ModuleNotFoundError for http package when using pip after setup

[silv-io]

Description:
When using pip to, e.g., install setuptools after running the action, we sometimes (but not always) get a ModuleNotFoundError: No module named 'http' error.
This doesn't happen always, and a re-run of the GitHub action usually resolves this error.
The error happens inside urllib, and is also filed there: urllib3/urllib3#3558 where they referred to the action.

Action version:
v5

Platform:

• Ubuntu
• macOS
• Windows

Runner type:

• Hosted
• Self-hosted (buildjet-2vcpu-ubuntu-2204)

Tools version:

Python 3.11

Repro steps:
A description with steps to reproduce the issue. If your have a public example or repo to share, please provide the link.

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install docker helper dependencies
        run: pip install --upgrade setuptools setuptools_scm

Expected behavior:

This should always work without issue.

Actual behavior:

Sometimes (but not always) we get the following error:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.9/x64/bin/pip", line 5, in <module>
    from pip._internal.cli.main import main
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_internal/cli/main.py", line 10, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_internal/cli/main_parser.py", line 9, in <module>
    from pip._internal.build_env import get_runnable_pip
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_internal/build_env.py", line 19, in <module>
    from pip._internal.cli.spinners import open_spinner
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_internal/cli/spinners.py", line 9, in <module>
    from pip._internal.utils.logging import get_indentation
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_internal/utils/logging.py", line 29, in <module>
    from pip._internal.utils.misc import ensure_dir
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_internal/utils/misc.py", line 43, in <module>
    from pip._internal.exceptions import CommandError, ExternallyManagedEnvironment
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_internal/exceptions.py", line 18, in <module>
    from pip._vendor.requests.models import Request, Response
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/requests/__init__.py", line 43, in <module>
    from pip._vendor import urllib3
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/urllib3/__init__.py", line 11, in <module>
    from . import exceptions
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/urllib3/exceptions.py", line 3, in <module>
    from .packages.six.moves.http_client import IncompleteRead as httplib_IncompleteRead
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/urllib3/packages/six.py", line 234, in create_module
    return self.load_module(spec.name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/urllib3/packages/six.py", line 209, in load_module
    mod = mod._resolve()
          ^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/urllib3/packages/six.py", line 118, in _resolve
    return _import_module(self.mod)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/urllib3/packages/six.py", line 87, in _import_module
    __import__(name)
ModuleNotFoundError: No module named 'http'

[gowridurgad]

Hi @silv-io ,
Thank you for reporting this issue. We will investigate it and get back to you as soon as we have some feedback.

[priya-kinthali]

Hello @silv-io👋,
It seems like the issue you're encountering may be related to the configuration of the runner you are using or an interaction between dependencies, rather than the setup-python action itself. We are unable to reproduce the error on our end. Please consider the following points that might help resolve the issue:
Potential solutions and workarounds:

• Retry Mechanism: Since re-running the action sometimes resolves the issue, we recommend adding a retry mechanism in your workflow to automatically attempt the failing step a few times. This can help mitigate transient errors. Here is an example of how you can implement it:

  - name: Install docker helper dependencies
    run: |
      for i in {1..3}; do
        pip install --upgrade setuptools setuptools_scm && break || sleep 5
      done

• Isolation of Dependencies: Running pip inside a virtual environment can help isolate dependencies and reduce the risk of conflicts with system packages. Consider adding a step to create and activate a virtual environment:

  - name: Set up virtual environment
    run: python -m venv venv
  - name: Activate virtual environment and install dependencies
    run: |
      source venv/bin/activate
      pip install --upgrade setuptools setuptools_scm

• Check for Network Issues: Intermittent network issues can sometimes cause errors like this. Please ensure there are no network interruptions during the installation process, as these may result in incomplete downloads or failures.

• Update urllib3 and six: Based on the error stack trace, it looks like there might be an issue with the urllib3 library, particularly related to its interaction with the six compatibility library. It may help to explicitly install or upgrade both urllib3 and six to their latest stable versions:

  - name: Upgrade urllib3 and six
    run: pip install --upgrade urllib3 six

We hope these suggestions help resolve or mitigate the issue you are facing.
Thank you!

[flying-sheep]

we’re running into inexplicable ModuleNotFoundErrors as well:

• Fix CI scverse/pertpy#704
• Fix Python 3.12 CI theislab/anndata2ri#211

Something’s wrong with maybe the runner image: actions/runner-images#11639

[priya-kinthali]

Hello @silv-io👋,
Just giving you a gentle ping to see if there have been any updates or if any of the suggested workarounds helped resolve the issue. Thank you!

[priya-kinthali]

Hello @silv-io👋,
Just a gentle reminder. Could you please let us know if you had a chance to try the workarounds mentioned in my previous comment and check if the issue has been resolved?
Thank you!

[flying-sheep]

@priya-kinthali check out actions/runner-images#11639. Even if a workaround works, it doesn‘t make sense to complicate things by creating a useless venv. The whole point of using the container’s system environment is to simplify things by only having one Python environment at any point.

[priya-kinthali]

Hello @flying-sheep👋,
While the virtual environment can help isolate dependencies, it is not mandatory if it complicates your setup.
Given the closure of the related issue and your observation that it worked with ubuntu-22.04 but not with ubuntu-latest, this seems to be more related to the runner or interaction between dependencies, rather than the setup-python action itself. We are unable to reproduce the error on our end. We recommend trying the suggested workarounds based on your project requirements to help mitigate this issue.

@silv-io 👋, We are proceeding with closing this issue for now. Please feel free to reach us out or create a new issue incase of any further concerns related to setup-python action. Thank you!