Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pytest doesn't run in venv on Remote workstation and so fails #24841

Open
strubbly opened this issue Feb 24, 2025 · 9 comments
Open

Pytest doesn't run in venv on Remote workstation and so fails #24841

strubbly opened this issue Feb 24, 2025 · 9 comments
Assignees
Labels
bug Issue identified by VS Code Team member as probable bug triage-needed Needs assignment to the proper sub-team

Comments

@strubbly
Copy link

Type: Bug

Behaviour

Some of my pytest tests rely on environment variables set during venv activation. They work when run in the integrated terminal (because the venv is activated) and under debug (that is set to run in a Debug terminal and that too gets the venv activated). But they don't run in the Test Explorer - they fail because the environment variables are not set correctly.

My settings include

   "python.testing.unittestEnabled": false,
   "python.testing.pytestEnabled": true,
   "python.terminal.activateEnvInCurrentTerminal": true,

I have tried various permutations of

     "python.experiments.optOutFrom": [
        "pythonTestAdapter",
        "pythonTerminalEnvVarActivation"
    ],
    "terminal.integrated.inheritEnv": false,

but none of those seem to make any difference.

I am aware that similar bugs have been reported before and that fixes have been released but this is happening with the latest VS code and extension versions. The test explorer used to work correctly with these settings and much earlier versions - specifically Python v2022.8.1

Steps to reproduce:

  1. Use Remote SSH to log in to remote machine.
  2. Create a venv.
  3. Edit the venv activate script to set a new environment variable.
  4. Create python code including a test that checks for the new environment variable.
  5. Choose the venv python interpreter
  6. Execute the test from the Test Explorer

Diagnostic data

Output for Python in the Output panel (ViewOutput, change the drop-down the upper-right of the Output panel to Python)

2025-02-24 15:13:54.853 [info] Discover tests for workspace name: design - uri: /users/bens/Sources/design
2025-02-24 15:13:54.891 [info] arg: --rootdir already exists in args, not adding.
2025-02-24 15:13:54.891 [info] All environment variables set for pytest discovery: {"WORKSPACE":"design", ... }
2025-02-24 15:13:54.899 [info] > ./.venv/bin/python -m pytest -p vscode_pytest --collect-only --rootdir=./src/design-code/src
2025-02-24 15:13:54.900 [info] cwd: ./src/design-code/src
2025-02-24 15:13:57.618 [info] ============================= test session starts ==============================
platform linux -- Python 3.9.16, pytest-8.3.4, pluggy-1.5.0
2025-02-24 15:13:57.618 [info] 

2025-02-24 15:13:57.627 [info] rootdir: /users/bens/Sources/design/src/design-code/src
configfile: pytest.ini
2025-02-24 15:13:57.628 [info] 
plugins: typeguard-4.4.2, subprocess-1.5.3, cov-6.0.0

2025-02-24 15:14:04.407 [info] collected 389 items / 65 errors

The third line of the output includes a listing of environment variables but these do not include the ones that would have been set had the
activation taken place. I have redacted the detail.

Extension version: 2025.0.0
VS Code version: Code 1.97.2 (e54c774e0add60467559eb0d1e229c6452cf8447, 2025-02-12T23:20:35.343Z)
OS version: Darwin arm64 24.3.0
Modes:
Remote OS version: Linux x64 4.18.0-553.32.1.el8_10.x86_64

  • Python version (& distribution if applicable, e.g. Anaconda): 3.9.16
  • Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
  • Value of the python.languageServer setting: Default
User Settings


languageServer: "Pylance"

testing
• cwd: "<placeholder>"
• pytestArgs: "<placeholder>"
• pytestEnabled: true

experiments
• optOutFrom: ["pythonTestAdapter","pythonTerminalEnvVarActivation"]

Installed Extensions
Extension Name Extension Id Version
debugpy ms- 2025.0.1
gitlens eam 16.3.0
isort ms- 2023.10.1
js-debug ms- 1.97.1
live-server ms- 0.4.15
python ms- 2025.0.0
ruff cha 2025.10.0
skill her 0.10.0
vscode-coverage-gutters rya 2.13.0
vscode-edit-csv jan 0.11.2
vscode-js-profile-table ms- 1.0.10
vscode-pylance ms- 2025.2.1
vsliveshare ms- 1.0.5948
System Info
Item Value
CPUs Apple M2 (8 x 2400)
GPU Status 2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
webgl: enabled
webgl2: enabled
webgpu: enabled
webnn: disabled_off
Load (avg) 2, 2, 2
Memory (System) 8.00GB (0.11GB free)
Process Argv --crash-reporter-id 1726e94e-8d6f-469e-8e9f-16be0da7f3fd
Screen Reader no
VM 0%
Item Value
Remote SSH: workstation3
OS Linux x64 4.18.0-553.32.1.el8_10.x86_64
CPUs AMD EPYC 7571 (2 x 0)
Memory (System) 7.36GB (5.72GB free)
VM 0%
A/B Experiments
vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805cf:30301675
binariesv615:30325510
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
a9j8j154:30646983
962ge761:30959799
2e7ec940:31000449
pythontbext0:30879054
cppperfnew:31000557
dwnewjupyter:31046869
nativerepl2:31139839
pythonrstrctxt:31112756
nativeloc1:31192215
iacca1:31171482
5fd0e150:31155592
dwcopilot:31170013
stablechunks:31184530
6074i472:31201624
dwoutputs:31242946
customenabled:31241370
9064b325:31222308
copilot_t_ci:31222730
jda6j935:31233686
copilothoveroff:31240707

@github-actions github-actions bot added the triage-needed Needs assignment to the proper sub-team label Feb 24, 2025
@karthiknadig
Copy link
Member

@strubbly Can you share a test that we can use to repro this? which variables are you relying on?

May I ask why you are relying on activated env variables? https://docs.python.org/3/library/venv.html#how-venvs-work
The venv docs mention not to rely on VIRTUAL_ENV variable.

When a virtual environment has been activated, the VIRTUAL_ENV environment variable is set to the path of the environment. Since explicitly activating a virtual environment is not required to use it, VIRTUAL_ENV cannot be relied upon to determine whether a virtual environment is being used.

@github-actions github-actions bot added the info-needed Issue requires more information from poster label Feb 24, 2025
@strubbly
Copy link
Author

This is an example of a test that demonstrates the issue:

def test_objects():
    import os

    env = os.environ["VSC_TEST"]
    assert env == "abc"

The activate script in the venv contains this line as its last line:

export VSC_TEST="abc"

If I run pytest in the integrated terminal then it succeeds. But if I run it in the test explorer, I get this output

2025-02-24 16:31:42.709 [info] arg: --rootdir already exists in args, not adding.
2025-02-24 16:31:42.709 [info] Attempting to use temp directory for test ids file, file name: test-ids-d1f07d6fd24f1c6f68e3.txt
2025-02-24 16:31:42.718 [info] All environment variables set for pytest execution: {"__MODULES_LMINIT":"module use --append /tools/modules/modulefiles", ...}
2025-02-24 16:31:42.718 [info] Running pytest with arguments: /users/bens/.vscode-server/extensions/ms-python.python-2025.0.0-linux-x64/python_files/vscode_pytest/run_pytest_script.py --rootdir=/users/bens/Sources/vscode_test for workspace /users/bens/Sources/vscode_test 

2025-02-24 16:31:42.718 [info] > ./.venv/bin/python ~/.vscode-server/extensions/ms-python.python-2025.0.0-linux-x64/python_files/vscode_pytest/run_pytest_script.py --rootdir=.
2025-02-24 16:31:42.718 [info] cwd: .
2025-02-24 16:31:43.707 [info] Test Result named pipe /run/user/596/python-test-results-42fa8e07ecc18f438c66  cancelled

and the test fails.

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Feb 24, 2025
@strubbly
Copy link
Author

strubbly commented Feb 24, 2025

I can see that it is not recommended to rely on environment variables being set in the activation script when using the venv. However, we are where we are and nearly everything (including the integrated terminal) does indeed correctly activate with the activate script before using the venv. And a lot of our infrastructure relies on that. So far the test explorer is the only thing we can't get to work.
In this case our specific requirement is to enable the use of a third party library which requires various PYTHONPATH and LD_LIBRARY_PATH changes to be enabled. Furthermore these are externally configurable per venv. So doing them in the activate is particularly useful.

@karthiknadig
Copy link
Member

@strubbly is it standard one that is generated when using venv or are you modifying the activate script to add anything additional?

@github-actions github-actions bot added the info-needed Issue requires more information from poster label Feb 24, 2025
@karthiknadig karthiknadig added the bug Issue identified by VS Code Team member as probable bug label Feb 24, 2025
@strubbly
Copy link
Author

The activate script is modified. In our real use case it has an automated modification to enable the third party library when activated but for the repro I simply added the export line as stated.

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Feb 24, 2025
@karthiknadig
Copy link
Member

@strubbly another question, would it work if you update the .bashrc or equivalent for the Remote to add the export?

@github-actions github-actions bot added the info-needed Issue requires more information from poster label Feb 24, 2025
@strubbly
Copy link
Author

@strubbly another question, would it work if you update the .bashrc or equivalent for the Remote to add the export?

No that wouldn't really work because the environment variables need to be set per virtual environment and we can have more than one per user. The environment variables control library versions (and other things) and need to match the things installed in the venv.

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Feb 24, 2025
@strubbly
Copy link
Author

As a temporary workaround I thought I would try adding one version of the possible environment variables into my .bashrc and see if that would work. However, I don't see the new environment variables in my environment during the test. I tried uninstalling the remote server and still I don't see the changes. Is the old environment being cached somewhere? I do see the new environment in a new terminal (for example) but not in the environment of pytest.

@strubbly
Copy link
Author

strubbly commented Mar 5, 2025

I wondered why the correct environment is used when running under debug. At first I thought it might be because I was running the debug sessions with a launch.json with "console": "integratedTerminal",. But that's not it. Even without this, the debug sessions run with the correct environment. So where does the environment come from? It seems like it is NOT set up by executing the activate script on each test run. Instead it seems like the activate script is used during test discovery, the environment is saved somehow, and then used in the debug session. So it seems odd that that environment saved during discovery is only used in the debugger and not when the test is run directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issue identified by VS Code Team member as probable bug triage-needed Needs assignment to the proper sub-team
Projects
None yet
Development

No branches or pull requests

3 participants