Tests are discovered but not ran

[andre-guilhon-rxscale]

Type: Bug

Behaviour

When you have a few tests, and you'd like to run them all at once (e.g. by clicking the "Run tests" button, it should run all of the tests, but instead it raises an error and no test is ran.

Steps to reproduce:

1. Create a python project
2. Add a src and a tests directories in the root dir
3. Create some logic inside the src dir to make your app run
4. Add some tests to the tests folder.
5. To me, all that was needed was two test files, with simple tests:
   test_1.py:

def test():
    assert True

test_2.py:

class TestClass:

    def test(self):
        assert True

settings.json:

{
    "python.testing.cwd": "${workspaceFolder}/src",
    "python.testing.pytestArgs": [
        "../tests",
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true,
}

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2025-03-18 09:18:13.308 [debug] Received shell integration sequence for bash
2025-03-18 09:19:37.069 [debug] Testing: Trigger refresh after saving /home/guilhon/Documents/Projects/public-api-py/.vscode/settings.json
2025-03-18 09:19:37.261 [debug] Extension ms-python.vscode-pylance accessed getActiveEnvironmentPath with args: undefined
2025-03-18 09:19:37.262 [debug] Extension ms-python.vscode-pylance accessed resolveEnvironment with args: {"id":"/home/guilhon/Documents/Projects/public-api-py/.venv/bin/python","path":"/home/guilhon/Documents/Projects/public-api-py/.venv/bin/python"}
2025-03-18 09:19:37.262 [debug] Extension ms-python.vscode-pylance accessed getEnvironmentVariables with args: undefined
2025-03-18 09:19:37.262 [debug] Extension ms-python.vscode-pylance accessed getActiveEnvironmentPath with args: undefined
2025-03-18 09:19:37.263 [debug] Extension ms-python.vscode-pylance accessed resolveEnvironment with args: {"id":"/home/guilhon/Documents/Projects/public-api-py/.venv/bin/python","path":"/home/guilhon/Documents/Projects/public-api-py/.venv/bin/python"}
2025-03-18 09:19:37.265 [debug] Extension ms-python.vscode-pylance accessed getActiveEnvironmentPath with args: undefined
2025-03-18 09:19:37.265 [debug] Extension ms-python.vscode-pylance accessed resolveEnvironment with args: {"id":"/home/guilhon/Documents/Projects/public-api-py/.venv/bin/python","path":"/home/guilhon/Documents/Projects/public-api-py/.venv/bin/python"}
2025-03-18 09:19:37.266 [debug] Extension ms-python.vscode-pylance accessed getEnvironmentVariables with args: undefined
2025-03-18 09:19:37.266 [debug] Extension ms-python.vscode-pylance accessed getActiveEnvironmentPath with args: undefined
2025-03-18 09:19:37.266 [debug] Extension ms-python.vscode-pylance accessed resolveEnvironment with args: {"id":"/home/guilhon/Documents/Projects/public-api-py/.venv/bin/python","path":"/home/guilhon/Documents/Projects/public-api-py/.venv/bin/python"}
2025-03-18 09:19:37.274 [debug] Delay Trigger[Refresh Test Data]: triggered=6, called=6
2025-03-18 09:19:37.274 [info] Discover tests for workspace name: public-api-py - uri: /home/guilhon/Documents/Projects/public-api-py/.vscode/settings.json
2025-03-18 09:19:37.299 [debug] Starting Test Discovery named pipe
2025-03-18 09:19:37.299 [debug] Test Discovery named pipe /run/user/1000/python-test-discovery-b20e055d602db627e4f7 connected
2025-03-18 09:19:37.300 [info] All environment variables set for pytest discovery, PYTHONPATH: "/home/guilhon/.vscode/extensions/ms-python.python-2025.2.0-linux-x64/python_files"
2025-03-18 09:19:37.300 [debug] Running pytest discovery with command: -m pytest -p vscode_pytest --collect-only ../tests --rootdir=/home/guilhon/Documents/Projects/public-api-py/src for workspace /home/guilhon/Documents/Projects/public-api-py.
2025-03-18 09:19:37.323 [debug] Executable path for pytest discovery: /home/guilhon/Documents/Projects/public-api-py/.venv/bin/python.
2025-03-18 09:19:37.323 [info] > ./.venv/bin/python -m pytest -p vscode_pytest --collect-only ../tests --rootdir=./src
2025-03-18 09:19:37.323 [info] cwd: ./src
2025-03-18 09:19:37.737 [debug] Delay Trigger[Test Configuration Change]: triggered=9, called=7
2025-03-18 09:19:37.987 [debug] Delay Trigger[Refresh Test Data]: triggered=7, called=7
2025-03-18 09:19:37.987 [info] Discover tests for workspace name: public-api-py - uri: /home/guilhon/Documents/Projects/public-api-py
2025-03-18 09:19:37.987 [error] Test discovery already in progress, not starting a new one.
2025-03-18 09:19:41.257 [info] ============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-7.4.3, pluggy-1.5.0

2025-03-18 09:19:41.260 [info] Using --randomly-seed=1980590724
rootdir: /home/guilhon/Documents/Projects/public-api-py/src
configfile: ../tox.ini
plugins: randomly-3.16.0, sqlalchemy-mock-0.1.5

2025-03-18 09:19:41.263 [info] collected 2 items

<Module test_2.py>
  <Class TestClass>
    <Function test>
<Module test_1.py>
  <Function test>

2025-03-18 09:19:41.263 [info] [vscode-pytest]: Session path not a parent of test paths, adjusting session node to common parent.
[vscode-pytest]: Session node now set to:  /home/guilhon/Documents/Projects/public-api-py

2025-03-18 09:19:41.264 [debug] Test Discovery named pipe /run/user/1000/python-test-discovery-b20e055d602db627e4f7 received data
2025-03-18 09:19:41.264 [debug] Testing: Resolving item /home/guilhon/Documents/Projects/public-api-py
2025-03-18 09:19:41.264 [debug] Testing: Resolving item /home/guilhon/Documents/Projects/public-api-py/tests
2025-03-18 09:19:41.264 [info] 
========================== 2 tests collected in 0.01s ==========================

2025-03-18 09:19:41.469 [debug] Testing: Resolving item /home/guilhon/Documents/Projects/public-api-py/tests/test_2.py
2025-03-18 09:19:41.470 [debug] Testing: Resolving item test_2.py::TestClass
2025-03-18 09:19:41.470 [debug] Testing: Resolving item /home/guilhon/Documents/Projects/public-api-py/tests/test_1.py
2025-03-18 09:19:48.781 [debug] Testing: Manually triggered test refresh
2025-03-18 09:19:48.781 [debug] Testing: Clearing all discovered tests
2025-03-18 09:19:48.781 [debug] Testing: Forcing test data refresh
2025-03-18 09:19:48.781 [debug] Testing: Refreshing all test data
2025-03-18 09:19:48.781 [info] Discover tests for workspace name: public-api-py - uri: /home/guilhon/Documents/Projects/public-api-py
2025-03-18 09:19:48.805 [debug] Starting Test Discovery named pipe
2025-03-18 09:19:48.806 [debug] Test Discovery named pipe /run/user/1000/python-test-discovery-561ba61fc04e19de5ae5 connected
2025-03-18 09:19:48.807 [info] arg: --rootdir already exists in args, not adding.
2025-03-18 09:19:48.807 [info] All environment variables set for pytest discovery, PYTHONPATH: "/home/guilhon/.vscode/extensions/ms-python.python-2025.2.0-linux-x64/python_files"
2025-03-18 09:19:48.807 [debug] Running pytest discovery with command: -m pytest -p vscode_pytest --collect-only ../tests --rootdir=/home/guilhon/Documents/Projects/public-api-py/src for workspace /home/guilhon/Documents/Projects/public-api-py.
2025-03-18 09:19:48.828 [debug] Executable path for pytest discovery: /home/guilhon/Documents/Projects/public-api-py/.venv/bin/python.
2025-03-18 09:19:48.828 [info] > ./.venv/bin/python -m pytest -p vscode_pytest --collect-only ../tests --rootdir=./src
2025-03-18 09:19:48.828 [info] cwd: ./src
2025-03-18 09:19:51.976 [info] ============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-7.4.3, pluggy-1.5.0

2025-03-18 09:19:51.982 [info] Using --randomly-seed=862211250
rootdir: /home/guilhon/Documents/Projects/public-api-py/src
configfile: ../tox.ini
plugins: randomly-3.16.0, sqlalchemy-mock-0.1.5

2025-03-18 09:19:51.986 [info] collected 2 items

<Module test_2.py>
  <Class TestClass>
    <Function test>
<Module test_1.py>
  <Function test>

2025-03-18 09:19:51.987 [info] [vscode-pytest]: Session path not a parent of test paths, adjusting session node to common parent.
[vscode-pytest]: Session node now set to:  /home/guilhon/Documents/Projects/public-api-py

2025-03-18 09:19:51.987 [debug] Test Discovery named pipe /run/user/1000/python-test-discovery-561ba61fc04e19de5ae5 received data
2025-03-18 09:19:51.987 [debug] Testing: Resolving item /home/guilhon/Documents/Projects/public-api-py
2025-03-18 09:19:51.987 [debug] Testing: Resolving item /home/guilhon/Documents/Projects/public-api-py/tests
2025-03-18 09:19:51.988 [info] 
========================== 2 tests collected in 0.01s ==========================

2025-03-18 09:19:52.191 [debug] Testing: Resolving item /home/guilhon/Documents/Projects/public-api-py/tests/test_2.py
2025-03-18 09:19:52.191 [debug] Testing: Resolving item test_2.py::TestClass
2025-03-18 09:19:52.192 [debug] Testing: Resolving item /home/guilhon/Documents/Projects/public-api-py/tests/test_1.py
2025-03-18 09:19:55.059 [debug] Starting Test Result named pipe
2025-03-18 09:19:55.060 [debug] Test Results named pipe /run/user/1000/python-test-results-66a6e7d54b25e3b3ed6d connected
2025-03-18 09:19:55.061 [debug] Executable path for pytest execution: /home/guilhon/Documents/Projects/public-api-py/.venv/bin/python.
2025-03-18 09:19:55.061 [info] arg: --rootdir already exists in args, not adding.
2025-03-18 09:19:55.061 [info] Attempting to use temp directory for test ids file, file name: test-ids-90e11db3d7f2062fe806.txt
2025-03-18 09:19:55.084 [info] All environment variables set for pytest execution, PYTHONPATH: "/home/guilhon/.vscode/extensions/ms-python.python-2025.2.0-linux-x64/python_files"
2025-03-18 09:19:55.084 [info] Running pytest with arguments: /home/guilhon/.vscode/extensions/ms-python.python-2025.2.0-linux-x64/python_files/vscode_pytest/run_pytest_script.py --rootdir=/home/guilhon/Documents/Projects/public-api-py/src for workspace /home/guilhon/Documents/Projects/public-api-py 

2025-03-18 09:19:55.084 [info] > ./.venv/bin/python ~/.vscode/extensions/ms-python.python-2025.2.0-linux-x64/python_files/vscode_pytest/run_pytest_script.py --rootdir=./src
2025-03-18 09:19:55.084 [info] cwd: ./src
2025-03-18 09:19:58.294 [debug] Test Result named pipe /run/user/1000/python-test-results-66a6e7d54b25e3b3ed6d received data
2025-03-18 09:19:58.294 [error] Test Results named pipe /run/user/1000/python-test-results-66a6e7d54b25e3b3ed6d error: TypeError: Cannot convert undefined or null to object
    at Function.keys (<anonymous>)
    at t.PythonResultResolver._resolveExecution (/home/guilhon/.vscode/extensions/ms-python.python-2025.2.0-linux-x64/out/client/extension.js:2:917078)
    at t.PythonResultResolver.resolveExecution (/home/guilhon/.vscode/extensions/ms-python.python-2025.2.0-linux-x64/out/client/extension.js:2:916387)
    at /home/guilhon/.vscode/extensions/ms-python.python-2025.2.0-linux-x64/out/client/extension.js:2:960049
    at b.callback (/home/guilhon/.vscode/extensions/ms-python.python-2025.2.0-linux-x64/out/client/extension.js:2:930145)
    at Object.thunk (/home/guilhon/.vscode/extensions/ms-python.python-2025.2.0-linux-x64/out/client/extension.js:2:1994360)
2025-03-18 09:19:58.553 [debug] Test run finished, subprocess closed.
2025-03-18 09:19:58.553 [info] Test Result named pipe /run/user/1000/python-test-results-66a6e7d54b25e3b3ed6d  cancelled
2025-03-18 09:19:58.553 [debug] Test Results named pipe /run/user/1000/python-test-results-66a6e7d54b25e3b3ed6d disposed
2025-03-18 09:19:58.553 [debug] Finished running tests, ending runInstance.

Extension version: 2025.2.0
VS Code version: Code 1.98.1 (2fc07b811f760549dab9be9d2bedd06c51dfcb9a, 2025-03-10T15:38:08.854Z)
OS version: Linux x64 6.8.0-55-generic
Modes:

• Python version (& distribution if applicable, e.g. Anaconda): 3.12.3
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
• Value of the python.languageServer setting: Default

User Settings

languageServer: "Pylance"

testing
• cwd: "<placeholder>"
• pytestArgs: "<placeholder>"
• pytestEnabled: true
• pytestPath: "<placeholder>"

Installed Extensions

Extension Name	Extension Id	Version
black-formatter	ms-	2025.2.0
code-spell-checker	str	4.0.40
copilot	Git	1.284.0
copilot-chat	Git	0.25.1
debugpy	ms-	2025.4.1
git-blame	sol	0.2.58
js-debug	ms-	1.97.1
js-debug-companion	ms-	1.1.3
LiveServer	rit	5.7.9
minimal-kiwi	pra	2.0.2
python	ms-	2025.2.0
vetur	oct	0.37.3
volar	Vue	2.2.8
vscode-js-profile-table	ms-	1.0.10
vscode-pylance	ms-	2025.3.2

System Info

Item	Value
CPUs	13th Gen Intel(R) Core(TM) i7-1355U (12 x 4678)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off webnn: disabled_off
Load (avg)	1, 1, 2
Memory (System)	31.02GB (13.42GB free)
Process Argv	--crash-reporter-id 38360e0f-61bf-4ee1-8b40-5a3c4d8c5b26
Screen Reader	no
VM	0%
DESKTOP_SESSION	ubuntu
XDG_CURRENT_DESKTOP	Unity
XDG_SESSION_DESKTOP	ubuntu
XDG_SESSION_TYPE	wayland

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805cf:30301675
binariesv615:30325510
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyonecf:30548226
a9j8j154:30646983
962ge761:30959799
h48ei257:31000450
pythontbext0:30879054
cppperfnew:31000557
dwnewjupytercf:31046870
nativerepl2:31139839
pythonrstrctxt:31112756
nativeloc1:31192215
iacca1:31171482
5fd0e150:31155592
dwcopilot:31170013
6074i472:31201624
dwoutputs:31242946
customenabled:31248079
9064b325:31222308
copilot_t_ci:31222730
f5992895:31259995
jda6j935:31233686
pythoneinst12cf:31257680
fh1c7952:31258891
4gafe986:31261448
5b33h341:31253271
h6j33387:31256343
0339e702:31259951

[eleanorjboyd]

will investigate thanks

[andre-guilhon-rxscale]

One thing I forgot to mention and that might help figuring this out, is that when you try to run individual tests, or even individual classes, it works fine.

[cyalen]

Seeing an similar issue as well VSCode version1.98.2. Like @andre-guilhon-rxscale, I can also run individual tests fine, but in addition I can also sub-directories of tests with/without coverage etc. Project-level runs, however I can only collect, no actual tests are ran...

In addition I've tried multiple versions of the extension, and this issue seems to still be present on all of them. Could this be a problem with the 1.98 version of the core app? I haven't yet attempted a downgrade myself.

[andre-guilhon-rxscale]

One weird thing is that I have a bunch of projects configured in the same way, and some of them work as they should, and some don't. What can I do to debug it further? Any ideas?

[eleanorjboyd]

things that would be helpful for me to try and understand and debug your issue

• set log levels to trace via the command palette command "Developer: set log level"
• including logs from the "python" output channel
• a screen recording, especially if clicking a button isn't responding so I can understand (to make this even better have the python logs open during the screen recording so I can see if any messages are sent that are expected / not when you take a UI action)

[andre-guilhon-rxscale]

Sorry for the long time! I've set the log level to trace and ran everything again. Attached you'll find a screen recording (with the output from python opened), a log from a correct run (where I run just a part of the tests and it works), and a log with the failing try to run the tests!
I couldn't attach the screen recording, so I uploaded it to youtube: https://youtu.be/XNKvIWvmFVw

tests-python.log
test-error-python.log

[andre-guilhon-rxscale]

Any news on this? :)