`pytest==8.1.1` Import regression in some namespace package layouts

[aaraney]

As first noted in #12074, below is an example project structure with two packages under the ns namespace that now fail to import after the changes introduced in 8.1.x:

ns
└── python
    ├── bar
    │   ├── ns
    │   │   ├── bar
    │   │   └── test
    │   │       ├── __init__.py
    │   │       ├── bar.py
    │   │       └── test_bar.py
    │   └── pyproject.toml
    └── foo
        ├── ns
        │   ├── foo
        │   └── test
        │       ├── __init__.py
        │       ├── foo.py
        │       └── test_foo.py
        └── pyproject.toml

Below are the contents of test_foo.py and foo.py. test_bar.py and bar.py look nearly identical.

# python/foo/ns/test/test_foo.py
from .foo import value

def test_foo():
    assert value == "foo"

# python/foo/ns/test/foo.py
value = "foo"

In pytest==8.0.2, python -m pytest --import-mode=importlib correctly discovers and runs the tests from the top level ns directory. In pytest==8.1.1, python -m pytest --import-mode=importlib -o "consider_namespace_packages=true", results in the following error during collection:

========================== test session starts ===========================
platform darwin -- Python 3.9.16, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/user/pytest-12074
collected 0 items / 2 errors

================================= ERRORS =================================
____________ ERROR collecting python/bar/ns/test/test_bar.py _____________
ImportError while importing test module '/home/user/pytest-12074/python/bar/ns/test/test_bar.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
python/bar/ns/test/test_bar.py:1: in <module>
    from .bar import value
E   ModuleNotFoundError: No module named 'test.bar'
____________ ERROR collecting python/foo/ns/test/test_foo.py _____________
ImportError while importing test module '/home/user/pytest-12074/python/foo/ns/test/test_foo.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
python/foo/ns/test/test_foo.py:1: in <module>
    from .foo import value
E   ModuleNotFoundError: No module named 'test.foo'
======================== short test summary info =========================
ERROR python/bar/ns/test/test_bar.py
ERROR python/foo/ns/test/test_foo.py
!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!
=========================== 2 errors in 0.04s ============================

To reproduce the issue clone https://github.com/aaraney/pytest-12074, pip install ns.foo and ns.bar, pip install the different versions of pytest and run with the aforementioned commands.

[nicoddemus]

Thanks @aaraney for the report, I will take a look ASAP. 👍

[nicoddemus]

@aaraney,

How are you setting up your PYTHONPATH? I managed to get your example to work by putting ns/python/bar and ns/python/foo in the PYTHONPATH, which seems to be the recommendation for using namespace packages.

λ python -m pytest .tmp\ns\python --import-mode=importlib -o consider_namespace_packages=true --no-header
======================== test session starts ========================
collected 2 items

.tmp\ns\python\bar\ns\bar\test\test_bar.py .                   [ 50%]
.tmp\ns\python\foo\ns\foo\test\test_foo.py .                   [100%]

========================= 2 passed in 0.05s =========================

λ echo %PYTHONPATH%
e:\projects\pytest\.tmp\ns\python\bar;e:\projects\pytest\.tmp\ns\python\foo

λ cd
e:\projects\pytest

One thing you can use to check if your namespace package is working is to import it inside Python directly:

λ python
Python 3.12.2 (tags/v3.12.2:6abddd9, Feb  6 2024, 21:26:36) [MSC v.1937 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ns.bar.test.test_bar
>>> import ns.foo.test.test_foo
>>> ns
<module 'ns' (namespace) from ['e:\\projects\\pytest\\.tmp\\ns\\python\\bar\\ns', 'e:\\projects\\pytest\\.tmp\\ns\\python\\foo\\ns']>

Can you try the above and show what you get?

[joshbode]

I am seeing something similar - I think it is because there is effectively a collision between ns.test in each subpackage.
I deleted the __init__.py and following that it ran successfully using:

$ pytest --import-mode=importlib -o "consider_namespace_packages=true"

Without consider_namespace_packages if still failed, though.

BTW, @aaraney, there's a typo in the original command above - a missing g (consider_namespace_packaes), which may help :)

[joshbode]

Also, and this may be a different bug, but if I run python -m pytest ... instead of the pytest ... command directly then I don't need to delete the __init__.py in the test directories, though either way I do require the consider_namespace_packages option.

With pytest==8.0.2 it works fine either way (with/without running via python -m pytest) but it fails without the __init__.py for a different reason (relative imports within the test directory don't work).

[nicoddemus]

I suggest to also test using the Python REPL to import the namespace packages, taking pytest out of the picture, to ensure your configuration is correct.

[aaraney]

@nicoddemus, thanks again for looking into this! The example I provided was incomplete. When I adding __init__.py files to python/ns/bar/ns/bar/__init__.py and python/ns/foo/ns/foo/__init__.py, so python / setuptools would recognize them as actual submodules (as you alluded to) pytest ran successfully!

In the project I thought I discovered this bug in, it turns out there was an erroneous __init__.py file at the same level as a submodule's pyproject.toml / setup.py / setup.cfg (e.g. python/ns/bar/__init__.py). For some reason pytest==8.0.2 did not balk at this and collected correctly, however with pytest==8.1.1, I received ImportError: attempted relative import beyond top-level package and ModuleNotFoundError: No module named 'some-module' errors. I think that is the correct behavior per PEP 420.

@joshbode, great catch! Just fixed it!

[nicoddemus]

@aaraney thanks for the clarification!

[aaraney]

@nicoddemus, I think this can be closed unless you feel the need for it to stay open. It seems that it was a false alarm on my part. Thanks again for looking into this!

[nicoddemus]

If somebody sees this and is not working as expected, please try importing your modules using the Python REPL first, to ensure the configuration is correct. 👍

I'm closing this for now then, with "working as intended".

Thanks everyone!

[joshbode]

The REPL tip is a good one - it helped me figure out that my imports were being masked and that it wasn't being treated correctly as a namespace package, even before it hit pytest

[jaraco]

I have encountered a related issue.

In this job, the tests are failing with:

ModuleNotFoundError: No module named 'test.support'

The module test.support is part of the standard library, so should be importable. Pinning to pytest < 8.1 works around the issue.

The problem is that the namespace package jaraco is being missed in the doctest discovery, so jaraco.test is being imported as test:

py: commands[0]> pytest --pdb
============================================================== test session starts ===============================================================
platform darwin -- Python 3.12.2, pytest-8.1.1, pluggy-1.4.0
cachedir: .tox/py/.pytest_cache
rootdir: /Users/jaraco/code/jaraco/jaraco.test
configfile: pytest.ini
plugins: jaraco.test-5.3.1.dev79+g23d6c10, enabler-3.0.0, checkdocs-2.10.1, ruff-0.3.1, mypy-0.10.3, cov-4.1.0
collected 23 items                                                                                                                               

docs/conf.py ....                                                                                                                          [ 17%]
jaraco/test/__init__.py ....                                                                                                               [ 34%]
jaraco/test/cpython.py ...F
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
001 
002 Compatibility shims for getting stuff from test.support across
003 Python versions (for compatibility with Python 3.9 and earlier).
004 
005 >>> os_helper = try_import('os_helper') or from_test_support('temp_dir')
UNEXPECTED EXCEPTION: ModuleNotFoundError("No module named 'test.support'")
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/doctest.py", line 1361, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest test.cpython[0]>", line 1, in <module>
  File "/Users/jaraco/code/jaraco/jaraco.test/jaraco/test/cpython.py", line 25, in from_test_support
    import test.support
ModuleNotFoundError: No module named 'test.support'
/Users/jaraco/code/jaraco/jaraco.test/jaraco/test/cpython.py:5: UnexpectedException
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /Users/jaraco/code/jaraco/jaraco.test/jaraco/test/cpython.py(25)from_test_support()
-> import test.support
(Pdb) import test
(Pdb) test.__file__
'/Users/jaraco/code/jaraco/jaraco.test/jaraco/test/__init__.py'

Essentially, #3396 re-emerges with pytest 8.1.