Poetry stuck at `pyenv` Python version active during `install-poetry`, broken after version uninstall

[jian-robo]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
  • no exception occurs, only Shell error

• OS version and name:
  • macOS Big Sur 11.2.3
• Poetry version:
  • 1.1.7 via install-poetry.py
• Link of a Gist with the contents of your pyproject.toml file:
  • provided inside [Issue] section

Issue

Below scenarios demonstrate that:

1. Poetry 1.1.7 which was installed via install-poetry.py, does not recognize the changes in Python version made via pyenv local <version>. The expected behavior is to recognize Python version activated via pyenv. Similar issue: Poetry does not use active pyenv when creating virtual environment #651
2. Poetry becomes broken after pyenv Python version, which have been active at a time of Poetry installation via install-poetry.py, was uninstalled via pyenv.

Out-of-the-head suggestion is that/Users/username/Library/Application Support/pypoetry/venv/bin/python should link to pyenv Python shims, not to a specific pyenv Python version, as it happens currently.

The environment:

• pyenv 2.0.3 via Homebrew
• Python 3.8.8 installed via pyenv, activated in a Poetry project dir via .python-version file. No other versions of Python installed via pyenv
• Poetry 1.1.7 installed via curl + install-poetry.py from the Poetry project dir, i.e. an active version of Python at a time of installation was 3.8.8
• Poetry appended to PATH via .zshrc: export PATH="$PATH:$HOME/.local/bin", echo $PATH output:
  /Users/username/.pyenv/shims:/usr/local/bin:<skipped>:/Users/username/.local/bin
• pyproject.toml specifies Python version requirement via [tool.poetry.dependencies]: python = "^3.8.8"
• poetry.lock specifies Python version requirement: python-versions = "^3.8.8"
• poetry.toml: in-project = true
• at the beginning, no .venv present in a Poetry project dir

Commands in below scenarios executed from Poetry project dir /Users/username/workspace/project/.

Scenario 1 (positive). Poetry Python version is the same as pyenv Python version:

• pyenv versions. The result (w and p are softlinks to workspace and project dirs):

  system
  3.8.8 (set by /Users/username/w/p/.python-version)

• python --version. The result:
  Python 3.8.8
• poetry run python --version. The result:

Creating virtualenv project in /Users/username/workspace/project/.venv
Python 3.8.8

Scenario 2 (negative). Switchinig Python version via pyenv:

• follow [Scenario 1]
• delete .venv from Poetry project dir
• install Python 3.7.11 via pyenv, activate it: pyenv install 3.7.11, pyenv local 3.7.11
• pyenv versions. The result:

  system
* 3.7.11 (set by /Users/username/w/p/.python-version)
  3.8.8

• python --version. The result:
  Python 3.7.11
• poetry run python --version. The result:

Creating virtualenv project in /Users/username/workspace/project/.venv
Python 3.8.8

Expected result:
poetry prints Python version Python 3.7.11, or maybe a Python version compatibility error (because the Poetry project requires a version 3.8.8 or later).

Scenario 3 (negative). Uninstalling of non-active pyenv Python version, .venv is still present:

• follow [Scenario 2]
• pyenv uninstall 3.8.8, after that: poetry run python --version. The result:

/Users/username/.local/bin/poetry: line 2: /Users/username/Library/Application Support/pypoetry/venv/bin/python: No such file or directory
/Users/username/.local/bin/poetry: line 2: exec: /Users/username/Library/Application Support/pypoetry/venv/bin/python: cannot execute: No such file or directory

Expected result:
Poetry is not broken, and either prints the actual Python version, or a reasonable and meaningful error.

Scenario 4 (negative). Uninstalling of non-active pyenv Python version, .venv is absent:

• follow [Scenario 3]
• delete .venv from Poetry project dir
• poetry run python --version. The result:
  the same result as for [Scenario 3].

Expected result:
The same expected result as for [Scenario 3].

[finswimmer]

Hello @sandpipersburg,

thanks a lot for your detailed issue report 👍

The short answers are:

• The "pyenv workflow" is not possible anymore with the install-poetry.py script
• Removing the python interpreter used during creation of a venv always leads to a broken venvs, because symlinks are used within the venv. This is not poetry specific.

The "pyenv workflow" doesn't work any longer, because the install-poetry.py scripts creates a venv in which Poetry gets installed. When running a poetry command, poetry has now it's own interpreter and has no access to the currently activated python interpreter in the shell. The get-poetry.py installation script vendors poetry's dependencies instead and poetry was always running with the current active python interpreter. So it was easy to change the python version for a venv by switching the activated python with poetry.

We are still looking for a user friendly alternative to define the python interpreter for the fresh venv. One solution under discussion is #4179 Another solution that come to my mind these days is something like poetry env use --shell /path/to/python which doesn't write to the envs.toml. Will see if I have some time for testing an implementation next week.

fin swimmer

[zume-dev]

Scenario 2 (negative). Switchinig Python version via pyenv:

I tried to execute poetry env use `which python` before poetry run python --version. The result:.

After that I tried to execute poetry run python --version, poetry raises NoCompatiblePythonVersionFound.
(The execution result is shown below)

I think this is the expected behavior presented by @sandpipersburg , but how about it?

▲  test/  poetry env use `which python`

  NoCompatiblePythonVersionFound

  The specified Python version (3.7.10) is not supported by the project (^3.8.8).
  Please choose a compatible version or loosen the python constraint specified in the pyproject.toml file.

  at ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/env.py:676 in create_venv
       672│             # If an executable has been specified, we stop there
       673│             # and notify the user of the incompatibility.
       674│             # Otherwise, we try to find a compatible Python version.
       675│             if executable:
    →  676│                 raise NoCompatiblePythonVersionFound(
       677│                     self._poetry.package.python_versions, python_patch
       678│                 )
       679│
       680│             io.write_line(

[mbahri]

I think this change although understandable is a bit unfortunate, as the pyenv + poetry workflow is quite popular and powerful.

It would be good to at least explain this limitation in the documentation, though. I spent hours today trying to figure out why the workflow was broken, before I noticed it was working fine with the old get-poetry.py

[ferreteleco]

Any update / solution on this? I would love to keep mantaining my 'pyenv workflow' using poetry as smooth as possible.

Thanks in advance!

[finswimmer]

Any update / solution on this?

The next preview release will contain an experimental option to detect the current activated python, regardless on how poetry was installed. See #4852

Poetry's own environment shouldn't break anymore, when the Python version used during installation get's removed. Symlinks are not used anymore:

https://github.com/python-poetry/install.python-poetry.org/blob/767db614c92afb79fde1b85cd0614bf3f75e8389/install-poetry.py#L299

https://github.com/python-poetry/install.python-poetry.org/blob/767db614c92afb79fde1b85cd0614bf3f75e8389/install-poetry.py#L316

[princelySid]

Just in case someone end up here after hours debugging do the usual flow then poetry env use <python version>. Should work after that.
Would be nice if the documentation changed to reflect this though

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.