Skip to content

Handle wrong PyPI detection better#3095

Draft
rasmusfaber wants to merge 3 commits intoUKGovernmentBEIS:mainfrom
METR:faber/handle-pypi-detection-for-binaries
Draft

Handle wrong PyPI detection better#3095
rasmusfaber wants to merge 3 commits intoUKGovernmentBEIS:mainfrom
METR:faber/handle-pypi-detection-for-binaries

Conversation

@rasmusfaber
Copy link
Contributor

This PR contains:

  • New features
  • Changes to dev-tools e.g. CI config / github tooling
  • Docs
  • Bug fixes
  • Code refactor

What is the current behavior? (You can also link to an open issue here)

In the case of a combination of factors, we can reach a strange failure mode:

  • Running with UV_NO_INSTALLER_METADATA=1 (standard recommendation when using uv in a Dockerfile)
  • Using a non-PyPI version of Inspect AI
  • Using the text_editor-tool (or one of the other tools that uses the sandbox tools
  • Failure was then made even worse by running in a non-interactive TTY

This resulted in:

  • install_state being detected as "pypi" since there is no direct_url.json when using UV_NO_INSTALLER_METADATA=1
  • binaries was not present due to using a direct install
  • In PyPI mode the S3 download is skipped, but it still falls back to building
  • The user is queried for combination, but there is no TTY, so the root cause is swallowed by a failure to prompt the user

What is the new behavior?

  • When detecting the install state as "pypi", we still emit a warning if the binary is not present, but we now also try downloading from S3 before we try to build it.
  • We test whether the TTY is interactive before we try to prompt the user for confirmation.

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

No

Other information:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant