[Feature Request] allow exporting virtual environments

[danielgafni]

It would be great if uv provided a way to export virtual environments to some formats like .tar.gz.

This is useful when preparing environments for PySpark.

venv-pack is a tool which does exactly that, but it's painfully slow. I'm sure uv could do the same task in a fraction of what it takes for venv-pack 🦀

[paveldikov]

That's exactly the use case that made me contribute uv venv --relocatable :-) all that is missing is the archival/compression which in your case is a simple tar czf of the finished venv.

[danielgafni]

I did notice the relocatable flag and used it. That's really nice! I guess all what's missing is a bunch of ergonomics and docs.

[danielgafni]

Hey, getting back to this...
It doesn't seem like the --relocatable flag is working as expected.

❯ uv venv --relocatable
Using CPython 3.11.9
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
❯ cd ./.venv/bin         
❯ ls -lh
total 44K
-rw-r--r-- 1 dan dan 3.7K Oct  2 13:42 activate
-rw-r--r-- 1 dan dan 2.2K Oct  2 13:42 activate.bat
-rw-r--r-- 1 dan dan 2.6K Oct  2 13:42 activate.csh
-rw-r--r-- 1 dan dan 4.2K Oct  2 13:42 activate.fish
-rw-r--r-- 1 dan dan 3.8K Oct  2 13:42 activate.nu
-rw-r--r-- 1 dan dan 2.8K Oct  2 13:42 activate.ps1
-rw-r--r-- 1 dan dan 2.4K Oct  2 13:42 activate_this.py
-rw-r--r-- 1 dan dan 1.7K Oct  2 13:42 deactivate.bat
-rw-r--r-- 1 dan dan 1.2K Oct  2 13:42 pydoc.bat
lrwxrwxrwx 1 dan dan   79 Oct  2 13:42 python -> /home/dan/.local/share/uv/python/cpython-3.11.9-linux-x86_64-gnu/bin/python3.11
lrwxrwxrwx 1 dan dan    6 Oct  2 13:42 python3 -> python
lrwxrwxrwx 1 dan dan    6 Oct  2 13:42 python3.11 -> python

Looks like the python executable is a symlink (which is not exactly relocatable).

Is there anything I'm missing here?

[bluss]

I think relocatable currently covers this use case: you rename the project folder, like if project1 is now project1.1. I don't think it does more. Without --relocatable, even that breaks (absolute paths in .venv/bin/ executable stubs break)

[zanieb]

Yeah we don't support bundling the interpreter in the environment yet, but want to.

[paveldikov]

Yes, that's right, this was not part of the initial use case that drove me to add --relocatable -- I was happy to take the base interpreter installation for granted.

This works very well for distribution formats like RPM or DEB (where you can register a dependency on a base Python package) but not at all end-user/desktop installs (whereby a system-wide Python installation is not expected or even desirable).

Some more thoughts on how one could solve the end-user-redistributability use case from first principles:

On terminology/semantics: by the time you bundle an interpreter with the virtual environment it's arguably no longer 'virtual'. Rather what you have is a fully fledged Python installation (could indeed be one coming from uv python install), that you are augmenting with further packages installed at system-site-packages-level.

So, this is mostly already achievable using stock uv functionality -- it's uv python install followed by uv pip install --system. The only problem here is the relocatable entrypoints. Currently the only user-facing way to generate relocatable entrypoint scripts is by using uv venv --relocatable... but in this case there is no venv at all! So there is no way to propagate the relocatable: true argument all the way down to the wheel installer.

A few options seem to naturally come out of this:

• uv package command that does a fully opinionated one-stop-shop thing.
  • Pros: simplest UX.
  • Cons: this would probably be strongly-coupled to uv python install, which is currently really enterprise-unfriendly (in a lot of regulated enterprises, downloading arbitrary Python binaries direct from GitHub is a fireable offence)
• uv pip install --system could imply relocatable: true.
  • Pros: this lines up with Python binary distributions already being relocatable -- if python{,.exe} is relocatable then it makes a lot of sense its sibling/dependent scripts to also be relocatable.
  • Cons: is this an implicit behaviour we are comfortable with? Relocatable entrypoints do have a few corner cases e.g. Scripts that use "from future" can result in syntax error when used with uvx #6489 which means that there is a minor loss of compatibility.
• Add --relocatable param to uv pip install
  • Pros: low level control
  • Cons: fragmented UX?
• Deliberately ignore the 'virtual' semantics and implement a mode of uv venv that actually completely clones the interpreter in its entirety.
  • Pros: UX remains consistent. And we maintain a decent level of low-level control for power users.
  • Cons: can we live with this breach of semantics?

[chrisrodrigue]

Thanks for brainstorming this @paveldikov. I think this would be a killer feature especially for offline environment deployments. I expect --relocatable allows one to move a venv anywhere on the file system and still function, is that correct?

Symlinks are somewhat tricky on Windows since they can require elevation to make them (unlike hardlinks) which isn't usually available in enterprise environments, so hopefully --relocatable doesn't depend on symlinks for the Windows implementation. This would have implications for Windows because hardlinks are only valid within the same file system, unlike symlinks which can span file systems since they just point to the other file.

I think --with-python and --with-uv or --with uv,python or --with uv --with python would be awesome options to add for uv venv since these flags would let someone move the venv onto another computer that might not have python or uv installed at all. This is somewhat the inverse of options that virtualenv provides such as --no-pip, --no-wheel and --no-setuptools. See here.

Maybe there could also be another flag so that one can target additional platforms that the venv should function in. By default, it targets the current platform, but it would be nice to create a venv that also grabs the wheels necessary for a different platform so that the venv could be moved there and still function. I think it would be a powerful feature, but I don't know what the API would look like, maybe --platform current,win_amd64,win32,macosx_10_12_x86_64,... or --platform win --platform macosx with --platform current being the implied default? If many platforms are specified, I would also expect it to apply to other options such as --with-python and --with-uv (i.e., the python and uv binaries for all specified platforms should be in the venv too).

--platform all would be nice too, for a universal (and gigantic) venv. Cosmopolitan python anyone?

[chrisrodrigue]

Cons: this would probably be strongly-coupled to uv python install, which is currently really enterprise-unfriendly (in a lot of regulated enterprises, downloading arbitrary Python binaries direct from GitHub is a fireable offence)

I think it would be cool if uv just bundled the latest python interpreter inside the respective uv platform binary for bootstrapping purposes, but not sure how the Astral team would feel about this or if it makes sense without having the rest of the standard library.

Deliberately ignore the 'virtual' semantics and implement a mode of uv venv that actually completely clones the interpreter in its entirety.

I think this option makes the most sense to me.

[zanieb]

downloading arbitrary Python binaries direct from GitHub is a fireable offence

For what it's worth, we're maintaining those binaries — they're not arbitrary. How's uv being installed? Isn't that a binary from GitHub? Note there's a UV_PYTHON_INSTALL_MIRROR which allows you to download from an internal mirror of the binaries.

[zanieb]

I also think we can only create environments with a bundled Python interpreter if the interpreter itself is relocatable, i.e., it would need to be one of our managed Python interpreters not an arbitrary system interpreter.

[paveldikov]

downloading arbitrary Python binaries direct from GitHub is a fireable offence

For what it's worth, we're maintaining those binaries — they're not arbitrary. How's uv being installed? Isn't that a binary from GitHub? Note there's a UV_PYTHON_INSTALL_MIRROR which allows you to download from an internal mirror of the binaries.

Oh, by arbitrary I meant 'unapproved' -- no slight intended, it's just that it doesn't fall under a use case that is known and expressly permitted.

It is quite common for enterprises to deploy scanning proxies of various kinds (cf. #5260) -- downloading direct from github.com or pypi.org is considered a breach of policy as you'd be bypassing said proxy. In fact in most cases you wouldn't even have direct connectivity to github.com or pypi.org as these would be blocked at firewall level.

Actually I was not aware of UV_PYTHON_INSTALL_MIRROR -- I am curious, is this also exposed in uv.toml? It does actually make a significant difference -- enterprises could use it to benefit from uv python in a way that still falls under existing/well-understood control implementations.