uv should skip parsing non-platform dependencies or sources

[pplmx]

Summary

I have a pyproject.toml where I configure platform-specific dependencies and source settings as follows:

[project]
name = "example"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
    "megatron-core ; sys_platform == 'linux'",
]

[tool.uv.sources]
megatron-core = { git = "https://github.com/NVIDIA/Megatron-LM", marker = "sys_platform == 'linux'" }

On Linux, everything works fine. However, on Windows, running uv sync throws the following error:

uv sync
  × Failed to build `megatron-core @ git+https://github.com/NVIDIA/Megatron-LM`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit code: 1)

      [stderr]
      Traceback (most recent call last):
        ...
      FileNotFoundError: [WinError 3] The system cannot find the path specified.

      hint: This usually indicates a problem with the package or the build environment.

Steps to Reproduce

# run the following commands on Windows
mkdir example
uv init -p 3.12
# copy the above pyproject.toml to this project
uv sync

Expected Behavior

On non-target platforms (e.g., Windows), uv should skip parsing dependencies or sources that have platform-specific markers, thereby avoiding build errors.

FYI

uv 0.6.8
Python 3.12.9

[charliermarsh]

uv sync performs a universal resolution (see the docs), so we need to resolve metadata for megatron-core even if it won't ultimately be installed.

[pplmx]

Since Megatron-LM does not support Windows, we use an alternative solution on Windows. In this case, how should the pyproject.toml be configured to work properly on Windows?

[charliermarsh]

Normally I would recommend adding a [[tool.uv.dependency-metadata]] for Megatron-LM so that the resolver has static metadata available and doesn't need to build the project: https://docs.astral.sh/uv/concepts/resolution/#dependency-metadata. That looks difficult for Megatron-LM since they use dynamic metadata and it seems to vary quite a bit? Your best bet might be to lock on Linux, then use --frozen when installing on Windows.

[pplmx]

Thank you, @charliermarsh. Here's what I've tried:

1. Generated uv.lock on Linux using the provided pyproject.toml.
2. Copied uv.lock to Windows.
3. Ran uv sync on Windows—this step worked as expected.
4. Installed a package (e.g., numpy) on Windows with uv add numpy --frozen and received no error output.
5. However, when I ran uv tree or uv pip list, no packages were listed.

Could you please help me understand why the package isn’t showing up?

[charliermarsh]

Can you explain what you mean by "no packages were listed", and/or provide me with exact steps I can take to reproduce?

[pplmx]

Steps to Reproduce

# On Linux
mkdir example; cd example
uv init -p 3.12
uv add "git+https://github.com/NVIDIA/Megatron-LM; sys_platform == 'linux'"
# show current packages
uv tree -d2

# On Windows
mkdir example; cd example
uv init -p 3.12
# copy the above pyproject.toml and uv.lock generated on Linux to replace here
uv sync
uv tree -d2
uv add numpy --frozen
uv tree -d2 # This will throw error
uv tree -d2 --frozen # output: example v0.1.0
uv pip list # output nothing

[zanieb]

uv add --frozen won't update the lockfile with your new dependency (and uv tree --frozen similarly will not update the lockfile). You need to add your dependencies up-front on the Linux machine (to workaround this specific problem).

[pplmx]

Thank you, @zanieb.
I understand the process now. For this case, to use numpy on Windows, I need to run uv add numpy on Linux first to generate the uv.lock file, and then execute uv sync --frozen on Windows using that lock file.

[pplmx]

Personally, uv should skip parsing non-platform dependencies or sources is a valid requirement :)

[zanieb]

I don't think we'll do that. The point of uv's lockfile is that it's platform independent and the problem is rather that this package is not following best practices for declaring metadata.

[pplmx]

Thank you for your assistance.
I'm not entirely sure what the best practices are for declaring metadata. Could you please provide some guidance?
Additionally, how should a Linux-only package declare the correct metadata?
Once I have this information, I'll be happy to open an issue on Megatron-LM to further discuss the topic.
Thanks again!

[zanieb]

The problem is that we need build the package to determine its metadata, but the package can only be built on Linux. Metadata can be declared statically, so the build does not need to happen. Or, they can build wheels for supported platforms, in which the resolved metadata can be stored.

For background, see

• https://packaging.python.org/en/latest/specifications/pyproject-toml/#declaring-project-metadata-the-project-table
• https://peps.python.org/pep-0621/

The project is using a pyproject.toml, but specifically defines critical core metadata fields as dynamic: https://github.com/NVIDIA/Megatron-LM/blob/cbc89b322c454a2de46edcbd1fc708669aeafd59/pyproject.toml#L11

[pplmx]

Thank you for the detailed explanation and the reference links.