Fix PEP-723 script metadata parsing.

[jsirois]

This pre-emptively fixes the issue reported here:
astral-sh/uv#12303

[jsirois]

Apparently @ofek thinks this is bad form, but as a tool maintainer that generally has to work around PEPs that are too loose leading to crazy situations in the wild, I think this is just par for the course. I'd rather Pex not choke and ruin a user's day than tell them they're doing it wrong. Of course I think the underlying issue is, yet again, the PyPA. It should not be a forked entity from Python. If PEP-723 were implemented in Python by Python and shipped with Python, it could unambiguously parse this data as part of its normal bytecode parsing phase.

[ofek]

Of course I think the underlying issue is, yet again, the PyPA.

I'm not sure why you always mention that the issue at the end of the day is the PyPA; wouldn't the issue literally be the spec? I don't see how any other conclusion can be made 😅

If PEP-723 were implemented in Python by Python and shipped with Python, it could unambiguously parse this data as part of its normal bytecode parsing phase.

No, that's a bad idea and was rejected: https://peps.python.org/pep-0723/#why-not-use-possibly-restricted-python-syntax

Python isn't a closed ecosystem and many kinds of tools from all kinds of languages need to be able to parse the block.

[jsirois]

I'm not sure why you always mention that the issue at the end of the day is the PyPA; wouldn't the issue literally be the spec? I don't see how any other conclusion can be made

My point is there is no spec involving Python source code that can avoid ambiguity without parsing the Python source code. As such, if there were instead no spec and Python shipped with its own single set of tooling, then this would all be avoided. Non-python ecosystem tooling could either shell out to python tooling - easy to do if there is 1 and only 1 tooling set - or else bite the bullet and parse the source code too. The PyPA and CPython though seem to consistently reject the idea the very split in the two groups is a deep issue.

The Python ecosystem keeps having to pay for the division between Python and its tooling. The JS ecosystem has tread this road as well. Perhaps some folks are satisfied with that? I only have had exposure to the opposite.

[ofek]

Non-python ecosystem tooling could either shell out to python tooling - easy to do if there is 1 and only 1 tooling set - or else bite the bullet and parse the source code too.

I view that as completely unreasonable for all of those other non-Python projects, so I guess we just have to agree to disagree here about the burden of work we are comfortable with imposing on others.

[jsirois]

@ofek agreed. You appear to bias towards not imposing on the vanishing set of non-Python tool authors that want to read metadata from Python source code and I bias towards the vanishing set of users that might have these strings in their Python scripts.

As such, I'll proceed with this fix despite wishing Python & PyPA would get their act together and put me and many other Python tool authors out of a job.

[ofek]

Hmm, based on your last comment the scope of what we both are talking about appears to differ. Just to be clear, in your ideal future of unified tooling for the Python ecosystem (a good goal indeed!) the non-Python projects I'm talking about still exist and see high usage. For example, no one is going to stop using Dependabot based on what our community does and it is written in Ruby.

[jsirois]

For example, no one is going to stop using Dependabot based on what our community does and it is written in Ruby.

We are speaking in the same terms. If dependabot wants to parse Python deps in script files, it can call python -t deps --type 3rdparty this/file/here.py ... assuming syntax of that rosy future where python ships with built-in tools (-t).

Heck - they could do this today with just a stdlib module via -m <tool module>. But then it would need to be built in and that appears to be anathema to CPython / PyPA as it stands.

And FWIW I ate this dogfood. When I had to get Pants v1 supporting Go, I did not expect Go source files to provide metadata in any other way than parsing its source code. I did not want to do that from Python; so I used the very handy built in go list, which, with options, can tell you everything about 1st and 3rd party deps in go.... at least it could back on 2016.