Python 3.11 launcher fails with "Unable to create process"

[pfmoore]

Bug report

When running a pip.pyz file with the py.exe launcher (itself launched from a Rust program), the command fails with the error:

Unable to create process using 'C:\Users\Gustav\AppData\Local\Programs\Python\Python311\"py C:\Users\Gustav\.local\apps\pip.pyz --version': The system cannot find the file specified.

Unfortunately, I haven't been able to isolate a fully reproducible test case so far. However, I have set $env:PYLAUNCHER_DEBUG=1 and the output of the failing command is

❯ pip --version
argv0: py.exe
version: 3.11.0b5
# Read 4096 bytes from C:\Users\Gustav\.local\apps\pip.pyz to find shebang line
Shebang: /usr/bin/env python
# Reading from C:\Users\Gustav\AppData\Local\py.ini for commands/python
# Did not find file C:\Users\Gustav\AppData\Local\py.ini
# Reading from C:\WINDOWS\py.ini for commands/python
# Did not find file C:\WINDOWS\py.ini
# Treating shebang command 'python' as 'py'
SearchInfo.originalCmdLine: "py.exe" C:\Users\Gustav\.local\apps\pip.pyz --version
SearchInfo.restOfCmdLine:  C:\Users\Gustav\.local\apps\pip.pyz --version
SearchInfo.executablePath: (null)
SearchInfo.scriptFile: C:\Users\Gustav\.local\apps\pip.pyz
SearchInfo.executable: "py
SearchInfo.executableArgs:
SearchInfo.company: (null)
SearchInfo.tag: (empty)
SearchInfo.oldStyleTag: True
SearchInfo.lowPriorityTag: False
SearchInfo.exclude32Bit: False
SearchInfo.only32Bit: False
SearchInfo.allowDefaults: False
SearchInfo.allowExecutableOverride: False
SearchInfo.windowed: False
SearchInfo.list: False
SearchInfo.listPaths: False
SearchInfo.help: False
 -V:3.11          C:\Users\Gustav\AppData\Local\Programs\Python\Python311\python.exe
 -V:3.10          C:\Users\Gustav\AppData\Local\Programs\Python\Python310\python.exe
 -V:3.9           C:\Users\Gustav\AppData\Local\Programs\Python\Python39\python.exe
 -V:3.8           C:\Users\Gustav\AppData\Local\Programs\Python\Python38\python.exe
env.company: PythonCore
env.tag: 3.11
# about to run: C:\Users\Gustav\AppData\Local\Programs\Python\Python311\"py C:\Users\Gustav\.local\apps\pip.pyz --version
Unable to create process using 'C:\Users\Gustav\AppData\Local\Programs\Python\Python311\"py C:\Users\Gustav\.local\apps\pip.pyz --version': The system cannot find the file specified.

The key issue seems to be

SearchInfo.executable: "py

(note the unbalanced quotes) which appears to be a result of incorrectly parsing the following:

SearchInfo.originalCmdLine: "py.exe" C:\Users\Gustav\.local\apps\pip.pyz --version

I can create the same debug output using the internal _winapi.CreateProcess function:

_winapi.CreateProcess("C:/Windows/py.exe", '"py.exe" -0', None, None, 0, 0, None, None, 0)

So the problem appears to be when the command line passed to CreateProcess contains a double quoted but relative filename.

Your environment

• Python and launcher from 3.11.0b5.
• Operating system and architecture: Windows 11, 64-bit.

[pfmoore]

Maybe another release blocker?

Edit: Possibly not - I rebuilt my Rust program, and cannot reproduce the issue there, now. So while there is definitely an issue, it's not as easily triggerable as I first thought. I updated my copy of rust, so maybe the rust stdlib changed how it calls CreateProcess?

[mrabarnett]

I notice that there's a stray quote in that first example: ... \Python311\"py ...

Was that a typo or actually present?

[pfmoore]

Actually present. I'm assuming it comes from treating SearchInfo.executable: "py as relative and prepending a directory to it...

[zooba]

Yeah, something funky going on with the shebang command there it looks like. I'll take a look

[zooba]

Okay, seems it's actually argv0 processing that's off, and you need to be launching from cmd.exe to trigger it (or probably through a custom subprocess command).