Skip to content

python311: 3.11.0a7 -> 3.11.0b3 - #173998

Merged
mweinelt merged 2 commits into
NixOS:masterfrom
mweinelt:cpython311
Jun 14, 2022
Merged

python311: 3.11.0a7 -> 3.11.0b3#173998
mweinelt merged 2 commits into
NixOS:masterfrom
mweinelt:cpython311

Conversation

@mweinelt

Copy link
Copy Markdown
Member

https://www.python.org/downloads/release/python-3110b1/
https://blog.python.org/2022/05/python-3110b1-is-now-available.html

The darwin-libutil.patch does not apply anymore, we'll see shortly if it is still required.

Description of changes
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@mweinelt
mweinelt requested a review from FRidh as a code owner May 22, 2022 14:18
@github-actions github-actions Bot added the 6.topic: python Python is a high-level, general-purpose programming language. label May 22, 2022
@ofborg ofborg Bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. labels May 22, 2022
@mweinelt

Copy link
Copy Markdown
Member Author

@ofborg build python311

@mweinelt

Copy link
Copy Markdown
Member Author

@ofborg build python311.tests

@mweinelt

mweinelt commented May 23, 2022

Copy link
Copy Markdown
Member Author

Apparently builds on all platforms … and passthru.tests pass on aarch64-darwin. Okay. Everywhere else fails around hatch-vcs.

Edit: Builds on aarch64-darwin, but not on x86_64-darwin

@mweinelt
mweinelt requested a review from a team May 26, 2022 12:58
@uri-canva

Copy link
Copy Markdown
Contributor

I'm getting an error on x86_64-darwin:

./Modules/posixmodule.c:7320:9: error: implicit declaration of function 'openpty' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    if (openpty(&master_fd, &slave_fd, NULL, NULL, NULL) != 0)
        ^
./Modules/posixmodule.c:7320:9: note: did you mean 'openat'?
/nix/store/5vn284y91qrpvhw6rfq7947p2qxxsr2c-Libsystem-1238.60.2/include/sys/fcntl.h:516:5: note: 'openat' declared here
int     openat(int, const char *, int, ...) __DARWIN_NOCANCEL(openat) __OSX_AVAILABLE_STARTING(__MAC_10_10, __IPHONE_8_0);
        ^
./Modules/posixmodule.c:7417:9: error: implicit declaration of function 'login_tty' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    if (login_tty(fd) == -1) {
        ^
./Modules/posixmodule.c:7471:11: error: implicit declaration of function 'forkpty' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    pid = forkpty(&master_fd, NULL, NULL, NULL);

@reckenrode

Copy link
Copy Markdown
Contributor

This appears to be an SDK issue. If I use the 11.0 SDK stdenv from #176661, beta 3 builds on x86_64-darwin. The official Python releases support back to 10.9, but I don’t know how they are building those binaries.

@mweinelt

Copy link
Copy Markdown
Member Author

So how do you propose we progress here?

@reckenrode

Copy link
Copy Markdown
Contributor

The only thing I can think of is marking x86_64-darwin broken until #176661 is merged. 🫤

@mweinelt mweinelt mentioned this pull request Jun 14, 2022
@veprbl

veprbl commented Jun 14, 2022

Copy link
Copy Markdown
Member

I doubt the issue has much to do with the SDK version requirements. Those calls are quite ancient and are available in the current SDK (e.g., openpty in /nix/store/46qj01lcjz48wb4hc7mvn3mhb3syda7h-Libsystem-osx-10.11.6/include/util.h).

@veprbl

veprbl commented Jun 14, 2022

Copy link
Copy Markdown
Member

I think, since this is only a beta, we should merge and file an issue for darwin maintainers to figure things out.

@veprbl veprbl changed the title python311: 3.11.0a7 -> 3.11.0b1 python311: 3.11.0a7 -> 3.11.0b3 Jun 14, 2022
@reckenrode

Copy link
Copy Markdown
Contributor

I doubt the issue has much to do with the SDK version requirements. Those calls are quite ancient and are available in the current SDK (e.g., openpty in /nix/store/46qj01lcjz48wb4hc7mvn3mhb3syda7h-Libsystem-osx-10.11.6/include/util.h).

The Libsystem the SDK is using on my system is <hash>-Libsystem-1238.60.2, but it also has both functions defined. Hmm. So I did some digging.

If I manually add #include <util.h> to posixmodule.c, Python builds again. According to config.log, that header and openpty are detected, so why isn’t it working? It turns out that the 10.12 SDK in nixpkgs also has libutil.h while the 11.0 SDK does not. Python assumes that libutil.h includes declarations of openpty, login_tty, and forkpty, which is not true on Darwin (though it is at least on FreeBSD).

This is the problem addressed by ./3.7/darwin-libutil.patch, but it fails to apply due to python/cpython#29658. I’ve included an updated patch below for Python 3.11.

diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
index 40229bce0f..3cc604930e 100644
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -7258,7 +7258,7 @@ os_sched_getaffinity_impl(PyObject *module, pid_t pid)
 #ifdef HAVE_UTMP_H
 #include <utmp.h>
 #endif /* HAVE_UTMP_H */
-#elif defined(HAVE_LIBUTIL_H)
+#elif defined(HAVE_LIBUTIL_H) && !defined(__APPLE__)
 #include <libutil.h>
 #elif defined(HAVE_UTIL_H)
 #include <util.h>

https://www.python.org/downloads/release/python-3110b2/
https://pythoninsider.blogspot.com/2022/05/python-3110b2-is-now-available.html

With updated darwin-libutil.patch provided by Randy Eckenrode.

Co-Authored-By: Randy Eckenrode <randy@largeandhighquality.com>
@mweinelt

Copy link
Copy Markdown
Member Author

@ofborg build python311

@mweinelt
mweinelt merged commit 0b63d22 into NixOS:master Jun 14, 2022
@mweinelt
mweinelt deleted the cpython311 branch June 14, 2022 23:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants