test_thread_state_ensure is flaky on Fedora Linux s390x

[befeleme]

Bug report

Bug description:

test_thread_state_ensure is flaky on Fedora Linux on s390x architecture. Sometimes we see the traceback below:

test_thread_state_ensure (test.test_embed.AuditingTests.test_thread_state_ensure) ... --- ['/builddir/build/BUILD/python3.15-3.15.0_b2-build/Python-3.15.0b2/build/freethreading/Programs/_testembed', 'test_thread_state_ensure'] failed ---
stdout:
stderr:
_testembed: /builddir/build/BUILD/python3.15-3.15.0_b2-build/Python-3.15.0b2/Programs/_testembed.c:2736: test_thread_state_ensure: Assertion `_Py_atomic_load_int(&data.done) == 1' failed.
------
FAIL
======================================================================
FAIL: test_thread_state_ensure (test.test_embed.AuditingTests.test_thread_state_ensure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/python3.15-3.15.0_b2-build/Python-3.15.0b2/Lib/test/test_embed.py", line 2000, in test_thread_state_ensure
    self.run_embedded_interpreter("test_thread_state_ensure")
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/python3.15-3.15.0_b2-build/Python-3.15.0b2/Lib/test/test_embed.py", line 138, in run_embedded_interpreter
    self.assertEqual(p.returncode, returncode,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
                     "bad returncode %d, stderr is %r" %
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                     (p.returncode, err))
                     ^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 0 : bad returncode -6, stderr is "_testembed: /builddir/build/BUILD/python3.15-3.15.0_b2-build/Python-3.15.0b2/Programs/_testembed.c:2736: test_thread_state_ensure: Assertion `_Py_atomic_load_int(&data.done) == 1' failed.\n"
----------------------------------------------------------------------

This has been added in #149116
cc @encukou

CPython versions tested on:

3.15

Operating systems tested on:

Linux

Linked PRs

• gh-151177: Fix race condition in _testembed #151293
• [3.15] gh-151177: Fix race condition in _testembed (GH-151293) #151312

[encukou]

If you have the box handy, could you test with guarding the write to done, like this?

diff --git a/Programs/_testembed.c b/Programs/_testembed.c
index 7246cc06fff..74d0fe37dda 100644
--- a/Programs/_testembed.c
+++ b/Programs/_testembed.c
@@ -2715,8 +2715,8 @@ do_tstate_ensure(void *arg)
     PyThreadState_Release(tokens[2]);
     PyThreadState_Release(tokens[1]);
     PyThreadState_Release(tokens[0]);
-    PyInterpreterGuard_Close(guard);
     _Py_atomic_store_int(&data->done, 1);
+    PyInterpreterGuard_Close(guard);
 }
 
 static int

cc @ZeroIntensity

[ZeroIntensity]

If you have the box handy, could you test with guarding the write to done, like this?

That looks like the right fix.

[ZeroIntensity]

I put up a fix using Petr's suggested change (#151293). This isn't specific to Linux or Fedora; it's just a general race condition in the test case.

[encukou]

Thanks!