Question
The aid pty tests flake on a loaded cargo test --workspace run found and fixed one mechanism — send_line delivering a paste as two pty writes — and measured it away, 86/100 to 100/100. While measuring (280 suite runs, 6 full-workspace runs, load average 12-25) it turned up four more flaky tests and one test that cannot fail. None were touched.
Flaked during measurement, mechanism unknown:
flows::launch::tests::a_contended_up_of_a_running_workspace_runs_no_up_at_all
timing::tests::stages_closed_out_of_order_do_not_stay_on_the_clock (twice)
dl/tests/interrupt.rs::closing_the_terminal_mid_up_removes_the_token_file_and_kills_the_up
dl/tests/picker.rs::what_is_typed_... — seen separately on another branch, a pty echo race ("> wayfind" vs "> wayfinder"); passes 3/3 alone
The first two are in-process and hold timing::exclusive(), the repo's process-global test mutex — worth checking whether contention on it, rather than pty timing, is the shared cause.
A test that cannot fail, which is the more interesting half:
aid/tests/interactive.rs::a_ctrl_c_at_the_editor_tears_the_whole_boot_down stakes its "the orphaned devpod up must have been killed" assertion on a wait_for whose subject is a sleep 30. A kill that never happened satisfies it anyway, 30 seconds later — which is also where the occasional 30-second run comes from. It is asserting the passage of time, not the kill.
Also unreproduced
#401 recorded the symptom "press Enter" never appeared after the 60s deadline. That did not occur once in 280 suite runs or 6 full-workspace runs while the paste mechanism was being hunted. Either it is a second mechanism still live, or the original observation was the paste failure seen through a different assertion. Worth establishing which before assuming the pty tests are now sound.
Testing the outcome
Same method that settled #401: reproduce at a measured rate first, name the mechanism, fix, re-measure with the same harness. A fix for a mechanism not confirmed by a knob is a guess — that is what #401 demonstrated.
The Ctrl-C test is different: it needs an assertion that observes the kill rather than the clock, and the red is that the current assertion passes against a build where nothing is killed.
Question
The aid pty tests flake on a loaded cargo test --workspace run found and fixed one mechanism —
send_linedelivering a paste as two pty writes — and measured it away, 86/100 to 100/100. While measuring (280 suite runs, 6 full-workspace runs, load average 12-25) it turned up four more flaky tests and one test that cannot fail. None were touched.Flaked during measurement, mechanism unknown:
flows::launch::tests::a_contended_up_of_a_running_workspace_runs_no_up_at_alltiming::tests::stages_closed_out_of_order_do_not_stay_on_the_clock(twice)dl/tests/interrupt.rs::closing_the_terminal_mid_up_removes_the_token_file_and_kills_the_updl/tests/picker.rs::what_is_typed_...— seen separately on another branch, a pty echo race ("> wayfind"vs"> wayfinder"); passes 3/3 aloneThe first two are in-process and hold
timing::exclusive(), the repo's process-global test mutex — worth checking whether contention on it, rather than pty timing, is the shared cause.A test that cannot fail, which is the more interesting half:
aid/tests/interactive.rs::a_ctrl_c_at_the_editor_tears_the_whole_boot_downstakes its "the orphaneddevpod upmust have been killed" assertion on await_forwhose subject is asleep 30. A kill that never happened satisfies it anyway, 30 seconds later — which is also where the occasional 30-second run comes from. It is asserting the passage of time, not the kill.Also unreproduced
#401 recorded the symptom
"press Enter" never appearedafter the 60s deadline. That did not occur once in 280 suite runs or 6 full-workspace runs while the paste mechanism was being hunted. Either it is a second mechanism still live, or the original observation was the paste failure seen through a different assertion. Worth establishing which before assuming the pty tests are now sound.Testing the outcome
Same method that settled #401: reproduce at a measured rate first, name the mechanism, fix, re-measure with the same harness. A fix for a mechanism not confirmed by a knob is a guess — that is what #401 demonstrated.
The Ctrl-C test is different: it needs an assertion that observes the kill rather than the clock, and the red is that the current assertion passes against a build where nothing is killed.