Skip to content

Four more flaky tests, and one that passes whether or not the kill happens #416

Description

@blooop

Question

The aid pty tests flake on a loaded cargo test --workspace run found and fixed one mechanism — send_line delivering a paste as two pty writes — and measured it away, 86/100 to 100/100. While measuring (280 suite runs, 6 full-workspace runs, load average 12-25) it turned up four more flaky tests and one test that cannot fail. None were touched.

Flaked during measurement, mechanism unknown:

  • flows::launch::tests::a_contended_up_of_a_running_workspace_runs_no_up_at_all
  • timing::tests::stages_closed_out_of_order_do_not_stay_on_the_clock (twice)
  • dl/tests/interrupt.rs::closing_the_terminal_mid_up_removes_the_token_file_and_kills_the_up
  • dl/tests/picker.rs::what_is_typed_... — seen separately on another branch, a pty echo race ("> wayfind" vs "> wayfinder"); passes 3/3 alone

The first two are in-process and hold timing::exclusive(), the repo's process-global test mutex — worth checking whether contention on it, rather than pty timing, is the shared cause.

A test that cannot fail, which is the more interesting half:

aid/tests/interactive.rs::a_ctrl_c_at_the_editor_tears_the_whole_boot_down stakes its "the orphaned devpod up must have been killed" assertion on a wait_for whose subject is a sleep 30. A kill that never happened satisfies it anyway, 30 seconds later — which is also where the occasional 30-second run comes from. It is asserting the passage of time, not the kill.

Also unreproduced

#401 recorded the symptom "press Enter" never appeared after the 60s deadline. That did not occur once in 280 suite runs or 6 full-workspace runs while the paste mechanism was being hunted. Either it is a second mechanism still live, or the original observation was the paste failure seen through a different assertion. Worth establishing which before assuming the pty tests are now sound.

Testing the outcome

Same method that settled #401: reproduce at a measured rate first, name the mechanism, fix, re-measure with the same harness. A fix for a mechanism not confirmed by a knob is a guess — that is what #401 demonstrated.

The Ctrl-C test is different: it needs an assertion that observes the kill rather than the clock, and the red is that the current assertion passes against a build where nothing is killed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions