Skip to content

Retry a suite that recorded no test case, not one that left no directory - #35

Merged
yschimke merged 1 commit into
mainfrom
claude/ech-conscrypt-build-oncco9
Aug 14, 2026
Merged

Retry a suite that recorded no test case, not one that left no directory#35
yschimke merged 1 commit into
mainfrom
claude/ech-conscrypt-build-oncco9

Conversation

@yschimke

Copy link
Copy Markdown
Owner

Follow-up to #33, whose retry never fired on the failure it was written for.

The defect

The retry asked whether the results directory was absent. A run whose APK install never happened still leaves that directory behind — empty. So on the merge commit of #33 the Android public suite did this:

'package install-create ...' returns error 'Unknown failure: cmd: Can't find service: package'
Starting 0 tests on emulator-5554
PublicEncryptedClientHelloTest failed; recorded, not fatal.

Zero tests, no retry attempted, non-fatal path taken, job green. Exactly the outcome the retry existed to prevent, and indistinguishable on the status page from a suite that had nothing to say.

The fix

Ask about test cases rather than about the directory. Checked against all four states:

results state retries?
no directory yes
empty directory yes
XML with zero test cases (the observed failure) yes
XML with real results no

Only the last declines, which is the intent.

Also: one wait, not two

wait_for_package_service is removed. #31 landed a real readiness gate a few commits after #33, and it is strictly stronger on every axis — it waits on the settings provider and external storage as well as the package service, proves the path with a live call, and fails the run instead of shrugging when the device never arrives. Two waits where the weaker one runs second is just a claim that the first didn't work. #31's gate is what should prevent zero-test runs; the retry is the backstop for when it doesn't.

What this does not do

It does not say whether ECH works on API 37. No run has yet executed those six public cases — the first two never installed, and the third failed on the PublicSuffixDatabase.list asset that #33 fixed. This change only ensures that a run which doesn't execute them stops looking like one that did.


Generated by Claude Code

The retry added in #33 never fired on the failure it was written for. It asked
whether the results directory was absent; a run whose APK install never
happened still leaves that directory behind, empty. So the run on the merge
commit reported `Starting 0 tests`, took the non-fatal path, and the job went
green with the public ECH suite having said nothing at all — which is what
the retry existed to prevent.

Ask about test cases instead. Checked against all four states: no directory,
empty directory, XML with zero cases (the observed failure), and real results
— only the last declines to retry.

Also drop `wait_for_package_service`. #31 landed a real readiness gate a few
commits later, and it is strictly stronger than this one on every axis: it
waits on the settings provider and external storage as well as the package
service, it proves the path with a live call, and it fails the run instead of
shrugging when the device never arrives. Two waits where the weaker one runs
second is just a claim that the first didn't work.

This does not yet say whether ECH works on API 37. No run has executed those
six cases: the first two never installed, and the third failed on the public
suffix asset. It says that a run which doesn't execute them will stop looking
like one that did.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CqiK79k9uoWsn2AzgXpHMA
@yschimke
yschimke merged commit e0c7c1d into main Aug 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants