Skip to content

Re-run the emulator suite once when it fails, and read an h3-only ALPN list - #65

Merged
yschimke merged 1 commit into
mainfrom
claude/okhttp-testbed-issue-5-y78q89
Aug 15, 2026
Merged

Re-run the emulator suite once when it fails, and read an h3-only ALPN list#65
yschimke merged 1 commit into
mainfrom
claude/okhttp-testbed-issue-5-y78q89

Conversation

@yschimke

Copy link
Copy Markdown
Owner

Progresses #5 and #12.

The re-runs

android-ech fails on infrastructure often enough to be a nuisance — the emulator comes up half-dead, package install-create answers Can't find service: package, and the run ends having executed zero tests. It happened twice while #64 was open, and re-running it by hand was the only response anyone had. rerun-flaky.yml does that automatically now.

The three suites already have daily crons and workflow_dispatch, so this is the missing piece rather than a change to when things run.

It is narrow on purpose, because an auto-retry is a way to hide real failures:

  • Only android-ech. The container and network suites fail for reasons worth reading — The rest of the roadmap: DNS, HTTP/2, Alt-Svc, TLS policy, client certificates and SvcParams #64's container failure was a genuine bug in one of my own tests. Covering them would be claiming their failures are noise too, which is a much bigger claim.
  • Only once, guarded on run_attempt == 1. Something that fails twice is either broken or flaky enough to be worth fixing rather than absorbing.
  • Visible. The retry appears as attempt 2 of the same run, so the history shows the failure as well as the retry rather than presenting a green first attempt.

If it starts firing regularly, that's the signal to fix the emulator setup, not to widen the retry.

The h3-only record

This closes #12's remaining bullet, which I couldn't do before the fixture could publish arbitrary records.

An HTTPS record whose alpn offers h3 and not h2 is the shape that would break connection setup if a client read the list as a requirement: OkHttp has no HTTP/3, and concluding the origin is unreachable would be wrong — http/1.1 is implied into the set and is perfectly usable. No public name publishes such a record, so the ECH fixture's resolver now does.

SvcParamTest asserts both halves: the implied default is present, and h2 is not invented to make the list more palatable — a client that did would try a protocol the origin never offered.

Verification

The Go change is gofmt-clean and the suite compiles against 5.5.0-SNAPSHOT and against the pinned release (where the 5.5.0 gate excludes it). The record shape is the same one already verified live against a natively-run resolver in #64; this one adds a fourth name in the same table.

The workflow itself can only really be verified by the next emulator flake — it is a workflow_run trigger, so it cannot fire from a pull request. Its YAML parses and the if guard is exact.


Generated by Claude Code

…N list

Two unrelated things the last round left behind.

`rerun-flaky.yml` retries `android-ech` once on failure. That job fails on
infrastructure often enough to be a nuisance — the emulator comes up half-dead,
`package install-create` answers `Can't find service: package`, and the run ends
having executed zero tests — and re-running it by hand was the only response
anyone had. It happened twice while the last pull request was open.

Narrow on purpose, because an auto-retry is a way to hide real failures. Only
`android-ech`: the container and network suites fail for reasons worth reading,
and covering them would be claiming their failures are noise too. Only once,
guarded on `run_attempt == 1` — something that fails twice is either broken or
flaky enough to fix rather than absorb. And the retry is visible as attempt 2 of
the same run, so the history shows the failure as well as the retry. If it
starts firing regularly that is the signal to fix the emulator setup, not to
widen the retry.

The second thing closes #12's remaining bullet. An `HTTPS` record whose `alpn`
offers `h3` and not `h2` is the shape that would break connection setup if a
client read the list as a requirement: OkHttp has no HTTP/3, and concluding the
origin is unreachable would be wrong, since `http/1.1` is implied into the set
and is perfectly usable. No public name publishes such a record, so the ECH
fixture's resolver now does, and `SvcParamTest` asserts both halves — the
implied default is present, and `h2` is not invented to make the list more
palatable.

Progresses #5, #12.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LTGos7kiusobSSixhsD19N

Copy link
Copy Markdown
Owner Author

android-ech failed here, which is the flake this PR exists to retry — a third occurrence, and the first one that is evidence rather than an anecdote.

Zero tests ran, on the first attempt and on the harness's own retry:

Starting 0 tests on test(AVD) - 17
Finished 0 tests on test(AVD) - 17
EncryptedClientHelloTest produced no results; retrying once.

The device died rather than the code failing:

Test run failed to complete. No test results.
onError: commandError=false message=INSTRUMENTATION_ABORTED: System has crashed.

with cmd: Can't find service: activity repeated afterwards. Gradle then reports "There were failing tests", which is misleading — nothing ever executed.

I checked whether my own change caused it, since this PR edits the ECH fixture's main.go: it did not. :ech-fixture:runEchFixture succeeded, and there is no Go compile error, no no ECH fixture for, no DoH resolver error and no container startup failure anywhere in the log. The new h3only.svcb.test case only matches .svcb.test names, so the ECH names the Android suite uses are untouched.

Re-run queued by hand. The workflow added here cannot help on this PR — workflow_run triggers run the copy of the file on the default branch, so it only starts working once this is merged.


Generated by Claude Code

@yschimke
yschimke merged commit 8c64060 into main Aug 15, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTTP/3 and Alt-Svc: record what OkHttp does today

2 participants