Publish a status site to GitHub Pages

[yschimke]

The daily runs already produced everything a status page needs, and put it in an artifact that expires in 30 days. This builds the page from it.

Live at https://yschimke.github.io/okhttp-testbed/ once Pages is set to deploy from GitHub Actions (Settings → Pages → Source: GitHub Actions — a one-time repository setting the workflow can't make for itself).

What's here

site/ — deployed exactly as it sits: plain HTML and CSS, no static site generator, no build step, no dependencies. The only generated files are site/data/latest.json and site/data/history.json, which are gitignored.

The status page reports the most recent results per OkHttp version and per suite, across both test workflows — that per-version comparison being the thing this repository exists to make — with the failing assertions in full and a history strip of the last 120 published updates. Topic pages cover ECH, DNS, TLS, proxies and virtual threads, each with the specifications the area is tested against and the servers involved.

site/tools/collect_results.py — JUnit XML to JSON, keyed by the OkHttp version under test rather than by the run that produced it, so containers and android-ech results for 5.5.0-SNAPSHOT land on one card. It keeps the distinction the build already draws: the Gradle task a suite ran under decides whether a failure is a failure or a finding, so BasicLoomTest's pinning assertion renders amber rather than red and the repository doesn't read as broken for reporting something true about OkHttp.

.github/workflows/pages.yml — rebuilds the page when either test workflow finishes on main. Whichever triggered it, it collects the most recent completed run of both, so a container run finishing doesn't blank the ECH results or the reverse. History is not committed: the workflow fetches data/history.json back off the deployed site, appends the snapshot it just built, and republishes. The deployed site is its own datastore. A pull request's run is deliberately not published — it is about the pull request, not the state of the repository — and a rebuild that finds no artifacts (they expire) carries the published results forward rather than blanking the page.

containers.yml and android-ech.yml — both now record a run-metadata.json in their artifact: which OkHttp version they actually resolved, and which run produced it. The Android suite lays its XML out by device rather than by Gradle task, so its task name comes from that metadata while the container suites keep reading theirs from the artifact layout.

Testing

collect_results.py was run against fabricated artifact directories covering both workflows, both versions, a passing suite, a fatal failure, a skip, and a loomTest finding — including the Android suite's nested by-device layout — and history append and same-snapshot replacement were both checked. The page was then rendered in headless Chromium against that data: summary, cards, suite matrix, failure details and history strip all populate, with no console or page errors.

[yschimke]

Fix merge conflicts

[yschimke]

Merged main and pushed. The README conflict was two additive sections, so both stay.

The site needed more than a textual merge, though — it was built around one workflow and would have quietly omitted android-ech entirely. So results are now keyed by the OkHttp version under test rather than by the run that produced them: the container suites and the ECH suite both test 5.5.0-SNAPSHOT, and that's one card. A page build now collects the most recent completed run of each workflow rather than only the one that triggered it, since they finish hours apart and either finishing must not blank the other's results. android-ech.yml gained the same run-metadata.json step containers.yml has, naming connectedAndroidTest explicitly because that suite's XML is laid out by device rather than by task.

The ECH topic page now leads with the fixture — origin, DoH resolver, and the three hostnames covering accepted/retried/disabled — with the public deployments kept as the interoperability check a fixture can't be.

Re-verified against fabricated artifacts from both workflows, including the nested by-device layout: the Android suite merges into the snapshot card, tasks resolve correctly from each source, and the page renders clean.

---

Generated by Claude Code

[yschimke]

android-ech (pinned snapshot) failed on the previous head, and it isn't this PR's doing — it's an emulator boot flake inside the android-emulator-runner step, before run-ech-test.sh ran at all:

adb: device offline          (repeatedly, for ~60s of boot)
Boot completed in 59647 ms
cmd: Failure calling service settings: Broken pipe (32)
##[error]The process '.../adb' failed with exit code 224

Gradle never started ("No Gradle build results detected"), and the suite passed on run #1 and on main's run #2. Nothing in this diff touches that path. The push just now should re-run it.

It did surface a real gap on my side, though, so that's fixed here: the failed job still uploaded its artifact, containing run-metadata.json and no XML. The page would have rendered that version as "unknown" with no suites and no explanation — indistinguishable from nothing having happened. Collection now records each contributing run's job status and suite count, and the summary line says no results (job failure) for a run that uploaded none. Suites that did report are unaffected, which is the point of collecting each workflow's latest run independently.

---

Generated by Claude Code

[yschimke]

android-ech failed again on the new head — containers (pinned release) is green. It got further this time (emulator booted, :ech-fixture:runEchFixture succeeded), then died installing the test APK:

Failed to commit install session 592834445 with command package install-commit 592834445.
Error: Unknown failure: cmd: Failure calling service package: Broken pipe (32)
> Could not load test results from '.../TEST-emulator-5554 - 17-_android-ech-.xml'.

Same root symptom as the previous run, one step later: cmd: Failure calling service <X>: Broken pipe (32) — first settings, now package. That's the emulator's system_server going away underneath adb, not a test failure. Run #3 never reached Gradle at all; run #4 reached it and lost the device mid-install.

I'm not fixing this here, and I want to be explicit about why. Nothing in this PR touches the android-ech build, its test code, or the emulator step — the only line I added to that workflow writes a JSON file after the tests are over. The suite passed on run #1 and on main's run #2 with the identical emulator configuration. Making it stable means changing the emulator setup of a suite that isn't this PR's — bumping the boot timeout, adding a retry around connectedDebugAndroidTest, or pinning a different system image — and that's your call on your suite, not something I should slip into a status-site PR.

Happy to do any of those in a separate change if you want one.

Worth noting the two failures are now both handled gracefully by the page: run #4 uploaded metadata plus an unparseable results XML, which the collector skips, leaving the run with zero suites — so it renders as no results (job failure) beside the container results rather than as a silent blank. Which is arguably the more useful outcome: an emulator that can't hold a device long enough to install an APK is exactly the kind of thing the status page should be showing rather than hiding.

---

Generated by Claude Code