Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -762,7 +762,7 @@ regression hides. The status page separates three things a non-gating failure ca

| | shown as | means |
|---|---|---|
| **expected** | amber, folded shut | The repository predicted this and can say why. `EchTest`'s `JDK` cases can't pass until a Conscrypt carrying the ECH API is released — that *is* the finding, so it is recorded with its reason and kept quiet. Declared per case in `site/tools/collect_results.py`. |
| **expected** | amber, folded shut | The repository predicted this and can say why. Examples include `EchTest`'s JVM limitation and `BasicLoomTest`'s monitor pinning before JDK 24 — those are the findings, so they are recorded with reasons and kept quiet. Declared per case in `site/tools/collect_results.py`. |
| **unexpected, critical** | red | A surprise in a suite whose question the repository is currently trying to answer. The ECH suites are critical today; move one back to `watch` when its question is settled. |
| **unexpected, watch** | amber | A surprise in a suite being kept honest, where a server nobody here operates is as likely a cause as the client. |

Expand Down
8 changes: 4 additions & 4 deletions containers/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,10 @@ tasks.withType<Test>().configureEach {
// virtual thread pins its carrier, which today it does — Http2Connection.newStream holds
// a monitor across Http2Writer.flush's blocking write. That is a true finding about the
// published artifact on JDK 21 (JEP 491 removes the pinning on 24+), not a broken test,
// so the assertion stands as written and the result is recorded in the JUnit XML. It
// just doesn't fail the build: a finding about OkHttp shouldn't read as this repo being
// broken. Every other suite stays fatal, which is what caught the MockServer version
// mismatch.
// so the assertion stands as written and the result is recorded in the JUnit XML. The
// status collector classifies that predicted failure as expected only before JDK 24, and
// this task doesn't fail the build. Every other suite stays fatal, which is what caught
// the MockServer version mismatch.
val loomTestPattern = "**/BasicLoomTest.class"

// HostileRetryTest asks how many times OkHttp sends a request the server killed under it. The
Expand Down
7 changes: 3 additions & 4 deletions site/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -142,10 +142,9 @@ <h2>About these results</h2>
<p>
<span class="pill expected">expected</span> is a failure this repository predicted and can
explain: the assertion is right and the answer is currently no, which is the finding rather
than a problem with it. ECH on the JVM is the present example — OkHttp's
<code>ConscryptPlatform</code> drops the config list, and no released Conscrypt has the
method for it to call. These are folded shut, and each carries its reason where a stack
trace would otherwise be. When one starts passing, it becomes an unexpected pass to look at.
than a problem with it. Examples include ECH on the JVM and monitor-based virtual-thread
pinning before JDK 24. These are folded shut, and each carries its reason where a stack trace
would otherwise be. When one starts passing, it becomes an unexpected pass to look at.
</p>
<p>
<span class="pill skipped">not run</span> against a suite whose endpoint is down is the
Expand Down
25 changes: 21 additions & 4 deletions site/tools/collect_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,8 +103,8 @@ def severity_of(suite_name: str) -> str:
# Failures that are the point rather than the problem.
#
# A suite here is asking a question whose answer is currently "no", and saying so is why it
# exists — EchTest reports that OkHttp can't do ECH on the JVM, and a green EchTest would mean
# the finding had been lost, not that the bug was fixed. Those failures are shown, and shown in
# exists — EchTest reports that OkHttp can't do ECH on the JVM, and BasicLoomTest reports the
# monitor pinning that JEP 491 fixes only from JDK 24. Those failures are shown, and shown in
# amber, but folded away: the page's red is reserved for a result nobody predicted.
#
# The reason is required, and is what the page shows instead of a stack trace. Cases are named
Expand Down Expand Up @@ -169,8 +169,21 @@ def normalise_case(name: str) -> str:
return " ".join(name.split())


def expected_reason(suite_name: str, case_name: str) -> str:
def expected_reason(
suite_name: str,
case_name: str,
platform: str,
variant: str,
) -> str:
"""Why this case failing is the expected answer, or empty if it isn't."""
if suite_name == "BasicLoomTest" and case_name == "testHttpsRequest":
jdk = re.search(r"\bJDK (\d+)", f"{variant} {platform}")
if jdk and 21 <= int(jdk.group(1)) < 24:
return (
"Before JDK 24, Http2Connection.newStream holds intrinsic monitors across "
"Http2Writer.flush's blocking write, which pins the virtual thread's carrier. "
"JEP 491 removes this monitor-based pinning from JDK 24."
)
return EXPECTED_FAILURES.get(suite_name, {}).get(case_name, "")


Expand Down Expand Up @@ -224,7 +237,11 @@ def parse_suite(

raw_name = case.get("name", "")
case_name = normalise_case(raw_name)
reason = expected_reason(simple_name, case_name) if status == "failed" else ""
reason = (
expected_reason(simple_name, case_name, platform, variant)
if status == "failed"
else ""
)
if reason:
status = "expected"

Expand Down
11 changes: 6 additions & 5 deletions site/topics/loom.html
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,11 @@ <h2>An observed pinning signature</h2>
<p>
<code>Http2Connection.newStream</code> holds a monitor across <code>Http2Writer.flush</code>'s
blocking write. That is a true statement about the published artifact, not a broken test — so
the suite runs under the <code>loomTest</code> Gradle task with <code>ignoreFailures</code>.
The assertion stays exactly as written. When present, the failure is recorded in the JUnit XML and shown above and on
the <a href="../">status page</a> as a <span class="pill finding">findings</span> result, and
the build stays green. A finding about OkHttp should not read as this repository being red.
the suite runs under the <code>loomTest</code> Gradle task with <code>ignoreFailures</code>, and
the assertion stays exactly as written. On JDK 21 through 23, the status collector classifies
this predicted failure as <span class="pill expected">expected</span> and supplies the reason
in place of the stack trace. On JDK 24 and later the exemption does not apply: a pinning trace
there is an unpredicted <span class="pill finding">finding</span>.
</p>

<h2>Why the answer depends on the JDK</h2>
Expand All @@ -76,7 +77,7 @@ <h2>Why the answer depends on the JDK</h2>
code that pins on 21 should not pin on 24 and later. Recording that difference — same artifact,
same test, different runtime — is exactly what a testbed running against published versions is
for. The suite is gated with <code>@EnabledForJreRange(min = JRE.JAVA_21)</code>, and the
daily run currently runs on 21.
daily run includes both 21 and 25, showing the behavior before and after that JDK change.
</p>

<h2>References</h2>
Expand Down