[Perf] Linux/x64: 1 Improvement on 9/22/2026 4:34:39 PM +00:00

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	ViperUbuntu
Baseline	d4b2951c1fa53ed95aa5bea8ad8e8a0e0a5a2fe7
Compare	206bf81aa71b157cb03ae7ed1a42d1ed7d3aa2dc
Diff	Diff
Configs	CompilationMode:wasm, RunKind:micro, RuntimeType:coreclr

Improvements in System.Collections.TryGetValueFalse<SmallClass, SmallClass>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableSortedDictionary - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	12.71 ms	8.77 ms	0.69	0.26	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<SmallClass, SmallClass>*'

Details

System.Collections.TryGetValueFalse<SmallClass, SmallClass>.ImmutableSortedDictionary(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[LoopedBard3]

🔍 Automated Triage Analysis

Summary: One test reported a 31.1% improvement, but historical analysis classifies it as borderline because the compare result is near the long-term mean after an elevated baseline tail. Finding confidence is 4/5 given 291 prior runs; improvement confidence is 2/5 because variability is high (CV 11.8%), the z-score is only -0.52, and no commit in the 17-commit range is a direct match. Bisection found no change for 1 test(s) — it ran but the change didn't reproduce (noisy or bimodal pattern).

Finding Confidence: 4/5 (analysis accuracy)
Improvement Confidence: 2/5 (likelihood of true improvement)

⚠️ Note: Low improvement confidence - attribution uncertain. See rationale in full analysis.

🤖 Proposed Next Actions: Likely noise — close & label — 1 done, 2 dry-run (rule R1b_bisection_noise — see Full Analysis for the table).

📊 Full Analysis (click to expand)

Summary: One test reported a 31.1% improvement, but historical analysis classifies it as borderline because the compare result is near the long-term mean after an elevated baseline tail. Finding confidence is 4/5 given 291 prior runs; improvement confidence is 2/5 because variability is high (CV 11.8%), the z-score is only -0.52, and no commit in the 17-commit range is a direct match.

Issue Overview

• Issue: #79899 - [Perf] Linux/x64: 1 Improvement on 9/22/2026 4:34:39 PM +00:00
• Type: Performance Improvement
• Date: 2026-09-22 | Queue: Ubuntu.2204.Amd64.Viper.Perf | OS/Arch: Ubuntu 22.04/x64 (WASM)
• Runtime stack: WASM + CoreCLR
• Commits: d4b2951c1fa5 -> 206bf81aa71b (17 commits)
• Labels: untriaged, perf-improvement, arch-x64, os-linux, branch-refs/heads/main, arch-wasm, kind-micro, runkind-micro, compilationmode-wasm, runtimetype-coreclr
• Historical Data: Available (291 prior runs; 298 total runs examined, including 7 after the reported change)
• Description: The issue reports one faster ImmutableSortedDictionary.TryGetValue benchmark for missing SmallClass keys.

Test Analysis

Test Name	Baseline	Compare	Delta	mu	sigma	n	z	Thresh%	Noise?	Assessment
System.Collections.TryGetValueFalse<SmallClass, SmallClass>.ImmutableSortedDictionary(Size: 512)	12.715 ms	8.766 ms	-31.05%	9.343 ms	1.106 ms	291	-0.522	23.67%	No	Borderline; delta exceeds the adaptive threshold, but the compare value is near the historical mean and follows an elevated baseline tail

Legend: mu=mean, sigma=standard deviation, n=prior runs, z=z-score, Thresh%=adaptive threshold

Finding Confidence: 4/5 - The analysis has comprehensive history (291 prior runs) and complete commit metadata. Confidence is not 5/5 because the signal has not been reproduced with isolated commit builds or bisection.

Candidate Stack Relevance

Commit	Touched Paths (summary)	Relevance	Notes
30ef94429b52	src/coreclr/jit/, including codegenwasm.cpp; CoreCLR helper definitions	Relevant	Adds a small-struct bulk write-barrier path, but the benchmark is dominated by read-only tree lookups and the reported gain is much larger than the commit's demonstrated 4-15% struct-copy gains.
1e619c3ab529	Broad src/coreclr/jit/ local-definition, SSA, value-numbering, and copy-propagation changes	Relevant	Could alter generated WASM code indirectly, but the change is broad and has no demonstrated connection to ImmutableSortedDictionary.TryGetValue.
4e6ff09589e4	src/coreclr/jit/lowerwasm.cpp	Relevant	WASM-specific address-mode correctness fix; its own performance evidence anticipated a small regression, not a large lookup improvement.
e4194874e40e	src/coreclr/System.Private.CoreLib/src/System/Collections/Generic/ComparerHelpers.cs	Relevant	Collection/comparer code is superficially close, but the surgical change only avoids an allocation while creating a comparer for Nullable<T>. This benchmark uses SmallClass keys and creates its comparer before the timed lookup loop.
7dcb8f855a94	JavaScript interop libraries and src/mono/browser/runtime/	Shared	Browser proxy-lifetime changes do not touch the CoreCLR lookup path; shared JS interop is exercised only around hosting, not inside this benchmark loop.

Confirmed Improvements

No improvement is confirmed. The autofiler's 31.05% decrease exceeds the 23.67% adaptive threshold, so it is not classified as noise, but the historical evidence conflicts with interpreting it as a commit-induced step change.

System.Collections.TryGetValueFalse<SmallClass, SmallClass>.ImmutableSortedDictionary(Size: 512) (-31.05%, borderline)

• Delta: 12.715 ms -> 8.766 ms (-31.05%)
• Historical Context: mu=9.343 ms (sigma=1.106 ms, n=291); z-score=-0.522; adaptive threshold=23.67%; CV=11.84%
• Assessment: Gradual drift/elevated baseline tail. The compare measurement is only 0.52 standard deviations below the historical mean, so it is not an exceptional fast result.
• Finding Confidence: 4/5 - Extensive history supports the classification, but no isolated verification has been run.
• Improvement Confidence: 2/5 - The raw delta is large, but high historical variance, an insignificant z-score, and the elevated baseline make a true code-induced improvement unlikely without reproduction.
• Likely Cause: No clear attribution
  • Affected Areas: Immutable collections, generic comparison, WASM/CoreCLR JIT
  • Stack Relevance: Unknown
  • Rationale: None of the 17 commits directly changes ImmutableSortedDictionary.TryGetValue or its timed lookup path. The only comparer-specific commit optimizes nullable comparer construction, which does not apply to SmallClass and occurs outside the measured loop; the remaining plausible CoreCLR JIT commits are too indirect to attribute against a drift-shaped, high-variance signal.
  • Candidates Considered:
    • 30ef94429b52 - Directly relevant to WASM/CoreCLR code generation, but optimizes GC-containing struct copies rather than dictionary reads.
    • 1e619c3ab529 - Broad JIT data-flow refactoring could affect generated code but provides no specific lookup mechanism.
    • 4e6ff09589e4 - WASM-specific lowering change is a correctness fix expected to be slightly slower.
    • e4194874e40e - Improves nullable comparer creation only; it does not apply to the benchmark's key type or timed operations.
• Verification Status: Not verified

Open Questions

• Historical Data: History is sufficient, but variability is high (CV=11.84%) and the baseline appears to come from an elevated tail.
• Confidence Limitations: The 17-commit range has multiple indirect JIT candidates and no direct benchmark-path change.
• Verification Needed: Rerun the baseline and compare commits in the same Helix job, then bisect only if the 31% improvement reproduces.
• Bisection Status: Not performed.

Recommended Actions

1. Reproduce endpoints - Run System.Collections.TryGetValueFalse<SmallClass, SmallClass>.ImmutableSortedDictionary(Size: 512) at d4b2951c1fa5 and 206bf81aa71b in one Helix job to control for the elevated baseline.
2. Bisect only after reproduction - If the endpoint run confirms a material gain, bisect the 17 commits, prioritizing 30ef94429b52, 1e619c3ab529, and 4e6ff09589e4.
3. Monitor subsequent runs - Confirm whether measurements remain near the 9.343 ms long-term mean; that outcome would support baseline-tail normalization rather than a durable new performance level.

Bisection Results

Test	Status	Culprit Commit	Relevance	Reported Diff (issue)	Bisected Diff (at culprit)
ImmutableSortedDictionary(Size: 512)	🔁 Did not reproduce	-	-	12.7→8.8 ms (-31%)	≈ (baseline ≈ head)

Total bisection time: 6m 38s
Tests measured: 1/1
Culprit found: 0/1

System.Collections.TryGetValueFalse<SmallClass, SmallClass>.ImmutableSortedDictionary(Size: 512)

• Status: inconclusive

Bisection details

• Baseline: 12.715 ms

Per-commit measurements (oldest → newest) (ms)

#	Commit	Result	Measurement
0	d4b2951c1fa5	baseline	8.647 ms
6	7dcb8f855a94	-	11.247 ms
11	e29c15ec947c	-	11.246 ms
17	206bf81aa71b	head	11.983 ms

Helix jobs

Job ID
78117ae2-b6b9-4d3f-a74d-2337ad727d25

🤖 Proposed Next Actions

Rule: R1b_bisection_noise — Likely noise — close & label.

Bisection measured the endpoints and the reported improvement did not reproduce on the harness for most tests (1/1, 100% ≥ 80% threshold) with no confirmed culprit — most likely measurement noise (or an un-labelled harness shift), not a real improvement.

#	Action	Status	Details
1	Add label	✅ Done	noise
2	Remove label	⏭️ Skipped (dry-run)	untriaged
3	Close issue	⏭️ Skipped (dry-run)	reason not_planned

🦆 Pre-bisection Re-analysis (advisory)

⚠️ Disputes the pre-bisection plan — advisory only, not applied

Flags: possible-noise, wasted-bisection-slot

Rationale: The sole finding has only 2/5 confidence that the improvement is real, with no z-score, sustained after-values, variance evidence, or likely commit provided. That is insufficient support for spending a bisection slot.

Suggested drop from bisection: System.Collections.TryGetValueFalse<SmallClass, SmallClass>.ImmutableSortedDictionary(Size: 512) (advisory — not applied)

This critique is advisory in the current configuration and did not change which tests were bisected.

🦆 Re-analysis (advisory)

⚠️ Disputes the analysis — advisory only, not applied

Flags: weak-bisection-evidence, possible-noise

Rationale: The low improvement confidence is appropriately cautious, but finding confidence of 4/5 and an actionable classification are not well supported by an inconclusive bisection, unknown relevance, and no reported variance or sustained-change evidence.

Suggested confidence: Finding Confidence → 3/5 (advisory — not applied)

This critique is advisory in the current configuration and did not change the confidences, culprit, or proposed actions above.

---

Generated by PerfTriageAgent on 2026-09-24 10:47 UTC