Nightly slow tests failed

[github-actions]

Nightly slow tests failed

Run: 27119252332

Check the run logs for which steps failed and close this issue once resolved.

[github-actions]

Also failed on run 27186791351.

[github-actions]

Also failed on run 27397920962.

[roryford]

Scoping: current nightly failure cause (run 27397920962, Jun 12)

TL;DR — already fixed on main. The Jun 12 failure was not a coverage-threshold miss. It was a single failing test inside the "Coverage thresholds" step, and the test in question has since been replaced. No action needed unless nightly fails again with a new cause.

What actually failed

The "Coverage thresholds" step runs swift test ... --enable-code-coverage and then scripts/check-coverage.sh, under set -euo pipefail. The swift test invocation itself failed:

Tests/ManifoldRuntimeTests/PromptContextPipelineConcurrencyTests.swift:59: error:
  -[ManifoldRuntimeTests.PromptContextPipelineConcurrencyTests test_threeProvidersRunConcurrently]
  XCTAssertLessThan failed: ("0.18033175 seconds") is not less than ("0.12 seconds")
ManifoldKitPackageTests.xctest failed ... Executed 2434 tests, 1 failure

Because swift test returned exit 1, pipefail aborted the step before check-coverage.sh ran. So:

• COVERAGE_THRESHOLDS_FAILED: true in the diagnostics only means the step failed — it does not mean a module fell below threshold.
• The per-module coverage table never printed; no module is actually under-covered. (For reference, thresholds live in scripts/check-coverage.sh: ManifoldInference/Runtime/PersistenceSwiftData = 74%, ManifoldMCP = 75%, set 10pp below the 2026-05-15 baseline. They are not implicated here.)

Root cause of the failing test (and why it's already resolved)

PromptContextPipelineConcurrencyTests.test_threeProvidersRunConcurrently asserted the pipeline runs three 50 ms providers concurrently (< 120 ms). But PromptContextPipeline.assemble actually walks providers sequentially (~150 ms with await between each). The test's contract contradicted the implementation, so it failed (and would be flaky at best on a loaded runner).

PR #1694 (commit 6adbcec7) replaced that file with PromptContextPipelineSequentialTests.swift, flipping the assertion to >= 140 ms to match the real sequential behavior (tracked by #1684 as the parallelization candidate). The failing run 57c14325 predates that commit; the old file no longer exists on main. This cause cannot recur.

Recommended fix path

None required — the offending test is gone from main. Optionally close this issue once a clean nightly run confirms green. If a future maintainer wants the pipeline to actually run providers concurrently, that's the open work in #1684 (fan-out via async let/withTaskGroup, then flip the sequential assertion to < 80 ms).

Broader pattern (this issue is a moving target)

This nightly job has failed on three distinct causes across recent runs:

1. Jun 8 (27119252332) — TrafficBoundaryAuditTest flagged ManifoldHuggingFace/DiffusionDownloadDelegate.swift (direct network I/O not in allowlist). That file was refactored/renamed; audit now passes. Move TrafficBoundaryAuditTest to per-PR CI #1706 moves TrafficBoundaryAuditTest to per-PR CI so this class of failure is caught before merge rather than ~24h later.
2. Jun 12 (27397920962) — the PromptContextPipelineConcurrencyTests timing failure above; fixed by docs: testing/CI principles, QA evaluation, and repeatable audit process #1694.
3. (Multiple earlier short-duration failures — likely build/compile or other intermittent steps; investigate per-run if they resurface.)

The takeaway for whoever picks this up: COVERAGE_THRESHOLDS_FAILED=true is a red herring here — always open the step log and look for the actual error: line, since coverage-step failures are dominated by the swift test sub-run, not the threshold gate.