Skip to content

fix(ci): resolve two pre-existing test-infra failures (event-bus flake + gate-smoke allowlist)#5063

Closed
YellowSnnowmann wants to merge 4 commits into
tinyhumansai:mainfrom
YellowSnnowmann:fix/event-bus-test-broadcast-capacity
Closed

fix(ci): resolve two pre-existing test-infra failures (event-bus flake + gate-smoke allowlist)#5063
YellowSnnowmann wants to merge 4 commits into
tinyhumansai:mainfrom
YellowSnnowmann:fix/event-bus-test-broadcast-capacity

Conversation

@YellowSnnowmann

@YellowSnnowmann YellowSnnowmann commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fixes two pre-existing full-suite unit-test flakes via real test isolation, and clears a stale gate-smoke allowlist entry.
    • learning::startup::tests::learning_subscriber_fires_*
    • web_chat::event_bus::tests::automation_halt_subscriber_handle_publishes_correct_payload
  • Root cause: both tests asserted against the process-global broadcast bus. Under cargo test --lib (~12,600 tests in one process, run concurrently) they observed DomainEvents published by other tests — cross-test interference, not a capacity shortfall.
  • Fix: give each test its own bus / filter for its own event, so no foreign publisher can perturb the assertion. #[cfg(test)]-only; the shipped build is byte-identical.

Problem

The event bus is a single OnceLock<EventBus> shared by every unit test (they all run in one process). Two tests read from that shared global:

  1. learning_subscriber_fires_* subscribed to the global bus and published a DocumentCanonicalized, then asserted the email-signature subscriber consumed it — but any of the other ~12.6k tests publishing to the same global bus could be delivered first.
  2. automation_halt_subscriber_handle_publishes_correct_payload called handle(...) directly, then try_recv()'d the shared web-channel bus — under load it picked up a foreign test's event first and failed with an "event name mismatch".

Both pass in isolation and on the changed-files fast lane, and fail only under the full parallel suite — which any Cargo.toml-touching PR triggers (the rust-core-full coverage lane).

Note: an earlier attempt raised #[cfg(test)] broadcast/candidate-ring capacity floors. CI proved it ineffective (a capacity bump cannot stop cross-test interference), so that commit was reverted here in favour of the isolation fix below.

Solution

  • learning::startup — rewrite the subscriber test to publish DocumentCanonicalized on a private EventBus::create(64) with a locally-subscribed EmailSignatureSubscriber, asserting delivery on that owned bus instead of the shared global. Add a synchronous email_signature_subscriber_registers_on_channel_independent_path guard covering the channel-independent registration path (fix(learning): email-signature subscriber + rebuild trigger silently depend on channels being configured #5003).
  • web_chat::event_bus — drain foreign events from the shared web-channel bus and match on event == "automation_halt" rather than taking whatever is first in the buffer.
  • Revert the ineffective #[cfg(test)] capacity floors in event_bus/bus.rs and learning/candidate.rs back to origin/main.

No new dependency, no serial_test, no weakened assertion, no production change.

Verification

  • Locally: the affected modules pass (cargo test --lib learning::startup web_chat::event_bus → 11 passed).
  • The flake only reproduces under the full parallel suite, so this PR's full-suite coverage run is the definitive venue — the previously-failing tests should now pass there.

Second fix: gate-smoke allowlist drift

Acknowledges openhuman/composio/action_tool.rs in the Rust Feature-Gate Smoke lane's
EXPECTED allowlist. That file gained a #[cfg(feature = "flows")] test in #4853 but was
never allowlisted, so the guard's ACTUAL-vs-EXPECTED diff has failed on every PR since.
This is the guard's own sanctioned maintenance (its error says "Update the EXPECTED
allowlist"), not a weakening — it still validates every other file. Replicated the guard's
grep+diff locally to confirm this is the only drift and the diff is now empty.

Submission Checklist

  • Tests added or updated — the fix is the test change (isolation + a new synchronous registration guard); the full suite is the regression venue for the load-only flake.
  • Diff coverage ≥ 80% — changed lines are all in #[cfg(test)] test bodies, executed by the run itself.
  • Coverage matrix updated — N/A: test-infrastructure fix, no user-facing feature.
  • All affected feature IDs listed — N/A.
  • No new external network dependencies — none.
  • Manual smoke checklist updated — N/A: #[cfg(test)]-only; the shipped build is byte-identical.
  • Linked issue closed via Closes #NNNN/A: no tracking issue; see ## Related.

Impact

Related

…lake

Under cfg(test) the crate's ~12.6k unit tests share two process-global bounded
resources sized for production: the event bus's 256-slot broadcast channel
(src/core/event_bus/bus.rs) and the learning candidate ring's 1024 entries
(src/openhuman/learning/candidate.rs, whose drainer does not run in the unit
suite). Under the concurrent full-suite publish load a slow subscriber lags and
drops events (RecvError::Lagged), and the candidate ring evicts a test's own
candidates — intermittently failing bus-delivery tests such as
learning::startup::tests::learning_subscriber_fires_with_no_channel_configured.

Raise both floors to 65536 for the TEST BINARY ONLY (cfg(test)); production keeps
256 / 1024 and ships byte-identical. No new dependency, no serialization, no
weakened assertion — the shared globals simply get enough headroom that neither
drops nor evicts under the artificial suite-wide load.
@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 4 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 760f8975-a640-4040-b735-e7480aee1f53

📥 Commits

Reviewing files that changed from the base of the PR and between 26b8250 and e872481.

📒 Files selected for processing (2)
  • src/openhuman/learning/startup.rs
  • src/openhuman/web_chat/event_bus.rs
📝 Walkthrough

Walkthrough

Test builds now allocate larger capacities for the global event bus and learning buffer to reduce drops during concurrent tests. The CI feature-gate smoke-test allowlist also includes an additional gated test module.

Changes

Test capacity configuration

Layer / File(s) Summary
Configure test singleton capacities
src/core/event_bus/bus.rs, src/openhuman/learning/candidate.rs
The global event bus and buffer use capacities of at least 1 << 16 under tests, while production defaults remain unchanged.
Update feature-gate allowlist
.github/workflows/ci-lite.yml
The smoke-test guard acknowledges openhuman/composio/action_tool.rs as an expected gated test module.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

Suggested labels: bug, rust-core

Suggested reviewers: oxoxdev

Poem

A rabbit watched the tests race by,
So widened channels filled the sky.
The bus and buffer hold much more,
While CI guards one module’s door.
Fewer dropped hops—hip-hip hooray!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the two main test-infra fixes in the changeset.

Comment @coderabbitai help to get the list of available commands.

@YellowSnnowmann
YellowSnnowmann marked this pull request as ready for review July 20, 2026 12:15
@YellowSnnowmann
YellowSnnowmann requested a review from a team July 20, 2026 12:15
@coderabbitai coderabbitai Bot added the working A PR that is being worked on by the team. label Jul 20, 2026
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 20, 2026
The `Rust Feature-Gate Smoke` lane guards that every feature-gated test module is
acknowledged in an EXPECTED allowlist. `composio/action_tool.rs` gained a
`#[cfg(feature = "flows")]` test in tinyhumansai#4853 but was never added to the list, so the
guard's ACTUAL-vs-EXPECTED diff fails on it — on every PR since. This is the guard's
own sanctioned maintenance (its error message says "Update the EXPECTED allowlist"),
not a weakening: the guard still validates every other file. Replicated the guard's
grep+diff locally to confirm this is the only drift and the diff is now empty.
@YellowSnnowmann YellowSnnowmann changed the title fix(event-bus): raise test-only capacity floors to fix a full-suite flake fix(ci): resolve two pre-existing test-infra failures (event-bus flake + gate-smoke allowlist) Jul 20, 2026
@coderabbitai coderabbitai Bot added bug rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure. and removed working A PR that is being worked on by the team. labels Jul 20, 2026
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 20, 2026
…city floor

The earlier test-only capacity floor (5c49a96) did not fix the flake:
the two failing tests share the process-global broadcast bus, so under the
parallel suite they receive events published by *other* tests — a capacity
bump cannot fix cross-test interference. Replace it with real isolation.

- learning::startup: rewrite `learning_subscriber_fires_*` to publish a
  DocumentCanonicalized event on a private `EventBus::create(64)` with a
  locally-subscribed `EmailSignatureSubscriber`, instead of asserting on the
  shared global bus. Add a synchronous `register_email_signature_subscriber`
  guard test for the channel-independent registration path (tinyhumansai#5003).
- web_chat::event_bus: the automation-halt test reads from the shared
  web-channel bus; drain foreign events and match on `event == "automation_halt"`
  rather than taking whatever is first in the buffer.
- Revert the `#[cfg(test)]` capacity floors in event_bus/bus.rs and
  learning/candidate.rs back to origin/main — no longer needed.
@github-project-automation github-project-automation Bot moved this from Todo to Done in Team Openhuman Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant