Skip to content

Add concurrency failure-harness checks - #14

Closed
lucarlig wants to merge 1 commit into
mainfrom
user/luca/concurrency-ci-failure
Closed

Add concurrency failure-harness checks#14
lucarlig wants to merge 1 commit into
mainfrom
user/luca/concurrency-ci-failure

Conversation

@lucarlig

@lucarlig lucarlig commented May 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds dedicated Loom/Miri checks for concurrency and interpreter-friendly regression coverage.

This PR intentionally does not fix the issues the suite exposes. It only adds the test harness and CI jobs so the failures are visible and reproducible.

No test-only public API or internal-test-hooks feature is exposed. Loom uses a normal loom Cargo feature. Miri uses cfg(miri).

Test layout

crates/contextforge-gateway-rs-lib/src/gateway/
  session_manager.rs       # Loom concurrency model tests live beside the private helpers
  miri_checks/
    namespace_routing.rs
    config_serialization.rs

Reproduce from main

git checkout main
git pull
git checkout -b reproduce-concurrency-ci-failure
git cherry-pick 4e65148

Run the Loom check:

cargo test --locked -p contextforge-gateway-rs-lib \
  --features loom \
  gateway::session_manager::concurrency

Expected result: 3 pass, 1 fail.

Run the Miri checks:

cargo +nightly miri setup
cargo +nightly miri test --locked -p contextforge-gateway-rs-lib miri_checks

Expected result: 4 pass, 2 fail.

Regression coverage map

Every issue called out below has a regression test in this PR.

Issue Regression test(s) Command
Concurrent session return can erase a live backend service gateway::session_manager::concurrency::concurrent_borrowers_do_not_erase_returned_transport cargo test --locked -p contextforge-gateway-rs-lib --features loom gateway::session_manager::concurrency
Overlapping backend prefixes can route tool calls to the wrong backend gateway::mcp_gateway::miri_checks::longest_backend_prefix_wins_for_tools cargo +nightly miri test --locked -p contextforge-gateway-rs-lib miri_checks
Overlapping backend prefixes can route resource reads to the wrong backend gateway::mcp_gateway::miri_checks::longest_backend_prefix_wins_for_resources cargo +nightly miri test --locked -p contextforge-gateway-rs-lib miri_checks

Additional non-failing coverage in the suite:

  • gateway::session_manager::concurrency::cleanup_does_not_resurrect_returned_transport
  • gateway::session_manager::concurrency::different_sessions_do_not_interfere
  • gateway::session_manager::concurrency::multiple_backends_preserve_unborrowed_services
  • gateway::mcp_gateway::miri_checks::missing_separator_does_not_match
  • gateway::mcp_gateway::miri_checks::hyphen_and_underscore_backend_names_are_distinct
  • gateway::session_store::miri_checks::session_mapping_msgpack_round_trip
  • gateway::session_store::miri_checks::user_session_key_msgpack_round_trip

Issues exposed

1. Concurrent session return can erase a live backend service

Concurrent calls for the same downstream MCP session can erase a live backend service in SessionManager.

Bad interleaving:

  1. Request A borrows backend backend-a, so the map temporarily stores None.
  2. Request B borrows while A owns the service and gets a holder containing None.
  3. Request A returns the real service.
  4. Request B returns None and overwrites the real service.

Observed Loom failure:

left: Some(None)
right: Some(Some(7))

2. Overlapping backend prefixes can route to the wrong backend

Backend tool/resource names are split by matching the first backend-name prefix. If one backend name prefixes another, the shorter name can be selected first.

Example:

backends: counter, counter-one
tool: counter-one-increment

Bad parse:

backend: counter
tool: one-increment

Correct parse:

backend: counter-one
tool: increment

Observed Miri test failures:

left: Some(("counter", "one-increment"))
right: Some(("counter-one", "increment"))
left: Some(("counter", "one-memo://insights"))
right: Some(("counter-one", "memo://insights"))

@lucarlig
lucarlig force-pushed the user/luca/concurrency-ci-failure branch 4 times, most recently from 29e9d57 to 8022cc0 Compare May 15, 2026 10:08
@lucarlig lucarlig changed the title Add concurrency checks and fix session return handling Add concurrency failure-harness checks May 15, 2026
@lucarlig
lucarlig force-pushed the user/luca/concurrency-ci-failure branch from 8022cc0 to d75bf46 Compare May 15, 2026 10:19
Signed-off-by: lucarlig <luca.carlig@ibm.com>
@lucarlig
lucarlig force-pushed the user/luca/concurrency-ci-failure branch from d75bf46 to 4e65148 Compare May 15, 2026 10:27
@lucarlig

Copy link
Copy Markdown
Contributor Author

@dawid-nowak what do you think about miri and loom/shuttle will this tool help us or we can close and ignore?

@dawid-nowak

dawid-nowak commented May 20, 2026

Copy link
Copy Markdown
Contributor

I haven't looked into it to be honest. Let's leave it hanging it might be useful.

@lucarlig

Copy link
Copy Markdown
Contributor Author

this is legacy at this point not relevant for sessionless

@lucarlig lucarlig closed this Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants