Skip to content

test_serve_low_tools: the concurrency-cap assertion races the server-side counter and reads 3 under load #428

Description

@localai-bot

test_serve_low_tools asserts an exact peak concurrency of 2 and observes 3 under load.

Found while landing #421 (it is not caused by that change: the merged tree differs from main in fifteen files, none under tests/tools/, so the code under test was byte-identical to main).

What fails

tests/tools/test_serve_low_client.py:130, in test_request_shape_concurrency_cap_and_error_propagation:

AssertionError: 3 not less than or equal to 2
  self.assertLessEqual(_CompletionHandler.peak, 2)

Why it is a real defect and not just noise

The test drives run_usage_batch(..., max_concurrency=2) against an in-process ThreadingHTTPServer and then asserts the SERVER-side observed peak is exactly 2 — assertGreaterEqual(peak, 2) and assertLessEqual(peak, 2).

The client's semaphore slot is released after it finishes reading the response, while the server's active counter is decremented in the handler's finally after the body is written. Those two instants are not the same instant, so under scheduling delay the client can legitimately start request #3 while handler #1 has written its body but not yet run its finally. The server then observes 3 concurrent handlers although the client never exceeded its cap of 2. The assertion tests the server's accounting race, not the client's cap.

Evidence

  • FAIL: ctest --test-dir build-cpu -j 6 at load average 134 — 384/385, this the only failure, 40.91 s.
  • PASS: same binary, same tree, standalone ×2 (19.8 s, 18.7 s).
  • PASS: full sweep re-run at load 34 — 385/385.
  • PASS: both sweeps on the later tip (389/389 at load 89).

So it is load-gated, which is what a race looks like from outside.

Suggested shape of a fix

Assert what the test means: that the CLIENT never exceeded its cap. Either count in-flight requests on the client side, or have the handler decrement before writing the response body, or (weakest) keep the >= 2 lower bound and drop the exact upper bound. Whichever is chosen, the fix wants a RED-first demonstration under induced load, since an unloaded box passes today either way.

Roadmap intake placement in .agents/roadmap_v1.md is owed and is not in this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions