Skip to content

Windows OpenAI socket smoke terminates instead of reporting a failed response #537

Description

@localai-bot

Summary

PR #524 run 31641616323 reaches the native Windows OpenAI API test, serves the final real-socket chat request, then exits with -1073740791 (0xC0000409) before doctest can print its result.

Evidence

  • Windows CPU job: https://github.com/mudler/vllm.cpp/actions/runs/31641616323/job/94265239385
  • test_openai_api_server.exe logs the final response from the first real-socket test, then the process terminates with no doctest summary.
  • tests/vllm/entrypoints/openai/test_api_server.cpp starts a raw std::thread, has aborting REQUIRE assertions after the thread becomes joinable, and only stops/joins at the normal end of the test. A failed response therefore unwinds through a joinable std::thread and calls std::terminate, masking the assertion.
  • The vendored cpp-httplib v0.49.0 closes each accepted socket immediately after writing. Upstream commit 8e702d3837b2164765ca1d98cb6d180ae4711e70 explains that this can generate an abortive RST on Windows and make a fully-written response appear as a failed read, then fixes it by half-closing and bounded-draining before close.
  • Upstream commit ae8356d86eabfd3ad4a969b55266fb3ecc2aa834 independently documents the same joinable-thread assertion masking and adds scoped teardown.

Required fix

  1. Make every real-socket OpenAI test tear its server thread down during assertion unwinding, so doctest failures remain observable.
  2. Backport the minimal upstream accepted-socket drain fix, preserving its 100 ms / 1 MiB bounds, so Windows clients do not see a successful response as a reset.
  3. Add RED-first behavioral coverage and mutation evidence for both guarantees; retain all existing request/status/body assertions and Windows release test execution.
  4. Rerun native Windows CPU and Vulkan jobs, the full PR gate, and the exact merged-SHA ten-platform release dry run before tagging.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions