Skip to content

Clean up search execution and add optional query CPU pool#87

Open
antonchen wants to merge 2 commits into
lightonai:mainfrom
antonchen:pr/search-execution-cleanup
Open

Clean up search execution and add optional query CPU pool#87
antonchen wants to merge 2 commits into
lightonai:mainfrom
antonchen:pr/search-execution-cleanup

Conversation

@antonchen
Copy link
Copy Markdown
Contributor

Summary

  • clean up the search handlers around a shared prepared-search execution path
  • add an optional query CPU worker pool so CUDA ingest can stay separate from query and rerank encoding
  • keep the change focused on search execution behavior and model-pool routing

Validation

  • cargo check -p next-plaid-api --features "openblas model"
  • cargo clippy -p next-plaid-api --all-targets --features "openblas model" -- -D warnings
  • cargo test -p next-plaid-api --lib --features model query_on_cpu_routes_logical_lanes_to_distinct_pools -- --nocapture
  • cargo test -p next-plaid-api --lib --features model build_search_request_config_rejects_zero_top_k -- --nocapture
  • cargo test -p next-plaid-api --lib --features model map_keyword_search_error_marks_malformed_match_as_bad_request -- --nocapture

Implemented entirely by Codex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant