Skip to content

perf(vector): parallelize large IVF scans - #144

Merged
ppodolsky merged 1 commit into
mainfrom
benchmark-implementation
Aug 11, 2026
Merged

perf(vector): parallelize large IVF scans#144
ppodolsky merged 1 commit into
mainfrom
benchmark-implementation

Conversation

@ppodolsky

Copy link
Copy Markdown
Contributor

Summary

  • parallelize large IVF-TQ and binary-IVF scans with bounded worker-local tasks
  • cover both distinct top-k retrieval and multi-value combined document scoring
  • preserve serial execution for small scans, single-thread pools, and WASM
  • add forced-parallel correctness tests plus ignored reproducible benchmarks

Benchmarks

Release mode on 8 Rayon threads:

  • IVF-TQ distinct, 131072 postings, dim 128: 1.042 ms -> 0.460 ms (2.26x)
  • IVF-TQ combined, 131072 postings, dim 128: 4.397 ms -> 1.902 ms (2.31x)
  • Binary IVF distinct, 131072 postings, 2048 bits: 0.800 ms -> 0.465 ms (1.72x)
  • Binary IVF combined, 131072 postings, 2048 bits: 1.463 ms -> 1.304 ms (1.12x)

At the 65536-posting threshold, all four paths remain faster than forced serial execution. Binary combined gains are smaller because document aggregation and sorting dominate its runtime.

Validation

  • cargo fmt --all -- --check
  • cargo clippy -p hermes-core --features native --all-targets -- -D warnings
  • cargo clippy -p hermes-core --no-default-features --features native --all-targets -- -D warnings
  • RUSTFLAGS=-Dwarnings cargo check -p hermes-core --no-default-features --features wasm
  • cargo test -p hermes-core --lib (1051 passed, 9 ignored)

@ppodolsky
ppodolsky merged commit 899034a into main Aug 11, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant