Skip to content

[ENH]: granular lock for posting list #4937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: 06-18-_enh_more_concurrent_blockfile
Choose a base branch
from
Open

[ENH]: granular lock for posting list #4937

wants to merge 2 commits into from
+86 −66

Conversation

sanketkedia
Copy link
Contributor

@sanketkedia sanketkedia commented Jun 25, 2025 •
edited
Loading

Description of changes

Summarize the changes made by this PR.

  • Improvements & Bug fixes
    • Takes a partitioned mutex on the posting list head instead of a big fat mutex. This combined with the granular locking in the blockfiles should give more concurrency to spann index add.
  • New functionality
    • ...

Test plan

How are these changes tested?

  • Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Documentation Changes

None

@sanketkedia sanketkedia mentioned this pull request Jun 25, 2025
Open
1 task
@sanketkedia Graphite App
Copy link
Contributor Author

sanketkedia commented Jun 25, 2025 •
edited
Loading

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@github-actions GitHub Actions
Copy link

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

  • Can you think of any use case in which the code does not behave as intended? Have they been tested?
  • Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
  • If appropriate, are there adequate property based tests?
  • If appropriate, are there adequate unit tests?
  • Should any logging, debugging, tracing information be added or removed?
  • Are error messages user-friendly?
  • Have all documentation changes needed been made?
  • Have all non-obvious changes been commented?

System Compatibility

  • Are there any potential impacts on other parts of the system or backward compatibility?
  • Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

  • Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

@sanketkedia sanketkedia changed the title [ENH]: granular lock in spann index [ENH]: granular lock for posting list Jun 25, 2025
@sanketkedia sanketkedia marked this pull request as ready for review June 25, 2025 06:08
@propel-code-bot propel-code-bot
Copy link
Contributor

Implement Granular Locking for Posting List in Spann Index

This PR refactors the locking mechanism around the posting list in the Spann index by replacing a coarse-grained global mutex with an AysncPartitionedMutex per posting list head. This change aims to increase concurrency for write operations and complements granular locking at the blockfile level. The update touches both spann internals and distributed_spann tests to ensure synchronization updates do not degrade correctness.

Key Changes:
• Replaces a single global mutex protecting the posting list with a per-head partitioned mutex (AysncPartitionedMutex<u32>) in SpannIndexWriter.
• Removes the use of Arc<tokio::sync::Mutex<BlockfileWriter>> for posting lists; posting list writer is now managed with partitioned locks.
• Updates all code paths (including add, update, split, merge, and GC operations) to take granular locks when accessing or modifying a specific posting list head.
• Replaces usages of .lock().await with granular lock acquisitions and direct accesses in all affected methods and tests.
• Modifies tests and consumers (including distributed_spann.rs) to align with the new concurrency primitives.

Affected Areas:
• rust/index/src/spann/types.rs
• rust/segment/src/distributed_spann.rs
• SpannIndexWriter posting list update, split, merge, and commit logic
• Unit and integration tests for spann index

This summary was automatically generated by @propel-code-bot

propel-code-bot[bot]
propel-code-bot bot reviewed Jun 25, 2025
View reviewed changes
rust/index/src/spann/types.rs
@@ -11,6 +11,7 @@ use chroma_blockstore::{
provider::{BlockfileProvider, CreateError, OpenError},
BlockfileFlusher, BlockfileReader, BlockfileWriter, BlockfileWriterOptions,
};
use chroma_cache::AysncPartitionedMutex;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[NitPick]

There's a typo in the class name AysncPartitionedMutex which should be AsyncPartitionedMutex.

propel-code-bot[bot]
propel-code-bot bot reviewed Jun 25, 2025
View reviewed changes
rust/index/src/spann/types.rs
// be changed in future if perf is not satisfactory.
pub posting_list_writer: Arc<tokio::sync::Mutex<BlockfileWriter>>,
pub posting_list_writer: BlockfileWriter,
pub posting_list_partitioned_mutex: Arc<AysncPartitionedMutex<u32>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BestPractice]

Same typo in the field name - AysncPartitionedMutex should be AsyncPartitionedMutex.

propel-code-bot[bot]
propel-code-bot bot reviewed Jun 25, 2025
View reviewed changes
rust/index/src/spann/types.rs
@@ -429,7 +429,8 @@ impl SpannIndexWriter {
cleaned_up_hnsw_index: None,
hnsw_provider,
blockfile_provider,
posting_list_writer: Arc::new(tokio::sync::Mutex::new(posting_list_writer)),
posting_list_writer,
posting_list_partitioned_mutex: Arc::new(AysncPartitionedMutex::new(())),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BestPractice]

Same typo again in the initialization - AysncPartitionedMutex should be AsyncPartitionedMutex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@propel-code-bot propel-code-bot[bot] propel-code-bot[bot] left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@sanketkedia