perf: reduce memory allocations with intelligent capacity pre-allocation

[am-miracle]

Description

Optimizes vector allocations across hot paths by using Vec::with_capacity() instead of Vec::new(), reducing memory reallocations and improving performance

Related Issues

Vector reallocations are a hidden performance tax in hot code paths. When Vec::new() is used without capacity hints, Rust doubles the vector size each time it runs out of space, causing:

• Repeated memory allocations during batch operations
• Memory copying as vectors grow
• Cache misses from memory fragmentation
• Increased GC pressure from temporary allocations

This is especially costly in operations that process hundreds or thousands of items in tight loops.

Solution

Systematically apply intelligent capacity pre-allocation using three strategies:

1. Exact-Size Pre-allocation (When Size is Known)

// Before: Dynamic reallocation in every loop iteration
for item in items {
    let mut kvs = Vec::new();              // starts at 0 capacity
    let mut key_parts = Vec::new();        // starts at 0 capacity
    for prop in properties {               // reallocates multiple times
        kvs.push(...);
    }
}

// After: Pre-allocate exact size
let properties_len = properties.len();     // calculate once
for item in items {
    let mut kvs = Vec::with_capacity(properties_len);        // allocates once
    let mut key_parts = Vec::with_capacity(properties_len);  // allocates once
    for prop in properties {               // no reallocations
        kvs.push(...);
    }
}

Applied to: aggregate.rs, group_by.rs

2. Heuristic Pre-allocation (When Typical Size is Known)

// Before: Grows from 0 → 4 → 8 → 16 → 32 (multiple allocations)
let mut connected_nodes = Vec::new();

// After: Single allocation for typical case
let mut connected_nodes = Vec::with_capacity(32);  // Most nodes have 5-50 connections

Applied to: node_connections.rs (32 connections), update.rs (16 items for mutations)

3. Iterator Size Hints (When Iterator Provides Bounds)

// Smart capacity based on iterator hints with sensible fallbacks
let mut vec = match self.inner.size_hint() {
    (_, Some(upper)) => Vec::with_capacity(upper),           // use upper bound
    (lower, None) if lower > 0 => Vec::with_capacity(lower), // use lower bound
    _ => Vec::with_capacity(16),                             // reasonable default
};

Applied to: update.rs

4. Explicit Collection (Avoid Hidden Allocations)

// Before: .collect() doesn't know final size
let mut results: Vec<_> = doc_scores.into_iter().collect();

// After: Pre-allocate exact size
let mut results = Vec::with_capacity(doc_scores.len());
results.extend(doc_scores);

Applied to: bm25.rs (2 locations)

Closes #

Checklist when merging to main

• No compiler warnings (if applicable)
• Code is formatted with rustfmt
• No useless or dead code (if applicable)
• Code is easy to understand
• Doc comments are used for all functions, enums, structs, and fields (where appropriate)
• All tests pass
• Performance has not regressed (assuming change was not to fix a bug)
• Version number has been updated in helix-cli/Cargo.toml and helixdb/Cargo.toml

Additional Notes

Greptile Overview

Updated On: 2025-10-16 22:05:05 UTC

Summary

Optimizes vector allocations across hot paths by pre-allocating capacity instead of starting at zero, reducing memory reallocations and improving performance in batch operations.

Key Changes

Highly Effective (Exact-Size Pre-allocation)

• aggregate.rs, group_by.rs: Pre-allocate kvs and key_parts vectors with exact size from properties.len() in hot loops - excellent optimization since size is known upfront
• bm25.rs: Replace .collect() with Vec::with_capacity() + .extend() for search results - eliminates hidden reallocations
• update.rs: Use iterator size hints with sensible 16-element fallback - good balance

Needs Reconsideration

• node_connections.rs: Pre-allocates 32 elements for connections without data supporting the "5-50 connections" claim - may waste memory for sparse graphs or be insufficient for dense nodes
• nodes_by_label.rs: Pre-allocates based on limit parameter without accounting for label filtering - if limit=100_000 but only 10 nodes match the label, wastes 99,990 slots

Testing

• Comprehensive test suite with correctness tests and performance benchmarks
• Covers aggregate, group_by, update, and BM25 operations with varying dataset sizes

Recommendations

• The core optimizations (aggregate.rs, group_by.rs, bm25.rs) are excellent
• Consider reverting the speculative pre-allocations in node_connections.rs and nodes_by_label.rs unless backed by profiling data

Important Files Changed

File Analysis

Filename	Score	Overview
helix-db/src/helix_engine/traversal_core/ops/util/aggregate.rs	5/5	Added exact-size pre-allocation for kvs and key_parts vectors using properties.len(), eliminating reallocations in hot loop
helix-db/src/helix_engine/traversal_core/ops/util/group_by.rs	5/5	Added exact-size pre-allocation for kvs and key_parts vectors using properties.len(), identical pattern to aggregate.rs
helix-db/src/helix_engine/bm25/bm25.rs	5/5	Replaced .collect() with explicit Vec::with_capacity() + .extend() to pre-allocate exact size for search results (2 locations)
helix-db/src/helix_gateway/builtin/node_connections.rs	4/5	Pre-allocated vectors with capacity 32 for node connections; may waste memory if nodes have fewer connections or be insufficient for highly-connected nodes
helix-db/src/helix_gateway/builtin/nodes_by_label.rs	3/5	Pre-allocates based on limit parameter without accounting for label filtering; may waste significant memory when few nodes match the label

Sequence Diagram

sequenceDiagram
    participant Client
    participant API as Gateway API
    participant Agg as Aggregate/GroupBy
    participant Update as Update Operation
    participant BM25 as BM25 Search
    participant Storage as Graph Storage

    Note over Agg,BM25: Hot Paths with Capacity Optimizations

    Client->>API: Query request (aggregate/group_by)
    API->>Storage: Read transaction
    Storage->>Agg: Iterator over nodes
    
    rect rgb(200, 255, 200)
        Note over Agg: BEFORE: Vec::new() reallocates<br/>in every loop iteration
        Note over Agg: AFTER: Vec::with_capacity(properties.len())<br/>pre-allocates exact size once
    end
    
    loop For each node
        Agg->>Agg: Push to kvs & key_parts<br/>(no reallocation needed)
    end
    
    Agg-->>Client: Aggregated results

    Client->>API: Update request
    API->>Storage: Write transaction
    Storage->>Update: Iterator over nodes
    
    rect rgb(200, 255, 200)
        Note over Update: Uses iterator size_hint()<br/>or defaults to capacity 16
    end
    
    Update->>Storage: Update nodes & indices
    Update-->>Client: Updated nodes

    Client->>API: BM25 search request
    API->>BM25: search(query, limit)
    BM25->>Storage: Read term postings
    
    rect rgb(200, 255, 200)
        Note over BM25: BEFORE: .collect() doesn't know size<br/>AFTER: Vec::with_capacity(doc_scores.len())<br/>then .extend() - exact allocation
    end
    
    BM25->>BM25: Sort & truncate results
    BM25-->>Client: Ranked document IDs

Loading

[greptile-apps]

_{10 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

style: Missing newline at end of file - rustfmt typically adds one

Suggested change

	[[test]]
	name = "capacity_optimization_benches"
	path = "benches/capacity_optimization_benches.rs"
	[[test]]
	name = "capacity_optimization_benches"
	path = "benches/capacity_optimization_benches.rs"

Prompt To Fix With AI

This is a comment left during a code review.
Path: helix-db/Cargo.toml
Line: 80:83

Comment:
**style:** Missing newline at end of file - `rustfmt` typically adds one

```suggestion
[[test]]
name = "capacity_optimization_benches"
path = "benches/capacity_optimization_benches.rs"
```

How can I resolve this? If you propose a fix, please make it concise.

[greptile-apps]

style: Pre-allocating 32 may waste memory if most nodes have fewer connections (1-5), or be insufficient if nodes have 100+ connections. Without profiling data supporting the "5-50 connections" assumption, using HashSet::new() and Vec::new() would be safer

Suggested change

	// Pre-allocate with reasonable capacity for typical graph nodes
	// Most nodes have 5-50 connections; 32 is a good middle ground (power of 2 for efficient allocation)
	let mut connected_node_ids = HashSet::with_capacity(32);
	let mut connected_nodes = Vec::with_capacity(32);
	let mut connected_node_ids = HashSet::new();
	let mut connected_nodes = Vec::new();

Prompt To Fix With AI

This is a comment left during a code review.
Path: helix-db/src/helix_gateway/builtin/node_connections.rs
Line: 87:90

Comment:
**style:** Pre-allocating 32 may waste memory if most nodes have fewer connections (1-5), or be insufficient if nodes have 100+ connections. Without profiling data supporting the "5-50 connections" assumption, using `HashSet::new()` and `Vec::new()` would be safer

```suggestion
    let mut connected_node_ids = HashSet::new();
    let mut connected_nodes = Vec::new();
```

How can I resolve this? If you propose a fix, please make it concise.

[greptile-apps]

style: Pre-allocating based on limit doesn't account for filtering by label. If the label matches few nodes, this wastes memory. If limit=100_000 but only 10 nodes match, you've allocated 100k slots unnecessarily. Consider using Vec::new() or a smaller default like 16

Suggested change

	// Pre-allocate based on limit parameter (if provided) or reasonable default
	// Cap at 100_000 for DoS protection while allowing legitimate large queries
	let initial_capacity = match limit {
	Some(n) if n <= 100_000 => n, // Trust reasonable limits
	Some(_) => 100_000, // Cap excessive limits
	None => 100, // Reasonable default for unlimited queries
	};
	let mut nodes_json = Vec::with_capacity(initial_capacity);
	// Start with reasonable default and let Vec grow as needed
	let mut nodes_json = Vec::new();

Prompt To Fix With AI

This is a comment left during a code review.
Path: helix-db/src/helix_gateway/builtin/nodes_by_label.rs
Line: 82:89

Comment:
**style:** Pre-allocating based on `limit` doesn't account for filtering by label. If the label matches few nodes, this wastes memory. If `limit=100_000` but only 10 nodes match, you've allocated 100k slots unnecessarily. Consider using `Vec::new()` or a smaller default like 16

```suggestion
    // Start with reasonable default and let Vec grow as needed
    let mut nodes_json = Vec::new();
```

How can I resolve this? If you propose a fix, please make it concise.

[xav-db]

this might be redundant now given our partial rewrite using arenas?