Use sliding rate limit windows

[erskingardner]

Summary

• replace fixed per-key rate limit counters with sliding minute/hour timestamp windows
• prune expired hits before admission so boundary crossings cannot reset the full budget
• keep rollback and cleanup behavior compatible with the existing bounded cache

Tests

• cargo test rate_limiter::tests -- --nocapture
• cargo test
• cargo clippy -- -D warnings
• just ci

Summary by CodeRabbit

• Bug Fixes
  • Improved rate limiting for more consistent throttling across minute and hour boundaries, reducing false rejections and smoothing request handling.
  • Rollbacks and stale-entry cleanup now behave reliably so admitted requests and stale limits are correctly reversed and removed, improving overall service stability.

[qodo-code-review]

ⓘ You've reached your Qodo monthly free-tier limit. Reviews pause until next month — upgrade your plan to continue now, or link your paid account if you already have one.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 886fe253-9445-4b93-a219-12fc30fd2c0c

📥 Commits

Reviewing files that changed from the base of the PR and between 05e8570 and 28c8290.

📒 Files selected for processing (1)

• src/rate_limiter.rs

---

Walkthrough

RateLimiter transitions from fixed-window counters to sliding-window per-key timestamp queues. Admission prunes expired timestamps, checks queue lengths against limits, and records hits. Rollback and cleanup now operate on timestamp queues. Tests validate minute- and hour-boundary sliding behavior.

Changes

Sliding-window rate limiter

Layer / File(s)	Summary
Data structure and documentation src/rate_limiter.rs	RateLimitEntry replaces fixed-window counters and window-start times with VecDeque<Instant> queues (minute_hits, hour_hits) and pruning helpers. Module imports and RateLimiter documentation updated for sliding-window semantics.
Sliding-window core operations src/rate_limiter.rs	check_and_increment prunes expired timestamps per key, enforces limits by comparing queue lengths, and records hits by pushing the current Instant into both queues; rollback_increment pops the latest timestamps from both queues and removes the key when empty; cleanup determines staleness from hour_hits.back(); test helper peek_counts returns queue lengths.
Boundary validation tests src/rate_limiter.rs	New tests use paused tokio time and incremental advance to verify minute- and hour-limit sliding-window allow/deny transitions across boundaries.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• marmot-protocol/transponder#50: Modifies rate limiter admission and result handling for cache-capacity rejection in the same rate limiter module.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Use sliding rate limit windows' directly and clearly describes the main change: replacing fixed counters with a sliding-window algorithm for rate limiting.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/issue-59-sliding-window

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

✅ Code Coverage Report

Metric	Value
Line Coverage	93.28%
Lines Covered	8398 / 9003
Change from master	+0.03% ✅

View detailed coverage

Download the coverage-html artifact from this workflow run to view the detailed coverage report.

Or run locally:

cargo llvm-cov --html
open target/llvm-cov/html/index.html

[coderabbitai]

🧹 Nitpick comments (1)

src/rate_limiter.rs (1)

123-132: 💤 Low value

Consider documenting memory scaling characteristics.

The sliding-window approach stores individual timestamps in VecDeque, which significantly increases per-entry memory compared to fixed-window counters. With default limits (240/min, 5000/hr), each entry at full utilization stores ~5240 Instant values (~84KB). At 100K entries under sustained load, this could reach several GB.

This tradeoff is inherent to true sliding windows and acceptable if expected, but consider:

1. Documenting the memory profile in the struct-level doc comment
2. Using VecDeque::shrink_to_fit() in prune() if entries frequently spike then idle

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/rate_limiter.rs` around lines 123 - 132, The struct-level comment for
RateLimiter should document its memory-scaling tradeoffs: note that
RateLimitEntry stores per-event timestamps in a VecDeque (true sliding window)
and with defaults (240/min, 5000/hr) each hot entry can hold ~5240 Instants
(~84KB) so N entries can consume many GB; add this explanation to the doc
comment for RateLimiter. Also modify prune() to call VecDeque::shrink_to_fit()
on entries that have been pruned to release excess allocation after spikes
(update logic around RateLimitEntry manipulation inside prune() to call
shrink_to_fit when an entry becomes idle or after heavy trimming).

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/rate_limiter.rs`:
- Around line 123-132: The struct-level comment for RateLimiter should document
its memory-scaling tradeoffs: note that RateLimitEntry stores per-event
timestamps in a VecDeque (true sliding window) and with defaults (240/min,
5000/hr) each hot entry can hold ~5240 Instants (~84KB) so N entries can consume
many GB; add this explanation to the doc comment for RateLimiter. Also modify
prune() to call VecDeque::shrink_to_fit() on entries that have been pruned to
release excess allocation after spikes (update logic around RateLimitEntry
manipulation inside prune() to call shrink_to_fit when an entry becomes idle or
after heavy trimming).

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0b158bc9-334e-4161-8eb5-abf8d5dd38f1

📥 Commits

Reviewing files that changed from the base of the PR and between 8bd605e and 05e8570.

📒 Files selected for processing (1)

• src/rate_limiter.rs