Skip to content

Release/v2.33.1 - #120

Merged
Lyellr88 merged 2 commits into
MARM-mainfrom
release/v2.33.1
Jul 31, 2026
Merged

Release/v2.33.1#120
Lyellr88 merged 2 commits into
MARM-mainfrom
release/v2.33.1

Conversation

@Lyellr88

@Lyellr88 Lyellr88 commented Jul 31, 2026

Copy link
Copy Markdown
Owner

v2.33.1 - Recall and Consolidation Correctness Fixes

This patch carries forward two independent correctness fixes found during the Phase 4 fusion bake-off. RRF itself was not merged; min-max remains the shipped fusion method.

  • Recall now scores every candidate in one request against the same reference time, preventing equal timestamps from being reordered by per-row clock drift.
  • Write-time consolidation now compares CONSOLIDATION_THRESHOLD against raw cosine similarity not the blended keyword/semantic/temporal ranking score. It explicitly uses the semantic lane, checks multiple candidates, and declines to merge when cosine is unavailable. Fixes Hybrid recall similarity is RRF-scaled, breaking cosine-based consolidation threshold #113.
  • Added regression coverage for temporal stability, raw-cosine thresholding, syntax-heavy content, missing-cosine safety, and candidates displaced by fusion ranking.
  • Updated v2.33.1 version surfaces, changelog, benchmark documentation, packaged docs, and Console Dependabot coverage.

Validation completed: 899 passed, 2 skipped; Ruff, MCP schema validation, version audit, and wheel/sdist build all pass.

Summary by CodeRabbit

  • Bug Fixes

    • Improved semantic duplicate detection by evaluating multiple candidates and using raw meaning similarity.
    • Corrected recency scoring so results share a consistent reference time.
    • Preserved exact keyword search behavior while improving hybrid retrieval accuracy.
  • Documentation

    • Updated installation guides, technical documentation, benchmarks, and performance figures for v2.33.1.
    • Clarified consolidation thresholds and revised estimated consolidation costs.
  • Release

    • Updated the MARM version to v2.33.1 across published packages and deployment configurations.

Lyellr88 added 2 commits July 30, 2026 23:36
Score each recall candidate set against one reference time and compare consolidation thresholds against raw cosine similarity.

Force semantic duplicate checks, inspect a bounded candidate set, and preserve normal MCP response shapes. Add regression coverage, refresh benchmark documentation, and prepare v2.33.1.

Fixes #113
Enable Dependabot coverage for the console, tighten scoring comments, and sync the packaged README navigation.
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Changes

Retrieval and consolidation corrections

Layer / File(s) Summary
Shared temporal scoring reference
marm-mcp-server/marm_mcp_server/core/memory_recall.py, marm-mcp-server/marm_mcp_server/core/memory_utils.py, marm-mcp-server/tests/test_hybrid_search.py
Temporal scoring uses one UTC reference timestamp per result set. Tests cover deterministic semantic and fallback scoring and preserve exact-lane behavior.
Raw cosine duplicate detection
marm-mcp-server/marm_mcp_server/core/consolidation.py, marm-mcp-server/marm_mcp_server/core/memory.py, marm-mcp-server/marm_mcp_server/core/memory_recall.py, marm-mcp-server/tests/*
Semantic consolidation requests raw cosine scores, evaluates five candidates, and compares the highest cosine score with the threshold. Tests cover missing scores, syntax-heavy queries, and lower-ranked candidates.

v2.33.1 release metadata and documentation

Layer / File(s) Summary
Release version alignment
marm-mcp-server/{Dockerfile,docker-compose.yml,pyproject.toml,server.json}, marm-mcp-server/marm_mcp_server/*, docs/INSTALL-*, README.md
Package, server, container, manifest, and installation references are updated to v2.33.1.
Release notes and benchmark documentation
CHANGELOG.md, README.md, marm-mcp-server/README.md, marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md
Release notes, benchmark measurements, consolidation costs, and threshold semantics are updated for v2.33.1.

Console UI dependency automation

Layer / File(s) Summary
Console UI Dependabot configuration
.github/dependabot.yml
Weekly npm updates are enabled for /marm-console, with grouped development and production updates and a three-pull-request limit.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR preserves raw cosine scores, compares consolidation thresholds against cosine, inspects multiple candidates, handles missing cosine, and keeps consolidation opt-in as required by #113.
Out of Scope Changes check ✅ Passed The changes support the stated release, temporal-scoring fix, consolidation fix, regression coverage, documentation, version surfaces, and Dependabot objectives.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Line 11: Update the changelog entry to call this “semantic duplicate
detection” or “semantic merging,” and clarify that semantic merging is skipped
when no meaning-similarity is available while exact normalized-content
deduplication still handles identical content, including syntax-heavy content.

In `@marm-mcp-server/README.md`:
- Line 182: Update the benchmark section’s release labels to match the actual
benchmark run: use v2.33.1 for both the re-measurement note and the LoCoMo
“Current” row only if the data was rerun against v2.33.1; otherwise mark both
references as historical v2.33.0 results.

In `@README.md`:
- Line 236: Correct the benchmark run-count methodology statement in README.md
at lines 236-236 and
marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md at lines 210-210:
scope the one-run claim to sections 1–4 and explicitly state that LoCoMo results
use two identically configured runs, keeping both documentation copies
consistent.
- Line 180: Update the benchmark version labels in README.md at lines 180-180
and marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md at lines
154-154 to identify the results as a v2.33.0 baseline, removing any implication
that they are current v2.33.1 measurements; keep both documents consistent.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: cd72cf90-b564-4629-aee4-6c270c5b0bea

📥 Commits

Reviewing files that changed from the base of the PR and between 92f37da and a57d8ed.

📒 Files selected for processing (24)
  • .github/dependabot.yml
  • CHANGELOG.md
  • README.md
  • docs/INSTALL-DOCKER.md
  • docs/INSTALL-LINUX.md
  • docs/INSTALL-PLATFORMS.md
  • docs/INSTALL-WINDOWS.md
  • docs/TECHNICAL-OVERVIEW.md
  • marm-mcp-server/Dockerfile
  • marm-mcp-server/README.md
  • marm-mcp-server/docker-compose.yml
  • marm-mcp-server/marm_mcp_server/__init__.py
  • marm-mcp-server/marm_mcp_server/config/settings.py
  • marm-mcp-server/marm_mcp_server/core/consolidation.py
  • marm-mcp-server/marm_mcp_server/core/memory.py
  • marm-mcp-server/marm_mcp_server/core/memory_recall.py
  • marm-mcp-server/marm_mcp_server/core/memory_scoring.py
  • marm-mcp-server/marm_mcp_server/core/memory_utils.py
  • marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md
  • marm-mcp-server/marm_mcp_server/server.py
  • marm-mcp-server/pyproject.toml
  • marm-mcp-server/server.json
  • marm-mcp-server/tests/test_consolidation_write_time.py
  • marm-mcp-server/tests/test_hybrid_search.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (11)
**/*.{py,json,md,toml,yml,yaml,Dockerfile}

📄 CodeRabbit inference engine (AGENTS.md)

Keep changes surgical: modify only what the task requires, match existing style, and preserve behavior during refactors.

Files:

  • marm-mcp-server/pyproject.toml
  • marm-mcp-server/marm_mcp_server/server.py
  • marm-mcp-server/docker-compose.yml
  • docs/TECHNICAL-OVERVIEW.md
  • marm-mcp-server/server.json
  • docs/INSTALL-LINUX.md
  • docs/INSTALL-WINDOWS.md
  • marm-mcp-server/marm_mcp_server/config/settings.py
  • docs/INSTALL-PLATFORMS.md
  • marm-mcp-server/marm_mcp_server/core/memory_utils.py
  • docs/INSTALL-DOCKER.md
  • marm-mcp-server/marm_mcp_server/core/memory.py
  • CHANGELOG.md
  • marm-mcp-server/marm_mcp_server/__init__.py
  • marm-mcp-server/tests/test_consolidation_write_time.py
  • marm-mcp-server/marm_mcp_server/core/consolidation.py
  • marm-mcp-server/marm_mcp_server/core/memory_scoring.py
  • README.md
  • marm-mcp-server/README.md
  • marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md
  • marm-mcp-server/marm_mcp_server/core/memory_recall.py
  • marm-mcp-server/tests/test_hybrid_search.py
**/*

📄 CodeRabbit inference engine (AGENTS.md)

**/*: Never commit changes without an explicit user request.
Use semantic versioning: MAJOR for breaking changes, MINOR for new tools, parameters, or features, and PATCH for fixes and documentation updates.

Files:

  • marm-mcp-server/pyproject.toml
  • marm-mcp-server/Dockerfile
  • marm-mcp-server/marm_mcp_server/server.py
  • marm-mcp-server/docker-compose.yml
  • docs/TECHNICAL-OVERVIEW.md
  • marm-mcp-server/server.json
  • docs/INSTALL-LINUX.md
  • docs/INSTALL-WINDOWS.md
  • marm-mcp-server/marm_mcp_server/config/settings.py
  • docs/INSTALL-PLATFORMS.md
  • marm-mcp-server/marm_mcp_server/core/memory_utils.py
  • docs/INSTALL-DOCKER.md
  • marm-mcp-server/marm_mcp_server/core/memory.py
  • CHANGELOG.md
  • marm-mcp-server/marm_mcp_server/__init__.py
  • marm-mcp-server/tests/test_consolidation_write_time.py
  • marm-mcp-server/marm_mcp_server/core/consolidation.py
  • marm-mcp-server/marm_mcp_server/core/memory_scoring.py
  • README.md
  • marm-mcp-server/README.md
  • marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md
  • marm-mcp-server/marm_mcp_server/core/memory_recall.py
  • marm-mcp-server/tests/test_hybrid_search.py
marm-mcp-server/{pyproject.toml,server.json,marm_mcp_server/__init__.py,marm_mcp_server/config/settings.py,marm_mcp_server/server.py,Dockerfile,docker-compose.yml}

📄 CodeRabbit inference engine (AGENTS.md)

When bumping the version, update every listed package, manifest, server, Docker, and configuration version occurrence, and audit with python scripts/find-versions.py.

Files:

  • marm-mcp-server/pyproject.toml
  • marm-mcp-server/Dockerfile
  • marm-mcp-server/marm_mcp_server/server.py
  • marm-mcp-server/docker-compose.yml
  • marm-mcp-server/server.json
  • marm-mcp-server/marm_mcp_server/config/settings.py
  • marm-mcp-server/marm_mcp_server/__init__.py
marm-mcp-server/marm_mcp_server/server.py

📄 CodeRabbit inference engine (AGENTS.md)

Keep HTTP MCP tools listed in MCP_TOOL_OPERATIONS; tools absent from this whitelist must not be exposed over HTTP.

Files:

  • marm-mcp-server/marm_mcp_server/server.py
marm-mcp-server/marm_mcp_server/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

marm-mcp-server/marm_mcp_server/**/*.py: All memory writes must use the serialized asynchronous write queue; do not add bypass paths.
A semantic-store failure during marm_log_entry must never fail the corresponding log write.
Keep the memory database and isolated concept-graph database on separate connection pools; never share connections between them.
Graph and concept failures must never break the seven core memory tools.
marm_smart_recall must keep primary memory ranking authoritative; graph enrichment is bounded, read-only, fail-open, and must be trimmed before primary results when enforcing response limits.
Use one lazy-loaded jinaai/jina-embeddings-v2-small-en encoder with 512 dimensions, serialized behind a lock; writes must succeed if the encoder is unavailable.
Keep orchestration in its current owner file and extract modules only at real boundaries consistent with the existing endpoint split.
Comments must be minimal, explain only non-obvious reasons, and never narrate the next line.

Files:

  • marm-mcp-server/marm_mcp_server/server.py
  • marm-mcp-server/marm_mcp_server/config/settings.py
  • marm-mcp-server/marm_mcp_server/core/memory_utils.py
  • marm-mcp-server/marm_mcp_server/core/memory.py
  • marm-mcp-server/marm_mcp_server/__init__.py
  • marm-mcp-server/marm_mcp_server/core/consolidation.py
  • marm-mcp-server/marm_mcp_server/core/memory_scoring.py
  • marm-mcp-server/marm_mcp_server/core/memory_recall.py
**/*.{md,py}

📄 CodeRabbit inference engine (AGENTS.md)

When adding or removing an MCP tool, update every full tool list and tool count in the README, protocol documentation, FAQ, and maintained marm-docs copies; add tests covering both transports.

Files:

  • marm-mcp-server/marm_mcp_server/server.py
  • docs/TECHNICAL-OVERVIEW.md
  • docs/INSTALL-LINUX.md
  • docs/INSTALL-WINDOWS.md
  • marm-mcp-server/marm_mcp_server/config/settings.py
  • docs/INSTALL-PLATFORMS.md
  • marm-mcp-server/marm_mcp_server/core/memory_utils.py
  • docs/INSTALL-DOCKER.md
  • marm-mcp-server/marm_mcp_server/core/memory.py
  • CHANGELOG.md
  • marm-mcp-server/marm_mcp_server/__init__.py
  • marm-mcp-server/tests/test_consolidation_write_time.py
  • marm-mcp-server/marm_mcp_server/core/consolidation.py
  • marm-mcp-server/marm_mcp_server/core/memory_scoring.py
  • README.md
  • marm-mcp-server/README.md
  • marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md
  • marm-mcp-server/marm_mcp_server/core/memory_recall.py
  • marm-mcp-server/tests/test_hybrid_search.py
**/*.py

⚙️ CodeRabbit configuration file

**/*.py: Prioritize runtime correctness, async/concurrency safety, SQLite transaction safety, auth/rate-limit behavior, release-breaking packaging issues, and MCP protocol compatibility.

Files:

  • marm-mcp-server/marm_mcp_server/server.py
  • marm-mcp-server/marm_mcp_server/config/settings.py
  • marm-mcp-server/marm_mcp_server/core/memory_utils.py
  • marm-mcp-server/marm_mcp_server/core/memory.py
  • marm-mcp-server/marm_mcp_server/__init__.py
  • marm-mcp-server/tests/test_consolidation_write_time.py
  • marm-mcp-server/marm_mcp_server/core/consolidation.py
  • marm-mcp-server/marm_mcp_server/core/memory_scoring.py
  • marm-mcp-server/marm_mcp_server/core/memory_recall.py
  • marm-mcp-server/tests/test_hybrid_search.py
**/*.md

⚙️ CodeRabbit configuration file

**/*.md: Only flag documentation issues that are materially wrong, misleading for installation/release behavior, or inconsistent with live MCP behavior. Skip style, phrasing, formatting, and wording preferences.

Files:

  • docs/TECHNICAL-OVERVIEW.md
  • docs/INSTALL-LINUX.md
  • docs/INSTALL-WINDOWS.md
  • docs/INSTALL-PLATFORMS.md
  • docs/INSTALL-DOCKER.md
  • CHANGELOG.md
  • README.md
  • marm-mcp-server/README.md
  • marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md
marm-mcp-server/{server.py,server_stdio.py,server.json}

📄 CodeRabbit inference engine (AGENTS.md)

When adding or removing an MCP tool, update the endpoint implementation, HTTP route and whitelist, STDIO registration/wrapper, and server.json tools array.

Files:

  • marm-mcp-server/server.json
marm-mcp-server/tests/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

marm-mcp-server/tests/**/*.py: Run tests with pytest from marm-mcp-server/; tests should exercise real FastAPI endpoints and real SQLite, using mocks only when they preserve at least 95% fidelity and provide meaningful speedup.
Every new MARM Console API route must have at least one happy-path FastAPI response-contract test with the MCP adapter stubbed.
Do not write existence-check or coded-to-pass tests; prefer deep tests exercising real paths over broad shallow coverage.
Use pytest.mark.skip only for genuinely unavailable dependencies, never merely to avoid implementation effort.

Files:

  • marm-mcp-server/tests/test_consolidation_write_time.py
  • marm-mcp-server/tests/test_hybrid_search.py
**/tests/**

⚙️ CodeRabbit configuration file

**/tests/**: Focus on tests that are flaky, non-isolated, incorrectly asserting behavior, or missing coverage for a changed high-risk path. Skip minor naming, comments, and layout preferences.

Files:

  • marm-mcp-server/tests/test_consolidation_write_time.py
  • marm-mcp-server/tests/test_hybrid_search.py
🔇 Additional comments (24)
.github/dependabot.yml (1)

14-27: LGTM!

marm-mcp-server/marm_mcp_server/core/memory_recall.py (1)

5-5: LGTM!

Also applies to: 146-168, 292-332, 378-385

marm-mcp-server/marm_mcp_server/core/memory_utils.py (1)

76-93: LGTM!

marm-mcp-server/marm_mcp_server/core/memory_scoring.py (1)

14-17: LGTM!

marm-mcp-server/tests/test_hybrid_search.py (1)

1366-1409: LGTM!

Also applies to: 1412-1528, 1534-1658, 1661-1741, 1745-1800

marm-mcp-server/marm_mcp_server/core/memory.py (1)

412-425: LGTM!

marm-mcp-server/marm_mcp_server/core/consolidation.py (1)

12-13: LGTM!

Also applies to: 60-89

marm-mcp-server/tests/test_consolidation_write_time.py (1)

132-156: LGTM!

Also applies to: 166-176

README.md (1)

8-8: LGTM!

Also applies to: 937-937, 1001-1001

docs/INSTALL-DOCKER.md (1)

5-5: LGTM!

docs/INSTALL-LINUX.md (1)

5-5: LGTM!

Also applies to: 323-323

docs/INSTALL-PLATFORMS.md (1)

1-1: LGTM!

docs/INSTALL-WINDOWS.md (1)

5-5: LGTM!

Also applies to: 297-297

docs/TECHNICAL-OVERVIEW.md (1)

3-3: LGTM!

marm-mcp-server/Dockerfile (1)

76-76: LGTM!

marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md (1)

1-1: LGTM!

Also applies to: 905-905, 969-969

marm-mcp-server/README.md (2)

238-240: 📐 Maintainability & Code Quality

Reconcile the benchmark provenance.

Line 238 says the accuracy range comes from two runs, but the benchmark introduction states that all numbers come from one run. Scope the single-run statement to the timed benchmarks or document the multi-run accuracy methodology explicitly.

Source: Path instructions


10-10: LGTM!

Also applies to: 939-939, 1003-1003

marm-mcp-server/docker-compose.yml (1)

8-8: LGTM!

Also applies to: 21-21

marm-mcp-server/marm_mcp_server/__init__.py (1)

17-20: LGTM!

marm-mcp-server/marm_mcp_server/config/settings.py (1)

182-182: LGTM!

marm-mcp-server/marm_mcp_server/server.py (1)

8-8: LGTM!

marm-mcp-server/pyproject.toml (1)

7-7: LGTM!

marm-mcp-server/server.json (1)

6-6: LGTM!

Also applies to: 20-25

Comment thread CHANGELOG.md
### Fixed: Duplicate Detection Compared the Wrong Score

- `CONSOLIDATION_THRESHOLD` (default `0.92`) is documented as the meaning-similarity a memory needs before it is merged into an existing one. The check was comparing it against the final ranking score instead, which also folds in keyword relevance and a recency boost. A memory could therefore be merged away when its actual meaning-similarity was below the threshold, because keyword overlap and freshness had pushed its ranking score above it.
- Duplicate detection now compares the raw meaning-similarity, as documented. It also asks for the meaning-based search lane explicitly: syntax-heavy content such as config keys, file paths and commands is normally routed to exact keyword lookup, which produces no meaning-similarity at all, so duplicate detection could not have worked for it either way. And it looks at a handful of candidates rather than only the top-ranked one, because the closest memory by meaning is not always the one that ranks first. If no meaning-similarity is available, nothing is merged.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Qualify “duplicate detection” as semantic duplicate detection.

Exact normalized-content deduplication still handles identical content, including syntax-heavy content. The current wording says duplicate detection cannot work when the semantic lane has no cosine value. State that semantic merging is skipped while exact deduplication remains available.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CHANGELOG.md` at line 11, Update the changelog entry to call this “semantic
duplicate detection” or “semantic merging,” and clarify that semantic merging is
skipped when no meaning-similarity is available while exact normalized-content
deduplication still handles identical content, including syntax-heavy content.

Source: Path instructions

Comment thread marm-mcp-server/README.md

These measurements use the fastembed-backed `jinaai/jina-embeddings-v2-small-en` encoder and a throwaway local SQLite database. Every timed path calls the shipped `MARMMemory` code, not a benchmark-local reimplementation. All numbers below come from a single run of [`scripts/benchmarking/performance/bench_hotpath.py`](scripts/benchmarking/performance/bench_hotpath.py) on local hardware; absolute milliseconds vary by machine, so treat the scaling shape as the signal.

Re-measured for v2.33.0. Latency rose and the hybrid speedup in section 4 fell substantially, both because `FTS_CANDIDATE_LIMIT` was raised from `50` to `200` in v2.32.0 to recover multi-hop accuracy. That was a deliberate trade of speed for correctness, and the older figures below were published before it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf 'README candidate:\n'
fd -a 'README.md' . | sed 's#^\./##'

printf '\nRelevant README excerpts:\n'
if [ -f marm-mcp-server/README.md ]; then
  nl -ba marm-mcp-server/README.md | sed -n '160,250p'
fi

printf '\nSearch for v2.33 references:\n'
rg -n 'v2\.33\.0|v2\.33\.1|Current \((.*?)v2\.33|Re-measured for v2\.33' marm-mcp-server/README.md || true

printf '\nDiff stat/name only:\n'
git diff --stat || true
git diff -- marm-mcp-server/README.md | sed -n '1,220p' || true

Repository: Lyellr88/marm-memory

Length of output: 540


🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE=marm-mcp-server/README.md

printf 'README excerpts:\n'
sed -n '160,250p' "$FILE" | cat -n -ba

printf '\nSearch for v2.33 references in marm-mcp-server/README.md:\n'
rg -n 'v2\.33\.0|v2\.33\.1|Current \((.*?)v2\.33|Re-measured for v2\.33' "$FILE" || true

printf '\nDiff status and changed README hunks:\n'
git diff -- marm-mcp-server/README.md | sed -n '1,260p'

Repository: Lyellr88/marm-memory

Length of output: 241


🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE=marm-mcp-server/README.md

printf 'README excerpts:\n'
sed -n '160,250p' "$FILE"

printf '\nSearch for v2.33 references in marm-mcp-server/README.md:\n'
rg -n 'v2\.33\.0|v2\.33\.1|Current \((.*?)v2\.33|Re-measured for v2\.33' "$FILE" || true

printf '\nDiff status and changed README hunks:\n'
git diff -- marm-mcp-server/README.md | sed -n '1,260p'

Repository: Lyellr88/marm-memory

Length of output: 10274


Resolve the v2.33 benchmark release label.

marm-mcp-server/README.md is for v2.33.1, but the benchmark section says Re-measured for v2.33.0 and labels the LoCoMo row Current (v2.33.0). If these numbers are from v2.33.0, label them as historical. Update both labels to v2.33.1 only after the benchmark run is from that release.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@marm-mcp-server/README.md` at line 182, Update the benchmark section’s
release labels to match the actual benchmark run: use v2.33.1 for both the
re-measurement note and the LoCoMo “Current” row only if the data was rerun
against v2.33.1; otherwise mark both references as historical v2.33.0 results.

Comment thread README.md

These measurements use the fastembed-backed `jinaai/jina-embeddings-v2-small-en` encoder and a throwaway local SQLite database. Every timed path calls the shipped `MARMMemory` code, not a benchmark-local reimplementation. All numbers below come from a single run of [`scripts/benchmarking/performance/bench_hotpath.py`](scripts/benchmarking/performance/bench_hotpath.py) on local hardware; absolute milliseconds vary by machine, so treat the scaling shape as the signal.

Re-measured for v2.33.0. Latency rose and the hybrid speedup in section 4 fell substantially, both because `FTS_CANDIDATE_LIMIT` was raised from `50` to `200` in v2.32.0 to recover multi-hop accuracy. That was a deliberate trade of speed for correctness, and the older figures below were published before it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Align benchmark version labels with the released code.

Both documents are labeled v2.33.1 but identify the benchmark results as current v2.33.0 measurements. Rerun the benchmark on v2.33.1, or label the results as a v2.33.0 baseline.

  • README.md#L180-L180: Update the benchmark version label and remove the current-release ambiguity.
  • marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md#L154-L154: Apply the same version-label correction to the packaged documentation.

As per path instructions: this is a materially misleading documentation claim.

📍 Affects 2 files
  • README.md#L180-L180 (this comment)
  • marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md#L154-L154
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@README.md` at line 180, Update the benchmark version labels in README.md at
lines 180-180 and marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md
at lines 154-154 to identify the results as a v2.33.0 baseline, removing any
implication that they are current v2.33.1 measurements; keep both documents
consistent.

Source: Path instructions

Comment thread README.md
| Jina v2 Small (v2.29.0) | 53.0% | 43.4% | 47.6% |
| **Current (v2.33.0)** | **62.9 - 63.5%** | **53.1 - 53.5%** | **57.4 - 57.9%** |

The v2.33.0 row is a range, not a point, because it is two identically-configured runs. Repeat runs of this harness disagree by up to **0.56pp** (19 of 1,977 questions), so differences smaller than that are not meaningful. The cause is understood: the corpus is written minutes before the benchmark, so `TEMPORAL_WEIGHT` reorders near-ties as the store ages between runs. Quote the range rather than the better run.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Align the benchmark run-count methodology.

The benchmark introduction claims one run, but the LoCoMo results use two runs. Scope the one-run statement to sections 1–4 and state the two-run method for LoCoMo.

  • README.md#L236-L236: Correct the run-count description in the main documentation.
  • marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md#L210-L210: Apply the same correction to the packaged documentation.

As per path instructions: this is a materially incorrect methodology statement.

📍 Affects 2 files
  • README.md#L236-L236 (this comment)
  • marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md#L210-L210
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@README.md` at line 236, Correct the benchmark run-count methodology statement
in README.md at lines 236-236 and
marm-mcp-server/marm_mcp_server/resources/marm-docs/README.md at lines 210-210:
scope the one-run claim to sections 1–4 and explicitly state that LoCoMo results
use two identically configured runs, keeping both documentation copies
consistent.

Source: Path instructions

@Lyellr88
Lyellr88 merged commit cb1c09e into MARM-main Jul 31, 2026
8 checks passed
Lyellr88 added a commit that referenced this pull request Jul 31, 2026
Addresses the three CodeRabbit findings on PR #120, all documentation:

- Scope the consolidation entry to semantic merging. Layer 1 exact
  content-hash deduplication is untouched by the cosine fix and still
  merges identical content, including syntax-heavy content. The previous
  wording implied dedup stops entirely when no cosine is available.
- Label the benchmark section v2.33.1. Both the LoCoMo accuracy run and
  the hot-path timings were measured on code carrying the temporal
  snapshot and consolidation fixes, so v2.33.0 was the wrong label rather
  than the numbers being stale.
- Scope the run-count claim. Sections 1-4 are one bench_hotpath.py run;
  section 5 is a separate accuracy benchmark reporting two runs.

Applied to the root docs and both packaged copies so all three agree.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Lyellr88
Lyellr88 deleted the release/v2.33.1 branch July 31, 2026 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hybrid recall similarity is RRF-scaled, breaking cosine-based consolidation threshold

1 participant