perf(memory): deterministic fast path for retrieve_memory — skip the model walk when data is present

[M3gA-Mind]

Problem (issue #4677)

retrieve_memory takes ~30–40s per call even when it finds data, dominating agent-turn latency (contrast goal_get 0.0s, read_workspace_state 0.07s, thread_message_list 0.007s). One turn spent retrieve_memory ×4 = 32.5 + 39.1 + 30.8 + 38.7 ≈ 141s.

Where the latency goes (verified in code)

retrieve_memory (the chat delegate) and call_memory_agent both spawn the agent_memory sub-agent via run_subagent (subagent_runner/ops/runner.rs). That agent runs a model-driven walk — up to max_iterations LLM round-trips (walk → drill_down → fetch_leaves → synthesize). On the eval tiers, those per-iteration round-trips are the ~30–40s; the retrieval work itself is not the bottleneck.

The deterministic retriever fast_retrieve (memory_tree/retrieval/fast.rs — E2GraphRAG: query-entity + dense/semantic recall + cosine rerank, no LLM in the loop) already returns the same tree hits in a single ~1–3s pass (one entity extraction + one query embed + DB reads). But it was only wired into the benchmark harness (agent_memory/ops.rs), never the production retrieve path — so every real retrieval paid the full model loop.

Fix (minimal, targeted, covers both entry paths)

In run_subagent (the shared seam for the retrieve_memory delegate and call_memory_agent), before the model walk, add a deterministic fast path gated on the agent being agent_memory:

• run fast_retrieve on the task prompt;
• when it finds data, return the hits directly (iterations = 0, Completed), skipping the model walk;
• fall through to the full sub-agent when the fast path is disabled, when config/retrieval errors, or when nothing is found — the empty/degraded case stays [memory] retrieve_memory sub-agent exhausts its tool-call budget → SUBAGENT_INCOMPLETE instead of failing fast on empty/degraded memory #4655's territory and still benefits from the model's judgement.

fast_retrieve reads the ambient source-scope, so memory scoping is preserved; these are the same recall calls the model's first tool step would have made. Env kill-switch OPENHUMAN_MEMORY_FAST_PATH=0 (false/no/off) forces the full walk for A/B without a rebuild.

Result: the data-present happy path drops from ~35s to ~1–3s (no LLM round-trips), while hard/empty queries still get the full walk.

This is distinct from #4655 (which only fixes the empty/degraded early-exit); this targets the happy path.

Repo note

The fix is in openhuman, not vendor/tinyagents: the harness sub-agent loop is generic and correct — the slowness came from openhuman routing memory retrieval through a model-driven agent when a deterministic retriever already existed. No tinyagents change needed.

Tests

Pure, deterministic unit tests in runner.rs (fast_path_tests): env opt-out parsing (default on; 0/false/no/off → off), the empty→None fallback, hit formatting (scope/content/score, numbering, singular/plural wording), and oversized-body truncation.

Checks

Per the active no-local-builds directive (disk pressure): cargo fmt only — no build/test locally; CI verifies.

Closes #4677

Summary by CodeRabbit

• New Features
  • Added an optional deterministic “memory fast path” for memory-related agent runs that can return results immediately when relevant matches are found.
  • Memory matches are now summarized in a compact “Retrieved … relevant memories” block with scope, per-hit relevance details, and safe truncation.
• Bug Fixes
  • Improved consistency and safety of result truncation with a clear truncation marker and multibyte-safe character limits.
• Tests
  • Added coverage for environment enablement, formatting rules, and truncation behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 426c2391-821c-499f-88c9-f41053d3312d

📥 Commits

Reviewing files that changed from the base of the PR and between 28d2c72 and 5e99319.

📒 Files selected for processing (1)

• src/openhuman/agent/harness/subagent_runner/ops/runner.rs

🚧 Files skipped from review as they are similar to previous changes (1)

• src/openhuman/agent/harness/subagent_runner/ops/runner.rs

---

📝 Walkthrough

Walkthrough

Adds an environment-gated deterministic retrieval fast path for the agent_memory sub-agent. Grounded, non-empty queries can return formatted retrieval hits directly; disabled, ungrounded, empty, failed, or empty retrievals fall back to the normal model-driven path. Result truncation is shared across both paths.

Changes

Memory fast path

Layer / File(s)	Summary
Fast path helpers and retrieval src/openhuman/agent/harness/subagent_runner/ops/runner.rs	Adds environment parsing, bounded retrieval, deterministic hit formatting, scope and relevance rendering, ellipsis handling, and character-safe max_result_chars truncation.
Runner integration and validation src/openhuman/agent/harness/subagent_runner/ops/runner.rs	Short-circuits agent_memory execution when deterministic retrieval returns an outcome, reuses shared truncation for typed results, and adds unit coverage for parsing, formatting, limits, and multibyte text.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant run_subagent
  participant try_deterministic_memory_retrieval
  participant fast_retrieve

  run_subagent->>try_deterministic_memory_retrieval: agent_memory delegation
  alt enabled, grounded, and non-empty query
    try_deterministic_memory_retrieval->>fast_retrieve: query with bounded options
    fast_retrieve-->>try_deterministic_memory_retrieval: hits or empty/error
    alt hits found
      try_deterministic_memory_retrieval-->>run_subagent: completed formatted outcome
    else no usable result
      try_deterministic_memory_retrieval-->>run_subagent: None
    end
  else fast path unavailable
    try_deterministic_memory_retrieval-->>run_subagent: None
  end

Loading

Possibly related PRs

• tinyhumansai/openhuman#4764: Both changes alter memory delegation and routing behavior around direct retrieval.
• tinyhumansai/openhuman#4778: Orchestration-triggered sub-agent runs may be affected by the updated run_subagent behavior.

Suggested labels: agent, enhancement

Poem

A rabbit found memories neat,
With bounded hops and text complete.
Fast hits appeared in tidy rows,
Safe truncation trimmed their prose.
“Fallback still waits below!” 🐇

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: a deterministic fast path for memory retrieval that skips model walking when data exists.
Linked Issues check	✅ Passed	The PR implements the requested low-latency data-present retrieve_memory fast path with fallback, grounding checks, and zero-iteration completed results.
Out of Scope Changes check	✅ Passed	The changes stay focused on memory retrieval latency, formatting, truncation, and tests, with no obvious unrelated scope creep.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

---

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b1a6469e61

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/openhuman/agent/harness/subagent_runner/ops/runner.rs`:
- Around line 186-204: The config load in the subagent runner currently swallows
failures via Config::load_or_init().await.ok()? with no observability. Update
the runner logic in the same path as fast_retrieve to handle load_or_init errors
explicitly, log them with tracing::warn! (including task_id and the error
details, similar to the fast_retrieve branch), and then fall back to None
instead of silently discarding the failure.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 14961c47-f3ec-421f-92a9-73c1c89328a0

📥 Commits

Reviewing files that changed from the base of the PR and between 54511e6 and b1a6469.

📒 Files selected for processing (1)

• src/openhuman/agent/harness/subagent_runner/ops/runner.rs

[YellowSnnowmann]

Verdict: approve. The PR satisfies #4677 — it short-circuits the agent_memory retrieval happy path to a single deterministic fast_retrieve pass, dropping the ~30–40s model walk to ~1–3s when data is present, while correctly falling through to the full agent when the fast path is disabled, errors, or finds nothing.

Verified:

• Gate on definition.id == AGENT_MEMORY_ID (runner.rs:335) matches the real id/delegate in agent_memory/agent/agent.toml, so it covers both retrieve_memory and call_memory_agent via the shared run_subagent seam.
• Memory scoping is preserved: fast_retrieve gates on current_source_scope() (fast.rs:250), a task-local set by with_source_scope around the whole turn (task_dispatcher/executor.rs:201), already active when the fast path runs — same allowlist the model walk's tools honor.
• Error handling is clean: config/retrieval failures are tracing::warn!-logged and degrade to the model walk; nothing swallowed, nothing routed to Sentry. Correct for graceful degradation.
• SubagentRunOutcome fields (iterations:0, usage:default, status:Completed) are consistent with the normal construction (runner.rs:1337) — no LLM ran, so zero usage is right.
• Tests: fixtures match RetrievalHit/QueryResponse exactly; env-parse, none-on-empty, singular/plural, truncation, and limit-invariant are covered. CI (Rust Core Coverage + Quality) green, so they compile and pass the diff-cover gate.

Non-blocking nit:

• The fast-path early return (runner.rs:336-342) bypasses the definition.max_result_chars truncation the normal path applies (runner.rs:404-422). No active bug — agent_memory sets no cap and the output is self-bounded (8 hits × 600 chars). Latent only: if a cap is later added to agent_memory it would be silently ignored. Worth a comment or reusing the cap to future-proof.

Both review threads are genuinely resolved: Codex's relevance-floor P2 is addressed by the entity-grounding guard (extract_query_entities gate, runner.rs:210-224 — defers ungrounded queries to the model), and CodeRabbit's silently-swallowed config-load is now tracing::warn!-logged before falling back (runner.rs:196-206). CodeRabbit is APPROVED.

[YellowSnnowmann]

Approving — feature satisfied, no blocker/major issues, review comments addressed in code, CI green. Detailed review in the PR comment above.

[M3gA-Mind]

@YellowSnnowmann good catch on the non-blocking nit — addressed in 28d2c72. The fast-path early return now routes through a shared apply_max_result_chars helper (extracted from run_subagent's existing truncation block), so both the deterministic fast path and the model-driven path honour a definition's max_result_chars cap. No behavior change today (agent_memory is uncapped and the fast-path block is self-bounded at 8×600 chars), but the two paths can no longer silently diverge if a cap is added to agent_memory later. Added unit tests for the helper (none/under-cap no-op, over-cap truncation with marker, multi-byte char-boundary safety).

[M3gA-Mind]

Rebased onto main, resolved conflicts in src/openhuman/agent/harness/subagent_runner/ops/runner.rs — dropped the now-removed TextModeProvider decorator (main's #4738 refactor replaced it with the in-band build_text_mode_tool_instructions() / with_text_mode() path) and re-applied the deterministic memory fast-path + its fast_path_tests module on top.

[oxoxDev]

Review ✅ — approve

Sound, low-risk. The fast path is gated strictly on definition.id == "agent_memory", placed after the spawn-depth + tier gates, so no other sub-agent routing through the generic run_subagent is affected. try_deterministic_memory_retrieval returns None (→ unchanged model walk) on any of {disabled, empty query, config err, ungrounded, retrieval err, zero hits}, so every degraded path falls back.

Verified

• Parity: agent_memory is sandbox_mode="read_only"; its tools record no access/recency/touch writes, so the walk has no side effects the fast path skips. The grounded branch hydrates leaf chunks (not summary-only), so a hit isn't categorically shallow. Zero-hit → fallback covers the empty-store false-negative.
• Determinism real: sort by coverage→latest_ts→id, no LLM sampling; spaCy-down degrades to regex grounding → more queries defer to the walk (safe).

Non-blocking

• [low] format_deterministic_memory_hits prints hit.score as (relevance X.XX), but in the grounded (resolve_local) branch that score is an entity-pair coverage count (2.0, 3.0…), not a 0–1 relevance — your own commit-2 note says it "is not a usable relevance floor." The parent model may misweight "(relevance 3.00)". Relabel per-branch (e.g. "entity matches N") or normalize.
• [low] Recall-depth reduction is silent: fast_retrieve is a bounded single pass (max_hops=2, no model-driven drill_down/re-query). A grounded but multi-hop query where the walk would drill deeper returns a shallower non-empty result that ships as Completed (fallback only fires on empty). Intended trade with the env kill-switch, but nothing measures parity — worth calling out; ideally a populated-store parity smoke.
• [low] extract_query_entities runs twice (guard + inside fast_retrieve) → 2× spaCy round-trips; thread the ids through. Test gap: nothing asserts fires-only-for-agent_memory / ungrounded-defers / empty-falls-through (unit-testable with a fake config).

CodeRabbit config-swallow, Codex no-match gating, and the max_result_chars divergence are all fixed in commits 2–3.