Web chat tool dispatch broken — compaction strips tool manifest + memory context, LLM hallucinates name instead of memory_recall

[ai-hpc]

Summary

The web chat UI on port 3000 is returning bare LLM completions with no tool calls executed, even though memory clearly has the data the user is asking about. The model answers from prior chat turns or hallucinates a value, instead of dispatching memory_recall / household-preference lookups.

This is the predictable downstream failure mode of the aggressive prompt compaction landed in PR #74 (called out in that PR's review as a follow-up risk).

Reproduction

Memory state (visible in the UI's memory / household-preference panel):

preference  household.preference  household  normal  recalls 10  5/18/2026, 2:46:59 AM
  Jared is interested in cooking and gardening.
  User's name is Jared
  instead of using the name Jared use the alias Tom

Chat transcript through :3000:

> hi
Hello! How can I assist you today?

> cool
Thanks for the cool input! How can I assist you today?

> did you remember my name?
Yes, I remember your name. How can I assist you today?

> what is my name?
Your name is GeniePod.

Expected:

• "what is my name?" → calls memory_recall (or whatever the household-preference lookup is named), reads back Tom (or Jared if the alias-mapping preference isn't being applied) — not GeniePod.
• "did you remember my name?" → likewise a tool-backed answer, not a content-free affirmation.

Why this is almost certainly the PR #74 compaction path

PR #74's compact_genie_runtime_messages (in crates/genie-core/src/llm/openai_compat.rs) replaces ALL prior messages — including the system prompt that lists the tool manifest and any injected memory context — with a single user-role turn:

{GENIE_RUNTIME_COMPACT_SYSTEM}\n\n{latest_user_content}

where GENIE_RUNTIME_COMPACT_SYSTEM is the hardcoded blurb "You are GeniePod Home. Answer the user's latest request directly and concisely.". That blurb says nothing about tools, has no memory context, and the model has no way to know memory_recall exists. So it just generates a chat completion.

Compaction triggers at body.size() > GENIE_RUNTIME_MAX_BODY_BYTES (4 KB). A real GenieClaw system prompt (tool definitions + household preferences + recent turns) easily exceeds 4 KB, so compaction fires on every web-chat request once the user has any non-trivial memory state.

Compounding factor — PR #76 widened the runtime context

PR #76 bumped GENIEPOD_AI_RUNTIME_CONTEXT from 2048 to 8192 with --int8-kv. The Jetson runtime can now comfortably hold ~16–24 KB of typical English chat text — so the 4 KB body-compaction trigger is now far more aggressive than necessary. The client is compacting prompts the runtime could trivially handle.

Acceptance criteria

• Asking "what is my name?" through the :3000 chat UI returns the value stored in memory (or in the household-preference store), not a hallucinated answer.
• Tool calls show up in journalctl -u genie-core for chat requests that should trigger them (memory_recall, household-preference lookups, etc.).
• Web-chat requests preserve the system prompt's tool manifest and at least the immediately-relevant memory context, so the LLM knows what tools exist and what facts it has.

Fix directions (pick one or combine)

Option A — Preserve the system prompt during compaction

Keep the original system message (tool manifest + memory injection) and only compact the history (older user/assistant turns). Latest user turn plus the system prompt; drop only the oldest turns until under budget. This matches what PR #74's body description claimed but isn't what the code actually does.

Option B — Raise GENIE_RUNTIME_MAX_BODY_BYTES to track context capacity

GENIE_RUNTIME_MAX_BODY_BYTES is currently a hardcoded 4 KB constant. With PR #76's 8192-token context, the runtime can comfortably handle ~16–24 KB. Either bump the constant or derive it from GENIEPOD_AI_RUNTIME_CONTEXT. Trade-off: simpler change, but compaction still wipes context the next time we widen prompts faster than we widen the runtime.

Option C — Pre-resolve memory tool calls before sending to the LLM

Genie-core's tool-dispatch layer already has memory recall plumbed. Run an explicit memory_recall before the LLM call when the prompt mentions identity / preferences keywords, inject the recalled values into a small prompt, then send that compact body. Sidesteps the compaction problem entirely for the most-common identity-query path.

I'd recommend A + B together: A is the structural fix (preserve system context); B unlocks today's bigger runtime context. C is a useful future direction but heavier.

Cross-references

• genie-core chat fails through genie-ai-runtime while health reports connected #73 — original issue that PR core: shape genie-ai-runtime chat requests #74 fixed; web chat going through at all
• core: shape genie-ai-runtime chat requests #74 — landed the compaction that's now too aggressive
• Evaluate raising genie-ai-runtime context default to 8192 #75 / setup: start genie-ai-runtime before memory-heavy services #76 — widened runtime context to 8192 tokens
• Related noted in core: shape genie-ai-runtime chat requests #74 squash body: "current compaction replaces ALL prior turns with the latest user message + system blurb. Web chat returns a working response but doesn't remember previous turns. Should evolve toward 'system blurb + recent N user/assistant pairs' once the runtime's context capacity is more reliable." — that follow-up is exactly this issue.