Skip to content

Reduce AI request and token usage - #38

Merged
Luckycat133 merged 3 commits into
remakefrom
agent/reduce-model-requests
Aug 14, 2026
Merged

Reduce AI request and token usage#38
Luckycat133 merged 3 commits into
remakefrom
agent/reduce-model-requests

Conversation

@Luckycat133

Copy link
Copy Markdown
Owner

What changed

  • enforce one configured text-model request per normal persona, Agent, or group turn, with no automatic fallback or hidden retry
  • move titles, durable memory capture, Moments background content, quizzes, idiom games, palettes, deterministic math, and learning-state tools to local execution
  • bound context and per-Agent output budgets; keep Scholar at one Tavily retrieval plus one bounded synthesis
  • keep image and speech APIs behind explicit user actions, and cache repeat TTS playback
  • standardize the OpenRouter preset on nvidia/nemotron-3.5-lightning:free
  • update English/Chinese README, changelog, and the full review report with request budgets and observed provider behavior

Why

OpenRouter activity showed 276 calls spread across several models. In addition to bounded model-selection QA, the previous architecture had multiple hidden request producers: model-generated titles, LLM memory extraction, group fan-out, recall rewrites, recursive model-backed tools, proactive images, background Moment generation, per-question game generation, and automatic retries. That made request count, latency, and token use hard for users to control.

User impact

  • ordinary chat and group turns now have a predictable one-model-request ceiling
  • deterministic and background features use zero model requests
  • provider failures remain visible and require an explicit manual retry
  • cross-topic memory still works through local IndexedDB capture
  • explicit search, image generation, and TTS retain their functionality with clear request boundaries

Real UX validation

  • Aurora remembered the user's public-speaking plan, fear of forgetting, and green pen across topics; one model request per reply
  • Muse translation completed with one model request and no second translation-tool call
  • a two-character group selected one responder and sent one model request
  • natural-language inventory math returned 66 with zero dynamic requests
  • local science trivia and 10 seconds of Moments idle time generated zero dynamic requests
  • Scholar used exactly one Tavily request plus one Lightning request and displayed both current OpenRouter official source links
  • Pixel explicit image generation used one MiniMax request and zero OpenRouter requests
  • repeat Read Aloud playback used the cached audio and made zero additional requests

Checks

  • npm test — 31 files, 443 tests passed
  • npm run lint — passed
  • npm run build — passed
  • production-preview browser request tracing — passed for the flows above
  • staged diff secret/private-path/large-file checks — passed

@Luckycat133
Luckycat133 merged commit 517d557 into remake Aug 14, 2026
8 checks passed
@Luckycat133
Luckycat133 deleted the agent/reduce-model-requests branch August 14, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant