fix(tools): don't leak unparsed tool-call JSON to the user (closes #378) - #380
Merged
Conversation
When the local model emits a tool-call-shaped output that is not valid JSON
(e.g. {"seconds": 60*60*12}, observed on Jetson), try_tool_call_with_context
returns None and the chat paths' else branch rendered the raw model output to
the user verbatim. Add is_unparsed_tool_call() + a fallback message and guard
the three API chat render sites (/api/chat/stream, /api/chat, and the
OpenAI-compatible bridge) so a botched tool call falls back to a graceful reply
instead of leaking raw tool-call JSON.
Voice and REPL stream/speak incrementally; guarding those is noted as follow-up.
The desktop session (display manager + X/Wayland + shell) holds a few hundred MB of the Orin Nano's 8 GB unified memory — the same pool the local LLM/KV-cache competes for. This adds a reversible, idempotent helper to drop to a console (multi-user) target and free that memory now and across reboots, with an --enable flag to restore the GUI.
Contributor
Author
This was referenced Jun 3, 2026
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #378. Found during Jetson validation (#375): when the local model emits a
tool-call-shaped output that is not valid JSON (e.g.
{"tool":"set_timer","arguments":{"seconds":60*60*12,...}}—60*60*12is a JS expression),try_tool_call_with_contextreturnsNoneand the chat paths rendered the raw model output to the user verbatim, leaking tool-call JSON.Changes
tools/parser.rs— addis_unparsed_tool_call()(structurally looks like a tool call but no valid tool-call JSON can be extracted) +UNPARSED_TOOL_CALL_FALLBACK, with unit tests (incl. the exact Jetson leak string).tools/mod.rs— re-export both.server.rs— guard the three API chat render sites —/api/chat/stream,/api/chat, and the OpenAI-compatible bridge — so a botched tool call falls back to a graceful reply instead of the raw JSON. Default/normal replies are unchanged.Scope note
Voice and REPL stream + speak the response incrementally, so guarding them is a separate, more involved change (post-stream rewrite); noted as follow-up. This PR fixes the observed surface (the port-3000 chat API). It does not close the umbrella #375.
The model emitting invalid JSON / picking the wrong tool is tracked separately in #379 (routing accuracy); this PR only stops the leak.
Real Behavior Proof
Tested profile / hardware:
laptopWhat I ran
What I observed