Observation
From conversation conv_30f049cdb75d464f on ws_mat: an agent needed to find the todo-board tools and went through three searches before hitting any:
| Query |
Result |
"todo task create" |
0 results |
"todo board" |
0 results |
"synapse-todo" |
26 results |
Every attempt after the first was a wasted tool-call round-trip. The app name is synapse-todo-board; the tools are create_board_task, batch_archive, etc. Natural-language terms ("todo", "task", "create") appear in tool names and descriptions but nb__search didn't surface them.
Behavior
nb__search appears to match on exact token or simple substring against tool names only, not against tool descriptions. Hyphenated app slugs (synapse-todo-board) aren't tokenized on -, so "todo" doesn't match.
Why this matters
- Visible cost: each failed search is a tool-call round-trip — LLM output tokens + latency + context growth. Three searches ≈ 300+ output tokens on the example above.
- Invisible cost: agents that can't find a tool in 2-3 tries typically give up or fall back to a worse approach. Users see "the agent doesn't know how to do X" when really the agent didn't know which tool to call.
Suggested
- Tokenize on
- and _: synapse-todo-board matches on todo, board, synapse.
- Match against descriptions, not just names: the tool description for
create_board_task starts with "Create a task on a specific board" — the query "create task" should hit it.
- Rank by query-term coverage: a match on all query terms beats a match on one.
- Fuzzy / edit-distance fallback (optional): handle
"todo boards" → matches "todo-board".
A simple BM25 or tf-idf over name + description per tool would cover (2) and (3) with ~20 LOC and no new deps.
Severity
Medium. Every agent-driven workflow involving tool discovery pays this cost. Affects UX more than correctness.
Source
conv_30f049cdb75d464f, ws_mat workspace, lines 9-22 in the JSONL log.
Observation
From conversation
conv_30f049cdb75d464fon ws_mat: an agent needed to find the todo-board tools and went through three searches before hitting any:"todo task create""todo board""synapse-todo"Every attempt after the first was a wasted tool-call round-trip. The app name is
synapse-todo-board; the tools arecreate_board_task,batch_archive, etc. Natural-language terms ("todo", "task", "create") appear in tool names and descriptions butnb__searchdidn't surface them.Behavior
nb__searchappears to match on exact token or simple substring against tool names only, not against tool descriptions. Hyphenated app slugs (synapse-todo-board) aren't tokenized on-, so "todo" doesn't match.Why this matters
Suggested
-and_:synapse-todo-boardmatches ontodo,board,synapse.create_board_taskstarts with "Create a task on a specific board" — the query"create task"should hit it."todo boards"→ matches"todo-board".A simple BM25 or tf-idf over
name + descriptionper tool would cover (2) and (3) with ~20 LOC and no new deps.Severity
Medium. Every agent-driven workflow involving tool discovery pays this cost. Affects UX more than correctness.
Source
conv_30f049cdb75d464f, ws_mat workspace, lines 9-22 in the JSONL log.