fix(tools): stop unusable browser tools from crowding out browser_nav… - #5762
Open
MummIndia wants to merge 1 commit into
Open
fix(tools): stop unusable browser tools from crowding out browser_nav…#5762MummIndia wants to merge 1 commit into
MummIndia wants to merge 1 commit into
Conversation
…igate Tool retrieval returns a top-K (8 by default), and @playwright/mcp alone exposes 30 tools. Measured across three typical browsing requests on a local setup, the agent was handed browser_drop, browser_handle_dialog, browser_close, browser_console_messages and the whole xy-mouse family -- while browser_navigate was missing in two cases out of three. On "click the login button", six of the eight slots went to mouse primitives. Without the tool that opens a page, every other browser tool is dead weight. So the model answered that it could not reach the internet, and that answer was correct. It reads as a refusal or a hallucination, which is what makes it expensive to diagnose: the browser is installed, the MCP server is connected and reports 30 tools, and calling browser_navigate by hand works fine. Two guards: MCP_INDEX_DENIED keeps 18 tools out of the index -- pointer primitives (browser_click and browser_hover work off the accessibility snapshot and need no coordinates), debugging aids, session plumbing, and browser_run_code_unsafe, which runs arbitrary JavaScript in the page. They stay connected and callable; they just no longer compete for a retrieval slot. MCP_COMPANIONS pulls browser_navigate and browser_snapshot in whenever any browser tool is retrieved. The server prefix comes from the hit itself rather than a constant, so the rule holds under any server id -- verified with a second prefix. Matching is on the bare tool name, after the server prefix, so neither guard depends on the browser being registered as builtin_browser. Measured on the same three queries after the change: browser_navigate present three times out of three, no denied tool leaking through, and every slot filled with something that works -- navigate, snapshot, find, click, type, press_key, select_option. The running app indexes 12 MCP tools instead of 30, and the agent then completed a real navigate-then-read task it had previously refused. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tool retrieval returns a top-K (8 by default), and
@playwright/mcpalone exposes30 tools. Measured on three typical browsing requests, the agent was handed
browser_drop,browser_handle_dialog,browser_close,browser_console_messagesand the whole xy-mouse family — while
browser_navigate, the only tool that opens apage, was missing in two cases out of three. On "click the login button", six of the
eight slots went to mouse primitives.
Without the tool that opens a page, every other browser tool is dead weight, so the
model answers that it cannot reach the internet. That answer is correct, but it reads
as a refusal or a hallucination, which is what makes it expensive to diagnose: the
browser is installed, the MCP server reports 30 tools, and calling
browser_navigateby hand works fine.
This adds two guards in
src/tool_index.py.MCP_INDEX_DENIEDkeeps 18 unusabletools out of the retrieval index — pointer primitives (
browser_clickandbrowser_hoverwork off the accessibility snapshot and need no coordinates),debugging aids, session plumbing, and
browser_run_code_unsafe, which runs arbitraryJavaScript in the page. They stay connected and callable; they just no longer compete
for a slot.
MCP_COMPANIONSpullsbrowser_navigateandbrowser_snapshotinwhenever any browser tool is retrieved. Matching is on the bare tool name, after the
server prefix, so neither guard depends on the browser being registered as
builtin_browser— verified with a second prefix.Target branch
dev, notmain.Linked Issue
Fixes #5763
Type of Change
Checklist
devdocker compose up) and verified the change works end-to-end.How to Test
show
MCP server connected: Built-in: Browser (builtin_browser) - 30 tools via stdio.Open https://example.com and summarise the page.[tool-rag] Retrieved tools for query:log line.mcp__builtin_browser__browser_navigateis absent, while entries such asbrowser_drop,browser_closeorbrowser_console_messagesoccupy the slots.The agent then reports that it cannot access the internet.
browser_navigateandbrowser_snapshot, and startup reportsIndexed 12 MCP toolsinstead of 30.Tool executed: mcp__builtin_browser__browser_navigate -> exit_code=0, thenfollows up with
browser_snapshotto read the page.Measured on three requests after the change:
browser_navigatepresent 3/3, nodenied tool leaking through, and every slot filled with something usable —
navigate,snapshot,find,click,type,press_key,select_option.Visual / UI changes
None. This PR only touches
src/tool_index.py, which has no rendering path — noHTML, CSS, SVG or
static/js/module is modified.