Summary
The website documents the product. The plugin ships its own SKILL.md / AGENTS.md / --help, and that is what an agent loads into context. In the concrete cases below the two disagree, and the agent has the version that is missing or wrong.
What is on the website and does not reach the agent
test run … uiautomator — with a worked example. The bundled docs do not carry it, and we concluded twice from them that instrumentation had no path on this platform at all. It is arguably the most valuable capability in the product.device forward — address format, a --mode mux|demux option we never knew existed, and "Transient network connection failures are retried for up to 5 minutes", which would explain a 4-minute hang we spent a session diagnosing — we have no capture tying that hang to a retry, but the sentence never reached us, because it is absent from the bundled docs the agent was reading.file push — full syntax.
Where they actively disagree
Two concrete cases we hit:
In both cases the plugin's version is the one in an agent's context, and the agent has no signal that a second, differing source exists — so it plans confidently from the wrong text and the error surfaces as a failed metered session rather than as a doc bug.
A related discoverability trap: one name, two features
The name "test run" resolves to two unrelated features that do not cross-reference each other: the docs use it for createTestRun (re-executing a recorded test case), and the CLI uses test run for instrumentation. A reader who reaches the recorded-case page has no signal the instrumentation feature exists — it reads coherently enough to look like the answer. This is part of why we twice concluded instrumentation did not exist on this platform: the name we searched for resolved to something else. A one-line cross-reference on each surface, naming what it is not and pointing at the other, is enough — a rename is more disruptive than it is worth.
Ask
Generate the plugin's bundled docs from the same source as the website, or have --help link to the canonical page per command. Any process that permits drift will keep producing this class of bug, and the reader least able to notice it is the one you are optimising for.
Summary
The website documents the product. The plugin ships its own
SKILL.md/AGENTS.md/--help, and that is what an agent loads into context. In the concrete cases below the two disagree, and the agent has the version that is missing or wrong.What is on the website and does not reach the agent
test run … uiautomator— with a worked example. The bundled docs do not carry it, and we concluded twice from them that instrumentation had no path on this platform at all. It is arguably the most valuable capability in the product.device forward— address format, a--mode mux|demuxoption we never knew existed, and "Transient network connection failures are retried for up to 5 minutes", which would explain a 4-minute hang we spent a session diagnosing — we have no capture tying that hang to a retry, but the sentence never reached us, because it is absent from the bundled docs the agent was reading.file push— full syntax.Where they actively disagree
Two concrete cases we hit:
test runhelp and its own worked example on the same page disagree about the framework argument — the contradiction reported in issue [Bug]test run's framework argument contradicts its own worked example on the same page #106.manual-testing/adb-commandspage, and neither matches what a CLI/MCP session actually allows (the divergence reported in issue [Bug] The argument allowlist rejects ordinary arguments and returns exit 0, so scripts read a rejection as success #107): commands the page forbids succeed, and permitted commands are blocked by an undocumented per-argument allowlist.In both cases the plugin's version is the one in an agent's context, and the agent has no signal that a second, differing source exists — so it plans confidently from the wrong text and the error surfaces as a failed metered session rather than as a doc bug.
A related discoverability trap: one name, two features
The name "test run" resolves to two unrelated features that do not cross-reference each other: the docs use it for
createTestRun(re-executing a recorded test case), and the CLI usestest runfor instrumentation. A reader who reaches the recorded-case page has no signal the instrumentation feature exists — it reads coherently enough to look like the answer. This is part of why we twice concluded instrumentation did not exist on this platform: the name we searched for resolved to something else. A one-line cross-reference on each surface, naming what it is not and pointing at the other, is enough — a rename is more disruptive than it is worth.Ask
Generate the plugin's bundled docs from the same source as the website, or have
--helplink to the canonical page per command. Any process that permits drift will keep producing this class of bug, and the reader least able to notice it is the one you are optimising for.