Groundedness identifier signal exempts every name that isn't literally CopilotKit* (useCopilotAction, CopilotChat, …)

[NathanTarbert]

The identifier signal is the ONLY thing that withholds a response after #143 (claim wording is penalty-only, deliberately — parsing English to make a publish/withhold decision failed three times in that PR). That makes the signal's coverage load-bearing, and it currently exempts most of the public API surface.

The gap

packages/outpost/ai/src/groundedness.ts filters candidate identifiers with a /copilotkit/i guard. Anything that does not contain the literal string copilotkit is never considered — so a fabricated useCopilotAction, CopilotChat, CopilotSidebar, CopilotTextarea, or useCopilotReadable is invisible to the gate, no matter how many are invented or whether any source mentions them.

Verified against the merged code:

extractCopilotKitIdentifiers('Use `.copilotKitGhostA` and `.copilotKitGhostB`')
  → ['copilotKitGhostA', 'copilotKitGhostB']   → suppress: true

extractCopilotKitIdentifiers('Call `useCopilotFabricated()` and `<CopilotInvented />`')
  → []                                          → suppress: false

The second line is the more likely fabrication in practice: the docs are full of useCopilotAction / CopilotChat, so a model inventing a neighbour of a real hook is a much more natural failure than inventing a .copilotKit* CSS class.

Two narrower related gaps

• TLD allowlist. BARE_HOST_PATTERN strips scheme-less hosts using a 9-TLD list (ai|app|co|com|dev|io|net|org|sh). A citation on another TLD (.cloud, .help) still leaks a phantom copilotkit identifier; conversely a genuine identifier that happens to end in one of those TLDs gets blanked as a host. Both directions verified.
• Extraction shape coverage. Nested parens and some JSX/prose forms are not matched, so an invented name in those shapes is not extracted.

Why this is filed rather than fixed in #143

Widening the guard changes what gets withheld, which needs its own review round and its own corpus rows (there is now a table-driven corpus in groundedness.test.ts keyed by response shape — the natural home). Doing it inside #143 would have meant a fifth review round on a PR that had already converged on the design.

Direction, when picked up: match on a CopilotKit-ish shape (Copilot* / useCopilot* / .copilotKit*) rather than the literal substring, and pin the widened surface with corpus rows in both directions — invented names suppress, documented names do not.

Found by the CR loop on #143 (round 3), reproduced against the merged code.

[NathanTarbert]

Fixed in 5bff9db on fix/ai-groundedness-gate (#143) — leaving this open until that PR merges.

The guard is now a shape test rather than a substring test: copilotkit anywhere (the original rule), or Copilot/useCopilot followed by more of a name. Verified end to end — useCopilotFabricated() + <CopilotInvented /> now suppresses where it previously published untouched, while CopilotChat stays grounded when a retrieved source mentions it.

Two things worth recording here:

Bare Copilot is deliberately excluded. The prefix test requires a character after Copilot, because "GitHub Copilot" and "Microsoft Copilot" appear in perfectly good answers and reading either as an invented API name would withhold a correct reply. Two tests pin it.

The check had to move after token unwrapping. It ran on the raw token, which a substring test tolerated (<CopilotKitFoo /> still contains copilotkit) but a prefix test does not — the leading < fails it, so <CopilotGhostPanel /> came back empty on the first attempt. The ordering is now load-bearing and commented as such.

What this costs, documented in the module: a real name counts as grounded only if retrieval actually returned a page mentioning it, so covering CopilotChat means a correct answer can be withheld when retrieval is thin and a second such name appears. Bounded by identifiers only counting in code shape, the haystack including each source's sourceUrl, and the suppression bar being 2. If correct answers do start getting withheld, the lever is retrieval breadth — narrowing this predicate back trades a visible false withhold for an invisible published fabrication.

[linear-code]

CPK-7928