fix(ai): reword low-confidence disclaimer (escalated to engineering)

[NathanTarbert]

What

Rewords the AI-response disclaimer shown on non-HIGH-confidence answers. Old copy read:

⚠️ This is an AI-generated response with low confidence. A human agent has been notified and will follow up.

New copy:

• Low confidence (these auto-escalate):

  ⚠️ This is an AI-generated response and may be incomplete. We've escalated this to our engineering team — someone will follow up in this thread shortly.

• Medium confidence (does NOT escalate):

  This is an AI-generated response. A member of our team will review and follow up if needed.

• Formatter default fallback: "…A member of our team will review it shortly."

Why

• "A human agent has been notified and will follow up" felt off; "escalated to our engineering team" reads better and matches what actually happens (LOW confidence enqueues the ESCALATION job).
• Dropped the user-facing "with low confidence" — it read as scary/negative to the reporter.
• MEDIUM keeps a lighter line since it doesn't escalate.

Scope

Copy only (pipeline.ts disclaimer text + formatter.ts default fallback). Not the cause of low-confidence answers — that's the Pathfinder retrieval timeout, fixed separately in #112. Keeps "AI-generated response" phrasing. 29 formatter/pipeline tests pass; typecheck clean.

[jerelvelarde]

Review — copy change is mostly a clear win, but the LOW disclaimer now over-promises

The MEDIUM and fallback rewords are improvements, and splitting severity between LOW/MEDIUM is the right instinct. One substantive issue, plus a coverage gap.

🔴 The LOW disclaimer promises escalation that doesn't always happen

The new LOW copy makes a concrete, thread-level commitment:

"We've escalated this to our engineering team — someone will follow up in this thread shortly."

But the disclaimer branch and the escalation branch are gated on two different thresholds:

• Disclaimer text is chosen on the level scheme — finalConfidence === ConfidenceLevel.LOW, i.e. score < MEDIUM_THRESHOLD (0.5) (pipeline.ts:120, types.ts:20-23).
• The ESCALATION job is enqueued on the action scheme — confidenceScore < AI_CONFIDENCE.ESCALATE (0.4) (ai-response.ts:248).

Thresholds: MEDIUM_THRESHOLD = 0.5, ESCALATE = 0.4 (shared/src/constants.ts).

Failure scenario: a ticket scores 0.45. It's classified LOW, so the response posted back to the user says "We've escalated this to our engineering team — someone will follow up in this thread shortly." But 0.45 >= 0.4, so no ESCALATION job is created. The reporter is promised a follow-up that never comes.

The PR body justifies the change as "matches what actually happens (LOW confidence enqueues the ESCALATION job)." That premise isn't true for the [0.4, 0.5) band. The old copy had the same latent gap, but it was vaguer — this hardens a soft inaccuracy into a specific, falsifiable promise, which is worse for the reporter.

Options:

1. (preferred) Gate the "escalated" wording on the same condition escalation actually uses — drive the disclaimer selection from the escalation decision so the strong copy only appears when a job is truly enqueued.
2. Soften the LOW copy so it doesn't claim escalation already happened (closer to the MEDIUM line).

Test coverage

Existing tests only assert the addDisclaimer boolean (pipeline.test.ts:137-164) and toContain('AI-generated response') (formatter.test.ts:53-68). Nothing asserts the LOW-vs-MEDIUM branch text, so "29 tests pass" doesn't exercise the changed strings — the copy can regress silently. If you take option 1, add a test asserting the "escalated" wording appears iff an escalation job is enqueued (same threshold); that test would have caught the gap above.

Verdict

Low-risk copy change with one real product-correctness bug: the LOW disclaimer over-promises for scores in [0.4, 0.5). Worth aligning the disclaimer promise with the actual escalation gate (or softening it) plus a regression test before merge. Everything else is a genuine improvement.

🤖 Generated with Claude Code