Skip to content

Reply editor prefills the AI disclaimer instead of the draft answer #145

Description

@NathanTarbert

The agent-facing reply editor prefills with the AI disclaimer instead of the draft answer.

Where

packages/outpost/queue/src/handlers/ai-response.ts:172

await prisma.ticket.update({
    where: { id: ticket.id },
    data: {
        suggestedResponse: pipelineResult.formatted.text,
    },
});

formatted.text is the published form of the reply: platform formatting, footer, and — for anything below HIGH confidence — a leading disclaimer block:

> ⚠️ This is an AI-generated response. We've escalated this to our engineering team — someone will follow up in this thread shortly.

apps/web/src/components/.../reply-editor.tsx:59 prefills the agent's reply box with that string verbatim. So a human picking up an escalation opens their editor and finds a draft that opens by declaring itself AI-generated and announcing an escalation — text that makes no sense coming from them, and that they have to delete before writing anything.

The raw draft is already persisted one call earlier as Message.content (ai-response.ts:158) and is available as pipelineResult.response. That's the right source for an editor.

Fix

-        suggestedResponse: pipelineResult.formatted.text,
+        suggestedResponse: pipelineResult.response,

Worth checking whether any other consumer depends on suggestedResponse carrying the formatted text before flipping it — as of #143's review, reply-editor.tsx is its only reader and it's human-driven, so nothing auto-posts it.

Why it's filed separately

Raised by @jerelvelarde in review of #143 and deliberately not fixed there: the line is pre-existing, isn't in that PR's diff, and #143 only makes it reachable more often.

It gets worse, though, with the always-answers work that's in progress (retrieval fan-out across all four MCP corpora + the no-answer reply). Under that change, a suppressed ungrounded draft has its formatted.text replaced with the no-answer copy:

I couldn't find an answer to this in the CopilotKit or AG-UI documentation or source code, so I don't want to guess. I've escalated this to our team — someone will follow up in this thread.

So the human's editor prefills with the apology, and the actual draft they were meant to work from lives only in Message.content, which the editor never reads. The stated justification for suppressing rather than discarding a draft — "a human can edit and send it" — stops being true.

Should land with, or before, that PR.

Acceptance

  • suggestedResponse holds the unformatted draft (pipelineResult.response), disclaimer-free
  • Reply editor prefills text an agent can send without deleting a preamble
  • A suppressed draft still reaches the editor as the draft, not as the no-answer copy
  • Test covering the suppressed case, since that's the path where the two diverge

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: aiAI response pipeline, groundedness, confidenceroadmapTracked on the Outpost roadmaproadmap: nowRoadmap horizon: V1 launch critical path / safety

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions