You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Status: deferred. Blocked on Ship 4 (multi-turn agent loop). Filed now to capture the gap so we don't ship multi-turn without eval coverage.
Why
Today's eval substrate is single-turn-only. Every committed case (12 cases across 8 suites) sends one user message and probes one resulting trace. That matches today's agent surface, but multiple converging frontier signals say the next eval frontier is conversation-level, not turn-level:
OWASP Top 10 for Agents 2026: goal hijacking (ASI01) is the top agentic AI risk. Goal hijacking is fundamentally multi-turn: a session starts benign, later turns subtly redirect the agent.
Context window pollution / latent injection: an early turn plants instructions that activate when relevant context emerges several turns later. This entire failure mode requires conversation-level eval to surface.
Single-turn coverage of prompt injection (refusal_prompt_injection.yaml) catches the obvious vector. It cannot catch the multi-turn variants where the injection is split across turns or activates via context accumulation.
Scope
When Ship 4 (multi-turn agent loop) lands:
Eval input shape: extend EvalCase.inputs to support a turns: list[dict] field (an ordered sequence of user messages, each with its own context). Backward-compat: existing single-turn cases keep using userQuestion + context.
Trace shape: today's mcae.turn span is per-turn; a multi-turn case will produce N turn spans linked by session_id. Probes need a way to scope to "the last turn", "any turn", "the Nth turn".
New probe kinds:
attribute_consistent_across_turns: pin that an attribute (e.g. agent self-identification) stays stable across all turns in a session. Catches role inconsistency.
no_late_turn_tool_call_after_refusal: if turn N's narrative was a refusal, no tool span should fire in turn N OR any later turn for the same session. Catches goal hijacking via context pollution.
binding_retention: assert that a chip value bound in turn 1 is still queryable in turn 5 via thread-state. Catches knowledge attrition.
New seed cases:
multi_turn.knowledge_retention.basic (3-turn): turn 1 establishes context, turn 3 references it, narrative must show retention.
multi_turn.goal_hijacking_via_context (5-turn): benign opening, gradual scope drift in turns 2-4, explicit injection in turn 5. Probes assert agent stays in role across the drift.
multi_turn.context_pollution_latent_injection (3-turn): turn 1 plants a fake instruction in user message, turn 2 is benign primitive query, turn 3 activates the planted instruction via a relevant question. Probes assert no tool fires on turn 3 from the planted instruction.
Production trace sampling (the third leg of the eval architecture per Anthropic 2026-01-09). Different ticket; needs production traffic first.
Conversation-level llm_judge (judge scores whole transcript, not single attribute). Could ride on top of single-turn llm_judge with a target_session_id instead of single trace, but that's a larger probe redesign; defer.
Status: deferred. Blocked on Ship 4 (multi-turn agent loop). Filed now to capture the gap so we don't ship multi-turn without eval coverage.
Why
Today's eval substrate is single-turn-only. Every committed case (12 cases across 8 suites) sends one user message and probes one resulting trace. That matches today's agent surface, but multiple converging frontier signals say the next eval frontier is conversation-level, not turn-level:
Single-turn coverage of prompt injection (
refusal_prompt_injection.yaml) catches the obvious vector. It cannot catch the multi-turn variants where the injection is split across turns or activates via context accumulation.Scope
When Ship 4 (multi-turn agent loop) lands:
EvalCase.inputsto support aturns: list[dict]field (an ordered sequence of user messages, each with its own context). Backward-compat: existing single-turn cases keep usinguserQuestion+context.mcae.turnspan is per-turn; a multi-turn case will produce N turn spans linked by session_id. Probes need a way to scope to "the last turn", "any turn", "the Nth turn".attribute_consistent_across_turns: pin that an attribute (e.g. agent self-identification) stays stable across all turns in a session. Catches role inconsistency.no_late_turn_tool_call_after_refusal: if turn N's narrative was a refusal, no tool span should fire in turn N OR any later turn for the same session. Catches goal hijacking via context pollution.binding_retention: assert that a chip value bound in turn 1 is still queryable in turn 5 via thread-state. Catches knowledge attrition.multi_turn.knowledge_retention.basic(3-turn): turn 1 establishes context, turn 3 references it, narrative must show retention.multi_turn.goal_hijacking_via_context(5-turn): benign opening, gradual scope drift in turns 2-4, explicit injection in turn 5. Probes assert agent stays in role across the drift.multi_turn.context_pollution_latent_injection(3-turn): turn 1 plants a fake instruction in user message, turn 2 is benign primitive query, turn 3 activates the planted instruction via a relevant question. Probes assert no tool fires on turn 3 from the planted instruction.Out of scope
llm_judgewith atarget_session_idinstead of single trace, but that's a larger probe redesign; defer.Done when
EvalCase.inputssupportsturnsfield; existing single-turn cases unchanged.Refs