diff --git a/docs/research.md b/docs/research.md index 66867f5..5b675d9 100644 --- a/docs/research.md +++ b/docs/research.md @@ -266,6 +266,20 @@ Autonomy contracts are not set-and-forget. The agent should re-confirm autonomy Beyond per-product autonomy settings, Anthropic positions its own model behavior on a "disposition dial" ranging from _fully corrigible_ (does whatever instructed) to _fully autonomous_ (acts on its own values). Current Claude models sit closer to the corrigible end — a deliberate safety choice with direct UX implications: agents defer to the principal hierarchy (Anthropic → operator → user) rather than override it. The autonomy contract a product designs sits on top of this model-level disposition. +### 4.5 Levels are analysis vocabulary, not Send-time UI (July 2026 reframe) + +> **Position (ratified 2026-07-05).** The level ladder in §4.1 is how we _analyze_ autonomy. It is not what the user sees at Send. At the dispatch moment the user sees a **behavioral posture badge** plus **verification defensibility gaps** — and, for complex tasks, a **plan→execute gate** that picks execution posture after plan review. + +The July 2026 web audits (`audits/`, 2026-07-03 series) converged on three findings that reshape how §4 applies to composer UI: + +1. **No product ships L0–L5 as a numbered control at Send** ([`audits/dispatch-moment-ui-web-2026-07-03.md`](../audits/dispatch-moment-ui-web-2026-07-03.md)). Mode, model, and placement are visible; Run Mode, sandbox, allowlists, and stopping conditions stay hidden. Where level surfaces at all, it works as behavior language, not a number. +2. **The strongest pre-dispatch gap is defensibility** ([`audits/verification-defensibility-web-2026-07-03.md`](../audits/verification-defensibility-web-2026-07-03.md)). Nothing on the market answers "is this autonomy level defensible for _this_ task?" before Send. The eight-field run contract (Addy Osmani, July 2026) is one safeguard among many — not the organizing UI concept; its fields ship shredded across products ([`audits/contract-field-reality-2026-07-03.md`](../audits/contract-field-reality-2026-07-03.md)). +3. **Plan gates that also pick execution posture are the strongest shipping pattern** — Claude Code's plan-approval options, Factory's Spec approval. The plan review moment is where autonomy gets renegotiated, not a settings page. + +**Canonical Send-time vocabulary.** agentic-craft ships posture as three behavioral badges — **"Asks first"** (pauses before edits and external actions), **"Edits freely"** (applies file edits without per-step approval), **"On its own"** (runs until done or a gate fires) — roughly the behavior of L2/L3/L4 in §4.1's terms. Before Send, the composer surfaces five defensibility gaps: **stopping condition, verification method, scope boundary, spend cap, risky actions**. Source of truth: `src/views/patterns/dispatch-moment-data.ts`; reference sketch: `/patterns/dispatch-moment`. + +**How this composes with the rest of §4.** The matrix view (§4.2) and renegotiation (§4.3) still hold — the badge is the compressed, legible face of the effective policy, and the plan→execute gate is a renegotiation moment. What changes is the UI claim: the ladder informs design analysis and this document's taxonomy; the badge, the gap checklist, and the plan gate are what earn pixels at Send. + ## 5. Trust through provenance > **Position.** Trust in agentic systems is not earned through confident prose. It is earned through _provenance_ — visible links between every claim and its source. An agent that says less but cites everything beats an agent that says more without sources. diff --git a/sessions/2026-07-03-eve-chat-composer-autonomy.md b/sessions/2026-07-03-eve-chat-composer-autonomy.md index 51e79a5..670436e 100644 --- a/sessions/2026-07-03-eve-chat-composer-autonomy.md +++ b/sessions/2026-07-03-eve-chat-composer-autonomy.md @@ -36,15 +36,25 @@ The Claude Code exploration may have been **solving the wrong problem well**: po --- -## Open composer questions (pick one to pursue) +## Composer direction — decided 2026-07-05 -| ID | Question | Research signal | -|----|----------|-----------------| -| **A** | Level selection legibility at Send | No product ships L0–L5 as a numbered control; level works better as behavioral badge | -| **B** | Verification defensibility before dispatch | Strongest gap — nothing answers "is this autonomy level defensible for *this* task?" pre-Send | -| **C** | Plan→execute transition | Strongest pre-dispatch patterns: Claude Code 5-option plan approval, Factory Spec approval, Plan→Implement | +**Decision: pursue B + C.** Verification defensibility before dispatch plus +the plan→execute transition are the composer problems agentic-craft works on. +Contract read-back is no longer the main problem. -Research points strongest at **B + C**, with level as behavioral badge rather than ladder UI. +| ID | Question | Resolution | +|----|----------|------------| +| **A** | Level selection legibility at Send | **Resolved as design position** — level ships as a behavioral badge ("Asks first" / "Edits freely" / "On its own"), never a numbered L0–L5 control | +| **B** | Verification defensibility before dispatch | **Chosen** — surface missing proof, unclear stopping conditions, missing scope, cost/budget ambiguity, and risky actions before Send | +| **C** | Plan→execute transition | **Chosen** — complex tasks gate on plan review, then explicitly pick execution posture | + +Canonical vocabulary (three postures, five defensibility gaps) lives in +`src/views/patterns/dispatch-moment-data.ts`; the pattern page at +[`/patterns/dispatch-moment`](../app/(reference)/patterns/dispatch-moment/page.tsx) +is the reference sketch. `docs/research.md` §4.5 carries the reframe. + +**Bare/B/C/D stay retired** unless B+C work surfaces a concrete need for a +contract-variant comparison. --- @@ -130,9 +140,9 @@ Fixture ask: *"migrate session-chat-page to the shared adapter — don't touch d --- -## Suggested next steps in agentic-craft +## Next steps in agentic-craft -1. Choose composer problem **A**, **B**, or **C** (recommend B + C) — **started:** [`/patterns/dispatch-moment`](../app/(reference)/patterns/dispatch-moment/page.tsx) sketches B+C on the registry composer +1. ~~Choose composer problem **A**, **B**, or **C**~~ — **done 2026-07-05:** B + C chosen (see decision record above); [`/patterns/dispatch-moment`](../app/(reference)/patterns/dispatch-moment/page.tsx) sketches B+C on the registry composer 2. Sketch against registry composer demos and/or compound-composer slots — not contract read-back -3. Cross-check [`docs/research.md`](../docs/research.md) §4 "Autonomy as a contract" — may need revision to match reframe -4. If promoting compound composer into the site, wire a playground route under agentic-craft (don't fork registry composer) +3. ~~Cross-check [`docs/research.md`](../docs/research.md) §4~~ — **done 2026-07-05:** §4.5 carries the reframe (behavioral badge, defensibility at Send) +4. Wire the compound-composer playground at `/playground/compound-composer` (URL-only lab; don't fork registry composer) — icons injectable per consumer shadcn config, `attachment` added via CLI diff --git a/src/components/DispatchMomentDemo.tsx b/src/components/DispatchMomentDemo.tsx index 4d522a0..c341603 100644 --- a/src/components/DispatchMomentDemo.tsx +++ b/src/components/DispatchMomentDemo.tsx @@ -16,7 +16,6 @@ import { ComposerPlan, ComposerDoneWhen, ComposerReceipt, - type ComposerDoneWhenItem, type ComposerTask, } from "@/components/ui/composer" import { @@ -31,9 +30,13 @@ import { HumanGateTitle, } from "@/components/ui/human-gate" import { cn } from "@/lib/utils" - -const FIXTURE_PROMPT = - "Migrate session-chat-page to the shared adapter — don't touch deps, stop when tests pass, keep it under $5" +import { + AUTONOMY_POSTURES, + FIXTURE_PROMPT, + evaluateDefensibility, + parseDispatchPrompt, + type AutonomyPosture, +} from "@/views/patterns/dispatch-moment-data" type ExecutionPosture = | "auto-review" @@ -41,30 +44,6 @@ type ExecutionPosture = | "review-each" | "keep-planning" -type AutonomyPosture = { - id: string - label: string - description: string -} - -const AUTONOMY_POSTURES: AutonomyPosture[] = [ - { - id: "asks-first", - label: "Asks first", - description: "Pauses before edits and external actions.", - }, - { - id: "edits-freely", - label: "Edits freely", - description: "Applies file edits without per-step approval.", - }, - { - id: "on-its-own", - label: "On its own", - description: "Runs until done or a gate fires.", - }, -] - const EXECUTION_OPTIONS: { id: ExecutionPosture label: string @@ -111,103 +90,6 @@ const PLAN_TASKS: ComposerTask[] = [ }, ] -type ParsedDispatch = { - summary: string - doneWhen: ComposerDoneWhenItem[] - exclusions: string[] - budget?: string - needsPlanGate: boolean -} - -function parseDispatchPrompt(text: string): ParsedDispatch { - const trimmed = text.trim() - const lower = trimmed.toLowerCase() - - const doneWhen: ComposerDoneWhenItem[] = [] - const exclusions: string[] = [] - let budget: string | undefined - - const stopMatch = lower.match( - /(?:stop|done)\s+when\s+([^—–.;]+(?:pass|green|complete)[^—–.;]*)/i - ) - if (stopMatch) { - doneWhen.push({ - id: "stop", - label: stopMatch[1].trim(), - }) - } - - const budgetMatch = trimmed.match( - /(?:under|below|keep\s+it\s+under)\s+\$?\d+/i - ) - if (budgetMatch) { - budget = budgetMatch[0].replace(/^keep it /i, "") - doneWhen.push({ id: "budget", label: budget }) - } - - const dontTouchMatch = trimmed.match(/don't touch\s+([^—–.;]+)/i) - if (dontTouchMatch) { - exclusions.push(dontTouchMatch[1].trim()) - doneWhen.push({ - id: "exclude", - label: `${dontTouchMatch[1].trim()} locked`, - }) - } - - const summary = - trimmed.length > 72 ? `${trimmed.slice(0, 69).trim()}…` : trimmed || "…" - - const needsPlanGate = - /migrate|refactor|implement|across|shared adapter/i.test(trimmed) && - trimmed.length > 40 - - return { - summary, - doneWhen, - exclusions, - budget, - needsPlanGate, - } -} - -type VerificationGap = { - id: string - label: string - met: boolean -} - -function buildVerificationGaps( - parsed: ParsedDispatch, - fullText: string -): VerificationGap[] { - const hasStop = parsed.doneWhen.some((item) => item.id === "stop") - const hasBudget = Boolean(parsed.budget) - const hasScope = parsed.exclusions.length > 0 - - return [ - { - id: "stop", - label: "Stopping condition", - met: hasStop, - }, - { - id: "proof", - label: "Verification method", - met: hasStop && /test|check|pass|green|ci/i.test(fullText), - }, - { - id: "scope", - label: "Scope boundary", - met: hasScope, - }, - { - id: "budget", - label: "Spend cap", - met: hasBudget, - }, - ] -} - function DispatchMomentDemo() { const [draft, setDraft] = React.useState("") const [postureIndex, setPostureIndex] = React.useState(0) @@ -222,7 +104,7 @@ function DispatchMomentDemo() { const parsed = React.useMemo(() => parseDispatchPrompt(draft), [draft]) const verificationGaps = React.useMemo( - () => buildVerificationGaps(parsed, draft), + () => evaluateDefensibility(parsed, draft), [parsed, draft] ) const missingVerification = verificationGaps.filter((gap) => !gap.met) @@ -466,4 +348,4 @@ function DispatchMomentDemo() { ) } -export { DispatchMomentDemo, FIXTURE_PROMPT, parseDispatchPrompt } +export { DispatchMomentDemo } diff --git a/src/views/patterns/dispatch-moment-content.tsx b/src/views/patterns/dispatch-moment-content.tsx index b40b680..af41dbb 100644 --- a/src/views/patterns/dispatch-moment-content.tsx +++ b/src/views/patterns/dispatch-moment-content.tsx @@ -7,12 +7,17 @@ import { PatternDemo } from "@/components/reference/pattern-demo" import { PatternPage } from "@/components/reference/pattern-page" import { PatternSection } from "@/components/reference/pattern-section" import { PatternSpecTable } from "@/components/reference/pattern-spec-table" +import { + AUTONOMY_POSTURES, + DEFENSIBILITY_GAPS, +} from "@/views/patterns/dispatch-moment-data" const PRINCIPLES = [ { title: "Behavioral badge, not a ladder", - description: - "Ship posture as language users recognize — asks first, edits freely — not L0–L5 numbers.", + description: `Ship posture as language users recognize — ${AUTONOMY_POSTURES.map( + (posture) => posture.label.toLowerCase() + ).join(", ")} — not L0–L5 numbers.`, }, { title: "Verification before posture", @@ -62,13 +67,13 @@ function DispatchMomentContent() { className="mb-10 grid scroll-mt-20 gap-4 border-y border-border/60 py-5 lg:grid-cols-[minmax(0,0.8fr)_minmax(0,1.2fr)]" >
-

Reframe

+

Direction — ratified 2026-07-05

Verification defensibility + plan→execute — not contract read-back.

-

Open questions pursued

+

Questions chosen

B · Verification defensibility @@ -78,12 +83,13 @@ function DispatchMomentContent() {

- Continues the July 3 composer/autonomy session handoff in{" "} + Decision recorded in{" "} sessions/2026-07-03-eve-chat-composer-autonomy.md - . Supersedes the Bare/B/C/D contract-variant framing until the - problem is reframed. + . Question A resolves as behavioral badge, not ladder UI; the + Bare/B/C/D contract-variant framing is retired unless B+C work + surfaces a concrete need for it.

@@ -119,6 +125,52 @@ function DispatchMomentContent() { + +
+
+

Posture badge

+
    + {AUTONOMY_POSTURES.map((posture) => ( +
  • + + {posture.label} + + + {posture.description} + +
    • + ))} +
+
+
+

Defensibility checklist

+
    + {DEFENSIBILITY_GAPS.map((gap) => ( +
  • + + {gap.label} + + + {gap.hint} + +
    • + ))} +
+
+
+ + diff --git a/src/views/patterns/dispatch-moment-data.ts b/src/views/patterns/dispatch-moment-data.ts new file mode 100644 index 0000000..aa9c5fa --- /dev/null +++ b/src/views/patterns/dispatch-moment-data.ts @@ -0,0 +1,196 @@ +import type { ComposerDoneWhenItem } from "@/components/ui/composer" + +// --------------------------------------------------------------------------- +// Canonical B+C dispatch-moment vocabulary. +// +// Ratified 2026-07-05 — see sessions/2026-07-03-eve-chat-composer-autonomy.md +// and docs/research.md §4.5. Single source of truth for the posture badge +// labels and the pre-Send defensibility checklist. Consumed by +// DispatchMomentDemo, the /patterns/dispatch-moment page, and the +// compound-composer playground. +// --------------------------------------------------------------------------- + +const FIXTURE_PROMPT = + "Migrate session-chat-page to the shared adapter — don't touch deps, stop when tests pass, keep it under $5" + +type AutonomyPostureId = "asks-first" | "edits-freely" | "on-its-own" + +type AutonomyPosture = { + id: AutonomyPostureId + label: string + description: string +} + +/** + * Behavioral posture badge vocabulary. Levels (L0–L5) stay analysis + * vocabulary; at Send the user sees behavior, not a number. + */ +const AUTONOMY_POSTURES: readonly AutonomyPosture[] = [ + { + id: "asks-first", + label: "Asks first", + description: "Pauses before edits and external actions.", + }, + { + id: "edits-freely", + label: "Edits freely", + description: "Applies file edits without per-step approval.", + }, + { + id: "on-its-own", + label: "On its own", + description: "Runs until done or a gate fires.", + }, +] as const + +type DefensibilityGapId = "stopping" | "proof" | "scope" | "budget" | "risk" + +type DefensibilityGapDefinition = { + id: DefensibilityGapId + label: string + /** What closes the gap — surfaced as guidance when the gap is open. */ + hint: string +} + +/** + * The five pre-Send defensibility gaps. A posture is defensible for a task + * when the ask closes these; open gaps are surfaced before Send, not after + * failure. + */ +const DEFENSIBILITY_GAPS: readonly DefensibilityGapDefinition[] = [ + { + id: "stopping", + label: "Stopping condition", + hint: "Say when the agent should stop.", + }, + { + id: "proof", + label: "Verification method", + hint: "Name the proof — tests green, CI pass, a check that closes the task.", + }, + { + id: "scope", + label: "Scope boundary", + hint: "Mark what is out of bounds.", + }, + { + id: "budget", + label: "Spend cap", + hint: "Cap cost or time before the run starts.", + }, + { + id: "risk", + label: "Risky actions", + hint: "Constrain deploys, deletes, sends, and other hard-to-undo actions.", + }, +] as const + +type ParsedDispatch = { + summary: string + doneWhen: ComposerDoneWhenItem[] + exclusions: string[] + budget?: string + needsPlanGate: boolean +} + +/** + * Fixture-grade parse of a dispatch ask. Heuristics are intentionally simple: + * good enough to drive the specimen and the playground fixtures, not a real + * intent parser. + */ +function parseDispatchPrompt(text: string): ParsedDispatch { + const trimmed = text.trim() + const lower = trimmed.toLowerCase() + + const doneWhen: ComposerDoneWhenItem[] = [] + const exclusions: string[] = [] + let budget: string | undefined + + const stopMatch = lower.match( + /(?:stop|done)\s+when\s+([^—–.;]+(?:pass|green|complete)[^—–.;]*)/i + ) + if (stopMatch) { + doneWhen.push({ + id: "stop", + label: stopMatch[1].trim(), + }) + } + + const budgetMatch = trimmed.match( + /(?:under|below|keep\s+it\s+under)\s+\$?\d+/i + ) + if (budgetMatch) { + budget = budgetMatch[0].replace(/^keep it /i, "") + doneWhen.push({ id: "budget", label: budget }) + } + + const dontTouchMatch = trimmed.match(/don't touch\s+([^—–.;]+)/i) + if (dontTouchMatch) { + exclusions.push(dontTouchMatch[1].trim()) + doneWhen.push({ + id: "exclude", + label: `${dontTouchMatch[1].trim()} locked`, + }) + } + + const summary = + trimmed.length > 72 ? `${trimmed.slice(0, 69).trim()}…` : trimmed || "…" + + const needsPlanGate = + /migrate|refactor|implement|across|shared adapter/i.test(trimmed) && + trimmed.length > 40 + + return { + summary, + doneWhen, + exclusions, + budget, + needsPlanGate, + } +} + +const RISKY_ACTION_PATTERN = + /migrat|deploy|delete|drop|push|publish|prod|send|email/i + +type DefensibilityGapStatus = DefensibilityGapDefinition & { + met: boolean +} + +/** + * Evaluate the five defensibility gaps against a parsed ask. The risk gap is + * met when the ask names no hard-to-undo action, or names one and also sets a + * boundary (scope or stopping condition) around it. + */ +function evaluateDefensibility( + parsed: ParsedDispatch, + fullText: string +): DefensibilityGapStatus[] { + const hasStop = parsed.doneWhen.some((item) => item.id === "stop") + const hasBudget = Boolean(parsed.budget) + const hasScope = parsed.exclusions.length > 0 + const namesRiskyAction = RISKY_ACTION_PATTERN.test(fullText) + + const met: Record = { + stopping: hasStop, + proof: hasStop && /test|check|pass|green|ci/i.test(fullText), + scope: hasScope, + budget: hasBudget, + risk: !namesRiskyAction || hasScope || hasStop, + } + + return DEFENSIBILITY_GAPS.map((gap) => ({ ...gap, met: met[gap.id] })) +} + +export { + AUTONOMY_POSTURES, + DEFENSIBILITY_GAPS, + FIXTURE_PROMPT, + evaluateDefensibility, + parseDispatchPrompt, + type AutonomyPosture, + type AutonomyPostureId, + type DefensibilityGapDefinition, + type DefensibilityGapId, + type DefensibilityGapStatus, + type ParsedDispatch, +}