Skip to content

feat(MAPCO-11434): implement a claimed ticket with the Agent SDK and verify it locally - #7

Open
razbroc wants to merge 1 commit into
feat/claim-and-releasefrom
feat/implement-and-verify
Open

feat(MAPCO-11434): implement a claimed ticket with the Agent SDK and verify it locally#7
razbroc wants to merge 1 commit into
feat/claim-and-releasefrom
feat/implement-and-verify

Conversation

@razbroc

@razbroc razbroc commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Stacked on #6 — review that first, and this PR retargets as the stack merges.

Question Answer
Bug fix
New feature
Breaking change
Deprecations
Documentation
Tests added
Chore

Hands a claimed ticket to the Claude Agent SDK inside the clone, runs the clone's own inferred test command, and retries up to a bound before giving up. The diff is left in the working tree — nothing is committed, nothing is pushed, no branch is created.

The tool surface is the control, not the prompt

tools is the SDK's base-set option, so the tools not named there are never built for the session: there is no Bash for the model to reach for, no subagent to delegate a git push to, and no MCP server it can be handed. On top of that, Bash, PowerShell, the worktree tools, Agent/Task, the network tools, Skill and mcp__* are denied by name — a deny rule outranks every other step of the permission evaluation, so they stay off even if someone later widens the base set.

settingSources: [] with strictMcpConfig matters more than it looks: the clone is a repository off the internet, and with the project source enabled its own .claude/settings.json would be read as permission rules — letting the thing being worked on widen what may be done to it. The cost is that the clone's CLAUDE.md is not loaded either, which is a real loss and the right side of the trade.

src/agent/prompt.ts also asks the model not to use git. That sentence is documented as not being the control. It is there so a model that goes looking is told why, and deniedTools reports it if it does.

⚠️ A write counts only when its result comes back

This was a genuine bug on the way in, and it is the one worth a reviewer's attention.

wroteFiles originally inferred "the tree changed" from an attempted tool_use block. So an Edit whose old_string did not match, or a Write the permission layer refused, reported changed — and the worker went on to run the clone's suite against an unmodified tree, watch it pass, and certify a verified diff that did not exist. Certifying the diff is the entire point of this slice.

It now pairs each write tool_use with the tool_result that answers it, by id, and treats a missing is_error as success rather than reading a falsy value as a pass. A call that was denied, or that the run never got back to, has no tool_result at all and so counts as no change. Four tests cover it: a refused write, a write that never came back, a mix where one of two landed, and a successful read result that must not satisfy an unrelated write attempt.

The test plan is taken before the model runs

The inferred command is read off the pristine clone before the first hand-off and every attempt is graded against that same snapshot, so a model cannot rewrite the command that grades it. Inference is test:citesttest:unit from the clone's own package.json, with the npm placeholder script rejected.

Install lifecycle scripts are compared before npm ci runs them, and a manifest the model moved installs with --ignore-scripts. spawnRunner uses shell: false and strips GITHUB_TOKEN/GH_TOKEN/GH_PAT from the child environment — proven by a real spawn, not a mock.

SDK bindings were read off the installed package

Worth stating because it nearly went wrong: fetching the TypeScript reference returned invented type names, so every option, message field and the mcp__* wildcard was verified against node_modules/@anthropic-ai/claude-agent-sdk/sdk.d.ts (v0.3.237) instead. Nothing here rests on a guessed binding.

One caveat a reviewer confirming the no-shell guarantee should know: the tools docstring notes that native builds may provide search via Bash find/grep rather than the dedicated Grep/Glob tools. That is about search availability, not about Bash becoming reachable.

Acceptance criteria

  • Partial — The worker authenticates with an Anthropic API key from an OpenShift Secret, never an interactive login. The code half is done and tested: ANTHROPIC_API_KEY only, no fallback to the login token, and the login token is stripped from the model's child env. The chart half is missing — helm/templates/deployment.yaml wires every sibling env var and no model credential, so the pod crash-loops with no value to set.
  • Partial — The ticket is handed to the Agent SDK as the task, working inside the clone. cwd and the prompt are right, but DescriptionPort has no implementation: the poll does not fetch a description and JiraTicket carries none, so every ticket is handed back before the first model turn. Four lines across POLL_FIELDS, McpTicket, toTicket and JiraTicket.
  • The repo's inferred test command runs and passes before the work counts as done
  • Repeated test failure, after a bounded number of attempts, takes the existing release path (bound is 3; every give-up funnels through one exit. The binding to handBackTicket arrives in the count-attempts PR.)
  • The model has no git or GitHub tooling available to it
  • Nothing is pushed and no branch is created in this slice
  • Tests assert only on observable outcomes

Not wired in

implementTicket has no caller. src/cycle.ts is untouched, and there is no clone-provisioning step yet (MAPCO-11433), so the ticket's Expected Result cannot be demonstrated end to end from this PR alone.

Refs: MAPCO-11434

…ally

MAPCO-11434. Hands a claimed ticket to the Claude Agent SDK inside the clone,
runs the clone's own inferred test command, and retries up to a bound before
giving up. The diff is left in the working tree; nothing is committed or pushed.

The model's tool surface is the security control, not the prompt: `tools` names
file and search tools only, so no Bash is built for the session, and Bash,
PowerShell, the worktree tools, subagents, the network tools and every MCP tool
are denied by name as well. Settings sources are off, so the clone cannot widen
its own permissions.

A write counts as a change only when its `tool_result` comes back without an
error. Reading the attempted `tool_use` alone meant a failed Edit or a denied
Write certified a pristine tree as a verified diff, which is the one thing this
slice exists to guarantee.

Not wired into runCycle yet, and `DescriptionPort` has no implementation: the
poll does not fetch a description and `JiraTicket` carries none, so every ticket
is handed back before the first model turn until that lands. The chart has no
ANTHROPIC_API_KEY secretKeyRef yet either.
@github-actions

Copy link
Copy Markdown

🎫 Related Jira Issue: MAPCO-11434

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant