Confine OpenCode reads to the current run task#28
Conversation
66a0123 to
b9a48d8
Compare
|
Declining this one, and we want to be clear up front: this is a direction call, not a quality call — the craft here is excellent, and the executed-inside-the-real-sandbox regression test is the best test in the current PR queue. That pattern (verify the sandbox by running reads inside it, not by inspecting the profile) is exactly this project's doctrine applied to its own tooling, and if we ever rework the wrapper we'll want it. The reasoning: this isolation exists to protect bakeoff integrity — stopping a worker from cribbing a sibling's answer when models race on the same task. We've landed on the position that Ringer isn't a model-testing harness: capability signal accrues from the eval log on real production work, where sibling tasks are different by design and a stray read corrupts nothing. Outside that comparison scenario, run-scoped read confinement is a restriction workers can trip over in normal operation — the Node module-resolution carve-out and the nested-task-key edge cases in this diff are early evidence of the debugging surface it adds. Our general bet is verification of outputs over confinement of workers, so we're keeping the sandbox at write-scope. If someone does want to build a rigorous bakeoff harness, this PR is a right piece of it — but that's its own project, and we'd rather not carry the machinery in the orchestrator. Your other open PRs (#35/#36/#37/#38) are aimed at things we actively want — infra-failure evidence handling especially — and reviews on those are coming shortly. Thank you for the consistently well-scoped, well-tested contributions; please keep them coming. |
Summary
Why
The existing Seatbelt profile restricted writes but left all reads open. In a six-model bakeoff, a worker whose implementation failed 9/18 public tests recovered by running:
The copied artifact then passed the hidden grader. Task directories were write-isolated, but not competitor-isolated.
Implementation
The wrapper already receives the canonical task directory. This change derives its parent workdir, passes both paths through
sandbox-exec -Dparameters, deniesfile-read*under the workdir, then re-allows the current taskdir.External source/reference paths remain readable.
--no-sandboxremains unchanged and continues to be gated by Ringer configuration.Verification
python3 -m unittest tests.test_opencode_sandbox -vbash -n engines/opencode-sandboxed.shopencode --versionstartup through the wrappergit diff --checkThe full suite produced 177/180 passes. The remaining three failures reproduce on unmodified
origin/main: a missing machine-specific design-reference file and two existing date/design assertions.Scope
This provides same-run sibling filesystem isolation on macOS. It does not claim complete filesystem confidentiality: reads outside the run workdir and shared OpenCode state remain available. Per-task OpenCode state isolation is a separate follow-up.