Skip to content

Confine OpenCode reads to the current run task#28

Closed
mlava wants to merge 1 commit into
NateBJones-Projects:mainfrom
mlava:codex/fix-bakeoff-read-isolation
Closed

Confine OpenCode reads to the current run task#28
mlava wants to merge 1 commit into
NateBJones-Projects:mainfrom
mlava:codex/fix-bakeoff-read-isolation

Conversation

@mlava

@mlava mlava commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • deny OpenCode worker reads across the run workdir
  • re-allow reads for the current taskdir while preserving external reference reads
  • add a real macOS Seatbelt regression test for sibling reads, symlink escapes, sibling writes, and special-character paths
  • document the tightened run-local boundary

Why

The existing Seatbelt profile restricted writes but left all reads open. In a six-model bakeoff, a worker whose implementation failed 9/18 public tests recovered by running:

cp ../gpt55--scheduler/scheduler.mjs ./

The copied artifact then passed the hidden grader. Task directories were write-isolated, but not competitor-isolated.

Implementation

The wrapper already receives the canonical task directory. This change derives its parent workdir, passes both paths through sandbox-exec -D parameters, denies file-read* under the workdir, then re-allows the current taskdir.

External source/reference paths remain readable. --no-sandbox remains unchanged and continues to be gated by Ringer configuration.

Verification

  • python3 -m unittest tests.test_opencode_sandbox -v
  • bash -n engines/opencode-sandboxed.sh
  • real opencode --version startup through the wrapper
  • git diff --check

The full suite produced 177/180 passes. The remaining three failures reproduce on unmodified origin/main: a missing machine-specific design-reference file and two existing date/design assertions.

Scope

This provides same-run sibling filesystem isolation on macOS. It does not claim complete filesystem confidentiality: reads outside the run workdir and shared OpenCode state remain available. Per-task OpenCode state isolation is a separate follow-up.

@justfinethanku

Copy link
Copy Markdown
Contributor

Declining this one, and we want to be clear up front: this is a direction call, not a quality call — the craft here is excellent, and the executed-inside-the-real-sandbox regression test is the best test in the current PR queue. That pattern (verify the sandbox by running reads inside it, not by inspecting the profile) is exactly this project's doctrine applied to its own tooling, and if we ever rework the wrapper we'll want it.

The reasoning: this isolation exists to protect bakeoff integrity — stopping a worker from cribbing a sibling's answer when models race on the same task. We've landed on the position that Ringer isn't a model-testing harness: capability signal accrues from the eval log on real production work, where sibling tasks are different by design and a stray read corrupts nothing. Outside that comparison scenario, run-scoped read confinement is a restriction workers can trip over in normal operation — the Node module-resolution carve-out and the nested-task-key edge cases in this diff are early evidence of the debugging surface it adds. Our general bet is verification of outputs over confinement of workers, so we're keeping the sandbox at write-scope.

If someone does want to build a rigorous bakeoff harness, this PR is a right piece of it — but that's its own project, and we'd rather not carry the machinery in the orchestrator.

Your other open PRs (#35/#36/#37/#38) are aimed at things we actively want — infra-failure evidence handling especially — and reviews on those are coming shortly. Thank you for the consistently well-scoped, well-tested contributions; please keep them coming.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants