openenv/tbench2: per-task Daytona cloud-sandbox execution backend

[nblintao]

Third of a three-PR stack: #1710 (per-task sandbox recipe) → #1711 (orphan TTL + ownership labels) → this. Pure additive change: the existing shared-server path is behaviorally unchanged (asserted by a dispatch unit test), and every #1487 mechanism — _apply_workdir, canonical exec + reward-marker scoring, rm-hack, policy.close() — is preserved as-is.

What

A second execution backend for the OpenEnv TB2 adapter: set OPENENV_TB2_TASKS_DIR to a terminal-bench-2 checkout and every episode runs in its own Daytona cloud sandbox, built from the task's official image (task.toml docker_image) plus a tbench2_env server layer, deleted when the episode ends.

Same per-task image fidelity as docker mode, with zero resident infrastructure: no Docker socket, no shared env server to size or babysit (no MAX_CONCURRENT_ENVS / ulimit tuning), no cross-episode state. Cost profile: first episode of a task builds its image (~10 min, layer-cached by definition hash), repeats start in ~1 min; no named snapshots, so no org snapshot quota. Sandboxes carry ownership labels and a TTL dead-man's switch (#1711) for safe operation in a shared org, and creation is throttled process-wide with jittered backoff (Daytona rate-limits creates).

Why the two backends score differently (on purpose)

Each leg matches what its env server actually provides:

• Shared-server leg (unchanged): adapter-driven canonical exec + reward-marker parse — this compensates for an UNMODIFIED upstream tbench2_env server, exactly as openenv: TB2 agentic RL adapter + GLM-4.7-Flash launcher #1487 established.
• Per-task leg (new): standard evaluate action. The sandbox recipe (tb2_task_sandbox, openenv/tbench2: per-task sandbox image recipe + Daytona materialization #1710) bakes the installed tbench2_env package into each task image; upstream tbench2_env (main, ≥ the fix(tbench2_env): canonical test.sh scoring, real timeouts, task-image workdir huggingface/OpenEnv#965 + [fix] [test] attention_output_gate TP slice when num_kv_heads < TP #972 merge) carries the server-side features this leg scores through — the same canonical tests/test.sh run natively, the task WORKDIR resolved server-side, and the verifier assets (tests/, solution/) withheld from the agent's filesystem outside the verify window (TB2_WITHHOLD_TESTS — the reward-hacking fix). The launcher preflights the installed server source for exactly this contract: an unpatched install wouldn't fail per-episode, it would silently mis-score every episode. No _apply_workdir prefix on this leg either — task WORKDIRs are not uniformly /app (fix-git = /app/personal-site, prove-plus-comm = /workspace), which the hardcoded prefix silently breaks.

Once the shared-server deployment also upgrades to that server (now plain upstream main), a follow-up can retire the adapter-driven machinery and unify both legs on evaluate.

Tooling (ships with the backend; produced the validation evidence)

• scan_golden.py — infra baseline with no LLM: replays each task's OFFICIAL solution/solve.sh (oracle-faithful: pushed from the LOCAL tasks checkout and staged at /solution — the image withholds verifier assets, so the solution is chunk-uploaded at golden time; DEBIAN_FRONTEND=noninteractive, [solution].env exported, task-workdir cwd) through the same sandbox + evaluate scoring; --logs captures solve/test log tails for anything below 1.0.
• eval_tbench2_via_api.py — the EXACT training agent-env loop (_multi_turn) with any OpenAI-compatible API as the policy: no GPU, no Ray/Megatron. For end-to-end smokes and for measuring base-model solve rates when picking a variance-band training subset.
• tests/ — offline unit tests (pytest examples/experimental/openenv/tests/ -q, not collected by the repo-level suite): backend dispatch on both legs, and the create-throttling retry/backoff/give-up behavior that only triggers under production rate limits.

Validation

• Per-task leg, no LLM (scan_golden.py): golden sweep passes 82/89 of the TB2 suite (0 infra errors; the 7 remaining have upstream-broken solutions, each with solve.log evidence). Golden regression on this exact branch: fix-git, chess-best-move, circuit-fibsqrt all 1.0 — covering per-task WORKDIR dispatch, apt paths, and long canonical evals.
• Per-task leg, end-to-end with a live API policy (eval_tbench2_via_api.py): full 89-task sweep with DeepSeek-V4-Flash as the policy (single sample per task, 30-turn cap, up to 38 concurrent episodes) — 33/89 solved, 0 systematic infra failures (10 non-scoring episodes: 8 heavyweight-task timeouts, i.e. reward 0 under training semantics, 1 transient registry network error, 1 sandbox-resource overrun).
• Shared-server leg: dispatch unit test asserts behavior identical to before this change (exec prefixed with cd /app, canonical-exec scoring, rm-hack present, evaluate never used).
• Post-reorg sanity (2026-07-17, after the openenv/tbench2: per-task sandbox image recipe + Daytona materialization #1710/openenv/tbench2: orphan TTL + ownership labels for per-task sandboxes #1711 split + in-repo recipe adoption): golden regression fix-git / chess-best-move / circuit-fibsqrt → 3/3 at 1.0 (solve_exit=0; circuit's 43s canonical eval intact); live-sandbox contract check → ownership labels present (openenv-launcher/openenv-run-id/task), TTL armed (auto_stop=30 / auto_delete=120), keepalive thread beating, clean delete; API-policy sanity (DeepSeek-V4-Flash through the full _multi_turn + per-task sandbox path) → 4/4 solved, 0 infra errors (fix-git 14 turns, prove-plus-comm 7, log-summary-date-ranges 6, largest-eigenval 30); full GRPO training smoke on 8×H200 (GLM-4.7-Flash colocated, TP=4 EP=2, 8-task subset, rollout-batch 4 × 4 samples) → rollout 16/16 episodes through per-task Daytona sandboxes, train step outcome=NORMAL, weight update, next rollout — full loop closed. raw_reward 0.1875 (3/16 solved, one prompt group with live GRPO signal), train/rollout logprob abs-diff 0.046 / KL 0.008 (token-faithful TITO recording), prefix-cache hit rate 0.96, sandboxes carried the TTL/ownership contract in production (launcher=tao-lin, stop=30, del=120, deleted on episode end).
• Unit tests: 15 passed (dispatch on both legs; throttle retry-until-success / give-up-after-budget / non-throttle-passthrough; throttle-error classification incl. the typed DaytonaRateLimitError path; recipe hygiene, labels, TTL keepalive lifecycle from the stack below).

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces support for per-task Daytona cloud sandboxes as an alternative to a shared environment server in the OpenEnv Terminal-Bench-2 adapter. It includes documentation updates, new standalone evaluation and validation scripts (eval_tbench2_via_api.py and scan_golden.py), and offline unit tests. The reviewer's feedback focuses on improving code robustness, such as replacing assert with ValueError for argument validation, using isinstance checks instead of parsing exception strings, and explicitly checking for environment variables rather than relying on exception handling.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[Shi-Dong]

Hmm if we save DAYTONA_API_KEY here, the ray job might print it out in the logs, which means that the key might be leaked.

[Shi-Dong]

Probably we should declare openai in Miles requirements?

[Shi-Dong]

I'd suggest leaving openenv_agent_function.py as-is, and create another script for the Daytona code path. Daytona is only one of the many sandbox providers that users can choose from, and mixing it with the generic sandbox-agnostic code path feels a little bit weird 😂