openenv/tbench2: Daytona sandbox mode — one cloud sandbox per episode, as its own agent function

examples/experimental/openenv/README.md

@@ -50,6 +50,55 @@ concurrency. Per-task containers are heavy on disk — if you'd rather not coloc
 them with the GPU workload, run the env server on a separate Docker host and point
 the launcher at it via `--openenv-env-url http://<env-host>:8003`.
 
+### 2b. Alternative: Daytona cloud sandboxes (no Docker host)
+
+Instead of one shared env server, the adapter can give **every episode its own
+[Daytona](https://www.daytona.io/) cloud sandbox**, built from the task's official
+image plus an env server layer and deleted when the episode ends. Same
+per-task image fidelity as docker mode, with zero resident infrastructure (no
+Docker socket, no shared server to size or babysit, no cross-episode state), at
+the cost of per-episode sandbox creation (~1 min warm; the first episode of each
+task builds its image in ~10 min, cached after that by definition hash).
+
+The image recipe lives in [`tb2_sandbox_recipe.py`](tb2_sandbox_recipe.py), its Daytona
+materialization in [`tb2_sandbox_daytona.py`](tb2_sandbox_daytona.py) (this
+directory). The recipe bakes the **installed** `tbench2_env` package — OpenEnv's
+Terminal-Bench-2 environment package, the same one step 2's shared server
+runs — into each task image, and in this mode the adapter scores via the
+standard `evaluate` action, so the install must carry the server-side fixes
+this leg relies on (canonical `tests/test.sh` scoring built into `evaluate`,
+per-task WORKDIR resolved server-side, `TB2_WITHHOLD_TESTS` verifier-asset
+withholding — all upstream since huggingface/OpenEnv#965 + #972; the launcher
+preflights the installed source and fails fast on an older install). Install
+from upstream main (editable: the recipe embeds the package source, which
+needs `pyproject.toml` present next to the package):
+
+```bash
+git clone https://github.com/huggingface/OpenEnv.git   # >= the #965/#972 merge (39c91bfd); pin that sha if you need frozen reward semantics across a long run
+pip install -e OpenEnv/envs/tbench2_env
+```
+
+Skip step 2 entirely and set:
+
+```bash
+pip install daytona   # the SDK is imported lazily, not installed with tbench2_env
+mkdir -p ~/.config/daytona && echo dtn_... > ~/.config/daytona/api_key   # or export DAYTONA_API_KEY
+export OPENENV_TB2_TASKS_DIR=/workspace/terminal-bench-2   # the checkout from step 1
+python run-openenv-tbench2.py
+```
+
+Key supply on multi-host clusters (and why only a file *path* is ever
+forwarded) is documented in the `openenv_daytona_agent_function.py` docstring —
+this mode's own agent function, which the launcher selects automatically
+when `OPENENV_TB2_TASKS_DIR` is set.
+
+Infra sanity checks without touching a GPU (both live beside the launcher):
+`scan_golden.py` replays each task's official solution through the full
+sandbox + scoring path (`--logs` captures failure evidence; 82/89 of the TB2
+suite pass, the rest have upstream-broken solutions), and
+`eval_tbench2_via_api.py` runs the identical agentic loop with any
+OpenAI-compatible API standing in for the policy.
+
 ## 3. Launch training
 
 ```bash
@@ -65,6 +114,8 @@ Common overrides:
 | `--num-rollout` | (launcher) | Number of GRPO steps |
 | `OPENENV_MAX_TURNS` | `30` | Max agent turns per episode |
 | `OPENENV_MAX_ROLLOUT_TIME_SECONDS` | `3600` | Per-episode wall-clock cap; a straggler that exceeds it is terminated and scored 0 |
+| `OPENENV_TB2_TASKS_DIR` + Daytona key | off | Daytona sandbox mode (section 2b); overrides `--openenv-env-url`. Key: `DAYTONA_API_KEY` in the env, else a key file (`~/.config/daytona/api_key`; `DAYTONA_API_KEY_FILE` overrides) |
+| `OPENENV_DAYTONA_CREATE_CONCURRENCY` | `4` | Max in-flight sandbox creates (Daytona rate-limits creation) |
 | `--dump-details <dir>` | off | Dump per-episode tokens/logprobs/masks/reward for inspection |
 | `WANDB_KEY`, `--wandb-project`, `--wandb-team` | — | W&B logging |
 

examples/experimental/openenv/eval_tbench2_via_api.py

@@ -0,0 +1,147 @@
+"""Standalone Terminal-Bench-2 eval: an OpenAI-compatible *API* as the policy,
+Daytona sandboxes as the env. No GPU, no miles training pipeline.
+
+This reuses the exact agent-env loop miles runs during training
+(``openenv_agent_function._multi_turn``: reset -> {policy emits a shell command
+-> exec -> feed output back} -> canonical tests/test.sh -> binary reward) but
+swaps miles' session-server policy for a plain API client. The machine running
+this only orchestrates; the policy runs in the cloud (e.g. DeepSeek) and each
+episode runs in its own Daytona sandbox (the task's OFFICIAL image +
+env server layer — recipe in the sibling ``tb2_sandbox_recipe`` module,
+materialized by ``tb2_sandbox_daytona``), created
+before the episode and deleted after.
+
+Why a separate script and not ``run-openenv-tbench2.py``: those launchers always
+bring up Megatron+sglang via Ray (the policy must be miles' own engine so the
+session server can record on-policy tokens for the backward pass). Pure eval
+needs none of that. The ``run()`` entry also hardcodes ``api_key="EMPTY"``
+(self-hosted engines don't check it), so we call each module's ``run_episode``
+with our own authenticated client instead.
+
+Env vars:
+  DEEPSEEK_API_KEY / POLICY_API_KEY   policy API key (required)
+  POLICY_BASE_URL   OpenAI-compatible root (default https://api.deepseek.com)
+  POLICY_MODEL      default deepseek-v4-flash
+  OPENENV_TB2_TASKS_DIR  Daytona sandbox mode (TB2 checkout path); with
+                    DAYTONA_API_KEY or a key file at ~/.config/daytona/api_key.
+                    Otherwise OPENENV_ENV_URL is used.
+  OPENENV_MAX_TURNS, OPENENV_MAX_ROLLOUT_TIME_SECONDS, ...   as in the adapter.
+
+Usage:
+  # DAYTONA_API_KEY may be omitted if ~/.config/daytona/api_key is provisioned
+  DEEPSEEK_API_KEY=sk-... DAYTONA_API_KEY=dtn_... \
+  OPENENV_TB2_TASKS_DIR=/path/to/terminal-bench-2 \
+  python eval_tbench2_via_api.py --tasks chess-best-move --concurrency 2
+  # or from make_tbench2_data.py output:
+  python eval_tbench2_via_api.py --data /root/tbench2_train.jsonl
+"""
+
+import argparse
+import asyncio
+import json
+import os
+import sys
+
+from openai import AsyncOpenAI
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import openenv_agent_function as oaf  # noqa: E402
+
+
+def _load_rows(args: argparse.Namespace) -> list[dict]:
+    """--tasks builds rows with the adapter's own agent-contract prompt, so it
+    reproduces exactly what make_tbench2_data.py emits for training."""
+    if args.tasks:
+        ids = [t.strip() for t in args.tasks.split(",") if t.strip()]
+        return [
+            {"prompt": [{"role": "system", "content": oaf.TB2_AGENT_SYSTEM_PROMPT}], "metadata": {"task_id": t}}
+            for t in ids
+        ]
+    with open(args.data) as f:
+        return [json.loads(line) for line in f if line.strip()]
+
+
+async def _eval_one(
+    run_episode, policy: AsyncOpenAI, model: str, row: dict, request_kwargs: dict
+) -> tuple[str, float | None, dict]:
+    task_id = row.get("metadata", {}).get("task_id", "?")
+    cap = float(os.getenv("OPENENV_MAX_ROLLOUT_TIME_SECONDS", "3600"))
+    try:
+        # Per-episode wall-clock cap: run_episode deliberately leaves timeout
+        # and failure semantics to the caller (run() maps a timeout to reward 0
+        # for training; a sweep must count it as errored instead). Bounds a
+        # task that loops on slow generations so one straggler can't stall the
+        # whole sweep.
+        reward, metrics = await asyncio.wait_for(
+            run_episode(
+                policy,
+                model,
+                row.get("prompt", []),
+                request_kwargs,
+                row.get("metadata", {}),
+            ),
+            timeout=cap,
+        )
+        return task_id, reward, metrics
+    except asyncio.TimeoutError:
+        return task_id, None, {"error": f"timeout>{cap:.0f}s"}
+    except Exception as e:  # noqa: BLE001 - one task failing must not sink the sweep
+        return task_id, None, {"error": f"{type(e).__name__}: {e}"}
+
+
+async def main() -> None:
+    ap = argparse.ArgumentParser()
+    src = ap.add_mutually_exclusive_group(required=True)
+    src.add_argument("--data", help="jsonl from make_tbench2_data.py (one row per task)")
+    src.add_argument("--tasks", help="comma-separated task_ids (system prompt inlined)")
+    ap.add_argument("--concurrency", type=int, default=4)
+    ap.add_argument("--temperature", type=float, default=0.8)
+    ap.add_argument("--out", default="", help="optional path to write per-task jsonl results")
+    args = ap.parse_args()
+
+    api_key = os.getenv("DEEPSEEK_API_KEY") or os.getenv("POLICY_API_KEY")
+    if not api_key:
+        sys.exit("set DEEPSEEK_API_KEY (or POLICY_API_KEY)")
+    base_url = os.getenv("POLICY_BASE_URL", "https://api.deepseek.com")
+    model = os.getenv("POLICY_MODEL", "deepseek-v4-flash")
+    rows = _load_rows(args)
+    tasks_dir = os.getenv("OPENENV_TB2_TASKS_DIR", "").strip()
+    if tasks_dir:
+        import openenv_daytona_agent_function as odaf
+
+        run_episode = odaf.run_episode
+        env_desc = f"daytona sandboxes (tasks_dir={tasks_dir})"
+    else:
+        run_episode = oaf.run_episode
+        env_desc = os.getenv("OPENENV_ENV_URL", oaf._DEFAULT_ENV_URL)
+    print(f"policy={model} @ {base_url} | env={env_desc} | " f"{len(rows)} tasks | concurrency={args.concurrency}")
+
+    policy = AsyncOpenAI(base_url=base_url, api_key=api_key)
+    request_kwargs = {"temperature": args.temperature}
+    sem = asyncio.Semaphore(args.concurrency)
+
+    async def _run(row: dict) -> tuple[str, float | None, dict]:
+        async with sem:
+            tid, reward, metrics = await _eval_one(run_episode, policy, model, row, request_kwargs)
+            tag = "ERR" if reward is None else f"{reward:.0f}"
+            extra = metrics.get("error") or f"turns={metrics.get('turns')}"
+            print(f"  [{tag}] {tid:40s} {extra}", flush=True)
+            return tid, reward, metrics
+
+    results = await asyncio.gather(*(_run(r) for r in rows))
+
+    scored = [(t, r) for t, r, _ in results if r is not None]
+    solved = sum(1 for _, r in scored if r >= 1.0)
+    errs = sum(1 for _, r, _ in results if r is None)
+    print(f"\n=== pass {solved}/{len(scored)} scored ({errs} errored) ===")
+    if scored:
+        print("solved:", sorted(t for t, r in scored if r >= 1.0))
+    if args.out:
+        with open(args.out, "w") as f:
+            for t, r, m in results:
+                f.write(json.dumps({"task_id": t, "reward": r, "metrics": m}) + "\n")
+        print(f"wrote {args.out}")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

examples/experimental/openenv/make_tbench2_data.py

@@ -24,19 +24,11 @@
 import json
 from pathlib import Path
 
+# The agent contract (one ```bash block per turn; TASK_COMPLETE to stop) is
+# defined by the adapter, next to the code that parses it.
+from openenv_agent_function import TB2_AGENT_SYSTEM_PROMPT
 from tap import Tap
 
-# The agent contract must match openenv_agent_function._multi_turn: one shell
-# command per turn inside a single ```bash block; TASK_COMPLETE to stop.
-_SYSTEM = (
-    "You are an autonomous terminal agent solving a Terminal-Bench task. You will "
-    "be given the task instruction, then interact with a real Linux shell. On each "
-    "turn respond with EXACTLY ONE shell command inside a single ```bash code block "
-    "and nothing else. Inspect the environment, make the required changes, and "
-    "verify your work. When you are confident the task is fully complete, reply with "
-    "TASK_COMPLETE (with no code block)."
-)
-
 
 class Args(Tap):
     tasks_dir: str = "/workspace/terminal-bench-2"  # TB2 repo checkout
@@ -67,7 +59,7 @@ def main() -> None:
     with open(args.output, "w") as f:
         for tid in task_ids:
             row = {
-                "prompt": [{"role": "system", "content": _SYSTEM}],
+                "prompt": [{"role": "system", "content": TB2_AGENT_SYSTEM_PROMPT}],
                 "metadata": {"task_id": tid},
             }
             f.write(json.dumps(row) + "\n")