Add a minimal Verifiers agentic regeneration demo

docs/.nav.yml

@@ -19,6 +19,7 @@ nav:
                 - Train P-Eagle Model Offline: user_guide/tutorials/train_peagle_offline.md
                 - Train MTP Model Online: user_guide/tutorials/train_mtp_online.md
                 - Response Regeneration: user_guide/tutorials/response_regeneration.md
+                - Agentic Regeneration: user_guide/tutorials/agentic_regeneration.md
                 - Evaluating Model Performance: user_guide/tutorials/evaluating_performance.md
                 - Serve in vLLM: user_guide/tutorials/serve_vllm.md
     - Developer Guide:

docs/user_guide/tutorials/agentic_regeneration.md

@@ -0,0 +1,107 @@
+# Agentic Regeneration with Verifiers
+
+Agentic regeneration should have a narrow integration boundary:
+
+1. the upstream Verifiers CLI runs an installed taskset and writes `config.toml` plus `traces.jsonl`;
+2. one generic adapter converts each native root-to-leaf trace branch to exact `input_ids` and `loss_mask`;
+3. Speculators replays those IDs through the same checkpoint to collect aligned hidden states.
+
+There is no Speculators environment wrapper or rollout loop.
+
+## Pinned UV environment
+
+The adapter has its own UV project under `scripts/agentic_regeneration/`. Its small `pyproject.toml` pins the compatibility boundary: Python 3.12, the stable `verifiers==0.2.0` release, and one exact research-environments commit.
+
+```bash
+uv sync --project scripts/agentic_regeneration
+```
+
+This environment is intentionally separate from the main Speculators environment. Updating Verifiers or an upstream taskset is an explicit `pyproject.toml` change, while UV resolves ordinary transitive dependencies at install time.
+
+## Environment profiles
+
+Three native CLI profiles provide a useful progression:
+
+| Profile | What it exercises | Cost |
+|---|---|---|
+| `prolog.toml` | Multi-turn code editing, execution feedback, and hidden verification | Public SWI-Prolog image; recommended smoke test |
+| `livecodebench.toml` | Python code generation with hidden execution tests | Shared Python image; useful correctness baseline |
+| `r2e.toml` | Multi-turn inspection and repair of a real Python repository | Per-task repository image; production acceptance test |
+
+Prolog is the quickest genuinely agentic check: tasks are generated locally and the public sandbox image is small. LiveCodeBench is lighter coding data but is capped at one turn in the supplied profile. R2E-Gym is the target for realistic Python repair, and its image and dataset cost is inherent to reproducibly executing arbitrary repositories.
+
+All three tasksets and the coding harness are upstream. R2E uses Verifiers' built-in stable `default` harness, which supplies bash and edit tools; Speculators contains no R2E-specific code.
+
+## 1. Start Qwen3-8B in vLLM
+
+```bash
+python scripts/launch_vllm.py Qwen/Qwen3-8B \
+  --hidden-states-path output/agentic_regen/server_hidden_states \
+  -- \
+  --host 127.0.0.1 \
+  --port 8000 \
+  --gpu-memory-utilization 0.5 \
+  --max-model-len 16384
+```
+
+The profiles use Verifiers' `train` client and its Qwen3 renderer with `enable_thinking = false`. Rendering and tool-call parsing therefore happen client-side, while vLLM only serves exact token generation. Disabling thinking keeps this coding profile from spending its whole turn on an unparsed reasoning block. OpenAI chat-completions tool-parser flags are not part of this path.
+
+## 2. Run an upstream taskset
+
+Start with Prolog:
+
+```bash
+VLLM_API_KEY=EMPTY uv run --project scripts/agentic_regeneration \
+  eval @ scripts/agentic_regeneration/configs/prolog.toml
+```
+
+Switch only the final profile path for another environment:
+
+```text
+scripts/agentic_regeneration/configs/livecodebench.toml
+scripts/agentic_regeneration/configs/r2e.toml
+```
+
+The stable CLI owns dataset loading, Docker lifecycle, tools, turns, retries, and scoring. Each profile is a one-task smoke run; increase `num_tasks`, `num_rollouts`, and `max_concurrent` in TOML for collection.
+
+The `train` client is important: its native trace contains the exact rendered prompt and sampled completion token spans. A normal chat-completions response is not sufficient for exact hidden-state alignment.
+
+## 3. Convert native traces
+
+For the Prolog profile:
+
+```bash
+uv run --project scripts/agentic_regeneration \
+  python scripts/agentic_regeneration/convert_traces.py \
+  --traces output/agentic_regen/prolog/traces.jsonl \
+  --outfile output/agentic_regen/prolog/trajectories.jsonl
+```
+
+The adapter targets the stable Verifiers `WireTrace` format. It reconstructs every root-to-leaf branch, retains messages and tool-call linkage, and reads model, endpoint, and taskset metadata from the sibling `config.toml`. Stable 0.2.0 does not duplicate tool schemas in the JSON trace; exact-token replay does not need to render them again.
+
+Completed zero-reward traces are retained because unsuccessful actions are valid on-policy data.
+
+## 4. Prepare and replay exact IDs
+
+```bash
+python scripts/prepare_data.py \
+  --model Qwen/Qwen3-8B \
+  --data output/agentic_regen/prolog/trajectories.jsonl \
+  --seq-length 16384 \
+  --minimum-valid-tokens 1 \
+  --num-preprocessing-workers 1 \
+  --output output/agentic_regen/prolog/preprocessed
+
+python scripts/data_generation_offline.py \
+  --model Qwen/Qwen3-8B \
+  --endpoint http://127.0.0.1:8000/v1 \
+  --preprocessed-data output/agentic_regen/prolog/preprocessed \
+  --output output/agentic_regen/prolog/eagle_data \
+  --concurrency 1 \
+  --validate-outputs \
+  --fail-on-error
+```
+
+When a record contains both `input_ids` and `loss_mask`, preprocessing passes them through together rather than applying the chat template again. `--validate-outputs` then checks that vLLM returned those same IDs and an aligned hidden-state sequence.
+
+The resulting states are on-policy for the served checkpoint because the model sampled every assistant turn against real observations, and replay teacher-forces that exact branch through the same weights. Regenerate after a policy update.

docs/user_guide/tutorials/index.md

@@ -44,6 +44,12 @@ Regenerate dataset responses using your target model for improved drafter alignm
 
 **Time required:** ~10 minutes
 
+## [Agentic Regeneration with Verifiers](agentic_regeneration.md)
+
+Collect exact-token, on-policy agent trajectories and aligned vLLM hidden states.
+
+**Time required:** ~10 minutes
+
 ## [Evaluating Model Performance](evaluating_performance.md)
 
 Benchmark and evaluate your trained speculator models.

scripts/agentic_regeneration/.gitignore

@@ -0,0 +1,2 @@
+.venv/
+uv.lock

scripts/agentic_regeneration/.python-version

@@ -0,0 +1 @@
+3.12

scripts/agentic_regeneration/configs/livecodebench.toml

@@ -0,0 +1,32 @@
+model = "Qwen/Qwen3-8B"
+num_tasks = 1
+num_rollouts = 1
+max_concurrent = 1
+max_turns = 1
+push = false
+rich = false
+output_dir = "output/agentic_regen/livecodebench"
+
+[taskset]
+id = "livecodebench-v1"
+difficulty = "easy"
+
+[harness]
+id = "default"
+
+[harness.runtime]
+type = "docker"
+
+[client]
+type = "train"
+base_url = "http://127.0.0.1:8000/v1"
+api_key_var = "VLLM_API_KEY"
+renderer_model_name = "Qwen/Qwen3-8B"
+
+[client.renderer]
+name = "qwen3"
+enable_thinking = false
+
+[sampling]
+temperature = 0.0
+max_tokens = 4096

scripts/agentic_regeneration/configs/prolog.toml

@@ -0,0 +1,31 @@
+model = "Qwen/Qwen3-8B"
+num_tasks = 1
+num_rollouts = 1
+max_concurrent = 1
+max_turns = 6
+push = false
+rich = false
+output_dir = "output/agentic_regen/prolog"
+
+[taskset]
+id = "prolog-v1"
+
+[harness]
+id = "default"
+
+[harness.runtime]
+type = "docker"
+
+[client]
+type = "train"
+base_url = "http://127.0.0.1:8000/v1"
+api_key_var = "VLLM_API_KEY"
+renderer_model_name = "Qwen/Qwen3-8B"
+
+[client.renderer]
+name = "qwen3"
+enable_thinking = false
+
+[sampling]
+temperature = 0.0
+max_tokens = 4096

scripts/agentic_regeneration/configs/r2e.toml

@@ -0,0 +1,31 @@
+model = "Qwen/Qwen3-8B"
+num_tasks = 1
+num_rollouts = 1
+max_concurrent = 1
+max_turns = 6
+push = false
+rich = false
+output_dir = "output/agentic_regen/r2e"
+
+[taskset]
+id = "r2e-gym-v1"
+
+[harness]
+id = "default"
+
+[harness.runtime]
+type = "docker"
+
+[client]
+type = "train"
+base_url = "http://127.0.0.1:8000/v1"
+api_key_var = "VLLM_API_KEY"
+renderer_model_name = "Qwen/Qwen3-8B"
+
+[client.renderer]
+name = "qwen3"
+enable_thinking = false
+
+[sampling]
+temperature = 0.0
+max_tokens = 4096