opencode-agent.md

OpenCode Agent

The opencode-agent runner drives OpenCode against the AssetOpsBench MCP servers. It is a peer to plan-execute, claude-agent, openai-agent, deep-agent, and stirrup-agent: same CLI contract, same persisted Trajectory, scored by the same uv run evaluate.

OpenCode is a general CLI coding agent, but this runner is configured as a tools-first benchmark agent by default. In the default mode, it can call the AssetOpsBench MCP servers while local file reads, shell commands, file edits, web access, external directory access, and follow-up questions are denied unless explicitly enabled.

The runner also supports an optional CLI workspace mode. In that mode, each benchmark run gets a dedicated workspace directory and OpenCode can be allowed to inspect files and/or run shell commands inside that run workspace.

Contents

• Why OpenCode
• Install
• Quick start
• Execution modes
• Model routing
• Permissions and web access
• Headless server mode
• CLI flags
• Benchmark-suite usage
• Validation runs
• Troubleshooting
• What was added

---

Why OpenCode

• CLI-backed agent loop, so it can be evaluated as an external agent without embedding another SDK directly in AssetOpsBench.
• MCP support, so it connects to the same six AssetOpsBench servers as every other tool-using runner.
• Headless automation, via opencode run --format json, which makes it usable in scripts and benchmark runs.
• Optional CLI workspace mode, so benchmark runs can evaluate agents that write and run local analysis code.
• Server/API mode, via opencode serve, which can keep OpenCode warm across repeated runs.
• Provider flexibility, including direct OpenCode providers and AssetOpsBench router prefixes such as tokenrouter/ and litellm_proxy/.

---

Install

The Python entry point is declared in pyproject.toml as the opencode-agent script. The OpenCode executable itself is installed outside the Python package.

Install the project dependencies from the repo root:

uv sync

Confirm the AssetOpsBench entry point resolved:

uv run opencode-agent --help

Confirm the OpenCode binary is available:

opencode --version

If opencode is missing, install it with one of the methods from the OpenCode docs, for example:

npm install -g opencode-ai

Run the unit tests for config, permissions, workspace handling, and trajectory mapping:

uv run pytest src/agent/opencode_agent/tests/ -q

---

Quick start

Set the provider credentials for the model route you want, then run a small question:

export TOKENROUTER_BASE_URL=https://api.tokenrouter.com/v1
export TOKENROUTER_API_KEY=...

uv run opencode-agent --show-trajectory \
  --model-id tokenrouter/MiniMax-M3 \
  "What sites are available?"

In the --show-trajectory output, look for domain tool calls such as iot_sites, iot_registry_assets, or wo_list_workorders. That confirms OpenCode discovered and called the AssetOpsBench MCP tools.

Quiet runs. opencode-agent runs OpenCode as a subprocess. During long questions, the terminal may look quiet until OpenCode finishes. The persisted trajectory is written only after the run completes.

---

Execution modes

Default MCP-only mode

By default, OpenCode is allowed to use AssetOpsBench MCP tools only.

The following capabilities are denied by default:

• local file inspection
• shell/bash commands
• file edits
• web fetch/search
• external directory access
• follow-up questions

This is the recommended mode for normal benchmark runs where answers should come from the configured MCP tools.

uv run python -m benchmark.scenario_suite_runner \
  --scenario-ids benchmarks/scenario_suite/scenarios.txt \
  --scenario-root /path/to/scenarios_data \
  --agent_name opencode_agent \
  --model-id tokenrouter/MiniMax-M3

CLI workspace mode

CLI workspace mode is opt-in. It gives each OpenCode run a dedicated workspace directory and can allow file inspection and/or shell execution.

This mode is useful when evaluating CLI-style agents that can write small Python scripts, run local analysis, and use intermediate artifacts while still using MCP tools for operational data.

uv run python -m benchmark.scenario_suite_runner \
  --scenario-ids benchmarks/scenario_suite/scenarios.txt \
  --scenario-root /path/to/scenarios_data \
  --agent_name opencode_agent \
  --model-id tokenrouter/MiniMax-M3 \
  --opencode-workspace-root traces/opencode_workspaces \
  --opencode-allow-files \
  --opencode-allow-bash

For scenario 401, this creates a workspace such as:

traces/opencode_workspaces/opencode_agent_401

Each scenario run starts with an empty per-run workspace directory. The benchmark loader fills CouchDB from the scenario repository, but the OpenCode process itself does not get the scenario's raw shared/ files staged into the workspace. This keeps the CLI run focused on MCP/CouchDB access instead of reading the source files directly.

Safety note. --opencode-allow-bash is not a hard OS-level sandbox. A shell or Python process can still attempt to access files outside the workspace. For strict filesystem isolation, run the benchmark inside Docker or another sandboxed environment.

---

Model routing

The runner's default model is opencode/gpt-5.1-codex.

--model-id prefix	OpenCode provider config	Required env vars
opencode/<model>	OpenCode built-in provider	OpenCode auth / provider setup
openai/<model>	OpenCode built-in OpenAI provider	OPENAI_API_KEY or OpenCode auth
anthropic/<model>	OpenCode built-in Anthropic provider	ANTHROPIC_API_KEY or OpenCode auth
tokenrouter/<model>	Custom OpenAI-compatible tokenrouter provider with explicit model registration	TOKENROUTER_BASE_URL, TOKENROUTER_API_KEY
litellm_proxy/<model>	Custom OpenAI-compatible litellm-proxy provider with explicit model registration	LITELLM_BASE_URL, LITELLM_API_KEY

For router prefixes, the runner generates inline OpenCode config using OPENCODE_CONFIG_CONTENT. For example:

tokenrouter/MiniMax-M3 -> tokenrouter/MiniMax-M3
litellm_proxy/azure/gpt-5.4 -> litellm-proxy/azure/gpt-5.4

The custom provider route is important because OpenCode's built-in openai/* provider validates model names against its own model registry. Router-hosted models such as MiniMax-M3 must be registered explicitly.

FMSR tool model

FMSR tools take asset_class. --model-id controls OpenCode; LLM-backed FMSR tools use FMSR_MODEL_ID.

MODEL_ID=tokenrouter/MiniMax-M3
FMSR_MODEL_ID="$MODEL_ID" \
uv run opencode-agent --model-id "$MODEL_ID" --show-trajectory \
  "Use generate_failure_mode_sensor_mapping for asset_class pump with failure_modes ['seal leakage'] and sensors ['Pressure sensor']."

---

Permissions and web access

The runner configures OpenCode permissions for benchmark use:

Capability	Default	Flag to allow
AssetOpsBench MCP tools	allowed	always enabled
read, glob, grep, lsp	denied	--allow-files
shell commands	denied	--allow-bash
file edits	denied	--allow-edit
web fetch/search	denied	--allow-web
external directory access	denied	not exposed
follow-up questions	denied	not exposed

Benchmark runs should not pass --allow-web. Without that flag, OpenCode's webfetch and websearch permissions are explicitly set to deny.

The CLI auto-approves allowed permissions by default so unattended benchmark runs do not block on prompts:

--dangerously-skip-permissions

This does not grant access to explicitly denied tools. For example, web access remains denied unless --allow-web is passed.

If you want OpenCode to ask before using allowed tools during local debugging, pass:

--ask-permissions

Do not use --ask-permissions for batch benchmark runs; it can wait for user input.

---

Headless server mode

OpenCode can also run as a long-lived headless server:

opencode serve --port 4096

Then attach benchmark calls to it:

uv run opencode-agent --attach http://localhost:4096 \
  --model-id tokenrouter/MiniMax-M3 \
  "What sites are available?"

This can reduce startup overhead when running many scenarios. The default runner does not require server mode; it starts OpenCode through opencode run.

---

CLI flags

In addition to the common flags (--model-id, --show-trajectory, --json, --run-id, --scenario-id, ...):

Flag	Description
--max-steps N	Maximum OpenCode agentic iterations (default: 30).
--agent-name NAME	Inline OpenCode agent name (default: assetops).
--opencode-bin PATH	OpenCode executable path (default: opencode).
--attach URL	Attach to a running opencode serve instance.
--timeout-s N	Wall-clock timeout for opencode run (default: 900).
--allow-files	Allow file inspection tools (read, glob, grep, lsp). Disabled by default.
--allow-bash	Allow shell commands. Disabled by default.
--allow-edit	Allow file edits. Disabled by default.
--allow-web	Allow web fetch/search. Disabled by default.
--workspace-dir PATH	Workspace directory required when files, bash, or edits are enabled.
--ask-permissions	Do not auto-approve allowed permissions. Not suitable for batch runs.

Direct CLI workspace example:

uv run opencode-agent \
  --model-id tokenrouter/MiniMax-M3 \
  --workspace-dir /tmp/assetopsbench-opencode/smoke \
  --allow-files \
  --allow-bash \
  "Analyze the maintenance cost distribution."

---

Benchmark-suite usage

The benchmark method name uses an underscore:

uv run python -m benchmark.scenario_suite_runner \
  --scenario-ids benchmarks/scenario_suite/scenarios.txt \
  --scenario-root /path/to/scenarios_data \
  --agent_name opencode_agent \
  --model-id tokenrouter/MiniMax-M3

The direct CLI command uses a hyphen:

uv run opencode-agent --model-id tokenrouter/MiniMax-M3 "What sites are available?"

For a faster smoke test, skip evaluation:

uv run python -m benchmark.scenario_suite_runner \
  --scenario-ids benchmarks/scenario_suite/scenarios.txt \
  --scenario-root /path/to/scenarios_data \
  --agent_name opencode_agent \
  --model-id tokenrouter/MiniMax-M3 \
  --no-evaluate

To run with CLI workspace capabilities:

uv run python -m benchmark.scenario_suite_runner \
  --scenario-ids benchmarks/scenario_suite/scenarios.txt \
  --scenario-root /path/to/scenarios_data \
  --agent_name opencode_agent \
  --model-id tokenrouter/MiniMax-M3 \
  --opencode-workspace-root traces/opencode_workspaces \
  --opencode-allow-files \
  --opencode-allow-bash

Scenario-suite OpenCode flags:

Flag	Description
--opencode-workspace-root PATH	Root directory for empty per-run OpenCode workspaces.
--opencode-allow-files	Pass --allow-files to opencode-agent.
--opencode-allow-bash	Pass --allow-bash to opencode-agent.
--opencode-allow-edit	Pass --allow-edit to opencode-agent.

If any OpenCode file, bash, or edit capability is enabled, then --opencode-workspace-root is required.

---

Validation runs

A reproducible checklist a teammate can follow to confirm a working setup:

# 0. install + entry point
uv sync
uv run opencode-agent --help
opencode --version

# 1. unit tests (no model call)
uv run pytest src/agent/opencode_agent/tests/ -q

# 2. direct MCP-only smoke test
export TOKENROUTER_BASE_URL=https://api.tokenrouter.com/v1
export TOKENROUTER_API_KEY=...
uv run opencode-agent --show-trajectory \
  --model-id tokenrouter/MiniMax-M3 \
  "What sites are available?"

# 3. direct CLI workspace smoke test
uv run opencode-agent --show-trajectory \
  --model-id tokenrouter/MiniMax-M3 \
  --workspace-dir /tmp/assetopsbench-opencode/smoke \
  --allow-files \
  --allow-bash \
  "What is the current time?"

# 4. persist a trajectory
export AGENT_TRAJECTORY_DIR=$(pwd)/traces/trajectories
uv run opencode-agent --run-id opencode-smoke --scenario-id smoke \
  --model-id tokenrouter/MiniMax-M3 \
  "What sites are available?"

# 5. FMSR get_failure_modes smoke test
uv run opencode-agent --show-trajectory \
  --model-id tokenrouter/MiniMax-M3 \
  "Use the FMSR get_failure_modes tool for asset_class pump. Return only the failure_modes list."

# 6. FMSR generate_failure_modes smoke test
MODEL_ID=tokenrouter/MiniMax-M3
FMSR_MODEL_ID="$MODEL_ID" \
uv run opencode-agent --show-trajectory \
  --model-id "$MODEL_ID" \
  "Use generate_failure_modes for asset_class pump with max_modes 3. Return known, generated, and message."

# 7. FMSR add_failure_modes smoke test
uv run opencode-agent --show-trajectory \
  --model-id tokenrouter/MiniMax-M3 \
  "Use add_failure_modes for asset_class pump with failure_modes ['bearing wear'], exhaustive false, and source 'manual smoke test'. Return added, total, source, and message."

# 8. FMSR mapping smoke test
MODEL_ID=tokenrouter/MiniMax-M3
FMSR_MODEL_ID="$MODEL_ID" \
uv run opencode-agent --show-trajectory \
  --model-id "$MODEL_ID" \
  "Use generate_failure_mode_sensor_mapping for asset_class pump with failure_modes ['seal leakage'] and sensors ['Pressure sensor', 'Vibration sensor']."

# 9. benchmark-suite dry run
uv run python -m benchmark.scenario_suite_runner \
  --scenario-ids benchmarks/scenario_suite/scenarios.txt \
  --scenario-root /path/to/scenarios_data \
  --agent_name opencode_agent \
  --model-id tokenrouter/MiniMax-M3 \
  --dry-run

# 10. benchmark-suite CLI workspace dry run
uv run python -m benchmark.scenario_suite_runner \
  --scenario-ids benchmarks/scenario_suite/scenarios.txt \
  --scenario-root /path/to/scenarios_data \
  --agent_name opencode_agent \
  --model-id tokenrouter/MiniMax-M3 \
  --opencode-workspace-root traces/opencode_workspaces \
  --opencode-allow-files \
  --opencode-allow-bash \
  --dry-run

---

Troubleshooting

Symptom	Cause / fix
opencode: command not found	Install OpenCode (npm install -g opencode-ai) or pass --opencode-bin /path/to/opencode.
Model not found: openai/MiniMax-M3	Old router mapping. Use the current runner, which maps tokenrouter/<model> to a custom tokenrouter provider instead of openai/<model>.
TOKENROUTER_BASE_URL and TOKENROUTER_API_KEY must be set	Set both env vars when using tokenrouter/*. The base URL should be OpenAI-compatible, usually ending in /v1.
LITELLM_BASE_URL and LITELLM_API_KEY must be set	Set both env vars when using litellm_proxy/*.
--workspace-dir is required when enabling files, edits, or bash	Direct opencode-agent runs require a workspace when --allow-files, --allow-bash, or --allow-edit is passed.
--opencode-workspace-root is required when enabling OpenCode files, bash, or edits	Scenario-suite runs require a workspace root when any OpenCode CLI capability is enabled.
Workspace folders are empty	Normal if OpenCode did not need to write scripts or intermediate files. The workspace is available as scratch space, but files are created only if the model/tooling writes them.
Run appears stuck after Command: uv run opencode-agent ...	Normal for long scenarios: scenario_suite_runner is quiet while OpenCode works. Check traces/trajectories/scenario_suite/opencode_agent/ after completion.
Evaluator fails even though the answer text contains the right label	The answer may be too verbose for static_json extraction. Prefer concise answers such as C or JSON like {"answer": "C"} for strict structured tasks.
Web access concern	Web is denied by default. Do not pass --allow-web for benchmark runs.
Bash isolation concern	--allow-bash is not a hard OS sandbox. Use Docker or another sandbox for strict filesystem isolation.
OpenCode prompts for permissions in a batch run	Do not pass --ask-permissions; the default auto-approves allowed tools and still denies explicitly denied tools.

---

What was added

• src/agent/opencode_agent/ — runner.py, cli.py, __init__.py, and tests/test_runner.py.
• pyproject.toml — opencode-agent entry point.
• src/benchmark/scenario_suite_runner.py — opencode_agent benchmark method and optional OpenCode workspace flags.
• docs/opencode-agent.md — OpenCode usage, permissions, and workspace-mode documentation.

No MCP servers are modified. The runner generates OpenCode MCP/provider config at runtime.