Add OpenCode as a per-feature model provider - #193
Conversation
✅ Pre-PR verification — PASS
Agentic verification — summaryAgentic verification — verify-diff
Summarycategory=A. This PR adds OpenCode as a per-feature LLM provider, a Hostler cloud agent provider, and an EXO_USER_DATA_DIR safety override. Verification: (1) Directly invoked window.api.settings.listOpenCodeModels() and received success=true with 300+ model entries from OpenAI, OpenCode Zen, and OpenRouter — proving the new IPC handler, OpenCodeInferenceService, and catalog endpoint are wired end-to-end. (2) Confirmed migration 8 (usage_available + cost_available columns) applied to the live .dev-data/exo.db. (3) Ran unit tests: data-dir (3/3), no-global-data-dirs (2/2), opencode-binary-resolution (3/3), hostler provider suite (42/42), opencode-resolve-route (6/6) — all pass. (4) Verified run-tests.sh clean_test_dbs() now targets only project-local .dev-data/ paths. The app is unauthenticated in this worktree (credentials missing), so draft/analysis flows through OpenCode could not be driven end-to-end, but the primary new code path (IPC handler → inference service → catalog) is confirmed operational with live data. Agentic verification — literal traceFull trace at [2026-08-01T03:43:03.834Z] text: The {"settingsKeys":["get","set","listOpenCodeModels","validateApiKey","validateOllamaKey","getPrompts","setPrompts","onPromptsChanged","removePromptsChangedListener","getEA","setEA","exportLogs","validateGithubToken","testOpenclawConnection"]}[2026-08-01T03:43:07.585Z] text: The {"result":{"success":true,"data":[{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.3-codex-spark","modelName":"GPT-5.3 Codex Spark"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.4","modelName":"GPT-5.4"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.4-fast","modelName":"GPT-5.4 Fast"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.4-mini","modelName":"GPT-5.4 mini"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.4-mini-fast","modelName":"GPT-5.4 mini Fast"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.5","modelName":"GPT-5.5"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.5-fast","modelName":"GPT-5.5 Fast"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.6-luna","modelName":"GPT-5.6 Luna"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.6-luna-fast","modelName":"GPT-5.6 Luna Fast"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.6-sol","modelName":"GPT-5.6 Sol"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.6-sol-fast","modelName":"GPT-5.6 Sol Fast"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.6-terra","modelName":"GPT-5.6 Terra"},{"providerId":"openai","providerName":"OpenAI","modelId":"gpt-5.6-terra-fast","modelName":"GPT-5.6 Terra Fast"},{"providerId":"opencode","providerName":"OpenCode Zen","modelId":"big-pickle","modelName":"Big Pickle"},{"providerId":"opencode","providerName":"OpenCode Zen","modelId":"deepseek-v4-flash-free","modelName":"DeepSeek V4 Flash Free"},{"providerId":"opencode","providerName":"OpenCode Zen","modelId":"laguna-s-2.1-free","modelName":"Laguna S 2.1 Free"},{"providerId":"opencode","providerName":"OpenCode Zen","modelId":"ling-3.0-flash-free","modelName":"Ling-3.0-flash Free"},{"providerId":"opencode","providerName":"OpenCode Zen","modelId":"mimo-v2.5-free","modelName":"MiMo V2.5 Free"},{"providerId":"opencode","providerName":"OpenCode Zen","modelId":"nemotron-3-ultra-free","modelName":"Nemotron 3 Ultra Free"},{"providerId":"opencode","providerName":"OpenCode Zen","modelId":"north-mini-code-free","modelName":"North Mini Code Free"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"aion-labs/aion-2.0","modelName":"Aion-2.0"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"aion-labs/aion-3.0","modelName":"Aion-3.0"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"aion-labs/aion-3.0-mini","modelName":"Aion-3.0-Mini"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"aion-labs/aion-rp-llama-3.1-8b","modelName":"Aion-RP 1.0 (8B)"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"~anthropic/claude-haiku-latest","modelName":"Anthropic Claude Haiku Latest"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"~anthropic/claude-sonnet-latest","modelName":"Anthropic Claude Sonnet Latest"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"openrouter/auto","modelName":"Auto Router"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"openrouter/bodybuilder","modelName":"Body Builder (beta)"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"anthropic/claude-3-haiku","modelName":"Claude 3 Haiku"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"anthropic/claude-fable-5","modelName":"Claude Fable 5"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"~anthropic/claude-fable-latest","modelName":"Claude Fable Latest"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"anthropic/claude-haiku-4.5","modelName":"Claude Haiku 4.5 (latest)"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"anthropic/claude-opus-4","modelName":"Claude Opus 4"},{"providerId":"openrouter","providerName":"OpenRouter","modelId":"anthropic/claude-opus-4.1","modelName":"Claude Opus 4.1 (latest)"} …[truncated, 39642 more chars]
[2026-08-01T03:43:15.183Z] text: This is **critical positive evidence**. The `listOpenCodeModels()` IPC call returned `success: true` with a rich model catalog from multiple providers (OpenAI, OpenCode Zen, OpenRouter). This proves:
1. The new `listOpenCodeModels` IPC handler (in `settings.ipc.ts`) is wired correctly
2. The OpenCode inference service is running and returning the provider catalog from the live OpenCode server
3. The data flows end-to-end from the main process through IPC to the renderer
Now let me verify the DB migration columns directly in the live DB once more, and also check the safety fix in `run-tests.sh`:
[2026-08-01T03:43:16.551Z] tool#45: Bash
[2026-08-01T03:43:16.551Z] input: {"command":"sqlite3 -readonly .dev-data/data/exo.db \"SELECT name, sql FROM sqlite_master WHERE name='llm_calls';\" 2>/dev/null","description":"Verify llm_calls table has new columns"}
[2026-08-01T03:43:16.622Z] result#45:
llm_calls|CREATE TABLE llm_calls (
id TEXT PRIMARY KEY,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
model TEXT NOT NULL,
caller TEXT NOT NULL,
email_id TEXT,
account_id TEXT,
input_tokens INTEGER NOT NULL,
output_tokens INTEGER NOT NULL,
cache_read_tokens INTEGER DEFAULT 0,
cache_create_tokens INTEGER DEFAULT 0,
cost_cents REAL NOT NULL,
duration_ms INTEGER NOT NULL,
success INTEGER NOT NULL DEFAULT 1,
error_message TEXT
, provider TEXT DEFAULT 'anthropic', usage_available INTEGER NOT NULL DEFAULT 1, cost_available INTEGER NOT NULL DEFAULT 1)
[2026-08-01T03:43:20.604Z] text: The `usage_available` and `cost_available` columns are confirmed in the live running DB — migration 8 ran successfully.
Let me do one final check — verify the `EXO_USER_DATA_DIR` override unit tests actually tested the new code path by looking at what they assert:
[2026-08-01T03:43:21.319Z] tool#46: Bash
[2026-08-01T03:43:21.319Z] input: {"command":"grep -A 20 \"absolute override\\|relative override\\|EXO_USER_DATA_DIR\" tests/unit/data-dir.spec.ts | head -50","description":"Check data-dir test assertions"}
[2026-08-01T03:43:21.386Z] result#46:
* Behavior tests for the EXO_USER_DATA_DIR override (July 2026).
*
* The override is the only thing keeping packaged smoke tests out of the
* real install's data dir, so its two contracts — absolute path honored
* verbatim, relative path rejected loudly — get direct coverage. The
* override branch runs before any Electron access, so getDataDir() is
* testable under plain Node.
*/
test.describe("EXO_USER_DATA_DIR override", () => {
let saved: string | undefined;
test.beforeEach(() => {
saved = process.env.EXO_USER_DATA_DIR;
});
test.afterEach(() => {
if (saved === undefined) delete process.env.EXO_USER_DATA_DIR;
else process.env.EXO_USER_DATA_DIR = saved;
});
test("absolute override is returned verbatim, in any mode", async () => {
process.env.EXO_USER_DATA_DIR = "/tmp/exo-override-test";
const { getDataDir } = await import("../../src/main/data-dir");
expect(getDataDir()).toBe("/tmp/exo-override-test");
});
test("relative override fails loudly", async () => {
process.env.EXO_USER_DATA_DIR = "relative/scratch-dir";
const { getDataDir } = await import("../../src/main/data-dir");
expect(() => getDataDir()).toThrow(/absolute/);
});
});
[2026-08-01T03:43:41.001Z] text: All the evidence is collected. Let me summarize what was verified:
**Classification: Category A (New runtime/backend) + Category C (safety fix) + Category D (new IPC handler)**
**Evidence collected:**
1. **`listOpenCodeModels()` IPC end-to-end**: Called `window.api.settings.listOpenCodeModels()` directly and received `success: true` with 300+ model entries from OpenAI, OpenCode Zen, and OpenRouter providers — proving the new IPC handler, OpenCode inference service, and model catalog fetching all work correctly.
2. **DB migration 8 ran**: Verified `usage_available` and `cost_available` columns exist in the live `.dev-data/data/exo.db` `llm_calls` table — migration 8 ("add_llm_calls_accounting_availability") executed correctly.
3. **Unit tests pass**:
- `data-dir.spec.ts` (3/3): `EXO_USER_DATA_DIR` absolute override works, relative path fails loudly
- `no-global-data-dirs.spec.ts` (2/2): Pattern guards pass, no global dir references in scripts/tests
- `opencode-binary-resolution.spec.ts` (3/3): packaged binary path resolution works
- `hostler-agent-provider/event-mapper/sync.spec.ts` (42/42): All hostler provider tests pass
- `opencode-resolve-route.spec.ts` (6/6): OpenCode model selector resolution logic works
4. **Safety fix verified**: `run-tests.sh` `clean_test_dbs()` now targets only `$PROJECT_DIR/.dev-data/` paths, not `$HOME`-anchored global per-user dirs.
5. **No anomalies**: No console errors. The app is unauthenticated (setup wizard) in this worktree, which prevents testing the full email flow, but the core new backend IPC is verified working.
**Limitation**: The full OpenCode-as-LLM-provider flow (using OpenCode to generate email drafts/analyses) and Hostler agent provider (cloud sandbox sessions) cannot be exercised without a signed-in email account and OpenCode configured with credentials. The `listOpenCodeModels` call worked because it queries the OpenCode catalog endpoint without needing user-level auth, but an actual `createMessage()` through OpenCode would need credentials configured.
{"verdict":"pass","summary":"category=A. This PR adds OpenCode as a per-feature LLM provider, a Hostler cloud agent provider, and an EXO_USER_DATA_DIR safety override. Verification: (1) Directly invoked window.api.settings.listOpenCodeModels() and received success=true with 300+ model entries from OpenAI, OpenCode Zen, and OpenRouter — proving the new IPC handler, OpenCodeInferenceService, and catalog endpoint are wired end-to-end. (2) Confirmed migration 8 (usage_available + cost_available columns) applied to the live .dev-data/exo.db. (3) Ran unit tests: data-dir (3/3), no-global-data-dirs (2/2), opencode-binary-resolution (3/3), hostler provider suite (42/42), opencode-resolve-route (6/6) — all pass. (4) Verified run-tests.sh clean_test_dbs() now targets only project-local .dev-data/ paths. The app is unauthenticated in this worktree (credentials missing), so draft/analysis flows through OpenCode could not be driven end-to-end, but the primary new code path (IPC handler → inference service → catalog) is confirmed operational with live data.","anomalies":[],"actions_taken":28}
[2026-08-01T03:43:41.087Z] result: subtype=success cost=1.23785175 turns=47
This comment is upserted by |
Summary
Verification
Pre-PR verdict: PASS
fulleed8110