Stage-Runtime mapping — runtime_profile.{interview,execute,evaluate,reflect}

[shaun0927]

Stage-Runtime mapping — runtime_profile.{interview,execute,evaluate,reflect}

Context

The Agent OS architecture diagram shared by maintainers assigns a
different harness per pipeline stage:

• Interview → Codex
• AC Tree / Execute → OMX
• Evaluate → Claude Code (CC)
• Wonder / Reflect → Hermes

PR #505 introduced an opt-in orchestrator.runtime_profile setting but
ships only the Codex worker mapping for Phase 1; OpenCode, Hermes,
Claude Code, and LiteLLM mappings are explicitly deferred ("matching the
staging in the design discussion on #488").

This issue extends runtime_profile from a single backend slot to a
per-stage map, so the diagram's stage→harness assignment becomes a
real configuration surface. It is the first concrete consumer of the Mesh
RFC (#511) at the single-installation layer (S3 — Multi-harness Parity in
RFC #476).

This is not the Mesh itself (cross-runtime IPC). It is the binding
table the Mesh and the orchestrator both read.

Scope

1. Configuration surface

   [orchestrator.runtime_profile]
   default = "claude_code"
   
   [orchestrator.runtime_profile.stages]
   interview = "codex"
   execute   = "opencode"     # OMX in the diagram
   evaluate  = "claude_code"
   reflect   = "hermes"

   • default keeps today's single-backend behavior for installs that
     don't opt in.
   • stages.* is additive: any unset stage falls back to default.

2. Stage enumeration

   • Initial stages: interview, execute, evaluate, reflect.
   • The list is closed — adding a new stage requires a justified PR (per
     the same narrow-membership commitment as AgentRuntimeContext).

3. Resolution rule

   • At workflow entry, the orchestrator resolves the active stage and
     selects the runtime via runtime_profile.stages.get(stage, default).
   • The chosen runtime is recorded on the AgentProcess (M6) as
     runtime_id, and emitted in control.directive.emitted as part of
     the extra payload so the journal answers "which harness ran which
     stage?" without log inspection.

4. Per-runtime mapping follow-ups

   • PR feat(orchestrator): Agent OS runtime_profile (Codex backend, supersedes #488) #505 already covers Codex worker profile mapping; this issue's
     PRs add four parallel slices:
     • feat(runtime_profile): add OpenCode mapping
     • feat(runtime_profile): add Hermes mapping
     • feat(runtime_profile): add Claude Code mapping
     • feat(runtime_profile): add LiteLLM mapping
   • Each slice is reviewable independently.

5. Validation

   • Setup-time validation: each referenced runtime must be installed and
     pass the existing ouroboros mcp doctor (feat(cli): add 'ouroboros mcp doctor' diagnostic command #445) check; otherwise the
     setting is rejected at startup with a clear error message rather
     than failing mid-workflow.
   • Diagnostics: ouroboros mcp doctor extends to print the resolved
     stage→runtime table for the current install.

6. Backward compatibility

   • Default runtime_profile=None preserves today's behavior exactly
     (same guarantee as feat(orchestrator): Agent OS runtime_profile (Codex backend, supersedes #488) #505).
   • Existing _build_command paths unchanged unless a stage is
     explicitly mapped to a different runtime.

Non-goals

• The Mesh / cross-runtime IPC envelope itself (sub-RFC of [RFC] Agent OS runtime contract: capabilities, policy, directives, and replayable agent processes #476, separate
  issue [RFC] MCP Mesh wire format & Coordinator handshake (sub-RFC of #476) #511).
• Per-AC runtime selection (this issue is per-stage; per-AC is a
  Tier-3 enhancement, gated by evidence).
• Dynamic re-routing mid-run (declarative configuration only).
• Cost-aware runtime selection (Tier-3 C3).

Acceptance

• runtime_profile.stages config surface defined and documented
• Stage enumeration closed with rationale
• Orchestrator resolves runtime per stage and records it on
  AgentProcess
• control.directive.emitted payload includes the resolved
  runtime_id for stage-relevant transitions
• Four follow-up PRs (OpenCode / Hermes / Claude Code / LiteLLM
  mappings) merged or referenced as separate issues
• ouroboros mcp doctor prints the active stage→runtime table
• Default runtime_profile=None behavior is unchanged
• Replay test: a fixture run with mixed stage runtimes shows the
  correct runtime_id per stage in the EventStore

Dependencies

• Builds on: feat(orchestrator): Agent OS runtime_profile (Codex backend, supersedes #488) #505 (Codex mapping, Phase 1), feat(cli): add 'ouroboros mcp doctor' diagnostic command #445 (mcp doctor)
• Coordinates with: [RFC] MCP Mesh wire format & Coordinator handshake (sub-RFC of #476) #511 (Mesh RFC — Mesh consumes this binding table),
  Introduce ControlBus on AgentRuntimeContext for control.directive.emitted #515 (ControlBus — emits the resolved runtime), M6 AgentProcess issue
  (carries runtime_id)
• Aligns with: RFC [RFC] Agent OS runtime contract: capabilities, policy, directives, and replayable agent processes #476 M4 (Capability Source of Truth) + S3 (Multi-harness
  Parity)
• 4-verb filter: strengthens compose

[ouroboros-agent]

This looks like a multi-PR effort that needs design discussion before implementation. Labeling needs-design for human planning.

Scope assessment:

• Affects configuration schema in src/ouroboros/config/models.py for orchestrator.runtime_profile and closed stage enumeration
• Affects orchestrator runtime resolution and persistence flow in src/ouroboros/orchestrator/runner.py, src/ouroboros/orchestrator/adapter.py, and related session/event paths
• Affects observability and diagnostics in src/ouroboros/orchestrator/events.py, EventStore-backed replay coverage, and ouroboros mcp doctor output
• Depends on adjacent work called out in the issue itself (#505, #445, #511, #515, and M6 runtime tracking), and the acceptance list explicitly references separate follow-up PR slices

Estimated complexity:

• Architectural, cross-cutting change rather than a single isolated fix
• Requires design agreement on stage taxonomy, fallback semantics, validation boundaries, and event contract changes before implementation

Suggested approach:

• Break this into a design-approved parent issue or spec update, then land narrowly-scoped follow-up PRs for config schema, runtime resolution/event emission, doctor diagnostics, and per-runtime mappings

ouroboros-agent[bot]

[shaun0927]

Sub-thread — stage enumeration and validation depth

The issue body locks the configuration surface and the relationship to #505. Two smaller decisions need to be settled before the per-runtime mapping PRs ship.

Decision A — Stage enumeration

The Agent-OS diagram suggests four stages: interview · execute · evaluate · reflect. Proposing this set as the closed initial vocabulary, with a stated rule for adding more.

[orchestrator.runtime_profile.stages]
interview = "codex"
execute   = "opencode"
evaluate  = "claude_code"
reflect   = "hermes"

Closed-list rule:

• The four stages above are the only valid keys in the stages table for now.
• Adding a new stage requires a PR that (a) names the stage, (b) documents which workflow phase it covers, (c) adds it to the Stage enum in code, and (d) justifies why an existing stage cannot host the work.
• Same narrow-membership commitment that AgentRuntimeContext has under [RFC] Agent OS runtime contract: capabilities, policy, directives, and replayable agent processes #476 Q1.

This stops the table from sprawling into per-handler entries (e.g., qa_judge, unstuck) which belong inside an AgentProcess (#518), not in the binding table.

Decision B — Validation depth at startup

When ouroboros mcp doctor (#445) inspects a runtime_profile.stages config, three depths are possible:

Depth	What is checked	Cost
Shallow	Each referenced runtime name is in the runtime registry	<50ms
Medium (proposed)	Shallow + each runtime's binary/path is reachable on $PATH (or matches the configured override)	a few hundred ms
Deep	Medium + a smoke RPC call (a ping-style directive) to confirm the runtime actually responds	seconds, requires network/IPC

Proposed: Medium at startup; Deep only when invoked explicitly via ouroboros mcp doctor --probe.

Reasoning:

• Medium catches the common failure modes (typo in runtime name, missing binary) without burning seconds at every startup.
• Deep is valuable but only when a user is explicitly debugging — not on every cold start, where it would slow non-MCP workflows.
• The diagnostic command's output should print the resolved stage→runtime table either way so users can read off the active configuration.

Decision C — Default fallback behavior

If a stage is unset in stages, fall back to runtime_profile.default. If default is also unset, fall back to today's hard-coded selection. This preserves the #505 commitment that runtime_profile=None is byte-for-byte today's behavior.

Risks

1. Stage enum sprawl — guarded by the closed-list rule above.
2. Per-AC runtime selection demand — explicitly Tier-3 territory. Users who need it should file an issue with usage evidence; the binding table stays per-stage in 1.0.
3. Provider not installed — Medium validation produces a clear error, not a mid-run failure. ouroboros mcp doctor prints the table.
4. stages typo silently mapped to default — no, because Medium validation rejects unknown keys at startup. A typo in interveiw fails fast.

Sign-off

• A — Closed initial enum: interview / execute / evaluate / reflect. Adding a new stage = explicit PR per the narrow-membership rule.
• B — Medium validation at startup; Deep probe only via ouroboros mcp doctor --probe.
• C — Fallback chain: stages.<stage> → runtime_profile.default → today's hard-coded selection.

If these land, the four per-runtime mapping PRs (OpenCode / Hermes / Claude Code / LiteLLM) can be written against a fixed binding contract.

[shaun0927]

Verification protocol — implementation PR

This PR extends runtime_profile from a single backend slot to a per-stage map (interview / execute / evaluate / reflect). It is the first concrete consumer of #511 at the single-installation layer (S3 multi-harness parity). Four follow-up PRs add per-runtime mappings (OpenCode / Hermes / Claude Code / LiteLLM).

Implementation steps

1. Define a closed Stage enum in orchestrator/:

   class Stage(StrEnum):
       INTERVIEW = "interview"
       EXECUTE   = "execute"
       EVALUATE  = "evaluate"
       REFLECT   = "reflect"

   Adding a new stage = explicit PR per the narrow-membership rule (sub-thread decision A).
2. Extend runtime_profile config:

   [orchestrator.runtime_profile]
   default = "claude_code"
   [orchestrator.runtime_profile.stages]
   interview = "codex"
   execute   = "opencode"
   evaluate  = "claude_code"
   reflect   = "hermes"

3. Implement resolution: runtime_profile.stages.get(stage, default) → falls back to today's hard-coded selection if both unset (feat(orchestrator): Agent OS runtime_profile (Codex backend, supersedes #488) #505 byte-for-byte commitment).
4. Record the resolved runtime_id on the AgentProcess (M6 AgentProcess lifecycle (lite) — spawn/pause/resume/cancel/replay #518) and emit it in control.directive.emitted extra payload at stage transitions.
5. Extend ouroboros mcp doctor (feat(cli): add 'ouroboros mcp doctor' diagnostic command #445) with Medium validation (sub-thread decision B):
   • Each referenced runtime name is in the runtime registry
   • The binary/path is reachable on $PATH (or matches the configured override)
   • Print the resolved stage→runtime table
6. Add --probe flag to ouroboros mcp doctor for Deep validation (a smoke RPC call per runtime).
7. Reject invalid stages keys at startup with a clear error message (no silent fallback to default for typos).
8. Tests as below.

Pre-merge verification

Unit

• Stage enum has exactly four members; Stage("interview") works; Stage("interveiw") raises
• Resolution rule: stages.<stage> → default → today's hard-coded selection
• runtime_profile=None (no opt-in) preserves today's behavior byte-for-byte (assert against a behavior fixture from feat(orchestrator): Agent OS runtime_profile (Codex backend, supersedes #488) #505)
• Unknown key in stages → startup raises with a clear error message naming the typo and the valid set

Validation

• Medium validation rejects a config that references a runtime not in the registry
• Medium validation rejects a config whose runtime binary is not on $PATH
• Medium validation latency < 500 ms on a four-stage config (the budget from sub-thread B)
• Deep probe (mcp doctor --probe) issues one ping per stage and reports per-stage round-trip time

Integration

• On a fixture that maps evaluate to a different runtime than default, AgentProcess (M6 AgentProcess lifecycle (lite) — spawn/pause/resume/cancel/replay #518) records the correct runtime_id for that stage
• control.directive.emitted events for stage transitions carry the resolved runtime_id in extra
• ouroboros mcp doctor output includes the resolved stage→runtime table in a stable, parseable format

Post-merge verification

• On a real install, set [orchestrator.runtime_profile.stages] evaluate = "claude_code" and run a fixture; verify via the EventStore that the evaluate stage's control.directive.emitted records runtime_id="claude_code"
• ouroboros mcp doctor on a fresh install with the default runtime_profile=None produces the same output as before this PR (byte-for-byte BC)
• Operator can switch a stage's runtime in the config and the next workflow run picks it up at startup (no restart of unrelated services)

Rollback plan

runtime_profile=None is the safety net: rolling back any mis-configuration is a single config edit; rolling back the code is reverting the PR. Rollback steps:

1. Revert the PR. Configs with [orchestrator.runtime_profile.stages] start failing at startup if the schema is no longer recognized — operators must remove the stages block.
2. To avoid the failure window, pre-revert workflow: ship a release that ignores unknown stages keys with a warning (compatibility shim), then revert the resolution code in a follow-up. Choose this path if rollback is anticipated; otherwise straight revert is acceptable.
3. AgentProcess (M6 AgentProcess lifecycle (lite) — spawn/pause/resume/cancel/replay #518) returns to using default only.
4. No data migration to undo.

Follow-up PRs (out of scope for this PR's verification)

Each of the four runtime-specific mapping PRs has its own verification scope:

• OpenCode mapping
• Hermes mapping
• Claude Code mapping
• LiteLLM mapping

Tracking entries linked from this issue.