Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,9 @@ tinyjuice = { version = "0.1", default-features = false }
# aligned to 0.40, avoiding duplicate `links = "sqlite3"` native bindings.
# Durable graph checkpoints still use `SqlRunLedgerCheckpointer` until the
# migration re-points those rows to the crate checkpointer.
tinyagents = { version = "1.5.0", features = ["sqlite"] }
# The `repl` feature adds the embedded Rhai `.ragsh` session runtime powering
# the `rlm` language-workflow tool (`src/openhuman/rlm/`).
tinyagents = { version = "1.5.0", features = ["sqlite", "repl"] }
# TinyCortex — Rust core for the memory engine (store/chunks/tree/retrieval/
# queue/ingest/score + long tail), vendored as a git submodule and patched
# below to `vendor/tinycortex`. OpenHuman's memory subsystem migrates onto this
Expand Down
19 changes: 17 additions & 2 deletions gitbooks/developing/architecture/agent-harness.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,12 +49,12 @@ icon: layer-group

## TinyAgents crate: features & compatibility

OpenHuman pins `tinyagents = { version = "1.5.0", features = ["sqlite"] }` (see [`Cargo.toml`](../../../Cargo.toml)). The rationale, so future upgrades don't silently regress it:
OpenHuman pins `tinyagents = { version = "1.5.0", features = ["sqlite", "repl"] }` (see [`Cargo.toml`](../../../Cargo.toml)). The rationale, so future upgrades don't silently regress it:

- **OpenHuman-owned providers only.** We do **not** enable any bundled provider feature. OpenHuman owns provider transport, credentials, OAuth, and billing classification, so the live model is always OpenHuman's `Provider` wrapped as [`ProviderModel`](../../../src/openhuman/tinyagents/model.rs), never an SDK-owned provider client. The `ChatModel` adapter is the seam that replaces feature-gated SDK providers.
- **`sqlite` feature enabled with one native sqlite chain.** OpenHuman's root and Tauri Cargo worlds pin `rusqlite = "=0.40.0"` and patch `rusqlite` / `libsqlite3-sys` locally to avoid the upstream `cfg_select!` build break on the current toolchain. Both worlds resolve to a single `libsqlite3-sys v0.38.0` chain. Durable graph checkpoints still run through [`SqlRunLedgerCheckpointer`](../../../src/openhuman/tinyagents/checkpoint.rs) until the migration re-points those rows to the crate checkpointer.
- **WhatsApp Web storage bridge.** `whatsapp-rust`'s Diesel-backed `sqlite-storage` feature links sqlite separately from rusqlite 0.40, so the optional `whatsapp-web` feature currently builds against `wacore::store::InMemoryBackend` and logs that sessions are not durable. A rusqlite-backed durable WhatsApp store is required before treating Web sessions as persistent again.
- **`repl` / expressive-language features unused.** OpenHuman drives graphs from Rust (`GraphBuilder`), not the crate's `.rag` REPL language.
- **`repl` feature enabled for language workflows; `.rag` expressive language unused.** OpenHuman still drives *graphs* from Rust (`GraphBuilder`), not the declarative `.rag` language. But the `repl` feature (the imperative Rhai `.ragsh` session runtime) is enabled to power the `rlm` language-workflow tool ([`openhuman::rlm`](../../../src/openhuman/rlm/README.md), see "Language workflows (`rlm`)" below).
- **Adapter map (feature-gated SDK piece → OpenHuman replacement):** provider clients → `ProviderModel`; crate SQLite checkpointer rows not yet adopted → `SqlRunLedgerCheckpointer`; task/status stores not yet controller-canonical → OpenHuman SQL/JSON run ledgers (`running_subagents`, `workflow_runs`, `agent_teams`, `command_center`). The generic harness/graph/middleware/event primitives are used as-is.

The agent harness is the runtime that turns a user message (or a webhook fire, or a cron tick) into a complete, tool-using LLM interaction. It owns the tool-call loop, sub-agent dispatch, the trigger-triage pipeline, and the hook surface around them. It does **not** own provider HTTP transport, tool implementations, prompt-section assembly, or memory storage - those are separate domains the harness composes.
Expand Down Expand Up @@ -282,6 +282,21 @@ Each `AgentDefinition` carries an `agent_tier` field (`chat` / `reasoning` / `wo

For Composio toolkits with hundreds of actions (GitHub alone has 500+), loading every action into the sub-agent's tool set balloons prompt size. The harness ranks the toolkit's actions against the parent-refined task prompt with a cheap CPU-only filter (verb detection, token overlap, verb-alignment boost) and only loads the top-ranked subset into the sub-agent. No model call, pure heuristic - fast and explainable.

## Language workflows (`rlm`)

The fixed delegation primitives (`spawn_subagent`, `spawn_parallel_agents`, `run_workflow`) can't express *ad-hoc control flow* — "spawn N readers, dedupe their findings, verify each survivor with 3 refuters, loop until dry". The **`rlm` tool** closes that gap: it exposes TinyAgents' Rhai-backed `.ragsh` REPL (the `repl` cargo feature) so the orchestrator can author and run its own workflow scripts.

**One tool call = one `eval_cell`.** The orchestrator's normal tool-call loop *is* the CodeAct driver loop: the model writes a Rhai cell, the cell runs against a persistent per-session namespace (top-level `let` bindings survive into the next cell via an optional `session_id`), and the structured result flows back as the tool result. Scripts reach the host only through capability functions — `tool_call`, `agent_query`, `model_query`, their `*_batched` fan-out variants, `emit`, and `answer`.

The domain lives in [`src/openhuman/rlm/`](../../../src/openhuman/rlm/README.md):

- **`policy.rs`** maps the autonomy tier + `tool_timeout` clamps onto a `tinyagents::ReplPolicy` (always bounded, never unbounded; `readonly` refused; `full` may raise call-count limits to a hard 2× ceiling).
- **`bridge.rs`** builds the `CapabilityRegistry`: the parent's visible tools (each re-wrapped so the **approval gate runs in the bridge** — it is *not* on the repl path, which bypasses the harness `wrap_tool` middleware), the turn's provider model, and a sub-agent capability per `allowed_subagent_ids`. Recursion/duplication hazards (`rlm`, `spawn_*`, workflow tools, `CliRpcOnly`-scoped tools) are excluded. Because `eval_cell` runs on `spawn_blocking` + `block_on`, the `agent_query` adapter re-installs the `PARENT_CONTEXT` task-local that `run_subagent` resolves.
- **`sessions.rs`** is a bounded (LRU + idle-TTL) manager of persistent sessions, one cell at a time (a concurrent call on a busy session returns a typed "busy" error).
- **`ops.rs`** runs the cell on `spawn_blocking` under a layered time bound (rhai `on_progress` deadline → `bridge_block_on` timer race → outer `tokio::timeout` backstop → harness `ToolTimeout`), wires the run-cancellation token to a fresh per-cell `ReplCancelFlag`, and maps every failure mode to a model-consumable result.

The tool is registered for the orchestrator on `supervised`/`full` tiers only, behind the `OPENHUMAN_RLM=0` kill switch. The TinyAgents-side host-embedding support (external cancellation, live capability events) landed in that crate's `repl` feature.

## Triage - handling external triggers

When a webhook fires, a cron ticks, or a Composio event arrives, the system can't just hand it straight to the orchestrator. Most triggers are noise; some warrant a notification; only a few deserve a full agent turn. The **trigger-triage pipeline** is the gate.
Expand Down
10 changes: 10 additions & 0 deletions src/openhuman/about_app/catalog_data.rs
Original file line number Diff line number Diff line change
Expand Up @@ -620,6 +620,16 @@ pub(super) const CAPABILITIES: &[Capability] = &[
status: CapabilityStatus::Beta,
privacy: DERIVED_TO_BACKEND,
},
Capability {
id: "intelligence.language_workflows",
name: "Language Workflows (RLM)",
domain: "intelligence",
category: CapabilityCategory::Intelligence,
description: "The orchestrator can author and run small Rhai workflow scripts to express ad-hoc control flow over delegated work — parallel fan-out, loops, and dedup-then-verify pipelines that fixed spawn/parallel primitives cannot. Each script runs bounded and fail-closed (per-cell timeout, per-session caps on tool/model/agent calls and recursion depth), and every effectful step still passes the same approval and permission gates as a direct tool call. Progress rides the existing tool-call timeline.",
how_to: "Runs automatically when the orchestrator chooses the `rlm` tool; disable with OPENHUMAN_RLM=0 or the read-only autonomy tier",
status: CapabilityStatus::Beta,
privacy: DERIVED_TO_BACKEND,
},
Capability {
id: "intelligence.agent_library",
name: "Agents Library",
Expand Down
3 changes: 2 additions & 1 deletion src/openhuman/agent_orchestration/tools/spawn_subagent.rs
Original file line number Diff line number Diff line change
Expand Up @@ -936,7 +936,8 @@ mod tests {

#[test]
fn build_worker_thread_title_collapses_whitespace_and_caps_length() {
let prompt = " draft\n a very long\tplan that\nrambles ".to_string() + &"x".repeat(200);
let prompt =
" draft\n a very long\tplan that\nrambles ".to_string() + "x".repeat(200).as_str();
let title = build_worker_thread_title(&prompt);
assert!(title.starts_with("draft a very long plan"));
assert!(title.chars().count() <= WORKER_THREAD_TITLE_MAX_CHARS + 1);
Expand Down
22 changes: 22 additions & 0 deletions src/openhuman/agent_registry/agents/orchestrator/prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,28 @@ their refs separate by `subagent_session_id` or `task_id` (`agentId` is only the
worker type), tick or wait on each independently, and synthesise only completed
outputs. Never fabricate a result for a worker that is still running or failed.

## Language workflows (rlm)

When a task needs **ad-hoc control flow** over delegated work — loops, conditionals, a
dedup-then-verify pipeline, "spawn N, filter, then verify each survivor with M checks" — that
the fixed `spawn_parallel_agents` / `delegate_*` primitives can't express, use the `rlm` tool.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add rlm to the orchestrator allowlist

This prompt teaches the orchestrator to call rlm, but the orchestrator uses a ToolScope::Named allowlist from src/openhuman/agent_registry/agents/orchestrator/agent.toml, and that list does not include rlm. The session builder filters prompt/provider tools through that non-empty allowlist before applying policy, so in normal chat sessions the model is instructed to use a tool that is not actually advertised or callable; add rlm to the orchestrator's named tools (or otherwise surface it through the scope) with this prompt change.

Useful? React with 👍 / 👎.

It evaluates a small **Rhai workflow cell** whose only side effects are capability calls:
`tool_call`, `agent_query`, `model_query`, and their `*_batched` fan-out variants (plus
`emit`/`answer`/`print`).

- **One call = one cell.** Top-level `let` bindings persist within a `session_id`, so pass the
same `session_id` back to continue a namespace across calls (`let findings = …` in cell 1,
reference it in cell 2). Omit `session_id` for a fresh session; set `close_session: true` when done.
- **Prefer `rlm`** over `spawn_parallel_agents` when you need iteration, branching, or a
reduce/verify step over results — not for a single delegation (use the matching `delegate_*`
or `spawn_subagent` for that).
- **It stays inside the gates.** Every effectful inner `tool_call` still hits the approval gate;
`agent_query` only reaches sub-agents already in your allowlist. `rlm` itself, `spawn_*`, and
workflow tools are not callable from a cell.
- **It is bounded and fail-closed.** Cells have a wall-clock timeout and per-session caps on
model/tool/agent calls and recursion depth. Exceeding one returns an error you can fix and
retry in the same session; the result reports `limits_remaining` so you can plan fan-out.

## Connecting external services

When the user asks to connect a service (Gmail, Notion, WhatsApp, Calendar, Drive, etc.) or a sub-agent reports `Connection error, try to authenticate`:
Expand Down
1 change: 1 addition & 0 deletions src/openhuman/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ pub mod provider_surfaces;
pub mod recall_calendar;
pub mod redirect_links;
pub mod referral;
pub mod rlm;
pub mod routing;
pub mod runtime_node;
pub mod runtime_python;
Expand Down
70 changes: 70 additions & 0 deletions src/openhuman/rlm/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# `rlm` — language-based workflows (Rhai `.ragsh` REPL)

Exposes TinyAgents' Rhai-backed `.ragsh` session runtime (the `repl` cargo
feature, `tinyagents::ReplSession`) as a first-class **`rlm` tool** so the
orchestrator model can author and execute its own workflow scripts — fan-out
over subagents, batched tool/model calls, loops, dedup/verify pipelines — and
run them deterministically, in the spirit of Claude Code Workflows and
Recursive Language Models.

One orchestrator tool call maps to **one `eval_cell`**: the model writes a Rhai
cell, the cell runs against a persistent per-session namespace (top-level `let`
bindings survive into the next cell), and the structured result flows back as
the tool result. The orchestrator's own turn loop *is* the CodeAct driver loop.

## Module shape

| File | Role |
| ---- | ---- |
| `mod.rs` | Exports only (no controller schemas in v1). |
| `types.rs` | `RlmSessionId`, `RlmEvalRequest`/`RlmEvalResponse`, `RlmLimitsOverride`, `RlmCallSummary`, serde types. |
| `policy.rs` | Maps openhuman autonomy tier + `tool_timeout` clamps → `tinyagents::ReplPolicy` (fail-closed, bounded). |
| `bridge.rs` | Builds the `CapabilityRegistry<()>`: openhuman tools (approval-gated, scope-filtered) + provider models + subagents. |
| `sessions.rs` | `RlmSessionManager`: LRU + idle-TTL bounded map of persistent `ReplSession`s, keyed `<thread>:<session_id>`, one cell at a time. |
| `ops.rs` | `eval_rlm_cell`: spawn_blocking + outer timeout, cancellation wiring, event forwarding, error → model-consumable result. |
| `tools.rs` | `RlmTool` (the `rlm` tool: schema, permission, scope, timeout, display). |

## Fail-closed guarantees

Every failure mode returns a **model-consumable tool result** — never a panic,
never a hung turn:

- **Layered time bounds:** (1) rhai `on_progress` deadline for pure script
loops; (2) `bridge_block_on` timer race for hung capability futures;
(3) an outer `tokio::time::timeout(policy.timeout + 5s)` around
`spawn_blocking`; (4) the harness `ToolTimeout::Secs` backstop above all of
them. The tool's own timeout is always set below the harness backstop.
- **Bounded sessions:** LRU cap (16) + idle TTL (30 min); a second concurrent
call on a busy session returns a typed "session busy" error rather than
deadlocking; a poisoned/errored session is dropped, never reused.
- **Bounded work per session:** `ReplPolicy` caps on operations, output bytes,
script bytes, and per-kind call counts. `full` tier may raise call-count
limits up to a hard 2× ceiling via the tool's `limits` arg; `readonly` tier
does not get the tool at all.
- **Cancellation end-to-end:** the turn's run-cancellation token drives a
`ReplCancelFlag` watcher, so a user cancel drops an in-flight cell (script or
capability call) promptly; the session is left resumable.

## Approval & security (bridged tools keep their own gates)

The RLM bridge restricts callable tools to the parent turn's
`visible_tool_names`, and **excludes** recursion/duplication hazards: `rlm`
itself, `spawn_subagent`/`spawn_parallel_agents` (use `agent_query` instead),
and `run_workflow`/`await_workflow`. `ToolScope::CliRpcOnly` tools are denied.

Approval gating is **not** on the tinyagents repl bridge path (it lives in the
harness `wrap_tool` middleware, which the REPL bypasses), so the bridge itself
invokes `ApprovalGate::intercept_audited` (+ `record_execution`) for any tool
whose `external_effect_with_args` is true, failing closed on denial.

## Capability surface exposed to scripts

`model_query`, `tool_call`, `agent_query`, their `*_batched` variants, `emit`,
and `answer`. Graph authoring/execution (`graph_*`) is **out of scope for v1**
(the REPL's `graph_run` returns a reference, not an execution).

## Kill switch & rollout

The tool is **not registered** when the autonomy tier is `readonly` or when
`OPENHUMAN_RLM=0`; default-on for `supervised`/`full`. Reverting the
registration line disables the surface without touching the domain.
Loading
Loading