-
Notifications
You must be signed in to change notification settings - Fork 3.4k
feat(core): add pluggable core runtime and fleet host #4791
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
senamakel
merged 29 commits into
tinyhumansai:main
from
senamakel:docs/pluggable-core-plan
Jul 12, 2026
Merged
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
ee32d2a
docs(plans): pluggable-core — analysis, target architecture, phase index
senamakel e2b09a6
docs(plans): pluggable-core phases 0-1 — de-fusion and CoreBuilder/Se…
senamakel 1fdce74
docs(plans): pluggable-core phases 2-4 — CoreContext/store traits, mu…
senamakel e27db7b
docs(plans): prettier-format pluggable-core docs
senamakel dae889c
refactor(core): Phase 0 de-fusion — extract service spawns + store in…
senamakel bb5391a
feat(core): Phase 1 — CoreBuilder/CoreRuntime/ServiceSet embeddable API
senamakel 7b97611
refactor(core): Phase 2 Stage A — CoreContext owns the init sequence
senamakel a9b51fa
feat(core): Phase 2 Stage B + registry collapse — ambient CoreContext…
senamakel 2ef1219
feat(fleet): Phase 4 MVP — openhuman-fleet process-per-user superviso…
senamakel aeb45b2
feat(core): Phase 2 Stage C / Phase 3 — first per-context store (peop…
senamakel ddafc44
fix(dependencies): downgrade windows-sys to version 0.60.2 across mul…
senamakel 8b5b2dc
chore(fleet): clean up prototype clippy warnings
senamakel 87dd04f
fix(core): address pluggable runtime review feedback
senamakel f574fe8
fix(fleet): authenticate tenant readiness probes
senamakel 268bcb6
fix(fleet): require secure token handoff
senamakel 8fc3874
fix(core): gate bootstrap background jobs
senamakel 9d699e6
fix(core): preserve degraded startup context
senamakel fbb1660
test(memory): remove global workspace order race
senamakel 2d464bb
fix(fleet): isolate headless tenant requests
senamakel 8a1df45
fix(fleet): harden tenant readiness proxy
senamakel d1f4834
fix(core): isolate bootstrap service gates
senamakel 72dd3ff
fix(fleet): separate proxy and credential safety
senamakel 81aa92d
Merge remote-tracking branch 'upstream/main' into docs/pluggable-core…
senamakel 7cf59cb
test(memory): use tool memory host adapter
senamakel 58c2a2d
fix(core): split bootstrap service gates
senamakel 4eba1e6
fix(fleet): harden proxy edge surfaces
senamakel 79dd32f
fix(fleet): isolate tenant action dirs
senamakel cbf6951
fix(core): defer runtime service spawns
senamakel 0a9616e
fix(core): rebind context workspace on user switch
senamakel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # Phase 0 — De-fusion cleanup | ||
|
|
||
| **Status:** planned. | ||
| **Goal:** pure motion inside `run_server_inner` so phase 1 is a composition | ||
| change, not a untangling change. No behavior change; verified by diffing | ||
| startup log lines. | ||
|
|
||
| ## Scope | ||
|
|
||
| ### 0.1 Delete the dead dispatch tier | ||
|
|
||
| `src/rpc/dispatch.rs:10-15` always returns `None` (module doc says the | ||
| registry is authoritative). Remove the module and its tier-3 call site in | ||
| `src/core/dispatch.rs::dispatch`. `src/rpc/` keeps `structured_error.rs` | ||
| (still used) — only the dispatch shim dies. | ||
|
|
||
| ### 0.2 Extract inline service spawns | ||
|
|
||
| Each inline `tokio::spawn` block in `run_server_inner` becomes a named | ||
| function in a new `src/core/runtime/services.rs`: | ||
|
|
||
| | Service | Today (approx.) | Extracted fn | | ||
| | --------------------------------------------------------------------------- | ----------------------- | ----------------------------------------- | | ||
| | Heartbeat + subconscious bootstrap | `jsonrpc.rs:~2083-2110` | `spawn_heartbeat_service(cancel, config)` | | ||
| | Update scheduler | `:~2113` | `spawn_update_scheduler(cancel)` | | ||
| | Cron scheduler (+ proactive-agent seeding, flow schedule-trigger reconcile) | `:~2124-2160` | `spawn_cron_service(cancel, config)` | | ||
| | Channel listeners | `:~2162-2191` | `spawn_channels_service(cancel, config)` | | ||
|
|
||
| Rules: | ||
|
|
||
| - Every extracted fn takes the `CancellationToken` explicitly (today some | ||
| blocks capture it, some don't — normalize). | ||
| - Existing gates stay _inside_ the fns for now (`config.cron.enabled`, | ||
| `config.heartbeat.enabled`, `OPENHUMAN_DISABLE_CHANNEL_LISTENERS`, | ||
| `has_listening_integrations()`), so call sites don't change semantics. | ||
| Phase 1 lifts the _selection_ (should this service exist) to `ServiceSet` | ||
| while the fns keep their _config_ gates (is it enabled for this user). | ||
| - Keep the deliberate asymmetry documented at `jsonrpc.rs:2259-2264`: the | ||
| flows trigger subscriber registers at unconditional core boot, not under | ||
| channels — that stays in `bootstrap_core_runtime`, not in | ||
| `spawn_channels_service`. | ||
|
|
||
| ### 0.3 Extract the store-init block | ||
|
|
||
| `jsonrpc.rs:1807-1892` (memory, whatsapp_data, people, attachments global | ||
| inits) → `fn init_stores(config: &Config, workspace_dir: &Path) -> | ||
| anyhow::Result<()>` in `src/core/runtime/context.rs` (file seeded here, | ||
| grows in phase 1/2). Preserve init order exactly; comment each step with the | ||
| prior `jsonrpc.rs` line range so the diff is auditable. | ||
|
|
||
| ## Out of scope | ||
|
|
||
| - Any signature or behavior change to the spawned services themselves. | ||
| - Touching `bootstrap_core_runtime` internals (phase 1 moves the call, phase | ||
| 2/3 decompose it). | ||
|
|
||
| ## Verification | ||
|
|
||
| - `cargo check` both crates; `pnpm test:rust`. | ||
| - Boot the desktop app and `openhuman-core serve`; diff the startup log | ||
| sequence (grep-stable prefixes) against `main` — must be identical. | ||
| - `rg 'rpc::try_dispatch|rpc/dispatch'` returns nothing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| # Phase 1 — `CoreBuilder` / `CoreRuntime` / `ServiceSet` | ||
|
|
||
| **Status:** planned. | ||
| **Goal:** split bootstrap (context init) from transport (HTTP is just another | ||
| service) behind a public builder API; port all existing entry points onto it. | ||
| This phase alone delivers the headline goal: embeddable library, programmatic | ||
| harness entry, thin CLI/Tauri. | ||
|
|
||
| ## New modules | ||
|
|
||
| ```text | ||
| src/core/runtime/ | ||
| ├── mod.rs # pub use CoreBuilder, CoreRuntime, ServiceSet, TokenSource | ||
| ├── builder.rs # CoreBuilder — validation + build() | ||
| ├── context.rs # CoreContext (Stage A facade — owns init order; see phase 2) | ||
| └── services.rs # from phase 0; gains spawn_rpc_http_service, spawn_socketio wiring | ||
| ``` | ||
|
|
||
| Public API per README §2.1. Additional decisions: | ||
|
|
||
| - **`TokenSource`**: `Fixed(Arc<String>)` (Tauri in-memory handoff), | ||
| `EnvOrFile` (read `OPENHUMAN_CORE_TOKEN` when present, otherwise generate | ||
| and write the standalone `{root}/core.token` fallback 0600). `build()` seeds | ||
| `auth::init_rpc_token*` exactly once, same precedence as today | ||
| (`src/core/auth.rs`). | ||
| - **`build()` is init-only**: no sockets and no detached jobs for | ||
| `ServiceSet::none()` / `ServiceSet::headless_api()`. It runs, in | ||
| order: controller registration (`all::all_registered_controllers`), master | ||
| key init, token seeding, `Config::load_or_init`, `init_stores` (phase 0), | ||
| `bootstrap_core_runtime(host_kind, services)` for pure registration plus | ||
| ServiceSet-gated legacy bootstrap jobs. Init-order regressions are the top | ||
| risk; add a startup-sequence integration test asserting the order via log | ||
| markers. | ||
| - **`serve()`** spawns only the services selected by `ServiceSet`. The HTTP | ||
| listener (bind, port fallback, router from `build_core_http_router`, | ||
| `axum::serve`) moves into `spawn_rpc_http_service`; **the | ||
| `set_var OPENHUMAN_CORE_RPC_URL` call keeps its exact timing** (post-bind, | ||
| `jsonrpc.rs:2010`) inside that service — child tools depend on it. It is | ||
| flagged in the drift ledger as single-runtime-only. | ||
| - **Ready signal**: `EmbeddedReadySignal` (port-fallback reporting) is kept | ||
| type-identical, relocated; the readiness sender is supplied to | ||
| `CoreRuntime::serve` and fires after bind (when `rpc_http` selected). | ||
| - **`invoke()`** delegates to the existing `invoke_method` | ||
| (`jsonrpc.rs:230`) — same tiered dispatch, no second path. | ||
|
|
||
| ## `AgentRuntime` (harness-as-library) | ||
|
|
||
| `CoreRuntime::agent_runtime()` returns the slice needed to run turns with | ||
| zero ports bound: | ||
|
|
||
| ```rust | ||
| pub struct AgentRuntime { ctx: Arc<CoreContext> } | ||
| impl AgentRuntime { | ||
| pub fn agent(&self, sel: AgentSelector) -> anyhow::Result<Agent>; // Agent::from_config | ||
| pub async fn run_turn(&self, agent: &Agent, input: TurnInput) -> …; // tinyagents seam | ||
| pub fn events(&self) -> broadcast::Receiver<CoreEvent>; | ||
| } | ||
| ``` | ||
|
|
||
| Its required bootstrap subset _defines_ what `ServiceSet::none()` must still | ||
| initialize: config, workspace, master key, event bus, | ||
| `AgentDefinitionRegistry`, memory/tinycortex stores, cost/x402 ledgers, | ||
| security live-policy + approval gate, tool registry. Explicitly not required: | ||
| HTTP, socket.io, cron, channels, heartbeat, update scheduler. | ||
|
|
||
| **Acceptance test for the whole phase:** an integration test (and | ||
| `examples/run_turn.rs`) that builds with `ServiceSet::none()`, runs one agent | ||
| turn through `run_turn_via_tinyagents_shared`, and asserts no listener was | ||
| bound. | ||
|
|
||
| ## Porting the consumers | ||
|
|
||
| | Consumer | Change | | ||
| | -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | `run_server` / `run_server_embedded*` (`jsonrpc.rs:1682-1737`) | become `#[deprecated]` shims over the builder; deleted one release later | | ||
| | Tauri (`app/src-tauri/src/core_process.rs:289`) | `CoreProcessHandle::ensure_running` builds `CoreBuilder::new(HostKind::TauriShell).token(TokenSource::Fixed(..)).services(ServiceSet::desktop())`, then passes readiness to `CoreRuntime::serve`; `CancellationToken` / restart / port-takeover logic unchanged | | ||
| | CLI `run`/`serve` (`src/core/cli.rs:66`) | maps flags → `ServiceSet` (`--jsonrpc-only` → `socketio: false`) | | ||
| | CLI `call` + namespace dispatch (`cli.rs:354,435`) | `ServiceSet::none()` build → `runtime.invoke()`; one-shot calls stop constructing server state | | ||
| | MCP stdio (`cli.rs` `mcp`) | adapter over `runtime.invoke()` | | ||
| | `src/lib.rs` | re-export the new surface; keep `run_core_from_args` | | ||
|
|
||
| Plus `examples/embed_headless.rs` (build + `serve()` with | ||
| `ServiceSet::headless_api()`, call `openhuman.ping` over HTTP) and | ||
| `examples/run_turn.rs` (above). | ||
|
|
||
| ## Risks & mitigations | ||
|
|
||
| | Risk | Mitigation | | ||
| | ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | | ||
| | Init-order regression (master key before stores; policy before approval gate) | single `CoreContext::init` encoding the order, line-ref comments, startup-sequence test | | ||
| | Tauri port-fallback / ready semantics drift | `EmbeddedReadySignal` unchanged; e2e boot test in CI Full already covers | | ||
| | `set_var` timing change breaks child tools | timing preserved inside `spawn_rpc_http_service`; grep test for env presence after ready | | ||
| | Double-bootstrap when shims + builder both run in tests | builder guards with the same idempotent `Once`s during this phase (removed in phase 3) | | ||
|
|
||
| ## Verification | ||
|
|
||
| - `pnpm test:rust`, `bash scripts/test-rust-with-mock.sh --test json_rpc_e2e`. | ||
| - Both examples run green in CI. | ||
| - Desktop app boots via ported `core_process.rs` (CI Full e2e matrix). | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.