v0.3 alpha: local file ingest + run chains

CHANGELOG.md

@@ -11,6 +11,33 @@ The format is loosely based on [Keep a Changelog](https://keepachangelog.com/).
 ## [Unreleased]
 
 ### Added
+- **v0.3 alpha — local file ingest + run chains.** `stacy brain create --file`
+  and `--dir`/`--glob`/`--ext` turn local text/markdown/json files into signed
+  Knowledge Objects with **no connector and no credentials** (leak-safe
+  cwd-relative source labels; dotfiles/`node_modules`/`.git`/symlinks/binary/
+  oversized files skipped). `stacy run --chain <spec.json>` composes multi-step
+  runs where a later step consumes an earlier step's `agent_output` via an
+  `@<stepId>` reference (validated before egress, gated once, one-hop
+  provenance, abort-on-step-failure). Together they make the whole ingest → run
+  → synthesize loop runnable offline. See
+  `docs/v0.3-files-and-chains-quickstart.md`.
+
+### Fixed
+- **Run-result caching now composes across chain steps** (verified live). Two
+  changes: (1) `agent_output` content no longer embeds a wall-clock
+  `generatedAt`; (2) the run cache keys inputs on their **content**, not on the
+  KO `contentHash` (which folds in `createdAt`, so it changes every time a KO is
+  created even for identical content). Without (2) a chain's downstream step —
+  whose input is the prior step's freshly-created output KO — missed the cache
+  on every run. Now an identical chain re-run makes **zero adapter calls**. KO
+  `contentHash` is still used for provenance; generation time still lives on the
+  KO record and the `run` receipt.
+
+### Changed
+- Extracted a reusable `runOnce` run executor (single-run verb and run chains
+  share it); single-run behavior is unchanged.
+
+### Added (v0.2 alpha)
 - **v0.2 alpha — connectors + AI runs.** GitHub connector (OAuth device-code
   flow, encrypted-at-rest token storage, sliding-window rate limiting), the
   `stacy connect` / `connectors list|status|disconnect` / `ingest` verbs that

TODOS.md

@@ -0,0 +1,93 @@
+# TODOS
+
+Deferred work, captured so it is not silently lost. Each item has enough context
+to pick up cold.
+
+## Later — Generic ingest refactor + `fs`/Linear as real Connectors
+
+- **What:** Refactor `verbs/ingest.ts` (currently `ingestGithubCommand`, GitHub-
+  specific) into a generic registry-driven `ingestCommand(connectorId, params)`,
+  and implement `fs` as a real `Connector` (with `authType: "none"`).
+- **Why deferred:** Eng review (Codex outside voice #12) showed this is premature
+  for the alpha. `brain create --file` (v0.3 W1) unblocks the zero-credential loop
+  without touching connector auth/registry/connect/status. The framework refactor
+  only pays off once a second OAuth connector (Linear) justifies it.
+- **Scope when revisited (holes the review found):** `authType: "none"` is a core
+  interface change — `Connector.authenticate/status/ingest` and
+  `IngestOptions.token` all currently require `TokenBundle`
+  (`connectors/types.ts:44,77`). `connectors list/status` derive "connected" from
+  keychain entries, so a no-auth connector needs a status story. The confirmation
+  count currently forces full enumeration before storing — reconcile with
+  streaming. Command registration lives in `packages/federation/verbs/index.ts`,
+  not just `cli/src/index.ts`.
+- **Depends on:** a concrete second connector (Linear) to justify the work.
+
+## ~~Bug — Postgres NOTICE lines pollute `--json` stdout~~ — FIXED 2026-06-03
+
+- **What:** Federation commands self-create tables/indexes with
+  `CREATE INDEX IF NOT EXISTS`; on an existing DB Postgres emitted a `NOTICE`
+  ("relation ... already exists, skipping") that the `postgres` driver printed to
+  **stdout**, ahead of the JSON, breaking `--json` for strict parsers.
+- **Fix:** `createDb` in `packages/db/src/client.ts` now passes
+  `onnotice: () => {}` (matching the existing `createUtilitySql` factory).
+  Verified live before/after: `brain create --file --json` against an
+  initialized DB now emits only the JSON object on stdout (`python3 -m
+  json.tool` parses it cleanly; before, three NOTICE objects preceded it).
+
+## Observation — KO content hash folds in `createdAt`
+
+- Not a bug (the v0.3 run cache now keys on content, so chain caching works), but
+  worth recording: `knowledge-object.ts:57,60` includes `createdAt` in the
+  hashed payload, so identical content created at different times yields
+  different KO ids/hashes. If true content-addressing is ever wanted
+  (re-creating identical content → same id), that is a core-model change needing
+  its own review (federation/consent/dedup implications). Deferred.
+
+## Later — `brain lineage <ko>` graph-walk command
+
+- **What:** A command/visualizer that walks `provenance.inputKoIds` edges to show
+  a KO's full lineage back to its original inputs.
+- **Why:** v0.3 uses one-hop provenance (each KO lists direct inputs only). Full
+  lineage is *derivable* by walking edges but there's no tool to do it. Eng
+  review (#9) flagged that a graph-walk exit criterion needs a real walker.
+- **Where:** new `brain lineage` verb over `agent-output` content `inputKoIds` +
+  run receipts.
+
+## v0.3.1 — Linear connector (deferred from Phase 3 / W2)
+
+- **What:** Add a Linear connector (issues → signed `document`/`issue` KOs) on
+  the W0 generic ingest path.
+- **Why:** Second real connector — proves the framework against a non-GitHub
+  OAuth + GraphQL shape and gives the run/chain loop a second real source.
+- **Blocker to resolve first:** Linear has **no OAuth device-code flow** (the
+  affordance GitHub's `connect` UX is built on). Decide: localhost-redirect
+  OAuth listener vs. **personal access token** (`authType: "api-key"`). PAT is
+  the leaning for alpha speed.
+- **Where to start:** `packages/federation/src/connectors/linear/`
+  (`connector.ts`, `api.ts` GraphQL client, `normalize.ts`); register in
+  `connectors/registry.ts`; CLI flag→params mapping in `cli/src/index.ts`
+  (`--team`, `--state`, `--label`, `--since`). Env `STACY_LINEAR_CLIENT_ID` or a
+  PAT entry in the keychain.
+- **Depends on:** Phase 3 W0 (generic `ingestCommand`) landing first.
+
+## v0.3.1 — Real-API robustness (deferred from Phase 3 / W4)
+
+- **What:** Rate-limit backoff/retry + resumable ingest cursor.
+- **Why:** Repeated dogfooding against real APIs hits 429s and transient
+  failures; today an interrupted `ingest` restarts from scratch.
+- **Where to start:** centralize retry/backoff in `connectors/http.ts` (honor
+  `Retry-After` / `X-RateLimit-Reset`; `rate-limiter.ts` already models the
+  window). Add a per-(connector, source) cursor file under
+  `<instance>/connectors/` so `ingest` resumes. Optional: a provider-agnostic
+  second reference adapter in `scripts/` to prove the adapter contract.
+- **Depends on:** best done once ≥2 connectors exist (after W2) so there is real
+  load to harden against.
+
+## Post-v0.2 — Independent validation gate (carryover)
+
+- **What:** Independent engineer validates the live GitHub + Anthropic paths
+  with their own credentials; meet design-partner metrics (15 alpha users, ≥10
+  real ingests, ≥5 `agent_output` KOs).
+- **Why:** `releases/v2026.602.0.md` is **Draft** until this clears; it also
+  gates tagging public v0.3.
+- **Where:** `PHASE-2-HANDOFF.md` §4 test plan.

cli/src/index.ts

@@ -27,7 +27,7 @@ import { registerPluginCommands } from "./commands/client/plugin.js";
 import { registerClientAuthCommands } from "./commands/client/auth.js";
 import { cliVersion } from "./version.js";
 import { applyStacyEnvAliases } from "./config/env-aliases.js";
-import { agentRunCommand, registerFederationCommands, runTaskCommand } from "@arpanstacy/stacy-federation/verbs";
+import { agentRunCommand, registerFederationCommands, runChainCommand, runTaskCommand } from "@arpanstacy/stacy-federation/verbs";
 
 const program = new Command();
 applyStacyEnvAliases();
@@ -148,6 +148,7 @@ program
   .option("--repair", "Attempt automatic repairs during doctor", true)
   .option("--no-repair", "Disable automatic repairs during doctor")
   .option("--use <ko_id>", "Input Knowledge Object for an AI run; repeat for multiple", collectOption, [])
+  .option("--chain <path>", "Run a multi-step chain from a JSON spec file (steps[] with @stepId refs)")
   .option("--model <name>", "Model identifier passed to the run adapter")
   .option("--adapter <name>", "Run adapter to use: anthropic (default) or deterministic")
   .option("--input <path>", "Input file for a public federation demo task")
@@ -164,6 +165,10 @@ program
   .option("--db-url <url>", "Database connection string")
   .option("--json", "Print raw JSON output", false)
   .action(async (task, opts) => {
+    if (typeof opts.chain === "string" && opts.chain.trim()) {
+      await runChainCommand({ ...opts, noCache: opts.cache === false });
+      return;
+    }
     if (typeof task === "string" && task.trim()) {
       if (Array.isArray(opts.use) && opts.use.length > 0) {
         await agentRunCommand(task, { ...opts, noCache: opts.cache === false });

docs/cli-reference.md

@@ -36,14 +36,30 @@ Operations on locally stored signed Knowledge Objects (KOs).
 ### `stacy brain create`
 Create and store a local signed Knowledge Object.
 
+Provide content one of four ways: inline JSON, a prompt, a single local file, or
+a directory of files. The file/directory modes are credential-free.
+
 | Option | Description |
 | --- | --- |
 | `--content-json <json>` | KO content as inline JSON. |
 | `--prompt <text>` | Generate KO content from a prompt. |
+| `--file <path>` | Create a KO from a local text/markdown/json file (no credentials). Wraps it in a `{ kind: "document", source, mediaType, text\|data }` envelope. |
+| `--dir <path>` | Create one KO per file under a directory (no credentials). |
+| `--glob <pattern>` | Glob pattern for `--dir`, e.g. `'**/*.md'` (relative to `--dir` or cwd). |
+| `--ext <list>` | Comma-separated extension allowlist for directory ingest, e.g. `md,txt,json`. |
+| `--source-label <label>` | Override the `--file` provenance label (default: cwd-relative path, basename if outside cwd). |
+| `--max-bytes <n>` | Max file size for `--file`/`--dir` in bytes (default `1048576`). |
+| `--yes` | Skip the directory-ingest confirmation prompt (for scripting). |
 | `--adapter-command <command>` | Adapter-like command that reads the prompt on stdin and writes output. |
 | `--adapter-arg <arg>` | Argument passed to `--adapter-command`; repeat for multiple. |
 | `--content-type <type>` | KO content type (default `application/json`). |
-| `--ko-id <id>` | Deterministic KO id (for harness runs). |
+| `--ko-id <id>` | Deterministic KO id (single-object modes only). |
+
+Exactly one of `--content-json`, `--prompt`, or `--file` may be given. Directory
+mode (`--dir`/`--glob`/`--ext`) skips dotfiles, `node_modules`, `.git`, and
+symlinks; binary and oversized files are skipped with a warning rather than
+aborting the batch. Source labels are always cwd-relative (or a bare filename
+when the file sits outside cwd), so absolute paths never leak into shareable KOs.
 
 ### `stacy brain list`
 List locally stored signed Knowledge Objects.
@@ -153,11 +169,12 @@ instead.
 
 | Option | Description |
 | --- | --- |
-| `--use <ko_id>` | Input Knowledge Object for the run; repeat for multiple. At least one is required. |
+| `--use <ko_id>` | Input Knowledge Object for the run; repeat for multiple. At least one is required (single-run mode). |
+| `--chain <path>` | Run a multi-step chain from a JSON spec file instead of a single run. Mutually exclusive with `--use`. |
 | `--model <name>` | Model identifier passed to the adapter (default `claude-sonnet-4-5`). |
 | `--adapter <name>` | `anthropic` (default) or `deterministic`. |
 | `--ack-egress` | Acknowledge that a non-deterministic adapter may send input KO content outside this install. |
-| `--ko-id <id>` | Deterministic output KO id (for harness runs). |
+| `--ko-id <id>` | Deterministic output KO id (single-run mode only). |
 | `--no-cache` | Skip the run-result cache and force a fresh adapter call. |
 
 **Adapters.** The `deterministic` adapter is offline, makes no network call, and
@@ -173,6 +190,32 @@ The cache key is independent of `--use` ordering.
 Every run loads and verifies each input KO, then emits `create`, `sign`, and
 `run` receipts for the new `agent_output` KO.
 
+### Run chains — `stacy run --chain <spec.json>`
+
+Compose runs: a later step consumes an earlier step's `agent_output` KO via an
+`@<stepId>` reference. The spec is an ordered `steps[]` array:
+
+```jsonc
+{
+  "steps": [
+    { "id": "per_doc",   "task": "Summarize each document", "use": ["ko_a", "ko_b"] },
+    { "id": "synthesis", "task": "Synthesize one report",   "use": ["@per_doc"] }
+  ]
+}
+```
+
+```bash
+stacy run --chain ./chain.json --adapter deterministic
+```
+
+Each step is a full run (`--model`/`--adapter`/`--ack-egress`/`--no-cache` apply
+to the whole chain; a step may override `model`). The spec is validated and all
+`@ref`s resolved **before** the egress gate or any KO read — a forward, unknown,
+or self reference fails immediately. Egress is acknowledged once for the whole
+chain. Provenance is one-hop (each step lists its direct inputs). A step failure
+aborts the chain and names the failed step; already-produced step KOs stay
+durable. See [Concept: AI Runs](concepts/ai-runs.md#run-chains-v03).
+
 ---
 
 ## Sharing & revocation

docs/concepts/ai-runs.md

@@ -91,3 +91,47 @@ acknowledgement.
 
 See the [v0.2 quickstart](../v0.2-connectors-and-runs-quickstart.md) for the
 end-to-end flow.
+
+## Run chains (v0.3)
+
+A single run is one task over a flat list of inputs. A **run chain** composes
+several runs, where a later step consumes an earlier step's `agent_output` KO as
+input. This expresses "summarize each source, then synthesize one report" as a
+single command.
+
+A chain is a JSON file with an ordered `steps[]` array. A step references a prior
+step's output with `@<stepId>`:
+
+```jsonc
+{
+  "steps": [
+    { "id": "per_doc",   "task": "Summarize each document", "use": ["ko_a", "ko_b"] },
+    { "id": "synthesis", "task": "Synthesize one report",   "use": ["@per_doc"] }
+  ]
+}
+```
+
+```bash
+stacy run --chain ./chain.json --adapter deterministic
+```
+
+```text
+per_doc   use: [ko_a, ko_b] ──run──► ko_out1
+synthesis use: [@per_doc]   ──run──► ko_final   (@per_doc resolves to ko_out1)
+```
+
+Guarantees:
+
+- **Validated up front.** The spec is parsed and every `@ref` is checked
+  *before* the egress gate and before any KO is read. A forward, unknown, or
+  self reference fails immediately — no partial run, no egress.
+- **Egress gated once.** A non-deterministic adapter still requires
+  `--ack-egress`, checked a single time before the first step.
+- **One-hop provenance.** Each step's `agent_output` lists only its direct
+  inputs (`synthesis → [per_doc output] → [ko_a, ko_b]`). Full lineage is the
+  graph walk across those edges; nothing is flattened or duplicated.
+- **Caching composes.** Each step is cached by `(task, model, adapter, input
+  content hashes)`. Because `agent_output` content is timestamp-free, an
+  identical chain re-run reuses every step and makes zero adapter calls.
+- **Abort on failure.** If a step fails, the chain stops and names the failed
+  step. Already-produced step KOs remain durable (no rollback).