Performance: acceleration headroom within the local+direct architecture — query latency, look-up round-trips, dev flow; subagent/haiku delegation hypothesis

[kiki830621]

Problem / Request

Original text (verbatim, from the maintainer):
「在這格架構下還有沒有可能加速」

Clarified scope (verbatim, follow-up answer):
「1,2,3都是,我在想有些東西是不是可以靠subagent或者workflows(似乎派haiku是夠的?)」

Interpretation: within LiveDocs' architecture boundary — local tool, direct to primary source, no central content cache (the anti-positioning line in docs/positioning.md is a hard constraint, not a preference) — explore acceleration on all three surfaces, with one cross-cutting hypothesis: delegate mechanical work to subagents / Workflows running a cheap model (haiku).

Type

feature (performance exploration — expect diagnosis to decompose into measured sub-issues)

The three surfaces (from clarification)

1. Product query latency (engine) — live fetch is structurally slower than a pre-built index; headroom candidates inside the boundary: concurrent probing of discovery-chain candidates (llms.txt path variants + registry in parallel rather than sequential), persisting the ETag cache across sessions (disk-backed, still local — the boundary forbids shared/central content, not local caching), connection reuse / HTTP2, registry-first short-circuit for pure version queries. Note: this is Swift-engine concurrency work — subagents don't apply inside the MCP server.
2. look-up skill round-trips (agent side) — each question can cost multiple main-loop MCP round-trips (resolve → fetch → reconcile), each paying main-model latency+tokens. This is where the subagent/haiku hypothesis bites: a cheap subagent could run the mechanical resolve→fetch→extract chain and return a distilled answer, freeing the main context. Counter-consideration to measure, not assert: subagent spawn overhead may exceed the saved time for single-shot queries; delegation likely pays off only for multi-fetch/summarize-heavy lookups.
3. Dev flow / CI — swift test ~2min is the CI ceiling (GitHub-side; model choice irrelevant there), but the verify/IDD side already has prior art for the hypothesis: the Workflow fan-out doctrine pins cheap models (sonnet/haiku) for wide fan-outs. Candidates: cheaper-model lenses for mechanical verify stages, parallel dev tasks via worktree-isolated agents.

Expected

A diagnosis that profiles before optimizing — the project's measured-not-asserted rule applies to performance claims exactly as it does to competitive claims:

• Where does wall-clock actually go on a typical look-up query (engine fetch vs MCP round-trip vs main-model thinking)?
• Which of the three surfaces has the largest measured headroom per unit of effort?
• For the haiku-delegation hypothesis: a measured spawn-overhead vs saved-latency comparison, not a vibe.
• Then decompose into concrete, individually-verifiable sub-issues (likely per surface).

Actual

• Engine: discovery-chain probing is (believed) largely sequential; ETag cache is in-memory per process — no cross-session persistence.
• Skill: look-up runs entirely in the main agent loop; no delegation path exists.
• Dev flow: verify ensembles run opus lenses uniformly; no cheap-model tiering by lens difficulty.
  (All three "Actual" claims are to be verified during diagnosis — stated here as the starting belief, not as measured fact.)

Impact

• Latency is the one structural axis the vs-context7 positioning concedes by silence — a pre-built index answers faster per query. Acceleration inside the boundary strengthens "fresh AND fast enough" without becoming context7 (docs/positioning.md anti-positioning: speed must come from local revalidation, never a shared content cache).
• Cheap-model delegation also cuts token cost, not just latency.

Constraints (architecture boundary — non-negotiable)

• No central/shared content cache (anti-positioning).
• Local caching / revalidation / concurrency are all fair game.
• Any competitive speed claim that comes out of this must be measured (dated capture, à la the freshness harness) — never asserted.

Clarification (added during filing)

The scope question (which of the 3 surfaces) was resolved interactively before filing — answer quoted verbatim above; no deferred clarity rows.

---

Do NOT add a close trailer — /idd-close after merge.

Children (measured decomposition, 2026-07-06 profile)

• Engine: parallelize discovery-chain probing (TaskGroup over candidates) #57 — engine: parallelize discovery-chain probing (fastapi 1055–1159 ms → ~400 ms class)
• Engine: disk-persist the ETag cache (bounded, local-only) so the 10× warm state survives restarts #58 — engine: disk-persist the ETag cache (344→32 ms warm state survives restarts)
• Composite one-call lookup: resolve+fetch server-side to cut look-up round-trips 2→1 #59 — composite one-call lookup (skill round-trips 2→1; the measured replacement for the refuted single-shot haiku delegation)

Current Status

Phase: closed (merged via PR #60 + verified; see Closing Summary)

[kiki830621]

Diagnosis

Type

feature (performance exploration — measured, per the issue's own "profile before optimizing" mandate)

Measured profile (captured 2026-07-06; reproduce commands will ship in the committed profile doc)

① Engine (query latency) — static + dynamic:

• Zero structured concurrency in the engine: grep withTaskGroup|async let over Sources/ → 0 hits. Every probe chain is sequential for+await (DiscoveryEngine.swift:89,142,173-188,402).
• Timed via MCP stdio against the installed signed binary (~/bin/CheLiveDocsMCP, n small, home network — directional):

call	cold	warm (same process)
latest_version react npm	344 ms	32 ms (ETag 10×)
resolve_source react npm	367–385 ms	271 ms
resolve_source fastapi pypi	1159 ms	—
fetch_docs react llms.txt	205 ms	—

• fastapi at 3× react = the sequential multi-candidate probe penalty made visible (more llms.txt candidates / slower host, paid linearly).
• ETag cache is in-memory only — its own docstring: "empty on restart. Persisting to disk is a possible follow-up." The 10× warm win dies with every MCP server restart.

② look-up skill (round-trips) — from the repo skill (source of truth): docs questions = resolve_source → then fetch_docs (2 sequential main-loop round-trips); version = 1; version-reconciliation adds introspect (up to 3). Each round-trip pays a main-model turn (seconds of thinking + tokens) — which dwarfs the 0.3–1.2 s tool execution. The dominant end-to-end cost is the number of main-loop turns, not the fetch itself.

③ Subagent/haiku delegation hypothesis — measured: claude -p "reply ok" --model haiku --strict-mcp-config from a clean dir = 9.0 s wall (real; user 2.4 s). Spawn overhead is 8–25× the fetch it would replace. Verdict: refuted for single-shot lookups; viable only when one spawn amortizes over a batch of lookups (≳10) or when the goal is context-isolation/token-cost rather than latency. For dev flow, delegation is already prior art (this repo's verify ensembles) — cheap-model tiering there is a cost win, not a latency one.

The measured insight that redirects the hypothesis

The user's instinct ("some things could go to a cheap worker") is right, but the cheapest worker isn't a subagent — it's the MCP server itself. A composite engine-side call (resolve + fetch top result in ONE tool call, returning distilled content) removes a whole main-model turn (~seconds) at zero spawn cost. Engine parallelization (TaskGroup over candidates) then compresses the in-call time; disk-persisted ETag makes the 10× warm state survive restarts.

Strategy

• Measurements (this diagnosis).
• Commit a dated perf profile evals/look-up/perf_profile.md — capture table + reproduce commands (stdio timing harness) + honesty caveats (small n, one machine/network, directional not benchmark).
• File measured sub-issues (children of Performance: acceleration headroom within the local+direct architecture — query latency, look-up round-trips, dev flow; subagent/haiku delegation hypothesis #56): (a) parallelize discovery-chain probing with TaskGroup (target: multi-candidate resolves approach slowest-single-probe, e.g. fastapi 1159→~400 ms); (b) disk-persist the ETag cache (keep LRU/byte bounds; local-only — inside the anti-positioning boundary); (c) composite one-call lookup (resolve+fetch server-side) to cut skill round-trips 2→1, updating the skill table; each sub-issue records the relevant numbers + the haiku-refutation so the hypothesis isn't re-litigated.
• PATCH Performance: acceleration headroom within the local+direct architecture — query latency, look-up round-trips, dev flow; subagent/haiku delegation hypothesis #56 body with the children task list.

Conflict Class

A_parallel_safe — this issue commits a docs artifact + files issues; no engine code changes here (those live in the children).

Complexity

Simple — the deliverable of #56 itself is the measured profile + decomposition (exactly what its Expected section asks for).

Risks

Numbers are directional (small n, one network) — the profile doc must say so loudly, per measured-not-asserted.

Residue

"加速" as perceived end-to-end speed also includes main-model thinking time between turns, which LiveDocs cannot control — conceded; the actionable part is turns × in-call latency.

[kiki830621]

Verify: #56 (pinned-opus agent lenses ×2 — Workflow blocked by session model-tier doctrine, flagged not skipped)

Verdict: FINDINGS → fixed (<fix commit on PR #60>). Both lenses ran to completion; dispatch = opus (pinned per-agent).

Scope/regression lens — PASS-grade: scope exactly the 3 stated files; pytest still 72 with the harness structurally uncollectable; CI untouched; no new deps; docs consistent with the anti-positioning boundary; children's numbers cross-checked. LOW (raw traceback on missing binary) + NIT (#57 range mismatch) → both fixed.

Measurement-honesty lens — the substantive catches, all fixed by re-measuring rather than re-wording:

Sev	Finding	Fix
MED	"cold" resolve react was connection/ETag-warmed by prior calls in the same process → understated ~2× (367 vs unbiased 649 ms); the "fastapi 3×" causal claim rested on it	Harness reordered; re-captured: fastapi ≈ 1.8× react true-cold; causal claim softened to "consistent with summed probes — #57 must instrument per-probe before projecting"; the biased capture kept as a worked example
MED	"largest measured lever" mislabeled — the composite-call lever's magnitude is the per-turn cost the engine-side harness deliberately doesn't measure	Relabeled expected; ② now splits measured (tool time) from asserted (turn cost) explicitly
LOW	warm 10× attributed to ETag alone (connection/DNS/TLS reuse confounded); #58 would not recover the full win	Conceded (observed 2–10× spread); implication written into #58's acceptance expectations
LOW	8–25× arithmetic (9/0.3 = 30, not 25); claude -p as Agent-tool proxy unlabeled	~7–30×; proxy equivalence explicitly labeled an assertion

The refutation of single-shot haiku delegation survives strengthened (the corrected loss range is wider), and the decomposition into #57/#58/#59 stands — with #57's projection now correctly conditional on per-probe instrumentation.

Process note: a cross-model/DA ensemble round was not run (doctrine hard-denies Workflow fan-outs from this session's model tier); the two lenses + the re-capture are the verification basis, stated honestly.

[kiki830621]

Closing Summary

Problem. 「在這個架構下還有沒有可能加速」— within the local+direct boundary (no central content cache), where is the acceleration headroom, and can subagent/Workflow delegation with a cheap model (haiku) provide it?

Solution — measure, then decompose. PR #60 (merged 5933624) ships a reproducible MCP-stdio timing harness (evals/look-up/profile_mcp.py) + a dated capture (evals/look-up/perf_profile.md, 2026-07-06) that separates structural facts (source-derived, stable) from timings (network-dominated, wide same-day spreads), and decomposes the headroom into three measured children:

• Engine: parallelize discovery-chain probing (TaskGroup over candidates) #57 — parallelize the sequential probe chains (0 TaskGroup/async let in the engine; fastapi true-cold ≈ 1.8× react; projection conditional on per-probe instrumentation)
• Engine: disk-persist the ETag cache (bounded, local-only) so the 10× warm state survives restarts #58 — disk-persist the per-process ETag cache (warm win 2–10× observed, confounded with connection reuse — only the revalidation share is recoverable; local-only, inside the anti-positioning boundary)
• Composite one-call lookup: resolve+fetch server-side to cut look-up round-trips 2→1 #59 — composite one-call lookup (docs questions cost 2 main-loop turns; turns — asserted, not measured — dominate perceived latency; largest expected lever)

The hypothesis, answered with numbers. Single-shot haiku delegation is refuted: headless spawn ≈ 9 s vs the 0.3–1.2 s of work it would absorb (~7–30× net loss). It survives only amortized over batches (≳10) or for token-cost/context-isolation goals. The instinct was right, the worker was wrong — the cheapest worker is the MCP server itself (#59).

Verification. Two pinned-opus agent lenses (Workflow doctrine-blocked this session — flagged, not skipped). The honesty lens caught a real ordering bias in the original capture (react's "cold" was connection-warmed → 367 vs unbiased 649 ms; "3×" was really ~1.8×) — fixed by re-measuring with a corrected harness, not re-wording; mislabeled "measured" claims de-asserted; arithmetic corrected (7–30×). The biased capture is kept in the doc as a worked example of why single runs mislead. Scope lens: clean (3 files, 72 pytest unchanged, CI untouched).

Residue (honest). Timings are one machine/one day, directional; the per-turn main-model cost is asserted (engine-side harness can't see it); claude -p is a proxy for the in-session Agent tool (order-of-magnitude equivalence asserted); perceived speed includes model thinking time no LiveDocs change can touch.

Next. The actual speedups are the children: /idd-all #57 #58 #59.