Decide (on paper) whether human-interactive chat belongs in the pull queue — TARGET_ARCHITECTURE Open Question 7

[obasilakis]

Summary

docs/planning/TARGET_ARCHITECTURE.md Open Question 7 — "Does human-interactive chat belong in the queue at all?" — is still marked "Under consideration, not decided." The doc instructs: "Decide before the #946 pilot, which routes chat_with_agent — an interactive kind — through the queue first."

#946 shipped. The decision is overdue, and it is now being made implicitly by code.

Why this needs closing now

PR #1982 (#1766 soak prep) makes a pull pilot's dispatch queue-only for autonomous triggers while excluding interactive ones — human chat and Session-tab turns keep today's synchronous push path and today's Redis session lock. That is an implementation of Open Question 7's own candidate answer (the "scope cut" rather than added machinery), and it is load-bearing: without it, one FIFO ordered by queued_at parks a human turn behind N batch tasks until the held connection times out, and N competing workers can claim two turns of one session concurrently — the concurrent --resume on a single JSONL that the session lock exists to prevent.

So the answer is now encoded in working code that ships to production, while the doc still says undecided. That gap is the problem:

• Nobody reviewing TARGET_ARCHITECTURE.md can tell that the boundary is already real.
• The trigger carve-out in services/pull_pilot.py::pull_owns_dispatch looks like a local implementation detail; it is actually an architectural commitment.
• Phase 4 of feat: pull / work-stealing coordination (push→pull dispatch) #1081 ("sync edge adapter") is specified against the other answer — chat as a thin edge adapter over the queue. Both cannot be right.

The two candidates (verbatim from the doc)

1. Edge adapter — the WebSocket or a ?wait=true call holds open, enqueues, forwards the reply on the completion event. Synchronous UX, async internals, held connection times out. Consistent with Principle Feature/vector log retention #3; but carries the starvation + per-session-serialization problems above, unaddressed.
2. Scope cut — the durable queue carries autonomous traffic only (schedules, webhooks, agent-to-agent, loops, channel messages); human chat/session turns keep the direct synchronous path and the session lock. Concurrency becomes one physical budget of N per agent: interactive turns take a slot on arrival, workers pull batch work only when slots are spare, optionally reserving one slot so long batch turns can't lock chat out. The doc leans this way; fix(pull): make the pilot flag a true either/or + soak measurement set (#1766) #1982 implements it.

Acceptance Criteria

• A decision is recorded in TARGET_ARCHITECTURE.md — Open Question 7 resolved, not merely annotated
• If the scope cut wins: §Coordination Model updated to state the queue carries autonomous traffic only, and feat: pull / work-stealing coordination (push→pull dispatch) #1081 Phase 4 re-scoped or dropped accordingly
• If the edge adapter wins: fix(pull): make the pilot flag a true either/or + soak measurement set (#1766) #1982's carve-out is a deliberate temporary measure and says so, with the starvation + session-serialization designs specified
• Either way the capacity model is stated explicitly — one physical budget of N, and whether a slot is reserved for interactive
• The decision is reflected in pull_owns_dispatch's docstring rather than that docstring being the only place it exists

Technical Notes

• Doc: docs/planning/TARGET_ARCHITECTURE.md Open Question 7 (+ §Async-First Communication, which assumes the edge adapter).
• Code: src/backend/services/pull_pilot.py::pull_owns_dispatch, capacity_manager.acquire.
• Note the replica interaction the doc flags: under replica_count > 1, session JSONLs are single-container state, so sessions need affinity or shared storage regardless of which answer wins (open question 5 / Replica groups: horizontal scaling for single agents #927).
• Related: feat: pull / work-stealing coordination (push→pull dispatch) #1081 (umbrella, Phase 4), experiment: pull-coordination pilot — route MCP chat_with_agent through the agent queue #946 (the pilot that was supposed to be gated on this), Production soak-test of pull/work-stealing coordination (#1081) before default-on #1766, fix(pull): make the pilot flag a true either/or + soak measurement set (#1766) #1982.

[obasilakis]

DECIDED — Open Question 7 resolves to the edge adapter. Chat goes on the queue; push is deleted.

Decision by @obasilakis, 2026-09-16. Recording it here because this issue is where the question lives, and because it has been getting answered by default, in code, for months while the doc said "undecided" — which is exactly what this issue was filed to stop.

The decision

1. Everything goes on the durable queue, including human-interactive chat. Candidate 1 (edge adapter) wins. Candidate 2 (scope cut) is rejected.
2. Interactive turns jump the queue, so a person's wait is no worse than today's push path. → feat(pull): interactive turns jump the queue — the starvation half of Open Question 7 #2842
3. A conversation never runs two turns at once, served by any free worker — mutual exclusion, explicitly not worker affinity. → feat(pull): one turn per conversation at a time — the transcript-safety half of Open Question 7 #2843
4. Once those land, the push coordination path is deleted, not kept alongside. One system, one set of books.

The carve-out from #1982 that keeps interactive turns on the synchronous path is hereby a deliberate temporary measure, which is what this issue's AC #3 requires it to say.

Why — the reasoning that settled it

The starvation objection was overstated, including by me earlier in the discussion. Push does not avoid the wait. Human chat dispatches with overflow_policy="queue_in_memory" — a bounded Redis FIFO — so at capacity a person already waits for a running turn to end. On eu2 the oracle agents average 339–468s per turn with a max of 1931s, so today's worst case is already ~32 minutes.

What push actually avoids is waiting behind queued batch work, because until #2391 autonomous overflow was rejected rather than accumulated. So the regression is not "pull makes humans wait" — it is "one undifferentiated FIFO makes humans wait behind batch."

	human's wait
push today	until a running turn ends
pull, single FIFO	until a running turn ends + the batch backlog ⟵ the actual regression
pull + priority (#2842)	until a running turn ends — parity with push
pull + reserved worker	immediate — better than push

Priority alone restores parity. That is the bar, and it is a much smaller piece of work than the objection implied.

The session-serialization objection is real but narrower than it reads. The requirement is only that two turns of one conversation never run concurrently; any worker may serve it. All workers share one container filesystem, so there is no file-access reason to pin a conversation to a worker — and pinning would actively defeat the priority fix by making a person wait on a busy worker while others idle. Affinity only becomes necessary under replica_count > 1, which is a separate open question.

What tipped it: the scope cut keeps two sets of books forever. Today there are two independent records of who is busy — rows in the database and a counter in Redis — neither authoritative alone, which is why three of the four slot/backlog canary invariants exist solely to detect the two disagreeing. Under one queue there is a single owner column with an expiry; the drift class disappears and its detectors have nothing to detect.

Measured scale of the simplification: roughly 6,000 lines of push-side coordination machinery against ~970 lines of pull code that already exists. Not all 6,000 goes — the cleanup module also carries retention work, and the capacity module shrinks rather than vanishes — so an honest figure is 3,000–4,000 lines retired, plus ~600 lines of reconciliation canaries. The scope cut retires none of it, because the second bookkeeper survives to serve chat.

Acceptance criteria of this issue, against the decision

• Recorded in TARGET_ARCHITECTURE.md — Open Question 7 marked resolved, §Coordination Model and §Async-First Communication updated to state the queue carries all traffic. ⚠️ Still outstanding — see below.
• Edge adapter wins ⇒ fix(pull): make the pilot flag a true either/or + soak measurement set (#1766) #1982's carve-out is explicitly temporary, and the starvation + session-serialization designs are specified — feat(pull): interactive turns jump the queue — the starvation half of Open Question 7 #2842 and feat(pull): one turn per conversation at a time — the transcript-safety half of Open Question 7 #2843
• Capacity model stated explicitly: one physical budget of N per agent; whether a worker is reserved for interactive is deferred to feat(pull): interactive turns jump the queue — the starvation half of Open Question 7 #2842 and must be recorded either way
• pull_owns_dispatch's docstring stops being the only place the boundary exists

⚠️ The one gap, stated plainly

The doc edit has not been made. These comments cannot close this issue on their own — AC #1 requires the decision in TARGET_ARCHITECTURE.md, "resolved, not merely annotated." Until someone lands that, a reader of the spec still finds "under consideration, not decided" and can reasonably relitigate all of this.

Concretely, what needs changing in the Trinity repo:

• docs/planning/TARGET_ARCHITECTURE.md — Open Question 7: replace "under consideration, not decided" with the decision above; §Coordination Model: state the queue carries all traffic including interactive; §Async-First Communication already assumes the edge adapter and becomes correct rather than aspirational.
• docs/planning/PULL_MIGRATION_STATUS.md — §3 reach table: interactive chat moves from "deliberately excluded — scope cut" to "pending feat(pull): interactive turns jump the queue — the starvation half of Open Question 7 #2842/feat(pull): one turn per conversation at a time — the transcript-safety half of Open Question 7 #2843"; §4 item 5 ("'everything is pull' is false by design") is deleted, it now says the opposite of the decision.
• docs/testing/PULL_MIGRATION_TESTING.md — §9's reach commentary documents the excluded triggers as correct behaviour; that framing inverts.

The PULL_MIGRATION_STATUS.md line is the load-bearing one — it is the document's own stated entry point, and it currently asserts the opposite of this decision in as many words.

Downstream work this decision creates

• feat(pull): interactive turns jump the queue — the starvation half of Open Question 7 #2842 — interactive turns jump the queue (starvation half)
• feat(pull): one turn per conversation at a time — the transcript-safety half of Open Question 7 #2843 — one turn per conversation at a time (transcript-safety half)
• feat(pull): async fan-out join + sync edge adapter — no autonomous trigger is stranded (#2524) #2532 — Phase 4 is now unambiguously on the critical path; it was specified against this answer all along
• bug(canary): B-02 has no pull awareness — fires critical on a healthy pull pilot, and Phase 5 does not retire it #2840 — canary B-02's disposition changes: under one queue every agent's rows pass through queued, so it is retired with the other slot/backlog invariants, not taught about pull
• Two gaps found on eu2 this week that now sit on the critical path rather than the margin: retry is in neither the autonomous nor the reachable trigger set (308 rows / 30 days on that instance), and the Phase-2 lease-renewal heartbeat was specified but never built, so a turn outliving its lease is re-run concurrently with itself. Neither is filed yet.

[obasilakis]

Ready-to-apply text for the doc edits, so AC #1 is copy-paste

The ops agent is barred from committing to this repo, so these are written out rather than pushed. Three edits close it.

---

1. docs/planning/TARGET_ARCHITECTURE.md, Open Question 7 (currently line 546)

Replace the whole numbered item with:

7. Does human-interactive chat belong in the queue at all? — DECIDED 2026-09-16 (@obasilakis): yes — the edge adapter, not the scope cut. The durable queue carries all traffic, interactive included, and the synchronous push dispatch path is deleted once the two supporting pieces land. The scope-cut alternative (queue carries autonomous traffic only, human chat keeps today's direct path) is rejected, because it preserves a second set of books permanently: two independent records of who is busy — rows in the database and a counter in Redis — which is the drift class that three of the four slot/backlog canary invariants exist solely to detect. One queue means one owner column with an expiry, and those detectors have nothing left to detect.

   The two consequences this question raised are addressed rather than avoided:

   (a) Interactive starvation → feat(pull): interactive turns jump the queue — the starvation half of Open Question 7 #2842. Note the premise was overstated: push does not avoid the wait. Human chat already dispatches with overflow_policy="queue_in_memory", a bounded Redis FIFO, so at capacity a person waits for a running turn to end regardless — measured on eu2, agent turns average 339–468s with a max of 1931s, so today's worst case is already ~32 minutes. What push avoids is waiting behind queued batch work. Priority ordering therefore restores parity with push; a reserved worker would be strictly better, and is deferred to feat(pull): interactive turns jump the queue — the starvation half of Open Question 7 #2842 with its capacity cost (one of N workers held idle) stated explicitly.

   (b) Per-session serialization → feat(pull): one turn per conversation at a time — the transcript-safety half of Open Question 7 #2843. The requirement is mutual exclusion per conversation — two turns of one conversation never run at once — served by any free worker. It is explicitly not worker affinity: all N workers share one container filesystem, so there is no file-access reason to pin, and pinning would make a person wait on a busy worker while others idle, defeating (a). Affinity does become necessary under replica_count > 1, where session JSONLs are single-container state — see open question 5 / Replica groups: horizontal scaling for single agents #927 — and is out of scope here.

   Concurrency remains one physical budget of N per agent.

---

2. docs/planning/PULL_MIGRATION_STATUS.md §3, the reach table (line 41 on the #2657 branch)

Replace:

| **Deliberately excluded** | interactive chat / Session-tab turns | scope cut, see §4 item 5 |

with:

| **Pending** | interactive chat / Session-tab turns | joins the queue once #2842 + #2843 land (decided 2026-09-16, #1989) |

While there, add the row that is missing entirely — it is in neither _AUTONOMOUS_TRIGGERS nor PULL_REACHABLE_TRIGGERS, and on eu2 it is 308 rows over 30 days (4.7% of traffic), going to push on pilot agents whose schedules go to pull:

| **Unreached, unclassified** | retry | in neither trigger set; looks like an oversight rather than a decision |

---

3. docs/planning/PULL_MIGRATION_STATUS.md §4 item 5 (line 74 on the #2657 branch)

Delete the item. It currently reads that "'everything is pull' is false by design, not by omission" and explains the scope cut as the reason. That is now the opposite of the decision, and by this PR's own argument — that a document someone can still open will be read and relitigated — it is the single sentence most likely to cause the next wasted review cycle.

Replace with:

5. Interactive chat joins the queue — feat(pull): interactive turns jump the queue — the starvation half of Open Question 7 #2842 (interactive turns jump the queue) and feat(pull): one turn per conversation at a time — the transcript-safety half of Open Question 7 #2843 (one turn per conversation at a time). Decided 2026-09-16 (Decide (on paper) whether human-interactive chat belongs in the pull queue — TARGET_ARCHITECTURE Open Question 7 #1989): the queue carries all traffic and the synchronous push path is deleted afterwards. Until both land, interactive turns still run on the old path — that is a migration state, not a design boundary.

---

Where to land them

Edits 2 and 3 are against #2657's branch (docs/pull-migration-consolidate), which rewrites that file and has not merged — applying them there avoids an immediate conflict. Edit 1 is against dev directly. Commented on #2657 separately.

Once edit 1 lands, this issue's AC #1 is met and it can close.

[obasilakis]

The doc edits are open as #2847, based on docs/pull-migration-consolidate (#2657) rather than dev, because that branch already rewrites all three affected files.

That closes this issue's outstanding AC #1 — "a decision is recorded in TARGET_ARCHITECTURE.md — resolved, not merely annotated" — on merge. Verified before pushing: no line in TARGET_ARCHITECTURE.md, PULL_MIGRATION_STATUS.md or PULL_MIGRATION_TESTING.md still asserts the scope-cut answer, and no internal markdown link is broken.

Leaving this issue open until #2847 merges, since the AC is about the doc and not about the comment.