Skip to content

pipeline: research findings reach only the planner, and research is gated on the class triage most often downgrades away #2415

Description

@macanderson

Problem

The pre-plan research stage (#1778 / PR #1953) exists to ground the turn in
verified facts about this workspace. Two wiring choices mean those facts
reach far less of the pipeline than the design intends, and on the most
common path they are never produced at all.

1. Research findings reach the planner only — never the worker

Pipeline::research_stage returns Vec<ResearchFinding>, and the sole
consumer is build_planner_prompt(goal, recall, research, repo_structure, revision)
(crates/stella-pipeline/src/plan.rs:212, called from
crates/stella-pipeline/src/pipeline.rs:1434).

The worker's own user message is built by
assemble_user_message(goal, frames, contract)
(crates/stella-pipeline/src/pipeline.rs:3065) — recalled context frames,
the goal, and the verification contract. No research findings. The
execute stage then appends only plan_steps::step_prompt(...) per step
(crates/stella-pipeline/src/pipeline/execute_stage.rs:118).

So a finding survives to the worker only as whatever residue of it the
planner chose to encode into a step string. The evidence is compressed
through a lossy intermediary that was never asked to preserve it.

2. Research is gated on the one class triage most often talks itself out of

triage_stage.rs:144:

let research = if resolved.class.plans() && !resolved.conversational { ... }

TaskClass::plans() is true only for MultiStep. So research requires
triage to answer multi — but resolve_task_class
(crates/stella-pipeline/src/triage.rs:669) explicitly lets the model lower
the deterministic floor by one rung, and the prompt actively encourages
cheapness. The stage that would supply grounding is gated behind the
decision that most often removes it.

Consequence: on a single task the pipeline has no mechanism at all to go
look something up before working — which is exactly the case where a worker
with no plan and no findings is most exposed.

Evidence

Across 34 Terminal-Bench tasks in three arm runs
(.../rig-runs/jobs/{dev2,DISCARDED-dev1,smoke1}-armA-stella), censused from
stella-events.jsonl:

  • stage events named research: 0.
  • stage events named triage: 18 in dev2 alone; plan: 5.

(In those runs triage was also timing out at its ceiling — see the companion
issue — so this is a lower bound on the gating effect, not a clean
measurement of it. It does establish that the path does not fire in
practice on that harness.)

Locally the stage does fire — .stella/private/store.db, telemetry:
call_role='research' has 49 rows, avg 5,559ms, $0.92 total. Those findings
were produced, paid for, handed to the planner, and never shown to the worker
that did the work.

Files

  • crates/stella-pipeline/src/pipeline/research_stage.rs (the stage)
  • crates/stella-pipeline/src/pipeline/triage_stage.rs:144 (the plans()
    gate)
  • crates/stella-pipeline/src/pipeline.rs:1434 (planner is the only sink),
    :3065 (assemble_user_message — the worker's message, no research)
  • crates/stella-pipeline/src/plan.rs:242 (the ## Research findings
    section)
  • crates/stella-pipeline/src/research.rs (bound_research_findings,
    RESEARCH_FINDING_CHARS)

Repro / verify

  1. Run a single-class goal that needs a workspace fact
    (add a retry to the thing that owns retry policy) and confirm no
    research stage event is emitted.
  2. Run a multi-class goal, confirm research fires, then dump the worker's
    first user message and confirm the findings are absent from it.
  3. sqlite3 .stella/private/store.db "SELECT COUNT(*), SUM(cost_usd) FROM telemetry WHERE call_role='research';"

Constraints

  • Research is advisory: every failure must degrade to fewer findings,
    never a failed turn, and zero findings must leave downstream prompts
    byte-for-byte unchanged (documented in research_stage.rs module docs).
    Adding a worker sink must preserve that — an empty findings list must not
    change assemble_user_message's output by a single byte.
  • Prompt-cache stability (AGENTS.md invariant 7): findings are volatile
    content and must ride after the stable prefix.
  • Findings are already bounded (bound_research_findings); a second sink
    must not double-bill the budget or re-run the children.
  • Ungating from MultiStep widens spend — the FanOutBudget pre-dispatch
    gate and research_latency_ceiling must still bound it, and the fast paths
    (L-E2) must stay free when triage names no questions.

Definition of done

  • Research findings reach the worker's prompt, not only the planner's, with
    a witness test asserting they appear in assemble_user_message's output
    when present and that the output is byte-identical when absent.
  • A decision (either way, stated) on whether single-class goals may ask
    research questions — and if yes, the gate widened with a spend bound.

Related: #1963 (research findings are paid for and thrown away — persist them
as reusable context chunks; this issue is the same-turn half of that
complaint), #2381 (stages cannot be ablated independently).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:pipelinestella-pipeline — triage→…→judge orchestrationin-progress

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions