feat(daemon): eval task progress plumbing — {done,total} on the task descriptor (part of #2216)#2244
Merged
Merged
Conversation
…descriptor (part of #2216) Daemon-side progress plumbing for the MCP eval task (P3a-2 PR1; no transport, no SSE — that is PR2). A tasks/get descriptor now carries an optional {done, total} so a follow-on transport lane can shape progress frames. The one frame (numerator and denominator through the SAME query_in_scope predicate, so the mirror bug is unrepresentable): - total = M = the post-filter query count (post_filter_count), stamped into the entry at admission BEFORE it is pollable, so a poll between insert and worker-start never serves {0,0}. - done = post-filter PROCESSED queries, bumped at iteration end (no-gold skips counted as processed), so it means finished, not started. Distinct from EvalOutput.query_count (scored only). - ProgressCounters rides EvalCoreParams as a non-serde Option side-channel (cancel-token shape), never on EvalOutput, so the dispatch/core parity guard holds by construction. None on the CLI and direct-daemon paths. - The descriptor reads status (Acquire) before the counters, matching the worker's counters -> result -> status(Release) order, so a terminal descriptor can never pair a terminal status with a stale done < total. - TaskProgress on the descriptor is additive + skip-if-none. Tests: post-filter denominator (incl no-gold, with full-set/unmatched negatives), numerator monotonicity, the mirror-bug debug_assert tripwire (should_panic), EvalOutput-carries-no-progress-keys, descriptor stamp-before-pollable (with a {0,0} negative control), terminal done==total with a mid-flight done<total read, and an embedder-gated end-to-end eval_core frame run. query_in_scope mutation-verified (disabling the filter flips M 2->4 and fails the frame test). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jamie8johnson
force-pushed
the
feat/p3a2-1-progress-plumbing
branch
from
4523f4a to
514faae
Compare
This was referenced Jul 5, 2026
jamie8johnson
added a commit
that referenced
this pull request
Jul 6, 2026
ROADMAP.md: add v1.54.0 + the post-v1.54.0 arc (MCP Phase 3 SSE #2244/#2250, is_test flag v33/PV18 #2249/#2254, eval gold_set+file-level #2252, off-self eval, testcode-demote knob), fix the stale 'Current: v1.49.0' marker. PROJECT_CONTINUITY.md: collapse five accumulated CURRENT-N paragraphs (all now-landed work) into one clean current-state block; update schema state (v32/PV14 -> v33/PV18), the open-issues table (17 open, verified vs gh, the old 2026-06-25 table had ~half closed), recent release history (add v1.50- v1.54), eval baselines (add the off-self finding + the testcode-demote A/B). notes.toml: the session's sweep-count-verification + env-docs-guard lessons. Co-authored-by: jamie8johnson <jamie8johnson@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 3a-2, PR1 of the SSE ladder: daemon-side eval-task progress plumbing (part of #2216)
Adds
{done, total}progress to the MCP task descriptor so atasks/getpoller (and the follow-on SSE lane) can observe an eval task's advance. Daemon-only — NO http.rs / SSE surface, NO eval_core scoring change (pure observability side-channel). Contract =docs/plans/2026-07-05-p3a2-sse-decisions.md§D2.What lands
ProgressCountersas anOptionside-channel onEvalCoreParams(mirrors the lane-1canceltoken; NON-serde, NEVER onEvalOutput— the dispatch==to_value(core) parity guard holds by construction).donecounts post-filter PROCESSED queries (bumped at iteration end; no-gold skips included),total= M = post-filter count stamped ONCE at admission before the entry is pollable. Numerator and denominator go through the SAME predicate (query_in_scope), so the mirror bug (progress >100% under--category) is structurally unrepresentable. Adebug_asserttripwire guards it.TaskDescriptor.progress: Option<TaskProgress>, read AFTERstatus(Acquire) so a terminal status can never pair with a staledone < total.descriptor_jsonnow serializesprogressskip-if-none (absent key, nevernull), sotask-getactually carries{done,total}— this is what makes PR1 observable rather than inert plumbing. Fence-safe (no panic token; wire.rs is on the daemon panic fence).Review
Fable code-review on the rebased tree (onto #2242): APPROVE, risk LOW, all six contract items verified (one-frame, stamp-before-pollable, read-order pin proven on the actual code, parity, the rebase-compose seam, tests-target-real-paths). The review flagged that
descriptor_jsondid not emit progress — I closed that here (it is PR1 scope per design line 35, not a PR2 carry-item), addeddescriptor_json_carries_stamped_progress, and corrected theTaskDescriptor.progressdoc comment to describe the now-real skip-if-none behavior.Tests / gates
fmt clean; clippy(all-targets, cuda-index) 0; provenance clean; batch::tasks 29/0 (incl. the new wire-emission test +
{0,0}negative control + terminaldone==total), eval runner 34/0 (incl. the mirror-bugshould_panictripwire + a real-embedder e2e that ran, not skipped), mcp::tasks 12/0, parity 64/0.query_in_scopemutation-verified (disabling the filter arm flips M and fails the denominator test).Follow-ons (not this PR)
PR2 = the SSE transport (http.rs) consuming this progress. The stdio bridge stays poll-only by design (progress is a receiver-MAY, no cross-transport parity requirement).
#2239(fence the eval worker) and#2240(e2e lifecycle test) are separately tracked.🤖 Generated with Claude Code