Skip to content

Preserve deterministic NodeAgent workbook recovery#205

Merged
HomenShum merged 1 commit into
mainfrom
codex/nodeagent-prod-route-evidence
Jul 16, 2026
Merged

Preserve deterministic NodeAgent workbook recovery#205
HomenShum merged 1 commit into
mainfrom
codex/nodeagent-prod-route-evidence

Conversation

@HomenShum

Copy link
Copy Markdown
Owner

Summary

  • preserve deterministic workbook plans across runtime compaction
  • stop stale follow-on plans after verified deterministic writes instead of delegating into repeated provider calls
  • accept only validated completed Excel cache finalization as recalculation proof
  • add multi-sheet and external-recalculation regressions

Proof

  • clean
    pm run floor: 350 files, 2,489 tests passed
  • SpreadsheetBench V2 Debugging/01_01: 5 steps, 0 model tokens, 22 verified formula repairs, overall 0.576332 baseline -> 0.580125
  • free-provider quota is no longer required for this deterministic task

Honest limits

  • task remains
    eeds_repair without fully completed Excel recalculation; stable-pending is not promoted to passed
  • full 321-task model-edit-plan receipt coverage remains incomplete

@vercel

vercel Bot commented Jul 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
noderoom Ready Ready Preview, Comment Jul 16, 2026 9:48pm

Request Review

@HomenShum
HomenShum force-pushed the codex/nodeagent-prod-route-evidence branch from 4e708e3 to e985074 Compare July 16, 2026 21:46
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown

Scaffold Handoff — For Your Coding Agent

Your coding agent (Codex, Claude Code, etc.) should apply the accepted
scaffold proposals below. Do NOT touch any immutable files.

Immutability Check

Mode: advisory

✅ No immutable files were modified in this branch.

Changed Files

  • docs/eval/OFFICIAL_BENCHMARK_READINESS.md
  • docs/eval/OFFICIAL_BENCHMARK_TASK_COVERAGE.md
  • docs/eval/OPENROUTER_CONVEX_BENCHMARK.md
  • docs/eval/agent-improvement-loop.md
  • docs/eval/agent-improvement-loop.svg
  • docs/eval/agent-improvement-loop/20260716T214714Z.json
  • docs/eval/agent-improvement-loop/latest.json
  • docs/eval/agent-workspace-sandbox-smoke.json
  • docs/eval/algorithm-artifact-smoke.json
  • docs/eval/bankertoolbench-official-contract.json
  • docs/eval/docker-sandbox-probe.json
  • docs/eval/eval-runs.jsonl
  • docs/eval/halo-convex-context-telemetry.json
  • docs/eval/halo-self-improvement-smoke.json
  • docs/eval/halo-variant-selection.json
  • docs/eval/official-benchmark-readiness.json
  • docs/eval/official-benchmark-task-coverage.json
  • docs/eval/openrouter-convex-benchmark.json
  • docs/eval/professional-catalog-proofs.json
  • docs/eval/professional-proof-ledger.json
  • docs/eval/spreadsheetbench-chart-visual-probe.json
  • docs/eval/traces/credit/20260716T214722695Z-eea1f191_dirty.c04d38c98d5641d1/cascade-healthy.json
  • docs/eval/traces/credit/20260716T214722695Z-eea1f191_dirty.c04d38c98d5641d1/delta-incomplete.json
  • docs/eval/traces/credit/20260716T214722695Z-eea1f191_dirty.c04d38c98d5641d1/mapping-correct.json
  • docs/eval/traces/credit/20260716T214722695Z-eea1f191_dirty.c04d38c98d5641d1/mapping-misbind.json
  • docs/eval/traces/credit/20260716T214722695Z-eea1f191_dirty.c04d38c98d5641d1/summit-stressed.json
  • docs/eval/traces/ladder/20260716T214722289Z-eea1f191_dirty.e309e2a67caf3b2e/ladder_L1_read_scripted.json
  • docs/eval/traces/ladder/20260716T214722289Z-eea1f191_dirty.e309e2a67caf3b2e/ladder_L2_edit_scripted.json
  • docs/eval/traces/ladder/20260716T214722289Z-eea1f191_dirty.e309e2a67caf3b2e/ladder_L3_conflict_scripted.json
  • docs/eval/traces/ladder/20260716T214722289Z-eea1f191_dirty.e309e2a67caf3b2e/ladder_L4_blocked_scripted.json
  • docs/eval/traces/ladder/20260716T214722289Z-eea1f191_dirty.e309e2a67caf3b2e/ladder_L5_large_range_scripted.json
  • docs/eval/traces/ladder/20260716T214722289Z-eea1f191_dirty.e309e2a67caf3b2e/ladder_L6_long_horizon_scripted.json
  • docs/eval/traces/ladder/20260716T214722289Z-eea1f191_dirty.e309e2a67caf3b2e/ladder_L7_resume_scripted.json
  • src/eval/spreadsheetBenchNodeAgentBridge.ts
  • tests/spreadsheetBenchNodeAgentBridge.test.ts

Needs Adversarial Review — Do NOT Apply Yet

These proposals passed the reject check but have not been approved by
an adversarial reviewer. A human or frozen LLM judge must approve them first.

  • scaf-001 (AGENTS.md): Add explicit instruction for step spreadsheetbench-runner-fixture: Step spreadsheetbench-runner-fixture failed — scaffold may need explicit instruction or evidence assertion.
  • scaf-002 (AGENTS.md): Add explicit instruction for step convex-boundaries: Step convex-boundaries failed — scaffold may need explicit instruction or evidence assertion.

Safety Boundary

Agent may improve the scaffold.
Agent may NOT weaken the proof gate.

Immutable files (never modify):

  • scripts/proofloop.mjs
  • scripts/agent-improvement-loop.ts
  • tests/harnessChangeEval.test.ts
  • .github/workflows/
  • src/eval/evalTrustPolicy.ts
  • src/eval/architectureBudget.ts
  • evals/evalStore.ts

Scaffold files (safe to modify):

  • AGENTS.md
  • CLAUDE.md
  • proofloop/scenarios/*.yaml
  • proofloop/rubrics/*.yaml
  • proofloop/subagents/*.md
  • proofloop/adapters/*.js
  • .proofloop/memory.jsonl
  • src/nodeagent/models/prompts/systemPrompt.ts

@HomenShum
HomenShum merged commit 15a24cb into main Jul 16, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant