bug: Ultracode can undercount judgePanel calls and set an insufficient maxAgents cap

[Friday023]

Environment

• @quintinshaw/pi-dynamic-workflows: 3.7.0
• Pi: 0.84.2
• Node.js: v24.15.0
• macOS: 27.0

Summary

With /ultracode enabled, Pi generated a workflow whose declared maxAgents was lower than the workflow's deterministic helper-expanded call count. The workflow looked bounded, but it would inevitably reach AGENT_LIMIT_EXCEEDED if its earlier phases succeeded.

The easy-to-miss part is that judgePanel(attempts, { judges: N }) launches N judges for every candidate, not N judges total.

Generated topology

The generated script used:

• 15 evidence agents
• 5 adversarial cross-check agents
• 3 synthesis candidates
• judgePanel(viable, { judges: 5 })
• 1 completenessCheck
• up to 1 repair agent
• maxAgents: 36

With all three synthesis candidates surviving:

15 evidence
+ 5 cross-check
+ 3 synthesis
+ (3 candidates × 5 judges)
+ 1 completeness critic
+ 1 optional repair
= 40 possible agent calls

Even without the optional repair, the workflow requires 39 calls, so a cap of 36 cannot complete it.

A smaller deterministic reproduction of the accounting problem is:

export const meta = {
  name: 'judge_panel_capacity_repro',
  description: 'Show helper-expanded agent cardinality.'
};

const attempts = await parallel([
  () => agent('Produce candidate A.', { label: 'candidate-a' }),
  () => agent('Produce candidate B.', { label: 'candidate-b' })
]);

// This consumes four more agent slots, not two.
const winner = await judgePanel(attempts, { judges: 2 });
return { winner };

That script requires six agent calls. Setting maxAgents: 5 guarantees a late failure.

Why this is easy for the authoring model to miss

The Ultra directive correctly tells the model to use wider fan-out, more judges, completeness checks, and a high maxAgents matching the plan. However, the workflow-authoring quality-helper reference describes judges without explicitly stating the capacity formula:

judgePanel agent slots = attempts.length × judges

The lifecycle guidance says to bound agents appropriately, but there is no explicit helper-cardinality checklist or preflight warning. The model therefore counted the visible phases but undercounted the calls hidden inside judgePanel.

Expected behavior

An Ultracode-authored workflow should not start when its deterministic planned call count already exceeds its own maxAgents cap.

At minimum, authoring guidance should make helper cardinality explicit so generated scripts calculate the cap correctly. Ideally, the runtime or authoring path could also surface an early warning before a guaranteed-to-fail run spends tokens.

Possible direction

A minimal fix could include:

1. Document that judgePanel consumes attempts.length × judges agent slots, verify consumes its reviewer count, and completenessCheck consumes one slot.
2. Add an authoring/comprehension regression requiring a generated workflow to include helper-expanded calls when choosing maxAgents.
3. Optionally add an early diagnostic when a helper's known expansion cannot fit in the remaining agent capacity.

This is related to #146, but distinct: #146 fixed raising maxAgents during journaled resume. This report concerns preventing Ultracode from generating a workflow with an insufficient cap in the first place.