feat(harness): no-progress escalation primitive (NoProgressTracker)

[M3gA-Mind]

Summary

• Adds harness::no_progress — a reusable, harness-agnostic detector that breaks the "retry the same failing tool call" loop. It tracks recent (tool, args, error) outcomes across a turn and returns a Continue / Nudge / Halt verdict.
• Escalation ladder caps same-strategy retries before halting: an identical repeated failure first nudges the model to change approach (a structured "no progress since step X" signal), and only halts if it keeps re-issuing the same failing call. A varied-failure backstop catches "different commands all failing," and a hard-policy-reject rung trips fastest.
• Pure state machine (no harness types), so it is unit-testable in isolation and reusable by higher-level reliability layers. A driver (typically a middleware) feeds each outcome into NoProgressTracker::record and maps the verdict onto steering (nudge) or a halt.

Context

Extracted from OpenHuman (tinyhumansai/openhuman#4389) as part of converging generic harness mechanics into this crate. After the OpenHuman → tinyagents harness migration, loop-control behavior like this belongs here so every crate consumer gets it; the app-specific driver (wiring the verdict onto steering) and policy stay in the consumer.

Public API

• NoProgressTracker::new(identical_halt_threshold) / record(step, &ToolAttempt) -> NoProgress / reset()
• ToolAttempt<'a> — { tool, arg_fingerprint, error, hard_reject, recoverable_miss }
• enum NoProgress { Continue, Nudge(String), Halt(String) }
• DEFAULT_IDENTICAL_HALT_THRESHOLD

Re-exported from harness. Additive only — no existing APIs change.

Tests

7 unit tests: nudge→halt on identical repeats, a success resets the ladder, changed-args clears the identical streak, hard-reject halts on the 2nd repeat, varied-failure nudge→backstop, unknown-tool recovery excluded from the backstop, and the halt-threshold clamp (nudge always precedes halt).

Summary by CodeRabbit

• New Features

  • Added a new progress-tracking capability for tool usage that can continue, prompt a change in approach, or stop after repeated failures.
  • Introduced clearer user-facing messages when progress stalls, including step-based guidance and concise error details.

• Bug Fixes

  • Improved handling of repeated identical failures, successful retries, and recoverable missing-tool cases so progress decisions are more consistent.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds a new no_progress harness submodule implementing a NoProgressTracker that records tool call outcomes and escalates through Continue, Nudge, and Halt verdicts based on identical-retry and varied-failure thresholds. The harness module publicly re-exports the new types and constant.

Changes

No-progress tracker feature

Layer / File(s)	Summary
Types, constants, and module wiring src/harness/no_progress.rs, src/harness/mod.rs	Adds module docs, threshold constants, ToolAttempt struct, NoProgress enum, and declares/re-exports the new submodule and its public types in the harness module.
Tracker construction and decision logic src/harness/no_progress.rs	Implements LadderState, NoProgressTracker::new/reset with threshold clamping, and the record method deciding Continue/Nudge/Halt based on identical-repeat and consecutive-failure counters, hard rejections, and recoverable misses.
Message formatting and tests src/harness/no_progress.rs	Adds private helpers for nudge/halt message text and error truncation, plus a unit test suite covering retry, reset, rejection, varied-failure, and clamping scenarios.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant NoProgressTracker
  participant LadderState

  Caller->>NoProgressTracker: record(step, attempt)
  NoProgressTracker->>LadderState: update streak counters
  alt success
    NoProgressTracker->>LadderState: reset()
    NoProgressTracker-->>Caller: Continue
  else hard_reject or identical retries exhausted
    NoProgressTracker->>LadderState: reset()
    NoProgressTracker-->>Caller: Halt(message)
  else identical cap reached or varied failures accumulate
    NoProgressTracker-->>Caller: Nudge(message)
  else
    NoProgressTracker-->>Caller: Continue
  end

Loading

Poem

A rabbit tracks each hop and try,
when steps repeat, I raise an eye 🐇
Nudge once, then halt if naught's begun,
no wasted hops beneath the sun.
Hop, retry, and reset clean —
the tidiest ladder you've ever seen!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: a new harness no-progress escalation primitive centered on NoProgressTracker.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

src/harness/mod.rs (1)

27-27: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider making no_progress private to keep a single public path.

pub mod no_progress combined with the flat pub use re-export exposes the same types via two public paths (harness::no_progress::NoProgress and harness::NoProgress). Since the intended surface is the flat re-export, dropping pub on the module keeps the API to the smallest useful set.

As per coding guidelines: "The module root should wire the pieces together and expose the smallest useful API."

♻️ Proposed change

-pub mod no_progress;
+mod no_progress;

Also applies to: 52-54

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/harness/mod.rs` at line 27, The `no_progress` module is being exposed
twice, once through `harness::no_progress` and again via the flat re-export, so
reduce the public API surface by making `no_progress` private in `harness::mod`
and keeping the existing `NoProgress` re-export as the single public entry
point. Update the module declaration and preserve the current re-export wiring
so callers continue using `harness::NoProgress` while `no_progress` remains
internal.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/harness/mod.rs`:
- Line 27: The `no_progress` module is being exposed twice, once through
`harness::no_progress` and again via the flat re-export, so reduce the public
API surface by making `no_progress` private in `harness::mod` and keeping the
existing `NoProgress` re-export as the single public entry point. Update the
module declaration and preserve the current re-export wiring so callers continue
using `harness::NoProgress` while `no_progress` remains internal.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: cefa8524-f71a-4e1e-be77-e7aa18e6789b

📥 Commits

Reviewing files that changed from the base of the PR and between 0b91c71 and be5e4b8.

📒 Files selected for processing (2)

• src/harness/mod.rs
• src/harness/no_progress.rs