Skip to content

Add recoverable InvalidArgsPolicy for schema-validation failures#42

Merged
senamakel merged 2 commits into
tinyhumansai:mainfrom
M3gA-Mind:harness/invalid-args-policy
Jul 8, 2026
Merged

Add recoverable InvalidArgsPolicy for schema-validation failures#42
senamakel merged 2 commits into
tinyhumansai:mainfrom
M3gA-Mind:harness/invalid-args-policy

Conversation

@M3gA-Mind

@M3gA-Mind M3gA-Mind commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Add InvalidArgsPolicy { Fail, ReturnToolError } on RunPolicy, mirroring UnknownToolPolicy, so a schema-validation failure on a registered tool call no longer hard-aborts the turn. Previously validate_call(call)? propagated TinyAgentsError::Validation out of the agent loop; a missing required field, wrong type, or bad enum killed the run with no way for the model to self-correct.

Under ReturnToolError, admission emits AgentEvent::InvalidToolArgs and returns a recoverable tool-error result carrying the validation detail + expected schema, consuming one tool-call budget slot (mirrors the existing unknown-tool recovery). Default stays Fail (backward-compatible).

Validation: cargo fmt clean; cargo test --lib green (996 passed, incl. a new recoverable-path test and the unchanged Fail baseline). Note: cargo clippy --lib --tests -D warnings currently fails on two pre-existing, unrelated lints under Rust 1.93.0 in files this PR does not touch (collapsible_if in src/harness/model/mod.rs, duplicated attribute in src/harness/stream/mod.rs) — reproducible on pristine b1f0d82; this diff introduces zero clippy warnings.

Closes tinyhumansai/openhuman#4639

Summary by CodeRabbit

  • New Features
    • Added configurable handling for invalid tool arguments, letting runs either stop on validation errors or return a tool error so the model can retry.
    • When recoverable handling is enabled, validation details are now included in the conversation for self-correction.
  • Bug Fixes
    • Improved agent loop behavior for malformed tool calls, preventing invalid inputs from being silently dropped and helping recover successful responses.

A schema-validation failure on a registered tool call previously
propagated TinyAgentsError::Validation out of the agent loop and
aborted the whole turn, with no recoverable analogue to
UnknownToolPolicy. A missing required field, wrong type, or bad enum
killed the run instead of letting the model self-correct.

Add InvalidArgsPolicy { Fail, ReturnToolError } on RunPolicy, mirroring
UnknownToolPolicy. Under ReturnToolError, admission emits an
AgentEvent::InvalidToolArgs and returns a recoverable tool-error result
carrying the validation detail and the expected schema, consuming one
tool-call budget slot. Default stays Fail (backward-compatible).
@coderabbitai

coderabbitai Bot commented Jul 7, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 30 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 59f0b307-307e-4305-b6b8-0ce6bd07ab15

📥 Commits

Reviewing files that changed from the base of the PR and between 4bad0b1 and dad42c3.

📒 Files selected for processing (2)
  • src/harness/model/mod.rs
  • src/harness/stream/mod.rs
📝 Walkthrough

Walkthrough

This PR adds a new InvalidArgsPolicy enum (Fail default, ReturnToolError) to RunPolicy, updates admit_tool_call to branch on this policy instead of hard-failing on schema-validation errors, adds an AgentEvent::InvalidToolArgs variant, and includes a regression test.

Changes

InvalidArgsPolicy Recovery Flow

Layer / File(s) Summary
Policy contract
src/harness/runtime/types.rs
Adds InvalidArgsPolicy enum with Fail (default) and ReturnToolError variants, extends RunPolicy with an invalid_args field, and initializes it in RunPolicy::default().
Event variant
src/harness/events/types.rs
Adds AgentEvent::InvalidToolArgs { call_id, tool_name, arguments, error, recovery } and maps it to "tool.invalid_args" in kind().
Validation branching in admit_tool_call
src/harness/agent_loop/tools.rs, src/harness/agent_loop/mod.rs
admit_tool_call branches on InvalidArgsPolicy: Fail propagates the validation error, ReturnToolError emits InvalidToolArgs, updates status, and returns an ErrorMessage resolution instead of aborting.
Regression test
src/harness/agent_loop/test.rs
Adds a test exercising InvalidArgsPolicy::ReturnToolError via StrictLookupTool, asserting recovery, no tool invocation, and injected validation error text.

Estimated code review effort: 2 (Simple) | ~15 minutes

Possibly related issues

Poem

A tool call stumbled, args gone wrong,
but now the loop just hums along! 🐇
No more hard-abort, no more dread—
"ReturnToolError" instead!
Hop, retry, and self-correct,
this bunny's build is now checked. ✅

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately describes the main change: recoverable handling for invalid tool arguments.
Linked Issues check ✅ Passed The changes match the linked issue requirements, including the new policy, recoverable error path, budget handling, and test coverage.
Out of Scope Changes check ✅ Passed The diff appears focused on the requested InvalidArgsPolicy feature and its test coverage, with no clear unrelated additions.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/harness/agent_loop/tools.rs (1)

203-231: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value

Solid mirror of the unknown-tool recovery path; minor: tool.schema() computed twice.

The branching logic correctly consumes the budget slot already reserved at line 140, emits InvalidToolArgs with the validation detail and schema, and avoids invoking the tool — matching the PR's requirements and the UnknownToolPolicy pattern above.

One small nit: tool.schema() is called once for validate_call (line 203) and again for .parameters (line 216), rebuilding the schema object twice on the error path. Not a functional issue, but avoidable.

♻️ Cache the schema lookup
-        if let Err(err) = tool.schema().validate_call(call) {
+        let schema = tool.schema();
+        if let Err(err) = schema.validate_call(call) {
             // The model called a registered tool with schema-invalid arguments.
             // Apply the run's `InvalidArgsPolicy` instead of unconditionally
             // aborting the turn (mirrors the unknown-tool recovery above).
             if matches!(self.policy.invalid_args, InvalidArgsPolicy::Fail) {
                 return Err(err);
             }
             ...
-            let schema_repr = serde_json::to_string(&tool.schema().parameters)
+            let schema_repr = serde_json::to_string(&schema.parameters)
                 .unwrap_or_else(|_| "<unserializable>".to_string());
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/harness/agent_loop/tools.rs` around lines 203 - 231, The error path in
tools.rs recomputes the tool schema twice, once for validate_call and again when
formatting the expected schema for the recovery message. Cache the result of
tool.schema() in the InvalidArgs handling block and reuse it for both validation
and schema serialization so the error path avoids rebuilding the schema object.
Keep the existing InvalidToolArgs event emission and ErrorMessage flow in the
same branch.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/harness/agent_loop/tools.rs`:
- Around line 203-231: The error path in tools.rs recomputes the tool schema
twice, once for validate_call and again when formatting the expected schema for
the recovery message. Cache the result of tool.schema() in the InvalidArgs
handling block and reuse it for both validation and schema serialization so the
error path avoids rebuilding the schema object. Keep the existing
InvalidToolArgs event emission and ErrorMessage flow in the same branch.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 24b930c6-6c48-4889-a6f4-2a981af810db

📥 Commits

Reviewing files that changed from the base of the PR and between b1f0d82 and 4bad0b1.

📒 Files selected for processing (5)
  • src/harness/agent_loop/mod.rs
  • src/harness/agent_loop/test.rs
  • src/harness/agent_loop/tools.rs
  • src/harness/events/types.rs
  • src/harness/runtime/types.rs

collapsible_if in StreamAccumulator::push_tool_chunk and a duplicated
#[cfg(test)] on the stream test module both fail CI's
cargo clippy --all-targets -- -D warnings under rust 1.96. Behavior-neutral.
@M3gA-Mind

Copy link
Copy Markdown
Contributor Author

CI guardian: the Rust SDK check was red on a pre-existing main clippy failure under stable rust 1.96 (a collapsible_if in StreamAccumulator::push_tool_chunk and a duplicated #[cfg(test)] on the stream test module), inherited by this branch. Pushed a behavior-neutral lint fix so CI can go green; no change to this PR's own logic.

@senamakel senamakel merged commit 4b3c588 into tinyhumansai:main Jul 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[TinyAgents][T2] Schema-validation failure hard-aborts the turn — add recoverable InvalidArgsPolicy (#4451 core)

2 participants