fix(agent): ignore Hermes INFO echoes that falsely fail completed runs (#5862)

[YikaJ]

What does this PR do?

Stops Multica from treating Hermes [INFO] root: conversation/tool echoes as provider failures.

Fixes #5862

Real error vs pollution error

This PR is specifically about pollution errors, not real Hermes/provider crashes.

Class	Meaning	Expected status	Observed before fix
Real error	Hermes/provider truly fails (rate limit, auth, AF_UNIX, timeout). No successful final reply.	failed	Correctly failed
Pollution error	Hermes already finished and emitted a correct final reply / side effect; Multica then suddenly flips the same run to failed because non-error stderr data matched error-looking tokens.	completed	Incorrectly failed

Pollution is characterized by timing:

1. run-messages already contains a successful final reply.
2. the task then suddenly gets status=failed.
3. the stored error is truncated conversation / tool / INFO JSON, not a short provider message.

Production evidence

• Real failures are short and self-contained (41–95 bytes), e.g. AF_UNIX path too long, HTTP 400: ..., HTTP 429: ..., Request timed out.
• Pollution failures are larger (265–87,286 bytes), contain conversation fingerprints (finish_reason, session_id, tool_call_id, [INFO] root:), and often start mid-JSON.
• Sampled polluted runs had already completed their requested work and only failed at final status promotion.
• The same pollution shape appears on at least two Hermes agents.
• A length-only guard is insufficient: shortest polluted sample observed was 265 bytes.

Root cause

Hermes writes conversation/tool JSON to stderr as:

YYYY-MM-DD HH:MM:SS [INFO] root: {...}

Those INFO records are data. Their payload may contain Error:, KeyError:, ❌, HTTP 429, or API call failed. Because one JSON record is one physical line, unrelated payload fragments can satisfy both the sniffer capture filter and terminal condition. After #2323 removed the empty-output precondition, that false terminal match overrides a valid completed turn.

Changes

• Ignore Hermes [INFO] root: records before provider-error matching.
• Keep acpMaxErrorLineLen as defense in depth for oversized echoes with an unexpected prefix.
• Keep genuine [ERROR] root: / short provider-error behavior.
• Add English-only regression cases covering four pollution shapes, plus a control case for real [ERROR] root: records.

How to test

cd server
go test ./pkg/agent/ -run 'TestHermesProviderErrorSniffer(IgnoresEchoedInfoRecords|StillCapturesErrorRootRecords)' -v
go test ./pkg/agent/ -run TestHermesProviderErrorSniffer
go test ./pkg/agent/
go vet ./pkg/agent/

Type of Change

• Bug fix (non-breaking change that fixes an issue)

[vercel]

@YikaJ is attempting to deploy a commit to the IndexLabs Team on Vercel.

A member of the Team first needs to authorize it.

[Bohan-J]

Thanks @YikaJ for the excellent diagnosis and the initial fix here 🙏

The pollution-vs-real-error framing, the production evidence, and the deterministic regression fixtures made this bug much easier to reason about — that's a high-quality report.

Since this is a live-impacting bug, we pushed the remaining boundary fixes directly on top of your branch rather than round-tripping review, so we could land it quickly:

• skip INFO/DEBUG root echoes via a structural JSON state machine (single- and multi-line records), instead of a prefix-only guard;
• keep genuine bare provider errors (⚠️/❌/📝 Error:) and non-root [ERROR] records, including when they follow a truncated INFO record;
• bound only the persisted error summary by length, never the error classification.

CI is green and the review is clear — merging now. Really appreciate the contribution!