Restore visible interim progress for long WebUI tool runs

[franksong2702]

Problem

Long WebUI sessions that spend most of their time in tool calls can appear silent even while Hermes is actively working. This is especially painful for long-running sessions and for users who switch between sessions: if no visible interim text is emitted, the browser can look stuck even though reasoning and tool events are moving underneath.

This is not the same as the SSE reattach/session-switching bug. Reattach can preserve visible text once it exists, but it cannot help when a run emits all progress only through reasoning/tool channels and no normal assistant content is produced until much later.

Why this matters

For long WebUI work, visible interim progress is part of the user-facing contract:

• Users need to know the run is still alive before the final answer arrives.
• Session switching should not make a long run feel blank or frozen.
• Some model/provider combinations are more likely to keep intermediate work inside reasoning/tool channels unless the WebUI instruction is explicit.
• The UI must not expose chain-of-thought or raw tool logs, but it should receive concise normal assistant text such as what phase is happening or what was found.

History and evidence

Relevant history from the repository:

• 2026-05-16 2dfe3ffb introduced a strong WebUI visible-progress contract: for multi-step tool work, the model should provide brief user-visible progress updates as normal assistant content before continuing with tool calls.
• 2026-05-18 PR fix(chat): align WebUI context with messaging sessions #2547 / 540292a7 softened that language from a contract to guidance and changed the key instruction to "you may provide" visible progress updates, while also asking the model not to add extra browser-only status chatter.
• 2026-05-25 5b9484b8 only suppresses duplicate reasoning/thinking echoes after text was already streamed as visible assistant output. It does not suppress normal assistant output and is not the root cause of invisible progress.

Recent local 8787 dogfood showed the practical impact: one long tool-heavy run had reasoning/tool activity but no token or interim_assistant events for a meaningful period, and another run did not emit its first visible token until after a large batch of tool events. The browser experience was "nothing is coming out" even though the backend was working.

Expected behavior

For long multi-step WebUI tasks that use tools, Hermes should instruct capable models to emit concise visible interim progress as normal assistant content before the next tool batch or after a meaningful finding.

The guidance should also make the boundaries explicit:

• Do not reveal chain-of-thought, hidden reasoning, secrets, or raw tool logs.
• Do not make direct or very short answers artificially chatty.
• Do not treat reasoning/thinking/tool-result streams as a substitute for visible interim assistant text.

Proposed scope

Restore a firmer WebUI progress contract in the streaming prompt and add regression coverage that prevents the instruction from drifting back to optional/silent wording.

This should not change SSE reconnect behavior, persisted session replay, or reasoning echo suppression.

Acceptance criteria

• The WebUI progress prompt uses explicit "provide/send" style language rather than optional "may" language for long tool-running turns.
• The prompt tells models not to keep all progress only in reasoning/thinking/tool-result channels.
• Existing safeguards against chain-of-thought/raw tool log leakage remain present.
• Static tests lock the wording/intent so future tone-parity work cannot accidentally remove visible interim progress again.

[nesquena-hermes]

Summary

Read the issue body, the cited SHAs (2dfe3ffb, 540292a7, 5b9484b8), and the current prompt at api/streaming.py:193-201 on origin/master. The diagnosis is accurate: the WebUI progress prompt was softened from "provide" to "you may provide" in PR #2547 (commit 540292a7, 2026-05-18), and that wording change is the proximate reason long tool-running turns can look silent.

Code reference

The current prompt text at api/streaming.py:193-201:

_WEBUI_PROGRESS_PROMPT = """
WebUI progress guidance:
- Match the normal Hermes messaging style; do not add extra status updates solely because this is a browser session.
- For long multi-step work that uses tools, you may provide brief user-visible progress updates before continuing with tool calls.
- Each update should say what you are about to check, what you just confirmed, or why the next tool call is needed.
- Keep updates concise, factual, and in the user's language. One or two short sentences are enough.
- Do not reveal hidden reasoning, chain-of-thought, private scratchpads, secrets, raw logs, or long tool output.
- For direct answers or very short tasks, skip progress updates and answer normally.
""".strip()

The diff that softened it (git show 540292a7 -- api/streaming.py):

- For multi-step work that uses tools, provide brief user-visible progress updates as normal assistant content before continuing with tool calls.
+ For long multi-step work that uses tools, you may provide brief user-visible progress updates before continuing with tool calls.

So the rename from _WEBUI_VISIBLE_PROGRESS_PROMPT → _WEBUI_PROGRESS_PROMPT was part of the same softening; the suppression at 5b9484b8 is a separate concern (only suppresses echoes after visible text has streamed, and runs in _is_visible_output_echo at api/streaming.py:3933-3940 — not the cause of invisible progress).

Existing related PR

There is already a draft PR addressing this exact symptom: #3015 — "Restore visible WebUI progress contract" by @franksong2702. It is currently in draft state and modifies _WEBUI_PROGRESS_PROMPT plus regression coverage at tests/test_sprint42.py. Worth coordinating with that PR rather than opening a second one — the maintainer can mark it ready when the wording is settled.

Recommendation

If #3015's wording is close to what this issue is asking for, the cleanest path is:

1. @franksong2702 marks Restore visible WebUI progress contract #3015 ready for review.

2. The PR pins the contract strength in tests/test_sprint42.py so future tone-parity passes can't silently revert it. The original 2dfe3ffb wording was:

   For multi-step work that uses tools, provide brief user-visible progress updates as normal assistant content before continuing with tool calls.

   A static assert like assert "provide brief user-visible progress updates as normal assistant content" in api.streaming._WEBUI_PROGRESS_PROMPT would lock the imperative form.

3. Keep the existing "do not reveal hidden reasoning / chain-of-thought / secrets / raw logs / long tool output" line — that's the safety boundary the acceptance criteria explicitly call out.

4. Add an additional line addressing the "do not keep all progress only in reasoning/thinking/tool-result channels" intent the issue spells out — that wasn't in the pre-softening text and is the new acceptance-criterion bit.

A suggested rewrite for _WEBUI_PROGRESS_PROMPT:

WebUI progress contract:
- For long multi-step work that uses tools, provide brief user-visible progress updates as normal assistant content before continuing with tool calls.
- Do not keep all intermediate progress inside reasoning, thinking, or tool-result channels — surface it as normal assistant text.
- Each update should say what you are about to check, what you just confirmed, or why the next tool call is needed.
- Keep updates concise, factual, and in the user's language. One or two short sentences are enough.
- Do not reveal hidden reasoning, chain-of-thought, private scratchpads, secrets, raw logs, or long tool output.
- For direct answers or very short tasks, skip progress updates and answer normally.

That swaps "guidance" → "contract", swaps "you may provide" → "provide", and adds the channel-suppression line the acceptance criteria asks for. The two terminal bullets (privacy boundary + short-answer carve-out) are unchanged.

Scope check

The acceptance criteria explicitly carve out SSE reconnect, persisted session replay, and reasoning-echo suppression. None of those touch _WEBUI_PROGRESS_PROMPT — the suppression at 5b9484b8 lives in _is_visible_output_echo/stream_delta_callback and only fires after visible text already streamed, so toughening this prompt does not interact with that path. Safe to limit the change to the prompt + a regression test.

[nesquena-hermes]

Cross-linking: the "Request timed out" symptom that several users hit alongside this was a separate connection-pool-exhaustion regression, now fixed in v0.51.348 (#3913 — approval/clarify moved off persistent SSE to HTTP polling, freeing browser connection slots).

This issue itself is the broader feature ask (restoring visible interim progress for long tool runs), which is distinct from the timeout regression — leaving it open. If the timeouts were the main pain point for you, v0.51.348 should resolve them; the interim-progress UX remains tracked here.