Redesign live-to-final assistant replies for running agent sessions

[franksong2702]

Summary

Hermes WebUI should redesign the live-to-final assistant reply experience for running agent sessions.

This is one of the most important UX surfaces in Hermes WebUI: when an agent is running, the user needs to understand what is happening now, what has already happened, and where the final answer begins. The current experience has repeatedly exposed edge cases where live progress, tool activity, replay/recovery, auto-compression, and final transcript rendering compete for the same visual surface.

The goal is not just to add a Worklog widget. The goal is to make the assistant reply lifecycle coherent:

• During a running turn, process text should remain the primary timeline.
• Tool activity should be visible, quiet, and close to the prose that caused it.
• When the turn settles, implementation detail should collapse into a compact activity summary above the final answer.
• The final answer must remain readable, complete, and clearly separate from the worklog.
• Recovery, session switching, and replay should reconstruct the same structure rather than creating a different UI state.

Why this matters

Running agent sessions are the core Hermes WebUI experience. Users spend most of their time watching an agent reason, inspect files, run tools, recover from reconnects, and eventually produce a final answer. If this surface is noisy or unstable, the product feels unreliable even when the backend is working correctly.

Mature agent clients such as Codex and Claude Code generally treat intermediate tool/progress details as supporting activity rather than as the final answer itself. They make running state visible, but they do not force every internal lifecycle marker to remain in the final transcript. Hermes WebUI should follow the same product direction while preserving its own stronger browser affordances: replay, session switching, resumability, and structured tool details.

Historical context

This issue intentionally cross-references prior Hermes WebUI problems because this UX has been revisited many times from different angles:

• Early working/final-answer separation: ux(chat): separate working/execution flow from final answer delivery #536 established the direction that working output and final answer should not blur together.
• Manual and automatic compression UX: feat: /compress with focus topic in Web UI (replace /compact) #469, feat: /compress with focus topic in Web UI (replace /compact, add transcript-aware UX) #619, fix(compress): prefer persisted reference handoff after completion #699, Normalize auto-compression transcript cards #1142, fix: compare compression_count against per-turn snapshot to stop repeated banner #1316, bug(streaming): auto-compression rotation leaves reference cards in the active turn #2355, Fix compression reference card anchoring #2357, Auto-compression running state keeps timing until the whole session ends #2973, and Auto-compression queuing is undiscoverable (misleading tooltip, no queue cue, no placeholder hint) #3079 show repeated attempts to represent compression state without confusing the user.
• Live/replay/reconnect durability: [Bug]: Session messages lost when WebUI server restarts during task execution #765, fix: clear stale WebUI stream state #1525, race: _clear_stale_stream_state mutates outside per-session lock (Opus follow-up from v0.50.279) #1533, (bug) Multiple browser sessions #1584, Add WebUI run event journal replay #2283, bug(streaming): show pending user message when reattaching to an active session #2341, Fix pending user row on active reattach #2342, Active agent streams lose and reorder the live timeline after session switches #2344, Preserve live agent timeline across session switches #2347, Live SSE stream does not reattach after session switch — user sees no live tokens until completion #2924, fix: reattach SSE on session-switch return + close leaked stream connections #2925, fix: reattach SSE on session-switch return and preserve live progress (closes #2924) #3005, fix(stream): replay active run activity on reconnect #3371, and streaming: emit SSE id: and honor Last-Event-ID for deterministic resume #3391 cover the long-running theme that a browser client must be able to recover the running transcript from durable stream state.
• Activity/tool/thinking/progress rendering: Bug: Activity panel collapses during streaming and latest user message disappears after Stop #1298, Restore visible interim progress for long WebUI tool runs #3014, and Restore visible WebUI progress contract #3015 cover the need for visible progress while avoiding noisy thinking/tool surfaces.
• Terminal/no-final edge cases: Streaming should show compression-exhausted tool-tail runs as errors #3315 and Fix compression-exhausted stream finalization #3316 show that compression exhaustion and tool-tail termination need explicit terminal classification rather than pretending a turn completed normally.
• Stream ownership and cancellation: cancelStream() should respect active stream ownership and close the SSE source #3344 and fix(runtime): make cancelStream() owner-aware and close its SSE source #3345 cover duplicate ownership/cancel interactions that directly affect whether the user sees the correct active stream.
• Sidebar/session awareness: feat(sessions): session list UX — loading spinner, unread dot, last-reply timestamp #856, Collapse agent session compression chains #1012, fix: persist context_length/threshold_tokens/last_prompt_tokens in Session model #1341, fix: collapse sidebar session lineage rows #1358, fix: expose session lineage metadata in API #1370, bug(context-indicator): older sessions show '100' / '890% used' on load — #1356 fix didn't cover GET /api/session path, falls back to cumulative input_tokens vs 128K default #1436 and related session-list work show that running state also needs to remain coherent when users switch sessions.

These are not separate random bugs. They are symptoms of one product surface: running agent sessions need one consistent live-to-final assistant reply model.

Product model

Live phase

A running assistant turn should show:

• Visible prose/progress as the primary timeline.
• Quiet L2 tool rows near the prose that triggered them.
• Collapsed tool rows/groups by default.
• L3 details only after expansion: full command, args, output, and long payloads.
• A bottom live status/timer within the active assistant turn, not at the top of the transcript.
• Running-only lifecycle markers, such as automatic compression, as transient status rows/dividers rather than persistent transcript content.

Final phase

A settled assistant turn should show:

• One compact L1 Activity summary at the top of the assistant reply.
• The L1 summary collapsed by default.
• Expanded L1 showing prose/tool Worklog details when the user asks for them.
• The final answer below the L1 summary, as normal assistant prose.
• No leftover running-only lifecycle markers that do not help the user interpret the final answer.

Design requirements

1. Process prose is the primary live timeline.
2. Tool rows are supporting activity, visually quieter than prose.
3. Tool groups and individual tool rows are collapsed by default.
4. Full command/output details are L3-only.
5. Final Answer must not be swallowed by the Worklog.
6. Session switching and refresh must rebuild the same live/final structure from durable state.
7. Auto Compression should be visible while it is happening, but should not persist as final transcript text.
8. Recovery-control/internal lifecycle messages must not leak into the visible transcript.
9. Duplicate stream ownership must not create a hidden active stream that steals live/final events.
10. Terminal edge cases should be classified explicitly instead of producing misleading final UI.

Non-goals for the first PR

The first implementation slice does not need to solve every related edge case:

• Queue composer behavior during compression can be handled later.
• A more explicit degraded/rebuild indicator during slow reattach can be handled later.
• Native SSE Last-Event-ID support can remain a follow-up.
• Compression-exhausted/no-final-answer and max-tool-call-limit terminal taxonomy can be refined in follow-up issues/PRs.
• Broader sidebar/session awareness improvements can remain separate.

Acceptance criteria

A first implementation slice is acceptable if it demonstrates:

• Live prose and tool rows interleave in the main assistant timeline.
• Live does not prematurely show the final L1 summary.
• Final shows one L1 Activity summary above the answer.
• L1 is collapsed by default and expands to prose plus tool rows.
• L2 tool rows are quieter than prose and use readable short labels.
• L3 expansion contains full command/args/output.
• Auto Compression shows only as a running/live status and disappears from final settled content.
• Switching away and back to a running session can rebuild prose/tool content from replay rather than showing only an empty Running shell.
• Recovery-control/internal lifecycle messages remain hidden from the visible transcript.
• Duplicate same-session stream starts are prevented or safely reused.

Suggested PR shape

The PR implementing this should stay focused on the live-to-final assistant reply lifecycle. Supporting fixes such as Automatic Compression display and duplicate stream ownership may be included when they are required to make the running-session experience coherent, but they should be framed as supporting edge-case fixes rather than the headline.

Use Refs rather than auto-close language unless the PR fully resolves the broader design space.

[nesquena-hermes]

Thanks — this is a substantial UX design issue, queued under Milestone 2 (Power user features), labeled enhancement / ux.

You're right that the live-to-final lifecycle is one of the most load-bearing surfaces in the UI and that process text, tool activity, replay/recovery, and auto-compression have repeatedly competed for the same visual space (it's the common thread behind several recent streaming bugs — #3315, #3014, the #2361 run-state-consistency umbrella). This deserves a coherent design pass rather than another point fix, so we'll treat it as a design-track item. A short RFC sketch of the proposed reply-lifecycle states (running-process timeline → tool rows → final-answer boundary) would help move it forward when it comes up the queue.

[franksong2702]

Opened #3464（and #3547 which refines it) as the related RFC for this umbrella .

The RFC anchors the design on long-running agent sessions and defines the product semantics for live process text, tool activity, recovery/replay, terminal outcomes, and the final-answer boundary. It also calls out the main long-running edge cases: Auto Compression, very long final answers, tool-call/retry ceilings, no-final/provider failure, reconnect/session switching, and live user intervention.

This is intended as a Refs #3400 design anchor, not as a close for the umbrella. Implementation can continue through #3401 and follow-up slices.

[franksong2702]

Live-to-Final tracking index

This comment is the maintained cross-reference index for #3400. The issue body keeps the umbrella problem statement and historical motivation; this comment tracks the current working map using the structure from the Live-to-Final / Long-Running Session RFC.

Reference points:

• Redesign live-to-final assistant replies for running agent sessions #3400 is the umbrella tracking issue.
• RFC: live-to-final replies for long-running sessions #3464 introduced the parent RFC baseline.
• Redesign live-to-final assistant replies #3401 was the first implementation slice. It was not directly merged; it was absorbed through release-stage PR [HELD — independent review pending] Release v0.51.294 — stage-3401 (live-to-final redesign #3401 + 4 deep-review fixes) #3741, which merged the live-to-final redesign plus additional deep-review fixes.
• docs(rfc): refine live-to-final RFC (fix issue/PR links, add ownership + sidebar sections) #3547 was a closed follow-up refinement PR; keep it as historical context, not an active vehicle.
• docs(rfc): Transparent Stream activity display mode (#3820) #3862 introduced the Transparent Stream display-mode RFC for Major UX regression: restore transparent chronological reasoning/tool-call stream in chat UI #3820, and docs(rfc): mark Transparent Stream RFC Accepted + spike note (#3820) #3864 marked it Accepted. feat: add Transparent Stream activity display #4024 merged on 2026-06-13 and Major UX regression: restore transparent chronological reasoning/tool-call stream in chat UI #3820 closed; bug(transparent-stream): agent text rendered above the full tool-call stack, buried on long turns #4096 later found a live Transparent Stream ordering regression that was fixed by fix(transparent-stream): keep live prose chronological, don't fold into top Worklog rail (#4096) #4158 and shipped in the v0.51.405 line.
• docs(rfc): add stable assistant turn anchors #3927 merged the Stable Assistant Turn Anchors RFC for RFC: Stable Assistant Turn Anchors for Live-to-Final rendering #3926. Stable assistant turn anchors phase 0 scaffold #3962, Add assistant turn anchor source normalizer #3980, Add assistant turn anchor registry and shadow source harness #4037, Add assistant turn anchor settled final projection #4092, Project assistant turn anchor activity scene rows #4093, Stable Assistant Turn Anchors: live shadow feed #4108, and Add assistant turn anchor dual-run reconciler #4120 have all shipped as inert / shadow / reconciliation foundation slices. Stabilize Compact Worklog with assistant turn anchors #4411 is the current active Compact Worklog renderer-integration vehicle on that anchor-normalized path. Future renderer replacement should continue through assistant-turn anchors instead of growing another renderer-local normalizer.
• Preserve live stream output across session switches #3427 was absorbed and shipped through release PR Release LQ — v0.51.353 (#3427 cross-client live-turn recovery) #3949 / v0.51.353; keep it as historical replay/reconnect infrastructure, not an active PR vehicle.
• Handle tool iteration limit as explicit terminal state #3823 merged through release PR Release MS (v0.51.380): tool-iteration-limit stops surfaced explicitly (#3821) #4099 on 2026-06-13, closing Handle max-iteration stops as tool_limit_reached instead of rendering synthetic summary prompts as user messages #3821; keep the tool-limit terminal-state work as historical shipped context.
• Hide Worklog reasoning when Thinking is disabled #3971 remains open as a stale/conflicting PR head, but the underlying Hide thinking does not hide Worklog/tool activity, and visible progress entries are hard to distinguish from final assistant output #3903 fix shipped through release PR Release MO (v0.51.376): Hide Thinking also hides Worklog reasoning (#3903) #4090 / v0.51.376; do not keep Hide Worklog reasoning when Thinking is disabled #3971 as an active code vehicle unless a maintainer explicitly wants the stale branch refreshed.
• State below was last checked on 2026-06-20. Keep Active limited to open issues/PRs that directly affect the Live-to-Final / Long-Running Session lifecycle.

Active tracking items

These are the currently open issues/PRs that should stay visible while #3400 remains open.

Track	Open issues / PRs	Tracking note
Parent product model	#3400	Umbrella for the long-running live-to-final reply lifecycle: live process text, supporting activity, terminal outcome, final answer, replay/recovery, and ownership.
First reply lifecycle / anchor-renderer integration	#4411	Current active implementation PR for Compact Worklog through assistant-turn anchors. It keeps final answer separate from supporting activity and covers live, settled, hard-refresh, session re-entry, cancel, and compression-adjacent Worklog rebuild behavior. Current state: open, non-draft, clean/mergeable on head 24f63b66; Browser smoke and Tests workflows passed on 2026-06-20 after the latest base refresh; next action is maintainer approval / merge gate, not scope expansion.
Visible interim progress / post-landing validation	#3014, #2403, #4295	#3014 and #2403 remain validation signals for whether long tool-heavy turns surface enough process prose. #4295 is the direct live-reading regression report: the viewport jumps upward while reading an in-progress streamed reply.
Live reading position / transcript virtualization scroll stability	#4207, #4346	#4207 reports completion/read-position jumps and remains needinfo. #4346 tracks the Phase B root-cause fix for variable-height anchor oscillation in transcript virtualization. Closed scroll/virtualization PR experiments are recorded in Historical below.
Long-running cost / runaway safety	#2957	Suspicious long-running sessions may need an opt-in operator pause/review point without making normal long sessions feel broken. Closed cost-protection PRs are recorded in Historical below.
Auto Compression / context-window / message visibility	#3133	#3133 remains the open reporter signal for post-compression message disappearance / context-count mismatch. #4248 is closed after the merged #4418 fix and is recorded in Historical below.
Replay, reconnect, session switch, and reattach	#1585	Refresh, reconnect, session switch, and terminal settle should rebuild or preserve the same lifecycle from durable/live state. Recent closed reconciliation PRs are recorded in Historical below.
Sidebar/session ownership and terminal indicator repair	#4451	#4354/#4355 are now closed and moved to Historical. #4451 is the current open sidebar-lineage streaming-state PR; it bubbles hidden lineage stream state to the visible parent but is dirty/conflicting, so the next action is conflict refresh rather than approval.
Pending intent / live user control	#3058, #2555, #3813, #3859	Queue / Steer / Stop-and-send / Interrupt semantics belong to the child control-surface lane. #3813 remains open as continuous observability vs approval gating feedback even after Transparent Stream shipped; #3859 remains a busy-composer control-surface signal.
Thinking / activity / final-answer boundary	#536	Working output, reasoning/activity, and final answer should remain distinct. Reasoning or activity should not be the only visible representation of a running or settled assistant reply. Closed thinking/final-answer visibility regressions are recorded in Historical below.
Cancel / stream ownership / partial-turn preservation	#4076	User-initiated cancel should preserve partial assistant text, reasoning, and tool calls when available. #4076 is still needinfo / repro-localization, not an implementation PR.

Historical / completed references

These are closed, merged, superseded, or historical signals. They remain useful context for why the umbrella exists, but they are not active tracking items.

Track	Historical references	Why they remain linked
Parent RFC history	#3464, #3547	#3464 landed the first RFC revision. #3547 was a closed refinement PR/discussion and should not be treated as an active vehicle.
First implementation / progress history	#3401, #3741, #3015	#3401 was absorbed through merged release-stage PR #3741. #3015 was the earlier progress-contract PR and was absorbed by the first implementation slice.
Long-running cost / runaway safety history	#2961	#2961 was a closed cost-protection pause PR. Keep it as historical design context while #2957 remains the open product tracking issue.
Stable anchor RFC / implementation history	#3926, #3927, #3962, #3977, #3980, #4037, #4092, #4100, #4093, #4101, #4108, #4115, #4120	#3927 merged the Stable Assistant Turn Anchors RFC. #3962 shipped the inert Phase 0 scaffold through release PR #3977 / v0.51.359. #3980 merged the inert Slice 2 normalizer and #3926 closed. #4037 merged the inert registry/shadow harness. #4092 shipped settled final-answer projection, #4093 shipped inert activity-scene projection, #4108 shipped the live shadow feed, and #4120 merged the inert dual-run reconciler. #4411 is the current Active renderer-integration vehicle rather than historical context.
Transparent Stream RFC / implementation and regression history	#3820, #3862, #3864, #4024, #4096, #4114, #4158	#3862 added the Transparent Stream display-mode RFC for #3820, #3864 marked it Accepted, and #4024 shipped the opt-in Transparent Stream activity display. #4096 later reported live prose bunching above tool stacks; #4114 was superseded, and #4158 fixed the actual live-path root cause in the v0.51.405 line.
Worklog defaults / settings alignment	#3868, #3902	#3868 aligned the settings surface with live-to-final model defaults and migration behavior. It shipped through release PR #3902 / v0.51.347. It is partial cleanup, not a full Transparent Stream implementation.
Pending-intent RFC, queue durability, and approval-card fallback history	#3061, #3839, #3906, #3917, #3108, #720, #965, #1062, #4107, #4111	#3061's pending-intent controls RFC landed through release PR #3839 / v0.51.328. #3906's Queue / draft restore durability fix shipped through release PR #3917 / v0.51.348, and #3108 is now closed. #4107's approval-card SSE/polling fallback reliability issue was fixed by #4111 and shipped in the v0.51.413 line. Active control-surface work remains #3058, #2555, #3813, and #3859.
Tool-limit and max-iteration terminal state	#3821, #3823, #4099	#3823 shipped through release PR #4099 on 2026-06-13 and closed #3821. Keep this as the shipped WebUI defensive compatibility line for limit reached state, control prompt visibility, and no fake final answer.
Interrupt successor guard history	#3822, #3808, #3836	#3822 was absorbed by release PR #3836 / v0.51.327. #3808 is closed after that fix. Keep as historical control-surface bug signal context.
Auto Compression and terminal/no-final history	#3315, #3316, #469, #2973, #3079, #4248, #4418	#3316/#3315 shipped through the v0.51.292 release line and remain context for compression-exhausted / tool-tail terminal states. Prior compression UX work explains why compression is treated as lifecycle status rather than final-answer content. #4248 was closed after merged PR #4418 fixed the large-context-window reload regression; current compression residuals are tracked above through #3133.
Replay / reconnect infrastructure	#2283, #2924, #3391, #3668, #3675, #3724, #3707, #3763, #3766, #3877, #3885, #3892, #3902, #3427, #3949, #4208, #4216	Prior recovery/reconnect work and issues that shaped the current recovery expectations. #3427's cross-client runtime_journal_snapshot live-turn recovery shipped through #3949 / v0.51.353. #4208 and #4216 are now closed-unmerged transcript/session reconciliation PRs, so they are not Active vehicles. #3877 is closed after #3892 preserved the live assistant turn DOM node across mid-stream renderMessages() rebuilds; #3892, #3885, and related streaming/render reliability fixes shipped through release PR #3902 / v0.51.347. #3763 was closed after being absorbed by merged release PR #3766 / v0.51.309, which closed #3707 and fixed the restored-live-tool-card reconnect residual.
Cancel / stream ownership history	#3344, #3345, #3475, #3476	Prior cancel/ownership issues and PRs. Keep as context for remaining ownership and terminal-state behavior.
Final answer readability / scroll stability history	#3545, #4110, #4367, #4368, #4383, #4384	Settling behavior after completion includes the long-stream scroll behavior fix and is part of the first live-to-final implementation chain. #4110 and the #4346 virtualization PR set (#4367/#4368/#4383/#4384) are closed-unmerged experiments; current scroll/read-position residuals are tracked above through #4207 / #4346.
Thinking / activity / final-answer boundary	#2492, #3599, #3869, #3876, #3886, #3971, #4090	Closed final-answer/thinking visibility signals, stale thinking-dot placeholder regression, and the show_thinking=false Worklog reasoning leak. #2492 is closed and kept as historical context. #3876 was absorbed and shipped through release PR #3886 / v0.51.341, closing #3869. #3971 remains a stale open PR head, but the underlying #3903 fix shipped through release PR #4090 / v0.51.376.
Artifact and output handoff history	#2655, #2673, #2881, #2938, #3329, #3348, #3528	Existing Artifacts and workspace:// surfaces are supporting context for long-running output handoff; this umbrella does not own rich artifact rendering.
Sidebar / session metadata history	#856, #1370, #1436, #4354, #4355	Prior sidebar/session metadata work that informs active-session ownership expectations. #4354/#4355 are closed stuck-Running terminal-indicator history; #4451 is the current open follow-up vehicle.
Activity / thinking rendering history	#1298	Earlier activity-panel behavior that informs the final-answer boundary.

Maintenance guidance:

• Keep Redesign live-to-final assistant replies for running agent sessions #3400 open as the umbrella until the product lifecycle, first implementation, and key follow-up slices are all settled.
• Add new items to Active tracking items only when they are open and directly affect the Live-to-Final / Long-Running Session lifecycle.
• Move items to Historical / completed references when they merge, close, or become superseded.
• Do not expand this umbrella to own unrelated Worklog, artifact rendering, sidebar IA, or control-surface semantics. Those can be linked here only when they affect the live-to-final lifecycle.