Skip to content

Shave streaming latency — chat should feel token-by-token live #4

Description

@8crsk

Chat already renders streamed deltas, but there's perceptible latency between the gateway emitting event:"agent" frames and pixels moving. Goal: the moment the model starts talking, the UI does too.

Where to look

  • data/network/ws/ChatSession.kt — demuxes agent frames into ChunkEvents
  • ui/chat/ChatViewModel.kt — accumulates TextDeltas
  • Compose recomposition frequency vs per-delta emission (batching? snapshotFlow? throttle window?)

Acceptance criteria

  • Measurable reduction in first-token-to-first-pixel and steady-state jank (profile before/after)
  • No dropped/reordered deltas (unit test on ChunkEvent stream)
  • Battery/CPU not worse

Metadata

Metadata

Assignees

No one assigned

    Labels

    ⚔️ questA meaty feature with clear scope — real XP

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions