Proposal: Add built-in support for AI streaming queries (partial responses, token-level updates, abort, and resume)

[elunno]

Proposal: Add built-in support for AI streaming queries (partial responses, token-level updates, abort, and resume)

Description:
Modern AI APIs (OpenAI, Anthropic, Google, AWS, Vercel AI SDK, etc.) use streaming responses that deliver output chunk-by-chunk. Many production AI apps need to:

• Show partial tokens as they arrive
• Update UI continuously during the stream
• Support cancel mid-stream
• Support resume on the same query key
• Cache the final output while ignoring partial chunks
• Integrate seamlessly with useQuery/useMutation

TanStack Query works well for REST/JSON, but AI streaming introduces unique challenges:

Problems:

1. Streaming responses don’t fit naturally into useQuery.
2. Cached data = final response, but UI = partial tokens → mismatch.
3. No standardized chunk buffering or reducer pattern.
4. Abort + resume logic must be custom-implemented.
5. Partial streams shouldn’t pollute cache.
6. Mutations are commonly used, but lack structure for streaming UIs.

Proposal:
Introduce a new “streaming query mode” with:

1. streamQuery API

• partial: live tokens
• data: final cached response
• isStreaming: state
• abort: cancel generation
• resume: continue generation
• appendChunk: controlled chunk merging

2. Built-in chunk reducer
   (chunk, prev) => next

3. Native AbortController integration

4. Cache only final output, not partial chunks

5. Streaming refetch policy

• continue?
• restart?
• reuse generation context?

6. Example use cases:

• AI chat completions
• Code generation
• Live markdown rendering
• Terminal-style token streaming

Why this matters:
Most modern AI apps depend on streaming. Without first-class support, teams must build custom fetchEventSource or ReadableStream handlers. Native streaming query support would make TanStack Query a top-tier choice for AI applications.

[TkDodo]

have you seen streamedQuery ?

https://tanstack.com/query/latest/docs/reference/streamedQuery