fix(langgraph): fall back A2UI tool_choice for reasoning/thinking-mode models - #2234
Open
contextablemark wants to merge 1 commit into
Open
fix(langgraph): fall back A2UI tool_choice for reasoning/thinking-mode models#2234contextablemark wants to merge 1 commit into
contextablemark wants to merge 1 commit into
Conversation
…e models
The A2UI render subagent binds the inner `render_a2ui` tool with a forced
named `tool_choice` so the subagent is guaranteed to emit a structured
surface. Reasoning / "thinking-mode" models on some OpenAI-compatible
providers reject a forced/object `tool_choice` and 400 the request
(observed on Alibaba DashScope qwen* with thinking enabled):
The tool_choice parameter does not support being set to required or
object in thinking mode
That aborts the render mid-stream — the surface never paints and the
client surfaces a `terminated` run error.
Two changes, both fully backward compatible:
1. Runtime fallback (langgraph adapter): if binding the forced choice
fails the request with that specific 400, downgrade to
`tool_choice="auto"` and retry — once, and for the rest of the run so
subsequent recovery attempts don't re-trip it. Providers that honor a
named `tool_choice` (OpenAI, etc.) hit the success path on the first
try and never enter the fallback, so their behavior is unchanged. The
error match is narrow (both "tool_choice" and "thinking mode" must
appear) so unrelated 400s propagate untouched, and the existing
validate/retry loop still gates the result — a model that skips the
tool under "auto" degrades gracefully instead of crashing.
2. `tool_choice` knob (shared toolkit `A2UIToolParams`): lets a caller
pin the choice up front (e.g. "auto" for a known reasoning model).
Defaults to the forced render tool, so existing callers are unaffected;
every framework adapter gets the knob for free.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Python Preview PackagesVersion
Install with uvAdd the TestPyPI index to your [[tool.uv.index]]
name = "testpypi"
url = "https://test.pypi.org/simple/"
explicit = trueThen install the packages you need: # Core SDK
uv add 'ag-ui-protocol==0.0.0.dev1784774805' --index testpypi
# Integrations (each already depends on the matching ag-ui-protocol preview)
uv add 'ag-ui-langgraph==0.0.0.dev1784774805' --index testpypi
uv add 'ag-ui-crewai==0.0.0.dev1784774805' --index testpypi
# NOTE: ag-ui-agent-spec depends on pyagentspec (git-only, not on PyPI).
# You will need to install pyagentspec separately from its git repo.
uv add 'ag-ui-agent-spec==0.0.0.dev1784774805' --index testpypi
uv add 'ag_ui_adk==0.0.0.dev1784774805' --index testpypi
uv add 'ag_ui_strands==0.0.0.dev1784774805' --index testpypiInstall with pippip install \
--index-url https://test.pypi.org/simple/ \
--extra-index-url https://pypi.org/simple/ \
ag-ui-protocol==0.0.0.dev1784774805
Commit: a30fbe9 |
@ag-ui/a2a-middleware
@ag-ui/a2ui-middleware
@ag-ui/event-throttle-middleware
@ag-ui/mcp-apps-middleware
@ag-ui/mcp-middleware
@ag-ui/a2a
@ag-ui/adk
@ag-ui/ag2
@ag-ui/agno
@ag-ui/aws-strands
@ag-ui/claude-agent-sdk
@ag-ui/crewai
@ag-ui/langchain
@ag-ui/langgraph
@ag-ui/llamaindex
@ag-ui/mastra
@ag-ui/pydantic-ai
@ag-ui/vercel-ai-sdk
@ag-ui/watsonx
@ag-ui/a2ui-toolkit
create-ag-ui-app
@ag-ui/client
@ag-ui/core
@ag-ui/encoder
@ag-ui/proto
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The A2UI render subagent binds the inner
render_a2uitool with a forced namedtool_choiceso the subagent is guaranteed to emit a structured surface:Reasoning / "thinking-mode" models on some OpenAI-compatible providers reject a forced/object
tool_choiceand 400 the request. Observed on Alibaba DashScopeqwen*with thinking enabled:That aborts the render mid-stream: the surface never paints and the client surfaces a
terminatedrun error. Real OpenAI (and any provider that honors a namedtool_choice) is unaffected — only reasoning-mode providers that forbid forced choices hit this.Fix
Two changes, both fully backward compatible:
1. Runtime fallback (
integrations/langgraph/python/ag_ui_langgraph/a2ui_tool.py)If binding the forced choice fails with that specific 400, downgrade to
tool_choice="auto"and retry — once, and for the rest of the run so subsequent recovery attempts don't re-trip it.tool_choicehit the success path on the first try and never enter the fallback → their behavior is byte-for-byte unchanged. The new path can only turn a hard crash into a success.tool_choiceandthinking modemust appear in the message), matched on the error text rather than theopenaiSDK's exception classes, so unrelated 400s propagate untouched.autodegrades gracefully (re-prompt, then structured hard-fail envelope) rather than crashing the stream.2.
tool_choiceknob (sharedA2UIToolParamsinsdks/python/a2ui_toolkit)Lets a caller pin the choice up front (e.g.
"auto"for a known reasoning model). Defaults to the forced render tool, so existing callers are unaffected, and every framework adapter gets the knob for free.Tests
integrations/langgraph/python/tests/test_a2ui_tool.py(12 pass):render_a2ui; explicittool_choiceoverride reachesbind_toolsautoand still emits a valid operations envelope (asserts["render_a2ui", "auto"]bind sequence)Toolkit suite (100 pass) confirms the
A2UIToolParams/resolver change is non-breaking.🤖 Generated with Claude Code