Skip to content

Fit OpenCode tool output to the Qwen context window - #25

Merged
bingran-you merged 1 commit into
mainfrom
bry/cap-model-bridge-context
Jul 14, 2026
Merged

Fit OpenCode tool output to the Qwen context window#25
bingran-you merged 1 commit into
mainfrom
bry/cap-model-bridge-context

Conversation

@bingran-you

@bingran-you bingran-you commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

What changed

  • Adds an explicit --max-context-tokens model-bridge contract, defaulting to the documented 49,152-token TRL server window.
  • Tokenizes every normalized OpenCode prompt with the pinned Qwen tokenizer before calling TRL.
  • Reserves the configured per-call generation budget and truncates only oldest tool-role outputs when the prompt would overflow.
  • Retains the largest fitting tool-output prefix using token-count binary search and appends an explicit truncation marker.
  • Never truncates system or user messages; non-tool overflow and non-convergent fitting fail before the TRL call.
  • Logs the number of affected tool outputs and before/after prompt-token counts.

Why

After PR #24 fixed Qwen tool-argument shape, the live red-wine canary still stalled on the second turn. Reconstructing the exact follow-up showed:

  • Qwen chat-template rendering succeeds after argument normalization.
  • The rendered prompt is 59,146 tokens.
  • The TRL vLLM server is configured for 49,152 tokens and the bridge reserves 4,096 tokens for the next completion.

The model had called OpenCode's read tool without a line limit, so the 1,600-line CSV result alone contributed about 51k characters. vLLM remained stuck at Rendering conversations: 0% instead of returning a usable overflow error. The bridge is the authoritative boundary because it has the actual normalized messages, tool schema, tokenizer, generation cap, and server context contract.

Validation

  • 217 package contract tests pass.
  • 48 focused bridge/CLI/GRPO tests pass.
  • Ruff check/format, Python compilation, and git diff --check pass.
  • Exact live replay fitted the previously failing prompt from 59,146 to 45,056 tokens and returned from the real Qwen3.5-9B TRL server in 8 seconds with two tool calls.
  • Full BenchFlow + OpenCode + Docker red-wine canary completed in 1.7 minutes with five tool calls, no idle timeout, healthy trajectories, verifier reward 1.0, and pass rate 1/1.
  • Unit coverage proves tool-only truncation, original-message immutability, maximum fitting under the prompt budget, oversized user-prompt rejection, CLI propagation, and invalid context configuration rejection.

@bingran-you
bingran-you merged commit e99338d into main Jul 14, 2026
1 check passed
@bingran-you
bingran-you deleted the bry/cap-model-bridge-context branch July 14, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant