Skip to content

fix(catalog): advertise GPT-5.6 reasoning efforts - #110

Merged
steipete merged 2 commits into
mainfrom
steipete/gpt-5-6-reasoning-efforts
Aug 8, 2026
Merged

fix(catalog): advertise GPT-5.6 reasoning efforts#110
steipete merged 2 commits into
mainfrom
steipete/gpt-5-6-reasoning-efforts

Conversation

@steipete

@steipete steipete commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add optional, bounded supportedReasoningEfforts metadata to provider model manifests and the compiled runtime catalog
  • advertise GPT-5.6's exact provider-native wire efforts: none, low, medium, high, xhigh, and max
  • preserve omission for models that do not declare effort metadata and leave /v1/models and request proxying unchanged

Root cause

ClawRouter's credential-scoped catalog exposed model identity, capabilities, and pricing, but omitted model-specific reasoning-effort metadata. Consumers therefore fell back to a stale generic effort profile capped at high, even though the existing GPT-5.6 route already passes advanced nested reasoning settings through to the provider.

The canonical fix records the provider-native contract in the model manifest, validates it during compilation, carries it through runtime types and the generated snapshot, and projects it through /v1/catalog. The paired OpenClaw consumer is openclaw/openclaw#120631.

Evidence

Authenticated local Worker runtime

The runtime candidate 4c69ef6fe5ee22f5541874456f359a016e0b7f4f ran through the repository's real local Wrangler Worker, local authority/KV state, and generated local proxy credential using:

pnpm worker:e2e

A temporary uncommitted assertion in the established smoke read the authenticated HTTP response and was removed after the run. Current exact head c35be57b80500f831656e1ab8682d335f90862cd changes only the release-owned changelog line; all runtime, manifest, generated-catalog, and test files remain byte-identical to the proven runtime candidate. Redacted observed result:

{"proof":"authenticated /v1/catalog","status":200,"model":"openai/gpt-5.6","supportedReasoningEfforts":["none","low","medium","high","xhigh","max"],"sibling":"openai/gpt-5.5","siblingHasSupportedReasoningEfforts":false}

The full smoke then completed successfully: local Worker smoke passed.

Consumer compatibility

The companion OpenClaw PR's source-blind validation used the public CLI with a disposable catalog endpoint. It discovered exactly none/low/medium/high/xhigh/max, rejected minimal with allowed choices off/low/medium/high/xhigh/max/ultra, and proved xhigh, max, and logical OpenClaw ultra reached the provider endpoint. It also proved absent metadata preserves the generic fallback. See openclaw/openclaw#120631.

The field is optional and additive. Consumers that do not read it continue parsing the pre-existing model fields and retain their prior fallback behavior.

Automated checks

  • focused provider compiler and catalog tests: 10/10 passed
  • pnpm worker:check: 126/126 passed
  • pnpm check: Worker, admin, and script suites passed
  • pnpm build: provider snapshot, admin production build, Worker typecheck, and Worker tests passed
  • git diff --check: passed

Autoreview

The coordinator ran the global Codex autoreview helper in branch mode against origin/main on 4c69ef6fe5ee22f5541874456f359a016e0b7f4f. The complete one-pass bundle was 202,469 bytes, including the generated snapshot. Result: clean, with no accepted or actionable findings.

After removing the release-owned changelog line, exact-head cleanup review ran:

/Users/steipete/.codex/skills/agent-scripts/autoreview/scripts/autoreview --mode local

It reviewed the complete 654-byte follow-up diff in one pass and returned autoreview clean: no accepted/actionable findings reported with overall correctness 0.99.

Change size

  • production/schema/config: +22/-3
  • tests: +45/-0
  • docs: +3/-0
  • generated snapshot: +1/-1

Source

The existing manifest model and upstream alias are documented by OpenAI at https://developers.openai.com/api/docs/models/gpt-5.6-sol.

Deployment status

Deployment status: not deployed.

Remaining risk

The catalog change is additive and request validation and proxy behavior are intentionally unchanged. The consumer behavior is delivered separately by the paired OpenClaw PR.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. labels Aug 8, 2026
@clawsweeper

clawsweeper Bot commented Aug 8, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed August 8, 2026, 1:04 PM ET / 17:04 UTC.

ClawSweeper review

What this changes

Adds optional GPT-5.6 reasoning-effort metadata to provider manifests, the compiled provider catalog, and authenticated /v1/catalog model results.

Merge readiness

⚠️ Ready for maintainer review - 1 item remains

Keep open: current main still omits model-specific reasoning efforts, while this clean branch carries the optional metadata from the provider manifest through the compiled and authenticated client catalog. The prior release-note finding is resolved, and the supplied real Worker proof supports normal maintainer merge review.

Priority: P2
Reviewed head: c35be57b80500f831656e1ab8682d335f90862cd

Review scores

Measure Result What it means
Overall readiness 🦞 diamond lobster (5/6) A focused additive implementation with exact-head real Worker evidence, compatibility tests, and no remaining actionable finding.
Proof confidence 🦞 diamond lobster (5/6) Sufficient (live_output): The PR records a successful authenticated local Worker /v1/catalog response with the six values and confirms the exact head changes only a release-owned changelog line.
Patch quality 🦞 diamond lobster (5/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (live_output): The PR records a successful authenticated local Worker /v1/catalog response with the six values and confirms the exact head changes only a release-owned changelog line.
Evidence reviewed 6 items Current-main gap: Current main’s catalog model projection returns identity, capabilities, and pricing only; it has no reasoning-effort field, so the central behavior is not already implemented.
Complete branch data path: The branch adds bounded manifest validation, preserves the optional field in compiled provider and model-index data, and projects it only when declared in catalog results.
Focused compatibility coverage: Tests verify GPT-5.6 exposes the declared six values, sibling models retain omission, and malformed metadata is rejected by the compiler.
Findings None None.
Security None None.

How this fits together

Provider manifests are compiled into ClawRouter’s Worker snapshot, then filtered by credential and executable endpoints before clients receive /v1/catalog. This metadata lets a client select supported reasoning levels without changing upstream request routing or /v1/models.

flowchart LR
A[Provider manifest] --> B[Manifest validation]
B --> C[Compiled provider snapshot]
C --> D[Credential and endpoint filtering]
D --> E[Authenticated catalog response]
E --> F[Client reasoning selection]
Loading

Before merge

  • Complete next step (P2) - No repair lane is needed; the PR is ready for normal maintainer merge review.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Changed surface 9 files, +71/-4 The change spans manifest schema, compiler, runtime projection, generated snapshot, documentation, and focused tests.
Production versus tests production/schema +22/-3, tests +45/-0 The bounded runtime addition has regression coverage for propagation, omission, and invalid declarations.

Technical review

Best possible solution:

Land the optional provider-native metadata contract and retain the omission behavior for models that do not declare it.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a bug reproduction: this is an additive catalog capability. The PR supplies high-confidence after-fix evidence from an authenticated local Worker response containing all six declared values.

Is this the best way to solve the issue?

Yes. A declarative optional manifest field is the narrow provider-neutral solution, and the Worker only carries it through existing compilation and catalog boundaries.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 7b0b6f60f7c6.

Labels

Label changes:

  • add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.

Label justifications:

  • P2: This is a bounded provider-catalog improvement that affects supported model selection but leaves proxy request behavior unchanged.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR records a successful authenticated local Worker /v1/catalog response with the six values and confirms the exact head changes only a release-owned changelog line.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR records a successful authenticated local Worker /v1/catalog response with the six values and confirms the exact head changes only a release-owned changelog line.

Evidence

What I checked:

  • Current-main gap: Current main’s catalog model projection returns identity, capabilities, and pricing only; it has no reasoning-effort field, so the central behavior is not already implemented. (worker/discovery.ts:206, 7b0b6f60f7c6)
  • Complete branch data path: The branch adds bounded manifest validation, preserves the optional field in compiled provider and model-index data, and projects it only when declared in catalog results. (scripts/compile-providers.mjs:110, 4c69ef6fe5ee)
  • Focused compatibility coverage: Tests verify GPT-5.6 exposes the declared six values, sibling models retain omission, and malformed metadata is rejected by the compiler. (worker/test/provider-compiler.test.mjs:19, 4c69ef6fe5ee)
  • Prior review blocker resolved: The exact head differs from the proven runtime candidate only in the release-owned changelog file; all runtime, manifest, generated snapshot, and test paths are byte-identical. (worker/discovery.ts:206, c35be57b8050)
  • Feature provenance: Blame shows the baseline catalog/compiler and OpenAI manifest lines date to the release commit, while the new optional field is introduced by this branch’s main implementation commit. (worker/discovery.ts:206, 4c69ef6fe5ee)
  • Unmerged and unreleased: The implementation commit is not an ancestor of current main and no local branch or tag contains it; it remains necessary for the next release. (worker/discovery.ts:206, 4c69ef6fe5ee)

Likely related people:

  • steipete: Relevant current-main baseline lines and this branch’s added manifest/compiler/catalog field all trace to commits authored by steipete. (role: baseline catalog/compiler contributor and latest feature contributor; confidence: high; commits: 13845ee2bdc0, 4c69ef6fe5ee, c35be57b8050; files: providers/openai.provider.yaml, scripts/compile-providers.mjs, worker/discovery.ts)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (2 earlier review cycles)
  • reviewed 2026-08-08T16:22:05.544Z sha 4c69ef6 :: needs real behavior proof before merge. :: none
  • reviewed 2026-08-08T16:58:02.479Z sha 4c69ef6 :: needs changes before merge. :: [P3] Remove the release-owned changelog entry

@steipete

steipete commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 8, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. labels Aug 8, 2026
@steipete

steipete commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 8, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Aug 8, 2026
@steipete
steipete merged commit ca27bc4 into main Aug 8, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant