feat(Anthropic): Add beta header for Claude 3.7 Sonnet 128k output #105
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change implements support for Anthropic's `output-128k-2025-02-19` beta header, specifically for the `claude-3-7-sonnet-20250219` model. Including this header allows users to leverage the increased 128k maximum output token limit available for this model. This implementation attempts to follow the suggested pattern for provider-specific capabilities, similar to the existing temperature normalization logic: - A new method `additional_headers_for_model` was added to `Anthropic::Capabilities` to encapsulate the provider-specific logic. - The shared `Provider#post` method was updated to conditionally call this capability method via `respond_to?` and merge the resulting headers before making the API request. - The necessary `model_id` context is now passed down through the relevant methods (`embed`, `paint`, `sync_response`, `stream_response`) to `post`. This approach avoids overriding shared methods within the provider module and keeps the specific logic contained within the capabilities definition. Closes crmne#85
crmne
requested changes
Apr 23, 2025
Comment on lines
+87
to
+96
| def post(url, payload, model_id: nil) | ||
| request_headers = headers | ||
|
|
||
| if model_id && capabilities.respond_to?(:additional_headers_for_model) | ||
| additional_headers = capabilities.additional_headers_for_model(model_id) | ||
| request_headers = request_headers.merge(additional_headers) unless additional_headers.empty? | ||
| end | ||
|
|
||
| connection.post url, payload do |req| | ||
| req.headers.merge! headers | ||
| req.headers.merge! request_headers |
Owner
There was a problem hiding this comment.
changing the post method is a bit heavy handed. especially when the post needs to know about the model_id which is a clear violation of SRP
Owner
|
Added in 177debc |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change implements support for Anthropic's
output-128k-2025-02-19beta header, specifically for theclaude-3-7-sonnet-20250219model. Including this header allows users to leverage the increased 128k maximum output token limit available for this model.This implementation attempts to follow the suggested pattern for provider-specific capabilities, similar to the existing temperature normalization logic:
additional_headers_for_modelwas added toAnthropic::Capabilitiesto encapsulate the provider-specific logic.Provider#postmethod was updated to conditionally call this capability method viarespond_to?and merge the resulting headers before making the API request.model_idcontext is now passed down through the relevant methods (embed,paint,sync_response,stream_response) topost.This approach avoids overriding shared methods within the provider module and keeps the specific logic contained within the capabilities definition.
Closes #85