-
Notifications
You must be signed in to change notification settings - Fork 4
[codex] Add generic model context window hints #23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -27,6 +27,82 @@ use crate::harness::usage::Usage; | |
|
|
||
| pub use types::*; | ||
|
|
||
| /// How a context-window pattern is matched against a model id. | ||
| #[derive(Clone, Copy, Debug, PartialEq, Eq)] | ||
| enum ContextPatternMatch { | ||
| /// Pattern may appear anywhere in the lowercased model id. | ||
| Substring, | ||
| /// Pattern must be a complete segment delimited by common provider/id | ||
| /// separators. This avoids false positives for short model ids such as | ||
| /// `o1` and `o3`. | ||
| Segment, | ||
| } | ||
|
|
||
| /// Generic context-window hints for common provider model families. | ||
| /// | ||
| /// These are deliberately provider-neutral fallbacks, not a pricing catalog. | ||
| /// Hosts should prefer authoritative provider/catalog metadata when available | ||
| /// and use this only when a raw model id needs a conservative pre-dispatch | ||
| /// budget. | ||
| /// | ||
| /// Order matters: lookup returns the first matching entry, so more-specific | ||
| /// substrings such as `gpt-4.1` and `gpt-4-turbo` must stay before broader | ||
| /// patterns such as `gpt-4` that would otherwise shadow them. | ||
| const MODEL_CONTEXT_PATTERNS: &[(&str, ContextPatternMatch, u64)] = &[ | ||
| ("claude-haiku-4.5", ContextPatternMatch::Substring, 200_000), | ||
| ("claude-haiku-4", ContextPatternMatch::Substring, 200_000), | ||
| ("claude-haiku", ContextPatternMatch::Substring, 200_000), | ||
| ("claude-sonnet-4", ContextPatternMatch::Substring, 200_000), | ||
| ("claude-opus-4", ContextPatternMatch::Substring, 200_000), | ||
| ("claude-3-5-sonnet", ContextPatternMatch::Substring, 200_000), | ||
| ("claude-3-5-haiku", ContextPatternMatch::Substring, 200_000), | ||
| ("claude-3-opus", ContextPatternMatch::Substring, 200_000), | ||
| ("gpt-4.1", ContextPatternMatch::Substring, 1_047_576), | ||
| ("gpt-4o", ContextPatternMatch::Substring, 128_000), | ||
| ("gpt-4-turbo", ContextPatternMatch::Substring, 128_000), | ||
| ("gpt-4", ContextPatternMatch::Substring, 128_000), | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OpenAI lists the plain Useful? React with 👍 / 👎. |
||
| ("gpt-3.5", ContextPatternMatch::Substring, 16_385), | ||
| ("o1", ContextPatternMatch::Segment, 200_000), | ||
| ("o3", ContextPatternMatch::Segment, 200_000), | ||
| ("deepseek", ContextPatternMatch::Substring, 128_000), | ||
| ("gemma3", ContextPatternMatch::Substring, 8_192), | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
For Gemma 3 ids such as Useful? React with 👍 / 👎. |
||
| ("gemma", ContextPatternMatch::Substring, 8_192), | ||
| ("llama-3", ContextPatternMatch::Substring, 128_000), | ||
| ("llama3", ContextPatternMatch::Substring, 128_000), | ||
|
Comment on lines
+70
to
+71
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Meta's Llama 3 model card lists an 8k context length for the Llama 3 8B/70B models (https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct), while the 128k window belongs to later Llama 3.1 models. With these generic patterns, Useful? React with 👍 / 👎.
Comment on lines
+70
to
+71
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Meta's Llama 3 model card lists an 8k context length for the Llama 3 8B/70B models (https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct), while the 128k window belongs to later Llama 3.1 models. With these generic patterns, Useful? React with 👍 / 👎. |
||
| ]; | ||
|
|
||
| fn matches_context_pattern(lower: &str, pattern: &str, mode: ContextPatternMatch) -> bool { | ||
| match mode { | ||
| ContextPatternMatch::Substring => lower.contains(pattern), | ||
| ContextPatternMatch::Segment => { | ||
| let model_name = lower.rsplit(['/', ':']).next().unwrap_or(lower); | ||
| model_name | ||
| .split(['-', '_', '.']) | ||
| .next() | ||
| .is_some_and(|segment| segment == pattern) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| /// Returns a generic context-window hint for a raw provider model id. | ||
| /// | ||
| /// Returns `None` for unknown ids rather than guessing. Hosts with product tier | ||
| /// aliases, local runtime profiles, or authoritative provider catalogs should | ||
| /// check those first and use this helper as a last generic fallback. | ||
| pub fn context_window_for_model_id(model: &str) -> Option<u64> { | ||
| let normalized = model.trim(); | ||
| if normalized.is_empty() { | ||
| return None; | ||
| } | ||
|
|
||
| let lower = normalized.to_ascii_lowercase(); | ||
| MODEL_CONTEXT_PATTERNS | ||
| .iter() | ||
| .find_map(|(pattern, mode, window)| { | ||
| matches_context_pattern(&lower, pattern, *mode).then_some(*window) | ||
| }) | ||
| } | ||
|
|
||
| impl std::fmt::Display for ProviderError { | ||
| /// Renders the same human-readable shape real provider adapters used to | ||
| /// build by hand before flattening it into a plain | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OpenAI lists the plain
gpt-4model's context window as 8,192 tokens (https://developers.openai.com/api/docs/models/gpt-4), socontext_window_for_model_id("gpt-4")now returns a 128k budget for a model that rejects prompts above 8k. Because this helper is intended for pre-dispatch budgeting, callers that use the plaingpt-4id can skip summarization/compaction and hit provider context-limit errors; keep the 128k value scoped togpt-4-turbo/gpt-4ovariants.Useful? React with 👍 / 👎.