Skip to content

feat(ai): Options for custom model cost mappings #94858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 4, 2025

Conversation

vgrozdanic
Copy link
Member

Since this is still a very young field, there is a lot of irregularities in model naming. This PR introduces an option to configure custom model mappings, where we can assign alternative names to existing models in our pricing map. Often the same models have different names, depending on where they are hosted.

For example:
image

We need to have a way to add custom mappings, which can be easily changed and quickly deployed - that's why the options are used.

@vgrozdanic vgrozdanic requested a review from a team as a code owner July 3, 2025 12:53
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Jul 3, 2025
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Custom Model Mapping Validation Errors

The ai-agent-monitoring.custom-model-mapping option lacks sufficient validation, leading to several issues:

  1. If the option is configured as a non-iterable value (e.g., None), iterating over it will raise a TypeError.
  2. If the option's list contains non-dictionary items, attempting to call .get() on them will result in an AttributeError.
  3. If a custom model mapping entry has a missing or None alternative_model_id, the code will incorrectly add None as a key to the models_dict, potentially causing downstream issues.
    These validation failures can cause the AI model costs fetching task to fail.

src/sentry/tasks/ai_agent_monitoring.py#L79-L88

# in different hosting providers. This allows us to map the alternative model id to the existing model id.
for model_mapping in options.get("ai-agent-monitoring.custom-model-mapping"):
alternative_model_id = model_mapping.get("alternative_model_id")
existing_model_id = model_mapping.get("existing_model_id")
if existing_model_id not in models_dict or alternative_model_id in models_dict:
continue
models_dict[alternative_model_id] = models_dict[existing_model_id]

Fix in Cursor


Was this report helpful? Give feedback by reacting with 👍 or 👎

@vgrozdanic vgrozdanic requested review from a team and removed request for a team July 4, 2025 07:27
@vgrozdanic vgrozdanic merged commit 59f5050 into master Jul 4, 2025
66 checks passed
@vgrozdanic vgrozdanic deleted the vg/options-for-model-costs-mapping branch July 4, 2025 07:40
andrewshie-sentry pushed a commit that referenced this pull request Jul 14, 2025
Since this is still a very young field, there is a lot of irregularities
in model naming. This PR introduces an option to configure custom model
mappings, where we can assign alternative names to existing models in
our pricing map. Often the same models have different names, depending
on where they are hosted.

For example:

![image](https://github.com/user-attachments/assets/3385b86c-1937-4b99-8365-1ba8779b2aca)

We need to have a way to add custom mappings, which can be easily
changed and quickly deployed - that's why the options are used.
@github-actions github-actions bot locked and limited conversation to collaborators Jul 19, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Scope: Backend Automatically applied to PRs that change backend components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants