Skip to content

Conversation

@wrisa
Copy link
Contributor

@wrisa wrisa commented Nov 6, 2025

  1. Assigns correct finish reason when tool call for llm invocation
  2. Skips evaluation when tool call for llm invocation when offer.
  3. Skips evaluation for error invocation. Adds error type on invocation object and verifies at the time of offer.

See below for 1) and 2)
skips eval for tool call chat span
Screenshot 2025-11-06 at 1 04 20 PM
Screenshot 2025-11-06 at 1 04 33 PM
shows evals for normal chat span
Screenshot 2025-11-06 at 1 05 56 PM

See below for 3)
Screenshot 2025-11-06 at 1 27 15 PM

Screenshot 2025-11-06 at 1 27 42 PM

@wrisa wrisa requested review from a team as code owners November 6, 2025 21:53
@wrisa wrisa merged commit f0682ca into main Nov 6, 2025
1 of 14 checks passed
@wrisa wrisa deleted the avoid-evaluations-for-skipped-and-error-invocations branch November 6, 2025 22:08
@github-actions github-actions bot locked and limited conversation to collaborators Nov 6, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants