Skip to content

Clarification on "Overall Score" calculation logic #25

Description

@yikeyang33

Hi team,

I have successfully ran the benchmark on my local model. However, I noticed that the final "Overall Score" (the metric used in the Leaderboard) is not explicitly saved in the generated benchmark_results_*.json file.

The JSON file contains detailed metrics for each task file (e.g., task_completion_score, tool_selection_score, etc.), but I couldn't find the aggregated final score or the explicit formula in the codebase/documentation to derive it.

Could you please clarify the exact formula used to calculate the Overall Score from these JSON metrics?

Thanks

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions