Skip to content

Conversation

muupan
Copy link
Contributor

@muupan muupan commented Aug 19, 2025

Resolves #927

This PR enables saving reasoning_content returned by litellm in the details file.

How I tested

I'm using vllm==0.9.2 and litellm==1.74.15.post2.

# Start vLLM server
vllm serve Qwen/Qwen3-1.7B --tensor-parallel-size 2 --trust-remote-code --dtype bfloat16 --gpu-memory-utilization 0.9 --max_model_len 32768 --reasoning-parser qwen3

# Evaluate via litellm entry point
lighteval endpoint litellm "model_name=hosted_vllm/Qwen/Qwen3-1.7B,base_url=http://0.0.0.0:8000/v1,api_key=\"\",generation_parameters={temperature:1,max_new_tokens:16000}" "lighteval|math_500|0|0" --save-details

# Check the content of the details file (only first example)
echo results/details/hosted_vllm/Qwen/Qwen3-1.7B/2025-08-19T22-27-32.231875/details_lighteval\|math_500\|0_2025-08-19T22-27-32.231875.parquet | python -c "import pandas as pd; print(pd.read_parquet(input().strip()).head(1).to_json())" | jq > reasonings_sample.json

"reasonings" is added under "model_response". Since it is too large to paste the content here, I attached the json file.
reasonings_sample.json

@HuggingFaceDocBuilderDev
Copy link
Collaborator

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@NathanHB NathanHB merged commit 50756ef into huggingface:main Aug 22, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FT] Save reasoning_content of responses as details
3 participants