Description
Create a benchmark script that measures tokens/second for each provider.
Implementation
- Create
benchmarks/speed_test.py
- Send the same prompt to each configured provider
- Measure time-to-first-token and tokens/second
- Output a comparison table
- Use a standard prompt (e.g., "Explain quantum computing in 200 words")
Expected output
Provider Model TTFT Tokens/s
─────────────────────────────────────────────────
SambaNova samba-llama-70b 0.3s 580 t/s
Groq llama-3.3-70b 0.2s 320 t/s
Cerebras llama3.1-8b 0.1s 450 t/s
...
Description
Create a benchmark script that measures tokens/second for each provider.
Implementation
benchmarks/speed_test.pyExpected output