Background
Follow-up from PR #113 review comment by @tnetennba3: #113 (comment)
The synthetic data generators under evals/synthetic/llm_generators/ (context, question, theme, response) currently instantiate openai.AsyncAzureOpenAI clients that hit Azure OpenAI directly. This deviates from Consult, where LLM requests are routed through the LiteLLM gateway.
Proposal
Point these generators at the LiteLLM gateway (potentially with a new key) so that:
- We get consistency with how Consult invokes models
- We can track and attribute spend centrally
Scope
evals/synthetic/llm_generators/context_generator.py
evals/synthetic/llm_generators/question_generator.py
evals/synthetic/llm_generators/theme_generator.py
evals/synthetic/llm_generators/response_generator.py
Notes
Not a correctness issue — synthetic data generation only — so this is intentionally a follow-up rather than a blocker for #113.
Background
Follow-up from PR #113 review comment by @tnetennba3: #113 (comment)
The synthetic data generators under
evals/synthetic/llm_generators/(context, question, theme, response) currently instantiateopenai.AsyncAzureOpenAIclients that hit Azure OpenAI directly. This deviates from Consult, where LLM requests are routed through the LiteLLM gateway.Proposal
Point these generators at the LiteLLM gateway (potentially with a new key) so that:
Scope
evals/synthetic/llm_generators/context_generator.pyevals/synthetic/llm_generators/question_generator.pyevals/synthetic/llm_generators/theme_generator.pyevals/synthetic/llm_generators/response_generator.pyNotes
Not a correctness issue — synthetic data generation only — so this is intentionally a follow-up rather than a blocker for #113.