Skip to content

Route synthetic data generators through LiteLLM gateway #133

@jjuritzno10

Description

@jjuritzno10

Background

Follow-up from PR #113 review comment by @tnetennba3: #113 (comment)

The synthetic data generators under evals/synthetic/llm_generators/ (context, question, theme, response) currently instantiate openai.AsyncAzureOpenAI clients that hit Azure OpenAI directly. This deviates from Consult, where LLM requests are routed through the LiteLLM gateway.

Proposal

Point these generators at the LiteLLM gateway (potentially with a new key) so that:

  • We get consistency with how Consult invokes models
  • We can track and attribute spend centrally

Scope

  • evals/synthetic/llm_generators/context_generator.py
  • evals/synthetic/llm_generators/question_generator.py
  • evals/synthetic/llm_generators/theme_generator.py
  • evals/synthetic/llm_generators/response_generator.py

Notes

Not a correctness issue — synthetic data generation only — so this is intentionally a follow-up rather than a blocker for #113.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions