Refactor models.py to a provider registry

[aallan]

Context

With #24 expanding to include Mistral, Grok, DeepSeek, and Gemini, we'll have 6+ LLM providers. Currently vera_bench/models.py has three nearly-identical classes:

• OpenAIClient (native)
• MoonshotClient (OpenAI-compatible, custom base URL)
• AnthropicClient (native SDK, different shape)

Adding Mistral/Grok/DeepSeek naively would mean three more copies of MoonshotClient with only the base URL and env var changing. That's ~120 lines of duplication, and any bug fix (e.g., the lazy-import issue we hit in the tests) would need to be made in 5+ places.

The rule of three

With two copies of a class, duplication is tolerable — the pattern hasn't stabilised yet. With three, the pattern is clear and abstraction pays for itself. We're about to add three or four more OpenAI-compatible providers, so this is the right moment to refactor.

Prior art

The Vera compiler is doing exactly this refactor in aallan/vera#413: replacing their _call_inference_provider() elif chain with a _ProviderConfig dataclass and a _PROVIDERS registry dict. Worth mirroring their approach so both repos stay consistent.

Proposed design

from dataclasses import dataclass

@dataclass(frozen=True)
class ProviderConfig:
    env_key: str        # e.g. "MOONSHOT_API_KEY"
    base_url: str       # OpenAI-compat base URL (e.g. "https://api.moonshot.ai/v1")
    prefix: str         # model prefix (e.g. "moonshot/")

_OPENAI_COMPAT_PROVIDERS: dict[str, ProviderConfig] = {
    "moonshot": ProviderConfig(
        env_key="MOONSHOT_API_KEY",
        base_url="https://api.moonshot.ai/v1",
        prefix="moonshot/",
    ),
    "mistral": ProviderConfig(
        env_key="MISTRAL_API_KEY",
        base_url="https://api.mistral.ai/v1",
        prefix="mistral/",
    ),
    "grok": ProviderConfig(
        env_key="XAI_API_KEY",
        base_url="https://api.x.ai/v1",
        prefix="xai/",
    ),
    "deepseek": ProviderConfig(
        env_key="DEEPSEEK_API_KEY",
        base_url="https://api.deepseek.com/v1",
        prefix="deepseek/",
    ),
}

class OpenAICompatClient:
    """Generic client for any OpenAI-compatible API."""
    def __init__(self, model: str, config: ProviderConfig) -> None:
        ...

create_client() iterates _OPENAI_COMPAT_PROVIDERS checking prefixes, returning an OpenAICompatClient with the matching config. Native OpenAIClient and AnthropicClient stay separate (native SDKs). Gemini gets its own class when added.

Scope

• Add ProviderConfig dataclass and _OPENAI_COMPAT_PROVIDERS registry
• Add generic OpenAICompatClient class
• Delete MoonshotClient (replaced by registry entry)
• Update create_client() to iterate the registry
• Update tests in tests/test_models.py to cover the generic client with a parametrised test for each provider
• Update the error message in create_client() to enumerate all registered prefixes

Relates to

• #24 — Multi-model benchmark runs (blocked on this refactor for clean provider addition)
• aallan/vera#413 — parallel refactor on the compiler side

[aallan]

PR #93 adds evidence for this, of the concrete kind rather than the speculative kind.

The openai-pro/ reasoning-mode entry required a second endpoint inside OpenAIClient — responses.create rather than chat.completions.create, because reasoning.mode exists only on the Responses API. So models.py now carries two request paths, two response shapes (output_text vs choices[0].message.content), two usage schemas (input_tokens / input_tokens_details.cached_tokens vs prompt_tokens / prompt_tokens_details.cached_tokens), and two module-level routing constants (REASONING_MODES, RESPONSES_API_MODELS) selecting between them per model id.

The duplication cost this issue predicts has now been paid, twice, in ways CI could not see:

1. _validate_openai_response_text has guarded empty responses on the OpenAI path since v0.0.15 — with a docstring explaining that without it "the harness would receive text="" and blame the model for did not define entry point when the real fault is API-side". The Anthropic path had no equivalent guard. Same hazard, same file, one provider protected. Found in review, fixed in v0.0.16: 3-tier matrix, charts, and four harness fixes found by preflight #93.
2. AuthenticationError → AuthError is mapped in _call_openai_compatible, so the runner aborts a sweep on a bad key rather than burning 60 problems into error rows. AnthropicClient.complete catches only APITimeoutError, so the same failure on Anthropic — now three of eight matrix slots, including the ceiling tier — produces 60 error rows and a completed run. Still open; deliberately not fixed in v0.0.16: 3-tier matrix, charts, and four harness fixes found by preflight #93 to keep that PR's scope defensible.

Neither is a hard bug in one place. Both are "the fix landed on one client and not its siblings", which is exactly the failure mode a shared base or registry removes structurally.

Worth noting for whoever picks this up: the three clients are no longer "nearly identical with only base URL and env var changing". OpenAIClient is now materially the most complex, and a naive registry that assumes one request shape per provider would not accommodate it. The abstraction needs a seam at request/response shape, not only at provider identity.

Related: #24 (this is its stated prerequisite for DeepSeek/Gemini).