Skip to content

Rework the /status provider matrix and update tracked models#474

Merged
jrhizor merged 3 commits into
mainfrom
jrhizor/bangkok
Jul 20, 2026
Merged

Rework the /status provider matrix and update tracked models#474
jrhizor merged 3 commits into
mainfrom
jrhizor/bangkok

Conversation

@jrhizor

@jrhizor jrhizor commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

What

Reworks the public /status provider matrix to be capability-accurate and adjusts which models Elmo tracks.

Tracked-model changes

  • Grok → OpenRouter only. xAI removed unauthenticated access, so the BrightData and Olostep scrapers can no longer reach Grok. Removed Grok from those scraper registries, the CLI scraper lists, the scraper docs, and the scraping STATUS_TARGETS. Grok stays trackable via OpenRouter (authenticated) — its OpenRouter target, model metadata, and docs remain.
  • Oxylabs → Google AI Overview. Added google-ai-overview scraping to the Oxylabs provider (google_search source, render: html), with text + citation extraction from the parsed ai_overviews block and unit tests. Adds the google-ai-overview:oxylabs:online target plus CLI/docs entries. The scheduled provider-test workflow exercises it live.
  • Kimi (moonshotai/kimi-k3) via OpenRouter. New tracked model with its label and Moonshot AI icon (SiMoonshotai), status/test target, and docs.

/status matrix presentation

  • Capability-accurate "not available". Empty cells were all a single grey dot. They're now split three ways:
    • N/A (hatched) — the combination can't exist. Pure LLMs (Claude, Grok, Mistral, DeepSeek, Kimi) are N/A on every scraper; the SERP/consumer surfaces (Google AI Mode/Overview, Copilot) are N/A on the Model APIs; and each scraper's capability set marks its own gaps (Oxylabs can't reach Gemini or Copilot, DataForSEO can't reach Copilot, …).
    • · (dot) — a combination Elmo could run but doesn't yet.
    • — tracked but no data yet (unchanged).
  • Column groups. The columns are grouped into Model APIs (Direct API, OpenRouter) and AI Search Scrapers (Olostep, BrightData, Oxylabs, DataForSEO), separated by a gap with group headers.

Resulting classification:

model               direct-api  openrouter  olostep   brightdata  oxylabs   dataforseo
chatgpt                ✓           ✓           ✓          ✓           ✓          ✓
claude                 ✓           ✓          N/A        N/A         N/A        N/A
copilot               N/A         N/A          ✓          ✓          N/A        N/A
deepseek               ·           ✓          N/A        N/A         N/A        N/A
gemini                 ·           ✓           ✓          ✓          N/A         ✓
google-ai-mode        N/A         N/A          ✓          ✓           ✓          ✓
google-ai-overview    N/A         N/A          ✓          ✓           ✓          ✓
grok                   ·           ✓          N/A        N/A         N/A        N/A
kimi                   ·           ✓          N/A        N/A         N/A        N/A
mistral                ✓           ✓          N/A        N/A         N/A        N/A
perplexity             ·           ·           ✓          ✓           ✓          ✓

Notes

  • The scraper capability sets are mirrored as display metadata in status-helpers.ts (the registries in @workspace/lib remain the runtime source of truth).
  • No pre-existing changesets conflict; three new patch changesets cover the tracked-model changes.
  • Marketing surfaces that mention Grok are untouched and remain accurate (Grok is still trackable).

Verification

  • Type-checks pass (@workspace/www, @workspace/web, @elmohq/cli).
  • Unit tests pass (@workspace/config, @workspace/lib provider + text-extraction — including new Oxylabs AI Overview extraction tests, @workspace/web model-filter).
  • pnpm --filter @workspace/www build succeeds.
  • Live-verified with test-provider.ts against real provider APIs: google-ai-overview:oxylabs:online and kimi:openrouter:moonshotai/kimi-k3 both PASS. The Oxylabs AI Overview run returned real overview text with 3 citations — all extracted from the ai_overviews block, confirming the parser against a production response (none of the pre-existing citation paths were present) — and Kimi returned a valid OpenRouter response.

xAI removed unauthenticated access to Grok, so the BrightData and Olostep scrapers can no longer reach it. Remove Grok from those scraper registries, the CLI setup lists, the scraper docs, and the /status + provider-test targets. Grok stays trackable via OpenRouter (authenticated), so its OpenRouter target, model metadata, trackable-model docs, and a docs example remain.

The /status matrix rendered every empty cell as the same grey dot. Split empty cells into "not available" (SERP-only surfaces like Google AI Mode/Overview and Copilot via Direct API or OpenRouter, which can't exist) shown as a hatched N/A, versus "not tracked yet" shown as the neutral dot, with legend entries and tooltips.
@vercel

vercel Bot commented Jul 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
elmo Ready Ready Preview, Comment Jul 20, 2026 5:55pm

Request Review

Replace the coarse scrape-only rule with a full capability model: pure LLMs (Claude, Grok, Mistral, DeepSeek) are N/A on every scraper, the SERP/consumer surfaces (Google AI Mode/Overview, Copilot) are N/A on the Model APIs, and each scraper's capability set marks its own gaps (Oxylabs can't reach Gemini/Copilot, DataForSEO can't reach Copilot). Group the matrix columns into Model APIs and AI Search Scrapers with a gap and group headers.

Add Google AI Overview scraping to the Oxylabs provider (google_search source, render html) with text + citation extraction and unit tests, plus the google-ai-overview:oxylabs:online target and CLI/docs entries. The scheduled provider-test workflow exercises it live.
Track Moonshot's Kimi through OpenRouter as an additional model — N/A on every scraper, like the other API-only models. Register its label and Moonshot AI icon (SiMoonshotai), add the status/test target, and document it.
@jrhizor jrhizor changed the title Drop Grok scraping and mark unavailable vs untracked status cells Rework the /status provider matrix and update tracked models Jul 20, 2026
@jrhizor
jrhizor merged commit c2fdd26 into main Jul 20, 2026
9 checks passed
@jrhizor
jrhizor deleted the jrhizor/bangkok branch July 20, 2026 23:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant