Keyless Exa search returns HTTP 429 continuously and degrades to a weaker provider silently

[sosidudku1]

Summary

The default web search provider (web.search.provider = "exa") runs keyless when EXA_API_KEY is unset. In a GAIA validation campaign this produced 1341 Exa returned HTTP 429 errors — 44% of all 3039 tool failures, making it the single largest source of failure in the run by a wide margin.

Search is the agent's primary way of grounding an answer. When it is rate-limited this often, the model's capability stops being the limiting factor.

Evidence that this is a standing limit, not a burst

429s were distributed evenly across all 24 hours of the campaign — between 8 and 20 per hour, with no clustering:

00h  18    06h  10    12h  11    18h  13
01h  13    07h  11    13h  13    19h   8
02h  15    08h  14    14h  12    20h  18
03h  13    09h  17    15h  12    21h  20
04h   9    10h  10    16h  16    22h  14
05h  16    11h  16    17h   8    23h  12

Concurrency during the campaign varied from 1 to 6 parallel agents; the error rate did not track it. This is the keyless tier's standing quota, not contention created by the harness.

Why it is easy to miss

web.search.fallback defaults to ["duckduckgo"] and works as designed — the search does not hard-fail. But DuckDuckGo's keyless HTML endpoint is materially weaker for the factual lookups this workload needs, so the observable outcome is quietly degraded answers rather than a visible error. Nothing in the run surfaced "your primary search provider is not working".

Configuration in use

{ "provider": "exa", "fallback": ["duckduckgo"], "cacheTtlMinutes": 15,
  "exa": { "apiKeyEnv": "EXA_API_KEY" } }

EXA_API_KEY unset; no BRAVE_SEARCH_API_KEY either, so brave was not an available fallback.

Suggestions

1. Warn at startup when the configured search provider has an apiKeyEnv that resolves to nothing. Silent degradation to a weaker provider is the worst of both worlds — it neither works well nor tells you why.
2. Raise the default cacheTtlMinutes (currently 15). Agent runs re-issue near-identical queries across steps and across tasks; a longer TTL would cut requests that currently spend quota to re-fetch the same result.
3. Consider backoff on 429 rather than immediate fallback, so a transient limit does not permanently downgrade a session to DuckDuckGo.
4. Document that a key is effectively required for search-heavy autonomous work, and what the keyless tier realistically supports.

Related

45 further 429s came from os.web.fetch and 82 from os.http.request hitting third-party rate limits (archive.org/wayback, api.semanticscholar.org), which the same backoff logic would help.

Data

All 1341 records with arguments and error text: rate_limit_429.jsonl in error-extracts.tar.gz, attached to the gaia-tool-reliability-2026-08 release.