[Bug]: All kbench.llms models return "User location not supported for this model/API"

[DanGlChris]

Bug Description

Description

As of today, every model registered in kbench.llms returns the same error:

User location not supported for this model/API

This affects all 27 listed models across all providers (Anthropic, DeepSeek, Google, Qwen, ZAI). The same notebook with the same account was working normally yesterday.

Environment

• Platform: Kaggle Cloud Notebook
• Package: kaggle_benchmarks (imported as kbench)
• Date of failure: 27.03.2026
• Date last working: 26.03.2026
• Internet enabled: Yes

Impact

• Complete blocker — no models are usable, making kbench entirely non-functional.
• Cannot run any benchmarks, tasks, or evaluations.
• Affects at minimum all users in [INSERT YOUR COUNTRY/REGION].
• Unknown how many other regions are affected.

Questions for Maintainers

1. Was there a deployment or configuration change in the last 24 hours?
2. Is "User location not supported" generated by kbench middleware or passed through from providers?
3. Which regions are currently in the allowlist?
4. Is there a way to check/override the detected region for debugging?

Kaggle Username

dagloxkankwanda

Execution Environment

Kaggle Notebook

Are you using Kaggle Benchmarks in your code?

Yes — I import and use kaggle_benchmarks directly

Steps to Reproduce

Minimal reproducer — one cell in a Kaggle notebook:

Code to Reproduce

import kaggle_benchmarks as kbench

print(f"kbench version: {getattr(kbench, '__version__', 'unknown')}")
print(f"Models listed: {len(list(kbench.llms.keys()))}")

for name in sorted(kbench.llms.keys()):
    try:
        model = kbench.llms[name]
        resp = model.prompt("Say hi.")
        print(f"  ✅ {name}: {str(resp)[:50]}")
    except Exception as e:
        print(f"  ❌ {name}: {e}")

Expected Behavior

Models listed: 27
  ✅ anthropic/claude-haiku-4-5@20251001: Hi!
  ✅ anthropic/claude-opus-4-1@20250805: Hi!
  ✅ anthropic/claude-opus-4-5@20251101: Hi!
  ✅ anthropic/claude-opus-4-6@default: Hi!
  ✅ anthropic/claude-sonnet-4-5@20250929: Hi!
  ✅ anthropic/claude-sonnet-4-6@default: Hi!
  ✅ anthropic/claude-sonnet-4@20250514: Hi!
  ✅ deepseek-ai/deepseek-r1-0528: Hi!
  ✅ deepseek-ai/deepseek-v3.1: Hi!
  ✅ deepseek-ai/deepseek-v3.2: Hi!
  ✅ google/gemini-2.0-flash: Hi!
  ✅ google/gemini-2.0-flash-lite: Hi!
  ✅ google/gemini-2.5-flash: Hi!
  ✅ google/gemini-2.5-pro: Hi!
  ✅ google/gemini-3-flash-preview: Hi!
  ✅ google/gemini-3-pro-preview: Hi!
  ✅ google/gemini-3.1-flash-lite-preview: Hi!
  ✅ google/gemini-3.1-pro-preview: Hi!
  ✅ google/gemma-3-12b: Hi!
  ✅ google/gemma-3-1b: Hi!
  ✅ google/gemma-3-27b: Hi!
  ✅ google/gemma-3-4b: Hi!
  ✅ qwen/qwen3-235b-a22b-instruct-2507: Hi!
  ✅ qwen/qwen3-coder-480b-a35b-instruct: Hi!
  ✅ qwen/qwen3-next-80b-a3b-instruct: Hi!
  ✅ qwen/qwen3-next-80b-a3b-thinking: Hi!
  ✅ zai/glm-5: Hi!

Actual Behavior

Models listed: 27
  ❌ anthropic/claude-haiku-4-5@20251001: User location not supported for this model/API
  ❌ anthropic/claude-opus-4-1@20250805: User location not supported for this model/API
  ❌ anthropic/claude-opus-4-5@20251101: User location not supported for this model/API
  ❌ anthropic/claude-opus-4-6@default: User location not supported for this model/API
  ❌ anthropic/claude-sonnet-4-5@20250929: User location not supported for this model/API
  ❌ anthropic/claude-sonnet-4-6@default: User location not supported for this model/API
  ❌ anthropic/claude-sonnet-4@20250514: User location not supported for this model/API
  ❌ deepseek-ai/deepseek-r1-0528: User location not supported for this model/API
  ❌ deepseek-ai/deepseek-v3.1: User location not supported for this model/API
  ❌ deepseek-ai/deepseek-v3.2: User location not supported for this model/API
  ❌ google/gemini-2.0-flash: User location not supported for this model/API
  ❌ google/gemini-2.0-flash-lite: User location not supported for this model/API
  ❌ google/gemini-2.5-flash: User location not supported for this model/API
  ❌ google/gemini-2.5-pro: User location not supported for this model/API
  ❌ google/gemini-3-flash-preview: User location not supported for this model/API
  ❌ google/gemini-3-pro-preview: User location not supported for this model/API
  ❌ google/gemini-3.1-flash-lite-preview: User location not supported for this model/API
  ❌ google/gemini-3.1-pro-preview: User location not supported for this model/API
  ❌ google/gemma-3-12b: User location not supported for this model/API
  ❌ google/gemma-3-1b: User location not supported for this model/API
  ❌ google/gemma-3-27b: User location not supported for this model/API
  ❌ google/gemma-3-4b: User location not supported for this model/API
  ❌ qwen/qwen3-235b-a22b-instruct-2507: User location not supported for this model/API
  ❌ qwen/qwen3-coder-480b-a35b-instruct: User location not supported for this model/API
  ❌ qwen/qwen3-next-80b-a3b-instruct: User location not supported for this model/API
  ❌ qwen/qwen3-next-80b-a3b-thinking: User location not supported for this model/API
  ❌ zai/glm-5: User location not supported for this model/API

Error Output / Traceback

Environment Info

LLM Model Used

No response

Additional Context

No response

Pre-submission Checklist

• I have searched the existing issues and this bug has not been reported before.
• I am using the latest version of kaggle-benchmarks (or have noted my version above).

[gastondana627]

Hey @DanGlChris I dug into this today and wanted to share what I found.

Root cause: This error originates server-side from Kaggle's model proxy backend, not from the kbench library itself. The PermissionDeniedError: User location not supported is being returned by the proxy before it even reaches the model providers. Since all 27 models across all providers (Anthropic, DeepSeek, Google, Qwen, ZAI) are failing simultaneously, this points to a backend configuration change on Kaggle's side between March 26–27, not anything in user code or the library itself.

This is the same underlying issue reported in #85 (where @yibinlin-google was assigned but hasn't posted an update yet).

What I've done: Opened PR #100 which adds better error handling in model_proxy.py so that when this error fires, users get a clear, actionable message explaining it's a backend restriction rather than a raw OpenAI traceback. It won't fix the backend outage itself, but it will make the failure mode much easier to understand for anyone hitting it.

Tagging @yibinlin-google in case this is connected to the same investigation from #85.

[neshdev]

This is not working because of a restricted region. See https://ai.google.dev/gemini-api/docs/available-regions
We updated the error message to be more descriptive such as which region is being blocked.

[DanGlChris]

I suspect the issue is unrelated to regional restrictions. My primary reason is that I am not located in a restricted region for Gemini. Furthermore, even if that were the case, the various models provided do not share the same restriction policies, so a region-based problem would not affect them all uniformly. The root cause must be something else

[neshdev]

It should post the message with the associated region. Have you retried running a model and see which region is showing up?

[DanGlChris]

Yes, it shows the region (RU), but I'm no longer in that location. Did Kaggle check my location based on the phone number I used? I created my Kaggle account when I was there and used an old phone number.