Skip to content

fix(vertex): EU/US REP + global endpoints, and fix v@ routing fall-through to OpenRouter#145

Open
nickoloss wants to merge 1 commit into
MadAppGang:mainfrom
nickoloss:feat/vertex-rep-global-endpoints
Open

fix(vertex): EU/US REP + global endpoints, and fix v@ routing fall-through to OpenRouter#145
nickoloss wants to merge 1 commit into
MadAppGang:mainfrom
nickoloss:feat/vertex-rep-global-endpoints

Conversation

@nickoloss

Copy link
Copy Markdown

Problem

The Vertex AI OAuth endpoint builder hardcodes a single host shape — https://<location>-aiplatform.googleapis.com — derived from VERTEX_LOCATION. That cannot address two endpoint families Vertex actually uses:

  1. Data-residency multi-region (REP)https://aiplatform.<loc>.rep.googleapis.com (e.g. aiplatform.eu.rep.googleapis.com, location=eu).
  2. Globalhttps://aiplatform.googleapis.com with location=global (note: no <location>- prefix).

The practical impact: Gemini models published only on those endpoints can't be used at all. gemini-3.5-flash is currently served only on the EU REP endpoint and the global endpoint — never on the per-region europe-west4-aiplatform.googleapis.com host the code builds. So EU users have no way to run 3.5 Flash with data residency.

Evidence (live :countTokens probes against a real project)

Endpoint (host / location) gemini-3.5-flash gemini-2.5-pro builder host before this PR
aiplatform.eu.rep.googleapis.com / eu ✅ 200 ❌ 404 ❌ unreachable
aiplatform.googleapis.com / global ✅ 200 ✅ 200 ❌ builds global-aiplatform… → 404
europe-west4-aiplatform.googleapis.com / europe-west4 ❌ 404 ✅ 200
eu-aiplatform.googleapis.com / eu (current output for VERTEX_LOCATION=eu) ❌ 404 ❌ 404 ✅ but dead host

So VERTEX_LOCATION=eu today produces eu-aiplatform.googleapis.com, which is not a valid Gemini host; the real EU multi-region host is aiplatform.eu.rep.googleapis.com.

Change

Add vertexApiHost(location) and use it in buildVertexOAuthEndpoint (and the smoke base-URL builder):

export function vertexApiHost(location: string): string {
  if (location === "global") return "aiplatform.googleapis.com";
  if (location === "eu" || location === "us") {
    return `aiplatform.${location}.rep.googleapis.com`;
  }
  return `${location}-aiplatform.googleapis.com`; // unchanged for normal regions
}

The locations/<location> path segment is untouched in every case. Existing single-region behavior (europe-west4, us-central1, …) is byte-for-byte unchanged.

Usage after this PR:

export VERTEX_PROJECT=my-project
export VERTEX_LOCATION=eu          # data-residency multi-region
claudish --model v@gemini-3.5-flash "hi"

Tests / verification

  • Added packages/cli/src/auth/vertex-auth.test.ts covering vertexApiHost and buildVertexOAuthEndpoint for eu (REP), global, and a normal region, plus the ?alt=sse streaming path.
  • The exact URLs the helper generates for eu, global, and europe-west4 were each confirmed to return HTTP 200 from Vertex via :countTokens.
  • I could not run bun test in my environment (bun not installed here) — please confirm CI is green.

Docs

  • docs/settings-reference.md: documented eu/us/global behavior for VERTEX_LOCATION.
  • CHANGELOG.md: [Unreleased] entry.

Refs: Vertex AI — locations/REP endpoints, data residency.

🤖 Generated with Claude Code

@nickoloss

Copy link
Copy Markdown
Author

Update: ran the suite locally on bun 1.3.14 — the new vertex-auth.test.ts passes (7 pass / 0 fail), and a bun build --compile binary builds cleanly. The generated hosts for eu (aiplatform.eu.rep.googleapis.com), global (aiplatform.googleapis.com), and europe-west4 were each verified to return HTTP 200 from Vertex :countTokens.

…hrough

Two related Vertex fixes so EU users can run Gemini models with data
residency (e.g. gemini-3.5-flash, which is published only on the EU REP
endpoint).

1. Endpoint host resolution (vertex-auth.ts): the OAuth endpoint builder
   hardcoded <location>-aiplatform.googleapis.com, which can't address the
   data-residency multi-region REP hosts (aiplatform.<loc>.rep.googleapis.com
   for eu/us) or the global host (aiplatform.googleapis.com). Add
   vertexApiHost(location); per-region behavior is unchanged.

2. Provider registry (remote-provider-registry.ts): Vertex has a static
   baseUrl of "" (endpoint built per-region in the transport), so the
   empty-baseUrl filter in getRemoteProviders() dropped it. resolveRemoteProvider
   returned null, resolveModelProvider categorized it "unknown", and every
   Vertex request silently fell through to the OpenRouter default -> HTTP 401.
   Keep providers whose transport self-constructs the URL.

Verified end-to-end: VERTEX_LOCATION=eu claudish --model v@gemini-3.5-flash
reaches Vertex and returns a response. Adds unit tests for both, docs, and a
.gitignore entry for the compiled binary / bun-build artifacts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@nickoloss nickoloss force-pushed the feat/vertex-rep-global-endpoints branch from 01fc4a5 to f8c47f2 Compare June 18, 2026 07:11
@nickoloss nickoloss changed the title feat(vertex): support EU/US REP multi-region and global endpoints fix(vertex): EU/US REP + global endpoints, and fix v@ routing fall-through to OpenRouter Jun 18, 2026
@nickoloss

Copy link
Copy Markdown
Author

Expanded this PR with a second, more fundamental Vertex bug found while testing the endpoint fix end-to-end:

v@/vertex@ models silently fall through to the OpenRouter default → HTTP 401. Vertex's provider definition has a static baseUrl: "" (its endpoint is built per-region in the vertex transport). The empty-baseUrl filter in getRemoteProviders() (added for LiteLLM) therefore drops Vertex from the registry, so resolveRemoteProvider("v@...") returns null, resolveModelProvider categorizes it "unknown", and the proxy never builds the Vertex handler — every Vertex request defaults to OpenRouter. Repro:

$ VERTEX_PROJECT=… claudish --model v@gemini-3.5-flash "hi"
[claudish] Error [OpenRouter]: HTTP 401. Check API key / OAuth credentials.

Fix: keep providers whose transport self-constructs the URL (transport === "vertex").

Verified end-to-end on a compiled binary (bun 1.3.14), both fixes together:

$ VERTEX_LOCATION=eu VERTEX_PROJECT=… claudish --model v@gemini-3.5-flash "Reply with exactly: VERTEX_OK"
VERTEX_OK

That round-trips Gemini 3.5 Flash through the EU data-residency REP endpoint. Tests: 10 pass / 0 fail across vertex-auth.test.ts and remote-provider-registry.test.ts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant