Skip to content

fix(litellm): provision Vertex sandbox runtime - #985

Open
kywch wants to merge 2 commits into
mainfrom
fix/vertex-litellm-sandbox
Open

fix(litellm): provision Vertex sandbox runtime#985
kywch wants to merge 2 commits into
mainfrom
fix/vertex-litellm-sandbox

Conversation

@kywch

@kywch kywch commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

Provision Vertex dependencies and ADC credentials inside sandboxed LiteLLM proxies.

Root cause

Vertex LiteLLM runs in a separate Daytona sandbox venv. #978 added google-cloud-aiplatform to BenchFlow host dependencies, but host packages are not visible in that proxy venv. The proxy also needed a local ADC credential-file path and explicit Vertex project/location.

Changes

  • Install google-cloud-aiplatform only for sandbox Vertex routes.
  • Stage ADC JSON as a sandbox-local credential file.
  • Remove raw ADC JSON before proxy launch config serialization.
  • Forward Vertex project/location to LiteLLM.

Scope

Complements #978. Does not change host provisioning or non-Vertex routes.

Validation

  • 74 focused tests passed.
  • git diff --check passed.

@kywch
kywch deployed to pypi-internal-preview August 14, 2026 02:01 — with GitHub Actions Active
@kywch
kywch deployed to pypi-internal-preview August 14, 2026 02:05 — with GitHub Actions Active
@kywch
kywch marked this pull request as ready for review August 14, 2026 02:11
@kywch
kywch deployed to pypi-internal-preview August 14, 2026 02:11 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant