Skip to content

GCP: Missing IAM roles and API enablement for VPC networking in SetUpCD() #2002

@simple-agent-manager

Description

@simple-agent-manager

Problem

When deploying a Compose file that uses managed Postgres (Cloud SQL) or Redis with private networking on GCP, the deployment fails during Pulumi execution because defang-cd lacks the IAM roles needed to create VPC peering connections and Serverless VPC Access connectors.

Root Cause

SetUpCD() in src/pkg/cli/client/byoc/gcp/byoc.go has three gaps:

1. Missing API: vpcaccess.googleapis.com

The API enablement list (lines 181-197) includes servicenetworking.googleapis.com but not vpcaccess.googleapis.com. The CD SA has roles/serviceusage.serviceUsageAdmin so it could enable it itself during Pulumi execution, but if the CD image doesn't do that explicitly, the Pulumi run fails when it tries to create a VPC Access Connector.

2. Missing IAM roles for defang-cd service account

The role list at lines 222-243 includes roles/compute.networkAdmin but is missing:

  • roles/servicenetworking.networksAdmin — required to create gcp:servicenetworking:Connection resources (VPC peering for Cloud SQL private IP). compute.networkAdmin does not include servicenetworking.* permissions.
  • roles/vpcaccess.admin — required to create Serverless VPC Access connectors that allow Cloud Run services to reach VPC-attached resources (Cloud SQL, Redis).

3. Missing serviceAccountTokenCreator self-binding for defang-cd

The CLI grants serviceAccountTokenCreator to the current principal on the defang-upload SA (line 266), but defang-cd does not get this role on itself. During Pulumi execution in Cloud Build, the CD SA needs to create tokens for itself when provisioning service accounts for deployed services.

How to Reproduce

  1. Create a compose.yaml with a service that depends on a managed Postgres database (or Redis)
  2. Run defang compose up targeting a GCP project
  3. Observe Pulumi failure during DB/networking provisioning

Suggested Fix

In src/pkg/cli/client/byoc/gcp/byoc.go:

Add missing API (around line 193):

"vpcaccess.googleapis.com",           // For Serverless VPC Access connectors

Add missing roles (around line 242):

"roles/servicenetworking.networksAdmin", // For VPC peering (Cloud SQL private IP)
"roles/vpcaccess.admin",                 // For Serverless VPC Access connectors

Add self-token-creator for defang-cd (after line 245):

// Give CD service account token creator on itself for self-impersonation during Pulumi
if err := b.driver.EnsurePrincipalHasServiceAccountRoles(ctx, "serviceAccount:"+b.cdServiceAccount, b.cdServiceAccount, []string{"roles/iam.serviceAccountTokenCreator"}); err != nil {
    return err
}

Environment

  • Defang CLI version: latest
  • Cloud provider: GCP
  • Authentication: Workload Identity Federation (service account impersonation)
  • Compose services: web app + managed Postgres

Workaround

Manually grant the missing roles and enable the missing API:

# Enable missing API
gcloud services enable vpcaccess.googleapis.com --project=$PROJECT_ID

# Grant missing roles to defang-cd
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:defang-cd@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/servicenetworking.networksAdmin"

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:defang-cd@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/vpcaccess.admin"

# Grant self-token-creator to defang-cd
gcloud iam service-accounts add-iam-policy-binding \
  defang-cd@$PROJECT_ID.iam.gserviceaccount.com \
  --project=$PROJECT_ID \
  --member="serviceAccount:defang-cd@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/iam.serviceAccountTokenCreator"

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions