refactor(helm): require external postgres for ha (#1844)

Author: TaylorMutch

.agents/skills/debug-openshell-cluster/SKILL.md

@@ -138,13 +138,14 @@ kubectl -n openshell rollout status statefulset/openshell
 
 Look for failed installs, unexpected values, missing namespace, wrong image tag, TLS settings that do not match the registered endpoint, and scheduling failures.
 
-For HA or PostgreSQL-backed installs, also check the service-binding Secret and
-bundled PostgreSQL workload:
+For HA or PostgreSQL-backed installs, also check the external database Secret
+referenced by `server.externalDbSecret` and the PostgreSQL workload if the test
+or operator deployed one in-cluster:
 
 ```bash
-kubectl -n openshell get secret -l app.kubernetes.io/instance=openshell
-kubectl -n openshell get statefulset,pod,pvc -l app.kubernetes.io/instance=openshell
-kubectl -n openshell logs statefulset/openshell-postgres --tail=200
+kubectl -n openshell get secret openshell-ha-pg -o yaml
+kubectl -n openshell get deployment,service,pod -l app.kubernetes.io/name=openshell-e2e-postgres
+kubectl -n openshell logs deployment/openshell-e2e-postgres --tail=200
 ```
 
 Check required Helm deployment secrets:

.agents/skills/helm-dev-environment/SKILL.md

@@ -66,9 +66,10 @@ generates mTLS secrets on first install. Envoy Gateway opt-in; see the Optional
 
 The gateway Service uses ClusterIP. Access is via Envoy Gateway (port `8080`) or `kubectl port-forward`.
 
-**HA test deploy** (two gateway replicas + bundled PostgreSQL): uncomment
+**HA test deploy** (two gateway replicas + external PostgreSQL Secret): uncomment
 `#- ci/values-high-availability.yaml` in `deploy/helm/openshell/skaffold.yaml`,
-then run `mise run helm:skaffold:run` or `mise run helm:skaffold:dev`.
+create the Secret named `openshell-ha-pg` with a `uri` key, then run
+`mise run helm:skaffold:run` or `mise run helm:skaffold:dev`.
 
 ### TLS behaviour
 
@@ -203,7 +204,7 @@ mise run helm:k3s:status
 | `deploy/helm/openshell/ci/values-skaffold.yaml` | Dev overrides (image pull policy, TLS disabled for local Skaffold) |
 | `deploy/helm/openshell/ci/values-cert-manager.yaml` | cert-manager PKI overlay (opt-in; disables pkiInitJob) |
 | `deploy/helm/openshell/ci/values-gateway.yaml` | Envoy Gateway GRPCRoute + Gateway overlay |
-| `deploy/helm/openshell/ci/values-high-availability.yaml` | HA test overlay (`replicaCount: 2` with bundled PostgreSQL) |
+| `deploy/helm/openshell/ci/values-high-availability.yaml` | HA test overlay (`replicaCount: 2` with external PostgreSQL Secret) |
 | `deploy/helm/openshell/ci/values-keycloak.yaml` | Keycloak OIDC overlay |
 | `deploy/helm/openshell/ci/values-tls-disabled.yaml` | Lint-only: TLS + auth disabled (reverse-proxy edge termination) |
 | `deploy/kube/manifests/envoy-gateway-openshell.yaml` | GatewayClass for Envoy Gateway (`mise run helm:gateway:apply`) |

.github/actions/release-helm-oci/action.yml

@@ -71,14 +71,6 @@ runs:
           exit 1
         fi
 
-    - name: Build chart dependencies
-      env:
-        CHART_DIR: ${{ steps.prep.outputs.chart_dir }}
-      shell: bash
-      run: |
-        set -euo pipefail
-        helm dependency build "${CHART_DIR}"
-
     - name: Package Helm chart
       env:
         CHART_DIR: ${{ steps.prep.outputs.chart_dir }}

.github/workflows/branch-e2e.yml

@@ -122,6 +122,7 @@ jobs:
       image-tag: ${{ github.sha }}
       job-name: Kubernetes HA E2E (Rust smoke)
       extra-helm-values: deploy/helm/openshell/ci/values-high-availability.yaml
+      external-postgres-secret: openshell-ha-pg
 
   core-e2e-result:
     name: Core E2E result

.github/workflows/e2e-kubernetes-test.yml

@@ -27,6 +27,11 @@ on:
         required: false
         type: string
         default: ""
+      external-postgres-secret:
+        description: "Create an ephemeral external PostgreSQL fixture and write its URI to this Secret"
+        required: false
+        type: string
+        default: ""
       mise-version:
         description: "mise version to install on the bare Kubernetes e2e runner"
         required: false
@@ -111,6 +116,7 @@ jobs:
         env:
           OPENSHELL_E2E_KUBE_CONTEXT: kind-${{ env.KIND_CLUSTER_NAME }}
           OPENSHELL_E2E_KUBE_EXTRA_VALUES: ${{ inputs.extra-helm-values }}
+          OPENSHELL_E2E_KUBE_EXTERNAL_POSTGRES_SECRET: ${{ inputs.external-postgres-secret }}
           IMAGE_TAG: ${{ inputs.image-tag }}
           OPENSHELL_REGISTRY: ghcr.io/nvidia/openshell
         run: mise run --no-deps --skip-deps e2e:kubernetes

architecture/compute-runtimes.md

@@ -86,7 +86,9 @@ runtime still owns GPU device injection.
 
 ## Deployment Shape
 
-Kubernetes deployments use the Helm chart under `deploy/helm/openshell`.
+Kubernetes deployments use the Helm chart under `deploy/helm/openshell`. The
+chart deploys the gateway and sandbox runtime integration, but HA deployments
+must point `server.externalDbSecret` at an operator-managed PostgreSQL database.
 Standalone local deployments start the gateway with a selected runtime such as
 Docker, Podman, or VM. The CLI can register multiple gateways and switch between
 them without changing the sandbox architecture.

deploy/helm/openshell/Chart.lock

@@ -11,9 +11,3 @@ type: application
 # empty), so a released chart automatically pulls the matching gateway and supervisor images.
 version: 0.0.0
 appVersion: "0.0.0"
-dependencies:
-  - name: postgresql
-    version: 18.6.7
-    repository: oci://registry-1.docker.io/bitnamicharts
-    condition: postgres.enabled
-    alias: postgres

deploy/helm/openshell/Chart.yaml

@@ -56,7 +56,7 @@ See [`values.yaml`](values.yaml) for source defaults. Selected overlays:
 - [`ci/values-gateway.yaml`](ci/values-gateway.yaml) - gateway-only configuration
 - [`ci/values-cert-manager.yaml`](ci/values-cert-manager.yaml) - cert-manager integration
 - [`ci/values-keycloak.yaml`](ci/values-keycloak.yaml) - Keycloak OIDC integration
-- [`ci/values-high-availability.yaml`](ci/values-high-availability.yaml) - HA gateway test overlay with bundled PostgreSQL
+- [`ci/values-high-availability.yaml`](ci/values-high-availability.yaml) - CI overlay for multi-replica external PostgreSQL testing
 
 ### Database backend
 
@@ -65,12 +65,15 @@ By default, OpenShell uses SQLite:
 ```yaml
 server:
   dbUrl: "sqlite:/var/openshell/openshell.db"
-postgres:
-  enabled: false
 ```
 
 #### External PostgreSQL
 
+Use external PostgreSQL when the gateway should connect to a database managed
+outside this chart. The OpenShell chart does not deploy a database; install
+PostgreSQL separately using the chart, operator, or managed service that fits
+your environment, then pass the connection URI through a Secret.
+
 Create a Secret containing the PostgreSQL connection URI if one does not
 already exist:
 
@@ -87,18 +90,6 @@ helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <vers
   --set server.externalDbSecret=my-pg-credentials
 ```
 
-#### Bundled PostgreSQL
-
-Deploy a PostgreSQL instance alongside the gateway using the bundled
-Bitnami subchart. A random password is generated automatically:
-
-```bash
-helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
-  --set postgres.enabled=true
-```
-
-To set an explicit password, add `--set postgres.auth.password=my-secret-password`.
-
 #### OpenShift
 
 Append these flags to any of the PostgreSQL commands above for OpenShift:
@@ -159,12 +150,6 @@ JWT signing Secret.
 | podLabels | object | `{}` | Extra labels to add to the gateway pod. |
 | podLifecycle.terminationGracePeriodSeconds | int | `5` | Grace period, in seconds, before Kubernetes terminates the gateway pod. |
 | podSecurityContext.fsGroup | int | `1000` | fsGroup assigned to the gateway pod. |
-| postgres.auth.database | string | `"openshell"` |  |
-| postgres.auth.password | string | `""` |  |
-| postgres.auth.username | string | `"openshell"` |  |
-| postgres.enabled | bool | `false` | Deploy the bundled Bitnami PostgreSQL subchart. |
-| postgres.primary.persistence.enabled | bool | `true` |  |
-| postgres.serviceBindings.enabled | bool | `true` |  |
 | probes.liveness.failureThreshold | int | `3` | Liveness probe failure threshold before the container is restarted. |
 | probes.liveness.initialDelaySeconds | int | `2` | Liveness probe initial delay, in seconds. |
 | probes.liveness.periodSeconds | int | `5` | Liveness probe period, in seconds. |
@@ -176,7 +161,7 @@ JWT signing Secret.
 | probes.startup.failureThreshold | int | `30` | Startup probe failure threshold before the container is killed. |
 | probes.startup.periodSeconds | int | `2` | Startup probe period, in seconds. |
 | probes.startup.timeoutSeconds | int | `1` | Startup probe timeout, in seconds. |
-| replicaCount | int | `1` | Number of OpenShell gateway replicas. |
+| replicaCount | int | `1` | Number of OpenShell gateway replicas. Values greater than 1 require server.externalDbSecret because the default SQLite backend is per pod. |
 | resources | object | `{}` | Gateway pod resource requests and limits. |
 | sandboxServiceAccount.annotations | object | `{}` | Annotations to add to the generated sandbox service account. |
 | sandboxServiceAccount.create | bool | `true` | Create a service account for sandbox pods. |

deploy/helm/openshell/README.md

@@ -56,7 +56,7 @@ See [`values.yaml`](values.yaml) for source defaults. Selected overlays:
 - [`ci/values-gateway.yaml`](ci/values-gateway.yaml) - gateway-only configuration
 - [`ci/values-cert-manager.yaml`](ci/values-cert-manager.yaml) - cert-manager integration
 - [`ci/values-keycloak.yaml`](ci/values-keycloak.yaml) - Keycloak OIDC integration
-- [`ci/values-high-availability.yaml`](ci/values-high-availability.yaml) - HA gateway test overlay with bundled PostgreSQL
+- [`ci/values-high-availability.yaml`](ci/values-high-availability.yaml) - CI overlay for multi-replica external PostgreSQL testing
 
 ### Database backend
 
@@ -65,12 +65,15 @@ By default, OpenShell uses SQLite:
 ```yaml
 server:
   dbUrl: "sqlite:/var/openshell/openshell.db"
-postgres:
-  enabled: false
 ```
 
 #### External PostgreSQL
 
+Use external PostgreSQL when the gateway should connect to a database managed
+outside this chart. The OpenShell chart does not deploy a database; install
+PostgreSQL separately using the chart, operator, or managed service that fits
+your environment, then pass the connection URI through a Secret.
+
 Create a Secret containing the PostgreSQL connection URI if one does not
 already exist:
 
@@ -87,18 +90,6 @@ helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <vers
   --set server.externalDbSecret=my-pg-credentials
 ```
 
-#### Bundled PostgreSQL
-
-Deploy a PostgreSQL instance alongside the gateway using the bundled
-Bitnami subchart. A random password is generated automatically:
-
-```bash
-helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
-  --set postgres.enabled=true
-```
-
-To set an explicit password, add `--set postgres.auth.password=my-secret-password`.
-
 #### OpenShift
 
 Append these flags to any of the PostgreSQL commands above for OpenShift: