NVIDIA · TaylorMutch · Jul 2, 2026 · Jul 2, 2026 · Jul 2, 2026 · Jul 3, 2026
@@ -268,6 +268,48 @@ kubectl -n openshell get configmap openshell-config -o jsonpath='{.data.gateway\
 kubectl -n <sandbox-namespace> get sandbox <sandbox-name> -o jsonpath='{.spec.template.spec.serviceAccountName}{"\n"}'
 ```
 
+If `supervisor_topology = "sidecar"` is rendered, sandbox pods should have an
+`openshell-network-init` init container running `--mode=network-init`, an
+`agent` container running `openshell-sandbox --mode=process`, and an
+`openshell-supervisor-network` container running `--mode=network`. The init
+container owns nftables setup and should be the only sidecar topology container
+with `NET_ADMIN`. It also needs `CHOWN`/`FOWNER` to hand shared emptyDir state
+to `sidecar_proxy_uid`. The long-running network sidecar runs as
+`sidecar_proxy_uid` with primary GID `sandbox_gid`; the pod `fsGroup` is also
+set to `sandbox_gid`.
+
+In sidecar topology only the network sidecar should mount the gateway bootstrap
+credentials (`openshell-sa-token` and `openshell-client-tls`). The process
+container should not receive `OPENSHELL_ENDPOINT`, gateway TLS env vars, the
+sandbox token file, or those credential mounts. Instead, the network sidecar
+writes `/run/openshell-sidecar/policy.pb` and
+`/run/openshell-sidecar/provider-env.json`, then writes the readiness file. If
+the process supervisor fails before launching the workload, inspect those
+snapshot files and the network sidecar logs. If new SSH/exec sessions do not
+pick up refreshed provider environment, inspect the provider-env snapshot
+revision and network sidecar settings-poll logs; the process container should
+consume newer provider-env snapshot revisions without receiving gateway
+credentials.
+
+The process container should also publish the workload entrypoint PID to
+`OPENSHELL_ENTRYPOINT_PID_FILE`
+(`/run/openshell-sidecar/entrypoint.pid` by default), and the network sidecar
+should read it for binary-scoped policy decisions; if allowed network rules are
+all denied, inspect that file and the network sidecar logs.
+The shared state directory should preserve `sandbox_gid` inheritance
+(`02775`), and the SSH socket should be group-connectable (`0660`) so the
+network sidecar can bridge gateway relay requests to the process supervisor.
+Inspect all three when sandbox registration or egress enforcement fails:
+
+```bash
+kubectl -n openshell get configmap openshell-config -o jsonpath='{.data.gateway\.toml}' | grep supervisor_topology
+kubectl -n <sandbox-namespace> get pod <sandbox-pod> -o jsonpath='{range .spec.initContainers[*]}{.name}{" "}{.command}{"\n"}{end}'
+kubectl -n <sandbox-namespace> get pod <sandbox-pod> -o jsonpath='{range .spec.containers[*]}{.name}{" "}{.command}{"\n"}{end}'
+kubectl -n <sandbox-namespace> logs <sandbox-pod> -c openshell-network-init --tail=200
+kubectl -n <sandbox-namespace> logs <sandbox-pod> -c openshell-supervisor-network --tail=200
+kubectl -n <sandbox-namespace> logs <sandbox-pod> -c agent --tail=200
+```
+
 ### Step 6: Check VM-Backed Gateways
 
 Use the VM driver logs and host diagnostics available in the user's environment. Verify:

@@ -60,9 +60,22 @@ mise run helm:skaffold:dev
 mise run helm:skaffold:run
 ```
 
+**Supervisor sidecar topology** (build once and leave running):
+```bash
+mise run helm:skaffold:run:sidecar
+```
+
+**Supervisor sidecar topology with TLS/mTLS enabled** (build once and leave running):
+```bash
+mise run helm:skaffold:run:sidecar-mtls
+```
+
 Both commands build the `gateway` and `supervisor` images and deploy the OpenShell Helm
-chart. The `pkiInitJob` hook (a pre-install Job that runs `openshell-gateway generate-certs`)
-generates mTLS secrets on first install. Envoy Gateway opt-in; see the Optional Add-ons section below.
+chart. The sidecar profile renders an `openshell-network-init` init container for
+nftables setup and a non-root `openshell-supervisor-network` runtime sidecar for
+proxying. The `pkiInitJob` hook (a pre-install Job that runs `openshell-gateway
+generate-certs`) generates mTLS secrets on first install. Envoy Gateway opt-in;
+see the Optional Add-ons section below.
 
 The gateway Service uses ClusterIP. Access is via Envoy Gateway (port `8080`) or `kubectl port-forward`.
 
@@ -126,6 +139,12 @@ openshell sandbox list --gateway-endpoint https://localhost:8090
 mise run helm:skaffold:delete
 ```
 
+For a sidecar-profile deployment:
+
+```bash
+mise run helm:skaffold:delete:sidecar
+```
+
 ### Delete the cluster entirely
 
 ```bash
@@ -250,6 +269,8 @@ for dependencies still declared in `Chart.yaml`.
 | `deploy/helm/openshell/ci/values-gateway.yaml` | Envoy Gateway GRPCRoute + Gateway overlay |
 | `deploy/helm/openshell/ci/values-high-availability.yaml` | HA test overlay (`replicaCount: 2` with external PostgreSQL Secret) |
 | `deploy/helm/openshell/ci/values-keycloak.yaml` | Keycloak OIDC overlay |
+| `deploy/helm/openshell/ci/values-sidecar.yaml` | Supervisor sidecar topology overlay for Kubernetes e2e/dev |
+| `deploy/helm/openshell/ci/values-sidecar-mtls.yaml` | Supervisor sidecar topology overlay with built-in TLS/mTLS re-enabled after Skaffold dev values |
 | `deploy/helm/openshell/ci/values-spire.yaml` | SPIFFE/SPIRE provider token grant overlay |
 | `deploy/helm/openshell/ci/values-spire-stack.yaml` | SPIRE hardened chart values for local dev |
 | `deploy/helm/openshell/ci/values-tls-disabled.yaml` | Lint-only: TLS + auth disabled (reverse-proxy edge termination) |

@@ -121,16 +121,25 @@ jobs:
         include:
           - agent_sandbox_api: v1beta1
             agent_sandbox_version: v0.5.0
+            topology: combined
+            extra_helm_values: ""
           - agent_sandbox_api: v1alpha1
             agent_sandbox_version: v0.4.6
+            topology: combined
+            extra_helm_values: ""
+          - agent_sandbox_api: v1beta1
+            agent_sandbox_version: v0.5.0
+            topology: sidecar
+            extra_helm_values: deploy/helm/openshell/ci/values-sidecar.yaml
     permissions:
       contents: read
       packages: read
     uses: ./.github/workflows/e2e-kubernetes-test.yml
     with:
       image-tag: ${{ github.sha }}
-      job-name: Kubernetes E2E (Rust smoke, Agent Sandbox ${{ matrix.agent_sandbox_api }})
+      job-name: Kubernetes E2E (Rust smoke, ${{ matrix.topology }}, Agent Sandbox ${{ matrix.agent_sandbox_api }})
       agent-sandbox-version: ${{ matrix.agent_sandbox_version }}
+      extra-helm-values: ${{ matrix.extra_helm_values }}
 
   kubernetes-ha-e2e:
     needs: [pr_metadata, build-gateway, build-supervisor]

@@ -91,10 +91,11 @@ Runtime layout:
   as a release artifact. Linux GNU VM driver binaries must not reference
   `GLIBC_*` symbols newer than `GLIBC_2.28`; release workflows verify this
   before publishing artifacts.
-- **Supervisor**: `scratch` base, static musl binary at `/openshell-sandbox`.
-  Static linkage is required because the image is mounted/extracted into
-  sandbox environments (Docker extraction, Podman image volumes, Kubernetes
-  init-container copy-self) and cannot rely on a dynamic loader.
+- **Supervisor**: Alpine base with `nftables`, static musl binary at
+  `/openshell-sandbox`. Static linkage keeps the binary usable when the image
+  is mounted/extracted into sandbox environments (Docker extraction, Podman
+  image volumes, Kubernetes init-container copy-self), while `nftables` supports
+  Kubernetes supervisor sidecar egress enforcement.
 
 Gateway image builds bake the corresponding supervisor image tag into the
 gateway binary so Docker sandboxes do not depend on `:latest` by default.

@@ -81,14 +81,33 @@ The supervisor must be available inside each sandbox workload:
 |---|---|
 | Docker | Bind-mounted local supervisor binary, or a binary extracted from the configured supervisor image. |
 | Podman | Read-only OCI image volume containing the supervisor binary. |
-| Kubernetes | Sandbox pod image or pod template configuration. |
+| Kubernetes | Supervisor image side-loaded into the sandbox pod by image volume or init container. |
 | VM | Embedded in the guest rootfs bundle. |
 | Extension | Defined by the out-of-tree driver. |
 
 Driver-controlled environment variables must override sandbox image or template
 values for sandbox ID, sandbox name, gateway endpoint, relay socket path, TLS
 paths, and command metadata.
 
+Kubernetes can run the supervisor in the default combined topology or in a
+sidecar topology. Combined mode keeps network and process supervision in the
+agent container. Sidecar mode runs network enforcement, the proxy, and gateway
+session in a dedicated sidecar, while the agent container runs only the
+process-supervision leaf and launches the user workload after the sidecar
+signals readiness. The sidecar writes local policy and provider-environment
+snapshots into shared state so the process leaf can start without gateway
+credentials. The network sidecar also refreshes the workload-facing provider
+environment snapshot after settings polls so future process sessions see
+updated provider env without giving the process leaf gateway access. In sidecar
+mode, an init container performs the privileged pod-network nftables setup with
+`NET_ADMIN` and hands shared state ownership to the configured proxy UID; the
+long-running network sidecar runs as that UID and does not keep `NET_ADMIN`.
+The agent container runs as the resolved sandbox UID/GID with no added Linux
+capabilities. Sidecar mode preserves gateway session and SSH behavior, but
+treats the process leaf as network-only by default: Landlock filesystem policy,
+process privilege dropping, and process/binary identity checks are not applied
+there.
+
 ## Images
 
 The gateway image and Helm chart are built from this repository. Sandbox images

@@ -167,9 +167,14 @@ async fn build_plain_channel(endpoint: &str) -> Result<Channel> {
             .into_diagnostic()
             .wrap_err_with(|| format!("failed to read client key from {key_path}"))?;
 
-        let tls_config = ClientTlsConfig::new()
+        let mut tls_config = ClientTlsConfig::new()
             .ca_certificate(Certificate::from_pem(ca_pem))
             .identity(Identity::from_pem(cert_pem, key_pem));
+        if let Ok(server_name) = std::env::var(sandbox_env::GATEWAY_TLS_SERVER_NAME)
+            && !server_name.is_empty()
+        {
+            tls_config = tls_config.domain_name(server_name);
+        }
 
         ep = ep
             .tls_config(tls_config)

@@ -63,6 +63,63 @@ impl ProviderCredentialState {
         }
     }
 
+    /// Build a static provider state from an already-prepared child
+    /// environment snapshot.
+    ///
+    /// Kubernetes sidecar topology uses this in the process-only supervisor:
+    /// the network sidecar owns provider credential resolvers and writes the
+    /// workload-facing env map into a shared local snapshot. The process leaf
+    /// must inject that map into child processes without re-placeholderizing it
+    /// or holding the gateway-side resolver material.
+    pub fn from_child_env_snapshot(revision: u64, child_env: HashMap<String, String>) -> Self {
+        let snapshot = Arc::new(ProviderCredentialSnapshot {
+            revision,
+            child_env,
+            dynamic_credentials: HashMap::new(),
+        });
+
+        Self {
+            inner: Arc::new(RwLock::new(ProviderCredentialStateInner {
+                current: snapshot,
+                generations: VecDeque::new(),
+                current_resolver: None,
+                combined_resolver: None,
+                suppressed_keys: HashSet::new(),
+            })),
+        }
+    }
+
+    /// Install an already-prepared child environment snapshot.
+    ///
+    /// This is intentionally narrower than [`Self::install_environment`]: it
+    /// updates only the workload-facing env map and clears resolver state so a
+    /// process that does not own gateway/provider resolver material can still
+    /// pick up refreshed provider env for future child processes.
+    pub fn install_child_env_snapshot(
+        &self,
+        revision: u64,
+        mut child_env: HashMap<String, String>,
+    ) -> usize {
+        let mut inner = self
+            .inner
+            .write()
+            .expect("provider credential state poisoned");
+
+        for key in &inner.suppressed_keys {
+            child_env.remove(key);
+        }
+
+        inner.current = Arc::new(ProviderCredentialSnapshot {
+            revision,
+            child_env,
+            dynamic_credentials: HashMap::new(),
+        });
+        inner.generations.clear();
+        inner.current_resolver = None;
+        inner.combined_resolver = None;
+        inner.current.child_env.len()
+    }
+
     pub fn snapshot(&self) -> Arc<ProviderCredentialSnapshot> {
         self.inner
             .read()
@@ -594,6 +651,39 @@ mod tests {
         );
     }
 
+    #[test]
+    fn child_env_snapshot_install_updates_env_without_resolver_material() {
+        let state = ProviderCredentialState::from_child_env_snapshot(
+            1,
+            HashMap::from([
+                ("GITHUB_TOKEN".to_string(), "old".to_string()),
+                ("GCE_METADATA_HOST".to_string(), "marker".to_string()),
+            ]),
+        );
+        state.remove_env_key("GCE_METADATA_HOST");
+
+        let env_count = state.install_child_env_snapshot(
+            2,
+            HashMap::from([
+                ("GITHUB_TOKEN".to_string(), "new".to_string()),
+                ("GCE_METADATA_HOST".to_string(), "marker".to_string()),
+            ]),
+        );
+
+        let snapshot = state.snapshot();
+        assert_eq!(snapshot.revision, 2);
+        assert_eq!(env_count, 1);
+        assert_eq!(
+            snapshot.child_env.get("GITHUB_TOKEN").map(String::as_str),
+            Some("new")
+        );
+        assert!(!snapshot.child_env.contains_key("GCE_METADATA_HOST"));
+        assert!(
+            state.resolver().is_none(),
+            "child-env snapshots must not install provider resolver material"
+        );
+    }
+
     #[test]
     fn stale_generation_falls_back_to_current_credential_after_retention_window() {
         let state = ProviderCredentialState::from_environment(

@@ -29,6 +29,51 @@ pub const SANDBOX_COMMAND: &str = "OPENSHELL_SANDBOX_COMMAND";
 /// Deployment-controlled telemetry toggle propagated to the sandbox supervisor.
 pub const TELEMETRY_ENABLED: &str = "OPENSHELL_TELEMETRY_ENABLED";
 
+/// Supervisor pod/runtime topology. Kubernetes sidecar mode sets this to
+/// `"sidecar"`; the default combined supervisor path omits it.
+pub const SUPERVISOR_TOPOLOGY: &str = "OPENSHELL_SUPERVISOR_TOPOLOGY";
+
+/// Network enforcement backend selected by the compute driver.
+pub const NETWORK_ENFORCEMENT_MODE: &str = "OPENSHELL_NETWORK_ENFORCEMENT_MODE";
+
+/// Process enforcement mode selected by the compute driver.
+///
+/// The default when unset is `"full"`, where the process supervisor enforces
+/// filesystem/process policy before spawning workloads. Kubernetes sidecar
+/// topology sets this to `"network-only"` so the process wrapper can run as
+/// the sandbox UID without Linux capabilities while preserving SSH/session
+/// behavior.
+pub const PROCESS_ENFORCEMENT_MODE: &str = "OPENSHELL_PROCESS_ENFORCEMENT_MODE";
+
+/// Whether network policy evaluation must bind requests to the peer binary.
+///
+/// The default when unset is `"required"`. Kubernetes sidecar experiments may
+/// set this to `"relaxed"` to enforce endpoint and L7 policy without per-binary
+/// `/proc` identity binding.
+pub const NETWORK_BINARY_IDENTITY: &str = "OPENSHELL_NETWORK_BINARY_IDENTITY";
+
+/// File written by the network supervisor when sidecar networking is ready.
+pub const SUPERVISOR_READY_FILE: &str = "OPENSHELL_SUPERVISOR_READY_FILE";
+
+/// File written by the process supervisor with the workload entrypoint PID and
+/// read by the network sidecar for process/binary-bound network policy checks.
+pub const ENTRYPOINT_PID_FILE: &str = "OPENSHELL_ENTRYPOINT_PID_FILE";
+
+/// Local protobuf policy snapshot written by the network sidecar and read by
+/// the process-only supervisor in Kubernetes sidecar topology.
+pub const SIDECAR_POLICY_SNAPSHOT_FILE: &str = "OPENSHELL_SIDECAR_POLICY_SNAPSHOT_FILE";
+
+/// Local provider environment snapshot written by the network sidecar and read
+/// by the process-only supervisor in Kubernetes sidecar topology.
+pub const SIDECAR_PROVIDER_ENV_SNAPSHOT_FILE: &str = "OPENSHELL_SIDECAR_PROVIDER_ENV_SNAPSHOT_FILE";
+
+/// Optional TLS server name override used when connecting to the gateway.
+pub const GATEWAY_TLS_SERVER_NAME: &str = "OPENSHELL_GATEWAY_TLS_SERVER_NAME";
+
+/// Directory where the network supervisor writes the proxy CA files consumed
+/// by workload child processes.
+pub const PROXY_TLS_DIR: &str = "OPENSHELL_PROXY_TLS_DIR";
+
 /// Path to the CA certificate for mTLS communication with the gateway.
 pub const TLS_CA: &str = "OPENSHELL_TLS_CA";
 

@@ -53,9 +53,29 @@ pods do not need direct external ingress for SSH.
 
 ## Container Security Context
 
-The driver grants the sandbox agent container the Linux capabilities the
-supervisor needs for namespace setup and policy enforcement. It can also request
-a Kubernetes AppArmor profile through `app_armor_profile`.
+The default `combined` supervisor topology grants the sandbox agent container
+the Linux capabilities the supervisor needs for namespace setup and process,
+filesystem, and network policy enforcement.
+
+The `sidecar` supervisor topology moves pod-level network setup into a root init
+container and runs the long-lived network sidecar as a non-root UID with no
+added Linux capabilities. The agent container also runs as the resolved sandbox
+UID/GID with `allowPrivilegeEscalation: false` and `capabilities.drop: ["ALL"]`.
+In this mode OpenShell preserves gateway session and SSH behavior, but the
+process supervisor defaults to network-only mode and does not apply Landlock
+filesystem policy, process privilege dropping, or process/binary identity
+checks. Network endpoint and L7 policy remain enforced by the network sidecar.
+Set `process_enforcement = "full"` only when you want combined-mode
+process/filesystem guards and accept the added agent-container permissions.
+
+Sidecar mode uses the pod `fsGroup` to make the projected service-account token
+and sandbox client TLS secret group-readable so the non-root process supervisor
+can authenticate to the gateway. Treat the agent container as trusted with
+respect to those in-pod gateway credentials until a narrower credential handoff
+exists.
+
+The driver can request a Kubernetes AppArmor profile through
+`app_armor_profile`.
 
 Supported values are `Unconfined`, `RuntimeDefault`, and
 `Localhost/<profile-name>`. An empty or unset value omits