|
| 1 | +--- |
| 2 | +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. |
| 3 | +# SPDX-License-Identifier: Apache-2.0 |
| 4 | +title: "Kubernetes Sandbox Topology" |
| 5 | +sidebar-title: "Topology" |
| 6 | +description: "Choose between combined and sidecar supervisor topology for Kubernetes sandbox pods." |
| 7 | +keywords: "Generative AI, Cybersecurity, Kubernetes, Sandboxing, Sidecar, Network Policy, RuntimeClass" |
| 8 | +position: 2 |
| 9 | +--- |
| 10 | + |
| 11 | +Kubernetes sandbox pods can run the OpenShell supervisor in `combined` or |
| 12 | +`sidecar` topology. Choose the topology based on which controls you need inside |
| 13 | +the pod and how much privilege your cluster allows on the agent container. |
| 14 | + |
| 15 | +## Choose a Topology |
| 16 | + |
| 17 | +The default `combined` topology preserves the full OpenShell enforcement model. |
| 18 | +Use `sidecar` only when you accept network-focused enforcement in exchange for a |
| 19 | +lower-privilege agent container. |
| 20 | + |
| 21 | +| Topology | Use when | Main tradeoff | |
| 22 | +|---|---|---| |
| 23 | +| `combined` | You need OpenShell network, filesystem, and process controls in the sandbox workload. | The agent container carries the Linux capabilities the supervisor needs. | |
| 24 | +| `sidecar` | You need the agent container to run as non-root without added Linux capabilities, and network policy is the primary control. | Filesystem policy, process privilege dropping, and process/binary identity checks are not applied by the process supervisor. | |
| 25 | + |
| 26 | +## Combined Topology |
| 27 | + |
| 28 | +Combined topology is the original Kubernetes mode and remains the default. The |
| 29 | +agent container starts the OpenShell supervisor, and the supervisor launches the |
| 30 | +workload after applying sandbox setup. |
| 31 | + |
| 32 | +Combined topology keeps these controls in one supervisor path: |
| 33 | + |
| 34 | +- Network endpoint and L7 policy enforcement. |
| 35 | +- Filesystem policy enforcement. |
| 36 | +- Process and binary identity checks. |
| 37 | +- Privilege drop into the sandbox user. |
| 38 | +- Gateway relay, SSH sessions, exec, and file sync. |
| 39 | + |
| 40 | +Because the supervisor performs network namespace setup and process/filesystem |
| 41 | +controls from the agent container, Kubernetes grants that container elevated |
| 42 | +Linux capabilities. Use this mode when you need the complete OpenShell sandbox |
| 43 | +contract and your cluster policy permits those capabilities. |
| 44 | + |
| 45 | +## Sidecar Topology |
| 46 | + |
| 47 | +Sidecar topology splits the supervisor into a network sidecar and a |
| 48 | +low-privilege process supervisor in the agent container. |
| 49 | + |
| 50 | +The pod contains these OpenShell-managed pieces: |
| 51 | + |
| 52 | +| Component | Runs as | Purpose | |
| 53 | +|---|---|---| |
| 54 | +| Network init container | Root with setup capabilities | Installs pod-level nftables rules and prepares shared sidecar state. | |
| 55 | +| Network sidecar | `supervisor.sidecarProxyUid` | Runs the proxy, enforces network policy, writes proxy TLS material, and forwards gateway traffic on loopback. | |
| 56 | +| Agent container | Resolved sandbox UID/GID | Runs the process supervisor and launches the user workload. | |
| 57 | + |
| 58 | +In this topology, the agent container runs with `runAsNonRoot: true`, |
| 59 | +`allowPrivilegeEscalation: false`, and `capabilities.drop: ["ALL"]`. The |
| 60 | +long-running network sidecar also drops all Linux capabilities. The root init |
| 61 | +container keeps the setup capabilities needed to configure pod networking. |
| 62 | + |
| 63 | +Sidecar mode preserves gateway session behavior, including SSH connectivity, |
| 64 | +because the process supervisor still owns the session relay. The network sidecar |
| 65 | +handles outbound enforcement and forwards the process supervisor's gateway |
| 66 | +traffic to the real gateway endpoint. |
| 67 | + |
| 68 | +<Warning> |
| 69 | +Sidecar mode runs the process supervisor in network-only mode. OpenShell still |
| 70 | +enforces network endpoint and L7 policy through the sidecar, but the process |
| 71 | +supervisor does not apply Landlock filesystem policy, process privilege |
| 72 | +dropping, or process/binary identity checks. |
| 73 | +</Warning> |
| 74 | + |
| 75 | +## Credential Exposure |
| 76 | + |
| 77 | +Sidecar topology uses pod `fsGroup` and group-readable projected credentials so |
| 78 | +the non-root process supervisor can authenticate to the gateway. This includes |
| 79 | +the projected ServiceAccount token used for sandbox token bootstrap and the |
| 80 | +sandbox client TLS secret. |
| 81 | + |
| 82 | +Treat the agent container as trusted with respect to those in-pod gateway |
| 83 | +credentials. Use `combined` topology when that credential exposure is not |
| 84 | +acceptable for your deployment. |
| 85 | + |
| 86 | +## RuntimeClass Isolation |
| 87 | + |
| 88 | +Sidecar topology pairs well with runtime classes such as gVisor or Kata |
| 89 | +Containers when the cluster supports them. A sandboxed runtime strengthens the |
| 90 | +container boundary while OpenShell focuses on network policy enforcement from |
| 91 | +the sidecar. |
| 92 | + |
| 93 | +Runtime classes do not re-enable the OpenShell filesystem and process controls |
| 94 | +that sidecar mode relaxes. Use them as an additional workload boundary, not as a |
| 95 | +replacement for the combined topology's full supervisor controls. |
| 96 | + |
| 97 | +You can set a default runtime class in the Kubernetes driver configuration or |
| 98 | +override it per sandbox with driver config: |
| 99 | + |
| 100 | +```shell |
| 101 | +openshell sandbox create \ |
| 102 | + --driver-config-json '{"kubernetes":{"pod":{"runtime_class_name":"kata-containers"}}}' \ |
| 103 | + -- claude |
| 104 | +``` |
| 105 | + |
| 106 | +## Enable Sidecar Mode |
| 107 | + |
| 108 | +Set `supervisor.topology=sidecar` in the Helm chart values: |
| 109 | + |
| 110 | +```yaml |
| 111 | +supervisor: |
| 112 | + topology: sidecar |
| 113 | + sidecarProxyUid: 1337 |
| 114 | +``` |
| 115 | +
|
| 116 | +`sidecarProxyUid` must be a non-root UID and must not match the sandbox UID. |
| 117 | +The network init container exempts this UID from proxy redirection so the |
| 118 | +sidecar can reach the gateway. |
| 119 | + |
| 120 | +For direct gateway TOML configuration, set the equivalent Kubernetes driver |
| 121 | +fields: |
| 122 | + |
| 123 | +```toml |
| 124 | +[openshell.drivers.kubernetes] |
| 125 | +supervisor_topology = "sidecar" |
| 126 | +sidecar_proxy_uid = 1337 |
| 127 | +``` |
| 128 | + |
| 129 | +Leave `supervisor.topology` unset, or set it to `combined`, to keep the original |
| 130 | +single-container supervisor path. |
| 131 | + |
| 132 | +## Next Steps |
| 133 | + |
| 134 | +- To install OpenShell on Kubernetes, refer to [Setup](/kubernetes/setup). |
| 135 | +- To configure gateway authentication, refer to [Access Control](/kubernetes/access-control). |
| 136 | +- To review the driver fields, refer to [Gateway Configuration File](/reference/gateway-config). |
0 commit comments