feat(kubernetes): add sidecar supervisor topology#2076
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
🌿 Preview your docs: https://nvidia-preview-pr-2076.docs.buildwithfern.com/openshell |
|
Label |
4f981c2 to
8c9ae53
Compare
Some comments with help of agent1. Central trust boundary — the workload can use the sandbox's gateway identity . 2. nftables fence only covers TCP/UDP. 3. 4. Doc drift in the debug skill. |
f48ebb5 to
e00a00b
Compare
Add the Kubernetes sidecar supervisor topology, its Helm/Skaffold configuration, topology documentation, and sidecar e2e matrix coverage. Skip root-only sandbox identity rewriting when process enforcement is network-only so the low-permission sidecar process container can start successfully. Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
e00a00b to
ed858eb
Compare
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Addressed this by moving the supervisor and sandbox to run with different GIDs. Updated the other areas with clarifying comments. |
Summary
Adds the Kubernetes
sidecarsupervisor topology after the combined-topology base from #2074.The default
combinedtopology remains unchanged, andsidecaris opt-in.sidecarmoves pod-level network enforcement and gateway forwarding into a dedicated supervisor sidecar. The agent container can run as the resolved sandbox UID/GID withrunAsNonRoot, no privilege escalation, and all Linux capabilities dropped.This draft intentionally includes the prerequisite UID/GID commits from #1973 for now. We expect to rebase those out after that work lands separately.
Runtime validation status:
sidecarhas been validated on GKE Autopilot clusters created withgcloud container clusters create-auto --workload-policies=allow-net-admin; the Google Cloud CLI reference listsallow-net-adminas the supported Autopilot workload policy: https://docs.cloud.google.com/sdk/gcloud/reference/container/clusters/create-auto#--workload-policiessidecaris experimental with Kata Containers.sidecaris known to fail with gVisor because it depends on pod-local network rule setup.Sidecar mode preserves gateway session and SSH behavior, but intentionally runs the process supervisor in network-only mode. Filesystem policy, process privilege dropping, and process/binary identity checks are not applied in this mode.
Related Issue
References #1827, #981, #899, #1305.
Related PRs: #1973, #2074, #2016.
Changes
sidecarsupervisor topology and the relatedprocessEnforcementandproxyUidconfiguration.network-onlybehavior for sidecar mode while keeping SSH/session relay behavior intact.Testing
git diff --check origin/main..feat/kubernetes-sidecar-topology-v2cargo check -p openshell-driver-kubernetes -p openshell-sandbox -p openshell-supervisor-process -p openshell-supervisor-networkcargo test -p openshell-driver-kubernetes --libcargo test -p openshell-supervisor-process --libmise run helm:testmarkdownlint-cli2 docs/kubernetes/topology.mdx docs/kubernetes/setup.mdxruby -e 'require "yaml"; YAML.load_file(".github/workflows/branch-e2e.yml"); puts "ok"'mise run rust:lintHELM_K3S_LB_HOST_PORT=18080 mise run e2e:kubernetes:sidecar--workload-policies=allow-net-admin.Checklist