Skip to content

Commit fd844d2

Browse files
committed
fix(docker): use supervisor image entrypoint path
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
1 parent 084c93b commit fd844d2

10 files changed

Lines changed: 35 additions & 23 deletions

File tree

.agents/skills/debug-openshell-cluster/SKILL.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ Use gateway metadata, deployment values, or the user's setup notes to identify t
6363
docker info
6464
docker ps --filter name=openshell
6565
docker logs <container> --tail=200
66+
docker run --rm --entrypoint /openshell-sandbox "${OPENSHELL_DOCKER_SUPERVISOR_IMAGE:-ghcr.io/nvidia/openshell/supervisor:latest}" --version
6667
openshell status
6768
```
6869

@@ -71,6 +72,7 @@ Common findings:
7172
- Docker daemon unavailable: start Docker Desktop or Docker Engine.
7273
- Gateway process stopped: inspect exit status and logs.
7374
- Sandbox image missing or pull denied: verify image reference and registry credentials.
75+
- Docker driver cannot initialize because it cannot find `openshell-sandbox`: verify `OPENSHELL_DOCKER_SUPERVISOR_BIN`, the sibling binary next to `openshell-gateway`, or the configured supervisor image contains `/openshell-sandbox`.
7476
- Sandbox never registers: check gateway logs and supervisor callback endpoint.
7577

7678
For source checkout development, restart the local gateway with:

.github/workflows/docker-build.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -238,7 +238,6 @@ jobs:
238238
--cache-to "type=gha,mode=max,scope=${{ inputs.component }}-${{ matrix.arch }}"
239239
240240
- name: Smoke check ${{ inputs.component }} image
241-
if: ${{ !inputs.push }}
242241
run: |
243242
set -euo pipefail
244243
image="${IMAGE_REGISTRY}/${{ inputs.component }}:${IMAGE_TAG}"

architecture/build.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ OpenShell builds these main artifacts:
1212
|---|---|
1313
| Gateway binary | `crates/openshell-server` |
1414
| CLI package and Python SDK | `python/openshell` plus Rust binaries where packaged |
15-
| Gateway container image | `deploy/docker/Dockerfile.images` |
15+
| Gateway and supervisor container images | `deploy/docker/Dockerfile.images` |
1616
| Helm chart | `deploy/helm/openshell` |
1717
| VM driver/runtime assets | `crates/openshell-driver-vm` |
1818
| Published docs site | `docs/` rendered by Fern config in `fern/` |

architecture/compute-runtimes.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ The supervisor must be available inside each sandbox workload:
3838

3939
| Runtime | Delivery model |
4040
|---|---|
41-
| Docker | Bind-mounted or extracted supervisor binary configured by the gateway. |
41+
| Docker | Bind-mounted local supervisor binary, or a binary extracted from the configured supervisor image. |
4242
| Podman | Read-only OCI image volume containing the supervisor binary. |
4343
| Kubernetes | Sandbox pod image or pod template configuration. |
4444
| VM | Embedded in the guest rootfs bundle. |

crates/openshell-driver-docker/README.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,17 @@ contract:
3434

3535
The agent child process does not retain these supervisor privileges.
3636

37+
## Supervisor Binary Resolution
38+
39+
The Docker driver bind-mounts a host-side Linux `openshell-sandbox` binary into
40+
each sandbox container. Resolution order is:
41+
42+
1. `--docker-supervisor-bin` / `OPENSHELL_DOCKER_SUPERVISOR_BIN`.
43+
2. A sibling `openshell-sandbox` next to the running `openshell-gateway` binary.
44+
3. A local Linux cargo target build for the Docker daemon architecture.
45+
4. `--docker-supervisor-image` / `OPENSHELL_DOCKER_SUPERVISOR_IMAGE`, or the
46+
release-matched default supervisor image, extracting `/openshell-sandbox`.
47+
3748
## Callback and TLS
3849

3950
`OPENSHELL_ENDPOINT` is injected from the gateway's configured gRPC endpoint

crates/openshell-driver-docker/src/lib.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1759,7 +1759,7 @@ async fn extract_supervisor_binary_bytes(docker: &Docker, image: &str) -> CoreRe
17591759
),
17601760
ContainerCreateBody {
17611761
image: Some(image.to_string()),
1762-
entrypoint: Some(vec!["/openshell-sandbox".to_string()]),
1762+
entrypoint: Some(vec![SUPERVISOR_IMAGE_BINARY_PATH.to_string()]),
17631763
cmd: Some(Vec::new()),
17641764
..Default::default()
17651765
},

crates/openshell-driver-kubernetes/src/driver.rs

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -688,19 +688,19 @@ fn supervisor_volume_mount() -> serde_json::Value {
688688

689689
/// Path of the supervisor binary inside the supervisor image.
690690
///
691-
/// The supervisor image places the binary at the filesystem root and ships
692-
/// nothing else. We invoke it directly — there is no shell, `cp`, or PATH
693-
/// resolution available inside the image.
691+
/// The supervisor image places the binary at the filesystem root. We invoke
692+
/// it directly so the init path does not depend on shell utilities or PATH
693+
/// resolution inside the image.
694694
const SUPERVISOR_IMAGE_BINARY_PATH: &str = "/openshell-sandbox";
695695

696696
/// Build the init container that copies the supervisor binary into the emptyDir.
697697
///
698-
/// The supervisor image contains only the supervisor binary at
699-
/// `/openshell-sandbox`. We invoke that binary with the `copy-self`
700-
/// subcommand so it copies itself into the shared emptyDir volume, where the
701-
/// agent container then executes it from a fixed, writable path. This pattern
702-
/// (binary self-copy) avoids requiring `sh`/`cp` in the supervisor image and
703-
/// mirrors the approach used by argoexec's emissary executor.
698+
/// The supervisor image contains the supervisor binary at `/openshell-sandbox`.
699+
/// We invoke that binary with the `copy-self` subcommand so it copies itself
700+
/// into the shared emptyDir volume, where the agent container then executes it
701+
/// from a fixed, writable path. This pattern (binary self-copy) avoids requiring
702+
/// `sh`/`cp` in the supervisor image and mirrors the approach used by argoexec's
703+
/// emissary executor.
704704
fn supervisor_init_container(
705705
supervisor_image: &str,
706706
supervisor_image_pull_policy: &str,
@@ -1559,8 +1559,8 @@ mod tests {
15591559
assert_eq!(init_containers[0]["image"], "supervisor-image:latest");
15601560
assert_eq!(init_containers[0]["imagePullPolicy"], "IfNotPresent");
15611561

1562-
// The supervisor image ships only the binary (no shell). The init
1563-
// container must invoke the binary directly with `copy-self <DEST>`.
1562+
// The init container must invoke the binary directly with
1563+
// `copy-self <DEST>` rather than depending on shell utilities.
15641564
let init_command = init_containers[0]["command"]
15651565
.as_array()
15661566
.expect("init container command should be set");
@@ -1573,7 +1573,7 @@ mod tests {
15731573
);
15741574
assert!(
15751575
!init_command.iter().any(|v| v == "sh"),
1576-
"init container must not depend on a shell (supervisor image ships only the binary)"
1576+
"init container must not depend on a shell"
15771577
);
15781578

15791579
// Agent container command should be overridden to the emptyDir path

crates/openshell-sandbox/src/main.rs

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,9 @@ use openshell_sandbox::run_sandbox;
1919

2020
/// Subcommand name used to self-copy the supervisor binary into a shared volume.
2121
///
22-
/// The supervisor image only ships the binary itself, so init containers
23-
/// cannot rely on `sh`/`cp` to copy the binary out. Invoking the binary itself
24-
/// with this argument performs the copy in pure Rust.
22+
/// Init containers invoke the binary directly instead of relying on `sh`/`cp`
23+
/// to copy the binary out. Invoking the binary itself with this argument
24+
/// performs the copy in pure Rust.
2525
const COPY_SELF_SUBCOMMAND: &str = "copy-self";
2626

2727
/// `OpenShell` Sandbox - process isolation and monitoring.
@@ -148,9 +148,8 @@ fn copy_self(dest: &str) -> Result<()> {
148148

149149
fn main() -> Result<()> {
150150
// Handle `copy-self <DEST>` before clap so it works without any of the
151-
// sandbox flags. The supervisor image only ships the binary itself, and
152-
// Kubernetes init containers invoke this path to seed an emptyDir volume
153-
// that the agent container then executes from.
151+
// sandbox flags. Kubernetes init containers invoke this path to seed an
152+
// emptyDir volume that the agent container then executes from.
154153
let raw_args: Vec<String> = std::env::args().collect();
155154
if raw_args.get(1).map(String::as_str) == Some(COPY_SELF_SUBCOMMAND) {
156155
let dest = raw_args.get(2).ok_or_else(|| {

deploy/docker/Dockerfile.images

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,3 +120,4 @@ CMD ["--bind-address", "0.0.0.0", "--port", "8080"]
120120
# the binary, breaking the Kubernetes init-container path.
121121
FROM nvcr.io/nvidia/base/ubuntu:noble-20251013 AS supervisor
122122
COPY --from=supervisor-binary /build/out/openshell-sandbox /openshell-sandbox
123+
ENTRYPOINT ["/openshell-sandbox"]

tasks/docker.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ run = "tasks/scripts/docker-build-image.sh gateway"
2727
hide = true
2828

2929
["build:docker:supervisor"]
30-
description = "Build the supervisor image (FROM scratch, binary only)"
30+
description = "Build the supervisor image"
3131
run = "tasks/scripts/docker-build-image.sh supervisor"
3232
hide = true
3333

0 commit comments

Comments
 (0)