Skip to content

Commit 411fc1f

Browse files
committed
fix(podman): harden sandbox callbacks
Pass gateway bind, TLS, and client certificate auth context into the Podman driver so rootless pasta callbacks use only bindability-checked listener addresses. Require TLS client certificate authentication before enabling the automatic non-loopback callback listener. Gateway JWT and OIDC alone do not protect the full listener surface exposed on host interfaces. Update callback listener tests and documentation to reflect the tightened protection boundary. Signed-off-by: Adam Miller <admiller@redhat.com>
1 parent eba5dd7 commit 411fc1f

19 files changed

Lines changed: 679 additions & 38 deletions

File tree

Cargo.lock

Lines changed: 10 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ metrics = "0.24"
6464
metrics-exporter-prometheus = { version = "0.18", default-features = false, features = ["http-listener"] }
6565

6666
# Unix/Process
67-
nix = { version = "0.29", features = ["signal", "process", "user", "fs", "term"] }
67+
nix = { version = "0.29", features = ["signal", "process", "user", "fs", "term", "net"] }
6868
rustix = { version = "1.1", features = ["process"] }
6969

7070
# Serialization

architecture/compute-runtimes.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ when a sandbox create request asks for GPU resources.
3131
| Runtime | Best fit | Sandbox boundary | Notes |
3232
|---|---|---|---|
3333
| Docker | Local development with Docker available. | Container plus nested sandbox namespace. | Uses host networking so loopback gateway endpoints work from the supervisor. |
34-
| Podman | Rootless or single-machine deployments. | Container plus nested sandbox namespace. | Uses the Podman REST API, OCI image volumes, and CDI GPU devices when available. |
34+
| Podman | Rootless or single-machine deployments. | Container plus nested sandbox namespace. | Uses the Podman REST API, OCI image volumes, CDI GPU devices when available, and bindability-checked callback listeners for direct-host loopback gateway binds. Bridge-mode gateways can listen on the Podman bridge IP when bindable; rootless pasta can add a protected non-loopback local-interface listener when TLS client certificate authentication is enabled. Containerized gateways skip host listener discovery and require explicit port publishing. |
3535
| Kubernetes | Cluster deployment through Helm. | Pod plus nested sandbox namespace. | Uses Kubernetes API objects, service accounts, secrets, PVC-backed workspace storage, and GPU resources. |
3636
| VM | Experimental microVM isolation. | Per-sandbox libkrun VM. | Managed endpoint-backed driver. The gateway spawns `openshell-driver-vm`, waits for its Unix socket, and then consumes it through the same remote `compute_driver.proto` path used by unmanaged endpoint drivers. The VM driver boots a cached bootstrap `rootfs.ext4`, prepares requested OCI images inside a bootstrap VM with `umoci`, attaches the prepared image disk read-only, and gives each sandbox a writable `overlay.ext4` for merged-root changes and runtime material. The driver persists each accepted launch request beside the overlay and restarts those VMs on driver startup without recreating the overlay. |
3737
| Extension | Out-of-tree drivers operated alongside the gateway. | Whatever boundary the driver implements. | Selected by a non-reserved custom `compute_drivers = ["<name>"]` entry with `[openshell.drivers.<name>].socket_path`, or at launch time by pairing `--drivers <name>` with `--compute-driver-socket=<path>`. Reserved built-in names such as `vm`, `docker`, `podman`, and `kubernetes` cannot be used as unmanaged socket endpoints. The gateway connects to a UDS the operator already provisioned, runs `GetCapabilities`, logs the advertised `driver_name`, and dispatches all sandbox lifecycle calls through `compute_driver.proto`. The driver process and socket lifecycle are operator-owned; the gateway does not spawn, supervise, or remove unmanaged extension drivers. The trust boundary is the socket's filesystem permissions: the operator must ensure only the gateway uid can read/write it. |

crates/openshell-driver-podman/NETWORKING.md

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -255,6 +255,29 @@ if config.grpc_endpoint.is_empty() {
255255
The bridge gateway IP is not a stable substitute in rootless mode because it
256256
can live inside the user namespace rather than on the host.
257257

258+
### Rootless pasta callback listener
259+
260+
When the gateway binds only to loopback, host-local clients can reach
261+
`127.0.0.1:<port>`, but sandbox containers do not share the host loopback
262+
namespace. In rootless Podman with pasta, `host.containers.internal` reaches the
263+
host through pasta's forwarding path and may not connect to a listener bound
264+
only to `127.0.0.1`.
265+
266+
For direct host gateways, the Podman driver detects rootless pasta from Podman
267+
system info (`rootless=true` and `rootlessNetworkCmd=pasta`). If the gateway
268+
bind address is loopback-only, TLS client certificate authentication is active,
269+
and auto-discovery is enabled, the driver adds a second callback listener on a
270+
bindable non-loopback local interface address. Candidate addresses come from
271+
local interface enumeration via `getifaddrs`; the driver does not probe an
272+
external IP to find this address. The supervisor endpoint remains
273+
`host.containers.internal:<port>`.
274+
275+
Set `enable_auto_callback_listener = false` in `[openshell.drivers.podman]` or
276+
`OPENSHELL_PODMAN_ENABLE_AUTO_CALLBACK_LISTENER=false` to disable the automatic
277+
listener. Containerized gateways skip host listener discovery because their
278+
network namespace is not the host namespace; publish the gateway container port
279+
on an address sandbox containers can reach.
280+
258281
### Layer 3 Inner Sandbox Network Namespace
259282

260283
Inside the container, the supervisor creates another network namespace for the
@@ -391,7 +414,10 @@ Gateway
391414

392415
The gateway binds to `127.0.0.1:17670` by default in the RPM packaging. Client
393416
certificates are auto-generated by `openshell-gateway generate-certs` on first
394-
start and bind-mounted into sandbox containers by the Podman driver.
417+
start and bind-mounted into sandbox containers by the Podman driver. In
418+
rootless pasta mode, the protected callback listener described above allows the
419+
supervisor to keep using `host.containers.internal:<port>` while the primary
420+
listener remains loopback-only.
395421

396422
## Differences from the Kubernetes Driver
397423

crates/openshell-driver-podman/README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -216,6 +216,11 @@ Key points:
216216
host. Linux defaults to Podman's `host-gateway` resolver. macOS Podman
217217
machine defaults to gvproxy's host-loopback IP, `192.168.127.254`, because
218218
stale Podman machines may fail to resolve `host-gateway`.
219+
- Callback listener: direct host gateways that bind to loopback can add a
220+
protected non-loopback listener for rootless pasta callbacks when TLS client
221+
certificate authentication is enabled. Disable this with
222+
`enable_auto_callback_listener = false`. Gateway containers skip host
223+
listener discovery and require explicit port publishing.
219224
- nsenter: the supervisor uses `nsenter --net=` instead of `ip netns exec` for
220225
namespace operations, avoiding the sysfs remount path that fails in rootless
221226
containers.
@@ -338,6 +343,7 @@ Podman resources after out-of-band container removal or label drift.
338343
| `OPENSHELL_GATEWAY_PORT` | `--gateway-port` | `17670` | Gateway port used for endpoint auto-detection by the standalone binary. |
339344
| `OPENSHELL_NETWORK_NAME` | `--network-name` | `openshell` | Podman bridge network name. |
340345
| `OPENSHELL_PODMAN_HOST_GATEWAY_IP` | `--host-gateway-ip` | empty on Linux, `192.168.127.254` on macOS | Host gateway IP used for sandbox host aliases. Empty uses Podman's `host-gateway` resolver. |
346+
| `OPENSHELL_PODMAN_ENABLE_AUTO_CALLBACK_LISTENER` | none | `true` | Enable the protected non-loopback callback listener for rootless pasta when the gateway is loopback-bound and TLS client certificate authentication is enabled. |
341347
| `OPENSHELL_SANDBOX_SSH_SOCKET_PATH` | `--sandbox-ssh-socket-path` | `/run/openshell/ssh.sock` | Supervisor Unix socket path in `PodmanComputeConfig`. |
342348
| `OPENSHELL_STOP_TIMEOUT` | `--stop-timeout` | `10` | Container stop timeout in seconds. |
343349
| `OPENSHELL_SANDBOX_PIDS_LIMIT` | `--sandbox-pids-limit` | `2048` | Podman cgroup PID limit for sandbox containers. Set `0` to inherit Podman's runtime/default PID limit. |

crates/openshell-driver-podman/src/client.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,6 +245,8 @@ pub struct HostInfo {
245245
#[serde(default)]
246246
pub network_backend: String,
247247
#[serde(default)]
248+
pub rootless_network_cmd: String,
249+
#[serde(default)]
248250
pub security: SecurityInfo,
249251
}
250252

crates/openshell-driver-podman/src/config.rs

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
// SPDX-License-Identifier: Apache-2.0
33

44
use openshell_core::config::{DEFAULT_STOP_TIMEOUT_SECS, DEFAULT_SUPERVISOR_IMAGE};
5-
use std::net::IpAddr;
5+
use std::net::{IpAddr, SocketAddr};
66
use std::path::PathBuf;
77
use std::str::FromStr;
88

@@ -68,6 +68,7 @@ impl FromStr for ImagePullPolicy {
6868

6969
#[derive(Clone, serde::Serialize, serde::Deserialize)]
7070
#[serde(default, deny_unknown_fields)]
71+
#[allow(clippy::struct_excessive_bools)]
7172
pub struct PodmanComputeConfig {
7273
/// Path to the Podman API Unix socket.
7374
/// Default: `$XDG_RUNTIME_DIR/podman/podman.sock` (Linux),
@@ -89,6 +90,15 @@ pub struct PodmanComputeConfig {
8990
/// so the correct port is used even when `--port` differs from the
9091
/// default. Defaults to [`openshell_core::config::DEFAULT_SERVER_PORT`].
9192
pub gateway_port: u16,
93+
/// Full gateway bind address supplied by the gateway process at runtime.
94+
#[serde(skip)]
95+
pub gateway_bind_address: Option<SocketAddr>,
96+
/// Whether the gateway has TLS enabled. Supplied by the gateway process.
97+
#[serde(skip)]
98+
pub gateway_tls_enabled: bool,
99+
/// Whether gateway callbacks are protected by listener-level client certificate auth.
100+
#[serde(skip)]
101+
pub gateway_callback_client_cert_auth_enabled: bool,
92102
/// Unix socket path the in-container supervisor bridges relay traffic to.
93103
pub sandbox_ssh_socket_path: String,
94104
/// Name of the Podman bridge network.
@@ -134,10 +144,19 @@ pub struct PodmanComputeConfig {
134144
/// Set to `0` to disable health checks entirely.
135145
/// Defaults to [`DEFAULT_HEALTH_CHECK_INTERVAL_SECS`] (10 seconds).
136146
pub health_check_interval_secs: u64,
147+
/// Automatically add a non-loopback callback listener for rootless Podman
148+
/// pasta when the primary gateway bind is loopback-only and TLS client
149+
/// certificate authentication is active.
150+
#[serde(default = "default_true")]
151+
pub enable_auto_callback_listener: bool,
137152
}
138153

139154
pub const DEFAULT_HEALTH_CHECK_INTERVAL_SECS: u64 = 10;
140155

156+
fn default_true() -> bool {
157+
true
158+
}
159+
141160
impl PodmanComputeConfig {
142161
/// Returns `true` when all three TLS paths are configured.
143162
#[must_use]
@@ -251,6 +270,9 @@ impl Default for PodmanComputeConfig {
251270
image_pull_policy: ImagePullPolicy::default(),
252271
grpc_endpoint: String::new(),
253272
gateway_port: openshell_core::config::DEFAULT_SERVER_PORT,
273+
gateway_bind_address: None,
274+
gateway_tls_enabled: false,
275+
gateway_callback_client_cert_auth_enabled: false,
254276
sandbox_ssh_socket_path: "/run/openshell/ssh.sock".to_string(),
255277
network_name: DEFAULT_NETWORK_NAME.to_string(),
256278
host_gateway_ip: Self::default_host_gateway_ip(),
@@ -262,6 +284,7 @@ impl Default for PodmanComputeConfig {
262284
sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT,
263285
enable_bind_mounts: false,
264286
health_check_interval_secs: DEFAULT_HEALTH_CHECK_INTERVAL_SECS,
287+
enable_auto_callback_listener: true,
265288
}
266289
}
267290
}
@@ -274,6 +297,12 @@ impl std::fmt::Debug for PodmanComputeConfig {
274297
.field("image_pull_policy", &self.image_pull_policy.as_str())
275298
.field("grpc_endpoint", &self.grpc_endpoint)
276299
.field("gateway_port", &self.gateway_port)
300+
.field("gateway_bind_address", &self.gateway_bind_address)
301+
.field("gateway_tls_enabled", &self.gateway_tls_enabled)
302+
.field(
303+
"gateway_callback_client_cert_auth_enabled",
304+
&self.gateway_callback_client_cert_auth_enabled,
305+
)
277306
.field("sandbox_ssh_socket_path", &self.sandbox_ssh_socket_path)
278307
.field("network_name", &self.network_name)
279308
.field("host_gateway_ip", &self.host_gateway_ip)
@@ -288,6 +317,10 @@ impl std::fmt::Debug for PodmanComputeConfig {
288317
"health_check_interval_secs",
289318
&self.health_check_interval_secs,
290319
)
320+
.field(
321+
"enable_auto_callback_listener",
322+
&self.enable_auto_callback_listener,
323+
)
291324
.finish()
292325
}
293326
}

0 commit comments

Comments
 (0)