feat(supervisor-middleware): add network egress middleware (#2027)

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Author: pimlock

AGENTS.md

@@ -42,7 +42,15 @@ These pipelines connect skills into end-to-end workflows. Individual skill files
 | `crates/openshell-tui/` | Terminal UI | Ratatui-based dashboard for monitoring |
 | `crates/openshell-driver-kubernetes/` | Kubernetes compute driver | In-process `ComputeDriver` backend for K8s sandbox pods |
 | `crates/openshell-driver-docker/` | Docker compute driver | In-process `ComputeDriver` backend for local Docker sandbox containers |
+| `crates/openshell-driver-podman/` | Podman compute driver | In-process `ComputeDriver` backend for local Podman sandbox containers |
 | `crates/openshell-driver-vm/` | VM compute driver | Standalone libkrun-backed `ComputeDriver` subprocess (embeds its own rootfs + runtime) |
+| `crates/openshell-prover/` | Policy prover | Policy verification and proof generation |
+| `crates/openshell-server-macros/` | Server macros | Compile-time helpers for gateway RPC authorization |
+| `crates/openshell-supervisor-middleware/` | Middleware runtime | Generic middleware registry, remote service integration, and chain execution |
+| `crates/openshell-supervisor-middleware-builtins/` | Built-in middleware | First-party in-process middleware implementations |
+| `crates/openshell-supervisor-network/` | Network supervisor | Proxying, L7 enforcement, policy evaluation, and inference routing |
+| `crates/openshell-supervisor-process/` | Process supervisor | Process lifecycle, namespace, and bypass monitoring |
+| `crates/openshell-vfio/` | VFIO support | PCI and GPU passthrough preparation and lifecycle |
 | `python/openshell/` | Python SDK | Python bindings and CLI packaging |
 | `proto/` | Protobuf definitions | gRPC service contracts |
 | `deploy/` | Docker, Helm, K8s | Dockerfiles, Helm chart, manifests |

CONTRIBUTING.md

@@ -75,6 +75,7 @@ Skills live in `.agents/skills/`. Your agent's harness can discover and load the
 | Contributing    | `create-github-pr`        | Create pull requests with proper conventions                                                        |
 | Reviewing       | `review-github-pr`        | Summarize PR diffs and key design decisions                                                         |
 | Reviewing       | `review-security-issue`   | Assess security issues for severity and remediation                                                 |
+| Reviewing       | `fix-security-issue`      | Implement an approved security remediation plan                                                     |
 | Reviewing       | `watch-github-actions`    | Monitor CI pipeline status and logs                                                                 |
 | Reviewing       | `launch-openshell-gator`  | Launch and supervise OpenShell gator agents for issue and PR monitoring                             |
 | Reviewing       | `test-release-canary`     | Dispatch and iterate on the Release Canary workflow that smoke-tests published artifacts            |

Cargo.lock

@@ -75,6 +75,7 @@ serde_yml = "0.0.12"
 toml = "0.8"
 apollo-parser = "0.8.5"
 tower-mcp-types = "0.12.0"
+regex = "1"
 
 # HTTP client
 reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls-native-roots"] }

Cargo.toml

@@ -369,6 +369,17 @@ config path. A gateway-global policy can override sandbox-scoped policy. The
 sandbox supervisor polls for config revisions and hot-reloads dynamic policy
 when the policy engine accepts the update.
 
+External supervisor middleware registration is operator-owned configuration
+under `[[openshell.supervisor.middleware]]`. At startup the gateway connects to
+each service and validates its described bindings and operator body limit.
+Policies attach a complete external middleware by its operator-owned registration
+name. Manifest bindings are identified by operation and phase, and each manifest
+may declare at most one binding for an operation and phase pair.
+Before persisting a policy, the gateway asks each selected implementation to
+validate its config. The effective sandbox config contains only the registered
+services required by that policy; supervisors invoke those services directly on
+the request path.
+
 Provider credential expiry is enforced during gateway-to-sandbox credential
 resolution and again by the sandbox placeholder resolver. This keeps expired
 credentials from resolving even when a running sandbox still has retained

architecture/gateway.md

@@ -66,6 +66,31 @@ matchers; generic JSON-RPC rules match only the method.
 JSON-RPC responses and server-to-client MCP messages on response or SSE streams
 are relayed but are not currently parsed for policy enforcement.
 
+For admitted HTTP requests, the proxy can run an ordered supervisor middleware
+chain after L7 policy evaluation and before credential injection. Destination
+host selectors choose the chain independently of the network rule that admitted
+the request. Policy-local map keys identify configs, while built-in names or
+operator-owned registration names identify implementations.
+
+Built-ins run in-process; operator services use the same bounded gRPC contract.
+`openshell-policy` validates policy-owned structure, and the active middleware
+registry validates implementation-owned config. The generic registry and chain
+runner live in `openshell-supervisor-middleware`; first-party implementations
+live in `openshell-supervisor-middleware-builtins`.
+
+The supervisor installs policy and middleware registry changes as one runtime
+generation and preserves the last-known-good generation if preparation fails.
+Policy-only updates reuse the connected registry, so an external middleware
+outage cannot block unrelated policy changes.
+
+Middleware cannot observe injected credentials or mutate supervisor-owned
+credential, routing, or framing headers. Body transformations are re-evaluated
+against body-aware L7 policy before later stages or the upstream can observe
+them. Requests, results, chain length, execution time, and diagnostics are
+bounded; external free-form diagnostic text is not exposed in responses or
+security logs. See [Supervisor Middleware](../docs/extensibility/supervisor-middleware.mdx)
+for configuration and protocol details.
+
 `https://inference.local` is special. It bypasses OPA network policy and is
 handled by the inference interception path:
 
@@ -206,8 +231,10 @@ engine with a gateway policy revision.
 ## Failure Behavior
 
 - If gateway config polling fails, the sandbox keeps its last-known-good policy.
-- If a live policy update is invalid, the supervisor rejects it and keeps the
-  current policy.
+- If a live policy or middleware-registry update is invalid, the supervisor
+  rejects the combined update and keeps the current runtime pair.
+- If an operator-run middleware call fails, the selected config's `on_error`
+  behavior decides whether to deny the request or continue without that stage.
 - Existing raw byte streams are connection scoped. Dynamic policy changes apply
   to new connections or the next parsed HTTP request where the proxy can safely
   re-evaluate.

architecture/sandbox.md

@@ -194,6 +194,10 @@ Sandbox events that represent observable behavior use OCSF structured logs:
 
 Use plain tracing for internal plumbing such as retries, debug state, and
 intermediate steps where the final observable event is logged separately.
+Forward-proxy success is a final observable event: emit it only after
+middleware, token grants, credential rewriting, policy-generation checks, and
+the HTTP relay have succeeded so a later denial cannot coexist with an allowed
+record for the same request.
 
 Never log secrets, credentials, bearer tokens, or query parameters in OCSF
 messages. OCSF JSONL output may be shipped to external systems.

architecture/security-policy.md

@@ -1818,39 +1818,9 @@ fn parse_driver_config_json(value: &str) -> Result<prost_types::Struct> {
         ));
     };
 
-    Ok(prost_types::Struct {
-        fields: fields
-            .into_iter()
-            .map(|(key, value)| json_to_protobuf_value(value).map(|value| (key, value)))
-            .collect::<Result<_>>()?,
-    })
-}
-
-fn json_to_protobuf_value(value: serde_json::Value) -> Result<prost_types::Value> {
-    use prost_types::{ListValue, Struct, Value, value::Kind};
-
-    let kind = match value {
-        serde_json::Value::Null => Kind::NullValue(0),
-        serde_json::Value::Bool(value) => Kind::BoolValue(value),
-        serde_json::Value::Number(value) => Kind::NumberValue(value.as_f64().ok_or_else(|| {
-            miette!("--driver-config-json contains a number that cannot be represented")
-        })?),
-        serde_json::Value::String(value) => Kind::StringValue(value),
-        serde_json::Value::Array(values) => Kind::ListValue(ListValue {
-            values: values
-                .into_iter()
-                .map(json_to_protobuf_value)
-                .collect::<Result<_>>()?,
-        }),
-        serde_json::Value::Object(fields) => Kind::StructValue(Struct {
-            fields: fields
-                .into_iter()
-                .map(|(key, value)| json_to_protobuf_value(value).map(|value| (key, value)))
-                .collect::<Result<_>>()?,
-        }),
-    };
-
-    Ok(Value { kind: Some(kind) })
+    openshell_core::proto_struct::json_object_to_struct(fields)
+        .into_diagnostic()
+        .wrap_err("--driver-config-json contains a value that cannot be represented")
 }
 
 fn validate_cpu_quantity(value: &str) -> Result<String> {

crates/openshell-cli/src/run.rs

@@ -11,6 +11,7 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+glob = { workspace = true }
 prost = { workspace = true }
 prost-types = { workspace = true }
 tonic = { workspace = true, features = ["channel", "tls-native-roots"] }

crates/openshell-core/Cargo.toml

@@ -50,3 +50,17 @@ router agree on provider defaults. Profiles define:
 
 Do not duplicate provider-specific inference behavior in callers. Add shared
 behavior here, then consume it from the gateway, sandbox, and router.
+
+## Middleware Contracts
+
+Built-in supervisor middleware identifiers, host-selector matching, and pure
+configuration validation live in `openshell_core::middleware`. Policy admission
+and the supervisor runtime consume the same contract without introducing a
+dependency from the policy crate to the supervisor implementation.
+
+## Protobuf Struct Conversion
+
+Use `openshell_core::proto_struct` when crossing between `serde_json` values and
+`prost_types::{Struct, Value}`. Both conversion directions live in this module;
+JSON-to-protobuf conversion is fallible so callers cannot silently replace an
+unrepresentable number.