feat(policy): add incremental sandbox policy updates (#860)

* feat(policy): add incremental sandbox policy updates

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>

* docs(policies): expand incremental update guidance

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>

* feat(policy): audit incremental updates in gateway logs

* docs(policy): quote glob specs in shell examples

---------

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>

Author: johntmyers

.agents/skills/debug-openshell-cluster/SKILL.md

@@ -182,7 +182,7 @@ Component images (server, sandbox) can reach kubelet via two paths:
 
 **Local/external pull mode** (default local via `mise run cluster`): Local images are tagged to the configured local registry base (default `127.0.0.1:5000/openshell/*`), pushed to that registry, and pulled by k3s via `registries.yaml` mirror endpoint (typically `host.docker.internal:5000`). The `cluster` task pushes prebuilt local tags (`openshell/*:dev`, falling back to `localhost:5000/openshell/*:dev` or `127.0.0.1:5000/openshell/*:dev`).
 
-Gateway image builds now stage a partial Rust workspace from `deploy/docker/Dockerfile.images`. If cargo fails with a missing manifest under `/build/crates/...`, verify that every current gateway dependency crate (including `openshell-driver-kubernetes`) is copied into the staged workspace there.
+Gateway image builds now stage a partial Rust workspace from `deploy/docker/Dockerfile.images`. If cargo fails with a missing manifest under `/build/crates/...`, or an imported symbol exists locally but is missing in the image build, verify that every current gateway dependency crate (including `openshell-driver-kubernetes` and `openshell-ocsf`) is copied into the staged workspace there.
 
 ```bash
 # Verify image refs currently used by openshell deployment

.agents/skills/openshell-cli/SKILL.md

@@ -421,10 +421,14 @@ Watch for `deny` actions that indicate the user's work is being blocked by polic
 
 When denied actions are observed:
 
-1. Pull current policy: `openshell policy get work-session --full > policy.yaml`
-2. Modify the policy to allow the blocked actions (use `generate-sandbox-policy` skill for content)
-3. Push the update: `openshell policy set work-session --policy policy.yaml --wait`
-4. Verify: `openshell policy list work-session`
+1. Prefer incremental updates for additive network changes:
+   `openshell policy update work-session --add-endpoint api.github.com:443:read-only:rest:enforce --binary /usr/bin/gh --wait`
+   `openshell policy update work-session --add-allow 'api.github.com:443:POST:/repos/*/issues' --wait`
+2. Use full YAML replacement when the change is broad or touches non-network fields:
+   `openshell policy get work-session --full > policy.yaml`
+   Modify the policy to allow the blocked actions (use `generate-sandbox-policy` skill for content)
+   `openshell policy set work-session --policy policy.yaml --wait`
+3. Verify: `openshell policy list work-session`
 
 The user does not need to disconnect -- policy updates are hot-reloaded within ~30 seconds (or immediately when using `--wait`, which polls for confirmation).
 
@@ -543,6 +547,7 @@ $ openshell sandbox upload --help
 | Create with custom policy | `openshell sandbox create --policy ./p.yaml` |
 | Connect to sandbox | `openshell sandbox connect <name>` |
 | Stream live logs | `openshell logs <name> --tail` |
+| Incremental policy update | `openshell policy update <name> --add-endpoint host:443:read-only:rest:enforce --binary /usr/bin/curl --wait` |
 | Pull current policy | `openshell policy get <name> --full > p.yaml` |
 | Push updated policy | `openshell policy set <name> --policy p.yaml --wait` |
 | Policy revision history | `openshell policy list <name>` |

.agents/skills/openshell-cli/cli-reference.md

@@ -268,9 +268,32 @@ View sandbox logs. Supports one-shot and streaming.
 
 ## Policy Commands
 
+### `openshell policy update <name>`
+
+Incrementally merge live network policy changes into the current sandbox policy. Multiple flags in one invocation are applied as one atomic batch and create at most one new revision.
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--add-endpoint <SPEC>` | repeatable | `host:port[:access[:protocol[:enforcement]]]`. Adds or merges an endpoint. `access`: `read-only`, `read-write`, `full`. `protocol`: `rest`, `sql`. `enforcement`: `enforce`, `audit`. |
+| `--remove-endpoint <SPEC>` | repeatable | `host:port`. Removes the endpoint or just the requested port from a multi-port endpoint. |
+| `--add-allow <SPEC>` | repeatable | `host:port:METHOD:path_glob`. Adds REST allow rules to an existing `protocol: rest` endpoint. |
+| `--add-deny <SPEC>` | repeatable | `host:port:METHOD:path_glob`. Adds REST deny rules to an existing `protocol: rest` endpoint that already has an allow base. |
+| `--remove-rule <NAME>` | repeatable | Deletes a named network rule. |
+| `--binary <PATH>` | repeatable | Adds binaries to each `--add-endpoint` rule. Valid only with `--add-endpoint`. |
+| `--rule-name <NAME>` | none | Overrides the generated rule name. Valid only when exactly one `--add-endpoint` is provided. |
+| `--dry-run` | false | Preview the merged policy locally without sending an update to the gateway. |
+| `--wait` | false | Wait for the sandbox to confirm the new policy revision is loaded. |
+| `--timeout <SECS>` | 60 | Timeout for `--wait`. |
+
+Notes:
+
+- `--add-allow` and `--add-deny` currently operate only on `protocol: rest` endpoints.
+- `--wait` cannot be combined with `--dry-run`.
+- Use `policy set` when replacing the full policy or changing static sections.
+
 ### `openshell policy set <name> --policy <PATH>`
 
-Update the policy on a live sandbox. Only the dynamic `network_policies` field can be changed at runtime.
+Replace the full policy on a live sandbox. Only the dynamic `network_policies` field can be changed at runtime.
 
 | Flag | Default | Description |
 |------|---------|-------------|

Cargo.lock

@@ -63,7 +63,7 @@ The incremental deploy (`cluster-deploy-fast.sh`) fingerprints local Git changes
 | Changed files | Rebuild triggered |
 |---|---|
 | Cargo manifests, proto definitions, cross-build script | Gateway + supervisor |
-| `crates/openshell-server/*`, `deploy/docker/Dockerfile.images` | Gateway |
+| `crates/openshell-server/*`, `crates/openshell-ocsf/*`, `deploy/docker/Dockerfile.images` | Gateway |
 | `crates/openshell-sandbox/*`, `crates/openshell-policy/*` | Supervisor |
 | `deploy/helm/openshell/*` | Helm upgrade |
 

architecture/build-containers.md

@@ -162,6 +162,24 @@ This guarantees that the same logical policy always produces the same hash regar
 
 **Idempotent updates**: `UpdateSandboxPolicy` compares the deterministic hash of the submitted policy against the latest stored revision's hash. If they match, the handler returns the existing version and hash without creating a new revision. The CLI detects this (the returned version equals the pre-call version) and prints `Policy unchanged` instead of `Policy version N submitted`. This makes repeated `policy set` calls safe and idempotent.
 
+### Incremental Merge Updates
+
+`UpdateConfigRequest.merge_operations` supports batched incremental changes to the dynamic `network_policies` section. The CLI exposes this as `openshell policy update`.
+
+Supported first-pass operations:
+
+- `--add-endpoint host:port[:access[:protocol[:enforcement]]]`
+- `--remove-endpoint host:port`
+- `--remove-rule <name>`
+- `--add-allow host:port:METHOD:path_glob`
+- `--add-deny host:port:METHOD:path_glob`
+
+`--add-allow` and `--add-deny` target existing `protocol: rest` endpoints only. `--binary` may be repeated with `--add-endpoint`, and `--rule-name` is allowed only when exactly one `--add-endpoint` is present.
+
+Each `openshell policy update` invocation is atomic at the revision level: the CLI sends one `merge_operations` batch, the server merges the whole batch into the latest policy, validates the result, and persists at most one new revision. Concurrency is handled with optimistic retries on the `(sandbox_id, version)` uniqueness boundary. If another writer wins first, the server refetches the latest policy, reapplies the full batch, revalidates it, and retries. This preserves batch atomicity without serializing all sandbox policy writes behind a sandbox-global mutex.
+
+The gateway emits per-sandbox OCSF `CONFIG:*` audit lines when incremental merge operations are applied and when draft chunks are approved or removed. These audit lines are streamed through the existing gateway log path, so operators can inspect the exact logical mutation that produced a policy revision without waiting for the sandbox poll loop to reload that revision.
+
 ### Policy Revision Statuses
 
 | Status | Meaning |
@@ -206,9 +224,20 @@ Failure scenarios that trigger LKG behavior include:
 
 ### CLI Commands
 
-The `openshell policy` subcommand group manages live policy updates:
+The `openshell policy` subcommand group manages live policy updates through full replacement (`policy set`) and incremental merges (`policy update`):
 
 ```bash
+# Merge endpoint/rule changes into the current sandbox policy
+openshell policy update <sandbox-name> \
+  --add-endpoint api.github.com:443:read-only:rest:enforce \
+  --binary /usr/bin/gh \
+  --wait
+
+# Add a REST allow rule to an existing endpoint
+openshell policy update <sandbox-name> \
+  --add-allow api.github.com:443:POST:/repos/*/issues \
+  --wait
+
 # Push a new policy to a running sandbox
 openshell policy set <sandbox-name> --policy updated-policy.yaml
 
@@ -255,6 +284,7 @@ Both `set` and `delete` require interactive confirmation (or `--yes` to bypass).
 
 When a global policy is active, sandbox-scoped policy mutations are blocked:
 - `policy set <sandbox>` returns `FailedPrecondition: "policy is managed globally"`
+- `policy update <sandbox>` returns `FailedPrecondition: "policy is managed globally"`
 - `rule approve`, `rule approve-all` return `FailedPrecondition: "cannot approve rules while a global policy is active"`
 - Revoking a previously approved draft chunk is blocked (it would modify the sandbox policy)
 - Rejecting pending chunks is allowed (does not modify the sandbox policy)
@@ -270,7 +300,7 @@ See [Gateway Settings Channel](gateway-settings.md#global-policy-lifecycle) for
 
 When `--full` is specified, the server includes the deserialized `SandboxPolicy` protobuf in the `SandboxPolicyRevision.policy` field (see `crates/openshell-server/src/grpc.rs` -- `policy_record_to_revision()` with `include_policy: true`). The CLI converts this proto back to YAML via `policy_to_yaml()`, which uses a `BTreeMap` for `network_policies` to produce deterministic key ordering. See `crates/openshell-cli/src/run.rs` -- `policy_to_yaml()`, `policy_get()`.
 
-See `crates/openshell-cli/src/main.rs` -- `PolicyCommands` enum, `crates/openshell-cli/src/run.rs` -- `policy_set()`, `policy_get()`, `policy_list()`.
+See `crates/openshell-cli/src/main.rs` -- `PolicyCommands` enum, `crates/openshell-cli/src/run.rs` -- `policy_update()`, `policy_set()`, `policy_get()`, `policy_list()`.
 
 ---
 

architecture/security-policy.md

@@ -12,6 +12,7 @@ pub mod auth;
 pub mod bootstrap;
 pub mod completers;
 pub mod edge_tunnel;
+pub(crate) mod policy_update;
 pub mod run;
 pub mod ssh;
 pub mod tls;

crates/openshell-cli/src/lib.rs

@@ -254,6 +254,8 @@ const POLICY_EXAMPLES: &str = "\x1b[1mALIAS\x1b[0m
 \x1b[1mEXAMPLES\x1b[0m
   $ openshell policy get my-sandbox
   $ openshell policy set my-sandbox --policy policy.yaml
+  $ openshell policy update my-sandbox --add-endpoint api.github.com:443:read-only:rest:enforce
+  $ openshell policy update my-sandbox --add-allow 'api.github.com:443:GET:/repos/**'
   $ openshell policy set --global --policy policy.yaml
   $ openshell policy delete --global
   $ openshell policy list my-sandbox
@@ -1438,6 +1440,54 @@ enum PolicyCommands {
         timeout: u64,
     },
 
+    /// Incrementally update policy on a live sandbox.
+    #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")]
+    Update {
+        /// Sandbox name (defaults to last-used sandbox).
+        #[arg(add = ArgValueCompleter::new(completers::complete_sandbox_names))]
+        name: Option<String>,
+
+        /// Add or merge an endpoint: host:port[:access[:protocol[:enforcement]]].
+        #[arg(long = "add-endpoint")]
+        add_endpoints: Vec<String>,
+
+        /// Remove an endpoint: host:port.
+        #[arg(long = "remove-endpoint")]
+        remove_endpoints: Vec<String>,
+
+        /// Add a REST allow rule: host:port:METHOD:path_glob.
+        #[arg(long = "add-allow")]
+        add_allow: Vec<String>,
+
+        /// Add a REST deny rule: host:port:METHOD:path_glob.
+        #[arg(long = "add-deny")]
+        add_deny: Vec<String>,
+
+        /// Remove a network rule by name.
+        #[arg(long = "remove-rule")]
+        remove_rules: Vec<String>,
+
+        /// Add binaries to each --add-endpoint rule.
+        #[arg(long = "binary", value_hint = ValueHint::FilePath)]
+        binaries: Vec<String>,
+
+        /// Override the generated rule name when exactly one --add-endpoint is provided.
+        #[arg(long = "rule-name")]
+        rule_name: Option<String>,
+
+        /// Preview the merged policy without sending it to the gateway.
+        #[arg(long)]
+        dry_run: bool,
+
+        /// Wait for the sandbox to load the policy revision.
+        #[arg(long)]
+        wait: bool,
+
+        /// Timeout for --wait in seconds.
+        #[arg(long, default_value_t = 60)]
+        timeout: u64,
+    },
+
     /// Show current active policy for a sandbox or the global policy.
     #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")]
     Get {
@@ -1988,6 +2038,37 @@ async fn main() -> Result<()> {
                             .await?;
                     }
                 }
+                PolicyCommands::Update {
+                    name,
+                    add_endpoints,
+                    remove_endpoints,
+                    add_allow,
+                    add_deny,
+                    remove_rules,
+                    binaries,
+                    rule_name,
+                    dry_run,
+                    wait,
+                    timeout,
+                } => {
+                    let name = resolve_sandbox_name(name, &ctx.name)?;
+                    run::sandbox_policy_update(
+                        &ctx.endpoint,
+                        &name,
+                        &add_endpoints,
+                        &remove_endpoints,
+                        &add_deny,
+                        &add_allow,
+                        &remove_rules,
+                        &binaries,
+                        rule_name.as_deref(),
+                        dry_run,
+                        wait,
+                        timeout,
+                        &tls,
+                    )
+                    .await?;
+                }
                 PolicyCommands::Get {
                     name,
                     rev,