docs(rfc): make middleware evaluation unary

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Author: pimlock

rfc/0009-supervisor-middleware/README.md

@@ -162,10 +162,8 @@ service SupervisorMiddleware {
   rpc Describe(google.protobuf.Empty) returns (MiddlewareManifest);
   rpc ValidateConfig(ValidateConfigRequest) returns (ValidateConfigResponse);
 
-  // method=HttpRequest, phase=pre_credentials. Declared as a bidirectional stream
-  // so large bodies can be chunked later; v1 exchanges exactly one
-  // HttpRequestEvaluation and one HttpRequestResult.
-  rpc EvaluateHttpRequest(stream HttpRequestEvaluation) returns (stream HttpRequestResult);
+  // method=HttpRequest, phase=pre_credentials.
+  rpc EvaluateHttpRequest(HttpRequestEvaluation) returns (HttpRequestResult);
 }
 
 message MiddlewareManifest {
@@ -183,7 +181,6 @@ message Operation {
 }
 
 // Context plus body as two top-level fields, so the body is cleanly separable.
-// v1 sets both in one message; a future stream sends body-only follow-ups.
 message HttpRequestEvaluation {
   Operation operation = 1;
   RequestContext context = 2;
@@ -233,7 +230,7 @@ The evaluation and result are shaped so middleware composes cleanly in a chain.
 
 Header mutation is deliberately narrow in v1. Middleware may only append safe, non-credential request headers approved by OpenShell. It cannot remove headers, rewrite existing headers, or set credential-bearing and request-routing headers such as `Authorization`, `Cookie`, `Host`, `X-Amz-*`, or OpenShell credential placeholders. This keeps request-body transformation useful while preserving the credential boundary and avoiding a second, middleware-owned routing surface.
 
-The interface is gRPC. The hot-path RPC is declared as a bidirectional stream, but v1 exchanges exactly one `HttpRequestEvaluation` and one `HttpRequestResult` over it: the supervisor buffers the bounded body and the middleware replies once. Declaring it as a stream now is deliberate, because gRPC method cardinality cannot change compatibly. It lets a later version chunk large payloads without altering the method signature. Possible extensions (chunked streaming, additional operation phases, semantic context) are collected in the [protocol-extensions appendix](appendices/protocol-extensions.md), including what streaming does and does not buy. The baseline middleware ships in the supervisor and is served in-process over the same gRPC contract, with no network hop. The exact field set is settled during implementation; the sketch above is the contract shape this RFC asks reviewers to evaluate.
+The interface is gRPC. The hot-path v1 RPC is unary: the supervisor buffers the bounded body, sends one `HttpRequestEvaluation`, and receives one `HttpRequestResult`. Streaming is deliberately not baked into `EvaluateHttpRequest`; if OpenShell later needs chunked or incremental processing, it should add a separate operation-specific method rather than changing the v1 method cardinality. Possible extensions (streaming operations, additional operation phases, semantic context) are collected in the [protocol-extensions appendix](appendices/protocol-extensions.md). The baseline middleware ships in the supervisor and is served in-process over the same gRPC contract, with no network hop. The exact field set is settled during implementation; the sketch above is the contract shape this RFC asks reviewers to evaluate.
 
 The `actor` process is the same identity OpenShell already resolves on the egress path - the binary, pid, and ancestor chain it uses for binary-scoped network policy and OCSF audit. It is resolved when the connection is established, so it is per-connection rather than strictly per-request: over a reused or pooled connection it identifies the process that opened the connection, which a middleware should not over-trust for per-request attribution. The field is optional because proxy-only or future shared-supervisor modes may not have reliable actor data; middleware must treat missing actor data as "unavailable" rather than as an authorization failure.
 
@@ -454,17 +451,17 @@ The cost of *not* doing this is leaving content-level egress control entirely ou
 ## Alternatives
 
 - **Build content checks into OpenShell directly.** A fixed, built-in set of DLP/redaction rules avoids a contract and an external service. Rejected as the primary model: OpenShell cannot embed every useful detection and transformation approach, and a stable contract lets dedicated tools and research scanners iterate without changing OpenShell. First-party built-in middleware still ships for narrow cases, over the same contract.
-- **REST instead of gRPC.** A REST/JSON hook is simpler to call, and with OpenAPI it could still offer a manifest handshake and a typed contract. Rejected because gRPC's typing is stronger and its streaming story is cleaner: the hot-path RPC is already declared as a stream so large bodies can be chunked later without a breaking signature change, which is awkward to match over REST. OpenShell already uses gRPC across its service contracts, so staying on a single toolchain avoids a second RPC stack to build, secure, and maintain.
+- **REST instead of gRPC.** A REST/JSON hook is simpler to call, and with OpenAPI it could still offer a manifest handshake and a typed contract. Rejected because gRPC's typing is stronger, OpenShell already uses gRPC across its service contracts, and gRPC leaves room for future streaming operations if large or incremental payload processing becomes necessary. Staying on a single toolchain avoids a second RPC stack to build, secure, and maintain.
 - **Other deployment modes (WASM, sidecar, in-sandbox).** In-process WASM filters or sidecars avoid a network hop and can tighten the trust boundary. Deferred rather than rejected: v1 commits to a single external-service shape to keep the contract small, and other shapes remain open. See [appendices/deployment-options.md](appendices/deployment-options.md).
 - **Doing nothing.** The cost of declining is covered at the end of Risks: content-level egress control stays outside OpenShell, and operators must build bespoke proxies that lose the policy integration, audit, and trust boundary the supervisor already provides.
 
 ## Prior art
 
 Calling an external service from a proxy to inspect, transform, or block in-flight traffic is well-established. The closest analogs:
 
-- **Envoy `ext_proc` (External Processing).** The primary model for this RFC. Envoy streams request headers and body to an external gRPC service that can mutate the body (for example redaction), allow, or deny, and the proxy and the processing service scale independently. Our `HttpRequest/pre_credentials` hook is effectively a buffered, single-hook v1 of the same boundary, with the proto shaped to grow toward `ext_proc`-style streaming later.
+- **Envoy `ext_proc` (External Processing).** The primary model for this RFC. Envoy streams request headers and body to an external gRPC service that can mutate the body (for example redaction), allow, or deny, and the proxy and the processing service scale independently. Our `HttpRequest/pre_credentials` hook is a buffered, single-hook v1 of the same boundary; if OpenShell later needs `ext_proc`-style streaming, it should add a separate streaming operation.
 - **Envoy `ext_authz` (External Authorization).** A narrower sibling: an external service returns an allow/deny decision per request. It validates the "delegate the per-request decision to an external service in the hot path" pattern, without the content-transformation half that this RFC needs.
-- **ICAP (RFC 3507).** HTTP proxies offload content adaptation, virus scanning, DLP, and content filtering to external ICAP servers that can modify or block request/response content. It is the closest *functional* precedent for content-aware egress control. Two details map directly onto our design: ICAP supports **pipelining** multiple servers (our middleware chain) and a **content preview** of the first bytes before full processing (our bounded-body buffering). What we avoid is its dated, text-based wire protocol; gRPC gives us typed contracts and a path to streaming.
+- **ICAP (RFC 3507).** HTTP proxies offload content adaptation, virus scanning, DLP, and content filtering to external ICAP servers that can modify or block request/response content. It is the closest *functional* precedent for content-aware egress control. Two details map directly onto our design: ICAP supports **pipelining** multiple servers (our middleware chain) and a **content preview** of the first bytes before full processing (our bounded-body buffering). What we avoid is its dated, text-based wire protocol; gRPC gives us typed contracts and room for future operation-specific streaming if we need it.
 - **HashiCorp `go-plugin` (Terraform, Vault).** Third-party plugins run as separate processes and communicate with the core exclusively over gRPC. It shows a strictly typed gRPC contract is a robust way to manage cross-language third-party extensions, which informs our registration plus manifest handshake (`Describe`, `ValidateConfig`).
 - **Kubernetes CSI / KMS.** Vendor-specific integrations are offloaded to external gRPC services rather than compiled into the core. Same "core defines the contract; integrators implement it out-of-process" split we use for middleware.
 - **Proxy-Wasm (Envoy/Istio Wasm filters).** In-process WebAssembly extensions with strong default-deny sandboxing and no IPC latency. Relevant to the future WASM deployment mode (see the deployment-options appendix); set aside for v1 because it is currently weak for GPU-backed or memory-heavy semantic guards.

rfc/0009-supervisor-middleware/appendices/protocol-extensions.md

@@ -6,37 +6,39 @@ The v1 contract is intentionally minimal: one HTTP request hook, buffered unary
 
 ## Streaming
 
-The hot-path `EvaluateHttpRequest` RPC is already declared as a bidirectional stream (see the contract in the RFC). v1 uses it in its degenerate form: the supervisor sends one `HttpRequestEvaluation` and the middleware returns one `HttpRequestResult`. This section records how the same method grows to carry chunked payloads, and importantly what streaming does and does not buy, since the distinction is easy to get wrong.
+The v1 `EvaluateHttpRequest` RPC is unary. The supervisor buffers the bounded request body, sends one `HttpRequestEvaluation`, and receives one `HttpRequestResult`. Streaming is deliberately left out of that method: if OpenShell later needs chunked payload transport or incremental processing, it should add a separate operation-specific method rather than changing `EvaluateHttpRequest` cardinality.
+
+This section records what such a future streaming operation would need to consider, and importantly what streaming does and does not buy, since the distinction is easy to get wrong.
 
 ### Transport streaming vs processing streaming
 
 These are different concepts and are easy to conflate:
 
-- **Transport streaming** - the gRPC call carries multiple messages (chunks). This is what a service advertises in its manifest and what the supervisor negotiates.
+- **Transport streaming** - a separate gRPC operation carries multiple messages (chunks). This is what a service would advertise in its manifest and what the supervisor would negotiate.
 - **Processing streaming** - the middleware can act on partial content before it has the whole body.
 
-The manifest field governs only the transport. It does not promise the middleware can process incrementally.
+The manifest field would govern only the transport. It would not promise the middleware can process incrementally.
 
 ### Full-body guards still buffer
 
 Many guards need the entire body to do anything: a JSON-aware redactor must parse the whole document, and a PII scan must see all of it. Such a guard, even over a streaming transport, accumulates every chunk internally, then parses, then emits a single response at end-of-stream - the decision still arrives after the last byte. Incremental processing only helps narrower cases such as byte-level regex redaction or secret scanning over a text stream.
 
-### Why support transport streaming at all
+### Why add a streaming operation later
 
-Even when the middleware must buffer the full body, chunked transport buys two things:
+Even when the middleware must buffer the full body, a separate chunked transport operation would buy two things:
 
 - It moves the large buffer off the supervisor. The supervisor does not hold a multi-MB body to put in a single message; the middleware, which needs it anyway and can be resourced for it, accumulates it.
 - It avoids gRPC's per-message size limit (4 MB by default). A 20 MB inference request cannot fit in one message without raising limits, but it can be chunked.
 
-This is the strongest reason to keep the door open for streaming, more so than incremental parsing.
+This is the strongest reason to keep the door open for a streaming operation, more so than incremental parsing.
 
 ### How it would work
 
-A service advertises chunked-transport support (and limits) in `Describe`. When supported, the supervisor may send the body as a sequence of messages; when not supported (or in v1), it buffers the bounded body and sends a single message, and a body over the cap uses the middleware config's `on_error` behavior.
+A service would advertise chunked-transport support (and limits) in `Describe`. When supported, the supervisor could use the streaming operation and send the body as a sequence of messages. When not supported, it would continue to use the unary v1 operation, and a body over the unary cap would use the middleware config's `on_error` behavior.
 
-Because the method is already a stream, chunking is field-additive rather than a signature change. Within a single streamed request, the first message carries the request context plus the first body bytes, and subsequent messages carry only further `body` bytes that the middleware appends; stream close marks end of request. This keeps the v1 messages flat and lets v1 stay a true single-message exchange.
+The streaming method should have its own messages instead of reusing `HttpRequestEvaluation` directly. Within a single streamed request, the first message would carry the request context plus the first body bytes, and subsequent messages would carry only further body chunks that the middleware appends; stream close would mark end of request. This keeps the v1 unary messages flat and gives streaming its own cleaner shape.
 
-A cleaner phased design -- a `oneof` over `context` and `body_chunk`, in the style of Envoy `ext_proc` - is the alternative, but it is a now-or-never choice rather than a later add-on. v1's flat message sets the context fields and `body` together, which a phase `oneof` forbids (only one member may be set), so a `oneof` cannot be retrofitted over the v1 message compatibly. We keep the flat shape because the append convention already covers the memory and message-size goals without forcing v1 into a multi-message exchange.
+A cleaner phased design -- a `oneof` over `context` and `body_chunk`, in the style of Envoy `ext_proc` - is available for a future streaming operation because it would not need to preserve the unary v1 message shape. V1 keeps the flat unary request because it is simpler for bounded bodies and avoids making every middleware implement streaming mechanics before the need is proven.
 
 ## Additional operation phases