diff --git a/aiac/docs/specs/ARCHITECTURE-SUMMARY.md b/aiac/docs/specs/ARCHITECTURE-SUMMARY.md new file mode 100644 index 00000000..95ade704 --- /dev/null +++ b/aiac/docs/specs/ARCHITECTURE-SUMMARY.md @@ -0,0 +1,270 @@ +# AIAC Architectural Summary + +## Abstract + +AI-based Access Control (AIAC) is a Kagenti platform extension that automates RBAC/ABAC policy +enforcement for AI agents running on Kubernetes. A LangGraph-based AI agent continuously translates +a natural-language access control policy — stored in a vector knowledge base — into concrete +permission configurations in the active Policy Decision Point (PDP), eliminating manual policy +administration and preventing policy drift as services and roles evolve. The PDP backend is OPA, +which evaluates LLM-generated Rego rules; Keycloak remains the identity provider for entity +management (subjects, roles, services). + +--- + +## Problem Description + +Kagenti AI agents call services across a shared platform. Every call must carry a token scoped to +exactly the permissions the caller's role entitles on the target service. Without a dedicated +policy management layer, access policy ends up scattered across per-deployment configuration, +creating three compounding problems: + +1. **Policy drift** — new services and roles are onboarded without corresponding permission + updates because there is no automated mechanism to apply them. +2. **Distributed policy intent** — no single authoritative source declares what roles may do; + policy knowledge is fragmented across deployments. +3. **Manual administration overhead** — keeping OPA policy rules consistent with a growing fleet + of agents and tools requires ongoing human attention with no audit trail. + +--- + +## Problem Solution + +AIAC introduces a strict three-layer model that cleanly separates policy concerns: + +| Layer | Component | Responsibility | +|---|---|---| +| **Policy Management** | AIAC Agent | Translates natural-language policy into PDP configuration on every trigger | +| **Policy Decision (PDP)** | OPA | Evaluates LLM-generated Rego rules; returns the caller's entitlements/decision | +| **Policy Enforcement (PEP)** | AuthBridge | Intercepts traffic; exchanges tokens; carries no policy knowledge | + +The AIAC Agent subscribes to an event stream (NATS JetStream) and reacts to entity lifecycle +events — new services, role changes, policy updates — by retrieving the current policy from a RAG +knowledge base, querying live PDP state, and applying the minimal required diff via a dedicated +PDP Policy Writer. AuthBridge performs RFC 8693 token exchanges sending only the target +`audience` — no `scope` parameter. OPA evaluates the caller's role against the Rego rules and +issues a token containing exactly the entitlements that role grants on the target service. +**Policy intent lives entirely in OPA, kept current by AIAC.** + +--- + +## Major Use-Cases + +### UC-1 · Continuous Access Reconciliation (On-boarding / Off-boarding) + +**Trigger:** A Role or Keycloak Client is created, updated, or removed. + +The Keycloak SPI listener publishes a scoped event to the Event Broker. The AIAC Agent retrieves +relevant context from the RAG store, reads the current OPA policy state, and asks the LLM to +compute the minimal permission diff scoped to the affected entity. The diff is validated by a +second LLM pass and applied to OPA as updated Rego rules. Supports both **auto-apply** (fully +automated, least-privilege) and **recommendation + human review** modes. + +### UC-2 · Policy Update Reconciliation + +**Trigger:** An operator ingests updated documents into the RAG store. + +After ingestion the RAG Ingest Service publishes a build event. The AIAC Agent retrieves all +relevant context, computes a full policy diff against current OPA state, and applies the delta. +A `rebuild` variant (operator-only, direct HTTP) first clears all OPA policy rules before +recomputing from scratch — used when policy changes are too broad for incremental diff. + +### UC-3 · Entitlements Review + +**Trigger:** Operator request (on-demand or scheduled). + +The agent evaluates all current OPA policy rules — including manually added ones that AIAC did +not create — against the natural-language policy. It reports compliant, non-compliant, and +policy-agnostic entitlements, enabling audit and remediation workflows. + +### UC-4 · Access Request + +**Trigger:** User request via chatbot. + +A user requests an entitlement grant. The agent verifies the request against the policy +(permissive approach) and either auto-grants or routes to a human approver (man-in-the-loop). +Manually granted entitlements are flagged as policy-agnostic and surfaced during UC-3 reviews. + +--- + +## AIAC Component Architecture + +Eight components across five Kubernetes Pods plus a Python library layer, all implemented in Python 3.12. External dependencies: Keycloak Admin API, an LLM API, and an embedding API. The Keycloak SPI listener is defined in a separate PRD. + +| # | Component | Description | +|---|-----------|-------------| +| 1 | **IdP Configuration Service** | REST service that exposes IdP entity data (subjects, roles, services, scopes) for read and write operations. Read methods enrich services with assigned roles/scopes and enrich roles with child roles. Backed by Keycloak. Python library: `aiac.idp.configuration`. | +| 2 | **PDP Policy Writer** | REST service that applies LLM-generated Rego rules to the OPA backend. Writes derived Rego packages to an `AuthorizationPolicy` Kubernetes CR. Exposed as ClusterIP service `aiac-pdp-policy-service:7072`. Python library: `aiac.pdp.policy.library`. | +| 3 | **Policy Store** | REST service that owns an in-memory `PolicyModel` cache backed by SQLite as the authoritative structured policy store. Enables the Policy Computation Engine to read current `AgentPolicyModel` state for additive merging. Deployed as a dedicated single-replica StatefulSet (`aiac-policy-store`) at `:7074`. Python library: `aiac.policy.store.library`. | +| 4 | **Policy Computation Engine** | Pure Python library module (`aiac.policy.computation`). No service, no Kubernetes deployment. Receives `list[PolicyRule]` from AIAC Agent sub-agents, queries IdP to resolve owning services, additively merges rules into `AgentPolicyModel` objects in the Policy Store, and pushes the updated `PolicyModel` to the PDP Policy Writer. Single entry point: `compute_and_apply(rules)`. | +| 5 | **Policy and Domain Knowledge RAG** | ChromaDB vector store holding the access control policy and domain knowledge in persistent, queryable form, populated via a co-located RAG Ingest Service. | +| 6 | **Event Broker** | NATS JetStream pod that decouples event producers (Keycloak SPI listener, RAG Ingest Service) from the AIAC Agent. Provides durable, at-least-once delivery with automatic replay on Agent pod restart. Competing consumer model ensures each event is processed exactly once. | +| 7 | **AIAC Agent** | LangGraph-based AI agent triggered by Event Broker subscriptions (`aiac.apply.>` subjects) and directly by the operator (`rebuild` only). Retrieves the current policy from the RAG store, interprets it against live PDP state, and applies the required policy changes immediately. | +| 8 | **Python library** | Python API library provides typed access to IdP and policy services via `aiac.idp.configuration`, `aiac.policy.model`, `aiac.policy.store.library`, `aiac.pdp.policy.library`, and `aiac.policy.computation` modules backed by generic Pydantic models. | + +``` + (𝗞𝗲𝘆𝗰𝗹𝗼𝗮𝗸 𝗔𝗣𝗜) (𝗞𝘂𝗯𝗲𝗿𝗻𝗲𝘁𝗲𝘀 𝗖𝗥 𝗔𝗣𝗜) + ▲ ▲ + │ | + (𝘶𝘴𝘦𝘳𝘴, 𝘳𝘰𝘭𝘦𝘴, 𝘤𝘭𝘪𝘦𝘯𝘵𝘴) (𝘈𝘶𝘵𝘩𝘰𝘳𝘪𝘻𝘢𝘵𝘪𝘰𝘯𝘗𝘰𝘭𝘪𝘤𝘺 𝘊𝘙) +┌──────────────┼──────────────────────┼───────────────────┐ +│ Kagenti Interface Pod │ │ +│ │ │ │ +│ ┌───────┴──────┐ ┌────────┴───────┐ │ +│ │ IdP Config │ │ PDP Policy │ │ +│ │ Service │ │ Writer (OPA) │ │ +│ └──────────────┘ └────────────────┘ │ +│ ▲ ▲ │ +└──────────────┼──────────────────────┼───────────────────┘ + │ │ + │ │ + │ │ + │ ┌──────────────────────────────────────┐ + │ │ Policy Store Pod │ + │ │ │ + │ │ ┌───────────────────────────────┐ │ + │ │ │ Policy Store Service │ │ + │ │ │ │ │ + │ │ │ (SQLite policy.db) │ │ + │ │ └───────────────────────────────┘ │ + │ │ ▲ │ + │ └──────────────────┼───────────────────┘ + │ │ +┌──────────────┼──────────────────────┼───────────────────┐ ┌────────────────────────────────┐ +│ Agent Pod └───────────────────┐ │ │ │ Event Broker Pod │ +│ │ │ │ │ │ +│ ┌──────────────────────┐ ┌────────────────┐ │ │ ┌──────────────────────────┐ │ +│ │ Policy Compute Engn │◄──│ AIAC Agent │◄─────────┼──┼──│ NATS JetStream │ │ +│ └──────────────────────┘ └────────────────┘ (𝘯𝘰𝘵𝘪𝘧𝘺) │ │ └──────────────────────────┘ │ +│ │ │ │ ▲ ▲ │ +│ │ │ │ │ │ │ +└─────────────────────────────────────┼───────────────────┘ └─────────┼──────────────┼───────┘ + │ (𝘱𝘶𝘣𝘭𝘪𝘴𝘩) (𝘱𝘶𝘣𝘭𝘪𝘴𝘩) +┌─────────────────────────────────────┼───────────────────┐ │ │ +│ Policy / Domain Knowledge RAG Pod │ │ (𝗞𝗲𝘆𝗰𝗹𝗼𝗮𝗸 𝗦𝗣𝗜) (𝗥𝗔𝗚 𝗜𝗻𝗴𝗲𝘀𝘁) +│ ▼ │ +│ ┌─────────────────────┐ ┌─────────────────────────┐ │ +│ │ RAG Ingest Service │──►│ ChromaDB (vector store) │ │ +│ └─────────────────────┘ └─────────────────────────┘ │ +└─────────────────────────────────────────────────────────┘ +``` + +All inter-pod traffic is Kubernetes ClusterIP. External access is exclusively via +`kubectl port-forward` (operator/developer) or NATS publish (Keycloak SPI, RAG Ingest). + +--- + +## Kagenti / Keycloak / OPA Interfaces + +**AIAC ↔ Kagenti platform** +The AIAC Agent reads `AgentRuntime` and `AgentCard` custom resources from the Kubernetes API to +extract service metadata during UC-1 service onboarding. The `aiac.idp.library` and +`aiac.pdp.library` Python packages are the integration surface for other Kagenti components +needing typed access to IdP configuration and PDP policy state. + +**AIAC ↔ Keycloak** +The IdP Configuration Service proxies Keycloak Admin REST endpoints under generic IdP entity +names (subjects, roles, services, scopes). The Keycloak SPI listener publishes entity lifecycle +events to NATS; it is a separate component outside the AIAC codebase. + +**AIAC ↔ OPA** +The PDP Policy Writer writes LLM-generated Rego rules to an `AuthorizationPolicy` Kubernetes CR. +Each agent pod's OPA plugin fetches its Rego packages from the CR at startup. + +**AIAC ↔ Policy Management Service** +The Policy Management Service writes structured `AgentPolicyModel` data to a SQLite store +(in-memory cache + write-through to `/data/state.db` on a dedicated PVC) — the source of truth +for policy state that the AIAC Agent diffs against before writing updated Rego rules to OPA. + +**AIAC ↔ Event Broker (NATS JetStream)** +The Agent subscribes to the event stream as a durable consumer with at-least-once delivery. +Unacknowledged messages survive pod restarts; failed messages are routed to a dead-letter subject. + +--- + +## Call Flows + +#### UC-1a · Service On-boarding (`aiac.apply.service.{id}`) + +``` + Keycloak SPI + │ CLIENT_CREATED + │ 1. publish aiac.apply.service.{id} + ▼ + NATS JetStream + │ (durable consumer, at-least-once delivery) + │ 2. deliver event + ▼ + AIAC Agent + │ 3. GET /services, /roles, /assignments ──► IdP Configuration Service ──► Keycloak Admin REST + │ 4. GET /services/{id}/roles, /services/{id}/scopes ──► IdP Configuration Service ──► Keycloak Admin REST + │ 5. semantic query (policy + domain knowledge) ──► ChromaDB + │ 6. [LLM] compute AgentPolicyModel for new service (inbound + outbound rules) + │ 7. [LLM] validate policy model against retrieved policy (second pass) + │ 8. POST /policy/agents/{service_id} (write agent policy) ──► PDP Policy Writer ──► AuthorizationPolicy CR + │ 9. ACK message + ▼ + NATS JetStream (message removed from pending) +``` + +#### UC-1b · Role On-boarding (`aiac.apply.role.{id}`) + +``` + Keycloak SPI + │ REALM_ROLE_CREATED / REALM_ROLE_UPDATED + │ 1. publish aiac.apply.role.{id} + ▼ + NATS JetStream + │ 2. deliver event + ▼ + AIAC Agent + │ 3. GET /roles, /services, /assignments ──► IdP Configuration Service ──► Keycloak Admin REST + │ 4. semantic query (policy + domain knowledge) ──► ChromaDB + │ 5. [LLM] compute PolicyModel delta for all services affected by the role change + │ 6. [LLM] validate policy model against retrieved policy (second pass) + │ 7. POST /policy (write updated PolicyModel) ──► PDP Policy Writer ──► AuthorizationPolicy CR + │ 8. ACK message + ▼ + NATS JetStream (message removed from pending) +``` + +#### UC-2a · Incremental Policy Update (`aiac.apply.policy.build`) + +``` + Operator + │ 1. POST /ingest/policy/{text|file|url} + ▼ + RAG Ingest Service + │ 2. upsert documents ──► ChromaDB + │ 3. publish aiac.apply.policy.build + ▼ + NATS JetStream + │ 4. deliver event + ▼ + AIAC Agent + │ 5. GET /roles, /services, /assignments ──► IdP Configuration Service ──► Keycloak Admin REST + │ 6. retrieve full policy context ──► ChromaDB + │ 7. [LLM] compute full PolicyModel delta against current OPA state + │ 8. POST /policy (write updated PolicyModel) ──► PDP Policy Writer ──► AuthorizationPolicy CR + │ 9. ACK message + ▼ + NATS JetStream (message removed from pending) +``` + +#### UC-2b · Full Rebuild (`POST /apply/policy/rebuild`, operator-only) + +``` + Operator + │ 1. POST /apply/policy/rebuild (kubectl port-forward → Agent pod) + ▼ + AIAC Agent + │ 2. DELETE /policy (clear all OPA policy rules) ──► PDP Policy Writer ──► AuthorizationPolicy CR + │ 3. GET /roles, /services (read fresh entity state) ──► IdP Configuration Service ──► Keycloak Admin REST + │ 4. retrieve full policy context ──► ChromaDB + │ 5. [LLM] compute complete PolicyModel from scratch + │ 6. POST /policy (write full PolicyModel) ──► PDP Policy Writer ──► AuthorizationPolicy CR + ▼ + (synchronous HTTP response to operator) +``` + +--- diff --git a/aiac/docs/specs/PRD.md b/aiac/docs/specs/PRD.md new file mode 100644 index 00000000..69f8ef6d --- /dev/null +++ b/aiac/docs/specs/PRD.md @@ -0,0 +1,590 @@ +# PRD: AI-based Access Control (AIAC) + +## Abstract + +AI-based Access Control (AIAC) is a Kagenti platform extension that automates RBAC/ABAC policy +enforcement for AI agents running on Kubernetes. A LangGraph-based AI agent continuously translates +a natural-language access control policy — stored in a vector knowledge base — into concrete +permission configurations in the active Policy Decision Point (PDP), eliminating manual policy +administration and preventing policy drift as services and roles evolve. The PDP backend is OPA, +which evaluates LLM-generated Rego rules; Keycloak remains the identity provider for entity +management (subjects, roles, services). + +--- + +## 1. Problem Description + +Kagenti AI agents call services across a shared platform. Every call must carry a token scoped to +exactly the permissions the caller's role entitles on the target service. Without a dedicated +policy management layer, access policy ends up scattered across per-deployment configuration, +creating three compounding problems: + +1. **Policy drift** — new services and roles are onboarded without corresponding permission + updates because there is no automated mechanism to apply them. +2. **Distributed policy intent** — no single authoritative source declares what roles may do; + policy knowledge is fragmented across deployments. +3. **Manual administration overhead** — keeping OPA policy rules consistent with a growing fleet + of agents and tools requires ongoing human attention with no audit trail. + +--- + +## 2. Problem Solution + +AIAC introduces a strict three-layer model that cleanly separates policy concerns: a **Policy +Management** layer (AIAC Agent) that translates natural-language policy into PDP configuration, a +**Policy Decision** layer (OPA) that evaluates caller entitlements, and a **Policy Enforcement** +layer (AuthBridge) that intercepts traffic and exchanges tokens but carries no policy knowledge of +its own. + +The AIAC Agent subscribes to an event stream (NATS JetStream) and reacts to entity lifecycle +events — new services, role changes, policy updates — by retrieving the current policy from a RAG +knowledge base, querying live PDP state, and applying the minimal required diff via a dedicated +PDP Policy Writer. **Policy intent lives entirely in the PDP, not in per-pod configuration.** + +--- + +## 3. Design Principles + +### PDP/PEP separation + +AIAC enforces a strict three-layer model: + +| Layer | Component | Role | +|---|---|---| +| **Policy Management** | AIAC Agent | Translates natural-language policy into PDP configuration on every trigger | +| **Policy Decision (PDP)** | OPA | Evaluates LLM-generated Rego rules; decides what a caller may access | +| **Policy Enforcement (PEP)** | AuthBridge | Intercepts traffic; exchanges tokens; carries no policy knowledge | + +The PEP (AuthBridge) is a pure enforcement layer. It performs RFC 8693 token exchanges sending only the target `audience` — no `scope` parameter. OPA evaluates the caller's role against the Rego rules and returns exactly the entitlements that role grants on the target service; Keycloak, as the authorization server, issues the token scoped to those entitlements. + +This means `token_scopes` is absent from `authproxy-routes`. Route configuration carries routing intent only (`host` → `target_audience`). Policy intent lives entirely in OPA, kept current by AIAC. + +--- + +## 4. Major Use-Cases + +### UC-1 · Continuous Access Reconciliation (On-boarding / Off-boarding) + +**Trigger:** A Role or Keycloak Client is created, updated, or removed. + +The Keycloak SPI listener publishes a scoped event to the Event Broker. The AIAC Agent retrieves +relevant context from the RAG store, reads the current OPA policy state, and asks the LLM to +compute the minimal permission diff scoped to the affected entity. The diff is validated by a +second LLM pass and applied to OPA as updated Rego rules. Supports both **auto-apply** (fully +automated, least-privilege) and **recommendation + human review** modes. + +### UC-2 · Policy Update Reconciliation + +**Trigger:** An operator ingests updated documents into the RAG store. + +After ingestion the RAG Ingest Service publishes a build event. The AIAC Agent retrieves all +relevant context, computes a full policy diff against current OPA state, and applies the delta. +A `rebuild` variant (operator-only, direct HTTP) first clears all OPA policy rules before +recomputing from scratch — used when policy changes are too broad for incremental diff. + +### UC-3 · Entitlements Review + +**Trigger:** Operator request (on-demand or scheduled). + +The agent evaluates all current OPA policy rules — including manually added ones that AIAC did not +create — against the natural-language policy. It reports compliant, non-compliant, and +policy-agnostic entitlements, enabling audit and remediation workflows. + +### UC-4 · Access Request + +**Trigger:** User request via chatbot. + +A user requests an entitlement grant. The agent verifies the request against the policy +(permissive approach) and either auto-grants or routes to a human approver (man-in-the-loop). +Manually granted entitlements are flagged as policy-agnostic and surfaced during UC-3 reviews. + +--- + +## 5. Architecture Overview + +Eight components across five Kubernetes Pods plus a Python library layer, all implemented in Python 3.12. External dependencies: Keycloak Admin API, an LLM API, and an embedding API. The Keycloak SPI listener is defined in a separate PRD. + +### Component Summary + +| # | Component | Description | +|---|-----------|-------------| +| 1 | **IdP Configuration Service** | REST service that exposes IdP entity data (subjects, roles, services, scopes) for read and write operations. Read methods enrich services with assigned roles/scopes and enrich roles with child roles. Backed by Keycloak. Python library: `aiac.idp.configuration`. | +| 2 | **PDP Policy Writer** | REST service that applies LLM-generated Rego rules to the OPA backend. Writes derived Rego packages to an `AuthorizationPolicy` Kubernetes CR. Exposed as ClusterIP service `aiac-pdp-policy-service:7072`. Python library: `aiac.pdp.policy.library`. | +| 3 | **Policy Store** | REST service that owns an in-memory `PolicyModel` cache backed by SQLite as the authoritative structured policy store. Enables the Policy Computation Engine to read current `AgentPolicyModel` state for additive merging. Deployed as a dedicated single-replica StatefulSet (`aiac-policy-store`) at `:7074`. Python library: `aiac.policy.store.library`. | +| 4 | **Policy Computation Engine** | Pure Python library module (`aiac.policy.computation`). No service, no Kubernetes deployment. Receives `list[PolicyRule]` from AIAC Agent sub-agents, queries IdP to resolve owning services, additively merges rules into `AgentPolicyModel` objects in the Policy Store, and pushes the updated `PolicyModel` to the PDP Policy Writer. Single entry point: `compute_and_apply(rules)`. | +| 5 | **Policy and Domain Knowledge RAG** | ChromaDB vector store holding the access control policy and domain knowledge in persistent, queryable form, populated via a co-located RAG Ingest Service. | +| 6 | **Event Broker** | NATS JetStream pod that decouples event producers (Keycloak SPI listener, RAG Ingest Service) from the AIAC Agent. Provides durable, at-least-once delivery with automatic replay on Agent pod restart. Competing consumer model ensures each event is processed exactly once. | +| 7 | **AIAC Agent** | LangGraph-based AI agent triggered by Event Broker subscriptions (`aiac.apply.>` subjects) and directly by the operator (`rebuild` only). Retrieves the current policy from the RAG store, interprets it against live PDP state, and applies the required policy changes immediately. | +| 8 | **Python library** | Python API library provides typed access to IdP and policy services via `aiac.idp.configuration`, `aiac.policy.model`, `aiac.policy.store.library`, `aiac.pdp.policy.library`, and `aiac.policy.computation` modules backed by generic Pydantic models. | + +### High-level architecture + +``` + (𝗞𝗲𝘆𝗰𝗹𝗼𝗮𝗸 𝗔𝗣𝗜) (𝗞𝘂𝗯𝗲𝗿𝗻𝗲𝘁𝗲𝘀 𝗖𝗥 𝗔𝗣𝗜) + ▲ ▲ + │ | + (𝘶𝘴𝘦𝘳𝘴, 𝘳𝘰𝘭𝘦𝘴, 𝘤𝘭𝘪𝘦𝘯𝘵𝘴) (𝘈𝘶𝘵𝘩𝘰𝘳𝘪𝘻𝘢𝘵𝘪𝘰𝘯𝘗𝘰𝘭𝘪𝘤𝘺 𝘊𝘙) +┌──────────────┼──────────────────────┼───────────────────┐ +│ Kagenti Interface Pod │ │ +│ │ │ │ +│ ┌───────┴──────┐ ┌────────┴───────┐ │ +│ │ IdP Config │ │ PDP Policy │ │ +│ │ Service │ │ Writer (OPA) │ │ +│ └──────────────┘ └────────────────┘ │ +│ ▲ ▲ │ +└──────────────┼──────────────────────┼───────────────────┘ + │ │ + │ │ + │ │ + │ ┌──────────────────────────────────────┐ + │ │ Policy Store Pod │ + │ │ │ + │ │ ┌───────────────────────────────┐ │ + │ │ │ Policy Store Service │ │ + │ │ │ │ │ + │ │ │ (SQLite policy.db) │ │ + │ │ └───────────────────────────────┘ │ + │ │ ▲ │ + │ └──────────────────┼───────────────────┘ + │ │ +┌──────────────┼──────────────────────┼───────────────────┐ ┌────────────────────────────────┐ +│ Agent Pod └───────────────────┐ │ │ │ Event Broker Pod │ +│ │ │ │ │ │ +│ ┌──────────────────────┐ ┌────────────────┐ │ │ ┌──────────────────────────┐ │ +│ │ Policy Compute Engn │◄──│ AIAC Agent │◄─────────┼──┼──│ NATS JetStream │ │ +│ └──────────────────────┘ └────────────────┘ (𝘯𝘰𝘵𝘪𝘧𝘺) │ │ └──────────────────────────┘ │ +│ │ │ │ ▲ ▲ │ +│ │ │ │ │ │ │ +└─────────────────────────────────────┼───────────────────┘ └─────────┼──────────────┼───────┘ + │ (𝘱𝘶𝘣𝘭𝘪𝘴𝘩) (𝘱𝘶𝘣𝘭𝘪𝘴𝘩) +┌─────────────────────────────────────┼───────────────────┐ │ │ +│ Policy / Domain Knowledge RAG Pod │ │ (𝗞𝗲𝘆𝗰𝗹𝗼𝗮𝗸 𝗦𝗣𝗜) (𝗥𝗔𝗚 𝗜𝗻𝗴𝗲𝘀𝘁) +│ ▼ │ +│ ┌─────────────────────┐ ┌─────────────────────────┐ │ +│ │ RAG Ingest Service │──►│ ChromaDB (vector store) │ │ +│ └─────────────────────┘ └─────────────────────────┘ │ +└─────────────────────────────────────────────────────────┘ +``` + +All inter-pod traffic is Kubernetes ClusterIP. External access is exclusively via +`kubectl port-forward` (operator/developer) or NATS publish (Keycloak SPI, RAG Ingest). + +### Call Flows + +#### UC-1a · Service On-boarding (`aiac.apply.service.{id}`) + +``` + Keycloak SPI + │ CLIENT_CREATED + │ 1. publish aiac.apply.service.{id} + ▼ + NATS JetStream + │ (durable consumer, at-least-once delivery) + │ 2. deliver event + ▼ + AIAC Agent + │ 3. GET /services, /roles, /assignments ──► IdP Configuration Service ──► Keycloak Admin REST + │ 4. GET /services/{id}/roles, /services/{id}/scopes ──► IdP Configuration Service ──► Keycloak Admin REST + │ 5. semantic query (policy + domain knowledge) ──► ChromaDB + │ 6. [LLM] compute list[PolicyRule] for new service (inbound + outbound rules) + │ 7. [LLM] validate policy rules against retrieved policy (second pass) + │ 8. compute_and_apply(rules) ──► Policy Computation Engine + │ ├── get_services_by_role / get_services_by_scope ──► IdP Configuration Service + │ ├── get_agent_policy / apply_agent_policy ──► Policy Store + │ └── apply_policy ──► PDP Policy Writer ──► AuthorizationPolicy CR + │ 9. ACK message + ▼ + NATS JetStream (message removed from pending) +``` + +#### UC-1b · Role On-boarding (`aiac.apply.role.{id}`) + +``` + Keycloak SPI + │ REALM_ROLE_CREATED / REALM_ROLE_UPDATED + │ 1. publish aiac.apply.role.{id} + ▼ + NATS JetStream + │ 2. deliver event + ▼ + AIAC Agent + │ 3. GET /roles, /services, /assignments ──► IdP Configuration Service ──► Keycloak Admin REST + │ 4. semantic query (policy + domain knowledge) ──► ChromaDB + │ 5. [LLM] compute list[PolicyRule] delta for all services affected by the role change + │ 6. [LLM] validate policy rules against retrieved policy (second pass) + │ 7. compute_and_apply(rules) ──► Policy Computation Engine + │ ├── get_services_by_role / get_services_by_scope ──► IdP Configuration Service + │ ├── get_agent_policy / apply_agent_policy ──► Policy Store + │ └── apply_policy ──► PDP Policy Writer ──► AuthorizationPolicy CR + │ 8. ACK message + ▼ + NATS JetStream (message removed from pending) +``` + +#### UC-2a · Incremental Policy Update (`aiac.apply.policy.build`) + +``` + Operator + │ 1. POST /ingest/policy/{text|file|url} + ▼ + RAG Ingest Service + │ 2. upsert documents ──► ChromaDB + │ 3. publish aiac.apply.policy.build + ▼ + NATS JetStream + │ 4. deliver event + ▼ + AIAC Agent + │ 5. GET /roles, /services, /assignments ──► IdP Configuration Service ──► Keycloak Admin REST + │ 6. retrieve full policy context ──► ChromaDB + │ 7. [LLM] compute list[PolicyRule] delta against current OPA state + │ 8. compute_and_apply(rules) ──► Policy Computation Engine + │ ├── get_services_by_role / get_services_by_scope ──► IdP Configuration Service + │ ├── get_agent_policy / apply_agent_policy ──► Policy Store + │ └── apply_policy ──► PDP Policy Writer ──► AuthorizationPolicy CR + │ 9. ACK message + ▼ + NATS JetStream (message removed from pending) +``` + +#### UC-2b · Full Rebuild (`POST /apply/policy/rebuild`, operator-only) + +``` + Operator + │ 1. POST /apply/policy/rebuild (kubectl port-forward → Agent pod) + ▼ + AIAC Agent + │ 2. DELETE /policy (clear all OPA policy rules) ──► PDP Policy Writer ──► AuthorizationPolicy CR + │ 3. DELETE /policy (clear Policy Store) ──► Policy Store + │ 4. GET /roles, /services (read fresh entity state) ──► IdP Configuration Service ──► Keycloak Admin REST + │ 5. retrieve full policy context ──► ChromaDB + │ 6. [LLM] compute complete list[PolicyRule] from scratch + │ 7. compute_and_apply(rules) ──► Policy Computation Engine + │ ├── get_services_by_role / get_services_by_scope ──► IdP Configuration Service + │ ├── get_agent_policy / apply_agent_policy ──► Policy Store + │ └── apply_policy ──► PDP Policy Writer ──► AuthorizationPolicy CR + ▼ + (synchronous HTTP response to operator) +``` + +### Component dependencies + +| Component | Called by | Calls | Returns | +|-----------|-----------|-------|---------| +| IdP Configuration Service (in Kagenti Interface Pod) | `aiac.idp.configuration.api` | Keycloak Admin REST API | Raw Keycloak JSON (generic endpoint names) | +| PDP Policy Writer — OPA (in Kagenti Interface Pod) | `aiac.pdp.policy.library` | Kubernetes CR (`AuthorizationPolicy`) | 204 on success | +| Policy Store (StatefulSet `aiac-policy-store`) | `aiac.policy.store.library` | SQLite (`agent_policies` table, in-memory cache) | `AgentPolicyModel` / `PolicyModel` on read; 204 on write | +| Policy Computation Engine (`aiac.policy.computation`) | AIAC Agent sub-UC agents | `aiac.idp.configuration.api`, `aiac.policy.store.library`, `aiac.pdp.policy.library` | None (fire-and-forget; logs exceptions) | +| `aiac.idp.configuration.models` | `aiac.idp.configuration.api`, `aiac.policy.model`, AIAC Agent | — | Pydantic model definitions for IdP entities (Subject, Role, Service, Scope) | +| `aiac.idp.configuration.api` | AIAC Agent, Policy Computation Engine, Python scripts | IdP Configuration Service (HTTP) | Typed Pydantic instances (reads and writes IdP configuration entities) | +| `aiac.policy.model` | `aiac.pdp.policy.library`, `aiac.policy.store.library`, `aiac.policy.computation`, AIAC Agent | — | Pydantic model definitions for policy entities (PolicyRule, AgentPolicyModel, PolicyModel) | +| `aiac.pdp.policy.library` | `aiac.policy.computation` | PDP Policy Writer — OPA (HTTP) | None (writes Rego policy rules to AuthorizationPolicy CR) | +| `aiac.policy.store.library` | `aiac.policy.computation` | Policy Store (HTTP) | `AgentPolicyModel` / `PolicyModel` on read; None on write/delete | +| ChromaDB | RAG Ingest Service (writes), AIAC Agent (reads) | — | Policy and domain knowledge vectors | +| RAG Ingest Service | Developer (via `kubectl port-forward`) | ChromaDB, Embedding API, Event Broker | — | +| Event Broker (NATS JetStream) | Keycloak SPI listener, RAG Ingest Service (publishers); NATS JetStream (DLQ routing) | — | Durable event delivery to AIAC Agent; DLQ on max retries | +| AIAC Agent | Event Broker (NATS consumer), operator (`/apply/policy/rebuild` HTTP direct) | Service Onboarding / Policy Update / Role Update orchestrators → `aiac.idp.configuration.api`, `aiac.policy.computation`, ChromaDB, LLM API, Kubernetes API | Rego policy written to AuthorizationPolicy CR; structured policy written to Policy Store (SQLite); provisioned service permissions/scopes (onboarding) | + +### Key architectural decisions + +- **Stateless PDP services are co-located in the Kagenti Interface Pod; the stateful Policy Store is separate.** IdP Configuration Service and PDP Policy Writer run as two containers in the Interface Pod, sharing a Kubernetes ServiceAccount. The Policy Store is a dedicated single-replica StatefulSet (`aiac-policy-store`) with its own PVC — decoupled from the Interface Pod's restart lifecycle. Three ClusterIP Services (`aiac-pdp-config-service:7071`, `aiac-pdp-policy-service:7072`, `aiac-policy-store-service:7074`) provide stable addressing. +- **Policy Computation Engine is a library, not a service.** `aiac.policy.computation` runs in-process within the AIAC Agent pod. It requires no Kubernetes deployment, no container image, and no ClusterIP Service. Sub-agents call `compute_and_apply(rules)` directly. +- **One CR + one SQLite store, distinct owners, distinct purposes.** The Policy Store owns a SQLite `agent_policies` table (backed by a 1 Gi RWO PVC) holding structured `AgentPolicyModel` data — the source of truth for policy state, served from an in-memory cache. The `AuthorizationPolicy` CR (one total, owned by the PDP Policy Writer) holds derived Rego packages for OPA runtime. The two services have no dependency on each other; both are driven by the PCE via their respective libraries. +- **`aiac.pdp.policy.library` has one caller: `aiac.policy.computation`.** AIAC Agent sub-agents do not call the PDP Policy Library directly; they call `compute_and_apply()` instead. This centralises all Policy Store ↔ PDP Policy Writer coordination. +- **Clean `idp` / `pdp` / `policy` Python namespace split.** IdP-related code (Keycloak entity management) lives under `aiac.idp.*`; PDP policy code (OPA Rego writing) lives under `aiac.pdp.*`; shared policy model and computation code lives under `aiac.policy.*`. +- **`aiac.policy.model` is dependency-free (only `pydantic` + `aiac.idp.configuration.models`).** `PolicyRule`, `AgentPolicyModel`, and `PolicyModel` live in a neutral namespace importable by any consumer — Policy Store library, PDP Policy Library, PCE — without forcing a dependency on any service namespace. +- **`PolicyRule.role` and `PolicyRule.scope` are typed objects.** They hold `Role` and `Scope` instances from `aiac.idp.configuration.models`, enabling the PCE to call `Configuration.get_services_by_role` and `Configuration.get_services_by_scope` without additional type conversion. +- **`AgentPolicyModel` relationship maps are keyed by string `id`.** `source_roles`, `subject_roles`, and `target_scopes` use the entity's string `id` as the dict key, so `Service`, `Role`, `Scope`, and `Subject` need no custom hash/eq and keep pydantic's default field-based equality. This also lets the maps serialize to JSON without a custom key serializer. +- **PCE merge semantics are additive only.** New rules are appended to existing `inbound_rules`/`outbound_rules`; existing rules are never removed. Rule revocation is TBD. +- **PDP services bind to `0.0.0.0`.** Exposed as Kubernetes ClusterIP Services so that the Agent Pod can reach them over the cluster network. +- **RBAC via OPA Rego rules.** AIAC manages role → service permission mappings by writing `AgentPolicyModel` instances to the `AuthorizationPolicy` CR. Each agent pod's OPA plugin fetches its packages from the CR at startup. +- **RAG Pod is a StatefulSet with persistent ChromaDB storage.** ChromaDB data is stored on a 1 Gi `ReadWriteOnce` PersistentVolumeClaim mounted at `/chroma/chroma` (ChromaDB default). On pod recreation, the StatefulSet rebinds the same PVC and ChromaDB resumes from persisted state without re-ingestion. The pod runs a single replica. +- **RAG Pod runs ChromaDB and RAG Ingest Service together.** Exposed as `aiac-rag-service` on ports 8000 (ChromaDB default) and 7073 (RAG Ingest Service). +- **AIAC Agent is stateless.** Changes are applied immediately on trigger — no pending session or human confirmation step. +- **Event Broker decouples all automated triggers from the Agent.** The Keycloak SPI listener and RAG Ingest Service publish to NATS subjects; the Agent subscribes as a durable competing consumer. This removes all direct dependencies between trigger sources and the Agent. +- **`rebuild` bypasses the Event Broker.** It is an operator-only command issued directly via HTTP (`kubectl port-forward`). It is never published to NATS and has no NATS listener. +- **NATS consumer is a thin adapter.** It receives events from the Event Broker and calls the same internal handler functions used by the debug HTTP endpoints. No business logic lives in the consumer. +- **Agent HTTP endpoints are retained for debugging.** They are not the primary trigger path; the NATS consumer is. `kubectl port-forward` to the Agent is used only for `rebuild` and debugging. +- **Event Broker uses WorkQueuePolicy.** Messages are removed from the stream after acknowledgement. Unacknowledged messages survive Agent pod restarts and are redelivered automatically. After 5 failed deliveries, messages are routed to `aiac.apply.dlq`. +- **AIAC init container gates Agent startup.** Before the Agent container starts, the init container waits for NATS, IdP Configuration Service, PDP Policy Writer, and RAG Ingest Service to be healthy, then creates the `aiac-events` JetStream stream idempotently. +- **All `__init__.py` files under `aiac.*` are empty.** Callers use explicit submodule paths: `from aiac.idp.configuration.models import Subject`, `from aiac.policy.model.models import PolicyModel`. +- **ChromaDB hosts two collections: `aiac-policies` and `aiac-domain-knowledge`.** Collection slug to ChromaDB name mapping: `policy` → `aiac-policies`, `domain-knowledge` → `aiac-domain-knowledge`. +- **`user/{id}` trigger not implemented.** OPA rules are role-scoped; individual user creation/update does not require agent intervention — OPA rule evaluation resolves entitlements from the caller's role automatically. + +--- + +## 6. Kagenti / Keycloak / OPA Interfaces + +**AIAC ↔ Kagenti platform** +The AIAC Agent reads `AgentRuntime` and `AgentCard` custom resources from the Kubernetes API to +extract service metadata during UC-1 service onboarding. The `aiac.idp.configuration` and `aiac.pdp.policy.library` Python packages are the integration surface for other Kagenti components needing typed access to the IdP and PDP respectively. + +**AIAC ↔ Keycloak** +The IdP Configuration Service proxies Keycloak Admin REST endpoints under generic entity names (subjects, roles, services, scopes, assignments). Read endpoints include per-service role and scope enrichment. The Keycloak SPI listener publishes entity lifecycle events to NATS; it is a separate component outside the AIAC codebase. + +**AIAC ↔ OPA** +The PDP Policy Writer (`aiac-pdp-policy-opa`) writes LLM-generated Rego packages to an `AuthorizationPolicy` Kubernetes CR. Each agent pod embeds two OPA plugin instances inside AuthBridge (one for the inbound pipeline, one for the outbound pipeline); each plugin fetches its Rego packages from the CR at startup. AuthBridge requires no changes when policy rules are updated. Full spec: [components/pdp-policy-writer-opa.md](components/pdp-policy-writer-opa.md). + +**AIAC ↔ Event Broker (NATS JetStream)** +The Agent subscribes to the event stream as a durable consumer with at-least-once delivery. +Unacknowledged messages survive pod restarts; failed messages are routed to a dead-letter subject. +See Section 7.5 (Event Broker) and Section 8 (Deployment) for subject names and handler mapping. + +--- + +## 7. AIAC System Components + +### 7.1 IdP Configuration Service + +FastAPI service (`0.0.0.0:7071`) co-located with the PDP Policy Writer in the **Kagenti Interface Pod**. Manages IdP (Keycloak) entity data (subjects, roles, services, scopes) via Keycloak Admin REST API. Exposes read and write endpoints for configuration entities. Stateless. All endpoints except `/health` require a `?realm=` query parameter; returns `422` if absent. `/health` requires no realm parameter — it uses `KEYCLOAK_ADMIN_REALM` directly. `KeycloakAdmin` instances are created lazily per realm and cached in a thread-safe map; the admin always authenticates via the realm in `KEYCLOAK_ADMIN_REALM`. + +**Full spec:** [components/idp-configuration-service.md](components/idp-configuration-service.md) + +--- + +### 7.2 PDP Policy Writer + +FastAPI service (`0.0.0.0:7072`, `aiac-pdp-policy-opa`) co-located with the IdP Configuration Service in the **Kagenti Interface Pod**. Writes LLM-generated Rego packages to an `AuthorizationPolicy` Kubernetes CR. Each AuthBridge OPA plugin instance fetches its Rego packages from the CR at startup. + +**Full spec:** [components/pdp-policy-writer-opa.md](components/pdp-policy-writer-opa.md) + +--- + +### 7.3 Policy Store + +FastAPI service (`0.0.0.0:7074`, `aiac-policy-store-service`) deployed as a dedicated single-replica StatefulSet (`aiac-policy-store`) with a `volumeClaimTemplate` PVC (1 Gi, `ReadWriteOnce`) mounted at `/data`. Owns an in-memory `PolicyModel` cache backed by a SQLite database (`/data/state.db`) as the authoritative structured policy store. All GET requests are served from the in-memory cache; mutations write through to SQLite synchronously; on pod restart the cache is repopulated from SQLite. The Policy Computation Engine reads current `AgentPolicyModel` state for additive merging and writes updated state after each computation. The PDP Policy Writer has no dependency on the Policy Store; the SQLite store and `AuthorizationPolicy` CR are written by distinct services and serve distinct purposes. + +**Full spec:** [components/policy-store.md](components/policy-store.md) + +--- + +### 7.4 Policy Computation Engine + +Pure Python library module (`aiac.policy.computation`). No FastAPI, no Kubernetes deployment, no container image. Runs in-process within the AIAC Agent pod. AIAC Agent sub-UC agents call `compute_and_apply(rules: list[PolicyRule]) -> None` to translate partial policy rule lists into merged `AgentPolicyModel` objects and push them to OPA. + +The PCE is the **single point of coordination** between the Policy Store and PDP Policy Writer: it reads current agent policy state, additively merges new rules, writes back to the Policy Store, then pushes the updated `PolicyModel` to `aiac.pdp.policy.library.apply_policy()`. All exceptions are logged; none propagate to the caller. + +**Full spec:** [components/policy-computation-engine.md](components/policy-computation-engine.md) + +--- + +### 7.5 Library + +Python package at `aiac/src/`. Clean `idp` / `pdp` / `policy` namespace split: + +**IdP library** (Keycloak entity management): +- **`aiac.idp.configuration.models`** — dependency-free Pydantic models for IdP entities (`Subject`, `Role`, `Service`, `Scope`). Plain pydantic models with default field-based equality; not hashable and not used as dict keys. +- **`aiac.idp.configuration.api`** — HTTP client wrapping the IdP Configuration Service; read and write access to configuration entities; returns typed Pydantic instances; all methods require a `realm: str` parameter. Includes `get_services_by_role(role)` and `get_services_by_scope(scope)` used by the PCE. + +**Policy model** (shared, dependency-light): +- **`aiac.policy.model`** — canonical Pydantic models for policy entities (`PolicyRule`, `AgentPolicyModel`, `PolicyModel`) with typed `Role`/`Scope`/`Service` fields. Importable by any consumer without pulling in HTTP or service dependencies. + +**Policy libraries** (OPA + Policy Store access): +- **`aiac.pdp.policy.library`** — HTTP client wrapping the PDP Policy Writer (OPA). Four module-level functions: `apply_policy`, `apply_agent_policy`, `delete_agent_policy`, `delete_policy`. Called exclusively by `aiac.policy.computation`. +- **`aiac.policy.store.library`** — HTTP client wrapping the Policy Store. Six module-level functions: `get_policy`, `get_agent_policy`, `apply_policy`, `apply_agent_policy`, `delete_agent_policy`, `delete_policy`. Returns `PolicyModel` and `AgentPolicyModel` directly. Called exclusively by `aiac.policy.computation`. + +**Computation library** (policy rule processing): +- **`aiac.policy.computation`** — library module implementing `compute_and_apply(rules: list[PolicyRule]) -> None`. Orchestrates IdP resolution, Policy Store merge, and PDP Policy Writer push. + +**Full specs:** [components/library-idp.md](components/library-idp.md) · [components/library-pdp-policy.md](components/library-pdp-policy.md) · [components/library-policy-store.md](components/library-policy-store.md) · [components/policy-model.md](components/policy-model.md) · [components/policy-computation-engine.md](components/policy-computation-engine.md) + +--- + +### 7.6 Event Broker + +NATS JetStream pod (`aiac-event-broker-service:4222`). Decouples event producers (Keycloak SPI listener, RAG Ingest Service) from the AIAC Agent. Provides at-least-once delivery, replay on pod restart via `WorkQueuePolicy`, and a dead-letter subject (`aiac.apply.dlq`) after 5 failed deliveries. No authentication — ClusterIP network isolation is the access control mechanism. Stream: `aiac-events`, subjects `aiac.apply.>`, consumer group `aiac-agent-consumer`. + +**Full spec:** [components/event-broker.md](components/event-broker.md) + +--- + +### 7.7 AIAC Agent + +FastAPI + LangGraph service (`0.0.0.0:7070`). Receives automated triggers via the **Event Broker** (NATS JetStream durable consumer, `aiac-agent-consumer` queue group) and the operator-only `rebuild` command directly via HTTP. Structured as a thin **Controller** (`controller/routes.py`) that dispatches `/apply/*` handlers to three **Orchestrators**, each owning one or more compiled `StateGraph` sub-agents. A **NATS consumer** (asyncio background task in the FastAPI `lifespan` handler) is a thin adapter that receives NATS events and calls the same internal handler functions used by the HTTP endpoints: + +| Orchestrator | Trigger(s) | Sub-agents | +|---|---|---| +| Service Onboarding | `aiac.apply.service.{id}` | Service Provision → Service Policy Builder (sequential) | +| Policy Update | `aiac.apply.policy.build`, `/apply/policy/rebuild` (HTTP) | Build sub-agent or Rebuild sub-agent (alternative) | +| Role Update | `aiac.apply.role.{id}` | Role sub-agent | + +All sub-agent `StateGraph` instances are logically separated modules running within a single pod and process. Sub-UC agents produce `list[PolicyRule]` and call `compute_and_apply(rules)` — they do not call `aiac.policy.store.library` or `aiac.pdp.policy.library` directly. The **Policy Update** sub-agents compute a minimal rule delta between the current ChromaDB policy and live OPA state. The **Rebuild** variant additionally clears the Policy Store and all OPA policy rules before recomputing. The **Role Update** orchestrator computes rules for all services affected by the role change. The **Service Onboarding** orchestrator classifies the new service via the pod's `kagenti.io/type` label (for agents reads the `AgentCard` CR; for tools calls `tools/list` on the MCP endpoint discovered via K8s Service label lookup), then computes rules and calls `compute_and_apply`. Stateless; changes are applied immediately. Integrated retry with differentiated error codes per upstream. + +**Full spec:** [components/aiac-agent.md](components/aiac-agent.md) + +--- + +### 7.8 RAG Knowledge Base + +ChromaDB vector store (`aiac-rag-service:8000`) hosting two collections: `aiac-policies` (access control policy rules) and `aiac-domain-knowledge` (org/business context such as team rosters, application ownership, and department mappings). Both collections are managed by the RAG Ingest Service and read by the AIAC Agent. Co-located with the RAG Ingest Service in the RAG Pod. ChromaDB data is persisted on a 1 Gi PVC mounted at `/chroma/chroma`; the RAG Pod is a StatefulSet. + +**Full spec:** [components/rag-knowledge-base.md](components/rag-knowledge-base.md) + +--- + +### 7.9 RAG Ingest Service + +FastAPI service (`0.0.0.0:7073`) co-located with ChromaDB. Thirteen collection-parameterized endpoints across three semantics: complete collection replacement (`POST /ingest/{collection}/{text|file|url}`), document-level upsert (`POST /ingest/{collection}/update/{text|file|url}`), and explicit removal (`DELETE /ingest/{collection}/{doc_id}`). The `{collection}` slug is validated against `AIAC_RAG_COLLECTIONS` (default: `policy,domain-knowledge`). After every successful ingest the service publishes to `aiac.apply.policy.build` on the Event Broker (`NATS_URL`). Developer access via `kubectl port-forward`. + +**Full spec:** [components/rag-ingest-service.md](components/rag-ingest-service.md) + +--- + +### 7.10 Keycloak SPI Listener + +A custom Keycloak Event Listener SPI (Java) that listens to Keycloak's internal event bus and translates entity-scoped events into NATS publish calls to the Event Broker. The AIAC Agent subject schema is authoritative; the SPI PRD references it. + +| Keycloak Event | Event Broker subject | +|---|---| +| `REGISTER`, `UPDATE_PROFILE` (user events) | — (dropped; OPA rules are role-scoped and resolve entitlements from the caller's role automatically) | +| `CLIENT_CREATED` | `aiac.apply.service.{id}` | +| Role created/updated | `aiac.apply.role.{id}` | + +**Full spec:** TBD (separate PRD). + +--- + +## 8. Deployment + +### Kubernetes manifests + +Four separate manifest files: + +| File | Contents | +|------|----------| +| `aiac/k8s/pdp-interface-deployment.yaml` | `aiac-pdp-config` ConfigMap + Kagenti Interface Pod Deployment (IdP Configuration Service container + PDP Policy Writer container) + two ClusterIP Services (`aiac-pdp-config-service:7071`, `aiac-pdp-policy-service:7072`) | +| `aiac/k8s/policy-store-statefulset.yaml` | `aiac-policy-store` StatefulSet (Policy Store container) + `volumeClaimTemplate` (1 Gi, `ReadWriteOnce`, mounted at `/data`) + headless Service + `aiac-policy-store-service:7074` ClusterIP Service | +| `aiac/k8s/agent-deployment.yaml` | Agent Pod Deployment (aiac-init container + AIAC Agent container) + ClusterIP Service | +| `aiac/k8s/event-broker-deployment.yaml` _(pending)_ | Event Broker Pod Deployment (NATS JetStream) + ClusterIP Service | +| `aiac/k8s/rag-statefulset.yaml` _(pending)_ | RAG StatefulSet (ChromaDB + RAG Ingest Service containers) + 1 Gi PVC template + ClusterIP Service | + +The two Interface Pod containers mount `aiac-pdp-config` (KEYCLOAK_URL, KEYCLOAK_REALM, KEYCLOAK_ADMIN_REALM) and `keycloak-admin-secret` (KEYCLOAK_ADMIN_USERNAME, KEYCLOAK_ADMIN_PASSWORD) as env vars. The IdP Configuration Service uses `KEYCLOAK_ADMIN_REALM` (admin auth realm) and ignores `KEYCLOAK_REALM`; the PDP Policy Writer uses `KEYCLOAK_REALM` as its default operating realm. The Policy Store container mounts `aiac-policy-store-config` for `AGENTPOLICY_DB_PATH` (default `/data/state.db`) — no Kubernetes API access or RBAC required. + +### Docker images + +Built independently. No entry in the repo's `build.yaml` CI matrix. + +```bash +# Build IdP Configuration Service (Kagenti Interface Pod container 1) +docker build -f aiac/src/aiac/idp/service/configuration/keycloak/Dockerfile -t aiac-pdp-config:latest aiac/src/ + +# Build PDP Policy Writer — Phase 1 mock (Kagenti Interface Pod container 2; writes Rego to filesystem) +docker build -f aiac/src/aiac/pdp/service/policy/keycloak/Dockerfile -t aiac-pdp-policy-keycloak:latest aiac/src/ + +# Build PDP Policy Writer — Phase 2 OPA (replaces mock via issue 4.18; writes to AuthorizationPolicy CR) +docker build -f aiac/src/aiac/pdp/service/policy/opa/Dockerfile -t aiac-pdp-policy-opa:latest aiac/src/ + +# Build Policy Store (deployed as StatefulSet aiac-policy-store) +docker build -f aiac/src/aiac/policy/store/service/Dockerfile -t aiac-policy-store:latest aiac/src/ + +# Build Agent (includes aiac-init container) +docker build -f aiac/src/aiac/agent/controller/Dockerfile -t aiac-agent:latest aiac/src/ + +# Build RAG Ingest Service +docker build -t aiac-rag-ingest:latest aiac/rag-ingest/ +``` + +The Event Broker uses the official `nats` Docker image with JetStream enabled (`-js` flag). No custom build required. + +### `aiac-pdp-config` ConfigMap template + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: aiac-pdp-config +data: + KEYCLOAK_URL: "http://keycloak-service.keycloak.svc:8080" + KEYCLOAK_REALM: "kagenti" + KEYCLOAK_ADMIN_REALM: "master" + AIAC_PDP_CONFIG_URL: "http://aiac-pdp-config-service:7071" + AIAC_PDP_POLICY_URL: "http://aiac-pdp-policy-service:7072" + AIAC_POLICY_STORE_URL: "http://aiac-policy-store-service:7074" + NATS_URL: "nats://aiac-event-broker-service:4222" + AIAC_RAG_INGEST_URL: "http://aiac-rag-service:7073" + AIAC_CHROMADB_URL: "http://aiac-rag-service:8000" +``` + +`AGENTPOLICY_DB_PATH` is absent — it belongs to `aiac-policy-store-config` (defined in `policy-store-statefulset.yaml`), not to the shared ConfigMap. + +### `aiac-policy-store-config` ConfigMap template + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: aiac-policy-store-config +data: + AGENTPOLICY_DB_PATH: "/data/state.db" +``` + +Update `KEYCLOAK_URL` and `KEYCLOAK_REALM` for the target environment before applying. + +--- + +## 9. Testing + +Tests live in `aiac/test/`. + +### Unit tests + +| Target | What to mock | What to assert | +|--------|-------------|----------------| +| IdP Configuration Service endpoints | `KeycloakAdmin` methods (return fixture dicts) | Correct JSON response, 502 on Keycloak error | +| PDP Policy Writer (OPA) endpoints | Kubernetes CR write (`AuthorizationPolicy`) | 204 on success, 502 on CR write error | +| Policy Store endpoints | SQLite `:memory:` database | Correct read/write/delete; 404 on missing agent; 502 on SQLite write error; 503 on SQLite open/query failure at `/health` | +| `aiac.policy.store.library` functions | Policy Store HTTP endpoints | Correct method + path per function; returns typed model on read; `RuntimeError` on non-2xx; default URL fallback | +| `aiac.policy.model` | No mock needed | `extra='ignore'` drops unknown fields; relationship maps keyed by string `id` round-trip through `model_dump(mode="json")` / `model_validate` with typed `Role` / `Scope` values preserved | +| `aiac.idp.configuration.api` functions | IdP Configuration Service HTTP endpoints | Returns correct Pydantic model instances; `RuntimeError` on non-2xx; default URL fallback; `get_services_by_role` and `get_services_by_scope` issue correct query params | +| `aiac.pdp.policy.library` functions | PDP Policy Writer HTTP endpoints | Correct serialisation; `RuntimeError` on non-2xx; default URL fallback | +| `aiac.policy.computation` | `aiac.idp.configuration.api`, `aiac.policy.store.library`, `aiac.pdp.policy.library` (import-boundary mocks) | Correct `apply_agent_policy` calls per resolved service; additive merge preserves existing rules; no duplicate rule insertion; `apply_policy` called once after all writes; exceptions logged, not propagated | +| Event Broker NATS consumer | NATS message delivery (mock `nats-py` subscription) | Correct handler dispatched per subject; ack issued on success; no ack on handler exception | +| Event Broker DLQ | NATS max redelivery exceeded | Message routed to `aiac.apply.dlq` after 5 failures | +| Init container health-check | HTTP 4xx then 200 sequence; NATS TCP refused then connected | Exits 0 only after all four dependencies healthy; `add_stream` called with correct config | +| AIAC Agent | TBD | TBD | + +### Integration tests + +Require a live Keycloak instance. Controlled by env vars: + +| Variable | Description | +|----------|-------------| +| `KEYCLOAK_URL` | Keycloak base URL | +| `KEYCLOAK_REALM` | Realm to query | +| `KEYCLOAK_ADMIN_USERNAME` | Admin username | +| `KEYCLOAK_ADMIN_PASSWORD` | Admin password | + +Integration tests call the live IdP Configuration Service (running locally or via port-forward) and assert that results are non-empty lists of the correct type. Event Broker integration tests require a live NATS JetStream instance. + +Use a pytest marker (e.g. `@pytest.mark.integration`) so unit tests and integration tests can be run independently: + +```bash +pytest aiac/ -m "not integration" # unit only +pytest aiac/ -m integration # integration only +``` + +### Integration test specifications + +Beyond the marker-gated pytest tests above, individual integration tests are specified **one spec per test** under `docs/specs/integration-test/` — a **sibling of `components/`**, following the same "one spec per unit" convention the component PRDs use. This section is the dedicated index of those specs (mirroring the Component Summary in §5) and grows as tests are added; each entry is a distinct integration test with its own spec. + +| Integration test | Description | Spec | +|---|---|---| +| PDP Policy Writer — `generate_rego.py` | Standalone launcher (no Docker) that boots the OPA stub locally, applies a `PolicyModel` through `aiac.pdp.policy.library`, and writes the generated Rego to a known directory for manual inspection. Write-only; not `@pytest.mark.integration`. | [integration-test/pdp-policy-writer.md](integration-test/pdp-policy-writer.md) | +| `policy-pipeline` — `policy_pipeline.py` | Standalone launcher (no Docker) driving the full identity→policy pipeline — provisions a Keycloak realm + entities, runs the three PRB mappings, applies via the PCE, and writes the generated Rego to a known directory for manual inspection. Write-only; not `@pytest.mark.integration`. | [integration-test/policy-pipeline.md](integration-test/policy-pipeline.md) | +| `uc1-onboarding-pipeline` — `test_uc1_onboarding_pipeline.py` | Discovery-driven sibling of `policy-pipeline` validating the **phase-1** deliverable: deploys the real `github-agent` + a simplified `github-tool` to a live Kagenti cluster, drives **real UC-1 onboarding** (`POST /apply/service/{id}`) to infer roles/scopes, and asserts the generated Rego with `opa eval`. Same scenario facts/tables as `policy-pipeline`; Rego is semantically similar (not byte-identical). `@pytest.mark.integration`. | [integration-test/uc1-onboarding-pipeline.md](integration-test/uc1-onboarding-pipeline.md) | + +Tracking issues: the live-Keycloak pytest integration tests in `testing/5.1-integration-tests.md`; the PDP Policy Writer integration test in `testing/5.2-pdp-writer-integration-test.md`; the policy-pipeline integration test in `testing/5.3-policy-pipeline-integration-test.md`; the UC-1 onboarding pipeline integration test in `testing/5.4-uc1-onboarding-integration-test.md`. + +--- + +## 10. Conventions and constraints + +- Python version: 3.12 +- Base Docker image: `python:3.12-slim` +- Linting: ruff (line length 120, target py312 per root `pyproject.toml`) +- Commits: DCO sign-off required (`git commit -s`); use `Assisted-By` not `Co-Authored-By` +- No auth on IdP Configuration Service, PDP Policy Writer, RAG Ingest Service, or Event Broker — network isolation (ClusterIP + `kubectl port-forward`) is the access control mechanism +- IdP Configuration Service, PDP Policy Writer, Agent, RAG Ingest Service, and Event Broker are not registered in the repo's `build.yaml` CI matrix; they have independent build processes +- `aiac/__init__.py` exists and is empty — `aiac` is a regular package, not a namespace package +- NATS consumer must **await** handler completion before issuing ack — fire-and-forget (`asyncio.create_task`) is prohibited; premature ack breaks at-least-once delivery guarantees +- AIAC provisioning marker: every role and client scope AIAC provisions carries the Keycloak attribute `aiac.managed` = `true`, distinguishing AIAC-provisioned entities from Keycloak's built-ins (default client scopes, `default-roles-`). Realm-role attribute values are lists (`["true"]`), client-scope values are plain strings (`"true"`). The IdP Configuration Service stamps it on create and returns full role representations so it survives reads; the Policy Computation Engine filters on it (`Role.aiac_managed` / `Scope.aiac_managed`) when embedding each agent's own roles/scopes (P2) diff --git a/aiac/docs/specs/components/aiac-agent.md b/aiac/docs/specs/components/aiac-agent.md new file mode 100644 index 00000000..5b2952c5 --- /dev/null +++ b/aiac/docs/specs/components/aiac-agent.md @@ -0,0 +1,244 @@ +# Component PRD: AIAC Agent + +## Description + +A LangGraph-based AI agent service that enforces a natural-language access control policy against the live PDP state. Triggered via the **Event Broker** (NATS JetStream) for all automated triggers, and directly via HTTP for the operator-only `rebuild` command: + +- **Event Broker** → `aiac.apply.service.{id}` subject (originated by Keycloak SPI `CLIENT_CREATED`) +- **Event Broker** → `aiac.apply.role.{id}` subject (originated by Keycloak SPI role created/updated) +- **Event Broker** → `aiac.apply.policy.build` subject (originated by RAG Ingest Service post-ingest) +- **Operator/admin call** → `POST /apply/policy/rebuild` directly via `kubectl port-forward` (HTTP only — not routed through Event Broker) + +The Agent subscribes to the Event Broker as a durable competing consumer (`aiac-agent-consumer` queue group). It acknowledges each message only after successful processing — ensuring at-least-once delivery and automatic replay on pod restart. + +The `/apply/*` HTTP endpoints are retained as a debugging escape hatch. The **NATS consumer is a thin adapter layer** that receives events from the Event Broker and calls the same internal `/apply/*` handler functions — there is no duplicated business logic. + +The service is structured as a **Controller** (FastAPI routes) that dispatches to the **Service Onboarding Orchestrator** (UC1) or directly to the Policy Update and Role Update sub-agents (UC2, UC3). Each producing sub-agent calls the **shared Policy Rules Builder** (`agent/policy_rules_builder/`) directly, merges the results internally, and returns a single `list[PolicyRule]` to the Controller. The Controller calls `compute_and_apply(merged_rules)` from `aiac.policy.computation` (PCE) once. + +| Use Case | Dispatch | Sub-agents | Sub-agent output | +|---|---|---|---| +| Service Onboarding (UC1) | via Orchestrator | Service Provision + Service Policy Builder | `list[PolicyRule]` | +| Policy Update (UC2) | Controller → sub-agent directly | Build or Rebuild (TBD) | `list[PolicyRule]` | +| Role Update (UC3) | Controller → sub-agent directly | Role sub-agent | `list[PolicyRule]` | + +Each producing sub-agent calls the **shared Policy Rules Builder** (`agent/policy_rules_builder/`) for each applicable (roles, scope) or (role, scopes) pair, merges the results, and returns a single `list[PolicyRule]` to the Controller. The Controller calls `compute_and_apply(merged_rules)` from `aiac.policy.computation` (PCE) once — no shared apply node exists. The PCE owns all Policy Store ↔ PDP Policy Writer coordination. Neither sub-agents nor the Policy Rules Builder call `aiac.pdp.policy.library` or `aiac.policy.store.library` directly. + +All components are **logically separated modules within a single pod and process** — no inter-service network calls between orchestrators and sub-agents. + +```mermaid +flowchart TD + NATS["Event Broker\nNATS JetStream\naiac.apply.>"] + NATS_CONSUMER["NATS Consumer\nasyncio background task\nthin adapter"] + TRIGGERS["HTTP Triggers\nPOST /apply/*\n(debugging + rebuild)"] + CTRL["Controller\nroutes.py"] + + NATS -->|"durable queue group\naiac-agent-consumer"| NATS_CONSUMER + NATS_CONSUMER -->|"calls internal handler"| CTRL + TRIGGERS --> CTRL + + subgraph CO["Service Onboarding"] + ORC1["Orchestrator"] + SA1["Service Provision"] + SA2["Service Policy Builder"] + ORC1 --> SA1 + ORC1 --> SA2 + end + + subgraph PU["Policy Update"] + SA4["Build"] + SA5["Rebuild"] + SA5 -->|"delegates"| SA4 + end + + subgraph RR["Role Update"] + SA6["Role"] + end + + PRB["Policy Rules Builder (shared)\nagent/policy_rules_builder/"] + PCE["Policy Computation Engine\naiac.policy.computation\ncompute_and_apply(rules)"] + + CTRL -->|"service/:id"| ORC1 + CTRL -->|"build"| SA4 + CTRL -->|"rebuild"| SA5 + CTRL -->|"role/:id"| SA6 + + SA2 -->|"calls"| PRB + SA4 -->|"calls"| PRB + SA6 -->|"calls"| PRB + + ORC1 -->|"list[PolicyRule]"| CTRL + SA4 -->|"list[PolicyRule]"| CTRL + SA5 -->|"list[PolicyRule]"| CTRL + SA6 -->|"list[PolicyRule]"| CTRL + + CTRL -->|"merged rules"| PCE +``` + +--- + +## NATS Consumer + +A thin adapter started as an **asyncio background task** in the FastAPI `lifespan` handler. It subscribes to the `aiac.apply.>` wildcard on the `aiac-events` NATS JetStream stream using the `aiac-agent-consumer` durable queue group. + +### Dispatch table + +| Subject pattern | Internal handler | +|---|---| +| `aiac.apply.service.{id}` | Service Onboarding Orchestrator (UC1) | +| `aiac.apply.role.{id}` | Role Update sub-agent (UC3, via Controller) | +| `aiac.apply.policy.build` | Policy Update Build sub-agent (UC2, via Controller) | + +### Ack contract + +The consumer **awaits** the internal handler before issuing the NATS acknowledgement. On handler success → ack. On handler exception → do not ack; NATS redelivers after `AckWait`. After 5 unacknowledged redeliveries, NATS routes the message to `aiac.apply.dlq`. + +Fire-and-forget (`asyncio.create_task`) is explicitly prohibited — acking before handler completion would break at-least-once guarantees. + +### Failure isolation + +The consumer and the FastAPI HTTP server share the same process. If the Agent pod crashes mid-processing, the in-flight message was never acked and NATS redelivers it to the next pod instance. This prevents the consumer from exhausting retry counts against an unavailable handler (which would occur if they were separate containers). + +### Configuration + +| Variable | Default | Source | +|---|---|---| +| `NATS_URL` | `nats://aiac-event-broker-service:4222` | ConfigMap (`aiac-pdp-config`) | + +--- + +## Controller + +The Controller is a FastAPI routes layer (`controller/routes.py`). Its responsibilities are: + +- Parse the trigger type and entity ID from the request path. +- Dispatch to the Service Onboarding Orchestrator (UC1) or directly to the Policy Update / Role Update sub-agents (UC2, UC3). +- Receive the `list[PolicyRule]` returned by the Orchestrator or sub-agent (already merged by the sub-agent). +- Call `compute_and_apply(merged_rules)` from `aiac.policy.computation` (PCE) once. +- Return a bare HTTP status code to the caller; write summary and debug info to the log. + +No per-use-case business logic, retry handling, or state assembly lives in the Controller. PRB calls are owned by the producing sub-agents; the Controller's shared step is the single PCE call. + +--- + +## Use Cases + +Each use case (and the UC1 Orchestrator) is specified in a dedicated sub-PRD: + +| Use Case | Sub-PRD | Trigger(s) | Notes | +|---|---|---|---| +| Service Onboarding | [aiac-agent/uc1-service-onboarding.md](aiac-agent/uc1-service-onboarding.md) | `aiac.apply.service.{id}`, `POST /apply/service/{id}` | Orchestrator sequences: Service Provision → Service Policy Builder (IdP reader + PRB invoker) | +| Policy Update | [aiac-agent/uc2-policy-update.md](aiac-agent/uc2-policy-update.md) | `aiac.apply.policy.build`, `POST /apply/policy/build`, `POST /apply/policy/rebuild` | | +| Role Update | [aiac-agent/uc3-role-update.md](aiac-agent/uc3-role-update.md) | `aiac.apply.role.{id}`, `POST /apply/role/{id}` | | + +> **Note:** Each producing sub-agent calls the **shared Policy Rules Builder** directly, merges the results, and returns `list[PolicyRule]` to the Controller. The Controller calls `compute_and_apply(merged_rules)` from `aiac.policy.computation` (PCE) once. Policy rule application is fully specified in [policy-computation-engine.md](policy-computation-engine.md). The Policy Rules Builder is specified in [aiac-agent/policy-rules-builder.md](aiac-agent/policy-rules-builder.md). + +### IdP access — library, not service + +Every sub-agent (UC1 Provision + Service Policy Builder, UC2 Build + Rebuild, UC3 Role) performs **all** IdP reads and writes through the **idp-library** API — `aiac.idp.configuration.api.Configuration` — and **never** calls the IdP Configuration **service** (`aiac.idp.service.configuration.*`) or its HTTP endpoints directly. The library owns the HTTP transport, retry/backoff, and Keycloak↔model mapping; sub-agents depend only on its typed `Configuration` methods (e.g. `get_service`, `get_roles`, `get_scopes`, `create_service_role`, `create_service_scope`, `set_service_type`). The shared service-type vocabulary is `aiac.idp.configuration.models.ServiceType` (`Agent`/`Tool`) — the same enum used by `Service.type`. See [library-idp.md](library-idp.md). + +--- + +## Endpoints + +| Method | Path | Orchestrator | Sub-agent | +|---|---|---|---| +| POST | `/apply/policy/build` | Policy Update | Build | +| POST | `/apply/policy/rebuild` | Policy Update | Rebuild | +| POST | `/apply/role/{role_id}` | Role Update | Role | +| POST | `/apply/service/{service_id}` | Service Onboarding | Provision | + +All endpoints return bare HTTP status codes: `200 OK` on success (no response body), and the status codes from the Error Handling table on upstream failure. Success responses carry no body; upstream failures are raised as FastAPI `HTTPException`s, so error responses carry FastAPI's default JSON error body (`{"detail": ...}`) alongside the status code. Summary, applied-rule details, and debug information are written to the service log. Validation failures surface as an error status and log entry; detailed reporting is specified in [policy-rules-builder.md](aiac-agent/policy-rules-builder.md). + +--- + +## Configuration + +| Variable | Default | Source | +|---|---|---| +| `NATS_URL` | `nats://aiac-event-broker-service:4222` | ConfigMap (`aiac-pdp-config`) | +| `AIAC_PDP_CONFIG_URL` | `http://aiac-pdp-config-service:7071` | ConfigMap (`aiac-pdp-config`) — used by `aiac.idp.configuration.api` (in-process via PCE) | +| `AIAC_PDP_POLICY_URL` | `http://aiac-pdp-policy-service:7072` | ConfigMap (`aiac-pdp-config`) — used by `aiac.pdp.policy.library` (in-process via PCE) | +| `AIAC_POLICY_STORE_URL` | `http://aiac-policy-store-service:7074` | ConfigMap (`aiac-pdp-config`) — used by `aiac.policy.store.library` (in-process via PCE) | +| `AIAC_CHROMADB_URL` | `http://aiac-rag-service:8000` | ConfigMap (`aiac-pdp-config`) | +| `KEYCLOAK_REALM` | — | ConfigMap (`aiac-pdp-config`) | +| `LLM_BASE_URL` | — | ConfigMap | +| `LLM_MODEL` | — | ConfigMap | +| `LLM_API_KEY` | — | Kubernetes Secret | +| `AIAC_AC_MODEL` | `RBAC` | ConfigMap (accepted: `RBAC`, `ABAC`, `REBAC`) | +| `CHROMA_N_RESULTS` | `10` | ConfigMap | +| `MAX_CHANGES_PER_RUN` | `50` | ConfigMap | +| `UPSTREAM_MAX_RETRIES` | `3` | ConfigMap | + +ChromaDB collections: `aiac-policies` and `aiac-domain-knowledge`. + +--- + +## Error Handling + +All upstream calls are retried up to `UPSTREAM_MAX_RETRIES` times with exponential backoff (`tenacity`) before propagating the error. The retry primitive is the project-level shared `run_upstream(fn)` helper (`aiac/shared/upstream.py`), which is transport-agnostic: it re-raises the original exception after the final attempt. Retry is applied at the **transport boundary**, not at the agent call sites — inside the idp-library `Configuration` (its `_request` helper), inside the provision MCP helper (`_mcp_tools_list`), and inside the provision Kubernetes seam (`uc/onboarding/provision/kube.py`). Each caller then maps the re-raised failure to the status below (e.g. an IdP/Kubernetes failure → `502`). + +| Upstream | HTTP status on final failure | +|---|---| +| ChromaDB | `503 Service Unavailable` | +| IdP Configuration Service | `502 Bad Gateway` | +| PDP Policy Writer | `502 Bad Gateway` | +| Kubernetes API | `502 Bad Gateway` | +| LLM API | `504 Gateway Timeout` | + +Upstream failures propagate as bare HTTP error responses (see table above), raised as FastAPI `HTTPException`s; the status code is authoritative and error responses carry FastAPI's default JSON error body (`{"detail": ...}`). All failure details are logged. + +--- + +## Runtime + +- Framework: FastAPI with uvicorn +- Bind: `0.0.0.0:7070` +- State: stateless — changes applied immediately, no pending session required +- Base image: `python:3.12-slim` + +--- + +## File Structure + +``` +aiac/src/aiac/ +├── shared/ ← project-level shared: run_upstream (upstream.py) — transport retry primitive +└── agent/ + ├── controller/ + ├── shared/ ← flatten_role (roles.py) + ├── uc/ + │ ├── onboarding/ + │ │ ├── orchestrator.py ← sequences provision → policy_builder, returns list[PolicyRule] + │ │ ├── provision/ ← LLM sub-agent: classify, analyze, write to IdP; kube.py = retrying K8s seam + │ │ └── policy_builder/ ← IdP reader + PRB invoker: read IdP, call PRB, return list[PolicyRule] + │ ├── policy_update/ + │ │ ├── build/ ← calls PRB, returns list[PolicyRule]; TBD internals + │ │ └── rebuild/ ← delegates to Build; TBD internals + │ └── role_update/ ← calls PRB with (role, all_scopes), returns list[PolicyRule] + └── policy_rules_builder/ ← shared; called by Service Policy Builder, Build, and Role sub-agent +``` + +Docker build command (run from repo root): + +```bash +docker build -f aiac/src/aiac/agent/controller/Dockerfile \ + -t aiac-agent:latest \ + aiac/src/ +``` + +--- + +## Dependencies (`requirements.txt`) + +``` +langgraph +langchain-openai +chromadb +tenacity +fastapi +uvicorn[standard] +requests +python-dotenv +kubernetes +nats-py +``` diff --git a/aiac/docs/specs/components/aiac-agent/policy-rules-builder.md b/aiac/docs/specs/components/aiac-agent/policy-rules-builder.md new file mode 100644 index 00000000..523c92a9 --- /dev/null +++ b/aiac/docs/specs/components/aiac-agent/policy-rules-builder.md @@ -0,0 +1,178 @@ +# Sub-PRD: AIAC Agent — Policy Rules Builder + +## Description + +The **Policy Rules Builder** (PRB) is a shared module at `agent/policy_rules_builder/`. It +exposes two module-level functions that producing sub-agents call directly. Each function +internally runs a LangGraph `StateGraph`; callers are decoupled from LangGraph mechanics. The +PRB fetches its own policy context (see **Policy source** below), reasons over it with an LLM, +and emits `list[PolicyRule]` scoped to the input. It does **not** call +`aiac.pdp.policy.library` or `aiac.policy.store.library` directly; only the PCE does. + +--- + +## Entry points + +```python +def build_role_rules(role: Role, scopes: list[Scope]) -> list[PolicyRule]: ... +def build_scope_rules(roles: list[Role], scope: Scope) -> list[PolicyRule]: ... +``` + +**`build_role_rules`** — role-centric: "given this role, which scopes does it get?" +Used for UC3 (Role Update). Called once per role with the full set of scopes relevant to the trigger. + +**`build_scope_rules`** — scope-centric: "given this scope, which roles may access it?" +Used as one of the calls for UC1 (Service Onboarding). See the Controller sub-PRD for the full UC1 dispatch pattern. + +Each call handles exactly **one focal entity** (the singular argument) against a list of +candidate counterparts; the caller (UC handler) does all iteration. + +--- + +## Policy source (two phases) + +Policy context is fetched behind a `PolicySource` seam, so the retrieval mechanism can change +without touching the rest of the graph. + +- **Phase 1 (current):** the entire access-control policy lives in a **single file**; the PRB + reads the whole file into the proposer prompt. No ChromaDB, no domain-knowledge collection. + Located via `AIAC_POLICY_FILE` (default `/etc/aiac/policy.md`), read as UTF-8; a + missing/unreadable file raises. +- **Phase 2 (later issue):** policy **and** domain knowledge live in a ChromaDB vector store; + the PRB does RAG retrieval over the `aiac-policies` and `aiac-domain-knowledge` collections + (query text derived from the focal entity), respecting `CHROMA_N_RESULTS`. This swaps in a + `ChromaPolicySource` at the same seam. + +--- + +## Contract + +| Aspect | Decision | +|---|---| +| Structure | LangGraph `StateGraph` — nodes `fetch → propose → precheck → audit → build`; `audit → propose` retry edge; two typed graphs (role / scope) sharing node helpers | +| Context retrieval | Two-phase via a `PolicySource` seam — Phase 1 whole-file read; Phase 2 ChromaDB RAG (both collections). See **Policy source** | +| Realm parameter | None — inputs are pre-resolved typed objects; the policy source is not realm-scoped | +| Trigger type in state | None — the function name encodes the direction; no routing field in state | +| Output shape | Proposer emits **names** (via `with_structured_output`); the PRB rebuilds `PolicyRule`s from the **typed inputs** filtered by name — never from LLM-produced fields | +| Dedup | PRB generates a full rule set; the PCE's additive merge handles dedup on write | +| LLM call pattern | **Propose → LLM auditor** (2 structured calls). Auditor rejection feeds its reason back into propose (bounded fix-and-retry, `MAX_AUDIT_RETRIES = 3`); raises on exhaustion | +| Empty result | An auditor-**approved** empty selection is a valid `[]` (deny-by-default). Empty proposals are still audited | +| Error contract | Raises on policy-source failure, LLM failure, or audit-budget exhaustion — no silent empty-list returns | + +--- + +## Internal graph design + +Both entry points compile the same node shape (two typed graphs sharing pure node helpers): + +``` +fetch ─► propose ─► precheck ─► audit ─┬─ approved ─► build ─► END + ▲ │ + └───────── retry ────────────┘ (audit feeds its reason back to propose) + │ + rejected & budget exhausted ─► RAISE (inside audit node) +``` + +- **fetch** — `PolicySource.fetch()` → `policy_text` (Phase 1: whole file). +- **propose** — proposer messages (policy + focal + candidates + any `audit_feedback`); + `with_structured_output(Selection)` → selected names + reasoning. +- **precheck** — deterministic: keep only names present in the candidate set (drop hallucinated + names; log drops). No LLM. +- **audit** — auditor messages; `with_structured_output(AuditVerdict)` → `{approved, reason}`. + Approved → continue; rejected → feed the reason back and retry, or raise once + `MAX_AUDIT_RETRIES` is exhausted. Empty proposals are audited too. +- **build** — reconstruct `PolicyRule`s from the typed inputs filtered by the approved names. + +### Structured-output schemas + +```python +class RoleSelection(BaseModel): # build_role_rules (role focal, scope candidates) + granted_scope_names: list[str] + reasoning: str + +class ScopeSelection(BaseModel): # build_scope_rules (scope focal, role candidates) + roles_with_access_names: list[str] + reasoning: str + +class AuditVerdict(BaseModel): + approved: bool + reason: str | None +``` + +The PRB rebuilds rules from the typed inputs, e.g. +`[PolicyRule(role=role, scope=s) for s in scopes if s.name in granted_scope_names]`. + +### State fields + +```python +class _PRBWorking(TypedDict): + policy_text: str + selected_names: list[str] + reasoning: str + approved: bool + audit_feedback: str | None + retry_count: int + rules: list[PolicyRule] + +class RoleRulesState(_PRBWorking): # role: Role; scopes: list[Scope] + ... +class ScopeRulesState(_PRBWorking): # roles: list[Role]; scope: Scope + ... +``` + +### Prompts + +Lean — task framing, the structured-output contract, and two **safety** meta-rules +(**deny-by-default / policy-silence** — grant a pair only if the policy supports it — and +**scope-strictly-to-focal**). On top of those, two shared **mapping** rules (`_MAPPING_RULES`) +govern how evidence becomes a grant: + +- **Capability projection** — a scope names a *set* of operations; any one covered operation + established for a candidate (by the policy or by the focal/candidate descriptions) grants the + whole scope, so partial (e.g. read-only) access still earns it. +- **Relationship scoping** — a policy may state several access relationships over the same + entities; each grant is judged only by evidence about *that* candidate and the focal entity, and + a statement about an entity that is neither the focal nor a candidate (even a same-theme one) is a + different relationship that never counts either way. + +No worked examples or domain heuristics; all substantive reasoning is deferred to the +(user-authored) policy content and the entity descriptions. The **proposer and auditor share the +same rule set** — both make the same grant decision, so a rule on only one side lets the two +diverge (they did: see issue 3.20 *Follow-up: cross-variant convergence*). The auditor adds only +its framing: approve only if every granted pair is policy-supported and nothing unsupported +slipped in. + +### LLM + retries + +`ChatOpenAI(base_url=LLM_BASE_URL, model=LLM_MODEL, api_key=LLM_API_KEY, temperature=0)`. Two +retry layers, kept distinct: + +- **`MAX_AUDIT_RETRIES`** (module constant, default `3`) — the semantic fix-and-retry loop + between audit and propose. +- **`UPSTREAM_MAX_RETRIES`** (env, default `3`) — tenacity (`stop_after_attempt`, exponential + backoff, `reraise=True`) around each LLM call for transport failures. The Phase-1 file read + does **not** retry; it raises directly. + +--- + +## Use-case dispatch + +| Use Case | Caller | Function(s) called | +|---|---|---| +| UC1 — Service Onboarding | Service Policy Builder sub-agent | `build_scope_rules(other_roles, scope)` per agent/tool scope + `build_role_rules(role, other_scopes)` per agent role (agent path only) | +| UC2 — Policy Update (Build) | Build sub-agent | TBD | +| UC3 — Role Update | Role sub-agent | `build_role_rules(role, all_scopes)` — one call | + +--- + +## Configuration + +| Variable | Used for | Phase | +|---|---|---| +| `AIAC_POLICY_FILE` | Path to the whole-file access policy (default `/etc/aiac/policy.md`) | 1 | +| `LLM_BASE_URL`, `LLM_MODEL`, `LLM_API_KEY` | LLM calls | 1 | +| `UPSTREAM_MAX_RETRIES` | Transport retry budget for LLM (and, in Phase 2, ChromaDB) calls (tenacity, default `3`) | 1 | +| `AIAC_CHROMADB_URL` | ChromaDB endpoint | 2 | +| `CHROMA_N_RESULTS` | Number of results per ChromaDB query (default `10`) | 2 | + +`MAX_AUDIT_RETRIES` (default `3`) is a module constant, not an env var. diff --git a/aiac/docs/specs/components/aiac-agent/uc1-service-onboarding.md b/aiac/docs/specs/components/aiac-agent/uc1-service-onboarding.md new file mode 100644 index 00000000..cacc53ca --- /dev/null +++ b/aiac/docs/specs/components/aiac-agent/uc1-service-onboarding.md @@ -0,0 +1,245 @@ +# Component Sub-PRD: UC1 — Service Onboarding + +> **Depends on:** [`../aiac-agent.md`](../aiac-agent.md) — NATS Consumer, Controller, Shared Module, Configuration, Error Handling, Runtime. + +> **IdP access — library, not service.** All IdP reads and writes go through the **idp-library** API (`aiac.idp.configuration.api.Configuration`), **never** the IdP Configuration **service** (`aiac.idp.service.configuration.*`) or its HTTP endpoints directly. See [aiac-agent.md → IdP access](../aiac-agent.md#idp-access--library-not-service). + +## Triggers + +| Source | Subject / Path | +|---|---| +| Event Broker (NATS) | `aiac.apply.service.{id}` (originated by Keycloak SPI `CLIENT_CREATED`) | +| HTTP (debug) | `POST /apply/service/{service_id}` | + +## Architecture overview + +UC1 is the only use case with an Orchestrator, because it is a two-stage pipeline: + +1. **Service Provision** (LLM-based): classify the new service, derive its roles + scopes, write them into the IdP. +2. **Service Policy Builder** (deterministic): read the full IdP role + scope universe (excluding the new service's own entities), call the PRB for each applicable pair, and return a merged `list[PolicyRule]` to the Orchestrator. + +The Orchestrator returns `(list[PolicyRule], override=False)` to the Controller. The Controller calls the PCE with that `override` flag; the PCE owns all rule reconciliation. UC1 is **incremental** — existing roles receive a partial new mapping and must not lose their other access — so the mode is always append (`override=False`). + +```mermaid +flowchart TD + NATS["Event Broker\nNATS JetStream\naiac.apply.service.{id}"] + NATS_CONSUMER["NATS Consumer\nasyncio background task\nthin adapter"] + TRIGGERS["HTTP Triggers\nPOST /apply/service/{service_id}\n(debug)"] + CTRL["Controller\nroutes.py"] + + NATS -->|"durable queue group\naiac-agent-consumer"| NATS_CONSUMER + NATS_CONSUMER -->|"calls internal handler"| CTRL + TRIGGERS --> CTRL + + subgraph CO["Service Onboarding"] + ORC["Orchestrator"] + SA_PROV["Service Provision\n(LLM)"] + SA_POL["Service Policy Builder\n(deterministic)"] + ORC --> SA_PROV + ORC --> SA_POL + end + + PRB["Policy Rules Builder (shared)\nagent/policy_rules_builder/"] + PCE["Policy Computation Engine\naiac.policy.computation\ncompute_and_apply(merged_rules, override)"] + + CTRL -->|"service/:id"| ORC + SA_POL -->|"calls"| PRB + ORC -->|"(list[PolicyRule], override=False)"| CTRL + CTRL -->|"merged rules, override=False"| PCE +``` + +## Orchestrator + +`onboarding/orchestrator.py` + +**Sequence:** +1. Call `ServiceProvisionGraph.invoke()` → get back `ServiceProvision { roles, scopes }` + `service_type`. +2. Call `ServicePolicyBuilder.build(service_id, service_type)` → get back `list[PolicyRule]`. Service Policy Builder re-reads the service's own roles/scopes from the IdP by `service_id` (Provision has already persisted them), so it needs only the id, not the `ServiceProvision`. +3. Return `(list[PolicyRule], override=False)` to the Controller. + +No LLM calls, retry logic, or response assembly in the Orchestrator beyond sequencing. + +**Replay safety (at-least-once delivery):** Service Provision IdP writes are **idempotent** (create-or-get by name: `create_service_role` / `create_service_scope` return the existing entity on a duplicate call). The PCE reconcile is also idempotent. If the pod crashes between Service Provision completing and the PCE call, NATS redelivers and the full pipeline re-runs safely to convergence. There is **no rollback logic**. + +--- + +## Sub-agent: Service Provision + +`onboarding/provision/` + +**Nature:** LLM-based. Classifies the new service (agent or tool), derives roles + scopes from AgentCard / MCP manifest, and **writes them into the IdP**. + +All IdP writes and reads target the **idp-library** — `aiac.idp.configuration.api.Configuration` — not the IdP service directly: +- `create_service_role(service_id, role)` — idempotent (create-or-get by name, then map) +- `create_service_scope(service_id, scope)` — idempotent (create-or-get by name, then map) + +### Graph + +``` +START → classify_service → [analyze_agent | analyze_tool] → provision_service → END +``` + +### Nodes + +- **`classify_service`**: resolves identity + determines service type from the operator's authoritative `kagenti.io/type` label (values `agent`/`tool`) — **not** from the `entity_id` format. + 1. Store `service_id = trigger.entity_id` (Keycloak `client_id`). + 2. Resolve identity: call `get_service(service_id)` from `aiac.idp.configuration.api` → `client.name`, which the kagenti-operator sets to `"{namespace}/{workload_name}"` for every workload (agents and tools, SPIRE-enabled or not). Split on the first `/` → store `namespace` and `workload_name`. `502` if `client.name` has no `/` (namespace unrecoverable). + 3. LIST pods in `namespace`; select the pod owned by `workload_name` via `ownerReferences` (Deployment → ReplicaSet name prefix, or `StatefulSet`/`Sandbox` name match). `502` on Kubernetes API failure or no matching pod. + 4. Read the `kagenti.io/type` label on that pod and normalize it to a `ServiceType` + member via `ServiceType(label.capitalize())` — the label is lowercase + (`agent`/`tool`); `ServiceType` values are capitalized (`Agent`/`Tool`): + - `agent` → `ServiceType.AGENT`; route to `analyze_agent`. + - `tool` → `ServiceType.TOOL`; route to `analyze_tool`. + - Absent or any other value (normalization raises `ValueError`) → `502` (inconsistent deployment). + + > K8s access: `list` on `pods` in the target namespace (both paths). + > `kagenti.io/type` is authoritative — applied by the kagenti-operator (via the AgentRuntime CR) and propagated to pod labels; it is the operator's own agent/tool discriminator (`SkipReason`, kagenti-operator `internal/clientreg/names.go`). The operator only registers a Keycloak client for a workload that already carries this label, so it is effectively guaranteed for operator-registered clients; a missing/invalid value still fails loud (`502`, naming the workload + label). The `entity_id` format (SPIFFE vs plain) reflects whether SPIRE is enabled, **not** the service type, so it is not used for classification. + +- **`analyze_agent`**: non-LLM node; reads AgentCard CR. + 1. LIST `AgentCard` CRs (`agent.kagenti.dev/v1alpha1`) in `namespace`; find the one matching `workload_name`. + 2. **AgentCard found** → produce `ServiceProvision`: + - `roles`: `[RoleDefinition(name=f"{workloadName}.agent", description="Agent role")]` + - `scopes`: `[ScopeDefinition(name=f"{workloadName}.{skill.name}", description=skill.description) for skill in card.skills]` + - `reasoning`: `f"derived from AgentCard: {len(skills)} skills"` + 3. **AgentCard not found** (legacy deployment) → produce minimal `ServiceProvision`: + - `roles`: `[RoleDefinition(name=f"{workloadName}.agent", description="Agent role")]` + - `scopes`: `[ScopeDefinition(name=f"{workloadName}.access", description="Default access scope")]` + - `reasoning`: `"partial: no AgentCard found, default scope assigned"` + + > K8s access: `list` on `agentcards.agent.kagenti.dev` in the target namespace. + +- **`analyze_tool`**: non-LLM node; discovers MCP tools. `namespace` + `workload_name` are already resolved by `classify_service` (from the `client.name` split). MCP endpoint lookup uses the **hybrid Keycloak→K8s strategy** decided in issue `docs/gh-issues/6.2-analyze-tool-lookup-strategy.md`: the Keycloak client name supplied the key `{namespace, workload_name}`; K8s supplies the reachable endpoint. + 1. Locate MCP endpoint: + a. GET the K8s `Service` named `workload_name` in `namespace` (operator convention: Service name == workload name). + b. Require the `protocol.kagenti.io/mcp` label present on that Service; `502` (actionable) if absent — the label is applied at deploy time, not stamped by the operator. + c. Build `http://{workload_name}.{namespace}.svc.cluster.local:{port}/mcp`, where `port` is the Service's first port (not hardcoded). + 2. Call `tools/list` (HTTP POST, MCP protocol) on the resolved endpoint. + 3. Produce `ServiceProvision`: + - `roles`: `[]` (tools do not initiate further calls) + - `scopes`: `[ScopeDefinition(name=f"{workload_name}.{tool.name}", description=tool.description) for tool in manifest.tools]` + - `reasoning`: `f"derived from MCP manifest: {len(tools)} tools"` + 4. Returns `502` on Service/label lookup failure or MCP call failure. + + > K8s access: `get` on `services` in the workload namespace (tool path). Identity is resolved by `classify_service` (config API). + > MCP path convention: all MCP tool services must serve at `/mcp` and carry the `protocol.kagenti.io/mcp` label. This label is a **deploy-time prerequisite** — the kagenti-operator does not stamp it today; automatic stamping is requested upstream (`docs/gh-issues/kagenti-operator-mcp-label-stamping.md`). Until then it must be applied at deploy time; `analyze_tool` fails loud (`502`, naming the workload + missing label) if it is absent. + +- **`provision_service`**: non-LLM node; calls `create_service_role` and `create_service_scope` from `aiac.idp.configuration.api` for each entry in `ServiceProvision`. Reads `service_id` from state. Writes are **idempotent** (create-or-get). + - Also persists the discovered `service_type` onto the Keycloak client via `Configuration.set_service_type(service, service_type)`, which stores it as the **`client.type`** attribute. This is the **authoritative origin** of the attribute that the IdP library's `Service._resolve_keycloak_fields` reads back (see the IdP library spec's type-resolution precedence). No case mapping is needed here: `service_type` is a `ServiceType` (values `Agent`/`Tool`), already matching `client.type` and `Service.type`. Case normalization happens once, upstream, when `classify_service` reads the lowercase `kagenti.io/type` label. + +### State: `OnboardingProvisionState` + +Extends `BaseAgentState` with: + +| Field | Type | Description | +|---|---|---| +| `service_id` | `str \| None` | Keycloak `client_id` = `trigger.entity_id` | +| `namespace` | `str \| None` | From the `client.name` split in `classify_service` (agents and tools) | +| `workload_name` | `str \| None` | From the `client.name` split in `classify_service` (agents and tools) | +| `service_type` | `ServiceType \| None` | `agent` or `tool`; routing field | +| `service_provision` | `ServiceProvision \| None` | Populated by `analyze_agent` or `analyze_tool` | + +### Types + +`ServiceType` is **not** redefined here — it is imported from `aiac.idp.configuration.models` +(the same enum backing `Service.type`), so the sub-agent, the IdP library, and the IdP service +share one vocabulary: + +```python +# aiac.idp.configuration.models — shared, reused by the sub-agent (do not duplicate): +class ServiceType(str, Enum): + AGENT = "Agent" # values capitalized to match the Keycloak client.type attribute + TOOL = "Tool" +``` + +The remaining types are sub-agent–local (in `provision/types.py`). `RoleDefinition` / +`ScopeDefinition` are deliberately distinct from the IdP `Role` / `Scope` models: a derived +role/scope is a pre-persistence *name + description* with no Keycloak `id` yet (idp `Role` +requires `id` + `composite`, `Scope` requires `id`), so it cannot be an idp model until +`provision_service` writes it. + +```python +class RoleDefinition(BaseModel): + name: str + description: str + +class ScopeDefinition(BaseModel): + name: str + description: str + +class ServiceProvision(BaseModel): + roles: list[RoleDefinition] + scopes: list[ScopeDefinition] + reasoning: str # machine-generated provenance string +``` + +--- + +## Sub-agent: Service Policy Builder + +`onboarding/policy_builder/` + +**Nature:** deterministic IdP reader + PRB invoker. + +**Purpose:** given the just-provisioned service's `service_id`, fetch its own roles + scopes from the IdP (`get_service(service_id)`), read the full IdP universe **excluding the new service's own entities**, call the PRB for each applicable (roles, scope) or (role, scopes) pair, and return a merged `list[PolicyRule]` to the Orchestrator. + +**Why `service_id`, not `ServiceProvision`:** own roles/scopes must be id-bearing `Role`/`Scope` — `flatten_role` needs a `Role` (with `childRoles`) and the PRB builds `PolicyRule(role=Role, scope=Scope)`. The Provision-time `RoleDefinition`/`ScopeDefinition` carry only name+description (no Keycloak id), so they cannot be passed to the PRB. Provision has already persisted these entities, so `get_service(service_id).roles` / `.scopes` returns them with ids. + +**Terminology — own vs other (used throughout this section):** +- **Own roles / own scopes** — the roles and scopes the just-provisioned service defines for *itself*, fetched from the IdP by `service_id`: `service.roles` / `service.scopes` (`get_service(service_id)`). These are exactly the entities Service Provision wrote. +- **Other roles / other scopes** — every *pre-existing* role/scope in the IdP universe **minus** the new service's own entities. These belong to other services. + +**Self-mapping invariant (must hold):** the PRB must **never** be handed an *(own role, own scope)* pair — a service's own role must never be mapped to its own scope. Onboarding only ever grants **cross-service** access: *who else* may call this service, and (agents only) *what else* this service may call. A service's own role reaching its own scope is not something onboarding needs to author (that access is intrinsic and out of scope here) and would pollute the policy set. The Service Policy Builder sub-agent guarantees the invariant **by construction** through two complementary guards: + +1. **Exclusion (own entities never appear on the "other" side).** Own roles are removed from `other_roles` and own scopes from `other_scopes` before any PRB call (steps 3–4). Flattening runs *after* exclusion and cannot reintroduce an own role: the just-provisioned roles are brand new and are not yet referenced as `childRoles` by any existing role. +2. **Call direction (each call's "self" side is one own entity of the *opposite* kind).** Each PRB call pairs a single own entity with the other-side universe, never own-with-own, and keeps the semantic intent crisp: + - `build_scope_rules(flattened_other_roles, own_scope)` = *who else may call this skill* (an **own scope** against **other roles**) + - `build_role_rules(own_role, other_scopes)` = *what else may this role call* (an **own role** against **other scopes**; agent path only) + +Neither guard alone is sufficient — exclusion keeps own entities off the other side, and the call direction keeps the self side and the other side of *opposite* kinds (a scope vs roles, or a role vs scopes). Together they make an *(own role, own scope)* pair unrepresentable in any PRB call. + +### Steps + +1. Receive `service_id: str` + `service_type: ServiceType` from the Orchestrator. +2. Fetch the service's **own roles + scopes** from the IdP by `service_id` via `aiac.idp.configuration.api` (`get_service(service_id)` → `service.roles` / `service.scopes`, id-bearing `Role`/`Scope`). +3. Read **all roles** from `aiac.idp.configuration.api`, **excluding** the service's own roles (i.e. exclude `role.name in {r.name for r in service.roles}`). +4. Read **all scopes** from `aiac.idp.configuration.api`, **excluding** the service's own scopes (i.e. exclude `scope.name in {s.name for s in service.scopes}`). +5. **Flatten roles to their closure** before any PRB call, via the shared `flatten_role` helper (see [Composite role flattening](#composite-role-flattening)): expand `other_roles` into the union of every role's closure, de-duplicated by `role.id` (call this `flattened_other_roles`); on the agent path, also expand each of the service's own roles. +6. Call PRB and merge: + - **`service_type = tool`:** call `build_scope_rules(flattened_other_roles, scope)` for each of the service's own scopes. Merge results into a single `list[PolicyRule]`. + - **`service_type = agent`:** call `build_scope_rules(flattened_other_roles, scope)` for each own scope; for each of the service's own roles, call `build_role_rules(r, other_scopes)` **once per role `r` in that role's closure**. Merge all results into a single `list[PolicyRule]`. +7. Return the merged `list[PolicyRule]` to the Orchestrator. (The Orchestrator pairs it with `override=False` for the Controller — see [Architecture overview](#architecture-overview).) + +**Note on "all relevant scopes":** relevance (which of `other_scopes` maps to each `agent_role`) is determined by the PRB, not here. This module always passes the full excluded-self scope universe; the PRB emits only the relevant rule mappings. See [`policy-rules-builder.md`](policy-rules-builder.md). + +### Composite role flattening + +Every role passed to the PRB is first flattened to its **closure** via the shared +`flatten_role` helper (aiac-agent Shared Module): recursively collect the role and all +descendant roles from `role.childRoles` into a flat list, de-duplicated by `role.id` +(`Role` is not hashable, so de-duplication tracks seen `id`s rather than adding `Role` +objects to a `set`). A non-composite role yields a list containing only itself. The PRB +therefore receives already-flattened roles, and the PCE performs no further flattening. + +## File structure + +``` +aiac/src/aiac/agent/uc/ +└── onboarding/ + ├── orchestrator.py + ├── provision/ + │ ├── __init__.py + │ ├── graph.py ← ServiceProvisionGraph (LLM-based StateGraph) + │ ├── nodes.py ← classify_service, analyze_agent, analyze_tool, provision_service + │ ├── state.py ← OnboardingProvisionState + │ └── types.py ← RoleDefinition, ScopeDefinition, ServiceProvision (ServiceType imported from aiac.idp.configuration.models) + └── policy_builder/ + ├── __init__.py + └── builder.py ← ServicePolicyBuilder.build(service_id, service_type) → list[PolicyRule] +``` + +## Out of scope + +- PRB internals — see [`policy-rules-builder.md`](policy-rules-builder.md). +- PCE reconcile mechanics — see [`../policy-computation-engine.md`](../policy-computation-engine.md). +- Response body shape — no success body; handlers return bare HTTP status codes (error responses carry FastAPI's default JSON error body from the raised `HTTPException`). Summary + debug go to the log. +- MCP endpoint lookup strategy for tools — **resolved** (hybrid Keycloak→K8s) in `docs/gh-issues/6.2-analyze-tool-lookup-strategy.md` and reflected in the `analyze_tool` node above. diff --git a/aiac/docs/specs/components/aiac-agent/uc2-policy-update.md b/aiac/docs/specs/components/aiac-agent/uc2-policy-update.md new file mode 100644 index 00000000..7dd923fe --- /dev/null +++ b/aiac/docs/specs/components/aiac-agent/uc2-policy-update.md @@ -0,0 +1,64 @@ +# Component Sub-PRD: UC2 — Policy Update + +> **Status: TBD.** The internal design of the Build and Rebuild sub-agents is not yet defined. A dedicated grill session is required. + +> **Depends on:** [`../aiac-agent.md`](../aiac-agent.md) — NATS Consumer, Controller, Shared Module, Configuration, Error Handling, Runtime. + +> **IdP access — library, not service.** All IdP reads and writes go through the **idp-library** API (`aiac.idp.configuration.api.Configuration`), **never** the IdP Configuration **service** (`aiac.idp.service.configuration.*`) or its HTTP endpoints directly. See [aiac-agent.md → IdP access](../aiac-agent.md#idp-access--library-not-service). + +## Triggers + +| Source | Subject / Path | +|---|---| +| Event Broker (NATS) | `aiac.apply.policy.build` (originated by RAG Ingest Service post-ingest) | +| HTTP (debug / operator) | `POST /apply/policy/build` | +| HTTP (operator only) | `POST /apply/policy/rebuild` (not routed through Event Broker) | + +## Architecture + +```mermaid +flowchart TD + NATS["Event Broker\nNATS JetStream\naiac.apply.policy.build"] + NATS_CONSUMER["NATS Consumer\nasyncio background task\nthin adapter"] + TRIGGERS["HTTP Triggers\nPOST /apply/policy/build\nPOST /apply/policy/rebuild\n(debug / operator)"] + CTRL["Controller\nroutes.py"] + + NATS -->|"durable queue group\naiac-agent-consumer"| NATS_CONSUMER + NATS_CONSUMER -->|"calls internal handler"| CTRL + TRIGGERS --> CTRL + + subgraph PU["Policy Update (TBD)"] + SA_BUILD["Build sub-agent\n(TBD)"] + SA_REBUILD["Rebuild sub-agent\n(TBD)"] + end + + PRB["Policy Rules Builder (shared)\nagent/policy_rules_builder/"] + PCE["Policy Computation Engine\naiac.policy.computation\ncompute_and_apply(merged_rules, override)"] + + SA_REBUILD -->|"delegates"| SA_BUILD + SA_BUILD -->|"calls"| PRB + + CTRL -->|"build"| SA_BUILD + CTRL -->|"rebuild"| SA_REBUILD + SA_BUILD -->|"(list[PolicyRule], override)"| CTRL + SA_REBUILD -->|"(list[PolicyRule], override=True)"| CTRL + CTRL -->|"merged rules, override"| PCE +``` + +## What is known + +- **Two sub-agents:** Build (responds to `aiac.apply.policy.build` + `POST /apply/policy/build`) and Rebuild (responds to `POST /apply/policy/rebuild` only). +- Build calls the PRB directly, merges the results, and returns `(list[PolicyRule], override)` to the Controller. +- **Composite role flattening:** before calling the PRB, Build flattens every role it reads to its **closure** via the shared `flatten_role` helper — the role plus all descendant roles from `role.childRoles`, de-duplicated by `role.id` (a non-composite role yields just itself). The PRB receives already-flattened roles; the PCE performs no flattening. (Same helper and semantics as UC1 and UC3.) +- Rebuild delegates to Build for rule generation and returns Build's rules to the Controller. +- **Append vs override:** the sub-agent conveys an `override` flag to the Controller alongside its rules. **Rebuild is the full-rebuild case (`override=True`)** — the PCE purges every input role's mappings before applying (see [`../policy-computation-engine.md`](../policy-computation-engine.md)). **Build's** `override` value is **TBD** (whether an incremental post-ingest build appends or replaces). +- The Controller calls `compute_and_apply(merged_rules, override)` via the PCE — the same pattern as all other UCs. +- Internal behavior (how Build/Rebuild sub-agents derive their tuple content, what IdP data they read, whether any LLM node is involved) is **deferred** — to be resolved in a dedicated grill session. + +## Out of scope (this stub) + +- Build sub-agent internal design. +- Rebuild sub-agent internal design. +- PRB internals — see [`policy-rules-builder.md`](policy-rules-builder.md). +- PCE reconcile mechanics — see [`../policy-computation-engine.md`](../policy-computation-engine.md). +- Response body shape — no success body; handlers return bare HTTP status codes (error responses carry FastAPI's default JSON error body from the raised `HTTPException`). Summary + debug go to the log. diff --git a/aiac/docs/specs/components/aiac-agent/uc3-role-update.md b/aiac/docs/specs/components/aiac-agent/uc3-role-update.md new file mode 100644 index 00000000..4f8fca80 --- /dev/null +++ b/aiac/docs/specs/components/aiac-agent/uc3-role-update.md @@ -0,0 +1,88 @@ +# Component Sub-PRD: UC3 — Role Update + +> **Depends on:** [`../aiac-agent.md`](../aiac-agent.md) — NATS Consumer, Controller, Shared Module, Configuration, Error Handling, Runtime. + +> **IdP access — library, not service.** All IdP reads and writes go through the **idp-library** API (`aiac.idp.configuration.api.Configuration`), **never** the IdP Configuration **service** (`aiac.idp.service.configuration.*`) or its HTTP endpoints directly. See [aiac-agent.md → IdP access](../aiac-agent.md#idp-access--library-not-service). + +## Triggers + +| Source | Subject / Path | +|---|---| +| Event Broker (NATS) | `aiac.apply.role.{id}` (originated by Keycloak SPI role created/updated) | +| HTTP (debug) | `POST /apply/role/{role_id}` | + +## Architecture + +Single path, no create/update branch. The sub-agent is **deterministic** (non-LLM). + +```mermaid +flowchart TD + NATS["Event Broker\nNATS JetStream\naiac.apply.role.{id}"] + NATS_CONSUMER["NATS Consumer\nasyncio background task\nthin adapter"] + TRIGGERS["HTTP Triggers\nPOST /apply/role/{role_id}\n(debug)"] + CTRL["Controller\nroutes.py"] + + NATS -->|"durable queue group\naiac-agent-consumer"| NATS_CONSUMER + NATS_CONSUMER -->|"calls internal handler"| CTRL + TRIGGERS --> CTRL + + subgraph RR["Role Update"] + SA["Role sub-agent\ndeterministic"] + end + + PRB["Policy Rules Builder (shared)\nagent/policy_rules_builder/"] + PCE["Policy Computation Engine\naiac.policy.computation\ncompute_and_apply(merged_rules, override)"] + + SA -->|"calls"| PRB + + CTRL -->|"role/:id"| SA + SA -->|"(list[PolicyRule], override=True)"| CTRL + CTRL -->|"merged rules, override=True"| PCE +``` + +## Sub-agent: Role sub-agent + +**Nature:** deterministic, non-LLM. Pure IdP reader. + +**Steps:** +1. Read the triggering role (`role_id`) from `aiac.idp.configuration.api`. +2. **Flatten the triggering role to its closure** via the shared `flatten_role` helper (see [Composite role flattening](#composite-role-flattening)): the role itself plus all descendant roles from `role.childRoles`, de-duplicated by `role.id`. A non-composite role yields just itself. +3. Read **all scopes** from `aiac.idp.configuration.api`. +4. Call `build_role_rules(r, all_scopes)` on the PRB **once per role `r` in the closure**, and merge the results into a single `list[PolicyRule]`. +5. Return the merged `list[PolicyRule]` (paired with `override=True` — see [Controller behaviour](#controller-behaviour-for-this-uc)). + +**Output:** `(list[PolicyRule], override=True)`. + +### Composite role flattening + +The triggering role is flattened to its **closure** via the shared `flatten_role` helper +(aiac-agent Shared Module): recursively collect the role and all descendant roles from +`role.childRoles` into a flat list, de-duplicated by `role.id` (`Role` is not hashable, so +de-duplication tracks seen `id`s rather than adding `Role` objects to a `set`). A +non-composite role yields a list containing only itself. `build_role_rules` is then called +once per role in the closure, so the PRB receives already-flattened roles and the PCE +performs no further flattening. + +## Controller behaviour (for this UC) + +1. Receives `(list[PolicyRule], override=True)` from the Role sub-agent (PRB already called and merged internally). +2. Calls `compute_and_apply(rules, override=True)` from `aiac.policy.computation`. + - With `override=True`, the PCE purges every input role's existing mappings (both directions, plus `target_scopes` reconciliation) before applying the fresh rules — an authoritative role-keyed replace. Because the sub-agent submits `build_role_rules(r, all_scopes)` output for the full closure, this replaces the complete mapping of the triggering role and every descendant. See [`../policy-computation-engine.md`](../policy-computation-engine.md). +3. Returns bare HTTP status; writes summary + debug to log. + +## File structure + +``` +aiac/src/aiac/agent/uc/ +└── role_update/ + ├── __init__.py + ├── graph.py ← Role sub-agent StateGraph (deterministic) + ├── nodes.py ← fetch_role, fetch_all_scopes, package_tuple + └── state.py ← RoleUpdateState +``` + +## Out of scope + +- PRB internals — see [`policy-rules-builder.md`](policy-rules-builder.md). +- PCE override (role-keyed replace) mechanics — see [`../policy-computation-engine.md`](../policy-computation-engine.md). +- Response body shape — no success body; handlers return bare HTTP status codes (error responses carry FastAPI's default JSON error body from the raised `HTTPException`). Summary + debug go to the log. diff --git a/aiac/docs/specs/components/event-broker.md b/aiac/docs/specs/components/event-broker.md new file mode 100644 index 00000000..0e3b90c7 --- /dev/null +++ b/aiac/docs/specs/components/event-broker.md @@ -0,0 +1,122 @@ +# Component PRD: Event Broker + +## Description + +A NATS JetStream pod that decouples event producers from the AIAC Agent. Producers (Keycloak SPI listener, RAG Ingest Service) publish lightweight trigger events to named NATS subjects. The AIAC Agent subscribes as a durable competing consumer, guaranteeing at-least-once delivery and automatic replay of unprocessed events after pod restarts. + +The Event Broker is a single-node NATS JetStream instance. It owns no business logic — it is a pure transport layer. All policy decisions, orchestration, and state remain in the AIAC Agent. + +--- + +## Stream Configuration + +| Property | Value | +|---|---| +| Stream name | `aiac-events` | +| Subjects | `aiac.apply.>` | +| Retention policy | `WorkQueuePolicy` — message deleted from stream after acknowledgement | +| Consumer name | `aiac-agent-consumer` | +| Consumer type | Durable push consumer with queue group (competing consumers) | +| Authentication | None — ClusterIP network isolation is the access control mechanism | +| Dead-letter subject | `aiac.apply.dlq` | +| Max delivery attempts | 5 — message routed to DLQ after 5 unacknowledged redeliveries | + +--- + +## Subjects + +| Subject | Publisher | Consumer | Trigger | +|---|---|---|---| +| `aiac.apply.service.{id}` | Keycloak SPI listener | AIAC Agent | Keycloak `CLIENT_CREATED` event | +| `aiac.apply.role.{id}` | Keycloak SPI listener | AIAC Agent | Keycloak role created/updated | +| `aiac.apply.policy.build` | RAG Ingest Service | AIAC Agent | Post-ingest completion (any collection) | +| `aiac.apply.dlq` | NATS JetStream (automatic) | Operator (manual inspection) | Max delivery attempts exceeded | + +**`rebuild` is not routed through the Event Broker.** It is an operator-only command issued directly via `POST /apply/policy/rebuild` on the AIAC Agent using `kubectl port-forward`. + +--- + +## Message Payload + +All messages carry a minimal JSON payload containing only the entity ID: + +```json +{ "id": "" } +``` + +For `aiac.apply.policy.build`, the payload is empty (`{}`). The AIAC Agent pulls all required state from the IdP Configuration Service at processing time — the event payload is a trigger, not a data carrier. + +--- + +## Delivery Guarantees + +- **At-least-once delivery** — NATS redelivers any message not acknowledged within the `AckWait` window. +- **Exactly-one processing** — the Agent subscribes via a queue group (`aiac-agent-consumer`). Only one Agent pod receives each message; other pods in the group are not notified. +- **Replay on restart** — `WorkQueuePolicy` retains all unacknowledged messages. A restarted Agent pod automatically receives pending messages on reconnection. +- **DLQ on repeated failure** — after 5 unacknowledged redeliveries, NATS routes the message to `aiac.apply.dlq` for operator inspection. No message is silently dropped. + +--- + +## Configuration + +| Variable | Default | Source | +|---|---|---| +| `NATS_URL` | `nats://aiac-event-broker-service:4222` | ConfigMap (`aiac-pdp-config`) | + +No authentication credentials are required. The NATS server runs with no-auth configuration. + +--- + +## Runtime + +- Image: `nats:latest` with JetStream enabled (`-js` flag) +- Bind: `0.0.0.0:4222` (NATS client port) +- Kubernetes ClusterIP service: `aiac-event-broker-service:4222` +- Base image: official `nats` Docker image + +--- + +## Kubernetes Manifest + +`aiac/k8s/event-broker-deployment.yaml` — NATS JetStream Pod Deployment + ClusterIP Service. + +--- + +## AIAC Init Container + +A dedicated `aiac-init` init container runs in the **Agent Pod** before the Agent container starts. It orchestrates the AIAC startup sequence: + +1. **Wait for NATS** — poll `aiac-event-broker-service:4222` until TCP connection succeeds. +2. **Wait for IdP Configuration Service** — poll `AIAC_PDP_CONFIG_URL/health` until HTTP 200. +3. **Wait for PDP Policy Writer** — poll `AIAC_PDP_POLICY_URL/health` until HTTP 200. +4. **Wait for RAG Ingest Service** — poll `AIAC_RAG_INGEST_URL/health` until HTTP 200 (confirms ChromaDB in the same RAG pod is also up). +5. **Create NATS JetStream stream** — call `js.add_stream()` idempotently with the `aiac-events` stream configuration. Safe to call on every restart. + +The init container uses `python:3.12-slim` with `nats-py` and `httpx`. It is version-controlled alongside the Agent. All dependency URLs are read from the `aiac-pdp-config` ConfigMap. + +### Init Container Configuration + +| Variable | Source | Resolves to | +|---|---|---| +| `NATS_URL` | ConfigMap (`aiac-pdp-config`) | `nats://aiac-event-broker-service:4222` | +| `AIAC_PDP_CONFIG_URL` | ConfigMap (`aiac-pdp-config`) | `http://aiac-pdp-config-service:7071` | +| `AIAC_PDP_POLICY_URL` | ConfigMap (`aiac-pdp-config`) | `http://aiac-pdp-policy-service:7072` | +| `AIAC_RAG_INGEST_URL` | ConfigMap (`aiac-pdp-config`) | `http://aiac-rag-service:7073` | + +### Init Container Dependencies (`requirements.txt`) + +``` +nats-py +httpx +``` + +--- + +## Testing + +| Target | What to mock | What to assert | +|---|---|---| +| Init container health-check loop | HTTP 4xx then 200 sequence | Exits 0 only after all four dependencies respond healthy | +| Init container stream creation | NATS JetStream `add_stream` call | Called with correct stream name, subjects, and retention policy; idempotent on second call | +| Agent NATS consumer dispatch | NATS message delivery | Correct `/apply/*` handler invoked for each subject pattern; message acked on success; message not acked on handler exception | +| DLQ routing | NATS max redelivery exceeded | Message appears on `aiac.apply.dlq` after 5 failures | diff --git a/aiac/docs/specs/components/idp-configuration-service.md b/aiac/docs/specs/components/idp-configuration-service.md new file mode 100644 index 00000000..fef5db1f --- /dev/null +++ b/aiac/docs/specs/components/idp-configuration-service.md @@ -0,0 +1,159 @@ +# Component PRD: IdP Configuration Service + +## Location +`aiac/src/aiac/idp/service/configuration/keycloak/` + +## Description +A FastAPI web service that proxies Keycloak Admin REST API endpoints. Returns IdP (Keycloak) entity state in generic form for consumption by the AIAC Agent and library clients. Consolidates all Keycloak interactions into a single container. Stateless — no caching. Backed exclusively by Keycloak. + +## Endpoints + +| Method | Path | Keycloak Admin API call | Description | +|--------|------|------------------------|-------------| +| GET | `/subjects` | `GET /admin/realms/{realm}/users` | All subjects (users) in realm; filtered to subjects with a specific role when `role_id` query param is provided | +| GET | `/roles` | `GET /admin/realms/{realm}/roles` (full representation, `brief_representation=False`) | All realm-level roles, including attributes (so the `aiac.managed` marker is visible) | +| GET | `/subjects/{subject_id}/assignments` | `GET /admin/realms/{realm}/users/{subject_id}/role-mappings` | Realm and service permission assignments for a subject | +| GET | `/services` | `GET /admin/realms/{realm}/clients` | All services (clients) | +| GET | `/services/{service_id}` | `GET /admin/realms/{realm}/clients/{service_id}` | Single service by ID | +| POST | `/services/{service_id}/type` | `admin.get_client(service_id)` → `admin.update_client(service_id, {"attributes": {...}})` | Set a service's type via the `client.type` client attribute | +| GET | `/scopes` | `GET /admin/realms/{realm}/client-scopes` | All scopes | +| GET | `/services/{service_id}/roles` | `admin.get_client_service_account_user(service_id)` → `admin.get_realm_roles_of_user(user_id)` | Realm roles assigned to a service's account | +| GET | `/services/{service_id}/scopes` | `admin.get_client_default_client_scopes(service_id)` | Default client scopes assigned to a service | +| GET | `/roles/{role_name}/composites` | `GET /admin/realms/{realm}/roles/{role-name}/composites` | Current composite permissions assigned to a role | +| POST | `/scopes` | `POST /admin/realms/{realm}/client-scopes` | Create realm-level scope | +| POST | `/services/{service_id}/scopes/{scope_id}` | `PUT /admin/realms/{realm}/default-default-client-scopes/{scope_id}` | Assign existing scope as default scope to service | +| POST | `/roles` | `POST /admin/realms/{realm}/roles` | Create realm-level role | +| POST | `/services/{service_id}/roles/{role_id}` | `admin.get_client_service_account_user(service_id)` → `admin.assign_realm_roles(user_id, ...)` | Assign existing realm role to service account | +| GET | `/health` | `admin.get_server_info()` — uses `KEYCLOAK_ADMIN_REALM`; no `?realm=` param | Readiness probe | + +`GET /subjects?role_id={role_id}` (filtered variant): +1. Calls `admin.get_realm_role_by_id(role_id)` to resolve the role name from its ID. +2. Calls `admin.get_realm_role_members(role_name)` (`GET /admin/realms/{realm}/roles/{role-name}/users`) to retrieve users directly assigned to the role. +3. For each returned user, enriches with realm role assignments by calling `GET /subjects/{id}/assignments?realm=` (same enrichment as the unfiltered `GET /subjects` endpoint). +4. Returns `200 OK` with a JSON array of enriched user objects. +5. Returns `[]` (empty array) when no subject holds the role directly. +6. Returns `502 Bad Gateway` with `{"error": ...}` on `KeycloakError`. + +`GET /services/{service_id}`: +1. Calls `admin.get_client(service_id)`. +2. Returns `200 OK` with the client JSON on success. +3. Returns `502 Bad Gateway` with `{"error": ...}` on `KeycloakError`. + +All service reads (`GET /services`, `GET /services/{service_id}`) return the Keycloak client representation **unmodified**, so client `attributes` — including `client.type` — flow through verbatim for the library's generic-model mapping (`Service._resolve_keycloak_fields`) to resolve service type. The Keycloak attribute name is confined to this service (writes) and the library mapping layer (reads); it is never exposed to library callers. + +`POST /services/{service_id}/type`: +Accepts JSON body `{"type": "Agent" | "Tool"}` (rejected with `422` otherwise). It: +1. Calls `admin.get_client(service_id)` and copies its existing `attributes`. +2. Sets the **`client.type`** attribute to the (capitalized, plain-string) type value and calls `admin.update_client(service_id, {"attributes": {...}})`. The existing attributes are merged, not clobbered. +3. Returns `200 OK` with the updated client JSON (re-fetched via `admin.get_client`). +4. Returns `502 Bad Gateway` with `{"error": ...}` on `KeycloakError`. + +`POST /scopes`: +Accepts JSON body `{"name": ..., "description": ...}`. It: +1. Calls `admin.create_client_scope({"name": ..., "description": ..., "protocol": "openid-connect", "attributes": {"aiac.managed": "true"}})` to create the scope at realm level. The `aiac.managed` attribute is the AIAC provisioning marker (client-scope attribute values are plain strings). +2. Returns `201 Created` with the created scope JSON (`{"id": ..., "name": ..., "description": ...}`). +3. Returns `409 Conflict` if a scope with that name already exists. +4. Returns `502 Bad Gateway` with `{"error": ...}` on `KeycloakError`. + +`POST /services/{service_id}/scopes/{scope_id}`: +1. Calls `admin.add_default_default_client_scope(service_id, scope_id)` to assign the scope as a default scope to the service. +2. Returns `201 Created` on success. +3. Returns `409 Conflict` if the scope is already assigned to the service. +4. Returns `502 Bad Gateway` with `{"error": ...}` on `KeycloakError`. + +`POST /roles`: +Accepts JSON body `{"name": ..., "description": ...}`. It: +1. Calls `admin.create_realm_role({"name": ..., "description": ..., "attributes": {"aiac.managed": ["true"]}})` to create the role at realm level. The `aiac.managed` attribute is the AIAC provisioning marker (realm-role attribute values are lists of strings). +2. Returns `201 Created` with the created role JSON (`{"id": ..., "name": ..., "description": ...}`). +3. Returns `409 Conflict` if a role with that name already exists. +4. Returns `502 Bad Gateway` with `{"error": ...}` on `KeycloakError`. + +`GET /services/{service_id}/roles`: +1. Calls `admin.get_client_service_account_user(service_id)` to get the service account user. +2. Extracts `user["id"]` from the result. +3. Calls `admin.get_realm_roles_of_user(user_id)` to return the realm roles assigned to the service account. +4. Returns `200 OK` with a JSON array of realm role objects. +5. Returns `[]` (empty array) if `KeycloakError` has `response_code == 400` (service has no service account — not an error). +6. Returns `502 Bad Gateway` with `{"error": ...}` on other `KeycloakError`. + +`GET /services/{service_id}/scopes`: +1. Calls `admin.get_client_default_client_scopes(service_id)` to return the realm-level client scopes assigned as defaults to the service. +2. Returns `200 OK` with a JSON array of client scope objects. +3. Returns `502 Bad Gateway` with `{"error": ...}` on `KeycloakError`. + +`POST /services/{service_id}/roles/{role_id}`: +1. Calls `admin.get_client_service_account_user(service_id)` to get the service account user. +2. Extracts `user["id"]` from the result. +3. Calls `admin.assign_realm_roles(user_id, [{"id": role_id}])` to assign the realm role to the service account. +4. Returns `201 Created` on success. +5. Returns `409 Conflict` if the role is already assigned. +6. Returns `502 Bad Gateway` with `{"error": ...}` on `KeycloakError`. + +All endpoints except `/health` require a `?realm=` query parameter specifying the Keycloak realm to operate in. Returns `422 Unprocessable Entity` if the parameter is absent. `/health` accepts no realm parameter — it calls `_get_or_create_admin(os.environ["KEYCLOAK_ADMIN_REALM"])` directly. + +All GET endpoints return `200 OK` with a JSON array on success, except `/subjects/{subject_id}/assignments` which returns a JSON object with `realmMappings` and `serviceMappings` fields. All endpoints return `502 Bad Gateway` with a JSON error body if the Keycloak Admin API call fails. + +### AIAC provisioning marker (`aiac.managed`) + +Every role and client scope this service creates is stamped with the Keycloak attribute `aiac.managed` = `true` — the AIAC naming convention that distinguishes AIAC-provisioned entities from Keycloak's own built-ins (default client scopes, the `default-roles-` composite). Attribute value shape differs by entity: realm-role attribute values are lists (`{"aiac.managed": ["true"]}`), client-scope attribute values are plain strings (`{"aiac.managed": "true"}`). Because Keycloak's brief role representation omits attributes, `GET /roles` requests the full representation so the marker survives the read. Downstream consumers (the Policy Computation Engine's P2 embed) filter on this marker to keep only domain entities. + +## Configuration + +Environment variables (injected via Kubernetes Deployment manifest): + +| Variable | Required | Description | +|----------|----------|-------------| +| `KEYCLOAK_URL` | Yes | Keycloak base URL, e.g. `http://keycloak-service.keycloak.svc:8080` | +| `KEYCLOAK_ADMIN_REALM` | Yes | Realm where the admin credentials live, e.g. `master` | +| `KEYCLOAK_ADMIN_USERNAME` | Yes | Admin username (from `keycloak-admin-secret`) | +| `KEYCLOAK_ADMIN_PASSWORD` | Yes | Admin password (from `keycloak-admin-secret`) | + +## Runtime + +- Framework: FastAPI +- Server: uvicorn +- Bind: `0.0.0.0:7071` +- Base image: `python:3.12-slim` +- Kubernetes ClusterIP Service: `aiac-pdp-config-service:7071` +- Deployment: co-located with PDP Policy Writer as a container in the **Kagenti Interface Pod** (`pdp-interface-deployment.yaml`) +- Python library: `aiac.idp.library.configuration` + +## Dependencies (`requirements.txt`) + +``` +fastapi +uvicorn[standard] +python-keycloak +``` + +## File structure + +``` +aiac/src/aiac/idp/service/ +├── __init__.py +└── configuration/ + ├── __init__.py + └── keycloak/ + ├── __init__.py + ├── Dockerfile + ├── requirements.txt + └── main.py +``` + +Build command: +```bash +docker build -f aiac/src/aiac/idp/service/configuration/keycloak/Dockerfile \ + -t aiac-pdp-config:latest aiac/src/ +``` + +## `main.py` behaviour notes + +- Maintain a `dict[str, KeycloakAdmin]` cache keyed by realm name, protected by a `threading.Lock`. +- `get_admin(realm: str = Query(...))` is a FastAPI dependency. On each call it checks the cache; on a miss it acquires the lock, double-checks, and constructs a new `KeycloakAdmin(realm_name=realm, user_realm_name=KEYCLOAK_ADMIN_REALM, ...)`. FastAPI returns `422` automatically if `realm` is absent. +- All endpoints except `/health` declare `admin: KeycloakAdmin = Depends(get_admin)`. `/health` calls `_get_or_create_admin` directly with `os.environ["KEYCLOAK_ADMIN_REALM"]` — no FastAPI dependency, no realm query param. +- Each GET endpoint calls the corresponding `python-keycloak` method and returns the result directly via `JSONResponse`. +- `GET /services/{service_id}/roles`: call `admin.get_client_service_account_user(service_id)` → extract `user["id"]` → call `admin.get_realm_roles_of_user(user_id)`. Returns `[]` if `KeycloakError.response_code == 400` (service has no service account); `502` on other `KeycloakError`. +- `GET /services/{service_id}/scopes`: call `admin.get_client_default_client_scopes(service_id)`. +- `GET /roles/{role_name}/composites`: call `admin.get_composite_realm_roles_of_role(role_name=role_name)`. +- `POST /services/{service_id}/roles/{role_id}`: call `admin.get_client_service_account_user(service_id)` → extract `user["id"]` → call `admin.assign_realm_roles(user_id, [{"id": role_id}])`. +- On `KeycloakError`, return HTTP 502 with `{"error": str(e)}`. diff --git a/aiac/docs/specs/components/keycloak-service.md b/aiac/docs/specs/components/keycloak-service.md new file mode 100644 index 00000000..362be176 --- /dev/null +++ b/aiac/docs/specs/components/keycloak-service.md @@ -0,0 +1,81 @@ +# ~~Component PRD: Keycloak Configuration Service~~ + +> **Superseded.** This component has been replaced by two separate services: +> - **IdP Configuration Service** (read endpoints) — see [idp-configuration-service.md](idp-configuration-service.md) +> - **PDP Policy Writer — Keycloak Implementation** (write endpoints) — see [pdp-policy-keycloak-service.md](pdp-policy-keycloak-service.md) +> +> The content below is retained for reference only. + +## Location +`aiac/src/aiac/keycloak/service/` + +## Description +A FastAPI web service that proxies Keycloak Admin REST API endpoints. Returns raw Keycloak JSON unchanged for read operations; forwards write operations directly. Stateless — no caching. + +## Endpoints + +| Method | Path | Keycloak Admin API call | Description | +|--------|------|------------------------|-------------| +| GET | `/users` | `GET /admin/realms/{realm}/users` | All users in realm | +| GET | `/roles` | `GET /admin/realms/{realm}/roles` | All realm-level roles | +| GET | `/users/{user_id}/role-mappings` | `GET /admin/realms/{realm}/users/{user_id}/role-mappings` | Realm and client role mappings for a user | +| GET | `/clients` | `GET /admin/realms/{realm}/clients` | All clients | +| GET | `/client-scopes` | `GET /admin/realms/{realm}/client-scopes` | All client scopes | +| GET | `/clients/{client_id}/roles` | `GET /admin/realms/{realm}/clients/{client_id}/roles` | Roles defined for a specific client | +| POST | `/users/{user_id}/role-mappings/clients/{client_id}` | `POST /admin/realms/{realm}/users/{user_id}/role-mappings/clients/{client_id}` | Assign client roles to a user | +| DELETE | `/users/{user_id}/role-mappings/clients/{client_id}` | `DELETE /admin/realms/{realm}/users/{user_id}/role-mappings/clients/{client_id}` | Revoke client roles from a user | +| POST | `/clients/{client_id}/roles` | `POST /admin/realms/{realm}/clients/{client_id}/roles` | Create a new role for a specific client | +| POST | `/clients/{client_id}/scopes` | `POST /admin/realms/{realm}/client-scopes` then `PUT /admin/realms/{realm}/clients/{client_id}/default-client-scopes/{scope_id}` | Create a realm-level scope and assign it to a client as a default scope (atomic) | +| DELETE | `/role-mappings` | loop: `GET /admin/realms/{realm}/users` → per user `GET /admin/realms/{realm}/users/{id}/role-mappings` → `DELETE /admin/realms/{realm}/users/{id}/role-mappings/clients/{client_id}` per client | Revoke all client role assignments for all users in the realm | + +Every endpoint accepts an optional `realm` query parameter. When supplied, the request targets the named Keycloak realm instead of the service default (`KEYCLOAK_REALM`); a new `KeycloakAdmin` bound to that realm is instantiated per request. When omitted, the singleton admin initialised at startup is used. + +The GET endpoints return `200 OK` with a JSON array on success, except `/users/{user_id}/role-mappings` which returns a JSON object with `realmMappings` and `clientMappings` fields. The POST and DELETE endpoints for role-mapping operations accept a JSON array of role representation objects in the request body and return `204 No Content` on success. `POST /clients/{client_id}/roles` and `POST /clients/{client_id}/scopes` accept a JSON object with `name` and `description` fields and return `201 Created` with the created resource as JSON. `DELETE /role-mappings` returns `204 No Content` on success. All endpoints return `502 Bad Gateway` with a JSON error body if the Keycloak Admin API call fails. + +## Configuration + +Environment variables (injected via Kubernetes Deployment manifest): + +| Variable | Required | Description | +|----------|----------|-------------| +| `KEYCLOAK_URL` | Yes | Keycloak base URL, e.g. `http://keycloak-service.keycloak.svc:8080` | +| `KEYCLOAK_REALM` | Yes | Realm name, e.g. `kagenti` | +| `KEYCLOAK_ADMIN_USERNAME` | Yes | Admin username (from `keycloak-admin-secret`) | +| `KEYCLOAK_ADMIN_PASSWORD` | Yes | Admin password (from `keycloak-admin-secret`) | + +## Runtime + +- Framework: FastAPI +- Server: uvicorn +- Bind: `0.0.0.0:7070` +- Base image: `python:3.14-slim` + +## Dependencies (`requirements.txt`) + +``` +fastapi +uvicorn[standard] +python-keycloak +``` + +## File structure + +``` +aiac/src/aiac/keycloak/service/ +├── Dockerfile +├── requirements.txt +└── main.py +``` + +## `main.py` behaviour notes + +- Instantiate the default `KeycloakAdmin` once at startup using env vars. +- `get_admin` is a FastAPI dependency accepting `realm: str | None = Query(None)`. When `realm` is `None` it returns the startup singleton; when `realm` is set it returns a new `KeycloakAdmin` for that realm. +- Each endpoint declares `admin: KeycloakAdmin = Depends(get_admin)` — no per-route changes needed for realm routing. +- Each GET endpoint calls the corresponding `python-keycloak` method and returns the result directly via `JSONResponse`. +- POST `/users/{user_id}/role-mappings/clients/{client_id}`: assign the provided roles and return `Response(status_code=204)`. +- DELETE `/users/{user_id}/role-mappings/clients/{client_id}`: revoke the provided roles and return `Response(status_code=204)`. +- POST `/clients/{client_id}/roles`: call `admin.create_client_role(client_id, {"name": ..., "description": ...})`; return `JSONResponse(status_code=201)` with the created role representation. +- POST `/clients/{client_id}/scopes`: call `admin.create_client_scope({"name": ..., "description": ..., "protocol": "openid-connect"})` to create the realm-level scope, then call `admin.add_default_default_client_scope(client_id, scope_id)` to assign it to the client; return `JSONResponse(status_code=201)` with the created scope representation. +- DELETE `/role-mappings`: fetch all users; for each user fetch role mappings; for each client key in `clientMappings` call `admin.delete_client_roles_of_user(user_id, client_id, roles)`; return `Response(status_code=204)` when all revocations complete. +- On `KeycloakError`, return HTTP 502 with `{"error": str(e)}`. diff --git a/aiac/docs/specs/components/library-idp.md b/aiac/docs/specs/components/library-idp.md new file mode 100644 index 00000000..97d5f857 --- /dev/null +++ b/aiac/docs/specs/components/library-idp.md @@ -0,0 +1,300 @@ +# Component PRD: IdP Configuration Library (`aiac.idp.configuration`) + +## Location +`aiac/src/aiac/idp/configuration/` + +## Package structure + +``` +aiac/src/aiac/idp/ +└── configuration/ + ├── __init__.py # empty + ├── models.py # Subject, Role, Service, Scope + └── api.py # Configuration class — reads + writes IdP entities +``` + +All `__init__.py` files are empty. Callers use explicit submodule paths: + +```python +from aiac.idp.configuration.models import Subject, Role, Scope, Service +from aiac.idp.configuration.api import Configuration +``` + +--- + +## Submodule: `aiac.idp.configuration.models` + +### Description +Dependency-free Pydantic `BaseModel` subclasses representing generic IdP configuration entities (subjects, roles, services, scopes). No HTTP client dependency — importable by any consumer without pulling in `requests` or `python-dotenv`. Model shapes are derived from Keycloak JSON but named generically. + +### Dependencies +``` +pydantic +``` + +### Pydantic models + +All models use `model_config = ConfigDict(extra='ignore')` to silently discard unknown fields. + +Model definition order: `Subject` → `Role` → `Service` → `Scope`. Because `Subject`, `Role`, and `Service` reference `Scope` (and `Subject` references `Role`) as forward references, the module calls `Subject.model_rebuild()`, `Role.model_rebuild()`, and `Service.model_rebuild()` after `Scope` is defined. + +`Service`, `Role`, `Scope`, and `Subject` use pydantic's default equality (field-based) and are **not hashable** — they define no custom `__hash__`/`__eq__` and are never used as dict keys or set members. The relationship maps in `AgentPolicyModel` (`source_roles`, `subject_roles`, `target_scopes`) are keyed by the entity's string `id` instead, so no identity override is needed. + +#### `Subject` + +Represents a user (Keycloak: `user`). + +| Field | Type | Keycloak field | Default | +|-------|------|----------------|---------| +| `id` | `str` | `id` | | +| `username` | `str` | `username` | | +| `email` | `str \| None` | `email` | | +| `firstName` | `str \| None` | `firstName` | | +| `lastName` | `str \| None` | `lastName` | | +| `enabled` | `bool` | `enabled` | | +| `roles` | `list[Role]` | _(populated by `Configuration.get_subjects()` from `GET /subjects/{id}/assignments` → `realmMappings`; not present in the raw Keycloak user object)_ | `[]` | + +#### `Role` + +Represents a role (Keycloak: realm role). + +| Field | Type | Keycloak field | Default | +|-------|------|----------------|---------| +| `id` | `str` | `id` | | +| `name` | `str` | `name` | | +| `description` | `str \| None` | `description` | | +| `composite` | `bool` | `composite` | | +| `childRoles` | `list[Role]` | `composites.realm` | `[]` | +| `attributes` | `dict[str, Any]` | `attributes` | `{}` | + +Roles also expose an `aiac_managed` property (`bool`): `True` when `attributes` carries the AIAC provisioning marker `aiac.managed` (realm-role attribute values are lists, so the marker appears as `["true"]`). See the naming convention in the idp-configuration-service spec. + +#### `Service` + +Represents a service (Keycloak: `client`). + +| Field | Type | Keycloak field | Default | +|-------|------|----------------|---------| +| `id` | `str` | `id` | | +| `serviceId` | `str \| None` | `clientId` | `None` | +| `name` | `str \| None` | `name` | | +| `description` | `str \| None` | `description` | `None` | +| `enabled` | `bool` | `enabled` | | +| `type` | `ServiceType \| None` | `attributes.client.type` | `None` | +| `roles` | `list[Role]` | _(roles for this client)_ | `[]` | +| `scopes` | `list[Scope]` | _(default client scopes)_ | `[]` | + +**Service type resolution** (`Service._resolve_keycloak_fields`, a `model_validator(mode="before")`). AIAC calls the concept "service type" everywhere; the backing Keycloak client attribute is named **`client.type`**. Resolution precedence: + +1. An explicit `type` already present on the input wins (never overridden). +2. Otherwise the Keycloak client attribute **`client.type`** ∈ {`Agent`, `Tool`} — a **plain string**. Client attribute values are plain strings; a **list** value (e.g. `["Agent"]`, the shape realm-role attributes use) fails the check and resolves to `None`. Capitalization matches the `ServiceType` values. +3. Otherwise `None`. + +The attribute is set via `Configuration.set_service_type` (below); its authoritative origin is UC1 Service Onboarding — `classify_service` **discovers** the type from the operator's `kagenti.io/type` label and `provision_service` **persists** it onto the client via `set_service_type` (see the aiac-agent UC1 spec). There is **no** `spiffe://` clientId fallback and **no** description-keyword inference — typing is `client.type`-attribute-only. (The former `spiffe:// ⇒ Agent` fallback was **removed**: a `spiffe://` clientId indicates a SPIRE-enabled workload, **not** necessarily an agent — it could mis-type a SPIRE-enabled tool — so clients without a `client.type` attribute now resolve to `None`.) + +> **`ServiceType`** (`aiac.idp.configuration.models`) is a `str` enum — `AGENT = "Agent"`, `TOOL = "Tool"` — shared by `Service.type`, `set_service_type`, and the aiac-agent sub-agents (one vocabulary, no duplication). Values are capitalized to match the `client.type` attribute; because it subclasses `str`, `ServiceType.AGENT == "Agent"`, so it is a drop-in for the former `Literal["Agent", "Tool"]`. The operator's lowercase `kagenti.io/type` pod label is normalized to a member via `ServiceType(label.capitalize())` in UC1 `classify_service`. + +#### `Scope` + +Represents a service scope (Keycloak: `client scope`). + +| Field | Type | Keycloak field | +|-------|------|----------------| +| `id` | `str` | `id` | +| `name` | `str` | `name` | +| `description` | `str \| None` | `description` | +| `attributes` | `dict[str, Any]` | `attributes` | + +Scopes also expose an `aiac_managed` property (`bool`): `True` when `attributes` carries the AIAC provisioning marker `aiac.managed` (client-scope attribute values are plain strings, so the marker appears as `"true"`). See the naming convention in the idp-configuration-service spec. + +### Usage + +```python +from aiac.idp.configuration.models import Subject, Role, Scope, Service + +raw = tool_result["content"] # raw JSON list +subjects = [Subject.model_validate(s) for s in raw] +``` + +--- + +## Submodule: `aiac.idp.configuration.api` + +### Description +HTTP client library that wraps the IdP Configuration Service REST API. Provides read and write access to IdP configuration entities (subjects, roles, services, scopes) and returns typed Pydantic model instances from `aiac.idp.configuration.models`. + +All Keycloak interactions are consolidated here; the PDP Policy Writer (OPA) does not touch Keycloak directly. + +**Transport retries.** Every HTTP call is issued through a private `_request(method, path, **kwargs)` helper that wraps the request in the project-level `run_upstream` retry primitive (`aiac.shared.upstream`): transient failures are retried up to `UPSTREAM_MAX_RETRIES` times (default `3`) with exponential backoff before a non-2xx status is raised as `RuntimeError`. Retry lives inside the library (not in callers), and applies at the leaf request, so composite methods (`create_service_role` / `create_service_scope`) retry each sub-request without compounding. + +### Dependencies +``` +requests +pydantic +python-dotenv +tenacity +``` + +### Class: `Configuration` + +Stateful client bound to a single realm. Construct via the factory method or directly. + +```python +class Configuration: + def __init__(self, realm: str) -> None: ... + + @classmethod + def for_realm(cls, realm: str) -> "Configuration": ... + + def get_subjects(self) -> list[Subject]: ... + def get_roles(self) -> list[Role]: ... + def get_services(self) -> list[Service]: ... + def get_service(self, service_id: str) -> Service: ... + def get_scopes(self) -> list[Scope]: ... + + def get_services_by_role(self, role: Role) -> list[Service]: ... + def get_services_by_scope(self, scope: Scope) -> list[Service]: ... + def get_subjects_by_role(self, role: Role) -> list[Subject]: ... + + def create_scope(self, scope_name: str, scope_description: str) -> Scope: ... + def map_scope_to_service(self, service: Service, scope: Scope) -> Service: ... + + def create_role(self, role_name: str, role_description: str) -> Role: ... + def map_role_to_service(self, service: Service, role: Role) -> Service: ... + + # Idempotent create-or-get (by name) + map to the service. Accept any object exposing + # .name / .description (e.g. the aiac-agent RoleDefinition / ScopeDefinition), so the + # library never imports the agent layer. + def create_service_role(self, service_id: str, role) -> Role: ... + def create_service_scope(self, service_id: str, scope) -> Scope: ... + + def set_service_type(self, service: Service, service_type: ServiceType) -> Service: ... +``` + +`get_scopes()` — simple read: +1. Issue `GET {AIAC_PDP_CONFIG_URL}/scopes`, always appending `?realm=`. +2. Raise `RuntimeError` on non-2xx HTTP status. +3. Parse the response into `list[Scope]` and return. + +`get_subjects()` — enriched with per-subject realm role assignments: +1. `GET {AIAC_PDP_CONFIG_URL}/subjects?realm=` — fetch the base user list. Keycloak does not include role assignments in the user representation. +2. Call `_all_roles_map()` once to build a `{id: Role}` lookup (fully hydrated via `get_roles()`). +3. For each subject, delegate to `_build_subject(raw, all_roles)` which issues `GET /subjects/{id}/assignments?realm=`, extracts `realmMappings` role IDs, filters the roles map, and returns a validated `Subject` with `roles` populated. +4. Raise `RuntimeError` on any non-2xx HTTP status (primary or secondary calls). + +`get_services()` — fully-enriched read: +1. `GET {AIAC_PDP_CONFIG_URL}/services?realm=` — fetch the base service list. +2. Call `get_roles()` and `get_scopes()` once upfront to build `{id: Role}` and `{id: Scope}` lookup maps. +3. For each service, delegate to `_build_service(raw, all_roles, all_scopes)` which issues: + - `GET /services/{id}/roles?realm=` → filter `all_roles` map → `Service.roles` + - `GET /services/{id}/scopes?realm=` → filter `all_scopes` map → `Service.scopes` +4. Raise `RuntimeError` on any non-2xx response. +5. Return `list[Service]` with fully-enriched `roles` (including `childRoles`) and `scopes` (including `description`). + +> **Performance note:** `get_services()` issues 2N + 1 + (roles overhead) HTTP requests where N is the number of services. `get_roles()` is called once and its fully-enriched objects are shared across all services. If this becomes a bottleneck, enrichment should be moved server-side. + +`get_service(service_id)` — fetch a single service with the same full enrichment: +1. `GET {AIAC_PDP_CONFIG_URL}/services/{service_id}?realm=` — fetch the single service. +2. Call `get_roles()` and `get_scopes()` to build lookup maps (same as `get_services()`). +3. Delegate to `_build_service(raw, all_roles, all_scopes)`. +4. Raise `RuntimeError` on any non-2xx response. +5. Return a single enriched `Service`. + +> **Note:** Callers that previously called `get_services()` and filtered by ID should be switched to `get_service(service_id)` to avoid fetching the full list. + +`get_roles()` — enriched read: +1. `GET {AIAC_PDP_CONFIG_URL}/roles?realm=` — fetch all realm roles. +2. For each role, if `role.composite` is `True`: `GET /roles/{name}/composites?realm=` → `Role.childRoles` +3. Raise `RuntimeError` on any non-2xx response. +4. Return `list[Role]` with `childRoles` populated. + +`get_services_by_role(role: Role) -> list[Service]`: +1. Fetches the fully-enriched service list via `get_services()` and filters it **client-side**: returns those services whose `.roles` contains a role with `role.id`. The server `GET /services` endpoint has no `role_id` filter, so filtering happens in the library. +2. Returns an empty list when no service owns the role (e.g. a realm-level role). +3. Raises `RuntimeError` on any underlying non-2xx (propagated from `get_services()` / `_build_service()`). + +`get_services_by_scope(scope: Scope) -> list[Service]`: +1. Fetches the fully-enriched service list via `get_services()` and filters it **client-side**: returns those services whose `.scopes` contains a scope with `scope.id`. The server `GET /services` endpoint has no `scope_id` filter, so filtering happens in the library. +2. Returns an empty list when no service exposes the scope. +3. Raises `RuntimeError` on any underlying non-2xx (propagated from `get_services()` / `_build_service()`). + +> **Performance note:** because both methods delegate to `get_services()`, each call inherits its full fan-out cost (see the `get_services()` performance note above — `2N + 1 + roles` HTTP requests for N services). Acceptable for the low-frequency PCE resolution path. If it becomes a bottleneck, the right fix is a real server-side `role_id` / `scope_id` filter on `GET /services`. + +`get_subjects_by_role(role: Role) -> list[Subject]`: +1. `GET {AIAC_PDP_CONFIG_URL}/subjects?role_id={role.id}&realm=` +2. Returns all subjects (users) that have this role directly assigned, enriched with their full realm role assignments (same enrichment as `get_subjects()`). +3. Raises `RuntimeError` on non-2xx. Returns an empty list when no subject holds the role. + +> **Note:** This method returns only subjects with a **direct** assignment of the given role. Composite role traversal (resolving `childRoles` and querying each) is the caller's responsibility — see PCE algorithm in `aiac.policy.computation`. + +`create_scope`: +1. Issues `POST {AIAC_PDP_CONFIG_URL}/scopes` with body `{"name": scope_name, "description": scope_description}`, appending `?realm=`. +2. Raises `RuntimeError` on non-2xx HTTP status (including 409 if a scope with that name already exists). +3. Returns the created `Scope` instance parsed from the response. + +`map_scope_to_service`: +1. Issues `POST {AIAC_PDP_CONFIG_URL}/services/{service.id}/scopes/{scope.id}`, appending `?realm=`. +2. Raises `RuntimeError` on non-2xx HTTP status (including 409 if the scope is already mapped to the service). +3. Re-fetches the service via `GET {AIAC_PDP_CONFIG_URL}/services/{service.id}`, appending `?realm=`. +4. Returns the updated `Service` instance parsed from the response. + +`create_role`: +1. Issues `POST {AIAC_PDP_CONFIG_URL}/roles` with body `{"name": role_name, "description": role_description}`, appending `?realm=`. +2. Raises `RuntimeError` on non-2xx HTTP status (including 409 if a role with that name already exists). +3. Returns the created `Role` instance parsed from the response. + +`map_role_to_service`: +1. Issues `POST {AIAC_PDP_CONFIG_URL}/services/{service.id}/roles/{role.id}`, appending `?realm=`. +2. Raises `RuntimeError` on non-2xx HTTP status (including 409 if the role is already mapped to the service). +3. Re-fetches the service via `GET {AIAC_PDP_CONFIG_URL}/services/{service.id}`, appending `?realm=`. +4. Returns the updated `Service` instance parsed from the response. + +`create_service_role(service_id: str, role) -> Role`: idempotent create-or-get + map. +1. `get_roles()` and reuse an existing realm role whose `name == role.name`; otherwise `create_role(role.name, role.description)`. +2. `map_role_to_service(get_service(service_id), resolved_role)` (itself idempotent). +3. Raises `RuntimeError` on any underlying non-2xx HTTP status. Returns the resolved `Role`. +4. `role` is any object exposing `.name` / `.description` (e.g. the aiac-agent `RoleDefinition`); the library does not import the agent layer. + +`create_service_scope(service_id: str, scope) -> Scope`: idempotent create-or-get + map. +1. `get_scopes()` and reuse an existing client scope whose `name == scope.name`; otherwise `create_scope(scope.name, scope.description)`. +2. `map_scope_to_service(get_service(service_id), resolved_scope)` (itself idempotent). +3. Raises `RuntimeError` on any underlying non-2xx HTTP status. Returns the resolved `Scope`. +4. `scope` is any object exposing `.name` / `.description` (e.g. the aiac-agent `ScopeDefinition`). + +`set_service_type(service: Service, service_type: ServiceType) -> Service`: +1. Issues `POST {AIAC_PDP_CONFIG_URL}/services/{service.id}/type` with body `{"type": }` (the `ServiceType`'s `Agent`/`Tool` value; a bare `"Agent"`/`"Tool"` string is accepted too since `ServiceType` is a `str` enum), appending `?realm=`. +2. The service persists the value onto the Keycloak client as the **`client.type`** attribute (a plain string, capitalized). The Keycloak attribute name is an IdP-Service/mapping-layer detail — callers pass the generic `service_type` and never see it. +3. Raises `RuntimeError` on non-2xx HTTP status. +4. Returns the updated `Service` instance parsed from the response (`type` now resolved from the new attribute). + +### Configuration + +Read from a `.env` file co-located with `api.py` (`aiac/src/aiac/idp/configuration/.env`) via `python-dotenv`. Falls back to the default if the file is absent or the key is not set. + +| Variable | Default | +|----------|---------| +| `AIAC_PDP_CONFIG_URL` | `http://127.0.0.1:7071` | + +> **TBD:** whether `AIAC_PDP_CONFIG_URL` should be renamed to `AIAC_IDP_CONFIG_URL`. Not yet decided — keep `AIAC_PDP_CONFIG_URL` until this is resolved. + +### Usage + +```python +from aiac.idp.configuration.api import Configuration + +cfg = Configuration.for_realm("kagenti") +subjects = cfg.get_subjects() +for s in subjects: + print(s.username, s.email) + +scope = cfg.create_scope(scope_name="read", scope_description="Read access") +service = cfg.get_service("abc123") # preferred over get_services() + filter +updated_service = cfg.map_scope_to_service(service, scope) + +role = cfg.create_role(role_name="reader", role_description="Read-only access") +updated_service = cfg.map_role_to_service(updated_service, role) + +# PCE usage — resolve services owning a given role or scope +services_with_role = cfg.get_services_by_role(role) +services_with_scope = cfg.get_services_by_scope(scope) +``` diff --git a/aiac/docs/specs/components/library-pdp-policy.md b/aiac/docs/specs/components/library-pdp-policy.md new file mode 100644 index 00000000..e6a2ba77 --- /dev/null +++ b/aiac/docs/specs/components/library-pdp-policy.md @@ -0,0 +1,112 @@ +# Component PRD: PDP Policy Library (`aiac.pdp.policy.library`) + +HTTP client module wrapping the PDP Policy Writer (OPA) REST API. These modules have no dependency on Keycloak — all IdP operations use `aiac.idp.configuration`. + +## Location +`aiac/src/aiac/pdp/policy/library/` + +## Package structure + +``` +aiac/src/aiac/pdp/policy/ +└── library/ + ├── __init__.py # empty + └── api.py # apply_policy, apply_agent_policy, delete_agent_policy, delete_policy +``` + +All `__init__.py` files are empty. Callers use explicit submodule paths: + +```python +from aiac.pdp.policy.library.api import apply_policy, apply_agent_policy, delete_agent_policy, delete_policy +from aiac.policy.model.models import PolicyModel, AgentPolicyModel +``` + +--- + +## Submodule: `aiac.pdp.policy.library.api` + +### Description +HTTP client module wrapping the PDP Policy Writer REST API. Exposes four module-level functions. Service URL is read from the `AIAC_PDP_POLICY_URL` environment variable (default: `http://127.0.0.1:7072`). All functions raise `RuntimeError` on non-2xx response. + +No `realm` parameter — the PDP Policy Writer operates on a Kubernetes CR, not a Keycloak realm. + +**Primary consumer:** `aiac.policy.computation` — the Policy Computation Engine is the only caller. AIAC Agent sub-UC agents do not call this library directly; they call `compute_and_apply` instead. + +### Dependencies +``` +requests +pydantic +python-dotenv +``` + +### Functions + +```python +def apply_policy(model: PolicyModel) -> None + # POST /policy — upsert Rego packages for all agents in the partial model + +def apply_agent_policy(agent_id: str, model: AgentPolicyModel) -> None + # POST /policy/agents/{agent_id} — upsert Rego packages for a single agent + +def delete_agent_policy(agent_id: str) -> None + # DELETE /policy/agents/{agent_id} — remove all Rego packages for agent (off-boarding) + +def delete_policy() -> None + # DELETE /policy — clear all Rego packages (rebuild pre-step) +``` + +### Configuration + +Read from `AIAC_PDP_POLICY_URL` environment variable (or `.env` file co-located with `api.py`). Falls back to the default if absent. + +| Variable | Default | +|----------|---------| +| `AIAC_PDP_POLICY_URL` | `http://127.0.0.1:7072` | + +### Usage + +```python +from aiac.pdp.policy.library.api import apply_policy, apply_agent_policy, delete_agent_policy, delete_policy +from aiac.policy.model.models import PolicyModel, AgentPolicyModel + +# Single-agent update (called by Policy Computation Engine) +apply_agent_policy("weather-agent", agent_model) + +# Full rebuild pre-step: clear all, then reapply +delete_policy() +apply_policy(full_model) + +# Off-boarding +delete_agent_policy("weather-agent") +``` + +--- + +## Testing Decisions + +**Seam:** HTTP boundary — mock responses from `AIAC_PDP_POLICY_URL`. + +**Prior art:** `3.14-unit-tests-write-api.md` (mock PDP Policy Writer HTTP; cover module-level functions). + +Key behaviors to assert: +- `apply_policy(model)` issues `POST /policy` with serialized `PolicyModel`. +- `apply_agent_policy(id, model)` issues `POST /policy/agents/{id}` with serialized `AgentPolicyModel`. +- `delete_agent_policy(id)` issues `DELETE /policy/agents/{id}`. +- `delete_policy()` issues `DELETE /policy`. +- Any non-2xx response raises `RuntimeError`. +- `AIAC_PDP_POLICY_URL` is read from env; falls back to `http://127.0.0.1:7072`. + +--- + +## Out of Scope + +- **Keycloak interaction:** this library never calls Keycloak directly. All IdP operations go through `aiac.idp.configuration`. +- **Policy computation:** translating `list[PolicyRule]` into `AgentPolicyModel` objects is the responsibility of `aiac.policy.computation`, not this library. +- **Policy persistence:** the Policy Store (`aiac.policy.store`) owns structured `AgentPolicyModel` durability. This library targets the OPA runtime only. + +--- + +## Further Notes + +- The `aiac.pdp.library.policy` module (old path) is deprecated. All consumers must update imports to `aiac.pdp.policy.library.api`. +- Models (`PolicyModel`, `AgentPolicyModel`) are imported from `aiac.policy.model.models`, not from the deprecated `aiac.pdp.library.models`. diff --git a/aiac/docs/specs/components/library-policy-store.md b/aiac/docs/specs/components/library-policy-store.md new file mode 100644 index 00000000..ad42598b --- /dev/null +++ b/aiac/docs/specs/components/library-policy-store.md @@ -0,0 +1,112 @@ +# Component PRD: Policy Store Library (`aiac.policy.store.library`) + +Companion library for the [AIAC Policy Store](policy-store.md). Follows the same pattern as `aiac.pdp.policy.library` — module-level functions, URL from env var via `python-dotenv`, `RuntimeError` on non-2xx. + +## Location +`aiac/src/aiac/policy/store/library/` + +## Package structure + +``` +aiac/src/aiac/policy/store/ +└── library/ + ├── __init__.py # empty + └── api.py # six module-level functions +``` + +All `__init__.py` files are empty. Callers use explicit submodule paths: + +```python +from aiac.policy.store.library.api import ( + get_policy, get_agent_policy, + apply_policy, apply_agent_policy, + delete_agent_policy, delete_policy, +) +from aiac.policy.model.models import PolicyModel, AgentPolicyModel +``` + +--- + +## Submodule: `aiac.policy.store.library.api` + +### Description +HTTP client module wrapping the [AIAC Policy Store](policy-store.md) REST API. Exposes six module-level functions returning `PolicyModel` and `AgentPolicyModel` objects directly — no Kubernetes client boilerplate. Service URL is read from the `AIAC_POLICY_STORE_URL` environment variable (default: `http://127.0.0.1:7074`). All functions raise `RuntimeError` on non-2xx response. + +### Dependencies +``` +requests +pydantic +python-dotenv +``` + +### Functions + +```python +def get_policy() -> PolicyModel + # GET /policy + +def get_agent_policy(agent_id: str) -> AgentPolicyModel + # GET /policy/agents/{agent_id} + +def apply_policy(model: PolicyModel) -> None + # POST /policy + +def apply_agent_policy(agent_id: str, model: AgentPolicyModel) -> None + # POST /policy/agents/{agent_id} + +def delete_agent_policy(agent_id: str) -> None + # DELETE /policy/agents/{agent_id} + +def delete_policy() -> None + # DELETE /policy +``` + +### Configuration + +Read from `AIAC_POLICY_STORE_URL` environment variable (or `.env` file co-located with `api.py`). Falls back to the default if absent. + +| Variable | Default | +|----------|---------| +| `AIAC_POLICY_STORE_URL` | `http://127.0.0.1:7074` | + +### Usage + +```python +from aiac.policy.store.library.api import ( + get_policy, get_agent_policy, + apply_policy, apply_agent_policy, + delete_agent_policy, delete_policy, +) +from aiac.policy.model.models import PolicyModel, AgentPolicyModel + +# Read current state for additive merge +current = get_agent_policy("weather-agent") + +# Write updated state +apply_agent_policy("weather-agent", updated_model) + +# Full rebuild +delete_policy() +apply_policy(full_model) + +# Off-boarding +delete_agent_policy("weather-agent") +``` + +--- + +## Testing Decisions + +**Seam:** HTTP boundary — mock responses from `AIAC_POLICY_STORE_URL`. + +**Prior art:** `3.14-unit-tests-write-api.md` (mock PDP Policy Writer HTTP; cover module-level functions). + +Key behaviors to assert: +- `get_policy()` issues `GET /policy`; response body deserialized to `PolicyModel`. +- `get_agent_policy(id)` issues `GET /policy/agents/{id}`; response body deserialized to `AgentPolicyModel`. +- `apply_policy(model)` issues `POST /policy` with serialized `PolicyModel`. +- `apply_agent_policy(id, model)` issues `POST /policy/agents/{id}` with serialized `AgentPolicyModel`. +- `delete_agent_policy(id)` issues `DELETE /policy/agents/{id}`. +- `delete_policy()` issues `DELETE /policy`. +- Any non-2xx response raises `RuntimeError`. +- `AIAC_POLICY_STORE_URL` is read from env; falls back to `http://127.0.0.1:7074`. diff --git a/aiac/docs/specs/components/pdp-policy-keycloak-service.md b/aiac/docs/specs/components/pdp-policy-keycloak-service.md new file mode 100644 index 00000000..afa72a7b --- /dev/null +++ b/aiac/docs/specs/components/pdp-policy-keycloak-service.md @@ -0,0 +1,80 @@ +# Component PRD: PDP Policy Writer — Keycloak Implementation + +## Location +`aiac/src/aiac/pdp/service/policy/keycloak/` + +## Description +A FastAPI web service that applies RBAC policy changes to Keycloak by managing composite role mappings. Roles are made composites of service (client) permissions (roles), so that any subject (user) assigned a role automatically inherits the associated service permissions. Stateless — no caching. + +This is the **Phase 1** implementation of the PDP Policy Writer. It is deployed as a container in the **Kagenti Interface Pod** alongside the IdP Configuration Service, behind the `aiac-pdp-policy-service:7072` ClusterIP. Phase 2 replaces only this container image (`aiac-pdp-policy-keycloak` → `aiac-pdp-policy-opa`) within the same pod. The service name and port remain stable so the AIAC Agent and library require no reconfiguration. + +## Endpoints + +| Method | Path | Keycloak Admin API call | Description | +|--------|------|------------------------|-------------| +| POST | `/roles/{role_name}/composites` | `POST /admin/realms/{realm}/roles/{role-name}/composites` | Add service permissions (client roles) to a role composite | +| DELETE | `/roles/{role_name}/composites` | `DELETE /admin/realms/{realm}/roles/{role-name}/composites` | Remove service permissions (client roles) from a role composite | +| DELETE | `/composites` | loop: `GET /admin/realms/{realm}/roles` → per role `GET .../composites` → `DELETE .../composites` | Revoke all composite mappings from all roles (rebuild) | +| POST | `/services/{service_id}/permissions` | `POST /admin/realms/{realm}/clients/{service_id}/roles` | Create a new permission (client role) for a specific service | +| POST | `/services/{service_id}/scopes` | `POST /admin/realms/{realm}/client-scopes` then `PUT .../clients/{service_id}/default-client-scopes/{scope_id}` | Create a realm-level scope and assign it to a service as a default scope (atomic) | + +Every endpoint accepts an optional `realm` query parameter, same as the IdP Configuration Service. + +`POST /roles/{role_name}/composites` and `DELETE /roles/{role_name}/composites` accept a JSON array of role representation objects `[{"id": "...", "name": "..."}]` and return `204 No Content` on success. + +`DELETE /composites` returns `204 No Content` on success. + +`POST /services/{service_id}/permissions` and `POST /services/{service_id}/scopes` accept a JSON object `{"name": "...", "description": "..."}` and return `201 Created` with the created resource as JSON. + +All endpoints return `502 Bad Gateway` with a JSON error body if the Keycloak Admin API call fails. + +## Configuration + +| Variable | Required | Description | +|----------|----------|-------------| +| `KEYCLOAK_URL` | Yes | Keycloak base URL, e.g. `http://keycloak-service.keycloak.svc:8080` | +| `KEYCLOAK_REALM` | Yes | Realm name, e.g. `kagenti` | +| `KEYCLOAK_ADMIN_USERNAME` | Yes | Admin username (from `keycloak-admin-secret`) | +| `KEYCLOAK_ADMIN_PASSWORD` | Yes | Admin password (from `keycloak-admin-secret`) | + +## Runtime + +- Framework: FastAPI +- Server: uvicorn +- Bind: `0.0.0.0:7072` +- Base image: `python:3.12-slim` +- Kubernetes ClusterIP Service: `aiac-pdp-policy-service:7072` +- Deployment: co-located with IdP Configuration Service as a container in the **Kagenti Interface Pod** (`pdp-interface-deployment.yaml`) + +## Dependencies (`requirements.txt`) + +``` +fastapi +uvicorn[standard] +python-keycloak +``` + +## File structure + +``` +aiac/src/aiac/pdp/service/ +├── __init__.py +└── policy/ + ├── __init__.py + └── keycloak/ + ├── __init__.py + ├── Dockerfile + ├── requirements.txt + └── main.py +``` + +## `main.py` behaviour notes + +- Instantiate the default `KeycloakAdmin` once at startup using env vars. +- `get_admin` is a FastAPI dependency accepting `realm: str | None = Query(None)`. +- `POST /roles/{role_name}/composites`: call `admin.add_composite_realm_roles_to_role(role_name, roles)`; return `Response(status_code=204)`. +- `DELETE /roles/{role_name}/composites`: call `admin.remove_composite_realm_roles_to_role(role_name, roles)`; return `Response(status_code=204)`. +- `DELETE /composites`: fetch all roles via `admin.get_realm_roles()`; for each role call `admin.get_composite_realm_roles_of_role(role_name)`; if composites are non-empty call `admin.remove_composite_realm_roles_to_role(role_name, composites)`; return `Response(status_code=204)`. +- `POST /services/{service_id}/permissions`: call `admin.create_client_role(service_id, {"name": ..., "description": ...})`; return `JSONResponse(status_code=201)` with the created permission representation. +- `POST /services/{service_id}/scopes`: call `admin.create_client_scope({"name": ..., "description": ..., "protocol": "openid-connect"})` to create the realm-level scope, then call `admin.add_default_default_client_scope(service_id, scope_id)` to assign it to the service; return `JSONResponse(status_code=201)` with the created scope representation. +- On `KeycloakError`, return HTTP 502 with `{"error": str(e)}`. diff --git a/aiac/docs/specs/components/pdp-policy-writer-opa.md b/aiac/docs/specs/components/pdp-policy-writer-opa.md new file mode 100644 index 00000000..797346e4 --- /dev/null +++ b/aiac/docs/specs/components/pdp-policy-writer-opa.md @@ -0,0 +1,304 @@ +# Component PRD: PDP Policy Writer (OPA) + +## Location +`aiac/src/aiac/pdp/service/policy/opa/` + +## Description +A FastAPI web service that translates a **Policy Model** into OPA Rego packages and writes them to an `AuthorizationPolicy` Kubernetes Custom Resource. The OPA plugin embedded in each AuthBridge instance fetches the Rego packages relevant to its pod from this CR at startup. + +The service is deployed as a container in the **Kagenti Interface Pod** alongside the IdP Configuration Service, behind the `aiac-pdp-policy-service:7072` ClusterIP. + +The service has no dependency on Keycloak. All Keycloak operations (entity reads) are handled by the **IdP Configuration Service** and its library (`aiac.idp.library.configuration`). + +--- + +## Pydantic models (`aiac.policy.model.models`) + +The Policy Writer deserializes the **canonical** `PolicyModel` / `AgentPolicyModel` / `PolicyRule` defined in [policy-model.md](policy-model.md) and imported from `aiac.policy.model.models`. This service does **not** define its own copies; the tables below summarize the fields the Rego generator consumes. (The former `aiac.pdp.library.models` module is deprecated — see policy-model.md "Replaces".) + +All models use `model_config = ConfigDict(extra='ignore')`. + +### `PolicyRule` + +A single access rule pairing a typed role with a typed scope. Used in both inbound and outbound rule sets. + +| Field | Type | +|-------|------| +| `role` | `Role` | +| `scope` | `Scope` | + +`Role` and `Scope` are the typed models from `aiac.idp.configuration.models`. The Rego generator emits their `.name` as the string literal OPA matches against. + +### `AgentPolicyModel` + +Complete policy definition for a single agent (service). Contains two sets of `PolicyRule` entries plus supporting data maps used by the Rego packages. + +| Field | Type | Description | +|-------|------|-------------| +| `agent_id` | `str` | Service ID from the AIAC trigger event (`aiac.apply.service.{id}`) | +| `agent_roles` | `list[Role]` | Realm roles assigned to this agent | +| `agent_scopes` | `list[Scope]` | Scopes this agent exposes | +| `source_roles` | `dict[str, list[Role]]` | Inbound: source (calling service) **id** → roles held. **Optional** gate input — an absent source passes. | +| `subject_roles` | `dict[str, list[Role]]` | Inbound: subject (end-user) **id** → roles held. **Mandatory** gate input. | +| `target_scopes` | `dict[str, list[Scope]]` | Outbound: target service **id** → scopes this agent may request on it | +| `inbound_rules` | `list[PolicyRule]` | Who may call this agent: `(subject_role, agent_scope)` tuples | +| `outbound_rules` | `list[PolicyRule]` | What this agent may call: `(this_agent_role, target_scope)` tuples | +| `outbound_subject_rules` | `list[PolicyRule]` | Which users may reach the agent's targets: `(user_role, tool_scope)` tuples. Defaults to `[]`. | + +**`agent_roles` / `agent_scopes` provenance:** these carry the agent's **own** identity — the service-account realm roles it holds and the scopes it exposes. The Policy Computation Engine resolves them from the agent's IdP `Service` record (P2) and embeds them on every agent model it writes; a realm-level agent with no owning service keeps `[]`. + +**Inbound rule semantics:** a subject holding realm role `role` is permitted to invoke this agent for the agent scope `scope`. Grouped by role, these rules become the `role_scopes` map (role → agent scopes) that the inbound package evaluates. + +**Outbound rule semantics:** this agent acting as realm role `role` is permitted to request the target scope `scope`. Grouped by role, these rules become the `agent_role_scopes` map (agent role → target scopes) that the outbound package evaluates. + +**Outbound subject rule semantics:** a subject holding realm role `role` (a **user** role) is permitted to reach a **tool** exposing scope `scope`. Grouped by role, these rules become the `outbound_subject_role_scopes` map (user role → tool scopes) that the **outbound** package's subject gate evaluates against `target_scopes[input.target]`. This is distinct from `inbound_rules` (user → *agent* scope): the outbound subject gate answers "may this user reach the tool?", not "may this user call the agent?". + +**Note on `target_scopes` direction:** the map is keyed by **target service id → allowed scopes** (the inverse of the former `scope_targets`, which was `scope → targets`). The outbound Rego generator emits this map **verbatim** and evaluates `target_scopes[input.target]` directly — there is no inversion (see below). + +### `PolicyModel` + +A partial or full system policy model. When sent to the PDP Policy Writer, contains only the agents whose policies have changed. + +| Field | Type | +|-------|------| +| `agents` | `list[AgentPolicyModel]` | + +### Usage + +```python +from aiac.policy.model.models import PolicyModel, AgentPolicyModel, PolicyRule +``` + +--- + +## Endpoints + +No `?realm=` parameter — the service operates on a Kubernetes CR, not a Keycloak realm. + +| Method | Path | Body | Operation | +|--------|------|------|-----------| +| `POST` | `/policy` | `PolicyModel` | Upsert Rego packages for all agents in the partial model | +| `POST` | `/policy/agents/{agent_id}` | `AgentPolicyModel` | Upsert Rego packages for a single agent | +| `DELETE` | `/policy/agents/{agent_id}` | — | Remove all Rego packages for a specific agent (off-boarding) | +| `DELETE` | `/policy` | — | Clear all Rego packages from the CR (rebuild pre-step) | +| `GET` | `/health` | — | Readiness probe | + +### Status codes + +| Endpoint | Success | Error | +|----------|---------|-------| +| `POST /policy` | `204 No Content` | `502 Bad Gateway` with `{"error": "..."}` if CR write fails | +| `POST /policy/agents/{agent_id}` | `204 No Content` | `502 Bad Gateway` if CR write fails | +| `DELETE /policy/agents/{agent_id}` | `204 No Content` | `502 Bad Gateway` if CR write fails | +| `DELETE /policy` | `204 No Content` | `502 Bad Gateway` if CR write fails | +| `GET /health` | `200 OK` `{"status": "ok"}` | `503 Service Unavailable` if CR is unreachable | + +--- + +## Rego package structure + +For each `AgentPolicyModel`, the service generates **two Rego packages**: one for the inbound pipeline and one for the outbound pipeline. The `agent_id` is slugified for use in the package name (hyphens → underscores, lowercase). + +**Input is identifiers only.** Both packages receive an input document of **IDs**, never roles or scopes: inbound input is `{subject, source}`, outbound input is `{subject, target}` (`subject` is the end-user id, `source` the calling service id, `target` the called service id). Every role/scope mapping is therefore **embedded in the package itself**, and the `allow` logic resolves IDs → roles → scopes internally. Because no per-request scope is supplied, the decision is **coarse**: a principal passes when it has access to **at least one** relevant scope. + +The generator embeds these symbols, derived from the `AgentPolicyModel`: + +| Rego symbol | Source | Shape | +|-------------|--------|-------| +| `subject_roles` | `model.subject_roles` | subject id → `[role.name, …]` | +| `source_roles` | `model.source_roles` | source id → `[role.name, …]` | +| `agent_scopes` | `model.agent_scopes` | `[scope.name, …]` | +| `agent_roles` | `model.agent_roles` | `[role.name, …]` | +| `role_scopes` | grouped `model.inbound_rules` | role.name → `[scope.name, …]` (agent scopes granted per subject role) — **inbound package only** | +| `outbound_subject_role_scopes` | grouped `model.outbound_subject_rules` | role.name → `[scope.name, …]` (tool scopes granted per user role) — **outbound package only** | +| `agent_role_scopes` | grouped `model.outbound_rules` | role.name → `[scope.name, …]` (tool scopes per agent role) | +| `target_scopes` | `model.target_scopes` | target id → `[scope.name, …]` | + +### Inbound package: `authz.{agent_slug}.inbound` + +Evaluated by the AuthBridge OPA plugin in the **inbound pipeline** — "who may call this agent". Input document: `{subject, source}` (IDs). **`subject` is mandatory; `source` is optional** (an absent source passes). A principal passes when it holds a role that grants at least one of the agent's own scopes (`agent_scopes`). + +```rego +package authz.{agent_slug}.inbound + +agent_scopes := ["{scope.name}", ...] # from agent_scopes + +subject_roles := { "{subject_id}": ["{role.name}", ...], ... } +source_roles := { "{source_id}": ["{role.name}", ...], ... } + +role_scopes := { "{role.name}": ["{scope.name}", ...], ... } # from inbound_rules + +subject_ok if { + some role in subject_roles[input.subject] + some scope in role_scopes[role] + scope in agent_scopes +} +source_ok if { not input.source } # optional: absent source passes +source_ok if { + some role in source_roles[input.source] + some scope in role_scopes[role] + scope in agent_scopes +} + +default allow := false +allow if { subject_ok; source_ok } +``` + +### Outbound package: `authz.{agent_slug}.outbound` + +Evaluated by the AuthBridge OPA plugin in the **outbound pipeline** — "what this agent may call". Input document: `{subject, target}` (IDs). The gate requires **both** the subject and the agent to pass, but the outbound **subject** gate is user→**tool** (distinct from the inbound user→agent gate): the subject must hold a role granting at least one **tool** scope the `target` accepts (via `outbound_subject_role_scopes`, grouped from `outbound_subject_rules`), **and** the agent (via its own `agent_roles`) must be permitted at least one scope that the `target` accepts. Both gates match against `target_scopes[input.target]`; `target_scopes` is consumed **directly** (target id → scopes) — it is not inverted. The inbound `role_scopes`/`agent_scopes` subject gate is **not** used here. + +```rego +package authz.{agent_slug}.outbound + +agent_roles := ["{role.name}", ...] # from agent_roles +agent_scopes := ["{scope.name}", ...] # from agent_scopes + +subject_roles := { "{subject_id}": ["{role.name}", ...], ... } + +outbound_subject_role_scopes := { "{role.name}": ["{scope.name}", ...], ... } # from outbound_subject_rules (user role → tool scopes) +agent_role_scopes := { "{role.name}": ["{scope.name}", ...], ... } # from outbound_rules (agent role → tool scopes) +target_scopes := { "{target_id}": ["{scope.name}", ...], ... } # from target_scopes + +# user may reach the tool: holds a role granting >=1 tool scope the target accepts +subject_ok if { + some role in subject_roles[input.subject] + some scope in outbound_subject_role_scopes[role] + scope in target_scopes[input.target] +} +# agent may reach the tool: agent role grants >=1 tool scope the target accepts +target_ok if { + some role in agent_roles + some scope in agent_role_scopes[role] + scope in target_scopes[input.target] +} + +default allow := false +allow if { subject_ok; target_ok } +``` + +A worked example (agent `github-agent`, users `developer`/`tester`, tool `github-tool`) is maintained alongside the tests. + +--- + +## Library: `aiac.pdp.policy.library` + +HTTP client module wrapping the PDP Policy Writer REST API. Exposes four module-level functions. Service URL is read from the `AIAC_PDP_POLICY_URL` environment variable (default: `http://127.0.0.1:7072`). All functions raise `RuntimeError` on non-2xx response. + +```python +def apply_policy(model: PolicyModel) -> None + # POST /policy + +def apply_agent_policy(agent_id: str, model: AgentPolicyModel) -> None + # POST /policy/agents/{agent_id} + +def delete_agent_policy(agent_id: str) -> None + # DELETE /policy/agents/{agent_id} + +def delete_policy() -> None + # DELETE /policy +``` + +### Dependencies + +``` +requests +pydantic +python-dotenv +``` + +### Usage + +```python +from aiac.pdp.policy.library.api import apply_policy, apply_agent_policy, delete_agent_policy, delete_policy +from aiac.policy.model.models import PolicyModel, AgentPolicyModel, PolicyRule + +apply_agent_policy("weather-agent", agent_model) +delete_policy() +apply_policy(full_model) +``` + +--- + +## Configuration + +| Variable | Required | Description | +|----------|----------|-------------| +| `AUTHORIZATION_POLICY_NAME` | TBD | Name of the `AuthorizationPolicy` CR to patch | +| `AUTHORIZATION_POLICY_NAMESPACE` | TBD | Namespace of the `AuthorizationPolicy` CR | + +Authentication to the Kubernetes API: in-cluster service account (auto-detected by the `kubernetes` Python client). The pod's `ServiceAccount` must be bound to a `ClusterRole` granting `get`/`patch`/`update` on `AuthorizationPolicy` resources. The `ServiceAccount`, `ClusterRole`, and `ClusterRoleBinding` are declared in `pdp-interface-deployment.yaml`. + +For local development, the `kubernetes` client falls back to `~/.kube/config` automatically. + +> **Note:** `AuthorizationPolicy` CR schema and ConfigMap source for env vars are TBD. + +--- + +## Runtime + +- Framework: FastAPI +- Server: uvicorn +- Bind: `0.0.0.0:7072` +- Base image: `python:3.12-slim` +- Kubernetes ClusterIP Service: `aiac-pdp-policy-service:7072` +- Deployment: co-located with IdP Configuration Service as a container in the **Kagenti Interface Pod** (`pdp-interface-deployment.yaml`) + +--- + +## Dependencies (`requirements.txt`) + +``` +fastapi +uvicorn[standard] +kubernetes +pydantic +``` + +--- + +## File structure + +``` +aiac/src/aiac/pdp/service/ +├── __init__.py +└── policy/ + ├── __init__.py + └── opa/ + ├── __init__.py + ├── Dockerfile + ├── requirements.txt + └── main.py + +aiac/src/aiac/pdp/ +├── __init__.py +└── library/ + ├── __init__.py + └── policy.py # apply_policy, apply_agent_policy, delete_agent_policy, delete_policy + # (models now imported from aiac.policy.model.models) +``` + +Build command: +```bash +docker build -f aiac/src/aiac/pdp/service/policy/opa/Dockerfile \ + -t aiac-pdp-policy-opa:latest aiac/src/ +``` + +--- + +## `main.py` behaviour notes + +- Load Kubernetes in-cluster config at startup via `kubernetes.config.load_incluster_config()`; fall back to `kubernetes.config.load_kube_config()` for local development. +- Instantiate a `kubernetes.client.CustomObjectsApi` for all CR operations. +- `_slugify(agent_id: str) -> str`: replace hyphens with underscores, lowercase — produces a valid Rego package name segment. +- `_generate_inbound_rego(model: AgentPolicyModel) -> str`: render the inbound Rego package string. Embeds `agent_scopes`, `subject_roles`, `source_roles`, and a `role_scopes` map (grouping `inbound_rules` by role → agent scope names); emits `subject_ok` (mandatory) and `source_ok` (optional — an absent `input.source` passes); `allow if { subject_ok; source_ok }`. +- `_generate_outbound_rego(model: AgentPolicyModel) -> str`: render the outbound Rego package string. Embeds `agent_roles`, `agent_scopes`, `subject_roles`, `outbound_subject_role_scopes` (from `outbound_subject_rules`), `agent_role_scopes` (from `outbound_rules`), and `target_scopes` (consumed directly, target id → scopes — **no inversion**); emits a user→tool `subject_ok` (matching `scope in target_scopes[input.target]`, **not** `role_scopes`/`agent_scopes`) and `target_ok`; `allow if { subject_ok; target_ok }`. The inbound `role_scopes` map is **not** embedded in the outbound package. +- `_upsert_agent(agent_id: str, inbound_rego: str, outbound_rego: str)`: patch the `AuthorizationPolicy` CR to upsert the two packages for `agent_id`. Schema TBD. +- `_delete_agent(agent_id: str)`: patch the CR to remove all packages for `agent_id`. +- `_delete_all()`: patch the CR to remove all packages. +- `POST /policy`: iterate `model.agents`; for each call `_generate_inbound_rego` + `_generate_outbound_rego` + `_upsert_agent`; return `Response(status_code=204)`. +- `POST /policy/agents/{agent_id}`: call `_generate_inbound_rego` + `_generate_outbound_rego` + `_upsert_agent`; return `Response(status_code=204)`. +- `DELETE /policy/agents/{agent_id}`: call `_delete_agent(agent_id)`; return `Response(status_code=204)`. +- `DELETE /policy`: call `_delete_all()`; return `Response(status_code=204)`. +- On Kubernetes API error, return HTTP 502 with `{"error": str(e)}`. +- `GET /health`: attempt to `get` the `AuthorizationPolicy` CR; return `200 {"status": "ok"}` on success, `503` on failure. diff --git a/aiac/docs/specs/components/policy-computation-engine.md b/aiac/docs/specs/components/policy-computation-engine.md new file mode 100644 index 00000000..6f2962eb --- /dev/null +++ b/aiac/docs/specs/components/policy-computation-engine.md @@ -0,0 +1,178 @@ +# Component PRD: Policy Computation Engine (`aiac.policy.computation`) + +## Problem Statement + +AIAC Agent sub-agents produce `list[PolicyRule]` objects representing partial policy updates — a new onboarding event may produce a handful of rules covering one agent's inbound and outbound access. Before this component, merging those rules into full `AgentPolicyModel` objects required each sub-agent to independently: + +1. Query the IdP Configuration Service to resolve which services own each role and scope. +2. Read the current `AgentPolicyModel` from the Policy Store. +3. Additively merge the new rules into the existing model. +4. Write the updated model back to the Policy Store. +5. Build a `PolicyModel` and push it to the PDP Policy Writer. + +This bespoke logic was duplicated across every sub-agent that produced policy rules, making the merge semantics inconsistent and the IdP query pattern scattered. + +## Solution + +A pure Python library module `aiac.policy.computation` centralises all policy computation. Sub-agents call a single function `compute_and_apply(rules: list[PolicyRule], override: bool = False) -> None`, which handles IdP resolution, merging (additive append or override-replace), Policy Store read/write, and PDP Policy Writer invocation. No FastAPI service, no Kubernetes deployment — the module is imported directly into the calling sub-agent's process. + +--- + +## User Stories + +1. As an AIAC Agent sub-UC agent, I want to submit a list of `PolicyRule` objects and have them automatically merged into the relevant `AgentPolicyModel` records, so that I do not need to implement IdP resolution or storage merge logic. +2. As an AIAC Agent sub-UC agent, I want the computation to be fire-and-forget, so that my sub-agent is not blocked waiting for Rego generation to complete. +3. As the Policy Computation Engine, I want to resolve which services own a given `Role`, so that I know which `AgentPolicyModel` records receive new outbound rules. +4. As the Policy Computation Engine, I want to resolve which services expose a given `Scope`, so that I know which `AgentPolicyModel` records receive new inbound rules. +5. As the Policy Computation Engine, I want to read each affected agent's current `AgentPolicyModel` before merging, so that additive append does not lose previously established rules. +6. As the Policy Computation Engine, I want to skip duplicate rules on append, so that re-processing the same event does not create redundant entries. +7. As the Policy Computation Engine, I want to push the updated `PolicyModel` to the PDP Policy Writer after all store writes succeed, so that OPA reflects the latest policy state. +8. As a developer, I want exceptions from the computation to be logged without propagating, so that a transient IdP or Policy Store failure does not crash the calling sub-agent. +9. As a developer, I want to import the engine from a stable path, so that the calling convention does not change as the module grows. + +--- + +## Implementation Decisions + +### Module Identity + +**Namespace:** `aiac.policy.computation` + +**Location:** `aiac/src/aiac/policy/computation/` + +**Package structure:** + +``` +aiac/src/aiac/policy/ +└── computation/ + ├── __init__.py # empty + └── engine.py # compute_and_apply +``` + +No FastAPI. No Kubernetes deployment. No container image. Imported as a library by AIAC Agent sub-UC agents. + +### Public API + +Single entry point: + +```python +def compute_and_apply(rules: list[PolicyRule], override: bool = False) -> None +``` + +- **Fire-and-forget:** the caller receives no return value. The function logs exceptions and does not propagate them — a transient failure in IdP resolution, Policy Store I/O, or PDP Policy Writer push must not crash the calling sub-agent. +- **`override`:** selects the merge mode (see [Merge Semantics](#merge-semantics)). `False` (default) appends additively; `True` authoritatively replaces every input role's mappings. Set by the caller (the Controller) from the producing UC's choice — UC1 = `False`, UC3 = `True`, UC2 Rebuild = `True`, UC2 Build = TBD. +- Import path: `from aiac.policy.computation.engine import compute_and_apply` + +### Algorithm + +Given `rules: list[PolicyRule]` and an `override` flag, the engine executes these steps. + +> **Input rules arrive with roles already flattened to their closure** by the calling +> sub-agent (UC1/UC2/UC3) — the role itself plus all descendant roles (recursively via +> `role.childRoles`), de-duplicated by `role.id`. The PCE performs **no role flattening**; +> each rule's `role` is treated as-is (it may be a composite role or one of its children). + +0. **Service catalog + classification.** Resolve the full service catalog once via `Configuration.get_services() -> list[Service]`, keyed as `serviceId → Service`. Each `Service` carries its `type` (inferred as `Agent` / `Tool`), its own service-account realm roles, and its exposed scopes. A service is an **agent** iff `type == "Agent"`; any other service (notably `Tool`) is a **pure target**. This catalog drives both agent identity (P2) and the "only agents are modelled" rule (P4). `get_services_by_role` / `get_services_by_scope` honor the **P1 client-side service filter**, so they return only services that genuinely own the role / expose the scope. + +1. **Classify and route each rule by kind (P5b).** The PCE is called with a single concatenated `list[PolicyRule]` spanning all three mappings, so each rule is classified by the kind of its role and scope and routed accordingly: + + | Rule kind (role, scope) | Routed to (on the agent model) | + |---|---| + | (user role, agent scope) | `inbound_rules` (+ `subject_roles`) | + | (user role, tool scope) | `outbound_subject_rules` (+ `subject_roles`) | + | (agent role, tool scope) | `outbound_rules` + `target_scopes[tool]` | + + - **Role kind** is read from ownership: `get_services_by_role(role)` returning an **agent** service ⇒ *agent role*; returning no agent (realm-level) ⇒ *user role*. + - **Scope kind** is read from exposure: `get_services_by_scope(scope)` returning an **agent** ⇒ *agent scope*; returning a **tool** ⇒ *tool scope*. + +2. **(agent role, tool scope) — mapping c:** for each agent owning the rule's role, add the rule to that agent's `outbound_rules`, and append the tool scope to `target_scopes[tool.serviceId]` for each tool exposing it (keyed by the tool's string `serviceId`, value is the typed `Scope`). This records "the agent may reach the tool". + +3. **(user role, agent scope) — mapping a:** for each agent exposing the rule's scope, add the rule to that agent's `inbound_rules`, and record the role's subjects — `get_subjects_by_role(role)` → append the typed `Role` to `subject_roles[subject.username]`. This records "the user may call the agent". + +4. **(user role, tool scope) — mapping b:** deferred until `target_scopes` is populated by mapping-c rules in the same batch. Then, for each agent model whose `target_scopes` already exposes that tool scope, add the rule to that agent's `outbound_subject_rules` and record the role's subjects in `subject_roles`. This records "the user may reach the tool the agent targets" — the outbound subject gate. A (user role, tool scope) rule with no agent targeting that tool is dropped (it cannot be attached to any agent). + +5. **P4 — only agents are modelled.** Routing only ever creates an `AgentPolicyModel` for a service identified as an **agent** (one owning the rule's role, or exposing the agent scope). A pure-target **Tool** therefore never gets its own model — no `github_tool.*.rego` is emitted. The agent→tool `target_scopes` edge is still recorded on the agent's model. + +6. **P2 — embed each agent's own identity.** For every agent model the PCE writes, set `agent_roles` / `agent_scopes` from that agent's `Service` record in the catalog (its own service-account realm roles and exposed scopes). Only **AIAC-provisioned** entities are embedded: the PCE keeps only roles/scopes carrying the `aiac.managed` marker (`Role.aiac_managed` / `Scope.aiac_managed`) and drops Keycloak's built-ins — the default client scopes (`profile`, `email`, `roles`, `web-origins`, `acr`, `basic`, `service_account`) and the `default-roles-` composite — that every OIDC client / service account carries. This keeps the embed to domain entities only and makes it deterministic across runs (built-ins are returned in an arbitrary order by Keycloak). A realm-level agent with no owning service in the catalog keeps `[]`. Without the embed, both generated gates would deny-all (inbound `subject_ok` needs a non-empty `agent_scopes`; outbound `target_ok` needs a non-empty `agent_roles`). + +7. **Merge (additive append, or override replace):** for each affected agent, read the current `AgentPolicyModel` from the Policy Store via `get_agent_policy(agent_id)`. + - **If `override` is `True`:** first purge the **distinct set of input roles** from the model — remove every stored `PolicyRule` whose `role.id` matches from `inbound_rules`, `outbound_rules`, **and** `outbound_subject_rules`, drop those role `id`s from all `source_roles` / `subject_roles` lists, and reconcile `target_scopes` by recomputing it from the surviving `outbound_rules`. This purge is done **once, up-front** for the whole input-role set — before any new rule is applied — so rules for a shared role are not wiped after being added. + - **Then (both modes):** append new rules and map entries that are not already present (de-duplicate rules by value; de-duplicate map list values by the entity's `id`). Because the maps are keyed by plain strings, merging is a plain dict-key lookup. P2's `agent_roles` / `agent_scopes` are then set from the catalog (authoritative for the agent's own identity). + + Write the updated model back via `apply_agent_policy(agent_id, model)`. + +8. **PDP push:** once all Policy Store writes complete, build a `PolicyModel` from the updated agents and call `aiac.pdp.policy.library.apply_policy(model)` (fire-and-forget within this function). + +### Merge Semantics + +The `override` flag (set by the caller from the producing UC's choice) selects the merge mode: + +- **`override=False` (default — additive append):** existing `inbound_rules` and `outbound_rules` entries are preserved; new rules and map entries are appended. De-duplication compares rules by value (`role.id` + `scope.id`) and map list values by the entity's `id`. This is the incremental path (e.g. UC1 Service Onboarding, where existing roles receive a partial new mapping and must not lose their other access). +- **`override=True` (authoritative role-keyed replace):** before appending, the engine purges every input role from the stored model (both directions + `target_scopes` reconciliation, per algorithm step 5) so the fresh rules become that role's complete mapping. Used by role-scoped recomputes (UC3 Role Update) and full rebuilds (UC2 Rebuild), where the input already represents each role's complete intended scope set. + +`override=True` provides **role-level** revocation — a role's stale mappings are removed before re-applying. Finer-grained single-rule revocation (removing one `PolicyRule` without replacing its whole role) is still **TBD**. + +### Dependencies + +| Module | Purpose | +|--------|---------| +| `aiac.policy.model` | `PolicyRule`, `AgentPolicyModel`, `PolicyModel` | +| `aiac.idp.configuration.library` | `Configuration` — `get_services` (catalog: type + own roles/scopes), `get_services_by_role`, `get_services_by_scope`, `get_subjects_by_role` | +| `aiac.policy.store.library` | `get_agent_policy`, `apply_agent_policy` | +| `aiac.pdp.policy.library` | `apply_policy` — push updated `PolicyModel` to OPA | + +### Not Called By + +The PCE is **not** called by: +- PDP Policy Writer — it is the downstream consumer, not a caller +- Policy Store — the store is pure CRUD with no computation +- IdP Configuration Service — the IdP service has no awareness of this module + +### Not Responsible For + +- Rule revocation (TBD) +- Bootstrapping `AgentPolicyModel` records for new agents (the store returns a 404; the engine creates a fresh model in that case) +- Translating `PolicyModel` → Rego packages (responsibility of `aiac.pdp.policy.library` / PDP Policy Writer) + +--- + +## Testing Decisions + +Good tests assert external behavior — what the engine does to the Policy Store and PDP Policy Writer — not internal merge logic directly. + +**Seam:** mock all downstream dependencies at their module-level import boundary: +- `aiac.idp.configuration.library` — mock `Configuration.get_services` (the catalog, carrying `type` + each service's own roles/scopes), `Configuration.get_services_by_role`, `Configuration.get_services_by_scope`, and `Configuration.get_subjects_by_role` +- `aiac.policy.store.library` — mock `get_agent_policy`, `apply_agent_policy` +- `aiac.pdp.policy.library` — mock `apply_policy` + +Key behaviors to assert: +- **Routing by kind (P5b):** a (user role, agent scope) rule lands in the exposing agent's `inbound_rules`; a (agent role, tool scope) rule lands in the owning agent's `outbound_rules` with a `target_scopes[tool]` entry; a (user role, tool scope) rule lands in `outbound_subject_rules` of the agent whose `target_scopes` already exposes that tool scope. +- A (user role, tool scope) rule with no agent targeting that tool produces no model. +- **P2:** each written agent model carries its own `agent_roles` / `agent_scopes` read from the service catalog, filtered to AIAC-provisioned entities (the `aiac.managed` marker) so Keycloak built-ins are excluded; an agent with no AIAC-managed catalog roles/scopes keeps `[]`. +- **P4:** a pure-target **Tool** service is never written as its own model, even though it exposes a scope the agent reaches; the agent→tool `target_scopes` edge is still recorded. +- `get_subjects_by_role` records `subject_roles` keyed by the subject's string `username` with the typed `Role` in the value list (for both user→agent and user→tool rules). +- `target_scopes` on the written model is keyed by the target service's string `serviceId` with the typed `Scope` in the value list; every relationship map has string keys, so `model_dump(mode="json")` round-trips without a custom key serializer. +- The PCE does **not** flatten roles: `get_services_by_role` is called once per rule's role as-is (rules arrive pre-flattened from the UC); passing a rule with a composite role does not trigger per-child calls inside the PCE. +- With `override=False` (default), existing rules and map entries in the fetched `AgentPolicyModel` are preserved after merge (additive append). +- With `override=True`, every input role's stored mappings are purged from `inbound_rules`, `outbound_rules`, **and** `outbound_subject_rules` (and dropped from `source_roles` / `subject_roles`, with `target_scopes` reconciled from surviving outbound rules) before the fresh rules are applied; the distinct input-role set is purged once, up-front. +- Duplicate rules (same role + scope already present) are not appended twice; duplicate map list entries (same `id`) are not appended twice. +- `apply_policy` is called exactly once after all `apply_agent_policy` writes complete. +- An exception from any dependency is logged and does not propagate to the caller. + +**Prior art:** `3.14-unit-tests-write-api.md` (mock HTTP boundary pattern — apply the same approach at the library import boundary here). + +--- + +## Out of Scope + +- **Fine-grained rule revocation:** removing an individual `PolicyRule` without replacing its whole role. `override=True` covers role-level replace (see [Merge Semantics](#merge-semantics)); single-rule revocation is not yet designed — marked TBD. +- **Full policy rebuild:** the PCE handles incremental updates only. Full rebuilds (clear + reapply all) are driven by higher-level orchestration outside this module. +- **Direct Keycloak calls:** all IdP queries go through `aiac.idp.configuration.library.Configuration`. The PCE never calls Keycloak directly. +- **Persistence of `PolicyRule` inputs:** the PCE does not store the input rule list — only the merged `AgentPolicyModel` output is persisted. + +--- + +## Further Notes + +- The PCE is the **only** caller of `aiac.pdp.policy.library.apply_policy` from AIAC Agent sub-agents. Sub-agents no longer call the PDP Policy Library directly; they call `compute_and_apply` instead. +- `aiac/src/aiac/agent/policy/api.py` retains `role_to_scopes` / `roles_to_scope` helpers used by AIAC Agent sub-UC agents. `PolicyRule` (now at `aiac.policy.model`) is imported from there. These helpers are not used by the PCE. diff --git a/aiac/docs/specs/components/policy-model.md b/aiac/docs/specs/components/policy-model.md new file mode 100644 index 00000000..55d194ab --- /dev/null +++ b/aiac/docs/specs/components/policy-model.md @@ -0,0 +1,171 @@ +# Component PRD: Policy Model (`aiac.policy.model`) + +## Problem Statement + +`PolicyRule`, `AgentPolicyModel`, and `PolicyModel` were previously defined in `aiac.pdp.library.models`. Three independent consumers now need these types: + +- `aiac.pdp.policy.library` — translates `PolicyModel` into HTTP calls to the PDP Policy Writer +- `aiac.policy.store.library` — reads/writes `AgentPolicyModel` from/to the Policy Store +- `aiac.policy.computation` — builds and merges `AgentPolicyModel` objects + +Keeping the canonical model definitions inside a PDP-namespaced module (`aiac.pdp.library.models`) forces both the Policy Store library and the Policy Computation Engine to take a dependency on the PDP package — a wrong-layer coupling. Any of the three consumers importing from `aiac.pdp.library.models` would create a transitive dependency on an unrelated service namespace. + +Additionally, the old `PolicyRule` used plain `str` for `role` and `scope`. The PCE algorithm requires typed `Role` and `Scope` objects (from `aiac.idp.configuration.models`) to invoke `Configuration.get_services_by_role` and `Configuration.get_services_by_scope`. + +Finally, the original `source_roles`, `subject_roles`, and `scope_targets` maps used pydantic model objects (`Service`, `Subject`, `Scope`) as dict keys. Model-object keys do not round-trip through `model_dump(mode="json")` / JSON without custom key handling, and they couple `aiac.policy.model` to the id-only `__hash__`/`__eq__` of the IdP models. The outbound map was also keyed by scope (`scope → targets`), whereas consumers need the inverse (`target → scopes`) to emit per-target authorization directly. + +## Solution + +A canonical, dependency-free model module at `aiac.policy.model` defines `PolicyRule`, `AgentPolicyModel`, and `PolicyModel` with typed fields. No HTTP client, no service code — importable by any consumer without side effects. `PolicyRule.role` and `PolicyRule.scope` are typed `Role` and `Scope` objects from `aiac.idp.configuration.models`. + +The relationship maps (`source_roles`, `subject_roles`, `target_scopes`) are keyed by the string `id` of the referenced entity rather than by a typed object, so they serialize to JSON natively and carry no hashability requirement into `aiac.policy.model`. Typed `Role` / `Scope` objects are retained as the map *values* (and in `PolicyRule`), preserving the typing the PCE needs for IdP queries. The outbound map is `target_scopes` (`target service id → scopes permitted`), the inverse of the former `scope_targets`. + +--- + +## User Stories + +1. As the Policy Computation Engine, I want to import `PolicyRule`, `AgentPolicyModel`, and `PolicyModel` from a shared, neutral namespace, so that I do not take an unwanted dependency on the PDP package. +2. As the PDP Policy Library, I want to import `PolicyModel` and `AgentPolicyModel` from `aiac.policy.model`, so that my HTTP serialization logic does not duplicate model definitions. +3. As the Policy Store Library, I want to import `AgentPolicyModel` and `PolicyModel` from `aiac.policy.model`, so that response deserialization uses the same canonical types as every other consumer. +4. As an AIAC Agent sub-UC agent, I want to construct a `PolicyRule` with typed `Role` and `Scope` objects, so that the PCE can use them for IdP queries without additional type conversion. +5. As the Policy Computation Engine, I want `source_roles`, `subject_roles`, and `target_scopes` keyed by string entity IDs, so that I build them with `entity.id` and they serialize to JSON without custom key handling. +6. As a developer, I want all models to silently ignore unknown fields from API responses, so that IdP API additions do not break deserialization. +7. As the PDP Policy Library, I want outbound permissions expressed as `target service id → allowed scopes`, so that I can emit per-target authorization directly without inverting a `scope → targets` map. +8. As a consumer serializing an `AgentPolicyModel` to JSON, I want every relationship map to have string keys, so that `model_dump(mode="json")` round-trips without a custom key serializer. + +--- + +## Implementation Decisions + +### Module Identity + +**Namespace:** `aiac.policy.model` + +**Location:** `aiac/src/aiac/policy/model/` + +**Package structure:** + +``` +aiac/src/aiac/policy/ +└── model/ + ├── __init__.py # empty + └── models.py # PolicyRule, AgentPolicyModel, PolicyModel +``` + +### Dependencies + +| Dependency | Purpose | +|------------|---------| +| `pydantic` | `BaseModel`, `ConfigDict` | +| `aiac.idp.configuration.models` | Typed `Role`, `Scope` (as map values and in `PolicyRule`) | + +No HTTP client dependency. No `requests`, no `python-dotenv`. + +### Pydantic Models + +All models use `model_config = ConfigDict(extra='ignore')`. + +#### `PolicyRule` + +A single access rule pairing a typed role with a typed scope. Used in both inbound and outbound rule sets. + +| Field | Type | Description | +|-------|------|-------------| +| `role` | `Role` | Typed role from `aiac.idp.configuration.models` | +| `scope` | `Scope` | Typed scope from `aiac.idp.configuration.models` | + +#### `AgentPolicyModel` + +Complete policy definition for a single agent (service). Inbound and outbound rule sets are typed collections. + +| Field | Type | Description | +|-------|------|-------------| +| `agent_id` | `str` | Service ID from the AIAC trigger event (`aiac.apply.service.{id}`) | +| `agent_roles` | `list[Role]` | Realm roles assigned to this agent | +| `agent_scopes` | `list[Scope]` | Scopes this agent exposes | +| `source_roles` | `dict[str, list[Role]]` | Inbound: source (calling service) **id** → roles held. **Optional** gate input — an absent source passes. | +| `subject_roles` | `dict[str, list[Role]]` | Inbound: subject (end-user) **id** → roles held. **Mandatory** gate input. | +| `target_scopes` | `dict[str, list[Scope]]` | Outbound: target service **id** → scopes this agent may request on it | +| `inbound_rules` | `list[PolicyRule]` | Who may call this agent: `(subject_role, agent_scope)` tuples | +| `outbound_rules` | `list[PolicyRule]` | What this agent may call: `(this_agent_role, target_scope)` tuples | +| `outbound_subject_rules` | `list[PolicyRule]` | Which users may reach the agent's targets: `(user_role, tool_scope)` tuples. Defaults to `[]`. | + +**Inbound rule semantics:** a subject holding realm role `role` is permitted to invoke this agent for the agent scope `scope`. The PDP Policy Writer consumes `inbound_rules` as a role → agent-scope map; its inbound gate is keyed on the subject id (mandatory), with the calling source id optional. + +**Outbound rule semantics:** this agent acting as realm role `role` is permitted to request the target scope `scope`. The PDP Policy Writer consumes `outbound_rules` as an agent-role → target-scope map; its outbound gate requires both the subject and the agent to be authorized. + +**Outbound subject rule semantics:** `outbound_subject_rules` holds `(user role, tool scope)` pairs — the outbound subject gate; a user holding `role` may reach a tool exposing `scope`. It is the outbound counterpart of `inbound_rules` (which pairs a user role with an *agent* scope): where `inbound_rules` answers "may this user call the agent?", `outbound_subject_rules` answers "may this user reach the tool the agent targets?". The PDP Policy Writer groups it into `outbound_subject_role_scopes` (user role → tool-scope names) and matches against `target_scopes[input.target]`, not against `agent_scopes`. + +#### `PolicyModel` + +A partial or full system policy model. When sent to `POST /policy` on the Policy Store, it may contain only the agents whose policies have changed. + +| Field | Type | +|-------|------| +| `agents` | `list[AgentPolicyModel]` | + +### Map keys are string IDs + +`source_roles`, `subject_roles`, and `target_scopes` are keyed by the string `id` of the referenced Keycloak entity (source service id, subject id, target service id) rather than by the typed `Service` / `Subject` / `Scope` object. Rationale: + +- JSON object keys must be strings. A dict keyed by a pydantic model does not round-trip through `model_dump(mode="json")` / JSON without a custom key serializer; a `str` key serializes natively. +- The IdP models are plain pydantic models (default field-based equality, not hashable). Consumers build these maps with `entity.id` as the key. + +As a result, no field in `aiac.policy.model` uses a typed object as a dict key, and this module imports only `Role` and `Scope` from `aiac.idp.configuration.models` (as map *values* and in `PolicyRule`). `Service` and `Subject` are no longer referenced here. + +### Usage + +```python +from aiac.policy.model.models import PolicyRule, AgentPolicyModel, PolicyModel +from aiac.idp.configuration.models import Role, Scope + +role = Role(id="r1", name="weather-reader", composite=False) +scope = Scope(id="s1", name="read") + +rule = PolicyRule(role=role, scope=scope) +agent_model = AgentPolicyModel( + agent_id="weather-agent", + agent_roles=[role], + agent_scopes=[scope], + source_roles={}, + subject_roles={"u1": [role]}, # keyed by subject id + target_scopes={"github-tool": [scope]}, # target service id → scopes + inbound_rules=[rule], + outbound_rules=[], + outbound_subject_rules=[], # (user_role, tool_scope) pairs; defaults to [] +) +model = PolicyModel(agents=[agent_model]) +``` + +### Replaces + +`aiac.pdp.library.models` is deprecated. All consumers must migrate their imports to `aiac.policy.model.models`. + +--- + +## Testing Decisions + +**Seam:** model instantiation and serialization — no HTTP boundary, no mocking required. + +Key behaviors to assert: +- `PolicyRule` accepts typed `Role` and `Scope` objects; rejects plain `str` where `Role`/`Scope` is expected. +- `AgentPolicyModel` with string-ID keys in `source_roles`, `subject_roles`, and `target_scopes` round-trips through `model_dump(mode="json")` / `model_validate()` with the typed `Role` / `Scope` list values preserved. +- `target_scopes` maps a target service id to the list of `Scope` objects permitted on it (outbound direction is `target → scopes`, not `scope → targets`). +- `outbound_subject_rules` defaults to `[]` (constructors that omit it still validate) and round-trips through `model_dump(mode="json")` / `model_validate()` with its `(user_role, tool_scope)` `PolicyRule` values preserved. +- A relationship map keyed by a plain string serializes to a JSON object without a custom key serializer. +- `ConfigDict(extra='ignore')` causes unknown fields to be silently discarded on `model_validate()`. + +--- + +## Out of Scope + +- HTTP serialization logic — handled by `aiac.policy.store.library`, `aiac.policy.store.service`, and `aiac.pdp.policy.library`. +- IdP API integration — `Service`, `Role`, `Scope` shapes are owned by `aiac.idp.configuration.models`. +- Rule revocation semantics — TBD; no model changes required until the design is finalised. + +--- + +## Further Notes + +- Keying maps by string `id` sidesteps the previous reliance on id-only hashing of the IdP models: two records for the same Keycloak entity fetched at different times (with potentially different enrichment fields) collapse to the same string key regardless of those differences. +- `aiac/src/aiac/agent/policy/api.py` imports `PolicyRule` from `aiac.policy.model`. The `role_to_scopes` / `roles_to_scope` helpers in that file remain in place and are used by AIAC Agent sub-UC agents directly; they are not consumed by the PCE. diff --git a/aiac/docs/specs/components/policy-store.md b/aiac/docs/specs/components/policy-store.md new file mode 100644 index 00000000..2856f78b --- /dev/null +++ b/aiac/docs/specs/components/policy-store.md @@ -0,0 +1,163 @@ +# Component PRD: AIAC Policy Store + +## Problem Statement + +The AIAC Agent's Policy Computation Engine and Policy Builder sub-agent produce and merge `AgentPolicyModel` objects representing the access control policy for each service. The PDP Policy Writer translates these into Rego packages and writes them to an `AuthorizationPolicy` Kubernetes CR — but this derived artifact cannot be reverse-engineered back into structured `AgentPolicyModel` data. Without a durable structured policy store: + +- The Policy Computation Engine cannot read current policy state for additive merging — it must re-derive the full state from the PDP snapshot on every trigger. +- Off-boarded agents leave no structured record of their removal. +- Pod restarts lose any in-flight policy construction context. + +## Solution + +A dedicated **AIAC Policy Store** owns an in-memory `PolicyModel` cache backed by a SQLite database for durability. A companion library [`aiac.policy.store.library`](library-policy-store.md) exposes module-level typed functions matching the `aiac.pdp.policy.library` pattern, used by the Policy Computation Engine to read and write policy state without any storage-layer boilerplate. + +The PDP Policy Writer retains sole ownership of the `AuthorizationPolicy` CR (Rego packages) and has no dependency on the Policy Store. The two persistence artifacts serve distinct purposes and are owned by distinct services: + +| Artifact | Owner | Contents | +|---|---|---| +| SQLite `agent_policies` table | Policy Store | Structured `AgentPolicyModel` — source of truth (cache-first, write-through) | +| `AuthorizationPolicy` CR (one total) | PDP Policy Writer | Derived Rego packages — OPA runtime artifact | + +--- + +## User Stories + +1. As the Policy Computation Engine, I want to read the current `AgentPolicyModel` for a specific agent, so that I can additively append new rules without overwriting existing ones. +2. As the Policy Computation Engine, I want to read the full `PolicyModel` (all agents), so that I can execute a whole-system policy rebuild. +3. As the Policy Computation Engine, I want to write an `AgentPolicyModel` to persistent storage, so that the current policy state survives pod restarts. +4. As the Policy Computation Engine, I want to delete a specific agent's policy on off-boarding, so that decommissioned services are removed from the structured policy store. +5. As the Policy Computation Engine, I want to clear all agent policies in a single call, so that a full policy rebuild can start from a clean state. +6. As a consumer of the Policy Store library, I want a typed Python library that returns `AgentPolicyModel` and `PolicyModel` objects directly, so that I can work with structured policy data without writing storage client code. +7. As an operator, I want the Policy Store deployed as its own single-replica StatefulSet with a dedicated PVC, so that its storage and restart lifecycle is decoupled from the stateless policy services. + +--- + +## Implementation Decisions + +### Policy Store Service + +**Location:** `aiac/src/aiac/policy/store/service/` + +**Port:** `0.0.0.0:7074` + +**ClusterIP Service:** `aiac-policy-store-service:7074` + +**Deployment:** dedicated single-replica `StatefulSet` `aiac-policy-store`, with a `volumeClaimTemplate` PVC (1 Gi, `ReadWriteOnce`, cluster-default StorageClass) mounted at `/data`. Fronted by a headless Service for stable pod DNS plus the `aiac-policy-store-service:7074` ClusterIP for clients. Not co-located with IdP Configuration / PDP Policy Writer. + +**Framework:** FastAPI + uvicorn. **Base image:** `python:3.12-slim`. + +**Storage backend:** SQLite via `sqlite3` stdlib (zero extra dependency — `sqlite3` ships with the Python standard library). Database file: `AGENTPOLICY_DB_PATH` (default `/data/policy_model.db`). + +**Schema:** + +```sql +CREATE TABLE IF NOT EXISTS agent_policies ( + agent_id TEXT PRIMARY KEY, + spec TEXT NOT NULL -- AgentPolicyModel.model_dump() as JSON +); +``` + +**In-memory cache:** the service owns a full `PolicyModel` in memory as the authoritative serving layer: +- All `GET` requests served from memory (storage never queried at runtime). +- Every mutation writes through to SQLite synchronously before returning `204`. +- On pod restart: load all rows from SQLite → populate cache → begin serving. + +**Transaction strategy:** +- Full rebuild (`POST /policy`): `BEGIN` → `DELETE FROM agent_policies` → per-agent `INSERT` → `COMMIT`. Crash before `COMMIT` leaves the prior state intact (SQLite WAL rollback). +- Per-agent upsert (`POST /policy/agents/{id}`): `INSERT OR REPLACE INTO agent_policies VALUES (?, ?)`. + +**Future normalization:** migrate to `agent_policies` + `policy_rules(agent_id, role, scope)` tables once `AgentPolicyModel`/`PolicyRule` schema stabilizes — a future observability UI will need queryable columns. JSON column in the current schema avoids migration churn during active development. + +**Endpoints:** + +| Method | Path | Body | Returns | +|---|---|---|---| +| `GET` | `/policy` | — | `PolicyModel` (all agents) | +| `GET` | `/policy/agents/{agent_id}` | — | `AgentPolicyModel` | +| `POST` | `/policy` | `PolicyModel` | `204 No Content` | +| `POST` | `/policy/agents/{agent_id}` | `AgentPolicyModel` | `204 No Content` | +| `DELETE` | `/policy/agents/{agent_id}` | — | `204 No Content` | +| `DELETE` | `/policy` | — | `204 No Content` | +| `GET` | `/health` | — | `200` / `503` | + +**Error responses:** +- `404 Not Found` with `{"error": "agent {id} not found"}` when `GET /policy/agents/{agent_id}` finds no entry in cache. +- `502 Bad Gateway` with `{"error": "..."}` on SQLite write error for all write endpoints. +- `503 Service Unavailable` if `GET /health` cannot open or query the SQLite file. + +**`main.py` functions:** + +- `_get_db() -> sqlite3.Connection` — open `AGENTPOLICY_DB_PATH` with `check_same_thread=False`; run `CREATE TABLE IF NOT EXISTS` on first open. +- `_upsert_agent(agent_id: str, model: AgentPolicyModel)` — `INSERT OR REPLACE INTO agent_policies VALUES (?, ?)` with `model.model_dump_json()`. +- `_get_agent(agent_id: str) -> AgentPolicyModel` — read from in-memory cache; raise `404` if absent. +- `_list_all() -> PolicyModel` — return in-memory cache directly. +- `_delete_agent(agent_id: str)` — `DELETE FROM agent_policies WHERE agent_id = ?`; remove from cache. +- `_delete_all()` — `DELETE FROM agent_policies`; replace cache with empty `PolicyModel`. +- `_rebuild(model: PolicyModel)` — `BEGIN` → `DELETE FROM agent_policies` → per-agent `INSERT` → `COMMIT`; replace cache atomically. + +**Configuration:** + +| Variable | Source | Default | +|---|---|---| +| `AGENTPOLICY_DB_PATH` | ConfigMap (`aiac-policy-store-config`) | `/data/policy_model.db` | + +**Dependencies:** `fastapi`, `uvicorn[standard]`, `pydantic`. `sqlite3` is stdlib (no new dependency). + +**Imports:** `from aiac.policy.model.models import PolicyModel, AgentPolicyModel` + +**File structure:** + +``` +aiac/src/aiac/policy/store/service/ +├── __init__.py +├── Dockerfile +├── requirements.txt +└── main.py +``` + +Build command (run from repo root): +```bash +docker build -f aiac/src/aiac/policy/store/service/Dockerfile \ + -t aiac-policy-store:latest aiac/src/ +``` + +--- + +## Testing Decisions + +Good tests assert external behavior at the system boundary — not internal implementation details such as private helpers or field serialization choices. + +### Policy Store Service + +**Seam:** SQLite `:memory:` database — pass an in-memory connection to the service on startup instead of opening `AGENTPOLICY_DB_PATH`. All behavioral assertions remain valid; only the storage seam changes. + +Key behaviors to assert: +- `GET /policy/agents/{id}`: returns `AgentPolicyModel` deserialized from cache; `404` when agent not in cache. +- `GET /policy`: returns `PolicyModel` for all agents; empty `PolicyModel(agents=[])` when cache is empty. +- `POST /policy/agents/{id}`: `spec` stored in SQLite; cache updated; `204` returned. +- `POST /policy` (rebuild): SQLite `DELETE` + per-agent `INSERT` issued inside one transaction; cache replaced atomically; `204` returned. +- `DELETE /policy/agents/{id}`: row removed from SQLite; removed from cache; `204`. +- `DELETE /policy`: all rows removed from SQLite; cache empty; `204`. +- SQLite write error on any write endpoint → `502`. +- SQLite file cannot be opened/queried on `GET /health` → `503`. + +See [library-policy-store.md](library-policy-store.md) for the companion library testing decisions. + +--- + +## Out of Scope + +- **Triggering Rego generation:** the Policy Store writes structured data only. Triggering Rego generation in the PDP Policy Writer is the responsibility of `aiac.pdp.policy.library` (called by `aiac.policy.computation`). +- **Pagination:** `GET /policy` returns all agents without pagination. At target scale (hundreds of agents), the full result fits within one SQLite query and one HTTP response. +- **In-cluster mTLS between Policy Computation Engine and Policy Store:** secured by Kubernetes network policy; no application-layer auth. +- **Multi-writer / replica scale-out:** the current design is single-writer (single-replica StatefulSet, RWO PVC, SQLite). Future migration to a shared DB (e.g. PostgreSQL) is a backend swap; the HTTP contract is unchanged. + +--- + +## Further Notes + +- The K8s manifests issue must create the `aiac-policy-store` StatefulSet, its `volumeClaimTemplate` PVC (1 Gi, `ReadWriteOnce`), and a headless Service. No CRD or RBAC is needed — the service does not touch the Kubernetes API. +- `spec` fields use snake_case (matching Pydantic's `model_dump()`) — consistent with the `AuthorizationPolicy` CR convention. The JSON column avoids a translation layer. +- `agent_id` is the SQLite `PRIMARY KEY`. The `aiac.apply.service.{id}` naming convention (lowercase alphanumeric + hyphens) should be maintained for consistency with trigger events. +- K8s resource names: StatefulSet `aiac-policy-store`, ClusterIP Service `aiac-policy-store-service:7074`, env var `AIAC_POLICY_STORE_URL`. diff --git a/aiac/docs/specs/components/rag-ingest-service.md b/aiac/docs/specs/components/rag-ingest-service.md new file mode 100644 index 00000000..5c869269 --- /dev/null +++ b/aiac/docs/specs/components/rag-ingest-service.md @@ -0,0 +1,89 @@ +# Component PRD: RAG Ingest Service + +## Description +A FastAPI REST service co-located with ChromaDB in the RAG Pod. Accepts knowledge documents for any configured collection, chunks and embeds them, and writes the resulting vectors into the ChromaDB instance in the same Pod. Supports both access control policies (`aiac-policies`) and org/business domain context (`aiac-domain-knowledge`) through a single collection-parameterized API surface. Developer-driven ingestion is performed via `kubectl port-forward`. + +After every successful ingest operation the service publishes a trigger event to the **Event Broker** (NATS JetStream) on the `aiac.apply.policy.build` subject. This causes the AIAC Agent to recompute and apply the updated policy against the live PDP state. All three ingest semantics (replace, update, delete) publish `build`; `rebuild` is an explicit operator-only command issued directly to the Agent and is never triggered by the ingest service. + +## Endpoints + +The `{collection}` path segment must be a slug from `AIAC_RAG_COLLECTIONS` (default: `policy,domain-knowledge`). Unknown slug → 404. + +### Replace — wipe and reload the named collection + +| Method | Path | Body | Description | +|--------|------|------|-------------| +| POST | `/ingest/{collection}/text` | `{"docs": [{"id": "...", "text": "..."}]}` | Replace entire named collection from a JSON body of text documents | +| POST | `/ingest/{collection}/file` | multipart upload (one or more files) | Replace entire named collection from uploaded files; `doc_id` = filename without extension | +| POST | `/ingest/{collection}/url` | `{"docs": [{"id": "...", "url": "..."}]}` | Replace entire named collection from a JSON body of URLs; service fetches each URL | + +**Replace semantics:** drops the ChromaDB collection and recreates it, then ingests all provided documents. Atomic at the collection level — partial failures roll back to an empty collection. An empty `docs` list wipes the collection. + +### Update — document-level upsert (additive, never deletes) + +| Method | Path | Body | Description | +|--------|------|------|-------------| +| POST | `/ingest/{collection}/update/text` | `{"docs": [{"id": "...", "text": "..."}]}` | Upsert documents by `doc_id`; absent `doc_id`s in the collection are left untouched | +| POST | `/ingest/{collection}/update/file` | multipart upload (one or more files) | Upsert documents from uploaded files; `doc_id` = filename without extension | +| POST | `/ingest/{collection}/update/url` | `{"docs": [{"id": "...", "url": "..."}]}` | Upsert documents from URLs; only named `doc_id`s are affected | + +**Update semantics:** for each incoming `doc_id`, deletes existing chunks for that `doc_id` then inserts new chunks. All other `doc_id`s in the collection are untouched. An empty `docs` list is a no-op. + +### Delete — explicit removal + +| Method | Path | Description | +|--------|------|-------------| +| DELETE | `/ingest/{collection}/{doc_id}` | Remove all chunks belonging to `doc_id` from the named collection. `doc_id` not found → 404 | + +**Delete** is the only path that removes content from a collection. `/update/*` endpoints never delete as a side effect. + +## Post-ingest Event Broker notification + +After every successful ingest operation (replace, update, or delete), the service publishes `{"id": ""}` to `aiac.apply.policy.build` on the Event Broker (`NATS_URL`). The publish is non-blocking: ingest success is reported to the caller before the NATS publish completes. Publish failures are logged but do not cause the ingest endpoint to return an error. This preserves ingest availability even when the Event Broker is temporarily unavailable. + +The AIAC Agent's durable consumer receives the event and acknowledges it after successful processing. Delivery guarantees (at-least-once, replay on Agent restart) are managed by the Event Broker — the RAG Ingest Service is fire-and-forget from its perspective. + +## Collection slug → ChromaDB name mapping + +| Slug | ChromaDB Collection Name | +|------|--------------------------| +| `policy` | `aiac-policies` | +| `domain-knowledge` | `aiac-domain-knowledge` | + +## Ingest conventions + +- Chunking and embedding are applied uniformly across all operations and both collections. +- `doc_id` is stored in ChromaDB chunk metadata on every write to enable document-level update and deletion. +- `/text` and `/url` endpoints take a JSON body `{"docs": [{"id": "...", "text/url": "..."}]}`. +- `/file` endpoints use multipart upload; `doc_id` is derived from the filename (extension stripped). Filename collisions within one call → 400. + +## Configuration + +| Variable | Default | Source | +|----------|---------|--------| +| `CHROMA_URL` | `http://localhost:8000` | ConfigMap | +| `AIAC_RAG_COLLECTIONS` | `policy,domain-knowledge` | ConfigMap | +| `NATS_URL` | `nats://aiac-event-broker-service:4222` | ConfigMap (`aiac-pdp-config`) | +| `EMBEDDING_BASE_URL` | — | ConfigMap | +| `EMBEDDING_MODEL` | — | ConfigMap | +| `EMBEDDING_API_KEY` | — | Kubernetes Secret | + +Adding a third collection is a configuration-only change: add a new slug to `AIAC_RAG_COLLECTIONS` and a corresponding entry in the slug→name map. No code modification required. + +## Runtime + +- Framework: FastAPI with uvicorn +- Bind: `0.0.0.0:7073` +- Base image: `python:3.12-slim` + +## Dependencies (`requirements.txt`) + +``` +fastapi +uvicorn[standard] +chromadb +httpx +nats-py +``` + +(Embedding model client TBD — depends on chosen embedding provider) diff --git a/aiac/docs/specs/components/rag-knowledge-base.md b/aiac/docs/specs/components/rag-knowledge-base.md new file mode 100644 index 00000000..1b4efe80 --- /dev/null +++ b/aiac/docs/specs/components/rag-knowledge-base.md @@ -0,0 +1,31 @@ +# Component PRD: RAG Knowledge Base + +## Description +A ChromaDB vector store that holds two named collections in a single instance: AIAC access control policies (`aiac-policies`) and org/business domain context (`aiac-domain-knowledge`). Deployed in a dedicated Kubernetes Pod alongside the RAG Ingest Service. The AIAC Agent retrieves relevant chunks from both collections at runtime via similarity search. + +## Technology +ChromaDB + +## Collections + +| Slug (wire) | ChromaDB Collection Name | Content | Written by | Read by | +|-------------|--------------------------|---------|------------|---------| +| `policy` | `aiac-policies` | Access control policy rules in natural language | RAG Ingest Service | Agent `fetch_policy` node | +| `domain-knowledge` | `aiac-domain-knowledge` | Org/business context — team rosters, application ownership, department mappings, who-does-what | RAG Ingest Service | Agent `fetch_domain_knowledge` node | + +The legal collection set is an open extension point governed by `AIAC_RAG_COLLECTIONS` on the RAG Ingest Service. Adding a new collection is a configuration-only change (new slug + ChromaDB name in the slug→name map) with no code modification required. + +## Deployment +Kubernetes **StatefulSet** in the RAG Pod, co-located with the RAG Ingest Service container. Exposed via the `aiac-rag-service` ClusterIP Service on port 8000 (ChromaDB default). Manifest: `rag-statefulset.yaml`. + +ChromaDB runs with `IS_PERSISTENT=TRUE` and `PERSIST_DIRECTORY=/chroma/chroma`. Data is stored on a 1 Gi `ReadWriteOnce` PersistentVolumeClaim mounted at `/chroma/chroma`. On pod recreation the StatefulSet rebinds the same PVC; ChromaDB resumes from persisted state without re-ingestion. The RAG Pod runs as a single replica. + +## Access patterns + +| Consumer | Operation | Collection | +|----------|-----------|------------| +| RAG Ingest Service | Write (replace / upsert / delete) | Either collection, selected by `{collection}` slug in the request URL | +| AIAC Agent `fetch_policy` | Read (similarity search, top-N chunks) | `aiac-policies` | +| AIAC Agent `fetch_domain_knowledge` | Read (similarity search, top-N chunks) | `aiac-domain-knowledge` | + +Each chunk written to ChromaDB stores `doc_id` in its metadata to enable document-level upsert and targeted deletion. diff --git a/aiac/docs/specs/demo/github-agent.md b/aiac/docs/specs/demo/github-agent.md new file mode 100644 index 00000000..e8d34a87 --- /dev/null +++ b/aiac/docs/specs/demo/github-agent.md @@ -0,0 +1,266 @@ +# Demo Spec: `github-agent` (A2A) — source + issue operations over `github-tool` + +> **Status:** spec (design source of truth). Implementation is decomposed into +> `docs/issues/demo/GA-*.md` and entry-pointed by +> `docs/handoffs/handoff-github-agent-implementation.md`. +> +> **This is a demo/reference agent**, not part of the AIAC service tree (`src/aiac/`). It is a +> self-contained, deployable A2A agent that realises the canonical `github-agent` used by the AIAC +> policy-pipeline integration test. + +--- + +## 1. Purpose & scenario mapping + +The AIAC policy-pipeline integration test +([`../integration-test/policy-pipeline.md`](../integration-test/policy-pipeline.md)) +and its fixture ([`../../../test/integration/scenario.py`](../../../test/integration/scenario.py)) are written +around a canonical **`github-agent`**: an autonomous A2A agent that acts on a user's behalf against +**source repositories** and an **issue tracker**, calling the **`github-tool`** MCP server. Until now +that agent has existed only as test data; this spec defines a **real, deployable** agent that matches it. + +The existing runnable reference, `agent-examples/a2a/git_issue_agent`, is **issue-only and single-skill**. +This agent generalises it to the scenario's **two capability areas**. + +### Mapping to the policy-pipeline scenario + +| Scenario element | This agent | +|---|---| +| Agent client `github-agent` (type **Agent**) | This A2A agent | +| Agent role `source-operator` → agent scope `source-access` | **Skill 1** — Source repository operations | +| Agent role `issues-operator` → agent scope `issues-access` | **Skill 2** — Issue & PR tracker operations | +| Tool `github-tool` scopes `source-read`/`source-write` | Source tool allow-list (see §5) | +| Tool `github-tool` scopes `issues-read`/`issues-write` | Issue/PR tool allow-list (see §5) | + +The agent does **not** know or enforce these scopes itself — AIAC/OPA + AuthBridge do. The agent simply +exposes the two capability areas; the AuthBridge sidecar performs inbound JWT validation and outbound +RFC-8693 token exchange, and the `github-tool` MitM swaps the exchanged token for a GitHub PAT by scope. + +### Related artefacts +- Scenario spec: [`../integration-test/policy-pipeline.md`](../integration-test/policy-pipeline.md) +- Scenario fixture: [`../../../test/integration/scenario.py`](../../../test/integration/scenario.py) +- Existing (issue-only) agent card: [`../../analysis/github-agent-card.json`](../../analysis/github-agent-card.json) +- `github-tool` MCP tool catalog (44 tools): [`../../analysis/github-mcp-tools-summary.json`](../../analysis/github-mcp-tools-summary.json) +- Reference agent: `agent-examples/a2a/git_issue_agent/` +- Reference deployment: `kagenti-extensions/authbridge/demos/github-issue/k8s/` +- **Sibling tool spec (UC-1 onboarding fixture):** [`github-tool.md`](github-tool.md) — a simplified + 4-tool stub (`source-read`, `source-write`, `issues-read`, `issues-write`) deployed as Service + `github-tool`. **This is not the tool this agent connects to.** The agent connects to the production + 44-tool server at `github-tool-mcp:9090/mcp`. Both coexist in namespace `team1` under different + Service names. + +--- + +## 2. Architecture + +Reuse the `git_issue_agent` stack verbatim — do not introduce a new framework: + +- **A2A SDK 1.x** server (route factories, `AgentInterface`, snake_case card fields). Binds `0.0.0.0` + on `PORT` (default **8000**). `create_jsonrpc_routes(..., enable_v0_3_compat=True)` — **required** + because Kagenti uses A2A 0.3 client libraries — plus the agent card at both + `/.well-known/agent-card.json` and legacy `/.well-known/agent.json`. +- **CrewAI** orchestration (`Agent`/`Crew`/`Task`, `Process.sequential`), LLM via **litellm** (`crewai.LLM`). +- **`crewai-tools[mcp]` `MCPServerAdapter`** — per-request connection to the MCP tool over + `transport="streamable-http"` at `MCP_URL` (default `http://github-tool-mcp:9090/mcp`), inside a + `with` block, torn down after each request. +- **Auth (three tiers, verbatim from the reference `GithubExecutor.execute`):** + 1. `GITHUB_TOKEN` env set → `Authorization: Bearer ` to MCP; + 2. else pass through the inbound request's `Authorization` header + (`context.call_context.state["headers"]["authorization"]`) — the AuthBridge/Envoy path; + 3. else unauthenticated + warning. + +``` + A2A client ──(JSON-RPC /)──► github-agent (:8000) + │ CrewAI: prereq extract → researcher + └──(streamable-http, MCP_URL)──► github-tool-mcp:9090/mcp ──► GitHub + (AuthBridge sidecar: inbound JWT validation; outbound RFC-8693 token exchange for MCP_URL host) +``` + +--- + +## 3. AgentCard (two skills — comprehensive) + +Built in `a2a_agent.py::get_agent_card()`. Fields: + +- **name:** `Github agent` +- **version:** `1.0.0` +- **description:** *"Autonomous Agent acting on a user's behalf against source repositories and an + issue tracker. It inspects and changes repository source contents and reads, creates, and updates + issues and their threads."* (verbatim from the scenario's `github-agent` description so the card and + the policy-pipeline fixture agree). +- **supported_interfaces:** `[AgentInterface(url=, protocol_binding="JSONRPC")]` +- **capabilities:** `AgentCapabilities(streaming=True)` +- **default_input_modes / default_output_modes:** `["text"]` +- **security_schemes:** single `Bearer` → `HTTPAuthSecurityScheme(scheme="bearer", bearer_format="JWT", description="OAuth 2.0 JWT token")` +- **skills:** two `AgentSkill`s: + +| id | name | description | tags | examples | +|---|---|---|---|---| +| `source_operations` | Source repository operations | Browse and search code; read, create, and modify repository file contents, branches, and commits. | `git, github, source, repositories, files, branches, commits` | "Show the README of kagenti/kagenti", "List the branches of owner/repo", "Create a branch and commit a fix to owner/repo" | +| `issue_operations` | Issue & PR tracker operations | Read, search, create, and update issues, comments, sub-issues, and pull requests. | `git, github, issues, pull-requests` | "List open issues in kubernetes/kubernetes", "Open an issue in owner/repo titled …", "Summarise PR #42 in owner/repo" | + +> The existing `git_issue_agent` card ([`../../analysis/github-agent-card.json`](../../analysis/github-agent-card.json)) +> has a single `github_issue_agent` skill — this card supersedes it with the two-skill shape. + +--- + +## 4. Agent internals + +Generalise the reference two-phase CrewAI flow from issue-only to source + issues: + +- **Phase A — prerequisite extraction.** A no-tools CrewAI agent parses the query into a generalised + `GithubQueryInfo`. Reuse the reference's robustness measures: **avoid `output_pydantic`** (fails on + small Ollama models) and parse raw JSON from the LLM text via a flat-object regex + (`re.search(r"\{[^{}]*\}", raw)`); coerce string arrays to lists. + + ```python + class GithubQueryInfo(BaseModel): + owner: str | None = None + repo: str | None = None + ref: str | None = None # branch / tag / sha, if named + path: str | None = None # file path, if named + numbers: list[int] | None = None # issue or PR numbers + ``` + + **Validation gates** (return a helpful message, no tool call, when unmet): + - `numbers` present → `owner` **and** `repo` required. + - `repo` present → `owner` required. + +- **Phase B — researcher.** One CrewAI "GitHub operations" agent wired with the **curated tool set** + (§5), `inject_date=True`, bounded `max_iter`/`max_retry_limit`, `respect_context_window=True`. A + generalised ReAct `TOOL_CALL_PROMPT` with a tool-selection decision tree spanning **source** + (files / branches / commits / code search) and **issues / PRs**, and a generalised `INFO_PARSER_PROMPT` + for Phase A. Identifiers (owner/repo/numbers) are copied verbatim; the final answer is grounded only + in tool output. + +`event.py` (streaming `Event` ABC) and `llm.py` (`CrewLLM`, incl. the Ollama `num_ctx=8192` bump) are +copied verbatim from the reference. + +--- + +## 5. Tool wiring (scenario source + issue) + +`github-tool` federates 44 GitHub tools at startup. This agent wires only the **source + issue** subset +that matches the scenario scopes, via an **explicit name allow-list** in `github_agent/tools.py` (robust +vs the reference's substring matching). The executor keeps only MCP tools whose `.name` is in the set +and raises `RuntimeError` if none resolve. An `ENABLED_TOOLS` env var (comma-separated) overrides the +default set. + +**Source (`source-access` → tool `source-read`/`source-write`):** +- read: `get_file_contents`, `list_branches`, `get_commit`, `list_commits`, `search_code` +- write: `create_or_update_file`, `delete_file`, `push_files`, `create_branch` + +**Issue & PR (`issues-access` → tool `issues-read`/`issues-write`):** +- read: `issue_read`, `list_issues`, `search_issues`, `list_issue_types`, `pull_request_read`, `list_pull_requests` +- write: `issue_write`, `add_issue_comment`, `sub_issue_write`, `create_pull_request`, `update_pull_request` + +**Excluded by default** (out of the policy-pipeline scenario; one edit / `ENABLED_TOOLS` to add back): +Teams & Users (`get_me`, `get_team_members`, `get_teams`), Security (`run_secret_scanning`), and +repo-lifecycle / release tools (`create_repository`, `fork_repository`, `list_tags`, `get_tag`, +`get_label`, releases). These are named in the module so they are trivially re-enabled. + +> The allow-list is a single editable constant grouped by skill/scope so the source/issue split stays +> legible and auditable against the scenario. + +--- + +## 6. Configuration & LLM presets + +Config via `github_agent/config.py` (`pydantic-settings`), adapted from the reference: + +| Variable | Description | Default | +|---|---|---| +| `TASK_MODEL_ID` | litellm model id | `ollama/ibm/granite4:latest` | +| `LLM_API_BASE` | OpenAI-compatible base URL | `http://host.docker.internal:11434` | +| `LLM_API_KEY` | LLM API key | `my_api_key` | +| `MODEL_TEMPERATURE` | Sampling temperature | `0` | +| `EXTRA_HEADERS` | Extra LLM headers (JSON) | `{}` | +| `MCP_URL` | MCP tool endpoint | `http://github-tool-mcp:9090/mcp` | +| `MCP_TIMEOUT` | MCP connect timeout (s) | `600` | +| `ENABLED_TOOLS` | Override the curated tool allow-list (comma-separated) | (unset → §5 default) | +| `PORT` | A2A listen port | `8000` | +| `LOG_LEVEL` | Log level | `INFO` | +| `GITHUB_TOKEN` | Static Bearer to MCP (else inbound passthrough) | (unset) | +| `ISSUER` | Expected `iss` of inbound JWTs (informational) | (unset) | +| `AGENT_ENDPOINT` | Override the URL advertised in the card | (unset) | + +**Env presets shipped:** +- `.env.ollama` **(default)** — `ollama/ibm/granite4:latest`, `LLM_API_BASE=http://host.docker.internal:11434`. +- `.env.openai` — `gpt-4o-mini`, key from k8s secret `openai-secret`. +- `.env.claude` — litellm Anthropic (`TASK_MODEL_ID=anthropic/claude-sonnet-4-6`, `ANTHROPIC_API_KEY` + from k8s secret `claude-secret`); native Anthropic endpoint (no `LLM_API_BASE`). +- `.env.template` — documented placeholders + `MCP_URL=http://github-tool-mcp:9090/mcp`. + +--- + +## 7. Deployment (aiac level) + +Manifests live under `aiac/demo/agents/github_agent/k8s/`, adapted from the github-issue demo. Namespace +`team1` (installer-provided ConfigMaps/secrets assumed present). + +- **`github-agent-deployment.yaml`** — `ServiceAccount` + `Deployment` + `Service` + `AgentRuntime`: + - **Deployment labels** (on `metadata.labels` **and** pod template): `protocol.kagenti.io/a2a: ""` + in addition to `app.kubernetes.io/name: github-agent`. The `protocol.kagenti.io/a2a` label is + required by the `AgentCardSyncReconciler` (`shouldSyncWorkload` gate): the operator stamps + `kagenti.io/type=agent` automatically via the `AgentRuntime`, but the protocol label must be set + in the manifest. Without it, no `AgentCard` CR is auto-created and `analyze_agent` falls back to + the default minimal scope. + - Pod labels `kagenti.io/inject: enabled`, `kagenti.io/spire: enabled`, `protocol.kagenti.io/a2a: ""`. + - Container port `8000`; env `MCP_URL=http://github-tool-mcp:9090/mcp`, + `JWKS_URI=http://keycloak-service.keycloak.svc:8080/realms/kagenti/protocol/openid-connect/certs`, + LLM vars, `PORT`, `LOG_LEVEL`; `/shared` `emptyDir` for operator-mounted client creds. + - `Service` (ClusterIP) with **two ports** — port order matters because + `AgentCardReconciler.getServicePort()` always takes `Ports[0]`: + 1. `agent: 8001 → 8001` (first) — direct path to the agent after authbridge port-stealing + (`8000 → 8001`); used by the operator's `AgentCardReconciler` to fetch `/.well-known/agent.json` + without going through the authbridge reverse proxy (which blocks unauthenticated requests). + 2. `proxy: 8080 → 8000` (second) — the public/authenticated path through the authbridge reverse + proxy; used by A2A clients that carry a valid JWT. + - `AgentRuntime{ type: agent, targetRef: this Deployment }` — enrolls the workload (operator applies + `kagenti.io/type=agent`, registers a Keycloak client, injects the AuthBridge sidecar). + - Image `github-agent:latest`, `imagePullPolicy: IfNotPresent` (kind-load; name is a documented knob). +- **`configmaps.yaml`** — `authbridge-config` (Keycloak URL/realm/issuer) + `authproxy-routes` with the + outbound token-exchange route: + ```yaml + - host: "github-tool-mcp" + target_audience: "github-tool" + token_scopes: "openid github-tool-aud github-full-access" + ``` +- **Prerequisite (reused, not created here):** the existing **production** `github-tool` Deployment/Service + (`authbridge/demos/github-issue/k8s/github-tool-deployment.yaml`, Service name `github-tool-mcp`) + + `github-tool-secrets`, a running Kagenti cluster (Keycloak realm `kagenti`, namespace `team1`). + The sibling UC-1 stub at `demo/tools/github_tool/` (Service `github-tool`) is a separate deployment + for AIAC onboarding discovery and is **not** a runtime dependency of this agent. + +**Wiring invariant:** agent `MCP_URL` host (`github-tool-mcp`) == `authproxy-routes` host == tool +Service name; exchanged audience (`github-tool`) == tool `AUDIENCE`. + +--- + +## 8. Verification + +**Local (no cluster — primary gate):** +1. `cd aiac/demo/agents/github_agent && uv lock && uv sync`. +2. `podman build -t github-agent:latest .`. +3. Startup + card: run `uv run --no-sync server` (or `test_startup.exp`), then + `curl -s localhost:8000/.well-known/agent-card.json | jq '.name, .skills[].id'` → + `"Github agent"` + `source_operations`, `issue_operations`. +4. (Optional; needs a GitHub PAT + reachable LLM) `GITHUB_TOKEN=… MCP_URL=https://api.githubcopilot.com/mcp/`, + send an A2A `message/send` read query and confirm a grounded, tool-cited answer. + +**Cluster (HITL — live Kagenti + Keycloak + LLM + tool PAT):** +5. `kind load docker-image github-agent:latest --name kagenti`. +6. Ensure `github-tool` + `github-tool-secrets` exist in `team1`. +7. `kubectl apply -f k8s/configmaps.yaml -f k8s/github-agent-deployment.yaml`. +8. Confirm AuthBridge injection + `kagenti.io/type=agent`; confirm the `AgentCard` CR was + auto-created (`kubectl get agentcard -n team1` → `github-agent-deployment-card`, `SYNCED=True`). + `kubectl port-forward svc/github-agent 8080:8080 -n team1` (proxy port); + send an authenticated A2A message; verify token exchange reaches `github-tool` and an answer returns. + +--- + +## 9. Out of scope +- Any changes to `github_tool` (reused as-is). +- Any changes to the AIAC pipeline, `policy-pipeline.md`, or the integration test. +- Wiring the agent into `agent-examples` CI (`build.yaml`) — aiac/demo images build independently. +- Onboarding this agent through the AIAC UC1 pipeline (separate activity; the card here is its input). diff --git a/aiac/docs/specs/demo/github-tool.md b/aiac/docs/specs/demo/github-tool.md new file mode 100644 index 00000000..fa1f8b0e --- /dev/null +++ b/aiac/docs/specs/demo/github-tool.md @@ -0,0 +1,290 @@ +# Demo Spec: `github-tool` (MCP) — minimal source + issue tool for UC-1 onboarding + +> **Status:** spec (design source of truth). This document describes the files to be built; +> it does not create them. +> +> **This is a demo/reference tool**, not part of the AIAC service tree (`src/aiac/`). It is a +> self-contained, deployable MCP tool server whose sole job is to make **UC-1 service onboarding** +> (`../components/aiac-agent/uc1-service-onboarding.md`, node `analyze_tool`) discover exactly the +> four canonical `github-tool` scenario scopes, deterministically. It is the tool sibling of the +> agent spec at [`github-agent.md`](github-agent.md). + +--- + +## 1. Purpose & scenario mapping + +The AIAC phase-1 deliverable ([`../../gh-issues/sub-issue-phase-1.md`](../../gh-issues/sub-issue-phase-1.md)) +demonstrates **service onboarding (UC-1)** end-to-end for one agent (`github-agent`) and one tool +(`github-tool`): AIAC classifies each service, discovers its capabilities, provisions the identities +they need, models access, and emits rules — which are then validated by **rule evaluation**, not by +live traffic. Phase 1 is explicitly **"Deploy + discover + evaluate": no live A2A traffic and no live +enforcement.** The tool is onboarded and its scopes are evaluated; it is **never actually driven by +the agent.** + +This spec defines the **tool half** of that demo: a **real, deployable MCP endpoint** that answers +`tools/list` with exactly four tools, so that UC-1's `analyze_tool` node derives exactly the four +canonical scenario tool scopes. + +### How UC-1 consumes this tool + +From `analyze_tool` (`../components/aiac-agent/uc1-service-onboarding.md`): + +1. **`classify_service`** resolves identity from the Keycloak client's `client.name` (the operator sets + it to `"{namespace}/{workload_name}"`), splits on the first `/` → `namespace` + `workload_name`, + LISTs pods in the namespace, finds the pod owned by `workload_name`, and reads the `kagenti.io/type` + pod label. It **must be `tool`** → routes to `analyze_tool`. +2. **`analyze_tool`** does `read_service(workload_name, namespace)` (the K8s Service named + `workload_name`), **requires the `protocol.kagenti.io/mcp` label** on that Service (a deploy-time + prerequisite — the operator does **not** stamp it), takes the Service's **first port**, and POSTs + JSON-RPC `tools/list` to + `http://{workload_name}.{namespace}.svc.cluster.local:{port}/mcp`. +3. UC-1 derives **one scope per returned tool**: + `ScopeDefinition(name=f"{workload_name}.{tool.name}", description=tool.description)`. + +So with workload `github-tool` and a tool named `source-read`, the provisioned scope is +`github-tool.source-read` with `description =` that tool's description. + +### Mapping to the policy-pipeline scenario + +The canonical scenario ([`../../../test/integration/scenario.py`](../../../test/integration/scenario.py), +`TOOL_SCOPES`) and its spec +([`../integration-test/policy-pipeline.md`](../integration-test/policy-pipeline.md), +*Role & scope descriptions → Tool scopes*) fix **four** `github-tool` scopes. This tool exposes +exactly the four MCP tools whose names + descriptions make UC-1 reproduce them: + +| Scenario tool scope | MCP tool name | UC-1-derived scope (`workload_name=github-tool`) | +|---|---|---| +| `source-read` | `source-read` | `github-tool.source-read` | +| `source-write` | `source-write` | `github-tool.source-write` | +| `issues-read` | `issues-read` | `github-tool.issues-read` | +| `issues-write` | `issues-write` | `github-tool.issues-write` | + +The tool does **not** know or enforce these scopes — AIAC/OPA + AuthBridge do (in later phases). +Phase 1 only discovers and evaluates them. + +### Related artefacts +- Phase-1 deliverable: [`../../gh-issues/sub-issue-phase-1.md`](../../gh-issues/sub-issue-phase-1.md) +- UC-1 `analyze_tool`: [`../components/aiac-agent/uc1-service-onboarding.md`](../components/aiac-agent/uc1-service-onboarding.md) +- Scenario fixture (`TOOL_SCOPES`): [`../../../test/integration/scenario.py`](../../../test/integration/scenario.py) +- Scenario spec: [`../integration-test/policy-pipeline.md`](../integration-test/policy-pipeline.md) +- Sibling agent spec: [`github-agent.md`](github-agent.md) +- Reference deployment: `kagenti-extensions/authbridge/demos/github-issue/k8s/` + +--- + +## 2. Relationship to the real `github-tool` (deliberate divergence) + +The sibling agent spec ([`github-agent.md`](github-agent.md) §2, §5, §7) is written against a +**production** `github-tool`: a real MCP server reachable at `github-tool-mcp:9090/mcp` that federates +**44 real GitHub tools** and proxies calls to the GitHub API (swapping the exchanged token for a GitHub +PAT by scope in a MitM). That tool is the runtime target for live A2A traffic in later phases. + +**This spec deliberately diverges from that production tool** in three ways, and the divergences are +intentional: + +1. **Four tools, not 44.** This tool exposes exactly `source-read`, `source-write`, `issues-read`, + `issues-write` — one per canonical scenario scope — so UC-1 discovery yields *exactly* the four + scenario tool scopes and nothing else. The production 44-tool catalog would yield 44 fine-grained + scopes that do not match the scenario truth table. +2. **Stub handlers, no GitHub.** Tool **calls** are trivial no-op/echo stubs; there are no real GitHub + API calls, no PAT, and no MitM. Phase 1 drives no live traffic, so nothing calls the tools. +3. **Workload name `github-tool`, not `github-tool-mcp`.** See §6 (naming invariants). The scenario + entity id is `github-tool`, which keeps UC-1 identity resolution clean. + +In short: this is a **purpose-built, minimal stand-in** used ONLY to make UC-1's discovery deterministic +for the phase-1 demo. It is distinct from — and not a replacement for — the production 44-tool +`github-tool` referenced by [`github-agent.md`](github-agent.md). When later phases wire live traffic, +they use the production tool; this stand-in serves the discover-and-evaluate loop only. + +--- + +## 3. The four MCP tools (verbatim scenario descriptions) + +Exactly four tools. Names are chosen so UC-1 derives `github-tool.`; descriptions are copied +**verbatim** from `scenario.py::TOOL_SCOPES` / the policy-pipeline spec's *Tool scopes* section (each is +≤255 chars, authored to be generic/keyword-free so the PRB maps them correctly — do not paraphrase): + +| Tool name | Description (verbatim) | +|---|---| +| `source-read` | `Read source repository contents: file listings and file bodies. Read-only.` | +| `source-write` | `Create, modify, or delete source repository contents; commit file changes.` | +| `issues-read` | `Read issues and their comment threads. Read-only.` | +| `issues-write` | `Create and update issues: open, edit, comment, and close.` | + +Each tool has a **minimal / empty** `inputSchema` (`{"type": "object", "properties": {}}`) — phase 1 +inspects only `name` + `description`, never arguments. + +> **Invariant:** the description strings here must remain byte-identical to `TOOL_SCOPES` in +> `scenario.py`. UC-1 copies them into `ScopeDefinition.description`, and the PRB LLM maps roles→scopes +> from that text; a drift would change the provisioned scope descriptions and could move the truth +> table. If `TOOL_SCOPES` changes, change this table with it. + +--- + +## 4. MCP server (minimal, real, deployable) + +A tiny, self-contained MCP server that serves **JSON-RPC 2.0 over HTTP at path `/mcp`** and correctly +answers `tools/list` with the four tools of §3. + +- **Transport / protocol:** JSON-RPC 2.0 over HTTP `POST /mcp` (the MCP streamable-HTTP endpoint + convention `analyze_tool` posts to). It must respond to a `tools/list` request with the four tools + (each `name` + `description` + minimal `inputSchema`). Implementing the MCP `initialize` handshake is + fine but not required by UC-1, which posts `tools/list` directly. +- **Stack:** a small Python server, consistent with the repo (Python 3.12). Either the official MCP + Python SDK in streamable-HTTP mode, or a hand-rolled minimal ASGI/HTTP handler for `POST /mcp` — pick + whichever keeps the image smallest and the `tools/list` contract obvious. No CrewAI, no A2A, no LLM. +- **Tool registry:** a single editable constant — the four `(name, description, inputSchema)` tuples of + §3 — so the list stays legible and auditable against `scenario.py::TOOL_SCOPES`. +- **Tool-CALL handlers:** trivial **no-op / echo stubs**. A `tools/call` for any of the four returns a + stub result (e.g. a text content block echoing the tool name + received arguments, or a fixed + `"stub: not implemented in phase-1 demo"` message). They perform **no** GitHub work. This is + acceptable because phase 1 drives no live traffic — but the endpoint must still be a **real, + deployable MCP server** that answers `tools/list`. +- **Location:** self-contained under `aiac/demo/tools/github_tool/`, mirroring the agent's + `aiac/demo/agents/github_agent/` layout. Ships its own `Dockerfile`, dependency manifest, and the + `k8s/` manifests of §7. +- **Listen:** binds `0.0.0.0` on a container `PORT` (see §5) and serves `/mcp`. + +``` + UC-1 analyze_tool ──(JSON-RPC POST /mcp: tools/list)──► github-tool (:PORT) ──► 4 tools + (resolves http://github-tool.team1.svc.cluster.local:{first-port}/mcp) + (phase 1: no tools/call traffic — stub handlers are never exercised by the agent) +``` + +--- + +## 5. Configuration + +Minimal env, adapted to the tiny server: + +| Variable | Description | Default | +|---|---|---| +| `PORT` | HTTP listen port serving `/mcp` | `9090` | +| `LOG_LEVEL` | Log level | `INFO` | + +No Keycloak, LLM, GitHub PAT, JWKS, or audience config — the demo tool performs no auth and no upstream +calls (contrast the github-issue demo's `github-tool`, which needs PATs + issuer/JWKS/audience). Any +auth enforcement in front of this tool is AuthBridge/MitM's job in later phases, not this container's. + +--- + +## 6. Naming & consistency invariants + +State these explicitly; UC-1 identity resolution depends on all of them holding: + +- **One name everywhere:** workload name == K8s `Deployment` name == K8s `Service` name == the + `AgentRuntime` `targetRef.name` == the Keycloak client's workload segment == **`github-tool`**. + Consequently: + - the operator sets the Keycloak client `client.name = "team1/github-tool"`, so `classify_service` + splits it into `namespace=team1`, `workload_name=github-tool`; + - `analyze_tool` resolves the endpoint to + `http://github-tool.team1.svc.cluster.local:{first-port}/mcp`; + - UC-1 derives scopes `github-tool.source-read`, `github-tool.source-write`, + `github-tool.issues-read`, `github-tool.issues-write`. +- **Keycloak client id == `github-tool`** — matches `scenario.py`'s `TOOL_ID = "github-tool"`, so the + scenario/integration test and this deployed tool agree on the entity id. +- **Divergence from the github-issue demo (`github-tool-mcp`) is intentional.** The github-issue demo + and [`github-agent.md`](github-agent.md) use a Service named **`github-tool-mcp`** on port `9090`. + This demo names both the workload and the Service **`github-tool`** because: + - the canonical scenario entity id is `github-tool` (`scenario.py` `TOOL_ID`), and UC-1 derives the + scope prefix from the **workload name** — a `github-tool-mcp` workload would yield + `github-tool-mcp.source-read`, not the canonical `github-tool.source-read`; + - `analyze_tool` relies on the operator convention **Service name == workload name**, so the Service + must also be `github-tool` for endpoint resolution to work off `client.name`. + Net: keeping a single `github-tool` name across workload / Service / Keycloak client keeps UC-1 + resolution clean and the derived scopes canonical. (The production live-traffic path continues to use + `github-tool-mcp:9090/mcp` per the agent spec; the two are separate deployments.) + +--- + +## 7. Deployment (aiac level) + +Manifests live under `aiac/demo/tools/github_tool/k8s/`, adapted from the sibling agent's §7 and the +github-issue demo's `github-tool-deployment.yaml`. Namespace **`team1`** (installer-provided +ConfigMaps/secrets assumed present), consistent with the agent spec. + +- **`github-tool-deployment.yaml`** — `Deployment` + `Service` + `AgentRuntime`: + - **`Deployment`** named `github-tool`. Container port serves `/mcp` (default `9090`). Env `PORT`, + `LOG_LEVEL`. Image `github-tool:latest`, `imagePullPolicy: IfNotPresent` (kind-load; name is a + documented knob). No GitHub PAT / issuer / JWKS / audience env (unlike the github-issue tool). + - **Pod label `kagenti.io/type: tool`** — this is what `classify_service` reads. Applied by the + operator via the `AgentRuntime` (see below); relying on the operator to stamp it, rather than + hand-setting it, keeps it consistent with the operator's own discriminator. + - **`Service`** named `github-tool` (ClusterIP), selecting the Deployment's pods. Its **first port** + maps to the container's `/mcp` port (e.g. `port: 9090 → targetPort: 9090`). `analyze_tool` uses the + Service's **first** port, so keep `/mcp`'s port first. + - **Service label `protocol.kagenti.io/mcp` MUST be present** — a **deploy-time prerequisite** for + `analyze_tool` (the operator does **not** stamp it; `analyze_tool` returns `502` if absent). Set it + explicitly on the Service metadata (e.g. `protocol.kagenti.io/mcp: "true"`). + - **`AgentRuntime{ type: tool, targetRef: { apiVersion: apps/v1, kind: Deployment, name: github-tool } }`** + — enrolls the workload so the kagenti-operator applies `kagenti.io/type=tool` to the pod and + registers a Keycloak client with `client.name = "team1/github-tool"`. Unlike the github-issue demo + (which omits `kagenti.io/type` to skip AuthBridge entirely), this demo **needs** `type: tool` so the + pod carries `kagenti.io/type=tool` for UC-1 `classify_service`. + +- **No `configmaps.yaml` needed here** — the tool has no `authbridge-config` / `authproxy-routes` + dependency (it neither validates inbound JWTs nor does outbound token exchange in phase 1). The agent + spec's `authproxy-routes` still targets the **production** `github-tool-mcp` host and is unrelated to + this stand-in. + +- **Prerequisite (reused, not created here):** a running Kagenti cluster with the kagenti-operator + (Keycloak realm as configured by the installer, namespace `team1`), so the `AgentRuntime` is + reconciled into a Keycloak client + pod label. + +**Wiring invariant:** `AgentRuntime.targetRef.name` == `Deployment` name == `Service` name == +`github-tool`; the `Service` carries the `protocol.kagenti.io/mcp` label and exposes `/mcp` as its first +port; the operator-applied pod label is `kagenti.io/type=tool`; the operator-registered Keycloak +`client.name` is `team1/github-tool`. + +--- + +## 8. Verification + +**Local (no cluster — primary gate):** +1. `cd aiac/demo/tools/github_tool` and build: `podman build -t github-tool:latest .`. +2. Run the container (`-e PORT=9090 -p 9090:9090`), then POST a JSON-RPC `tools/list` to `/mcp` and + confirm the four tool names: + ```bash + curl -s -X POST localhost:9090/mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' \ + | jq -r '.result.tools[].name' | sort + # → issues-read, issues-write, source-read, source-write + ``` +3. Confirm each returned tool's `description` is byte-identical to the matching `TOOL_SCOPES` entry in + `scenario.py` (e.g. `jq '.result.tools[] | {name, description}'`). + +**Cluster (HITL — live Kagenti + Keycloak + operator):** +4. `kind load docker-image github-tool:latest --name kagenti`. +5. `kubectl apply -f k8s/github-tool-deployment.yaml`. +6. Confirm the operator applied the pod label: `kubectl get pod -l app=github-tool -n team1 + -o jsonpath='{.items[0].metadata.labels.kagenti\.io/type}'` → `tool`. +7. Confirm the Service carries the MCP label: `kubectl get svc github-tool -n team1 + -o jsonpath='{.metadata.labels.protocol\.kagenti\.io/mcp}'` → present. +8. Confirm the operator registered a Keycloak client with `client.name = "team1/github-tool"` (via the + Keycloak admin API / IdP `Configuration` library). +9. From inside the cluster (or via `kubectl port-forward svc/github-tool 9090:9090 -n team1`), POST + `tools/list` to `http://github-tool.team1.svc.cluster.local:9090/mcp` and confirm the four tools — + the exact call UC-1 `analyze_tool` makes. +10. (End-to-end) Trigger UC-1 onboarding for `github-tool` and confirm it provisions scopes + `github-tool.source-read` / `github-tool.source-write` / `github-tool.issues-read` / + `github-tool.issues-write`, and writes **no rules for the tool alone** (per the phase-1 acceptance + criteria). + +--- + +## 9. Out of scope + +- **Real GitHub API calls / real source & issue operations** — tool-call handlers are stubs; there is + no GitHub PAT and no upstream. +- **Auth enforcement inside the tool** — no inbound JWT validation, no outbound token exchange, no + audience/scope checks. AuthBridge / the MitM handle enforcement in later phases. +- **The production 44-tool federation** — this stand-in exposes only the four canonical scenario tools + (see §2); the real `github-tool` (`github-tool-mcp:9090/mcp`) is unchanged and used by the live-traffic + path. +- **Being agent-executable** — phase 1 drives no A2A traffic, so the tool is discovered and evaluated but + never invoked by `github-agent`. +- **Any changes to the AIAC pipeline, UC-1, `policy-pipeline.md`, or the integration test** — this tool + is an input to UC-1 discovery, not a change to it. +- **Building this tool into `agent-examples` CI** — aiac/demo images build independently (as with the + agent spec). diff --git a/aiac/docs/specs/event-broker-redhat-amq-evaluation.md b/aiac/docs/specs/event-broker-redhat-amq-evaluation.md new file mode 100644 index 00000000..bd290041 --- /dev/null +++ b/aiac/docs/specs/event-broker-redhat-amq-evaluation.md @@ -0,0 +1,197 @@ +# Event Broker Alternatives: Red Hat AMQ Evaluation (Greenfield, Technology-Neutral) + +## Abstract + +This document evaluates Red Hat's AMQ messaging product family as a candidate for the AIAC Event +Broker role. The evaluation assumes a greenfield implementation — no part of the AIAC system +exists yet — and uses exclusively functional requirements derived from the PRD's use cases and +architectural principles. All technology-specific constraints (protocol names, client library +names, image identifiers, API call names) have been removed. The question asked is: can this +product satisfy what the system must do? + +**Conclusion:** AMQ Broker (Artemis) satisfies all functional requirements and is a technically +sound candidate. Remaining differentiators from NATS JetStream are operational: footprint, +configuration surface, and protocol fit for the AIAC event volume. AMQ Streams retains two +structural incompatibilities that cannot be resolved without changing the PRD's routing design. +AMQ Interconnect is disqualified by the durability requirement. + +--- + +## Functional Requirements + +Derived from the PRD's use cases and architectural decisions. All technology-specific language +has been removed — requirements state what the system must do, not how. + +| ID | Functional requirement | PRD source | +|----|------------------------|------------| +| FR1 | Events must survive the Agent pod going down and be re-delivered when it restarts — no silent event loss on transient consumer failure | §5 arch decisions, §7.4 | +| FR2 | Each event must be processed by exactly one Agent instance across competing replicas (work-queue semantics) | §7.4, §5 arch decisions | +| FR3 | Failed event processing must be retried a bounded number of times, then moved to a dead-letter destination for operator inspection — a persistently broken event must not block the queue | §7.4, §9 | +| FR4 | The broker must support per-entity event addressing: a new Keycloak client and a new role each produce distinct, independently routable events carrying the entity ID — the consumer must be able to subscribe to a wildcard that covers all entity-scoped events without inspecting message payloads | §7.4, §7.5, §7.8, §5 ("no business logic lives in the consumer") | +| FR5 | The Agent (Python 3.12, FastAPI, asyncio) must be able to consume events asynchronously as a background task; processing must complete before the event is acknowledged | §7.5, §10 | +| FR6 | The RAG Ingest Service (Python 3.12, FastAPI) must be able to publish events to the broker | §7.7 | +| FR7 | The Keycloak SPI (Java) must be able to publish events to the broker | §7.8 | +| FR8 | The broker must be deployable without authentication; Kubernetes ClusterIP network isolation is the intended access control boundary | §7.4, §10 | +| FR9 | The broker must run as a single Kubernetes pod with minimal configuration overhead; no Operator dependency | §8 | +| FR10 | An init container must be able to provision required broker resources idempotently at startup and health-check the broker before the Agent container starts | §5 arch decisions, §9 | + +--- + +## Candidate 1 — AMQ Broker (Apache ActiveMQ Artemis) + +**What it is:** A full-featured message broker with persistent storage (NIO journal), native +dead-letter support, and competing consumer semantics via `anycast` routing. Supports AMQP 1.0, +STOMP, MQTT, and OpenWire. Python client: `python-qpid-proton`. Java client: Artemis JMS or +Qpid Proton Java. + +### Assessment + +| Req | Requirement | Verdict | Detail | +|-----|-------------|---------|--------| +| **FR1** | Events survive Agent pod restart | **Pass** | NIO journal persists messages to disk. Consumer reconnects and pending messages re-deliver automatically | +| **FR2** | Work-queue: one consumer per event | **Pass** | `anycast` routing type delivers each message to exactly one consumer across competing receivers — equivalent to work-queue semantics | +| **FR3** | Bounded retry + dead-letter | **Pass** | Native `dead-letter-address` + `max-delivery-attempts` configuration in `broker.xml`. Max attempts is a simple numeric setting | +| **FR4** | Per-entity dynamic addressing | **Pass** | Artemis supports hierarchical address wildcards (`aiac.apply.#` for multi-level, `*` for single-level). Addresses are auto-created on first publish when auto-create address policy is enabled — a new entity ID at runtime creates a new routable address automatically. A consumer on the wildcard address receives all matching events without payload inspection | +| **FR5** | Python asyncio consumer, ack-after-processing | **Pass** | `python-qpid-proton` provides asyncio consumer with manual message settlement. Ack-after-processing is the standard AMQP disposition model — the consumer accepts or rejects a message after handler completion | +| **FR6** | Python publisher | **Pass** | `python-qpid-proton` sender is straightforward | +| **FR7** | Java publisher (Keycloak SPI) | **Pass** | Artemis JMS client and Qpid Proton Java are both mature and well-supported | +| **FR8** | No-auth deployment | **Pass** | Configurable: `false` in `broker.xml`. One XML element; low friction | +| **FR9** | Single pod, no Operator, low footprint | **Partial** | Artemis runs standalone without an Operator. However, a `broker.xml` configuration file must be authored and mounted via ConfigMap — there is no single-flag enablement. RAM footprint is ~256–300 MB. For AIAC's event volume (tens of lifecycle events per day) this is significantly over-provisioned | +| **FR10** | Init container idempotent provisioning | **Pass** | Addresses and queues can be pre-declared in `broker.xml` — on broker start they exist before any consumer connects, making provisioning declarative and idempotent by design. Alternatively, the Artemis management REST API supports idempotent address creation at runtime. Health-checking via HTTP management endpoint | + +### Residual concern + +**FR9** is the only meaningful remaining differentiator. Artemis is operationally heavier than +warranted for the AIAC use case: + +- `broker.xml` must be authored, maintained, and mounted as a ConfigMap in the Event Broker + deployment manifest +- RAM footprint (~300 MB) is ~15× higher than a comparable lightweight broker for an event bus + that carries a handful of events per day +- Artemis was designed for enterprise JMS workloads with high message throughput and complex + routing topologies; the AIAC Event Broker is a simple lifecycle event bus + +These are operational trade-offs, not functional failures. The system works correctly with +Artemis — it is simply heavier than necessary. + +--- + +## Candidate 2 — AMQ Streams (Apache Kafka / Strimzi) + +**What it is:** A Kafka-based event streaming platform managed by the Strimzi Operator on +Kubernetes. Python client: `aiokafka` or `confluent-kafka-python`. Java client: Kafka producer. + +### Assessment + +| Req | Requirement | Verdict | Detail | +|-----|-------------|---------|--------| +| **FR1** | Events survive Agent pod restart | **Pass** | Kafka topic retention provides message replay for a configured consumer group | +| **FR2** | Work-queue: one consumer per event | **Partial** | Kafka consumer groups deliver one message per partition. Work-queue semantics emerge only when partition count equals consumer count — it is not a single-setting guarantee and requires careful partition design | +| **FR3** | Bounded retry + dead-letter | **Fail** | Kafka has no native dead-letter queue or `max-delivery-attempts` concept. A DLQ must be implemented as application code: a separate retry topic, a retry consumer service, and a DLQ topic. This is infrastructure the PRD expects the broker to provide natively | +| **FR4** | Per-entity dynamic addressing | **Fail** | Kafka topics are static, pre-declared strings. A runtime-generated entity ID cannot become a new topic without administrator intervention. Per-entity routing collapses to per-type topics (e.g. one topic for all role events), requiring the consumer to inspect message payloads to identify the target entity. The PRD (§5) mandates a consumer that carries no business logic — payload-based routing contradicts this | +| **FR5** | Python asyncio consumer, ack-after-processing | **Pass** | `aiokafka` is a mature asyncio Kafka client; manual commit after processing is idiomatic | +| **FR6** | Python publisher | **Pass** | `aiokafka` or `confluent-kafka-python` | +| **FR7** | Java publisher (Keycloak SPI) | **Pass** | Kafka Java producer is the most mature producer client available | +| **FR8** | No-auth deployment | **Partial** | Kafka supports no-auth, but Strimzi configures mutual TLS by default; explicit opt-out via `KafkaListeners` spec is required | +| **FR9** | Single pod, no Operator | **Fail** | Strimzi Operator is the practical Kubernetes deployment path for Kafka. A single-node KRaft deployment without Operator is possible but not maintainable. Minimum footprint: ~1 GB RAM plus Operator pods | +| **FR10** | Init container idempotent provisioning | **Partial** | Kafka `AdminClient` supports idempotent topic creation. Feasible but requires a Kafka bootstrap connection and more setup than a simple health check | + +### Structural incompatibilities + +FR3 and FR4 fail regardless of implementation approach: + +**FR3 — no native DLQ:** Building bounded retry with dead-lettering requires a separate retry +consumer service, a retry topic, and a DLQ topic. This is application infrastructure, not broker +configuration. The PRD (§7.4) treats dead-lettering as a broker-level property. + +**FR4 — static topics vs dynamic addressing:** The PRD's per-entity event addressing is a +first-class routing feature. Collapsing `aiac.apply.role.{id}` to a static topic and +embedding the entity ID in the message payload changes the consumer contract and adds routing +logic to what §5 explicitly defines as a thin adapter with no business logic. This is a +PRD-level design change, not an implementation detail. + +--- + +## Candidate 3 — AMQ Interconnect (Apache Qpid Dispatch Router) + +**What it is:** A stateless AMQP 1.0 message router. Routes messages between endpoints with no +on-disk persistence. + +### Assessment + +| Req | Requirement | Verdict | Detail | +|-----|-------------|---------|--------| +| **FR1** | Events survive Agent pod restart | **Hard fail** | Interconnect is stateless by design. Messages in-flight when the Agent pod restarts are permanently lost | + +FR1 is an immediate disqualifier. UC-1 and UC-2 both depend on events surviving transient Agent +failures — this is a first-class architectural decision in §5. AMQ Interconnect can complement +AMQ Broker as a routing mesh but cannot serve as the AIAC Event Broker independently. + +--- + +## Consolidated Scorecard + +| Requirement | PRD anchor | NATS JetStream | AMQ Broker | AMQ Streams | AMQ Interconnect | +|-------------|------------|:--------------:|:----------:|:-----------:|:----------------:| +| FR1 Durable replay on pod restart | §5, §7.4 | ✅ | ✅ | ✅ | ❌ | +| FR2 Work-queue / competing consumers | §7.4, §5 | ✅ | ✅ | ⚠️ | ⚠️ | +| FR3 DLQ after bounded retries | §7.4, §9 | ✅ | ✅ | ❌ | ❌ | +| FR4 Per-entity dynamic addressing, wildcard consumer | §7.4, §7.5, §7.8, §5 | ✅ | ✅ | ❌ | ⚠️ | +| FR5 Python asyncio consumer, ack-after-processing | §7.5, §10 | ✅ | ✅ | ✅ | ❌ | +| FR6 Python publisher | §7.7 | ✅ | ✅ | ✅ | ✅ | +| FR7 Java publisher (Keycloak SPI) | §7.8 | ✅ | ✅ | ✅ | ✅ | +| FR8 No-auth viable | §7.4, §10 | ✅ | ✅ | ⚠️ | ✅ | +| FR9 Single pod, no Operator, low footprint | §8 | ✅ | ⚠️ | ❌ | ✅ | +| FR10 Init container idempotent provisioning | §5, §9 | ✅ | ✅ | ⚠️ | ❌ | + +✅ Satisfies requirement · ⚠️ Achievable with configuration trade-off · ❌ Cannot satisfy without design change + +--- + +## Comparative Analysis: AMQ Broker vs NATS JetStream + +With all technology-specific constraints removed, both candidates satisfy every functional +requirement. The decision reduces to operational fit. + +| Dimension | NATS JetStream | AMQ Broker (Artemis) | +|-----------|----------------|----------------------| +| FR1–FR3 delivery semantics | Native, zero-config | Native, `broker.xml` config | +| FR4 per-entity addressing | Hierarchical subjects, auto-routed | Hierarchical addresses, auto-create policy | +| FR5 Python asyncio consumer | Idiomatic, well-documented | Functional, lower-level API | +| FR7 Java publisher | NATS Java client | Artemis JMS / Qpid Proton Java | +| FR8 no-auth | Default, no config needed | One XML element in `broker.xml` | +| FR9 footprint | ~20 MB RAM, single flag | ~300 MB RAM, `broker.xml` ConfigMap | +| FR10 broker provisioning | Single API call, runtime | Declarative via `broker.xml` (no runtime call needed) | +| Protocol lineage | Purpose-built for microservice pub/sub | Enterprise JMS / AMQP 1.0 interoperability | +| Python ecosystem depth | Large community, extensive examples | Smaller community, sparser asyncio docs | +| Design envelope | Lightweight durable pub/sub | High-throughput enterprise queuing | + +The functional gap is closed. AMQ Broker is a technically valid choice. The remaining difference +is that it carries more operational weight (configuration, RAM, XML) than the AIAC Event Broker +role requires. + +--- + +## Conclusion + +**AMQ Broker (Artemis)** satisfies all ten functional requirements in a greenfield build. It +is a technically sound candidate. The single remaining concern — FR9, operational footprint — is +a trade-off, not a disqualifier: Artemis is ~15× heavier in RAM than warranted for an event bus +that carries tens of lifecycle events per day, and its XML-based configuration adds overhead +absent from simpler brokers. + +**AMQ Streams** retains two hard disqualifications regardless of implementation approach: no +native DLQ (FR3) and inability to support dynamic per-entity addressing (FR4). Resolving either +would require changing the PRD's routing design. + +**AMQ Interconnect** is disqualified by FR1. + +**Selection guidance:** + +| Condition | Recommended choice | +|-----------|-------------------| +| No pre-existing AMQ infrastructure; team is starting fresh | Lightweight broker matched to event volume — NATS JetStream | +| Team already operates AMQ Broker for other workloads | AMQ Broker — eliminate a dependency, reuse existing expertise | +| Platform standardised on AMQP 1.0 across services | AMQ Broker — protocol consistency has value | +| Minimising operational surface and configuration files is a priority | NATS JetStream | +| Red Hat support contract covers messaging infrastructure | AMQ Broker | diff --git a/aiac/docs/specs/integration-test/pdp-policy-writer.md b/aiac/docs/specs/integration-test/pdp-policy-writer.md new file mode 100644 index 00000000..53a49c75 --- /dev/null +++ b/aiac/docs/specs/integration-test/pdp-policy-writer.md @@ -0,0 +1,173 @@ +# Integration Test: PDP Policy Writer (OPA) — `generate_rego.py` + +> **One spec among several.** This document specifies a **single** integration test. +> Integration-test specs live **one spec per test** under `docs/specs/integration-test/` +> (a sibling of `components/`), and the master PRD's *Integration test specifications* section +> ([../PRD.md](../PRD.md)) is the index of them. This is the **PDP Policy Writer (OPA)** integration +> test — not the definition of integration testing in general, and not the only integration-test PRD. + +## Location +`aiac/test/pdp/policy/generate_rego.py` + +## Description + +A standalone launcher script that exercises the PDP Policy Writer (OPA) **filesystem stub** +end-to-end and leaves the generated Rego on disk for a human to eyeball. It is **not** a pytest +test, **not** part of CI, and **not** marked `@pytest.mark.integration` — it is run by hand when an +operator wants to see the actual `.rego` output for a known scenario. + +The script drives the service through its real HTTP surface using the real client library, so it +covers the whole path: `PolicyModel` → HTTP → ASGI app → Rego generator → files on disk. Nothing is +mocked; the only shortcut is that the OPA target is the filesystem stub +([../components/pdp-policy-writer-opa.md](../components/pdp-policy-writer-opa.md) §1.14) rather than +the Kubernetes-CR implementation, so the output is `.rego` files instead of a patched +`AuthorizationPolicy` CR. + +### What it does + +1. Choose a local `REGO_OUTPUT_DIR` (a known directory the operator can inspect afterward). +2. Launch `aiac.pdp.service.policy.opa.main:app` as a **`uvicorn` subprocess** (no Docker), passing + `REGO_OUTPUT_DIR` and `PORT` in its environment. +3. Poll `GET /health` until it returns `200 {"status": "ok"}` (the stub reports healthy once + `REGO_OUTPUT_DIR` exists and is writable), with a bounded timeout. +4. Build the fixed `PolicyModel` scenario (below) and apply it via + `aiac.pdp.policy.library.api.apply_policy` — a real `POST /policy` over HTTP. +5. Terminate the `uvicorn` subprocess. +6. Print the `REGO_OUTPUT_DIR` path so the operator knows where to look. + +**Write-only.** The script performs no read-back and makes **no assertions**. Verification is +manual: the operator opens the generated `.rego` files and confirms they match the package shapes in +[../components/pdp-policy-writer-opa.md](../components/pdp-policy-writer-opa.md) +(§ *Rego package structure*). There is no pass/fail exit contract beyond the script running to +completion. + +## Scenario + +A single agent, fixed so the generated Rego is reproducible and reviewable by inspection. The values +below are the canonical worked example referenced by +[../components/pdp-policy-writer-opa.md](../components/pdp-policy-writer-opa.md). + +| Element | Value | +|---------|-------| +| Agent | `github-agent` | +| Agent roles | `source-helper`, `issues-helper` | +| Agent scopes | `source-access`, `issues-access` | +| Subject (user) roles | `developer`, `tester` | +| Tool | `github-tool` | +| Tool scopes | `source-read`, `source-write`, `issues-read`, `issues-write` | +| Calling service (source) | none | + +Role → access, as encoded by the model's inbound and outbound rules: + +- `developer` — source read/write, issues read. +- `tester` — issues read/write. + +This user→tool access is encoded in the model's `outbound_subject_rules` (`(user_role, tool_scope)` +pairs), which the outbound package renders as `outbound_subject_role_scopes`. The model's +`inbound_rules` (user→agent-scope) and `outbound_rules` (agent-role→tool-scope) are unchanged. + +Applying this `PolicyModel` produces exactly two files in `REGO_OUTPUT_DIR`: + +- `github_agent.inbound.rego` — package `authz.github_agent.inbound` +- `github_agent.outbound.rego` — package `authz.github_agent.outbound` + +Both must match the **ID-only** package shapes in +[../components/pdp-policy-writer-opa.md](../components/pdp-policy-writer-opa.md): input is IDs only +(`{subject, source}` inbound, `{subject, target}` outbound); all role/scope maps are embedded in the +package; the inbound gate is subject-mandatory + source-optional; the outbound gate requires both +subject and agent to pass, but its **subject** gate is now user→**tool** — it reads +`outbound_subject_role_scopes` (grouped from `outbound_subject_rules`) and matches against +`target_scopes[input.target]`, distinct from the inbound user→agent gate — while `target_ok` +(agent→tool, from `agent_roles` × `agent_role_scopes`) is unchanged; and `target_scopes` is emitted +verbatim (target id → scopes, no inversion). Because the input carries no per-request scope, the +decision is coarse — a principal passes on having access to **at least one** relevant scope. + +The `PolicyModel` / `AgentPolicyModel` / `PolicyRule` objects come from `aiac.policy.model.models` +([../components/policy-model.md](../components/policy-model.md)); the script constructs them in +Python rather than reading them from Keycloak. + +## Configuration (env) + +| Variable | Purpose | Default | +|----------|---------|---------| +| `AIAC_PDP_POLICY_URL` | Base URL the library client posts to; must point at the launched stub | `http://127.0.0.1:7072` | +| `REGO_OUTPUT_DIR` | Directory the stub writes `.rego` files to; passed to the subprocess and printed at the end | operator-chosen local dir | +| `PORT` | Port the `uvicorn` subprocess binds; must agree with the host/port in `AIAC_PDP_POLICY_URL` | `7072` | + +`PORT` and `AIAC_PDP_POLICY_URL` must be kept consistent — the client posts to the URL, the +subprocess listens on the port. + +## Runbook + +```bash +.venv/bin/python test/pdp/policy/generate_rego.py +# then inspect the printed REGO_OUTPUT_DIR, e.g.: +# github_agent.inbound.rego +# github_agent.outbound.rego +``` + +To pin the output location and port explicitly: + +```bash +REGO_OUTPUT_DIR=/tmp/aiac-rego PORT=7072 \ +AIAC_PDP_POLICY_URL=http://127.0.0.1:7072 \ + .venv/bin/python test/pdp/policy/generate_rego.py +``` + +## Testing Decisions + +- **Highest seam available.** The test drives the service through its real HTTP boundary + (`AIAC_PDP_POLICY_URL`) using the real client library + ([../components/library-pdp-policy.md](../components/library-pdp-policy.md), `aiac.pdp.policy.library.api`), + and observes the real filesystem output. It asserts on **external behavior** (files produced on + disk), never on internal generator functions. +- **Launch as a `uvicorn` subprocess.** The script spawns + `uvicorn aiac.pdp.service.policy.opa.main:app` as a child process (no Docker), polls `GET /health` + before applying the model, and terminates the subprocess at the end. This exercises the full + HTTP + ASGI stack the way a caller would, and keeps the service lifecycle self-contained. +- **Write-only, human-verified.** The value of this test is the concrete `.rego` output for a known + scenario — so a reviewer can confirm the ID-only redesign renders correctly. It intentionally + makes no automated assertions; the generator's assertable behavior is covered by the OPA + service/`rego.py` unit tests. +- **Prior art.** The stub itself and its unit tests (`test/pdp/service/policy/opa/`, + covering `main.py` and `rego.py`) verify endpoint and rendering behavior automatically; this + launcher complements them with an eyeball-the-output workflow. The live-Keycloak pytest + integration tests (issue `testing/5.1-integration-tests.md`) are the marker-gated counterpart for + the read-side services. + +## Relationship to other integration tests + +This is **one** integration-test spec among several indexed by the master PRD +([../PRD.md](../PRD.md), § *Integration test specifications*). It is distinct from the +**live-Keycloak pytest integration tests**, which are a different flavor — `@pytest.mark.integration`, +run in/near CI against a live Keycloak/NATS, asserting on typed responses — tracked by issue +`testing/5.1-integration-tests.md`. This launcher, by contrast, is standalone, write-only, and +manually inspected. + +For the full identity→policy pipeline (Keycloak → PRB → PCE → OPA) — which drives the same +`github-agent` scenario end to end through the real Policy Computation Engine rather than a +hand-built `PolicyModel` — see [policy-pipeline.md](policy-pipeline.md). + +Tracking issue for this test: `testing/5.2-pdp-writer-integration-test.md`. + +## Out of Scope + +- **The Rego generator implementation** — package rendering, `_slugify`, and the ID-only gate logic + are specified by [../components/pdp-policy-writer-opa.md](../components/pdp-policy-writer-opa.md) + and covered by the OPA service unit tests, not here. +- **The canonical policy model** — `PolicyModel` / `AgentPolicyModel` / `PolicyRule` shapes and + semantics belong to [../components/policy-model.md](../components/policy-model.md). +- **The Kubernetes-CR Policy Writer (1.13)** — this test targets the filesystem **stub** (1.14). The + CR-backed implementation and its `AuthorizationPolicy` schema are out of scope. +- **Live-Keycloak integration** — the marker-gated pytest integration tests (issue + `testing/5.1-integration-tests.md`). +- **Automated pass/fail** — no assertions, no CI wiring, no `@pytest.mark.integration`. + +## Further Notes + +- Depends on the launcher script itself (created separately) and the OPA filesystem stub (issue + `pdp-policy-writer/1.14-pdp-policy-writer-opa-stub.md`, status: done). +- The scenario is deliberately fixed. If the canonical worked example in + [../components/pdp-policy-writer-opa.md](../components/pdp-policy-writer-opa.md) changes, update the + scenario table here to match so the generated Rego stays reviewable against a single source of + truth. diff --git a/aiac/docs/specs/integration-test/policy-pipeline.md b/aiac/docs/specs/integration-test/policy-pipeline.md new file mode 100644 index 00000000..01bc916c --- /dev/null +++ b/aiac/docs/specs/integration-test/policy-pipeline.md @@ -0,0 +1,470 @@ +# Integration Test: policy-pipeline — `test_policy_pipeline.py` + +> **One spec among several.** This document specifies a **single** integration test. +> Integration-test specs live **one spec per test** under `docs/specs/integration-test/` +> (a sibling of `components/`), and the master PRD's *Integration test specifications* section +> ([../PRD.md](../PRD.md)) is the index of them. This is the **policy-pipeline** integration test — +> the full identity→policy pipeline — not the definition of integration testing in general, and not +> the only integration-test PRD. + +## Location +`aiac/test/integration/test_policy_pipeline.py` — a pytest module marked `@pytest.mark.integration`. +It imports two shared modules: `aiac/test/integration/scenario.py` — the canonical `github-agent` +scenario as pure data (one of the role→access fact sources the *Further Notes* mandate — the pair-lists, +alongside the *Scenario* table and both `policy.md` variants) — and `aiac/test/integration/launcher.py` +— the shared `uvicorn` subprocess-lifecycle helpers. It also ships a new +`aiac/test/integration/probe.rego` — a small standalone Rego module used only as the outbound +verification query (see *[What it does](#what-it-does)*). The `5.2` launcher +`test/pdp/policy/generate_rego.py` was refactored onto the same `launcher.py` + `scenario.py` so the two +launchers cannot drift. + +## Description + +A `@pytest.mark.integration` test that drives the **whole identity→policy pipeline** — +**Keycloak → PRB → PCE → OPA Policy Writer** — end-to-end, then **asserts** the generated Rego decides +correctly by running the standalone `opa eval` binary as its verification oracle. The `.rego` files are +still left on disk per policy variant, so the test doubles as the eyeball workflow: running the test +*is* the eyeball. There is no separate standalone script. + +The generated Rego is the **artifact under test** — the LLM/PCE that produced it might be wrong — so the +test never trusts it. Instead it feeds `opa eval` requests derived from the **scenario spec (the +intended policy)** and asserts the real Rego admits/denies each one as the scenario truth table +requires. A mismatch fails the test and names the exact cell. + +This is the same *flavor* as the PDP Policy Writer launcher +([pdp-policy-writer.md](pdp-policy-writer.md), issue `testing/5.2-pdp-writer-integration-test.md`) but +**broader**: where `5.2` hand-builds a `PolicyModel` in Python and POSTs it to the OPA stub — +deliberately bypassing Keycloak, the PRB, and the PCE — this test provisions a **live Keycloak** realm, +calls the real **Policy Rules Builder (PRB)** to map roles→scopes with a real LLM, then calls the real +**Policy Computation Engine (PCE)** to build the `PolicyModel` and drive the **OPA Policy Writer** to +emit Rego. Nothing is mocked; the only shortcut is that the OPA target is the filesystem stub +([../components/pdp-policy-writer-opa.md](../components/pdp-policy-writer-opa.md) §1.14) rather than the +Kubernetes-CR implementation, so the output is `.rego` files instead of a patched +`AuthorizationPolicy` CR — identical to `5.2`. + +Because it needs a live Keycloak and a real LLM, it is `@pytest.mark.integration` and stays out of the +default unit-test run (`-m "not integration"`); it additionally `pytest.skip`s when no `opa` binary is +found. + +### What it does + +The pipeline (provision → PRB → PCE → OPA) is driven **once per `policy.md` variant** — `explicit` and +`abstract` — each writing into its own `rego_out//` directory. `opa eval` then asserts the +scenario truth table against **each** variant's Rego (step 7). Steps 1–6 below describe one such run. + +1. **Set service URLs in env before importing the aiac libraries.** Export `AIAC_PDP_CONFIG_URL`, + `AIAC_POLICY_STORE_URL`, `AIAC_PDP_POLICY_URL`, and `AIAC_REALM` *before* importing the aiac + libraries — the libraries read env at import time. This is the pattern + `test/pdp/policy/generate_rego.py` already follows. +2. **Spawn the three services as `uvicorn` subprocesses** (no Docker) and poll each `GET /health` + until ready, with a bounded timeout: + - IdP Configuration Service — `aiac.idp.service.configuration.keycloak.main:app` on `7071`. + - Policy Store — its ASGI app on `7074`, with `AGENTPOLICY_DB_PATH` pointed at a fresh temp dir. + - OPA Policy Writer — `aiac.pdp.service.policy.opa.main:app` on `7072`, with `REGO_OUTPUT_DIR` + (pointed at the current variant's `rego_out//`) and the Policy Store DB path in its env. +3. **Provision Keycloak** (idempotent — delete-if-exists the realm first, then create): + - via **`python-keycloak` `KeycloakAdmin`** (test fixture): create realm `AIAC_TEST_REALM`; create + users `dev-user`, `test-user`, and `devops-user`; create realm roles `developer`, `tester`, and + `devops`; assign roles to users (`devops-user`→`devops`, which maps to **no** agent/tool scope — + the inbound deny case, see *[Scenario](#scenario)*); create the `github-agent` and `github-tool` + clients with the descriptions in + *[Scenario inputs](#scenario-inputs-prb-functional-inputs)* and with the `client.type` + client attribute set to the plain string `"Agent"` / `"Tool"` respectively, so `Service` type + resolution tags them from the attribute (not from description prose). Set the type via the product + surface `Configuration.set_service_type(service, type)` (`POST /services/{id}/type`) or by writing + the `client.type` attribute directly at client create. The attribute value is a plain string, + **not** a list — a list fails the `in ("Agent","Tool")` check, resolves the type to `None`, and + yields empty pipeline output. + - via the **aiac IdP `Configuration` library** (the real product surface the PCE reads back): create + the client roles (`source-operator`, `issues-operator`) and scopes (`source-access`, `issues-access`, + `source-read`, `source-write`, `issues-read`, `issues-write`) with the descriptions in + *[Scenario inputs](#scenario-inputs-prb-functional-inputs)*, and map roles→services and + scopes→services so `get_services_by_role` / `get_services_by_scope` and `get_service().roles` / + `.scopes` resolve correctly. +4. **Read-back type guard** — after provisioning, call `Configuration.get_service` for both clients and + assert each resolved `.type` (`github-agent` ⇒ `Agent`, `github-tool` ⇒ `Tool`) **before** spawning + the pipeline; abort with a clear message otherwise. This is a provisioning sanity check on the + `client.type` attribute, distinct from the step-7 Rego-decision assertions. +5. **Proto-UC1 orchestration** — run the three PRB mappings against a pinned LLM (`temperature=0`) and + concatenate the results into one `list[PolicyRule]`: + - **(a)** `build_scope_rules(user_roles, agent_scope)` per agent scope → user→agent-scope rules. + - **(b)** `build_scope_rules(user_roles, tool_scope)` per tool scope → user→tool-scope rules. + - **(c)** `build_role_rules(agent_role, tool_scopes)` per agent role → agent-role→tool-scope rules. + + Concatenate into a single `list[PolicyRule]` and call + `aiac.policy.computation.engine.compute_and_apply(rules, override=False)` against a **fresh** Policy + Store. The PCE resolves the IdP relationships, builds the `github-agent` model (with `agent_roles` / + `agent_scopes`; mapping (b) routed into `outbound_subject_rules`; and **no** `github-tool` model), + writes it to the store, and pushes it to the OPA stub. +6. **Terminate the three subprocesses in `finally`.** The realm and the `.rego` files are left in + place for eyeballing. +7. **Assert the truth table with `opa eval`.** Once both variants' Rego is on disk, evaluate a matrix of + **(request JSON, rego file)** tuples with the standalone `opa` binary and hard-assert each decision + against the scenario truth table (see *[Expected output](#expected-output)*): + - **`opa` discovery** — `$OPA_BIN` → else `shutil.which("opa")` → else `pytest.skip("opa not + found")`. Missing `opa` skips (does not fail) the suite. + - **Inbound** — one node per `(variant × subject)`. Request `{"subject": }` (source omitted, so + the generated `source_ok` passes) is evaluated against the real + `data.authz.github_agent.inbound.allow`. Coarse "can this user reach the agent at all" — there is + no intent field. + - **Outbound** — one node per `(variant × subject × function_name)`, where `function_name` is the + agent's operation (a tool scope). Because the generated `allow` / `subject_ok` are existential and + ignore any scope, the outbound decision is evaluated by a **probe query**, + `data.probe.outbound.allow` (defined in `test/integration/probe.rego`), which binds + `input.function_name` against the generated data maps and requires **both** the user→tool gate and + the agent→tool gate to admit the function. Request shape `{"subject", "target", "function_name"}`. + - **Soft match** `function_name`↔scope — the probe compares names by splitting **both** on `[._-]+`, + lowercasing, and comparing as **sets** (token-set equality): `source.read`, `read_source`, and + `Source-Read` all match `source-read`; bare `source` matches nothing. + - The expected verdict for every cell is **computed from** the scenario pair-lists + (`INBOUND_PAIRS` / `OUTBOUND_SUBJECT_PAIRS` / `OUTBOUND_PAIRS` in `scenario.py`), not from a second + hand-maintained copy — a wrong LLM/PCE mapping therefore fails the test. A failing node names the + exact `variant / subject / function_name` cell. +8. **Assert grant-set equivalence (semantic, beyond the decision oracle).** The `opa eval` matrix in + step 7 is deliberately coarse: inbound `allow` only checks "reaches *some* agent scope," and the + agent→tool gate covers all four scopes so only the user gate discriminates — so a **verdict-neutral** + mapping error (a missing or spurious `(role, scope)` grant) passes step 7 unseen. To close that gap + the test also captures the PRB's `list[PolicyRule]` per variant and asserts, as order-independent + `(role, scope)` **sets** per gate, that **each variant equals the `scenario.py` truth table** and + **the two variants equal each other**. This compares grant *sets*, not Rego text (formatting/ordering + may differ; the grant set may not). This is what enforces the *both variants reproduce the same Rego* + intent stated in *Further Notes*. + +## Expected output + +The test passes when `opa eval` decides every cell of the scenario truth table as follows, for **both** +policy variants. Verdicts are **computed from** the `scenario.py` pair-lists (`INBOUND_PAIRS` / +`OUTBOUND_SUBJECT_PAIRS` / `OUTBOUND_PAIRS`), not a hand-maintained copy — this table is the human- +readable rendering of them. + +`USERS`: `dev-user`→`developer`, `test-user`→`tester`, `devops-user`→`devops`. + +**Inbound allow** (`data.authz.github_agent.inbound.allow`, from `INBOUND_PAIRS`, user-role→agent-scope): + +| Subject | Inbound | +|---|---| +| dev-user | ✅ | +| test-user | ✅ | +| devops-user | ❌ | + +**Outbound allow(subject, function)** (`data.probe.outbound.allow`, from `OUTBOUND_SUBJECT_PAIRS` +user→tool; the agent→tool gate covers all four scopes, so the user gate discriminates): + +| | source-read | source-write | issues-read | issues-write | +|---|---|---|---|---| +| dev-user | ✅ | ✅ | ✅ | ❌ | +| test-user | ❌ | ❌ | ✅ | ✅ | +| devops-user | ❌ | ❌ | ❌ | ❌ | + +Alongside the assertions, each variant leaves exactly **two** files on disk in its +`rego_out//` for eyeballing: + +- `github_agent.inbound.rego` — package `authz.github_agent.inbound`; the **user→agent** gate. + `subject_roles` = `{dev-user: [developer], test-user: [tester]}`; `agent_scopes` populated. + (`devops-user` holds `devops`, which maps to no agent scope, so it is absent from `subject_roles` and + denied inbound.) +- `github_agent.outbound.rego` — package `authz.github_agent.outbound`; `allow if { subject_ok; + target_ok }`. Its **`subject_ok`** is the new **user→tool** gate (mapping (b), grouped from + `outbound_subject_rules` into `outbound_subject_role_scopes`, matched against + `target_scopes[input.target]`); its **`target_ok`** is the **agent→tool** gate (mapping (c), over + `agent_roles` × `agent_role_scopes`). `agent_roles` and `target_scopes` are populated. + +Explicitly **no** `github_tool.*.rego` — the pipeline emits no tool model. Eyeball both files against +the **ID-only** package shapes in +[../components/pdp-policy-writer-opa.md](../components/pdp-policy-writer-opa.md) +(§ *Rego package structure*), the same source of truth `5.2` uses. + +## Scenario + +A single agent + tool + three users, fixed so the generated Rego is reproducible and reviewable by +inspection. This is the same canonical `github-agent` worked example as `5.2`, driven end to end +through the real pipeline rather than a hand-built `PolicyModel`, plus a third `devops-user` that +exercises the deny-by-default path. + +| Element | Value | +|---------|-------| +| Realm | `AIAC_TEST_REALM` (default `aiac-e2e`) | +| Agent | `github-agent` (client roles `source-operator`, `issues-operator`; scopes `source-access`, `issues-access`) | +| Tool | `github-tool` (scopes `source-read`, `source-write`, `issues-read`, `issues-write`) | +| Users | `dev-user` (role `developer`), `test-user` (role `tester`), `devops-user` (role `devops`) | +| `developer` | source read/write + issues read | +| `tester` | issues read/write | +| `devops` | no access (inbound deny; denied every outbound function) | + +Role → access (confirmed with the user; the fixed facts that both `policy.md` versions below and the +`scenario.py` pair-lists must agree with — the generic descriptions are not part of this triad): + +- `developer` — source read/write, issues read. +- `tester` — issues read/write. +- `devops` — no access. Conveyed by the **role description only** — it is absent from every pair-list + and both `policy.md` variants are **unchanged** (deny-by-default), so it is denied inbound and on + every outbound function. + +## Configuration (env) + +| Variable | Purpose | Default | +|----------|---------|---------| +| `KEYCLOAK_URL` | External Keycloak base URL | — (required) | +| `KEYCLOAK_ADMIN_REALM` | Realm the admin creds live in | `master` | +| `KEYCLOAK_ADMIN_USERNAME` / `KEYCLOAK_ADMIN_PASSWORD` | Keycloak admin creds | — (required) | +| `AIAC_TEST_REALM` | Realm the test provisions | `aiac-e2e` | +| `AIAC_REALM` | Realm the PCE reads back (= `AIAC_TEST_REALM`) | `aiac-e2e` | +| `AIAC_PDP_CONFIG_URL` | IdP Configuration Service base URL (set before import) | `http://127.0.0.1:7071` | +| `AIAC_POLICY_STORE_URL` | Policy Store base URL (set before import) | `http://127.0.0.1:7074` | +| `AIAC_PDP_POLICY_URL` | OPA Policy Writer base URL (set before import) | `http://127.0.0.1:7072` | +| `REGO_OUTPUT_DIR` | Base dir the OPA stub subprocess writes `.rego` to; the test points it at `rego_out//` per variant and leaves the files on disk | operator-chosen local dir | +| `AGENTPOLICY_DB_PATH` | Policy Store DB path for the subprocess (fresh temp dir) | temp | +| `AIAC_POLICY_FILE` | PRB whole-file policy — path to the `policy.md` variant fed to the PRB; the test sets it per variant (`policy.explicit.md`, `policy.abstract.md`) | `/etc/aiac/policy.md` | +| `LLM_BASE_URL` / `LLM_MODEL` / `LLM_API_KEY` | PRB LLM (pinned `temperature=0`) | — (required) | +| `OPA_BIN` | Path to the standalone `opa` binary used as the verification oracle; else `PATH` (`shutil.which`), else the test `pytest.skip`s | — (optional; PATH lookup) | + +> When the test is written, confirm the Policy Store's ASGI import path and its DB-path env-var +> name against the Policy Store component spec / issue — `AGENTPOLICY_DB_PATH` is the placeholder used +> here; use the real one. `AIAC_POLICY_FILE` selects which `policy.md` variant (see +> *[Scenario inputs](#scenario-inputs-prb-functional-inputs)*) the PRB reads. + +## Runbook + +Runnable only once the pipeline fixes (handoffs 01 + 02, P1–P5) have landed, and requires a live +Keycloak, a real LLM, and an `opa` binary on `PATH` (or `$OPA_BIN`). + +```bash +# env: KEYCLOAK_URL + admin creds + LLM_* set; realm defaults to aiac-e2e; opa on PATH or $OPA_BIN +.venv/bin/pytest test/integration/test_policy_pipeline.py -m integration -v +# ~30 parametrized nodes (variant × subject inbound; variant × subject × function_name outbound). +# A failing node names the exact cell, e.g.: +# test_outbound[abstract-test-user-source-read] — expected deny, opa allowed +# The generated Rego is left on disk per variant for eyeballing: +# rego_out/explicit/github_agent.{inbound,outbound}.rego +# rego_out/abstract/github_agent.{inbound,outbound}.rego +# (no github_tool.*.rego in either) +``` + +The suite `pytest.skip`s when no `opa` binary is found (`$OPA_BIN` → `PATH`). Eyeball the persisted +Rego against the adjusted package shapes in +[../components/pdp-policy-writer-opa.md](../components/pdp-policy-writer-opa.md); optionally inspect the +Policy Store DB and the provisioned Keycloak realm. + +## Testing Decisions + +- **Highest seam available, verified by a real oracle.** Real libraries + real services + real Keycloak + + real LLM. The test drives the pipeline through its real surfaces — the IdP `Configuration` library, + the PRB entry points (`build_scope_rules` / `build_role_rules`), and the PCE's `compute_and_apply` — + and then verifies the real filesystem output with the standalone **`opa eval`** binary. The only + shortcut is the OPA filesystem stub (same as `5.2`). A good test here asserts only **external + behavior** — the policy *decisions* the generated Rego makes for scenario-derived requests — never the + internal Rego structure (which the OPA Policy Writer's own unit tests own). +- **Rego is the artifact under test; the scenario is the oracle.** The LLM/PCE that produced the Rego + might be wrong, so the expected verdicts are **computed from** the scenario pair-lists + (`INBOUND_PAIRS` / `OUTBOUND_SUBJECT_PAIRS` / `OUTBOUND_PAIRS`), not from a second hand-maintained + copy or from the Rego itself. A wrong role→scope mapping therefore fails the test at the exact cell. +- **Outbound needs a probe.** The generated `allow` / `subject_ok` are existential and ignore any + scope, so a raw query cannot answer "may this subject invoke *this* function." A small + `test/integration/probe.rego` (`data.probe.outbound.allow`) binds `input.function_name` against the + generated data maps and requires **both** the user→tool and agent→tool gates to admit it. Names are + compared by **token-set equality** (split on `[._-]+`, lowercased) so `source.read` / `read_source` / + `Source-Read` all match `source-read` while bare `source` matches nothing. +- **Attribute-based client typing + read-back guard.** Clients are typed by the `client.type` + attribute (plain string `"Agent"` / `"Tool"`), provisioned by the test — not by description keywords. + Because that attribute drives whether the PCE emits an agent model (and suppresses the tool model), + the test reads each service back via `Configuration.get_service` and asserts its `.type` before + running the pipeline, aborting on mismatch. This is a **provisioning** sanity check, distinct from the + Rego-decision assertions. +- **Self-contained subprocess lifecycle.** The test spawns IdP (7071), Policy Store (7074), and OPA + (7072) as `uvicorn` subprocesses, polls each `GET /health` before use, and tears them all down in + `finally`. Keycloak and the LLM are **external** (reached via env); `opa` is an external binary. +- **LLM nondeterminism, contained.** The PRB LLM is pinned to `temperature=0`, and the **explicit** + `policy.md` variant states each `(role, scope)` grant outright, so its mapping is stable. The + **abstract** variant leans on the LLM to expand prose + descriptions into concrete scopes; both + variants are asserted not only cell-by-cell via `opa eval` (step 7) but at the **grant-set** level + (step 8) — each variant's `(role, scope)` set must equal the truth table *and* the other variant's. + Grant-set equivalence catches the verdict-neutral under/over-grants the decision oracle hides. Some + model-dependence remains, which is why the suite is `@pytest.mark.integration`, out of default CI. +- **Prior art, shared not copied.** `test/pdp/policy/generate_rego.py` (the `5.2` launcher) established + the shape this test reuses — `uvicorn` subprocess spawn, `GET /health` poll, env-before-import + ordering, and `finally` teardown. Rather than duplicate it, that machinery lives in the shared + `test/integration/launcher.py`, and the fixed scenario lives in `test/integration/scenario.py`; + `generate_rego.py` was refactored onto both (its `.rego` output verified byte-identical to before the + refactor). The live-Keycloak pytest suite (`testing/5.1-integration-tests.md`) is the sibling + marker-gated, decision-asserting counterpart for the read-side services and is the prior art for the + `@pytest.mark.integration` + `opa eval` shape. + +## Relationship to other integration tests + +This is **one** integration-test spec among several indexed by the master PRD +([../PRD.md](../PRD.md), § *Integration test specifications*). + +- Same flavor as the **live-Keycloak pytest integration tests** (`testing/5.1-integration-tests.md`) — + both are `@pytest.mark.integration`, run outside the default unit run against live dependencies, and + assert on decisions. This test additionally uses `opa eval` as its oracle and skips when `opa` is + absent. +- **Broader than** the OPA-stub-only **PDP Policy Writer** launcher + ([pdp-policy-writer.md](pdp-policy-writer.md), `testing/5.2-pdp-writer-integration-test.md`): `5.2` + hand-builds a `PolicyModel`, exercises only OPA, and is still a write-only eyeball launcher; this test + adds Keycloak provisioning + PRB + PCE in front of the **same** OPA stub and **asserts** the resulting + decisions with `opa eval`. Both still leave `.rego` on disk against the same package shapes. + +Tracking issue for this test: `testing/5.3-policy-pipeline-integration-test.md`. + +## Out of Scope + +- **Writing `test_policy_pipeline.py`, `probe.rego`, or any P1–P5 pipeline code** — this spec + *describes* the test; the test itself is written in a later session against the fixed pipeline + (tracked by `testing/5.3-policy-pipeline-integration-test.md` and the prerequisite issues). +- **The Rego generator, the canonical policy model, the PRB, and the PCE implementations** — specified + and unit-tested by their own components ([../components/pdp-policy-writer-opa.md](../components/pdp-policy-writer-opa.md), + [../components/policy-model.md](../components/policy-model.md), + [../components/policy-computation-engine.md](../components/policy-computation-engine.md), and the PRB + component spec), not here. In particular, the internal **structure** of the generated Rego is the + Policy Writer's concern; this test asserts only the **decisions** that Rego makes. +- **The Kubernetes-CR Policy Writer (1.13)** — this test targets the filesystem **stub** (1.14) only. +- **Default-CI wiring** — the test is `@pytest.mark.integration` and requires live Keycloak + LLM + an + `opa` binary, so it runs on demand, not in the default `-m "not integration"` unit run. + +## Further Notes + +- The scenario is deliberately fixed. The role→access facts are owned by **three** artefacts that must + agree: the *Scenario* table, **both** `policy.md` versions in *Scenario inputs*, and the + `scenario.py` pair-lists (`INBOUND_PAIRS` / `OUTBOUND_SUBJECT_PAIRS` / `OUTBOUND_PAIRS`). The + entity/role/scope **descriptions no longer encode those facts** — they are generic and functional and + drop out of the fact triad; they must stay generic and simply not contradict the facts. If the + role→access facts change, update the *Scenario* table, both `policy.md` variants, and the pair-lists + together so the eyeballed output stays reviewable. +- The least-privilege **deny-by-default** directive is supplied by the PRB prompt itself + (`_GRANT_ACCESS` in `agent/policy_rules_builder/prompts.py`), which prepends it — followed by the + bundled generic baseline policy (`generic_policy.md`) — ahead of the scenario `policy.md` on every + call, so every policy decision gets it regardless of which variant is read. The **explicit** variant + still spells the directive out (its whole point is to state everything outright); the **abstract** + variant relies on the prompt and does not restate it — do not re-add it to the abstract variant. +- Two `policy.md` variants are shipped on purpose (see *Scenario inputs*): an **explicit** one and an + **abstract** one. `AIAC_POLICY_FILE` selects which the PRB reads, so a reviewer can compare the PRB's + output on explicit vs. abstract policy text against the same expected Rego. The abstract variant + carries **no** agent-capability bullet; it relies on the elaborated `source-operator` / + `issues-operator` role descriptions (provisioned into Keycloak) for mapping (c), so it survives + deny-by-default and both variants reproduce the same Rego. +- Descriptions are ≤255 characters and written **verbatim** into Keycloak; there is no shortened / + verbatim split. (Keycloak caps role and client descriptions at 255 chars, and the generic descriptions + are authored to stay within that cap.) +- The `devops` role's **zero access** is conveyed by its **role description only**. It is absent from + every pair-list (`INBOUND_PAIRS` / `OUTBOUND_SUBJECT_PAIRS` / `OUTBOUND_PAIRS`) and both `policy.md` + variants are **unchanged**, so deny-by-default alone denies it inbound and on every outbound function — + which is precisely what the truth table's `devops-user` row asserts. Because `devops-user` lives in + the shared `scenario.py`, it also appears in the `5.2` launcher's eyeball output (denied everywhere); + that is intentional and keeps the two launchers consistent. + +## Blocked-by + +The pipeline can only produce correct output once handoffs 01 (P1, P3) and 02 (P2, P4, P5) land; those +are **resolved**, so this test is ready to be written. Component prerequisites: + +- PRB — `agent/3.20-policy-rules-builder.md` +- PCE — `policy/pce/8.10-policy-computation-engine.md` +- Policy model — `policy/model/8.1-policy-model.md` +- OPA filesystem stub — `pdp-policy-writer/1.14-pdp-policy-writer-opa-stub.md` +- Rego package generator — `pdp-policy-writer/1.10-rego-package-generator.md` +- pdp-policy library — `library/pdp/8.9-pdp-policy-library-rename.md` +- Policy Store library / service — `policy/store/8.7-policy-store-library.md` / + `policy/store/8.5-policy-store-service.md` + +## Scenario inputs (PRB functional inputs) + +These are **functional** inputs — the LLM reads the entity/role/scope descriptions and the `policy.md` +to produce the role→scope mappings, so they are part of the fixed scenario, not decoration. Confirmed +with the user; keep them in sync with the *Scenario* table (see *Further Notes*). + +### Entity descriptions + +The descriptions are **generic and keyword-free** — they describe what each entity/role/scope *does*, +carry no policy grant ("Resolves to…") and no owning-client naming, and stay within Keycloak's 255-char +cap so they are written verbatim (no shortened renderings). Client `type` is **not** inferred from +description prose: the test provisions each client's `client.type` attribute (the type UC1 +discovers from the agent card / `kagenti.io/type` label) as a plain string `"Agent"` / `"Tool"`, so +`Service` type resolution ([../../../src/aiac/idp/configuration/models.py:79-87](../../../src/aiac/idp/configuration/models.py#L79-L87)) +tags each client from the attribute without touching the TEMP description-keyword fallback. + +**`github-agent`** — client (Agent): +> Autonomous Agent acting on a user's behalf against source repositories and an issue tracker. It +> inspects and changes repository source contents and reads, creates, and updates issues and their +> threads. + +**`github-tool`** — client (Tool): +> Capability provider Tool for source repositories and an issue tracker. It performs read and write +> operations on repository source contents and on issues and their comment threads. + +**`developer`** — realm role (user): +> Developer — an engineering user who develops the source codebase (writing and maintaining code) and +> fixes code defects reported in the issue tracker; works primarily in source and consults issues for +> defect reports. + +**`tester`** — realm role (user): +> Tester — a quality-assurance user who verifies software quality and tracks defects through the issue +> tracker: filing, triaging, and updating issue reports; works in the issue tracker, not in source. + +**`devops`** — realm role (user): +> DevOps — an operations user who manages deployment infrastructure and runtime environments; does not +> author source code and does not manage the issue tracker. + +> The `devops` description is deliberately **unrelated** to source and issue work, so the PRB derives no +> agent or tool scope for it and deny-by-default leaves `devops-user` denied everywhere — the inbound +> deny case. It is added to the realm-role set only; the pair-lists and both `policy.md` variants stay +> unchanged (see *Further Notes*). + +### Role & scope descriptions + +**Client roles (agent):** + +- `source-operator` — Covers read and write access to source repository contents — listing, reading, + creating, and modifying files. +- `issues-operator` — Covers read and write access to the issue tracker — reading, filing, updating, + and commenting on issues and their threads. + +**Agent scopes:** + +- `source-access` — Scope granting use of a source-code capability — invoking source-code functions such + as reading and changing repository contents. +- `issues-access` — Scope granting use of an issue-management capability — invoking issue functions such + as reading and updating issues. + +**Tool scopes:** + +- `source-read` — Read source repository contents: file listings and file bodies. Read-only. +- `source-write` — Create, modify, or delete source repository contents; commit file changes. +- `issues-read` — Read issues and their comment threads. Read-only. +- `issues-write` — Create and update issues: open, edit, comment, and close. + +### `policy.md` — Version 1 (explicit) + +Each granted `(role, scope)` pair is spelled out; the three sections map 1:1 to PRB mappings (a)/(b)/(c) +and to the expected Rego gates. + +```markdown +# Access Control Policy — github-agent / github-tool + +Grant access on a least-privilege basis. Only grant a (role, scope) pair when this +policy supports it; deny by default. + +## Users → agent capabilities (inbound; user may call the agent) +- developer may use source-access and issues-access. +- tester may use issues-access. + +## Users → tool operations (outbound subject; user may reach the tool) +- developer may perform source-read, source-write, and issues-read. +- tester may perform issues-read and issues-write. + +## Agent roles → tool operations (outbound target; agent may reach the tool) +- source-operator may perform source-read and source-write. +- issues-operator may perform issues-read and issues-write. +``` + +### `policy.md` — Version 2 (abstract) + +Relies on the PRB / LLM to expand "read and modify source" into the concrete scopes. Encodes the same +role→access facts as Version 1. It carries **no** agent-capability bullet; mapping (c) +(agent-role→tool-scope) is instead derived from the elaborated `source-operator` / `issues-operator` +role descriptions (see *Role & scope descriptions*), so it survives the PRB's deny-by-default-on-silence +rule and both variants reproduce the same Rego. + +```markdown +- Developers work primarily in source — writing and maintaining code — and consult the issue tracker only to follow defect reports; grant them full read and write access to source contents, and read-only access to issues. +- Testers work exclusively in the issue tracker — filing, triaging, and updating defect reports — and do not work in source; grant them full read and write access to issues, and no access to source. +``` diff --git a/aiac/docs/specs/integration-test/uc1-onboarding-pipeline.md b/aiac/docs/specs/integration-test/uc1-onboarding-pipeline.md new file mode 100644 index 00000000..bb2fbf37 --- /dev/null +++ b/aiac/docs/specs/integration-test/uc1-onboarding-pipeline.md @@ -0,0 +1,445 @@ +# Integration Test: uc1-onboarding-pipeline — `test_uc1_onboarding_pipeline.py` + +> **One spec among several.** This document specifies a **single** integration test. Integration-test +> specs live **one spec per test** under `docs/specs/integration-test/` (a sibling of +> `components/`), and the master PRD's *Integration test specifications* section ([../PRD.md](../PRD.md)) +> is the index of them. This is the **uc1-onboarding-pipeline** integration test — the phase-1 +> service-onboarding demo driven end-to-end through the **real UC-1 agent** against **really-deployed** +> demo workloads — not the definition of integration testing in general. + +> **Relationship to `policy-pipeline`.** This test is the **discovery-driven sibling** of +> [policy-pipeline.md](policy-pipeline.md). The *scenario facts and the truth tables are identical* (same +> three users, same role→access facts, same inbound/outbound matrices). The **difference** is provenance: +> where `policy-pipeline` *hand-provisions* the agent/tool roles and scopes with clean fixed names and +> then calls the PRB mappings directly, this test *infers them via real UC-1 onboarding* of deployed +> workloads. That inference is what makes the generated Rego **semantically similar but not byte-identical** +> to `policy-pipeline` (see *[Semantic similarity, not byte-identity](#semantic-similarity-not-byte-identity)*). + +## Location + +`aiac/test/integration/test_uc1_onboarding_pipeline.py` — a pytest module marked +`@pytest.mark.integration`. It imports two shared modules: + +- `aiac/test/integration/scenario_uc1.py` — a **new** scenario module (sibling of the existing + `scenario.py`, kept separate so `5.2`/`5.3` are unaffected). It carries the phase-1 users/roles, the two + `policy.md` variants, and the pair-lists expressed over the **discovered, workload-prefixed** names + (`github-tool.source-read`, `github-agent.source_operations`, …), which are what the generated Rego + actually contains. +- `aiac/test/integration/launcher.py` — the shared `kubectl`/port-forward + `opa` helpers (extended from + the `5.3` launcher; see *[Testing Decisions](#testing-decisions)*). + +It also ships `aiac/test/integration/probe_uc1.rego` — a standalone Rego module used only as the outbound +verification query, adapted from `5.3`'s `probe.rego` to match against **full discovered scope names** +(no prefix-stripping soft match; see step 7). + +## Description + +A `@pytest.mark.integration` test that validates the **phase-1 deliverable** +([../../gh-issues/sub-issue-phase-1.md](../../gh-issues/sub-issue-phase-1.md)) and confirms the runnable +demo: it deploys the real **`github-agent`** and a simplified **`github-tool`** to a live Kagenti/Kind +cluster, drives the **real UC-1 Service Onboarding agent** (`onboard_service` via the Controller's HTTP +trigger) for each, and then **asserts** the generated Rego decides correctly by running the standalone +`opa eval` binary as its verification oracle. + +Phase-1 is explicit that **live enforcement / live traffic is out of scope** — correctness is shown by +**evaluating the generated rules**, not by routing real requests. So this test is *Deploy + discover + +evaluate*: the agent and tool are really deployed and really discovered by UC-1 (classify from the +`kagenti.io/type` label, read the AgentCard, read the MCP `tools/list`, provision roles/scopes into +Keycloak, model access, emit Rego), but **no A2A message is ever sent through the agent**. + +The generated Rego is the **artifact under test** — the LLM/PCE that produced it might be wrong — so the +test never trusts it. Expected verdicts are **computed from** the `scenario_uc1.py` pair-lists (the +intended policy), and the real Rego is asserted to admit/deny each scenario-derived request as the truth +table requires. A mismatch fails the test and names the exact cell. + +Because it needs a live Kagenti cluster + operator + Keycloak + a real LLM, it is `@pytest.mark.integration` +and stays out of the default unit run (`-m "not integration"`); it additionally `pytest.skip`s when no +`opa` binary is found. + +### Topology + +- **AIAC runs on the host? No — in-cluster.** The pipeline is driven **inside the cluster** so UC-1's + `analyze_tool` can reach the tool's MCP endpoint at its cluster-internal DNS name + (`github-tool.{ns}.svc.cluster.local`). The test triggers onboarding over HTTP against the in-cluster + AIAC Controller (`POST /apply/service/{service_id}`), reachable via `kubectl port-forward`. +- **Two AIAC stacks, one per policy variant.** Because the PRB reads its policy from `AIAC_POLICY_FILE` + (a file/ConfigMap **baked into the AIAC pod**, not selectable per request), the two `policy.md` + variants (`explicit`, `abstract`) are served by **two independent in-cluster AIAC stacks** — each an + AIAC agent + its own Policy Store + its own OPA Policy Writer, mounting its own variant. The **IdP + Configuration Service and Keycloak are shared** (provisioning is variant-independent and idempotent). +- **Rego capture.** Each stack's OPA Policy Writer writes `.rego` (the filesystem stub — phase-1 emits to + a filesystem location, **not** the K8s CR) into a mounted volume. The test `kubectl cp`/`exec`s each + variant's `.rego` out to a host temp dir `rego_out//`, then runs the `opa eval` oracle on the + host. + +### What it does + +The pipeline (deploy → onboard tool → onboard agent → emit Rego) is driven **once per `policy.md` variant** +— against that variant's AIAC stack — each writing into its own `rego_out//`. `opa eval` then +asserts the truth table against **each** variant's Rego (step 7). Steps 1–6 describe the run. + +0. **Point the demo namespace at the dedicated realm (test fixture).** Before deploying workloads, set + `KEYCLOAK_REALM=` in the demo namespace's **`authbridge-config`** ConfigMap. The + kagenti-operator reads the realm per-namespace from this key and **preserves an admin/CI-set value** + (operator issue #433), so the operator registers *this* namespace's clients into the test realm with + **no operator restart or cluster-wide change**. (Default without this is `kagenti`.) +1. **Provision the realm's users + realm roles (test fixture).** UC-1 does **not** create users or realm + roles, so the fixture provisions them via **`python-keycloak` `KeycloakAdmin`** into the dedicated test + realm (`AIAC_TEST_REALM`), **before** onboarding (the Service Policy Builder reads the full realm-role + universe): users `dev-user`, `test-user`, `devops-user`; realm roles `developer`, `tester`, `devops` + (with the generic ≤255-char descriptions in *[Scenario inputs](#scenario-inputs)* — the PRB reads + them); role assignments (`devops-user`→`devops`, which maps to **no** scope — the deny case). + Provisioning is idempotent (create-or-get); state is **left in place** on teardown (the shared realm is + never deleted/recreated — reruns converge). + +2. **Deploy the demo workloads.** `kubectl apply` the simplified **`github-tool`** + ([../demo/github-tool.md](../demo/github-tool.md)) and the real **`github-agent`** + ([../demo/github-agent.md](../demo/github-agent.md)) into the demo namespace. Wait until: pods Ready; + the kagenti-operator has **registered a Keycloak client** for each (with `client.name = + "{namespace}/{workload}"`) and applied the `kagenti.io/type` pod label (`tool`/`agent`); the + `github-agent` **AgentCard CR** is present; the tool **Service** carries the `protocol.kagenti.io/mcp` + label and answers `tools/list`. + +3. **Onboard the tool, then the agent (real UC-1), against each variant's AIAC stack.** Order matters — + the tool is onboarded first so its scopes exist when the agent's Service Policy Builder reads the scope + universe: + > **Resolving `{service_id}`.** The trigger takes the Keycloak **client id**, which is **not** the + > string `github-tool`: the operator sets `client.name = "{ns}/{workload}"` but the client *id* is + > `"{ns}/{workload}"` only when SPIRE is off — with `--spire-trust-domain` set it is a SPIFFE URI + > (`spiffe:///ns/{ns}/sa/{sa}`). The test resolves the id by looking up the client whose + > **name** is `"{ns}/github-tool"` (resp. `"{ns}/github-agent"`) via the Configuration library / + > Keycloak admin, then triggers with that id. + + - `POST /apply/service/{github-tool client id}` → UC-1 classifies it a **Tool** from the label, reads + the MCP manifest, provisions scopes `github-tool.{source-read, source-write, issues-read, + issues-write}` (verbatim tool descriptions), sets `client.type=Tool`. **No rules are written for the + tool directly.** + - `POST /apply/service/{github-agent client id}` → UC-1 classifies it an **Agent** from the label, + reads the AgentCard, provisions role `github-agent.agent` + scopes + `github-agent.{source_operations, issue_operations}` (from the two skills), sets `client.type=Agent`, + then the Service Policy Builder maps the realm roles→scopes via the real PRB (real LLM, + `temperature=0`) and the Controller calls `compute_and_apply(rules, override=False)` → the OPA writer + emits `github_agent.inbound.rego` + `github_agent.outbound.rego`. + +4. **Capture the Rego.** `kubectl cp`/`exec` each variant stack's OPA-writer output to + `rego_out//`. + +5. **Repeat steps 3–4 for the second variant** (the other AIAC stack). Provisioning re-runs are idempotent + and variant-independent; only the emitted Rego differs. + +6. **Teardown in `finally`.** Delete the demo workloads (operator de-registers the clients); leave the + realm, users, realm roles, and `.rego` files in place for eyeballing. AIAC stacks may be left running or + torn down per the harness. + +7. **Assert the truth table with `opa eval`.** For each variant's captured Rego, evaluate a matrix of + **(request JSON, rego file)** tuples with the standalone `opa` binary and hard-assert each decision: + - **`opa` discovery** — `$OPA_BIN` → `shutil.which("opa")` → `pytest.skip("opa not found")`. + - **Inbound** — one node per `(variant × subject)`. Request `{"subject": }` against + `data.authz.github_agent.inbound.allow`. Coarse existential "can this user reach the agent at all"; + the discovered agent-scope names (`github-agent.source_operations` / `.issue_operations`) do not need + to match anything — inbound has no function field. + - **Outbound (user gate only)** — one node per `(variant × subject × function_name)`, where + `function_name` is a **full discovered tool-scope name** (`github-tool.source-read`, …). Evaluated by + the probe query `data.probe.outbound.allow` (in `probe_uc1.rego`), which binds `input.function_name` + against the generated **user→tool** data maps (`subject_ok`) **only**. The agent→tool gate + (`target_ok`) is **not** probed — under UC-1's single generic `github-agent.agent` role it is + degenerate/empty (see *[The agent-to-tool gate](#the-agent-to-tool-gate-degenerate-by-design)*), matching + phase-1's "user-gating dimension only." + - **Name matching** — because `scenario_uc1.py` stores the **full discovered scope names**, the probe + matches `input.function_name` to a scope by **exact string equality** (no prefix-stripping token-set + soft match — that was `5.3`'s device for bare names; here both sides are already prefixed). + - The expected verdict for every cell is **computed from** the `scenario_uc1.py` pair-lists, not a + second hand-maintained copy. A failing node names the exact `variant / subject / function_name` cell. + +8. **Assert grant-set equivalence (semantic, from the Rego).** The pipeline runs inside the AIAC pod, so + the test never sees the intermediate `list[PolicyRule]`. Grant-set equivalence is therefore re-derived + from the **generated Rego data maps**: dump each variant's user→tool grant map (via `opa eval + data.authz.github_agent.outbound...`) and each variant's inbound `subject_roles`/`agent_scopes`, and + assert, as order-independent `(role, scope)` **sets**, that **each variant equals the `scenario_uc1.py` + truth table** and **the two variants equal each other**. This compares grant *sets*, not Rego text + (formatting/name-ordering may differ; the grant set may not) — the semantic-similarity guarantee this + test makes. It catches verdict-neutral over/under-grants the coarse `opa eval` oracle hides. + +## Expected output + +The test passes when `opa eval` decides every cell of the scenario truth table as follows, for **both** +policy variants. Verdicts are **computed from** the `scenario_uc1.py` pair-lists, not a hand-maintained +copy — these tables are the human-readable rendering of them. They are **identical to policy-pipeline's** +(only the underlying scope-name strings differ). + +`USERS`: `dev-user`→`developer`, `test-user`→`tester`, `devops-user`→`devops`. + +**Inbound allow** (`data.authz.github_agent.inbound.allow`, user-role→agent-scope, existential): + +| Subject | Inbound | +|---|---| +| dev-user | ✅ | +| test-user | ✅ | +| devops-user | ❌ | + +**Outbound allow(subject, function)** (`data.probe.outbound.allow`, user→tool gate only; `function_name` +is the full discovered scope name, shown here by its bare suffix for readability): + +| | github-tool.source-read | github-tool.source-write | github-tool.issues-read | github-tool.issues-write | +|---|---|---|---|---| +| dev-user | ✅ | ✅ | ✅ | ❌ | +| test-user | ❌ | ❌ | ✅ | ✅ | +| devops-user | ❌ | ❌ | ❌ | ❌ | + +Each variant leaves exactly **two** files on disk in its `rego_out//`: + +- `github_agent.inbound.rego` — package `authz.github_agent.inbound`; the **user→agent** gate. + `subject_roles` = `{dev-user: [developer], test-user: [tester]}`; `agent_scopes` populated with the + **discovered** names `[github-agent.source_operations, github-agent.issue_operations]`. (`devops-user` + holds `devops`, which maps to no agent scope, so it is absent and denied inbound.) +- `github_agent.outbound.rego` — package `authz.github_agent.outbound`; `allow if { subject_ok; target_ok + }`. Its **`subject_ok`** is the **user→tool** gate (populated: `subject_role_scopes` grouping + developer/tester → `github-tool.*` scopes). Its **`target_ok`** (agent→tool) is **degenerate/empty** + under the single generic `github-agent.agent` role — documented, not asserted (see below). + +Explicitly **no** `github_tool.*.rego` — the pipeline emits no tool model (the tool is a pure target; +phase-1 acceptance requires "no rules written for the tool alone"). + +### Semantic similarity, not byte-identity + +This test's Rego is **semantically similar** to `policy-pipeline`'s but **not byte-identical**, for two +baked-in reasons in the (frozen) UC-1 provisioning: + +1. **Workload-prefixed names.** UC-1 names every scope `{workload}.{name}`, so the data maps hold + `github-tool.source-read` / `github-agent.source_operations` where `policy-pipeline` holds bare + `source-read` / `source-access`. Same **file set + package shapes**; different name strings. +2. **Degenerate `target_ok`.** UC-1 emits one generic `github-agent.agent` role (description `"Agent + role"`), which the PRB cannot map to specific tool scopes under deny-by-default, so the agent→tool gate + is empty — where `policy-pipeline`'s two operator roles populate it across all four scopes. + +Both are inherent to running real UC-1; making the Rego byte-identical would require unfreezing UC-1 +(dropping the prefix + deriving per-skill agent roles) or abandoning UC-1 discovery (which would collapse +this test into `policy-pipeline`). The test therefore asserts **same files + same decisions + equivalent +grant sets**, not identical text. + +### The agent-to-tool gate (degenerate by design) + +Phase-1 states outbound access is an **intersection** of the user→tool gate and the agent→tool gate, but +that "the agent holds all of `github-tool`'s scopes, so this demo exercises the **user-gating dimension +only**; the agent-role gate is not exercised here" (deferred to a future two-tool demo). Under real UC-1 +the single generic `github-agent.agent` role yields an **empty** `target_ok`, so the generated `allow` +(`subject_ok AND target_ok`) would deny everything. The probe therefore evaluates **`subject_ok` only**, +which is exactly the user-gating slice phase-1 validates. The empty `target_ok` is documented as a known +UC-1 limitation, not a test failure. + +## Scenario + +The phase-1 scenario — identical role→access facts to `policy-pipeline`, driven end-to-end through real +UC-1 onboarding of deployed workloads rather than a hand-built provisioning step. + +| Element | Value | +|---------|-------| +| Realm | `AIAC_TEST_REALM` (dedicated; default `aiac-uc1-e2e`) | +| Agent | `github-agent` — **discovered** role `github-agent.agent`; scopes `github-agent.source_operations`, `github-agent.issue_operations` (from AgentCard skills) | +| Tool | `github-tool` (simplified) — **discovered** scopes `github-tool.source-read`, `github-tool.source-write`, `github-tool.issues-read`, `github-tool.issues-write` (from MCP `tools/list`) | +| Users | `dev-user` (role `developer`), `test-user` (role `tester`), `devops-user` (role `devops`) | +| `developer` | source read/write + issues read | +| `tester` | issues read/write | +| `devops` | no access (inbound deny; denied every outbound function) | + +Role → access (the fixed facts both `policy.md` variants and the `scenario_uc1.py` pair-lists must agree +with; the generic descriptions are not part of this triad): + +- `developer` — source read/write, issues read. +- `tester` — issues read/write. +- `devops` — no access. Conveyed by the **role description only** — absent from every pair-list and both + `policy.md` variants (deny-by-default), so denied inbound and on every outbound function. + +## Configuration (env) + +| Variable | Purpose | Default | +|----------|---------|---------| +| `KUBECONFIG` | Kubeconfig for the live Kagenti/Kind cluster | — (required) | +| `AIAC_DEMO_NAMESPACE` | Namespace the demo workloads deploy into | `team1` | +| `KEYCLOAK_URL` | External Keycloak base URL | — (required) | +| `KEYCLOAK_ADMIN_REALM` | Realm the admin creds live in | `master` | +| `KEYCLOAK_ADMIN_USERNAME` / `KEYCLOAK_ADMIN_PASSWORD` | Keycloak admin creds (for user/realm-role provisioning) | — (required) | +| `AIAC_TEST_REALM` | Dedicated realm the test uses; the fixture sets `KEYCLOAK_REALM` on the demo namespace's `authbridge-config` ConfigMap so the operator registers this namespace's clients into it (per-namespace, no cluster-wide change) | `aiac-uc1-e2e` | +| `AIAC_REALM` | Realm the AIAC stacks read back (= `AIAC_TEST_REALM`) | `aiac-uc1-e2e` | +| `AIAC_EXPLICIT_URL` / `AIAC_ABSTRACT_URL` | Base URL of each variant's in-cluster AIAC Controller (via port-forward) for `POST /apply/service/{id}` | `http://127.0.0.1:7070` / `:7080` | +| `AIAC_OPA_POD_EXPLICIT` / `AIAC_OPA_POD_ABSTRACT` | OPA-writer pod (or PVC) per variant to `kubectl cp` `.rego` from | — (resolved from labels) | +| `REGO_OUTPUT_DIR` | Host base dir the captured `.rego` is copied to; `rego_out//` per variant, left on disk | operator-chosen local dir | +| `LLM_BASE_URL` / `LLM_MODEL` / `LLM_API_KEY` | PRB LLM (pinned `temperature=0`); consumed by the in-cluster AIAC pods | — (required) | +| `OPA_BIN` | Path to the standalone `opa` binary (oracle); else `PATH`, else `pytest.skip` | — (optional; PATH lookup) | + +> When the test is written, confirm: the Controller trigger path and port; and the OPA writer's output +> path / how the two variant stacks are deployed and addressed. The realm mechanism (per-namespace +> `authbridge-config` `KEYCLOAK_REALM`, preserved by the operator — issue #433) and the `{service_id}` +> resolution (look up the client by `client.name = "{ns}/{workload}"`; the id is a SPIFFE URI under SPIRE) +> are **confirmed** against the operator source and reflected above. + +## Runbook + +Runnable only against a live Kagenti/Kind cluster (operator + Keycloak + SPIRE) **configured to register +clients into `AIAC_TEST_REALM`**, with the two AIAC variant stacks deployable, a real LLM, the demo agent +(GA-1…9) and simplified tool images built + kind-loaded, and an `opa` binary on `PATH` (or `$OPA_BIN`). + +```bash +# env: KUBECONFIG + KEYCLOAK_URL + admin creds + LLM_* set; realm defaults to aiac-uc1-e2e; opa on PATH or $OPA_BIN +.venv/bin/pytest test/integration/test_uc1_onboarding_pipeline.py -m integration -v +# Parametrized nodes: (variant × subject) inbound + (variant × subject × function_name) outbound + grant-set equivalence. +# A failing node names the exact cell, e.g.: +# test_outbound[abstract-test-user-github-tool.source-read] — expected deny, opa allowed +# The generated Rego is left on disk per variant for eyeballing: +# rego_out/explicit/github_agent.{inbound,outbound}.rego +# rego_out/abstract/github_agent.{inbound,outbound}.rego +# (no github_tool.*.rego in either) +``` + +The suite `pytest.skip`s when no `opa` binary is found. Eyeball the persisted Rego against the ID-only +package shapes in [../components/pdp-policy-writer-opa.md](../components/pdp-policy-writer-opa.md); +optionally inspect the provisioned Keycloak realm and the discovered scopes. + +## Testing Decisions + +- **Highest seam available, verified by a real oracle.** Real deployed workloads + real kagenti-operator + + real UC-1 onboarding + real PRB/PCE + real Keycloak + real LLM. The test drives the pipeline through + its production trigger (`POST /apply/service/{id}`) and verifies the real filesystem Rego with the + standalone `opa eval` binary. A good test here asserts only **external behavior** — the *decisions* the + generated Rego makes — never internal Rego structure. +- **Rego is the artifact under test; the scenario is the oracle.** Expected verdicts are computed from + `scenario_uc1.py`, not from the Rego. A wrong role→scope mapping fails the test at the exact cell. +- **Deploy + discover + evaluate, no live traffic.** Per phase-1, enforcement/token-exchange/live A2A is + out of scope; correctness is shown by evaluating the generated rules. The agent pod need only exist + (labelled `kagenti.io/type=agent`, AgentCard present); the simplified tool need only answer `tools/list` + — neither is driven with real requests. +- **In-cluster AIAC for MCP reachability.** UC-1's `analyze_tool` posts `tools/list` to a + cluster-internal DNS name, so the pipeline runs in-cluster (triggered over HTTP) rather than in-process + on the host, which cannot resolve `*.svc.cluster.local`. +- **Two AIAC stacks for two variants.** The PRB policy is baked into the pod (`AIAC_POLICY_FILE`), so each + `policy.md` variant is served by its own AIAC stack; Keycloak + the IdP Configuration Service are shared + (provisioning is variant-independent and idempotent). +- **User gate only.** UC-1's single generic agent role yields an empty `target_ok`; the outbound probe + evaluates `subject_ok` alone, matching phase-1's user-gating-only intent. The agent-role gate is + deferred (needs a second tool the agent is not mapped to). +- **Semantic similarity, from the Rego.** Since the pipeline runs in-pod, grant-set equivalence is + re-derived from the generated Rego data maps (not an in-process `list[PolicyRule]`), and compares grant + *sets* across variants and vs the scenario — the semantic-similarity guarantee, not byte-identity. +- **Dedicated realm, leave-in-place.** A dedicated test realm (`AIAC_TEST_REALM`) the operator is + configured for; the shared realm is never deleted/recreated, so cleanup is idempotent leave-in-place + (users/roles create-or-get; UC-1 writes idempotent; demo workloads deleted so the operator de-registers + the clients). +- **LLM nondeterminism, contained.** The PRB LLM is pinned `temperature=0`; both variants are asserted + cell-by-cell (`opa eval`) and at the grant-set level. Some model-dependence remains, which is why the + suite is `@pytest.mark.integration`, out of default CI. +- **Prior art, shared not copied.** Reuses the `5.3` shape (`@pytest.mark.integration`, `opa` + discovery/skip, per-variant `rego_out/`, scenario-as-oracle, probe query) via the shared + `launcher.py`/`scenario_uc1.py`, adapted from in-process/subprocess to deploy/port-forward/`kubectl cp`. + +## Relationship to other integration tests + +This is **one** integration-test spec among several indexed by the master PRD +([../PRD.md](../PRD.md), § *Integration test specifications*). + +- **Discovery-driven sibling of `policy-pipeline`** ([policy-pipeline.md](policy-pipeline.md), + `testing/5.3-policy-pipeline-integration-test.md`): identical scenario facts and truth tables, but this + test *infers* the agent/tool entities via **real UC-1 onboarding of deployed workloads** instead of + hand-provisioning them, so its Rego is semantically similar (not byte-identical) and it validates the + **phase-1** deliverable end-to-end. +- Same `@pytest.mark.integration` + `opa eval` flavor as the live-Keycloak pytest tests + (`testing/5.1-integration-tests.md`) and `policy-pipeline`; skips when `opa` is absent. + +Tracking issue for this test: `testing/5.4-uc1-onboarding-integration-test.md`. + +## Out of Scope + +- **Writing `test_uc1_onboarding_pipeline.py`, `probe_uc1.rego`, `scenario_uc1.py`** — this spec + *describes* the test; the test is written in a later session (tracked by + `testing/5.4-uc1-onboarding-integration-test.md`). +- **The UC-1 agent, PRB, PCE, OPA writer, and demo `github-agent` implementations** — specified/tested by + their own components/issues, not here. In particular UC-1's discovery naming and single-generic-role + behavior are **fixed**; this test observes them, it does not change them. +- **Building the simplified `github-tool`** — its own build issue (a `blocked-by` of this test); the tool + is specified in [../demo/github-tool.md](../demo/github-tool.md). +- **Live enforcement / A2A traffic / token exchange / K8s-CR Policy Writer** — Phase-2+; this test targets + the filesystem stub and evaluates rules, per phase-1 out-of-scope. +- **Default-CI wiring** — `@pytest.mark.integration`; runs on demand. + +## Further Notes + +- **Fact triad.** The role→access facts are owned by three artefacts that must agree: the *Scenario* + table, **both** `policy.md` variants, and the `scenario_uc1.py` pair-lists. Entity/role/scope + descriptions are generic and functional and must not contradict the facts. +- **Two variants in the discovered world** (see *Scenario inputs*): an **explicit** variant that + enumerates each `(user-role → discovered scope)` pair by its full prefixed name, and an **abstract** + variant (phase-1's intent-only prose). **Neither names the agent role** — doing so would populate + `target_ok` and break both the user-gate-only decision and cross-variant equivalence. Both are + user-intent-only, both leave `target_ok` degenerate, and both must produce the **same discovered grant + set**. +- **`devops` zero access** is conveyed by its role description only; it is absent from every pair-list and + both `policy.md` variants (deny-by-default denies it inbound and on every outbound function). +- **Descriptions ≤255 chars, verbatim into Keycloak.** The tool-scope descriptions are the verbatim + scenario descriptions the simplified tool returns from `tools/list`; the agent-scope descriptions come + from the AgentCard skills; the realm-role descriptions are provisioned by the fixture. + +## Blocked-by + +- Simplified `github-tool` build issue (see [../demo/github-tool.md](../demo/github-tool.md)). +- Demo `github-agent` implementation — `demo/GA-1…GA-9` (deployable agent + AgentCard). +- UC-1 Service Onboarding — `agent/service-onboarding/3.6-service-onboarding-orchestrator.md` (**done**). +- The PRB / PCE / OPA-writer / Policy-Store prerequisites shared with `5.3`. +- A live Kagenti/Kind cluster + operator (registers clients into `AIAC_TEST_REALM` via the demo + namespace's `authbridge-config` `KEYCLOAK_REALM`, set by the fixture — per-namespace, confirmed against + the operator source); the `protocol.kagenti.io/mcp` Service label applied at deploy time + (`../../gh-issues/kagenti-operator-mcp-label-stamping.md`); an `opa` binary at test time. + +## Scenario inputs + +These are **functional** inputs — the PRB reads the descriptions and the `policy.md` to produce the +role→scope mappings. The entity/role/scope descriptions are **generic and keyword-free**; client `type` +is set by UC-1 from the `kagenti.io/type` label (not description prose). + +### Discovered entities (what UC-1 provisions) + +- **`github-tool`** (Tool) → scopes, from the simplified tool's MCP `tools/list` (verbatim descriptions): + - `github-tool.source-read` — "Read source repository contents: file listings and file bodies. Read-only." + - `github-tool.source-write` — "Create, modify, or delete source repository contents; commit file changes." + - `github-tool.issues-read` — "Read issues and their comment threads. Read-only." + - `github-tool.issues-write` — "Create and update issues: open, edit, comment, and close." +- **`github-agent`** (Agent) → role `github-agent.agent` (description "Agent role") + scopes from the + AgentCard skills: + - `github-agent.source_operations` — "Browse and search code; read, create, and modify repository file contents, branches, and commits." + - `github-agent.issue_operations` — "Read, search, create, and update issues, comments, sub-issues, and pull requests." + +### Realm roles (provisioned by the fixture) + +- `developer` — "Developer — an engineering user who develops the source codebase (writing and maintaining code) and fixes code defects reported in the issue tracker; works primarily in source and consults issues for defect reports." +- `tester` — "Tester — a quality-assurance user who verifies software quality and tracks defects through the issue tracker: filing, triaging, and updating issue reports; works in the issue tracker, not in source." +- `devops` — "DevOps — an operations user who manages deployment infrastructure and runtime environments; does not author source code and does not manage the issue tracker." + +### `policy.md` — Version 1 (explicit) + +Enumerates each `(user-role → discovered scope)` pair by name. **No agent-role→tool-scope section** +(target_ok is deferred). Deny by default. + +```markdown +# Access Control Policy — github-agent / github-tool + +Grant access on a least-privilege basis. Only grant a (role, scope) pair when this +policy supports it; deny by default. + +## Users → agent capabilities (inbound; user may call the agent) +- developer may use github-agent.source_operations and github-agent.issue_operations. +- tester may use github-agent.issue_operations. + +## Users → tool operations (outbound subject; user may reach the tool) +- developer may perform github-tool.source-read, github-tool.source-write, and github-tool.issues-read. +- tester may perform github-tool.issues-read and github-tool.issues-write. +``` + +### `policy.md` — Version 2 (abstract) + +Phase-1's intent-only prose. Encodes the same facts; relies on the PRB/LLM to expand intent into the +discovered scopes via the entity/role descriptions. + +```markdown +Grant access on a least-privilege basis: allow only what this policy states; deny by default. + +- Developers may read and modify source, and read issues. +- Testers may read and modify issues. +```