Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions charts/kagenti-operator/templates/manager/manager.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,10 @@ spec:
- name: KAGENTI_SPIRE_TRUST_DOMAIN
value: {{ .Values.signatureVerification.spireTrustDomain | quote }}
{{- end }}
{{- if and .Values.signatureVerification .Values.signatureVerification.spireClusterName }}
- name: KAGENTI_SPIRE_CLUSTER_NAME
value: {{ .Values.signatureVerification.spireClusterName | quote }}
{{- end }}
{{- if .Values.controllerManager.container.env }}
{{- range $key, $value := .Values.controllerManager.container.env }}
- name: {{ $key }}
Expand Down
7 changes: 7 additions & 0 deletions charts/kagenti-operator/templates/rbac/role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -251,10 +251,17 @@ rules:
- apiGroups:
- operator.openshift.io
resources:
- spiffecsidrivers
- spireagents
- spireoidcdiscoveryproviders
- spireservers
- zerotrustworkloadidentitymanagers
verbs:
- create
- get
- list
- update
- watch
- apiGroups:
- operator.tekton.dev
resources:
Expand Down
2 changes: 2 additions & 0 deletions charts/kagenti-operator/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,8 @@ signatureVerification:
# Set here only to override auto-discovery. Used by ClientRegistrationReconciler
# to build SPIFFE-shaped Keycloak client IDs (spiffe://<trust-domain>/ns/<ns>/sa/<sa>).
spireTrustDomain: ""
# SPIRE cluster name for the ZTWIM CR. Defaults to "agent-platform" if empty.
spireClusterName: ""
# Key within the SPIRE trust bundle ConfigMap. Matches the SPIRE hardened Helm chart default
# and the binary flag default. Override to "bundle.crt" only for older ZTWIM deployments.
spireTrustBundle:
Expand Down
29 changes: 29 additions & 0 deletions kagenti-operator/cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -743,6 +743,35 @@ func main() {
}
}

if controller.SpireOperandCRDExists(mgr.GetConfig()) {
if spireTrustDomain == "" {
setupLog.Info("ZTWIM CRDs present but SPIRE trust domain not available; " +
"SPIRE operand controller will not start until trust domain is discoverable")
} else {
spireClusterName := os.Getenv("KAGENTI_SPIRE_CLUSTER_NAME")
if err := mgr.Add(&controller.SpireBootstrapRunnable{
Client: mgr.GetClient(),
TrustDomain: spireTrustDomain,
ClusterName: spireClusterName,
Log: ctrl.Log.WithName("spire-bootstrap"),
}); err != nil {
setupLog.Error(err, "unable to add SPIRE bootstrap runnable")
os.Exit(1)
}
if err = (&controller.SpireOperandReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
Recorder: mgr.GetEventRecorderFor("spire-operand-controller"),
TrustDomain: spireTrustDomain,
ClusterName: spireClusterName,
}).SetupWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "SpireOperand")
os.Exit(1)
}
setupLog.Info("SPIRE operand controller enabled", "trustDomain", spireTrustDomain)
}
}

// Validation webhooks
// For local testing without webhook certificates, set ENABLE_WEBHOOKS=false:
// ENABLE_WEBHOOKS=false ./bin/manager --leader-elect=false [other flags...]
Expand Down
7 changes: 7 additions & 0 deletions kagenti-operator/config/rbac/role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -250,10 +250,17 @@ rules:
- apiGroups:
- operator.openshift.io
resources:
- spiffecsidrivers
- spireagents
- spireoidcdiscoveryproviders
- spireservers
- zerotrustworkloadidentitymanagers
verbs:
- create
- get
- list
- update
- watch
- apiGroups:
- operator.tekton.dev
resources:
Expand Down
85 changes: 85 additions & 0 deletions kagenti-operator/docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,14 @@ The Kagenti Operator is a Kubernetes controller that implements the [Operator Pa
- On CR deletion: removes type label, managed-by label and config-hash annotation (causing the workload to lose sidecars)
- Coordinates with the AuthBridge mutating webhook (in-process) which injects sidecars at Pod CREATE time

#### SPIRE Operand Controller
- Creates and reconciles 5 SPIRE operand CRs (`operator.openshift.io/v1alpha1`) when ZTWIM CRDs are present on the cluster
- CRD-gated: no feature flag needed — automatically activates on OpenShift 4.19+ where ZTWIM operator is installed
- Manages: ZeroTrustWorkloadIdentityManager (parent), SpiffeCSIDriver, SpireServer, SpireAgent, SpireOIDCDiscoveryProvider (children)
- Drift correction: any manual change to CR spec or deletion is automatically corrected
- Replaces fragile Helm post-install hooks with proper reconciliation loop
- Uses `SpireBootstrapRunnable` to create initial ZTWIM CR at startup, triggering the controller's watch

### Supporting Components

#### Webhooks
Expand Down Expand Up @@ -108,13 +116,24 @@ graph TB
CardController[AgentCard Controller]
SyncController[AgentCardSync Controller]
RuntimeController[AgentRuntime Controller]
SpireController[SPIRE Operand Controller]
CardCR -->|Validates| ValidationWebhook
RuntimeCR -->|Validates| ValidationWebhook
Deployment -->|CREATE/UPDATE with kagenti.io/type| VAP

ValidationWebhook -->|Valid CR| CardController
end

subgraph "SPIRE Infrastructure"
ZTWIMCR[ZTWIM CR]
SpireCRs[SpireServer + SpireAgent + SpiffeCSIDriver + SpireOIDC]
ZTWIMOperator[ZTWIM Operator]
SpireController -->|Creates/Updates| ZTWIMCR
SpireController -->|Creates/Updates| SpireCRs
ZTWIMOperator -->|Reconciles| ZTWIMCR
ZTWIMOperator -->|Reconciles| SpireCRs
end

subgraph "Config Sources"
ClusterCM[Cluster Defaults ConfigMaps]
NsCM[Namespace Defaults ConfigMap]
Expand Down Expand Up @@ -261,6 +280,60 @@ AgentRuntime CR created/updated
| `ConfigResolved` | Configuration merged successfully. Reason is `ConfigResolved` when clean, `ConfigWarning` when ambiguity detected (e.g., multiple namespace defaults ConfigMaps). Warnings are surfaced in the condition message and as Kubernetes events. |
| `Ready` | Labels and config-hash applied successfully |

### SPIRE Operand Controller

The SPIRE Operand Controller manages the lifecycle of 5 SPIRE operand CRs on OpenShift clusters where the ZTWIM (Zero Trust Workload Identity Manager) operator is installed. It replaces fragile Helm post-install hooks with a proper reconciliation loop that corrects drift.

#### CRD Gating

The controller uses `SpireOperandCRDExists()` to check for the `ZeroTrustWorkloadIdentityManager` CRD via the Kubernetes discovery API (3 retries). No feature flag is needed — the controller activates automatically when CRDs are present (OpenShift 4.19+).

#### Bootstrap Flow

`SpireBootstrapRunnable` is a one-shot `manager.Runnable` that creates the initial ZTWIM CR at startup if absent. This triggers the controller's watch, which then creates the 4 child CRs.

#### Reconciliation Flow

```
1. Get ZTWIM CR "cluster" — if NotFound, return (bootstrap pending)
2. ensureUnstructuredCR(ZTWIM) — CreateOrUpdate with desired spec:
- trustDomain (auto-discovered), clusterName="agent-platform", bundleConfigMap="spire-bundle"
3. ensureChildren() — CreateOrUpdate for each of 4 children:
a. SpiffeCSIDriver: agentSocketPath, pluginName
b. SpireServer: caSubject, persistence, datastore, jwtIssuer
c. SpireAgent: nodeAttestor, workloadAttestors
d. SpireOIDCDiscoveryProvider: csiDriverName, jwtIssuer
4. Record events on create/update, return success
```

Children inherit `trustDomain` and `clusterName` from the parent ZTWIM CR — these fields are NOT set on child CRs (OCP 4.19 CRD constraint).

#### Managed CRs

All CRs are `operator.openshift.io/v1alpha1`, cluster-scoped, name `"cluster"`:

| CR | Key Spec Fields | Role |
|---|---|---|
| ZeroTrustWorkloadIdentityManager | trustDomain, clusterName, bundleConfigMap | Parent — created first |
| SpiffeCSIDriver | agentSocketPath, pluginName | CSI volume plugin for SVID mounting |
| SpireServer | caSubject, persistence, datastore, jwtIssuer | SPIRE server configuration |
| SpireAgent | nodeAttestor, workloadAttestors | Node-level SPIRE agent |
| SpireOIDCDiscoveryProvider | csiDriverName, jwtIssuer | OIDC endpoint for JWT-SVID |

#### Watches

| Resource | Scope | Purpose |
|----------|-------|---------|
| ZeroTrustWorkloadIdentityManager | Cluster | Primary resource — reconcile on create/update/delete |
| SpiffeCSIDriver | Cluster | Secondary — map to ZTWIM reconcile for drift correction |
| SpireServer | Cluster | Secondary — map to ZTWIM reconcile for drift correction |
| SpireAgent | Cluster | Secondary — map to ZTWIM reconcile for drift correction |
| SpireOIDCDiscoveryProvider | Cluster | Secondary — map to ZTWIM reconcile for drift correction |

#### Drift Correction

Uses `controllerutil.CreateOrUpdate` — if a CR exists but spec differs from desired state, it is updated. If a CR is deleted, it is recreated on next reconcile (triggered by the child watch). All CRs are labeled `app.kubernetes.io/managed-by: kagenti-operator`.

### NetworkPolicy Controller

The NetworkPolicy Controller enforces network isolation based on signature verification.
Expand Down Expand Up @@ -356,6 +429,18 @@ Source: `internal/controller/agentcard_networkpolicy_controller.go`
| `networking.k8s.io` | `networkpolicies` | get, list, watch, create, update, patch, delete | Create permissive/restrictive NetworkPolicies based on signature verification |
| `""` (core) | `pods` | get, list, watch, update, patch | Resolve pod selectors from workload pod template labels |

#### SPIRE Operand Controller Permissions

Source: `internal/controller/spire_operand_controller.go`

| API Group | Resources | Verbs | Purpose |
|-----------|-----------|-------|---------|
| `operator.openshift.io` | `zerotrustworkloadidentitymanagers` | create, get, list, update, watch | Create and reconcile ZTWIM parent CR |
| `operator.openshift.io` | `spiffecsidrivers` | create, get, list, update, watch | Create and reconcile SpiffeCSIDriver CR |
| `operator.openshift.io` | `spireservers` | create, get, list, update, watch | Create and reconcile SpireServer CR |
| `operator.openshift.io` | `spireagents` | create, get, list, update, watch | Create and reconcile SpireAgent CR |
| `operator.openshift.io` | `spireoidcdiscoveryproviders` | create, get, list, update, watch | Create and reconcile SpireOIDCDiscoveryProvider CR |

#### Cross-Namespace Considerations

- The operator uses a **ClusterRole** and **ClusterRoleBinding** in cluster-wide mode, allowing it to reconcile resources across all namespaces
Expand Down
3 changes: 3 additions & 0 deletions kagenti-operator/docs/operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,9 @@ Reconciles `AgentCard` CRs by resolving the backing workload via `spec.targetRef
### AgentCard Sync Controller
Watches Deployments and StatefulSets labeled with `kagenti.io/type=agent` and one or more `protocol.kagenti.io/<name>` labels (e.g., `protocol.kagenti.io/a2a`). Automatically creates AgentCard resources with `targetRef` pointing to the discovered workloads. Sets owner references for garbage collection.

### SPIRE Operand Controller
Creates and reconciles 5 SPIRE operand CRs (`operator.openshift.io/v1alpha1`) when ZTWIM CRDs are present on the cluster. CRD-gated — no feature flag needed, automatically activates on OpenShift 4.19+. Manages ZeroTrustWorkloadIdentityManager (parent), SpiffeCSIDriver, SpireServer, SpireAgent, and SpireOIDCDiscoveryProvider (children). Corrects drift on any manual change or deletion. Uses a bootstrap runnable to create the initial ZTWIM CR at startup.

### NetworkPolicy Controller
Watches AgentCard resources when `--enforce-network-policies` is enabled. Creates permissive NetworkPolicies for agents with verified signatures and restrictive NetworkPolicies for unverified agents. When identity binding is configured, both signature and binding must pass for permissive access.

Expand Down
Loading
Loading