Skip to content

Commit f8d3497

Browse files
committed
updated design images
Signed-off-by: Aswin A <[email protected]>
1 parent 8396b40 commit f8d3497

File tree

4 files changed

+6
-6
lines changed

4 files changed

+6
-6
lines changed

083-stretch-cluster.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
# Stretch Kafka cluster
22

3-
The Strimzi Kafka operator currently manages Kafka clusters within a single Kubernetes environment. This proposal aims to extend support to stretch Kafka clusters, where brokers and controllers are distributed across multiple Kubernetes clusters.
3+
The Strimzi Kafka operator currently manages Kafka clusters within a single Kubernetes cluster. This proposal aims to extend support to stretch Kafka clusters, where brokers and controllers of a single Kafka cluster are distributed across multiple Kubernetes clusters.
44

55
## Current situation
66

77
At present, the availability of Strimzi-managed Kafka clusters is directly tied to the availability of the underlying Kubernetes cluster. If a Kubernetes cluster experiences an outage, the entire Kafka cluster becomes unavailable, disrupting all connected Kafka clients.
88

99
## Motivation
1010

11-
By distributing Kafka nodes across multiple clusters, a stretch Kafka cluster can tolerate outages of individual Kubernetes clusters and will continue to serve clients seamlessly even if one of the clusters goes down.
11+
By evenly distributing Kafka nodes across multiple Kubernetes clusters, a stretch Kafka cluster can tolerate outages of individual Kubernetes clusters, and continue to serve clients seamlessly.
1212

1313
## Proposal
1414

@@ -29,10 +29,10 @@ The cluster operator will be deployed in all Kubernetes clusters and will manage
2929
This approach will allow users to specify/manage the definition of stretch Kafka cluster in a single location. The operators will then create necessary resources in target Kubernetes clusters, which can then be reconciled/managed by operators on those clusters.
3030

3131
### Reconciling Kafka and KafkaNodePool resources
32-
<img width="1321" alt="image" src="https://media.github.ibm.com/user/473362/files/0a1a4e7f-8227-47da-86b1-4b27a83fdb9d">
32+
![Reconciling Kafka and KafkaNodePool resources](./images/083-reconciling-kafka-knp.png)
3333

3434
### Reconciling StrimziPodSet resources
35-
<img width="1317" alt="image" src="https://media.github.ibm.com/user/473362/files/f4524847-3c9c-4a8c-a5fd-4a76b6ec0bf4">
35+
![Reconciling SPS](./images/083-reconciling-sps.png)
3636

3737
#### KafkaNodePool changes
3838
A new optional field (`target`) will be introduced in the KafkaNodePool resource specification, to allow users to specify the details of the Kubernetes cluster where the node pool should be deployed. This section will include the target cluster's URL (Kubernetes cluster where resources for this node pool will be created) and the secret containing the kubeconfig data for that cluster.
@@ -101,7 +101,7 @@ In a stretch Kafka cluster, we'll need bootstrap and broker services to be prese
101101
#### Cross-cluster communication
102102
Kafka controllers/brokers are distributed across multiple Kubernetes environments and will need to communicate with each other. Currently, the Strimzi Kafka operator defines Kafka listeners for internal communication (controlplane and replication) between brokers/controllers (Kubernetes services using ports 9090 and 9091). The user is not able to influence how these services are set up and exposed outside the cluster. We would remove this limitation and allow users to define how these internal listeners are configured in the Kafka resource, just like they do for Kafka client listeners.
103103

104-
Users will also be able to override listener configurations in each KafkaNodePool resource, if the listeners need to be exposed in different ways (ingress host names, Ingress annotations etc.) for each Kubernetes cluster. This will be similar to how KafkaNodePools are used to override other configuration like storage etc. To override a listener, KafkaNodePool will define configuration with same listner name as in Kafka resource.
104+
Users will also be able to override listener configurations in each KafkaNodePool resource, if the listeners need to be exposed in different ways (ingress host names, Ingress annotations etc.) for each Kubernetes cluster. This will be similar to how KafkaNodePools are used to override other configuration like storage etc. To override a listener, KafkaNodePool will define configuration with same listener name as in Kafka resource.
105105

106106
#### Resource cleanup on remote Kubernetes clusters
107107
As some of the Kubernetes resources will be created on a remote cluster, we will not be able to use standard Kubernetes approaches for deleting resources based on owner references. The operator will need to delete remote resources explicitly when the owning resource is deleted.
@@ -135,7 +135,7 @@ This proposal only impacts strimzi-kafka-operator project.
135135
- Use network technologies like skupper, submariner etc to allow internal services to be visible on other clusters
136136
- introduces additional dependencies and complexity to the Strimzi project
137137

138-
<img width="1057" alt="image" src="https://media.github.ibm.com/user/473362/files/9a9720f9-9ccf-4d4f-a56c-9a35917af711">
138+
![Synchronized ClusterOperator](./images/083-synchronized-clusteroperator.png)
139139

140140

141141
An alternative approach considered was setting up a stretch Kafka cluster with synchronized `KafkaStretchCluster` and `Kafka` custom resources (CRs). The idea was to introduce a new CR called `KafkaStretchCluster`, which would contain details of all the clusters involved in the stretch Kafka deployment. The spec would include information such as cluster names, secrets for connecting to each Kubernetes cluster, and a list of node pools across the entire stretch cluster.
1.75 MB
Loading

images/083-reconciling-sps.png

1.98 MB
Loading
2 MB
Loading

0 commit comments

Comments
 (0)