Skip to content

Commit c541a8e

Browse files
Merge pull request #6 from neeraj-laad/stretch-cluster
chore: Editorial fixes
2 parents b46b29f + 662bd5b commit c541a8e

File tree

1 file changed

+12
-10
lines changed

1 file changed

+12
-10
lines changed

083-stretch-cluster.md

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,21 @@
11
# Stretch Kafka cluster
22

33
The Strimzi Kafka operator currently manages Kafka clusters within a single Kubernetes cluster.
4-
This proposal aims to extend support to stretch Kafka clusters, where brokers and controllers of a single Kafka cluster are distributed across multiple Kubernetes clusters.
4+
This proposal aims to extend this support to stretch Kafka clusters, where brokers and controllers of a single Kafka cluster are distributed across multiple Kubernetes clusters.
55

66
## Current situation
77

8-
At present, the availability of Strimzi-managed Kafka clusters is directly tied to the availability of the underlying Kubernetes cluster.
8+
At present, the availability of Strimzi-managed Kafka clusters is limited by the availability of the underlying Kubernetes cluster.
99
If a Kubernetes cluster experiences an outage, the entire Kafka cluster becomes unavailable, disrupting all connected Kafka clients.
1010

1111
## Motivation
1212

13-
A stretch Kafka cluster allows Kafka nodes to be distributed across multiple Kubernetes clusters, significantly enhancing resilience by enabling the system to tolerate the outage of an entire Kubernetes cluster without disrupting service to clients.
14-
This configuration ensures high availability and seamless client operations, even in the event of cluster-specific failures.
13+
A stretch Kafka cluster allows Kafka nodes to be distributed across multiple Kubernetes clusters. This approach also facilitates maby valuable use cases, such as:
1514

16-
In addition to improving fault tolerance, this approach also facilitates other valuable use cases, such as:
15+
- **High Availability**: Distributing Kafka brokers across multiple Kubernetes cluster significantly enhances resilience by enabling the system to tolerate the outage of a Kubernetes cluster without disrupting service to clients.
16+
17+
- **Migration Flexibility**: The ability to move Kafka nodes between Kubernetes environments without downtime, supporting maintenance or migrations.
1718

18-
- **Migration Flexibility**: The ability to move Kafka clusters between Kubernetes environments without downtime, supporting maintenance or migrations.
1919
- **Resource Optimization**: Efficiently utilizing resources across multiple clusters, which can be advantageous in environments with varying cluster capacities or during scaling operations.
2020

2121
### Limitations and Considerations
@@ -24,18 +24,20 @@ While a stretch Kafka cluster offers several advantages, it also introduces some
2424
- **Increased Network Complexity and Costs**: The communication between brokers and controllers across clusters relies on network connectivity, which can be less reliable and more costly than intra-cluster communication.
2525
This necessitates careful consideration of network architecture and associated costs.
2626

27-
- **Latency Requirements**: The stretch Kafka cluster is best suited for environments with low-latency network connections between the Kubernetes clusters.High latency can adversely affect the performance and synchronization of Kafka nodes, potentially leading to delays or errors in replication and client communication.
27+
- **Latency Requirements**: Stretch Kafka clusters are best suited for environments with low-latency network connections between the Kubernetes clusters.
28+
High latency can adversely affect the performance and synchronization of Kafka nodes, potentially leading to delays or errors in replication and client communication.
2829
Defining the minimal acceptable latency between clusters is crucial to ensure optimal performance.
2930

3031
## Proposal
3132

3233
This proposal seeks to enhance the Strimzi Kafka operator to support stretch Kafka clusters, distributing brokers and controllers across multiple Kubernetes clusters.
33-
The intent is to focus on high-availability of the data plane.The proposal outlines high-level topology and design concepts for such deployments, with a plan to incrementally include finer design and implementation details for various aspects.
34+
The intent is to focus on high-availability of the data plane.
35+
The proposal outlines high-level topology and design concepts for such deployments, with a plan to incrementally include finer design and implementation details for various aspects.
3436

3537
### Prerequisites
3638

3739
- **Multiple Kubernetes Clusters**: Stretch Kafka clusters will require multiple Kubernetes clusters.
38-
Ideally, an odd number of clusters (at least three) is needed to maintain quorum in the event of a cluster outage.
40+
Ideally, an odd number of clusters (at least three) will be needed to maintain Kafka controller quorum and tolerate a Kubernetes cluster outage.
3941

4042
- **Low Latency**: Kafka clusters should be deployed in environments that allow low-latency communication between Kafka brokers and controllers.
4143
Stretch Kafka clusters should be deployed in environments such as data centers or availability zones within a single region, and not across distant regions where high latency could impair performance.
@@ -45,7 +47,7 @@ While Zookeeper-based deployments are still supported, they are outside the scop
4547

4648
### Design
4749

48-
The cluster operator will be deployed in all Kubernetes clusters and will manage Kafka brokers/controllers running on that cluster.
50+
The cluster operator will be deployed in all Kubernetes clusters and will manage Kafka brokers and controllers running on that cluster.
4951
One Kubernetes cluster will act as the control point for defining custom resources (Kafka, KafkaNodePool) required for stretch Kafka cluster.
5052
The KafkaNodePool custom resource will be extended to include information about a Kubernetes cluster where the pool should be deployed.
5153
The cluster operator will create necessary resources (StrimziPodSets, services etc.) on the target clusters specified within the KafkaNodePool resource.

0 commit comments

Comments
 (0)