| Purpose | Description |
|---|---|
| Store basic metadata of the cluster | Metadata like info about brokers, topics, partitions, mapping of partitions to nodes, partition leader/followers, consumer offset etc. - ZooKeeper and etcd are designed to hold small amounts of data that can fit entirely in memory (although they still write to disk for durability). - That small amount of data is replicated across all the nodes using a fault-tolerant total order broadcast algorithm. |
| Leader Election in the cluster | Electing leader in the leader-follower replication, through consensus algorithm. |
| Controller election in the Cluster | Through consensus algorithm |
| Handle new broker/node addition or failure in the cluster | Notifies consumers, producers, routing tier of the arrival of new broker or failure of existing broker. |
| Route requests to partition leader | Route all requests to partition's leaders. |
- As an application developer, you will rarely need to use ZooKeeper or etcd directly, because it is actually not well suited as a general-purpose database.
- It is more likely that you will end up relying on it indirectly via some other project - Kafka (uses Zookeeper), Kubernates (uses etcd) etc.