Decouple Committer from Kafka and Enable Custom Coordinator Election #12460
Conversation
…rrent design itself
…rent_design_in_ICR_mode
…handle backward compatibility
…handle backward compatibility
… owned offsets and this avoida extra work
…_data_loss_in_current_design_in_ICR_mode_test Resiliency Testing For New Changes
…_data_loss_in_current_design_in_ICR_mode_test_latest Handling no coordinator and data loss in current design in icr mode test latest
…sLeaderPartition and isLeader
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions. |
|
@bryanck can we please take some decision on this. This will at least allow users to plug in their committer logic. |
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions. |
|
@bryanck can we also take decision on this one as this will give freedom to users to plug their committers. |
|
@ajantha-bhat can you please review this. |
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions. |
|
@bryanck is there any update on this one? |
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions. |
|
@bryanck can we please take a call on this as this is getting marked stale by GitHub repetitively. |
|
Handled by PR |
2 participants
Summary
This PR refactors the Committer implementation to remove its tight coupling with Kafka and provide users the flexibility to define their own coordinator election mechanism by implementing the Committer Interface. Currently, the coordinator is elected based on the lexicographically smallest source topic partition, which limits adaptability in different environments.
Changes Introduced