Skip to content

KAFKA-18561: Remove withKip853Rpc and replace it with withRaftProtocol #18600

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 41 commits into
base: trunk
Choose a base branch
from

Conversation

frankvicky
Copy link
Contributor

JIRA: KAFKA-18561
Remove withKip853Rpc and replace it with withRaftProtocol and adjust the corresponding test.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@github-actions github-actions bot added triage PRs from the community tests Test fixes (including flaky tests) kraft labels Jan 17, 2025
@frankvicky
Copy link
Contributor Author

Hi @ahuang98
Please take a look when you have a free cycle.
Many thanks 🙇🏼

@ijuma ijuma requested a review from jsancio January 18, 2025 13:39
@frankvicky frankvicky force-pushed the KAFKA-18561 branch 2 times, most recently from c36f9bc to 312c113 Compare January 21, 2025 02:29
@ijuma
Copy link
Member

ijuma commented Jan 22, 2025

@jsancio You're ok with this?

Copy link

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

@frankvicky frankvicky force-pushed the KAFKA-18561 branch 2 times, most recently from b93875b to fabd9f6 Compare January 26, 2025 02:04
Copy link

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

@frankvicky frankvicky force-pushed the KAFKA-18561 branch 2 times, most recently from 4e56589 to 543467d Compare January 29, 2025 02:55
Copy link

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

@frankvicky frankvicky force-pushed the KAFKA-18561 branch 2 times, most recently from e135aed to 826ba56 Compare January 31, 2025 07:42
@ahuang98
Copy link
Contributor

ahuang98 commented Jan 31, 2025

@ijuma I introduced withRaftProtocol and believe @jsancio is okay with this change. I'll review first

Copy link
Contributor

@ahuang98 ahuang98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this change it's more obvious that KafkaRaftClientTest only covers KIP_853_PROTOCOL and lower, I wonder if it would be worth doing one of the following

  • have KafkaRaftClientTest test the whole range of RaftProtocols
  • replace KIP_853_PROTOCOL with latestProtocol() which will pick the last enum constant in RaftProtocol, and KafkaRaftClientTest will always test the newest protocol
  • java doc / rename for KafkaRaftClientTest to indicate this only covers a subset of the raft protocols

@@ -2265,4 +2267,17 @@ public ByteBuffer buffer() {
return data;
}
}

private boolean isSupport853(RaftProtocol raftProtocol) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can use RaftClientTestContext.isReconfigSupported()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.
I don't notice that we have such a helper method.

Comment on lines 67 to 68
private static final String KIP_595_PROTOCOL = "KIP_595_PROTOCOL";
private static final String KIP_853_PROTOCOL = "KIP_853_PROTOCOL";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my slight pref would be to remove these and just use the strings directly in the EnumSource annotations

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion should also be applied to KafkaRaftClientTest if you agree

@CsvSource({ "false,false", "false,true", "true,false", "true,true" })
public void testLeaderListenerNotified(boolean entireLog, boolean withKip853Rpc) throws Exception {
@MethodSource("generateKip853TestMatrix")
public void testLeaderListenerNotified(boolean entireLog, RaftProtocol raftProtocol) throws Exception {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess another option would be just renaming the second parameter to boolean isReconfigSupported, potentially easier for readers to know at a glance that we're not testing the entire set of raftProtocols (rather than navigating to the method source definition)

(then we would do .withRaftProtocol(isReconfigSupported ? KIP_853_PROTOCOL : KIP_595_PROTOCOL) which I understand is a bit bulkier)

ReplicaKey voter1 = replicaKey(localId + 1, withKip853Rpc);
ReplicaKey voter2 = replicaKey(localId + 2, withKip853Rpc);
ReplicaKey observer3 = replicaKey(localId + 3, withKip853Rpc);
boolean support853 = isSupport853(raftProtocol);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: support853 -> reconfigSupported

Copy link
Contributor

@ahuang98 ahuang98 Mar 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was this change applied everywhere? we should remove isSupport853() since it's a dupe of reconfigSupported()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ahuang98
Sorry for inconvience. Previously, I had a finger issue when rebasing.
Now all comments should be applied. PTAL

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there was no inconvenience 😅 sorry to hear about your finger! I missed your update today and will review tomorrow morning

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you 😆

@github-actions github-actions bot removed needs-attention triage PRs from the community labels Feb 1, 2025
@frankvicky frankvicky requested a review from jsancio April 18, 2025 03:45
@frankvicky
Copy link
Contributor Author

Hi @jsancio
Since this patch changes the base logic of the test, it would be painful when conflicts happen.
Could you please take a look when you have a free moment?

Many thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-approved kraft tests Test fixes (including flaky tests)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants