HBASE-29441 ReplicationSourceShipper should delegate the empty wal entries handling to ReplicationEndpoint #7145
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… of COMMITED for empty WAL batches
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
vinayakphegde
changed the title
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull Request Overview
This PR delegates empty WAL entry handling in ReplicationSourceShipper to each ReplicationEndpoint via a new EmptyEntriesPolicy enum and updates related code and tests.
- Introduce
EmptyEntriesPolicyandgetEmptyEntriesPolicy()inReplicationEndpointwith a default ofCOMMIT. - Update
ReplicationSourceShipper.shipEdits()to use endpoint policy for empty batches and add helpergetReplicationResult(). - Add sync and async endpoint implementations and corresponding tests; update
ContinuousBackupReplicationEndpointto returnSUBMIT.
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSource.java | Added AsyncReplicationEndpoint and SyncReplicationEndpoint tests for empty-batch behavior. |
| hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceShipper.java | Changed shipEdits visibility, delegated empty-batch logic using endpoint policy, and added getReplicationResult(). |
| hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationEndpoint.java | Added default getEmptyEntriesPolicy() method and documentation. |
| hbase-server/src/main/java/org/apache/hadoop/hbase/replication/EmptyEntriesPolicy.java | Introduced enum to distinguish COMMIT vs. SUBMIT for empty batches. |
| hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/replication/ContinuousBackupReplicationEndpoint.java | Override getEmptyEntriesPolicy() to return SUBMIT for async backup. |
Comments suppressed due to low confidence (3)
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceShipper.java:244
- [nitpick] The method name getReplicationResult is generic; consider renaming to getEmptyBatchReplicationResult or computeEmptyEntriesPolicyResult for clarity.
private ReplicationResult getReplicationResult() {
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceShipper.java:278
- [nitpick] Document the return value for updateLogPosition, explaining what
trueorfalseindicates about the update operation.
boolean updateLogPosition(WALEntryBatch batch, ReplicationResult replicated) {
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/EmptyEntriesPolicy.java:23
- [nitpick] The Javadoc bullet list in this enum is formatted inline; consider using standard Javadoc list tags (
<ul><li>) or separate lines for each policy for better readability.
* Policy that defines what a replication endpoint should do when the entry batch is empty. This is
| */ | ||
| private void shipEdits(WALEntryBatch entryBatch) { | ||
| void shipEdits(WALEntryBatch entryBatch) { |
There was a problem hiding this comment.
[nitpick] Consider annotating this package-private method with @VisibleForTesting (or a project-specific equivalent) to clearly signal its test-only visibility.
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
nit: VisibleForTesting seems the right comment, and it may save few words for testing only,
taklwu
approved these changes
Jul 15, 2025
| */ | ||
| private void shipEdits(WALEntryBatch entryBatch) { | ||
| void shipEdits(WALEntryBatch entryBatch) { |
There was a problem hiding this comment.
nit: VisibleForTesting seems the right comment, and it may save few words for testing only,
| /** | ||
| * Package-private for test visibility only. Do not use outside tests. | ||
| */ | ||
| boolean updateLogPosition(WALEntryBatch batch, ReplicationResult replicated) { |
There was a problem hiding this comment.
Got it.
I think we are not allowed to use @VisibleForTesting. I don't see any other instance in the codebase as well.
also, I see this
<restrictImports implementation="de.skuzzle.enforcer.restrictimports.rule.RestrictImports">
<includeTestCode>true</includeTestCode>
<commentLineBufferSize>512</commentLineBufferSize>
<reason>You should never use this style of annotations(i.e, 'this is for test only')
in IA.Public or IA.LimitedPrivate classes. Use IA.Private to tell users this is
not for public use.
For IA.Private classes, use RestrictedApi annotation in error prone instead.</reason>
<bannedImports>
<bannedImport>org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting</bannedImport>
</bannedImports>
</restrictImports>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.