HBASE-29310 Handle Bulk Load Operations in Continuous Backup #7150
+171
−25
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull Request Overview
This PR enhances bulk load tooling and point-in-time restore (PITR) to handle bulk-load operations in continuous backup workflows.
- Adds a post-bulkload suggestion to take a backup
- Extends PITR validation to detect bulk-loaded files since the last backup and adds a
--forceoption - Updates model and storage of bulk-load timestamps and adapts tests and CLI to support the new flow
Reviewed Changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| BulkLoadHFilesTool.java | Prints a suggestion to back up after successful bulk load |
| TestPointInTimeRestore.java | Exposed overloaded buildPITRArgs with force flag and added force restore tests |
| TestIncrementalBackupWithContinuous.java | Made test counters dynamic, updated performBulkLoad and loadTable signatures |
| BulkLoad.java | Introduced timestamp field and getter |
| BackupSystemTable.java | Captured cell timestamp and passed it into the BulkLoad constructor |
| BackupAdminImpl.java | Propagated isForce through PITR validation, added checkBulkLoadAfterBackup |
| PointInTimeRestoreRequest.java | Added force parameter to the request model |
| PointInTimeRestoreDriver.java | Parsed -f/--force CLI flag and wired it into the request builder |
| BackupRestoreConstants.java | Defined constants for the force-restore option |
Comments suppressed due to low confidence (1)
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupAdminImpl.java:633
- The Javadoc for
validatePitrdoes not mention the newisForceparameter. Please update the method’s Javadoc to describe this argument.
private void validatePitr(long endTime, TableName[] sTableArray, TableName[] tTableArray,
Comment on lines
+1199
to
+1200
| System.out.println("Bulk load completed successfully."); | ||
| System.out.println("IMPORTANT: Please take a backup of the table immediately if this table " |
There was a problem hiding this comment.
Use the framework’s logger (e.g., LOG.info) instead of System.out.println for consistency with the rest of HBase’s logging.
Suggested change
| System.out.println("Bulk load completed successfully."); | |
| System.out.println("IMPORTANT: Please take a backup of the table immediately if this table " | |
| LOG.info("Bulk load completed successfully."); | |
| LOG.info("IMPORTANT: Please take a backup of the table immediately if this table " |
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
this is not related, you can ignore it.
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BulkLoad.java
Show resolved
Hide resolved
| @@ -69,6 +72,12 @@ protected int executeRestore(boolean check, TableName[] fromTables, TableName[] | |||
| return -1; | |||
| } | |||
|
|
|||
| boolean force = cmd.hasOption(OPTION_FORCE_RESTORE); | |||
| if (force) { | |||
| LOG.debug("Found force option (-{}) in restore command, " | |||
There was a problem hiding this comment.
[nitpick] This debug-level message may not be visible in default logs; consider logging at INFO or printing a user-facing warning so users know they’ve forced a restore.
Suggested change
| LOG.debug("Found force option (-{}) in restore command, " | |
| LOG.info("Found force option (-{}) in restore command, " |
Copilot uses AI. Check for mistakes.
taklwu
reviewed
Jul 11, 2025
Comment on lines
119
to
121
| String OPTION_FORCE_RESTORE = "f"; | ||
| String LONG_OPTION_FORCE_RESTORE = "force"; | ||
| String OPTION_FORCE_RESTORE_DESC = |
There was a problem hiding this comment.
nit: can you explain why should we support this use case if missing backup could cause data loss ?
| @@ -69,6 +72,12 @@ protected int executeRestore(boolean check, TableName[] fromTables, TableName[] | |||
| return -1; | |||
| } | |||
|
|
|||
| boolean force = cmd.hasOption(OPTION_FORCE_RESTORE); | |||
| if (force) { | |||
| LOG.debug("Found force option (-{}) in restore command, " | |||
Comment on lines
+1199
to
+1200
| System.out.println("Bulk load completed successfully."); | ||
| System.out.println("IMPORTANT: Please take a backup of the table immediately if this table " |
There was a problem hiding this comment.
this is not related, you can ignore it.
|
also, please try to fix the checkstyle and javac if possible. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
kgeisz
approved these changes
Jul 14, 2025
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
🎊 +1 overall
This message was automatically generated. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR addresses these 2 improvements:
Logging Suggestion during successful bulkload operation:
After a bulk load operation completes, log a message suggesting the user perform a full or incremental backup.
PITR Enhancements:
During PITR, check if any bulk load operation occurred after the last successful backup.
If no such backup exists, inform the user and fail the process.
JIRA: https://issues.apache.org/jira/browse/HBASE-29310