Skip to content

Conversation

@charlesconnell
Copy link
Contributor

I argue that in addition to flushing the master region, the flushMasterStore() RPC should also roll the WAL files being written by the HMaster.

In clusters with small numbers of DataNodes (less than 9, in my experience), a rolling restart of the DataNodes will break WAL writers, in both HMasters and RegionServers. This can be worked around on RegionServers by calling admin.rollWALWriter() on every RegionServer after restarting each DataNode. Currently, there is no equivalent way to do this to an HMaster. I think the most elegant solution is to augment admin.flushMasterStore().

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 12s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+1 💚 mvninstall 5m 10s master passed
+1 💚 compile 4m 16s master passed
+1 💚 checkstyle 1m 14s master passed
+1 💚 spotbugs 2m 17s master passed
+1 💚 spotless 1m 7s branch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚 mvninstall 4m 43s the patch passed
+1 💚 compile 4m 16s the patch passed
+1 💚 javac 4m 16s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 10s the patch passed
+1 💚 spotbugs 2m 22s the patch passed
+1 💚 hadoopcheck 14m 25s Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
+1 💚 spotless 0m 58s patch has no errors when running spotless:check.
_ Other Tests _
+1 💚 asflicense 0m 13s The patch does not generate ASF License warnings.
51m 13s
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7407/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #7407
JIRA Issue HBASE-29682
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname Linux 375e5ee70007 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / b4c4ae9
Default Java Eclipse Adoptium-17.0.11+9
Max. process+thread count 71 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7407/1/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache9
Copy link
Contributor

Apache9 commented Oct 24, 2025

Which WAL implementation do you use? We should be able to deal with datanode restart. The data which has been hflushed should be OK, and once there is a write error, we will try rolling the WAL and then write the data out again.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 12s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 44s master passed
+1 💚 compile 1m 16s master passed
+1 💚 javadoc 0m 36s master passed
+1 💚 shadedjars 7m 0s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 4m 19s the patch passed
+1 💚 compile 1m 13s the patch passed
+1 💚 javac 1m 13s the patch passed
+1 💚 javadoc 0m 35s the patch passed
+1 💚 shadedjars 6m 51s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 454m 18s hbase-server in the patch passed.
485m 11s
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7407/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #7407
JIRA Issue HBASE-29682
Optional Tests javac javadoc unit compile shadedjars
uname Linux 2ca77c563354 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / b4c4ae9
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7407/1/testReport/
Max. process+thread count 3093 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7407/1/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants