[Store] refactor: config loading and CLI parsing

[cocktail828]

Description

The current command-line parsing uses gflags, which doesn't support hot updates and has limited extensibility. Additionally, the various config structure definitions and initialization code are verbose and difficult to read. Furthermore, logging has never supported persistence and always prints to screen, making it unsuitable for production environments.

This PR replaces gflags with CLI11, which allows configuration and command-line parsing to be handled together while flexibly supporting parameter deprecation mechanisms. It also makes configuration hot-reloading possible with minimal overhead - no need to write redundant code for parsing configuration files, just re-call the Parser interface. Priority mechanisms are also supported (Command Line > Configuration File > Default Values).

Additionally, duplicate code related to Master config has been refactored and streamlined. This modification significantly simplifies master-related code.

Type of Change

• Types
  • Refactor
    • Transfer Engine
    • Mooncake Store
    • Mooncake EP
    • Integration
    • P2P Store
    • Python Wheel
  • Breaking change
  • CI/CD
  • Documentation update
  • Other

How Has This Been Tested?

• Added comprehensive unit tests for configuration loading and command-line parsing scenarios
• Verified existing integration tests continue to pass
• Manual testing on Linux to confirm proper config resolution when executing from different working directories
• Confirmed backward compatibility with existing command-line usage patterns

Checklist

• I have performed a self-review of my own code.
• I have updated the documentation.
• I have added tests to prove my changes are effective.

[gemini-code-assist]

Summary of Changes

Hello @cocktail828, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request undertakes a significant refactoring of the Mooncake Store's configuration loading and command-line interface parsing. The primary motivation was to address the limitations of the existing gflags-based system, which lacked support for hot configuration updates and extensibility. By migrating to CLI11, the system now benefits from a unified approach to CLI and file-based configuration, enabling dynamic updates, flexible parameter deprecation, and a clear hierarchy for setting values. Additionally, the internal MasterConfig structures have been streamlined, and logging capabilities have been enhanced for better production readiness.

Highlights

• Configuration and CLI Parsing Overhaul: Replaced the gflags library with CLI11 for command-line argument and configuration file parsing. This significantly enhances flexibility, supports hot-reloading of configurations, and provides a clear priority mechanism (Command Line > Configuration File > Default Values).
• Streamlined Master Configuration: Refactored and simplified the MasterConfig structure, removing verbose boilerplate code and consolidating various configuration classes into a single, more manageable MasterConfig struct. This includes integrating Glog logging configuration directly into MasterConfig.
• Enhanced Logging Management: Introduced a dedicated GlogConfig struct, inherited by MasterConfig, to centralize and improve logging setup. This allows for persistent logging to files, which was previously unsupported, and offers fine-grained control over log levels and rotation.
• Improved Testability and Maintainability: The refactoring simplifies the configuration initialization logic across the codebase, making it easier to understand, test, and maintain. Test utilities and existing tests have been updated to reflect the new configuration paradigm.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request is a significant and valuable refactoring of the configuration and command-line parsing logic. It greatly simplifies the configuration structure by replacing multiple config classes with a single MasterConfig and leveraging CLI11 for parsing, which is a commendable improvement. The introduction of a dedicated GlogConfig for logging is also a positive step towards better production-readiness.

My review focuses on a few areas where the refactoring might have introduced issues or could be further improved. I've identified a critical issue related to High Availability where the view_version is no longer propagated, which could impact client behavior. There are also a couple of high-severity issues with command-line option parsing and error handling. Additionally, I've included a suggestion for a minor performance improvement in the logging initialization.

Overall, this is an excellent pull request that enhances code quality and maintainability. Addressing the identified issues will help ensure the refactoring is complete and robust.

[gemini-code-assist]

The view_version obtained from mv_helper.ElectLeader is not being passed to the WrappedMasterService. In the previous implementation, this was handled by WrappedMasterServiceConfig. After the refactoring, WrappedMasterService's constructor only takes MasterConfig, and MasterConfig no longer has a view_version field. This means MasterService::view_version_ will be default-initialized to 0, which is incorrect for HA mode. The view version is critical for clients to detect master changes. This appears to be a critical regression.

A possible fix is to add view_version back to MasterConfig and initialize it in the MasterService constructor, or to change the constructors of WrappedMasterService and MasterService to accept the view_version as a separate argument.

[cocktail828]

As far as I’m aware, this variable doesn’t seem to be used anywhere in the code.