[Design] How to handle modules in operator

[utdrmac]

This issue is an open discussion on how to handle modules within the operator. There were proposals, and discussions (#118) on the implementation for valkey configuration parameters, along with modules. It was then realized that modules are not currently part of the valkey/valkey:9.0.0 image that the operator uses. Thus, any attempt to specify a module config would fail to start the cluster. It was decided to move module support out of #44, and into a separate PR, and open this discussion.

There are currently two proposals:

• Create an operator-specific image that combines the base valkey binary, and all the modules in github.com/valkey-io/valkey-bundle
• Use the valkey-bundle image as an init-container, and copy/mount the modules into the pod

If you have a solution proposal, or comment, feel free to put below. For extra credit, consider if and how other 3rd party modules could also be supported.

[hieu2102]

I think option 1 is better, user can create custom images if they wanted to add more modules.

For the config, maybe just a simple config map with a single key module.conf? and then append the content to the main config file in the pod creation flow, so that

• the module load order is preserved
• we don't have to use include <sub-config-file.conf>. Since doing so will duplicate the loadmodule entries after running CONFIG REWRITE, thus crashing the pod if it restart

[sandeepkunusoth]

i think we need to provide user to use their own image with custom modules. and specify those in CR spec as part of module config.