Merging Configuration Files and Environment Variables

[ohkinozomu]

Hello,

• As the number of configurations increases, managing them becomes complex, so I would like to use a configuration file.
• In Kubernetes, IP addresses change dynamically, and at least for the Hashicorp Raft implementation, the bind-addr must be set to something other than 0.0.0.0 (while etcd allows this, I am still not sure if this is due to the comqtt implementation).

To address this, I am using initContainers to rewrite the bind-addr.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: node01
  namespace: comqtt
spec:
  serviceName: "node01"
  replicas: 1
  selector:
    matchLabels:
      app: node01
  template:
    metadata:
      labels:
        app: node01
        group: comqtt
    spec:
      initContainers:
      - name: copy-scripts
        image: busybox
        command:
        - sh
        - -c
        - >
          cp /scripts/entrypoint.sh /writable/entrypoint.sh &&
          chmod +x /writable/entrypoint.sh &&
          cp /configs/comqtt-config-template.yml /writable/comqtt-config-template.yml
        volumeMounts:
        - name: entrypoint-script
          mountPath: /scripts
        - name: config-template
          mountPath: /configs
        - name: writable-volume
          mountPath: /writable
      containers:
      - name: comqtt
        image: gcr.io/orelab/github.com/ohkinozomu/comqtt:f3e75ff4986431eec310a07fdc307f66c3958aa3
        ports:
        - containerPort: 1883
        - containerPort: 7946
        - containerPort: 8946
        - containerPort: 8080
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: RAFT_BOOTSTRAP
          value: "true"
        volumeMounts:
        - name: writable-volume
          mountPath: /writable
        - name: data-volume
          mountPath: /data
        command: ["/bin/sh", "-c"]
        args: ["/writable/entrypoint.sh"]
      volumes:
      - name: entrypoint-script
        configMap:
          name: comqtt-entrypoint
          defaultMode: 0777
      - name: config-template
        configMap:
          name: comqtt-config
      - name: writable-volume
        emptyDir: {}
      - name: data-volume
        persistentVolumeClaim:
          claimName: node01
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: node01
  namespace: comqtt
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: comqtt-entrypoint
  namespace: comqtt
data:
  entrypoint.sh: |
    #!/bin/sh
    sed -e "s/__POD_IP__/${POD_IP}/" -e "s/__POD_NAME__/${POD_NAME}/" -e "s/__RAFT_BOOTSTRAP__/${RAFT_BOOTSTRAP}/" /writable/comqtt-config-template.yml > /writable/comqtt-config.yml
    /comqtt-cluster -conf=/writable/comqtt-config.yml
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: comqtt-config
  namespace: comqtt
data:
  comqtt-config-template.yml: |
    storage-way: 3

    cluster:
      discovery-way: 0
      node-name: "__POD_NAME__"
      bind-addr: "__POD_IP__"
      bind-port: 7946
      members: [node01.comqtt.svc.cluster.local:7946,node02.comqtt.svc.cluster.local:7946,node03.comqtt.svc.cluster.local:7946]
      raft-impl: 0
      raft-port: 8946
      raft-bootstrap: __RAFT_BOOTSTRAP__
    redis:
      options:
        addr: valkey-master.valkey:6379
        password: {PASSWORD}

    mqtt:
      tcp: :1883
      http: :8080

    log:
      enable: true

This is complex, and it seems better if the configuration files and environment variables could be merged.
Any thoughts on this approach?