Setup hot-standby replica for clickhouse cluster

[hellais]

Currently our clickhouse cluster has 3 nodes, 2 data nodes and 3 zookeeper nodes.

Originally we were planning to add a third data node (data2.htz-fsn.prod.ooni.nu), however it's not possible to do that with the current disk setup on the host, since the disks are unevenly sized and a raid5 configuration would lead to too much wasted disk space. We could go for raid0, but that seems a bit risky to do on nvme drives.

In light of this, we should for the moment keep the cluster to be 2 nodes (data3 being the cold replica since it's HDD backed) and discontinue data2 to keep costs down.

For the time being we keep data2.htz-fsn.prod.ooni.nu a reserved hostname label for when we decide to finalize setting it up.

We probably want to do that once we have finished the backend-fsn and backend-hel migration so we can cut down the cost of our dedicated hosts.

[hellais]

I am going to request that data2 is terminated on hetzner.