fix: set pod anti affinity for docker-registry deployment and statefulset

[supershal]

What type of PR is this?

Sets pod anti affinity for docker-registry pods and deployments
exposes nodeAffinity configuration for docker-registry.

What this PR does/ why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:
Directly setting soft pod anti affinity on the stateful set to always distribute the pods on different nodes.
This is done directly in templates so that we can use helper template functions in matching labels.

I tested using single and multi CP node kind clusters.

Does this PR introduce a user-facing change?:

Checklist

• If a chart is changed, the chart version is correctly incremented.
• The commit message explains the changes and why are needed.
• The code builds and passes lint/style checks locally.
• The relevant subset of integration tests pass locally.
• The core changes are covered by tests.
• The documentation is updated where needed.

[supershal]

tolerations and nodeAffinity overrides will be set in CAREN's values.yaml.

[dkoshkin]

You don't think we can get away without this change and just leave it to the client to set the affinity values directly? This chart already exposes .Values.podLabels that can be used to select.

Making this change deviates us further from the upstream chart and removes existing ability to set podAffinity

[supershal]

The main chart does not use .Values.podLabels. but this patch does.

charts/staging/docker-registry/patch/nutanix/patch_2_support_statefulset.patch

Line 147 in f69eb21

+ {{- with .Values.podLabels }}

Can you explain how we apply this patched chart in CAREN?

[supershal]

As we discussed, we will instead use use matching cluster uuid in the labels and provide that value to the affinity using values.podLabels . closing this PR.