Change deployment strategy for agent when persistent volume is enabled

[jalev]

Why?

When the PVC feature is enabled you will run into the following issue when upgrading the chart:

Multi-Attach error for volume "pvc-401c5500-a013-4853-815d-0bfd7c43ad1e" Volume is already used by pod(s) cloudzero-agent-server-7f8cdfbb85-wjkdn

This is because the PVC claim is already being mounted by the previous version of the pod, while the new version of the pod is waiting for the claim to get released!

What

A quick fix for this is to change the rollout strategy for the deployment to Recreate. This will terminate the previously running pod and free up the PVC claim mount for use in the new pod.

How Tested

1. Enable PVC feature
2. Upgrade the helm chart
3. New agent-server container will work
4. Upgrade helm chart again (e.g. add label change)
5. New agent-server container will be blocked until previous container is deleted

[evan-cz]

Hey @jalev, thanks for the patch, we really appreciate it!

I opened up #562 to get it merged, since CI doesn't work for external contributors. Yet...

I changed the condition slightly to operate on server.persistentVolume. enabled instead of server.persistentVolume, but other than that all I did was add a bunch of tests to make sure we don't break this again :)