OCPNODE-3747: Disable Swap mode in Kubelet and enable drop-in directory

[ngopalak-redhat]

Fixes: https://issues.redhat.com/browse/OCPNODE-3747
Context:
With the release of 1.34 version of Kubernetes swap feature in Kubelet is GAed. This means that, customer can set FailSwapOn to false and SwapBehavior to LimitedSwap. OpenShift 4.21 is currently planned to adopt 1.34 version of Kubernetes. Hence users of OpenShift 4.21 and above are impacted by this upstream release of swap feature.

- What I did

1. Ensure that FailSwapOn and MemorySwap.SwapBehavior cannot be set by users of OpenShift
2. Ensure that FailSwapOn is set to false on worker nodes and true on all other types of nodes. MemorySwap.SwapBehavior is always set to "NoSwap"
3. Ensure that drop-in directory is enabled on the worker node kubelet so that CNV team can use it to set swapBehavior (won't be documented for external users)

- How to verify it

• Created a cluster in clusterBot with this PR and ensured that failSwapOn and swapBehavior is set

Worker Nodes:
  - All 3 worker nodes: failSwapOn: false, swapBehavior: "NoSwap"

  Control Plane Nodes:
  - All 3 master nodes: failSwapOn: true, swapBehavior: "NoSwap"

  The worker nodes have swap failure disabled (failSwapOn: false) while control plane nodes have it enabled (failSwapOn: true). All nodes have swapBehavior
  set to "NoSwap".

While this was tested the following PR was also required: openshift/api#2494 (Merged already)

The cluster bot command was:

launch 4.21,openshift/machine-config-operator#5294,openshift/api#2494 gcp

• Then created a drop-in file

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
memorySwap:
  swapBehavior: LimitedSwap

Checked the output of configz

oc get --raw "/api/v1/nodes/ci-ln-sptb06t-72292-d64jj-worker-a-4zrm5/proxy/configz" | \
jq '.kubeletconfig.memorySwap.swapBehavior'
"LimitedSwap"

Note to reviewer
Unit tests are included. The end-to-end (E2E) tests will be added to the origin repository, as defined by the work items in the Epic: OCPNODE-3646.

I've chosen the origin repository to ensure maintain consistency with other node features (ex: image volume).

- Description for the changelog

Disables openshift users to use swap mode on the Kubelet

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[ngopalak-redhat]

/test all

[ngopalak-redhat]

/test all

[ngopalak-redhat]

/test unit

[ngopalak-redhat]

/retest-required

[ngopalak-redhat]

/retest-required

[openshift-ci-robot]

@ngopalak-redhat: This pull request references OCPNODE-3747 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

Fixes: https://issues.redhat.com/browse/OCPNODE-3747
Context:
With the release of 1.34 version of Kubernetes swap feature in Kubelet is GAed. This means that, customer can set FailSwapOn to false and SwapBehavior to LimitedSwap. OpenShift 4.21 is currently planned to adopt 1.34 version of Kubernetes. Hence users of OpenShift 4.21 and above are impacted by this upstream release of swap feature.

- What I did

1. Ensure that FailSwapOn and MemorySwap.SwapBehavior cannot be set by users of OpenShift
2. Ensure that FailSwapOn is set to false on worker nodes and true on all other types of nodes. MemorySwap.SwapBehavior is always set to "NoSwap"
3. Ensure that drop-in directory is enabled on the worker node kubelet so that CNV team can use it to set swapBehavior (won't be documented for external users)

- How to verify it

• Created a cluster in clusterBot with this PR and ensured that failSwapOn and swapBehavior is set

Worker Nodes:
 - All 3 worker nodes: failSwapOn: false, swapBehavior: "NoSwap"

 Control Plane Nodes:
 - All 3 master nodes: failSwapOn: true, swapBehavior: "NoSwap"

 The worker nodes have swap failure disabled (failSwapOn: false) while control plane nodes have it enabled (failSwapOn: true). All nodes have swapBehavior
 set to "NoSwap".

While this was tested the following PR was also required: openshift/api#2494

The cluster bot command was:

launch 4.21,openshift/machine-config-operator#5294,openshift/api#2494 gcp

• Then created a drop-in file

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
memorySwap:
 swapBehavior: LimitedSwap

Checked the output of configz

oc get --raw "/api/v1/nodes/ci-ln-sptb06t-72292-d64jj-worker-a-4zrm5/proxy/configz" | \
jq '.kubeletconfig.memorySwap.swapBehavior'
"LimitedSwap"

- Description for the changelog

Disables openshift users to use swap mode on the Kubelet

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[ngopalak-redhat]

/verified later @BhargaviGudi

[openshift-ci-robot]

@ngopalak-redhat: This PR has been marked to be verified later by @BhargaviGudi.

In response to this:

/verified later @BhargaviGudi

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@ngopalak-redhat: This pull request references OCPNODE-3747 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

Fixes: https://issues.redhat.com/browse/OCPNODE-3747
Context:
With the release of 1.34 version of Kubernetes swap feature in Kubelet is GAed. This means that, customer can set FailSwapOn to false and SwapBehavior to LimitedSwap. OpenShift 4.21 is currently planned to adopt 1.34 version of Kubernetes. Hence users of OpenShift 4.21 and above are impacted by this upstream release of swap feature.

- What I did

1. Ensure that FailSwapOn and MemorySwap.SwapBehavior cannot be set by users of OpenShift
2. Ensure that FailSwapOn is set to false on worker nodes and true on all other types of nodes. MemorySwap.SwapBehavior is always set to "NoSwap"
3. Ensure that drop-in directory is enabled on the worker node kubelet so that CNV team can use it to set swapBehavior (won't be documented for external users)

- How to verify it

• Created a cluster in clusterBot with this PR and ensured that failSwapOn and swapBehavior is set

Worker Nodes:
 - All 3 worker nodes: failSwapOn: false, swapBehavior: "NoSwap"

 Control Plane Nodes:
 - All 3 master nodes: failSwapOn: true, swapBehavior: "NoSwap"

 The worker nodes have swap failure disabled (failSwapOn: false) while control plane nodes have it enabled (failSwapOn: true). All nodes have swapBehavior
 set to "NoSwap".

While this was tested the following PR was also required: openshift/api#2494

The cluster bot command was:

launch 4.21,openshift/machine-config-operator#5294,openshift/api#2494 gcp

• Then created a drop-in file

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
memorySwap:
 swapBehavior: LimitedSwap

Checked the output of configz

oc get --raw "/api/v1/nodes/ci-ln-sptb06t-72292-d64jj-worker-a-4zrm5/proxy/configz" | \
jq '.kubeletconfig.memorySwap.swapBehavior'
"LimitedSwap"

Note to reviewer
Unit tests are included. The end-to-end (E2E) tests will be added to the origin repository, as defined by the work items in the Epic: OCPNODE-3646.

I've chosen the origin repository to ensure maintain consistency with other node features (ex: image volume).

- Description for the changelog

Disables openshift users to use swap mode on the Kubelet

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[ngopalak-redhat]

@haircommander Can you please review?

[openshift-ci-robot]

@ngopalak-redhat: This pull request references OCPNODE-3747 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

Fixes: https://issues.redhat.com/browse/OCPNODE-3747
Context:
With the release of 1.34 version of Kubernetes swap feature in Kubelet is GAed. This means that, customer can set FailSwapOn to false and SwapBehavior to LimitedSwap. OpenShift 4.21 is currently planned to adopt 1.34 version of Kubernetes. Hence users of OpenShift 4.21 and above are impacted by this upstream release of swap feature.

- What I did

1. Ensure that FailSwapOn and MemorySwap.SwapBehavior cannot be set by users of OpenShift
2. Ensure that FailSwapOn is set to false on worker nodes and true on all other types of nodes. MemorySwap.SwapBehavior is always set to "NoSwap"
3. Ensure that drop-in directory is enabled on the worker node kubelet so that CNV team can use it to set swapBehavior (won't be documented for external users)

- How to verify it

• Created a cluster in clusterBot with this PR and ensured that failSwapOn and swapBehavior is set

Worker Nodes:
 - All 3 worker nodes: failSwapOn: false, swapBehavior: "NoSwap"

 Control Plane Nodes:
 - All 3 master nodes: failSwapOn: true, swapBehavior: "NoSwap"

 The worker nodes have swap failure disabled (failSwapOn: false) while control plane nodes have it enabled (failSwapOn: true). All nodes have swapBehavior
 set to "NoSwap".

While this was tested the following PR was also required: openshift/api#2494 (Merged already)

The cluster bot command was:

launch 4.21,openshift/machine-config-operator#5294,openshift/api#2494 gcp

• Then created a drop-in file

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
memorySwap:
 swapBehavior: LimitedSwap

Checked the output of configz

oc get --raw "/api/v1/nodes/ci-ln-sptb06t-72292-d64jj-worker-a-4zrm5/proxy/configz" | \
jq '.kubeletconfig.memorySwap.swapBehavior'
"LimitedSwap"

Note to reviewer
Unit tests are included. The end-to-end (E2E) tests will be added to the origin repository, as defined by the work items in the Epic: OCPNODE-3646.

I've chosen the origin repository to ensure maintain consistency with other node features (ex: image volume).

- Description for the changelog

Disables openshift users to use swap mode on the Kubelet

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[haircommander]

the commit history is a bit wonky can you fix it up please?

[ngopalak-redhat]

the commit history is a bit wonky can you fix it up please?

@haircommander Fixed it. Thanks

[ngopalak-redhat]

/retest-required

[fabiand]

I cna't review the details, but just enabling the dir and rejecting it on the cluster API level looks correct.

🚀

#just-commenting-to-leave-an-emoji

[haircommander]

/approve
/lgtm

[haircommander]

/retest-required

[ngopalak-redhat]

/assign djoshy
As per #5294 (comment)

[ngopalak-redhat]

/verified later @BhargaviGudi

[openshift-ci-robot]

@ngopalak-redhat: This PR has been marked to be verified later by @BhargaviGudi.

In response to this:

/verified later @BhargaviGudi

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[djoshy]

/lgtm
/approve

Took a look and it looks sane to me. Will defer to node team's expertise for the swap mode details.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: djoshy, haircommander, ngopalak-redhat

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [djoshy]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[djoshy]

/test unit

seems like a flake

[openshift-ci]

@ngopalak-redhat: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-gcp-op-ocl	fc21115	link	false	/test e2e-gcp-op-ocl
ci/prow/okd-scos-e2e-aws-ovn	fc21115	link	false	/test okd-scos-e2e-aws-ovn
ci/prow/e2e-aws-mco-disruptive	fc21115	link	false	/test e2e-aws-mco-disruptive
ci/prow/e2e-azure-ovn-upgrade-out-of-change	fc21115	link	false	/test e2e-azure-ovn-upgrade-out-of-change
ci/prow/bootstrap-unit	fc21115	link	false	/test bootstrap-unit

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[ngopalak-redhat]

/test e2e-gcp-op-1of2