diff --git a/skills/k8s-launch-kit-config/SKILL.md b/skills/k8s-launch-kit-config/SKILL.md index 186d65fa..d95f6eee 100644 --- a/skills/k8s-launch-kit-config/SKILL.md +++ b/skills/k8s-launch-kit-config/SKILL.md @@ -162,7 +162,7 @@ networkNamespaces: ["my-namespace"] - Use `l8k schema` to discover the Network Operator release keys supported by the installed l8k version. - `nvIpam` subnets are auto-generated if not specified — one per rail using non-routable ranges. -- `docaDriver.unloadThirdPartyRDMAModules: true` auto-populates `UNLOAD_THIRD_PARTY_RDMA_MODULES` from discovered OFED-dependent modules. +- `docaDriver.unloadThirdPartyRDMAModules: true` sets the `UNLOAD_THIRD_PARTY_RDMA_MODULES` environment variable to `"true"` (a boolean flag) in generated manifests. Discovered OFED-dependent modules are saved as `thirdPartyRDMAModules` for visibility and warnings, not passed to that environment variable. - For release 26.1+, SR-IOV requestor mode requires both the Network Operator drain requestor and the SR-IOV external drainer. l8k renders both; applying only CRs cannot enable their Deployment environment variables. - Updating an existing release to the generated requestor-mode Helm values requires `--overwrite-existing`. diff --git a/skills/k8s-launch-kit-config/references/config-reference.md b/skills/k8s-launch-kit-config/references/config-reference.md index 06a9e53e..9d945090 100644 --- a/skills/k8s-launch-kit-config/references/config-reference.md +++ b/skills/k8s-launch-kit-config/references/config-reference.md @@ -62,7 +62,7 @@ docaDriver: enableNFSRDMA: false # bool | default: false, auto-enabled by discovery when third-party RDMA modules are found - # When true, adds UNLOAD_THIRD_PARTY_RDMA_MODULES env var to the ofedDriver container. + # When true, sets UNLOAD_THIRD_PARTY_RDMA_MODULES env var to "true" in the ofedDriver container. # Third-party RDMA modules are blacklisted and unloaded before OFED driver reload. # Discovery automatically sets this to true when third-party RDMA modules are detected. unloadThirdPartyRDMAModules: true diff --git a/skills/k8s-launch-kit-discover/references/discovery-internals.md b/skills/k8s-launch-kit-discover/references/discovery-internals.md index 56704267..8278af55 100644 --- a/skills/k8s-launch-kit-discover/references/discovery-internals.md +++ b/skills/k8s-launch-kit-discover/references/discovery-internals.md @@ -150,9 +150,11 @@ directory to find modules that depend on OFED. Common dependents include: - `nvidia_peermem` -- modern GPUDirect RDMA peer memory module. - `mlx5_vdpa` -- vDPA offload module. -The discovered dependents are saved per group as `thirdPartyRDMAModules`. These are used -during manifest generation to configure the NicClusterPolicy's `ofedDriver` section -with the correct secondary module list. +The discovered dependents are saved per group as `thirdPartyRDMAModules` for visibility +and warnings. When `unloadThirdPartyRDMAModules` is true, manifest generation sets the +`UNLOAD_THIRD_PARTY_RDMA_MODULES` environment variable to `"true"` (a boolean flag) in +the NicClusterPolicy's `ofedDriver` section; the module names themselves are not passed +to that variable. --- diff --git a/skills/k8s-launch-kit-troubleshoot/references/common-failures.md b/skills/k8s-launch-kit-troubleshoot/references/common-failures.md index 2be86425..1bf3f57e 100644 --- a/skills/k8s-launch-kit-troubleshoot/references/common-failures.md +++ b/skills/k8s-launch-kit-troubleshoot/references/common-failures.md @@ -17,8 +17,8 @@ failures or dependency errors. 1. Check `thirdPartyRDMAModules` in your cluster config -- these modules must be unloaded. 2. Set `docaDriver.unloadThirdPartyRDMAModules: true` in your l8k config. 3. Re-run discovery (`--discover-cluster-config`) to refresh the dependent modules list. -4. Redeploy. The generated NicClusterPolicy will include `UNLOAD_THIRD_PARTY_RDMA_MODULES` env var - in the ofedDriver section. +4. Redeploy. The generated NicClusterPolicy will include `UNLOAD_THIRD_PARTY_RDMA_MODULES: "true"` + (a boolean flag) in the ofedDriver section. 5. If the issue persists, exec into the OFED pod and check `/sys/module/*/holders/` to find additional unlisted dependents. diff --git a/skills/k8s-network-engineer/references/config-schema.md b/skills/k8s-network-engineer/references/config-schema.md index 9eaee496..a756f859 100644 --- a/skills/k8s-network-engineer/references/config-schema.md +++ b/skills/k8s-network-engineer/references/config-schema.md @@ -32,9 +32,11 @@ Controls DOCA/OFED driver deployment in the NicClusterPolicy. | `enableNFSRDMA` | bool | `false` | Enable NFS over RDMA support | | `unloadThirdPartyRDMAModules` | bool | `true` | Unload kernel modules that depend on MLX/OFED drivers | -When `unloadThirdPartyRDMAModules` is true and dependent modules are discovered, -the generated NicClusterPolicy includes `UNLOAD_THIRD_PARTY_RDMA_MODULES` env var -(space-separated module names) in the ofedDriver section. +When `unloadThirdPartyRDMAModules` is true, the generated NicClusterPolicy +sets the `UNLOAD_THIRD_PARTY_RDMA_MODULES` environment variable to `"true"` +(a boolean flag) in the ofedDriver section. Dependent modules discovered on each +group are saved as `thirdPartyRDMAModules` for visibility and warnings, but are +not passed as module names to that environment variable. ## maintenance