Update DRA APIs to v1

[nojnhuh]

Kubernetes 1.34 introduces the resource.k8s.io/v1 version for the DRA APIs. The example driver should demonstrate how to implement and use the latest and greatest API. These are the rough steps to do that:

1. Update all of the k8s.io dependencies to the latest 1.34 pre-release version (currently v1.34.0-rc.1). See Bump k8s.io dependencies to 1.33 #95 for an example (which also includes some changes specific to that upgrade that won't apply here).

2. Importing the new k8s.io dependencies will require updating the driver itself to operate against the new API version. Imports like this should all be updated to use v1 instead of v1beta1:
   

   dra-example-driver/cmd/dra-example-kubeletplugin/driver.go

   Line 24 in fc2d0ba

   resourceapi "k8s.io/api/resource/v1beta1"

3. Update the validating webhook to use v1:
   

   dra-example-driver/cmd/dra-example-webhook/main.go

   Line 30 in fc2d0ba

   resourceapi "k8s.io/api/resource/v1beta1"

4. Update the kind config to remove the config enabling API versions and the feature gate. The v1 API and the DRA feature gate are enabled by default and I think it's valuable to exercise that.

5. Update YAML manifests to the new version, e.g. in examples, the Helm chart, and sample output.

6. Call out the new minimum required Kubernetes version in the Helm chart as 1.34:
   

   dra-example-driver/deployments/helm/dra-example-driver/Chart.yaml

   Line 28 in fc2d0ba

   kubeVersion: ">=1.32.0-0"

The differences between the APIs are fairly straightforward, e.g. from kubernetes/kubernetes@5c4f817:

• ResourceSlice no longer contains the basic level of indirection:

  diff --git a/staging/src/k8s.io/dynamic-resource-allocation/structured/internal/allocatortesting/allocator_testing.go b/staging/src/k8s.io/dynamic-resource-allocation/structured/internal/allocatortesting/allocator_testing.go
  index ebd22fc217eeb..3d8c81b1fde01 100644
  --- a/staging/src/k8s.io/dynamic-resource-allocation/structured/internal/allocatortesting/allocator_testing.go
  +++ b/staging/src/k8s.io/dynamic-resource-allocation/structured/internal/allocatortesting/allocator_testing.go
  @@ -257,16 +259,14 @@ const (
   // generate a Device object with the given name, capacity and attributes.
   func device(name string, capacity any, attributes map[resourceapi.QualifiedName]resourceapi.DeviceAttribute) wrapDevice {
    device := resourceapi.Device{
  -		Name: name,
  -		Basic: &resourceapi.BasicDevice{
  -			Attributes: attributes,
  -		},
  +		Name:       name,
  +		Attributes: attributes,
    }

• ResourceClaims now have an additional exactly level of indirection:

  diff --git a/test/e2e/dra/test-driver/deploy/example/pod-inline-multiple.yaml b/test/e2e/dra/test-driver/deploy/example/pod-inline-multiple.yaml
  index 520935c59265a..59cd7b63b59f5 100644
  --- a/test/e2e/dra/test-driver/deploy/example/pod-inline-multiple.yaml
  +++ b/test/e2e/dra/test-driver/deploy/example/pod-inline-multiple.yaml
  @@ -1,4 +1,4 @@
  -apiVersion: resource.k8s.io/v1beta1
  +apiVersion: resource.k8s.io/v1
   kind: ResourceClaimTemplate
   metadata:
     name: pause-template
  @@ -11,7 +11,8 @@ spec:
       devices:
         requests:
         - name: my-device
  -        deviceClassName: example
  +        exactly:
  +          deviceClassName: example
         config:
         - opaque:
             driver: test-driver.cdi.k8s.io

For reference, the v1beta2 APIs are documented here, which are equivalent to v1: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/

/assign @alimaazamat

[nojnhuh]

As @alimaazamat has found out the hard way before I thought of it, the kubelet plugin Helper for 1.33 still using v1beta1 in its API makes it impractical to use v1beta2 as widely as I'd hoped in the kubelet plugin binary here. Maybe we can re-scope this a bit more narrowly to focus only on updating the examples? That would pare down the list above into:

• Enable v1beta2 in the API server in the kind config
• Update examples to use v1beta2

I also forgot that we could update the validating webhook to also be aware of the new version like the NVIDIA driver is doing in NVIDIA/k8s-dra-driver-gpu#461. That isn't strictly necessary to do right away since the API server can handle the conversion before invoking our webhook, so let's save that for a follow-up for now.

[pohly]

Should we turn this issue into "use v1" using the v1.35 helper? Using v1beta2 as an intermediate step doesn't make that much sense anymore given how close we are to releasing v1.35.

[nojnhuh]

Jumping to v1 sounds good to me. My only concern was that the last release is pinned to Kubernetes 1.32 (i.e. fails on both older and newer versions), so raising the floor in the next release to 1.34 would leave a gap at 1.33. I think it's reasonable to steer users to build and install the driver from source in that case, especially since I haven't heard anyone ask for a new release of the example driver for 1.33 yet. Getting a head start on 1.34 seems more important at the moment.

/retitle Update DRA APIs to v1

@alimaazamat I've also updated the original description here to reflect this change.

[klueska]

I would recommend following what was done in NVIDIA/k8s-dra-driver-gpu@920a287, which allows the driver to work across all of v1, v1beta1, and v1beta2 by leveraging the helper in the staging repo.

[pohly]

I was about to say that: when using the helper from v1.34, a DRA driver should be compatible with Kubernetes >= v1.32.

The upstream DRA upgrade/downgrade testing covers that.

[alimaazamat]

Thank you so much for the help. One thing I noticed is here: NVIDIA/k8s-dra-driver-gpu
doesn't have kubeVersion in Chart.yaml.
With the muliversion client maybe we do not need to have this constraint? Or either way it should stay kubeVersion: ">=1.32.0-0" ?

[klueska]

I think you could change it to >= 1.32.0
What is the -0 at the end of the original one?

[klueska]

Looks like the -0 means include pre-releases, which we probably want to drop at this point.

[nojnhuh]

I think you could change it to >= 1.32.0 What is the -0 at the end of the original one?

That's primarily for GKE which advertises its versions like 1.32.0-gke. I'm not sure if that's still the case. I think @mortent pointed that out a while ago?