Users cannot create TrainedModel and inferencegraph resources

[gigabyte132]

Validation Checklist

• Is this a Kubeflow issue?
• Are you posting in the right repository ?
• Did you follow the Kubeflow installation guideline ?
• Is the issue report properly structured and detailed with version numbers?
• Is this for Kubeflow development ?
• Would you like to work on this issue?
• You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

Version

1.9

Describe your issue

The default kubeflow user service account forbids users to create resources of type TrainedModel, this is useful for a bunch of serving use cases where you want to serve multiple models in the same InferenceService.

I will shortly open a PR to fix this issue as well, as it is just a matter of adding trainedmodels to the kubeflow-kserve-edit and kserve-kubeflow-view ClusterRoles in contrib/kserve/kserve/kserve_kubeflow.yaml

Versions:
kubeflow: 1.9.1
k8s: v1.30.2

Steps to reproduce the issue

As a normal kubeflow user, in multi-user mode at least, you will get the following error when trying to create a resource of type TrainedModel from inside a Notebook. the serviceaccounts default-editor and default-viewer do not currently have permissions to create/view such a resource.

Error from server (Forbidden): error when retrieving current configuration of:
Resource: "serving.kserve.io/v1alpha1, Resource=trainedmodels", GroupVersionKind: "serving.kserve.io/v1alpha1, Kind=TrainedModel"
Name: "simple-string", Namespace: "rchiores"
from server for: "model2.yaml": trainedmodels.serving.kserve.io "simple-string" is forbidden: User "system:serviceaccount:rchiores:default-editor" cannot get resource "trainedmodels" in API group "serving.kserve.io" in the namespace "rchiores"

Put here any screenshots or videos (optional)

No response

[juliusvonkohout]

Please do not modify contrib/kserve/kserve/kserve_kubeflow.yaml directly. This must be fixed upstream and synchronized here with https://github.com/kubeflow/manifests/blob/master/hack/synchronize-kserve-kserve-manifests.sh. So fix it upstream first please

[gigabyte132]

Closing, in favor of opening another issue upstream in kserve.

[juliusvonkohout]

But lets leave it open here.

[gigabyte132]

Relevant Upstream issue with PR kserve/kserve#4224 kserve/kserve#4225