Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bff) crud for api/v1/model_registry/{model_registry_id}/artifacts #720

Merged
merged 2 commits into from
Jan 21, 2025

Conversation

ederign
Copy link
Member

@ederign ederign commented Jan 20, 2025

Description

This PR introduces the endpoints for GET, POST, and PATCH for artifacts.

In this PR:

  • clients/ui/bff/README.md: some formating and adding the new API endpoints
  • clients/ui/bff/internal/api/app.go: adding the new routes GET ../artifacts ../artifacts/{id}, POST and PATCH
  • clients/ui/bff/internal/api/artifacts_handler.go: handler for the following endpoints
  • clients/ui/bff/internal/api/artifacts_handler_test.go: companion test
  • clients/ui/bff/internal/api/suite_test.go and clients/ui/bff/internal/api/test_utils.go: code formating
  • clients/ui/bff/internal/mocks/k8s_mock.go: redundant error handling
  • clients/ui/bff/internal/mocks/model_registry_client_mock.go: simple mocks for when MOCK_MR_CLIENT=true
  • clients/ui/bff/internal/mocks/static_data_mock.go: companion mocks
  • clients/ui/bff/internal/mocks/types_mock.go: changed the ArtifactType for a hardcode value to avoid serialization issues
  • clients/ui/bff/internal/mocks/types_mock.go: repository for artifacts
  • clients/ui/bff/internal/repositories/model_registry_client.go: adding new repository for our ModelRegistryClient

How Has This Been Tested?

Tested both with mocked and real model registry. With MOCK_MR_CLIENT=false:

# GET /api/v1/model_registry/{model_registry_id}/artifacts
curl -i -H "kubeflow-userid: [email protected]" "http://localhost:4000/api/v1/model_registry/model-registry/artifacts?namespace=kubeflow"
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 20 Jan 2025 21:03:27 GMT
Transfer-Encoding: chunked

{
	"data": {
		"items": [
			{
				"artifactType": "model-artifact",
				"createTimeSinceEpoch": "1729029766683",
				"customProperties": {},
				"description": "fsdfsd",
				"id": "1",
				"lastUpdateTimeSinceEpoch": "1729029766683",
				"name": "fsdfsd-fsdfsd-artifact",
				"state": "LIVE",
				"uri": "s3://fsd/fsdfsd?endpoint=fsd\u0026defaultRegion=fsd"
			},
			...
			{
				"artifactType": "model-artifact",
				"createTimeSinceEpoch": "1737406383829",
				"customProperties": {},
				"description": "MNIST digit classification model trained on TensorFlow",
				"externalId": "33fdsfdmzodel-12345678",
				"id": "16",
				"lastUpdateTimeSinceEpoch": "1737406383829",
				"modelFormatName": "tensorfl2ow",
				"modelFormatVersion": "2.339.30",
				"name": "dora-classififsdfsder-v3333432",
				"storageKey": "models/mnist/1.0.0",
				"storagePath": "/models/mnist/1.0.0/model.savedmodel",
				"uri": "gs://my-models/mnist33-classifier/v234"
			}
		],
		"nextPageToken": "",
		"pageSize": 0,
		"size": 16
	}
}
# GET /api/v1/model_registry/{model_registry_id}/artifacts/{artifact_id}
curl -i -H "kubeflow-userid: [email protected]" "http://localhost:4000/api/v1/model_registry/model-registry/artifacts/{artifact_id}?namespace=kubeflow"
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 20 Jan 2025 21:04:06 GMT
Content-Length: 545

{
	"data": {
		"artifactType": "model-artifact",
		"createTimeSinceEpoch": "1737394565358",
		"customProperties": {},
		"description": "MNIST digit classification model trained on TensorFlow",
		"externalId": "mzodel-12345678",
		"id": "14",
		"lastUpdateTimeSinceEpoch": "1737394565358",
		"modelFormatName": "tensorfl2ow",
		"modelFormatVersion": "2.9.30",
		"name": "dora-classifier-v33432",
		"storageKey": "models/mnist/1.0.0",
		"storagePath": "/models/mnist/1.0.0/model.savedmodel",
		"uri": "gs://my-models/mnist33-classifier/v234"
	}
}
# POST /api/v1/model_registry/{model_registry_id}/artifacts
# externalId and name should be unique
curl -i \                                                                                                                                           
  -H "kubeflow-userid: [email protected]" \
  -X POST "http://localhost:4000/api/v1/model_registry/model-registry/artifacts?namespace=kubeflow" \
  -H "Content-Type: application/json" \
  -d '{
    "data": {
      "artifactType": "model-artifact",
      "name": "dora-classifsdier-v2",
      "description": "MNIST digit classification model trained on TensorFlow",
      "uri": "gs://my-models/mnist-classifier/v2",
      "externalId": "mfsdfodel-12345678",
      "modelFormatName": "tensorflow",
      "modelFormatVersion": "2.9.0",
      "storageKey": "models/mnist/1.0.0",
      "storagePath": "/models/mnist/1.0.0/model.savedmodel"
    }
  }'
HTTP/1.1 201 Created
Content-Type: application/json
Location: /api/v1/model_registry/model-registry/artifacts/17?namespace=kubeflow
Date: Mon, 20 Jan 2025 21:04:34 GMT
Content-Length: 540

{
	"data": {
		"artifactType": "model-artifact",
		"createTimeSinceEpoch": "1737407074319",
		"customProperties": {},
		"description": "MNIST digit classification model trained on TensorFlow",
		"externalId": "mfsdfodel-12345678",
		"id": "17",
		"lastUpdateTimeSinceEpoch": "1737407074319",
		"modelFormatName": "tensorflow",
		"modelFormatVersion": "2.9.0",
		"name": "dora-classifsdier-v2",
		"storageKey": "models/mnist/1.0.0",
		"storagePath": "/models/mnist/1.0.0/model.savedmodel",
		"uri": "gs://my-models/mnist-classifier/v2"
	}
}
# PATCH /api/v1/model_registry/{model_registry_id}/artifacts/{artifact_id}
curl -i -H "kubeflow-userid: [email protected]" -X PATCH "http://localhost:4000/api/v1/model_registry/model-registry/artifacts/1?namespace=kubeflow" \
     -H "Content-Type: application/json" \
-d '{ "data": {
  "artifactType": "model-artifact",
  "description": "New description 2"
}}'

This is currently broken on MR side but it will be fixed by https://github.com/kubeflow/model-registry/pull/718

Merge criteria:

  • All the commits have been signed-off (To pass the DCO check)
  • The commits have meaningful messages; the author will squash them after approval or in case of manual merges will ask to merge with squash.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work.
  • Code changes follow the kubeflow contribution guidelines.

Signed-off-by: Eder Ignatowicz <[email protected]>
Copy link
Contributor

@lucferbux lucferbux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Tested and working (except the part that needs a mr fix) great work here @ederign thanks, this is needed for the midstream update.

Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lucferbux

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit 39beea8 into kubeflow:main Jan 21, 2025
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants