TFX 1.16.0 Issue

[vkarampudi]

Please comment or link any issues you find with TFX 1.16.0.

[pritamdodeja]

In TFX 1.15.1, I could use

model.save(fn_args.serving_model_dir, save_format='tf', signatures=signatures)

and the model would work as expected in tensorflow serving. Given that keras3 has deprecated the SavedModel format as per https://keras.io/guides/serialization_and_saving/, specifically, "The only supported format in Keras 3 is the "Keras v3" format, which uses the .keras extension.",

The use of

tf.saved_model.save(model, export_dir=fn_args.serving_model_dir, signatures=signatures)

does not work as expected in tensorflow serving. What is the right way to store the model such that it can served?

Secondly, does the solution for the above also solve it for the InfraValidator component?

[surajk998]

Hi Team,
I am currently working on upgrading the tfx library from 1.15.0 to 1.16.0 but I am facing these recurring issues with the kubeflow v2 APIs.

here, in 1.16.0
I have observed that we are storing the runtime parameters via attach_parameter method ( https://github.com/tensorflow/tfx/blob/v1.16.0/tfx/orchestration/kubeflow/v2/parameter_utils.py#L56 ) in parameter_utils module which has a dependency on ParameterContext class ( https://github.com/tensorflow/tfx/blob/v1.16.0/tfx/orchestration/kubeflow/v2/parameter_utils.py#L22 )

and in the PipelineBuilder class of pipeline_builder.py module,
for the exit handler StepBuilder, we have not used it inside the ParameterContext scope.
https://github.com/tensorflow/tfx/blob/master/tfx/orchestration/kubeflow/v2/pipeline_builder.py#L258

[pritamdodeja]

In TFX 1.15.1, I could use

model.save(fn_args.serving_model_dir, save_format='tf', signatures=signatures)

and the model would work as expected in tensorflow serving. Given that keras3 has deprecated the SavedModel format as per https://keras.io/guides/serialization_and_saving/, specifically, "The only supported format in Keras 3 is the "Keras v3" format, which uses the .keras extension.",

The use of

tf.saved_model.save(model, export_dir=fn_args.serving_model_dir, signatures=signatures)

does not work as expected in tensorflow serving. What is the right way to store the model such that it can served?

Secondly, does the solution for the above also solve it for the InfraValidator component?

Hello, can you please advise about this, as soon I'll have to use the path foundation model released by Google, and that model uses the latest version of XLA/Tensorflow, and this SavedModel thing would be a blocker for me. Thank you!

[nikelite]

Hi @pritamdodeja, sorry for the late response.

TFX and underlying libraries are not currently supporting Keras 3, though we support TF 1.16.

In TF 1.16, model.save() api is deprecated, and you should use tf.saved_model API.
You can find more information in https://www.tensorflow.org/guide/saved_model

TF saved model should be compatible with TF serving, as both follows TF saved model format. You may be able to check some codes without TFX pipeline or components, and see if it's model issue or TFX component issue.
saved_model_cli will be the useful tool to investigate this.

Thanks!

[pritamdodeja]

Hello @nikelite - are there any other blockers other than keras3 dropping support for SavedModel format? I am sticking with 1.15 till things stabilize/integrate with respect to keras3. Thank you!