Running UFM on edge device NVIDIA jetson AGX orin

[rkakash59]

Great work. I am working on a use case where the device which I have is jetson AGX orin.

I would like to give UFM a try as it might solve the problem more easily.

Do you think that UFM can be run on the edge device easily?
Or do you think there might be some challenges for running it on edge device?

[infinity1096]

Hi, thank you for your interest in deploying UFM.

We’re currently deploying UFM on the Orin AGX as well. The base model has been successfully converted to ONNX and then to TensorRT without any issues. At a resolution of 420×560 with a batch size of 2 pairs of images, the model runs at 2.67 FPS.

We haven’t deployed the refinement model yet, but that’s planned for the near future.

Additionally, we’re actively working on a lightweight version of UFM tailored for robotics applications—stay tuned, and let me know should you need help converting models.

[rkakash59]

Thank you for reply.

Great to know that you have already deployed it on Orin AGX. I will try it in few days time.

Also looking forward for for the release of light weight version and Apache licensed model.

[rkakash59]

Hi, Will the lightweight version of UFM will also be released by Oct 23rd?

[infinity1096]

Hi, during our deployment (visual odometry), we finetuned and ran the UFM560-Base model at 336x224 resolution. Under this resolution the FPS is usable. We will release this model before Oct. 23.

We are still working on distilling/training a smaller model to run at the same FPS at larger resolution.

[HarborC]

Hi, during our deployment (visual odometry), we finetuned and ran the UFM560-Base model at 336x224 resolution. Under this resolution the FPS is usable. We will release this model before Oct. 23.

We are still working on distilling/training a smaller model to run at the same FPS at larger resolution.

@infinity1096 We're very much looking forward to your work—would it be possible to share your release plan and expected goals prior to Oct. 23?

[infinity1096]

Hi @rkakash59, the full model, finetuned to operate at 336x252 resolution, is released at https://huggingface.co/infinity1096/UFM-Refine-336.

It operates with the same interface as all other model variants.

[RohanAsokan-Perceptyne]

Hi! What is the FPS numbers you get on tensorrt converted UFM-Refine-336?

And what is your timeline for UFM-Tiny? Is it still in ideation stage or training stage?