About Application

[ikai-zkdlin]

Thank you for your work and code. when I adapt this method to X-ray images and try modifying the loss function (e.g., using SSIM), the model training fails to converge and the gradients become quite large. Did you observe the same issue on natural images? What is your opinion on this? I look forward to your reply.

[ou524u]

hi, while I haven’t tried applying our method to X-ray images, here are some possible reasons you might be seeing this issue:

1.DINOv2 features – Our model relies on DINOv2 for feature extraction, but X-ray images could be out-of-domain. You might consider visualizing the features to check if they are effective. Trying the latest DINOv3 could also help.

2.Camera estimation – Our latent Plücker estimator learns camera estimation with transformers. This works reasonably well on dense features, but it may fail on X-rays, as how feature-based methods (e.g., COLMAP) fails. If you already have camera poses, falling back to LVSM-style networks could be a good option. Otherwise, methods like RayZer with more input views may also help.

3.Correct inductive bias – While our approach tries to minimize inductive bias, incorporating domain-specific priors can be important for X-rays. Following the progression from Gaussian → LVSM → UP-LVSM and starting from works like SAX-NeRF or X-Gaussian might be a strong starting point.

Thanks a lot for your interest, and wish you success with your experiments!