About Mapillary-Planet-Scale-Depth(MPSD) for the training dataset

[zck921031]

Hi:

After processing the Mapillary-Planet-Scale-Depth(MPSD) dataset following https://github.com/facebookresearch/map-anything/tree/main/data_processing,
I found that the result images were distorted, but the depth images were un-distorted.

they are not well aligned, e.g.,

Checking this config: https://github.com/facebookresearch/map-anything/blob/main/data_processing/wai_processing/configs/launch/mpsd.yaml,
"undistortion" not in mpsd's stages.

No distortion coefficient in scene_meta.json .

Does it mean that distortion correction should not be performed on MPSD images?

Or, are there any issues with wai_processing when using MPSD?

Best Regards!

[Nik-V9]

Hi, thanks for your interest in MPSD!

We do not do any additional undistortion on top of the images. Since the dataset is crowdsourced/in-the-wild, we have noticed that there might be instances where the image appears distorted due to the dashboard.

@mlopezantequera would know better if this is a major issue that occurs with high frequency.

[zck921031]

Thank you for your reply!

Another question, I notice that the input image and the depth map are not pixel-by-pixel corresponding.

Does this mean that the model predicts a distortion-free depth map from the original image, rather than directly predicting the depth of each pixel?

Hi, thanks for your interest in MPSD!

We do not do any additional undistortion on top of the images. Since the dataset is crowdsourced/in-the-wild, we have noticed that there might be instances where the image appears distorted due to the dashboard.

@mlopezantequera would know better if this is a major issue that occurs with high frequency.

[Nik-V9]

Based on my experience going through MPSD scenes, I believe such cases might be rare in the data and when training at large scale, I doubt this would bias the model.

From my experience with running MapAnything on distorted images, I've observed it to predict pixel corresponding depth values as opposed to undistorted ones.