Having difficulties in replicating experiment

[fine20030510]

Hi！ Your work is excellent！
I copy your source code and data, and I tried to run your code.
But some error happened.

File "/data/Python-Super/super/deform_mesh.py", line 644, in forward
loss, losses, boundary_edge_type, boundary_face_type = self.get_losses(i, deform_verts, inputs, trg, src, src_graph, new_verts, src_edge_index, models, init_iter=i==0)
File "/data/Python-Super/super/deform_mesh.py", line 477, in get_losses
point_plane_loss = DataLoss.autograd_forward(self.opt, inputs, self.new_data, trg,
File "/data/Python-Super/super/loss.py", line 395, in autograd_forward
sample_trg, _, sample_valid = bilinear_sample([trg.points, trg.norms, trg.seg_conf],
File "/data/Python-Super/super/loss.py", line 86, in bilinear_sample
U_nm_valid = U_nm_valid.view(len(U_nm_valid), -1)
RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1] because the unspecified dimension size -1 can be any value and is ambiguous

Can you help you solve this problem?
Looking forward to you reply!

[gkw0010]

Hi, I meet the same error as you. Have you solved this problem?

[ShanL03]

Hi,

Thank for the comments! We just cleaned up the code base, checked those errors, and also cleaned up the data. Could you try the new code and data? All comments to run the code are provided in readme.

[gkw0010]

/home/ren2/anaconda3/envs/super/lib/python3.9/site-packages/skimage/metrics/_structural_similarity.py:251: RuntimeWarning: invalid value encountered in divide
S = (A1 * A2) / D
45%|████████████████████████████████████████████████████████████████▊ | 68/151 [03:50<04:41, 3.39s/it]
Traceback (most recent call last):
File "/home/ren2/data/wgk/super/run_semantic_super.py", line 25, in
main()
File "/home/ren2/data/wgk/super/run_semantic_super.py", line 22, in main
models.super(models, inputs)
File "/home/ren2/anaconda3/envs/super/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ren2/data/wgk/super/super/super.py", line 55, in forward
deform_param = self.fusion(models, inputs, sfdata)
File "/home/ren2/data/wgk/super/super/super.py", line 70, in fusion
deform_param = self.graph_fit(inputs, self.sf, sfdata, models)
File "/home/ren2/anaconda3/envs/super/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ren2/data/wgk/super/super/deform_mesh.py", line 247, in forward
return self.deform_superedg(inputs, src, trg, models)
File "/home/ren2/data/wgk/super/super/deform_mesh.py", line 313, in deform_superedg
loss, losses = self.get_losses(deform_verts,
File "/home/ren2/data/wgk/super/super/deform_mesh.py", line 83, in get_losses
point_plane_loss = DataLoss.autograd_forward(self.opt,
File "/home/ren2/data/wgk/super/super/loss.py", line 348, in autograd_forward
sample_trg, _, sample_valid = bilinear_sample([trg.points,
File "/home/ren2/data/wgk/super/super/loss.py", line 83, in bilinear_sample
U_nm_valid = U_nm_valid.view(len(U_nm_valid), -1)
RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1] because the unspecified dimension size -1 can be any value and is ambiguous

The problem still exists in Trial3(SuPerV2-T1). I think this is caused by the “deform_verts” in super/deform_mesh.py: 286. When "deform_verts" contains "nan", an error will be reported.

[ShanL03]

Seems like the tracking crushed. Have you checked the tensorboard, does the depth looks correct? It’s possible that the pre-trained model didn’t read properly.

…

On Oct 23, 2023, at 1:03 AM, Guankun Wang ***@***.***> wrote: /home/ren2/anaconda3/envs/super/lib/python3.9/site-packages/skimage/metrics/_structural_similarity.py:251: RuntimeWarning: invalid value encountered in divide S = (A1 * A2) / D 45%|████████████████████████████████████████████████████████████████▊ | 68/151 [03:50<04:41, 3.39s/it] Traceback (most recent call last): File "/home/ren2/data/wgk/super/run_semantic_super.py", line 25, in main() File "/home/ren2/data/wgk/super/run_semantic_super.py", line 22, in main models.super(models, inputs) File "/home/ren2/anaconda3/envs/super/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl return forward_call(*input, **kwargs) File "/home/ren2/data/wgk/super/super/super.py", line 55, in forward deform_param = self.fusion(models, inputs, sfdata) File "/home/ren2/data/wgk/super/super/super.py", line 70, in fusion deform_param = self.graph_fit(inputs, self.sf, sfdata, models) File "/home/ren2/anaconda3/envs/super/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl return forward_call(*input, **kwargs) File "/home/ren2/data/wgk/super/super/deform_mesh.py", line 247, in forward return self.deform_superedg(inputs, src, trg, models) File "/home/ren2/data/wgk/super/super/deform_mesh.py", line 313, in deform_superedg loss, losses = self.get_losses(deform_verts, File "/home/ren2/data/wgk/super/super/deform_mesh.py", line 83, in get_losses point_plane_loss = DataLoss.autograd_forward(self.opt, File "/home/ren2/data/wgk/super/super/loss.py", line 348, in autograd_forward sample_trg, _, sample_valid = bilinear_sample([trg.points, File "/home/ren2/data/wgk/super/super/loss.py", line 83, in bilinear_sample U_nm_valid = U_nm_valid.view(len(U_nm_valid), -1) RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1] because the unspecified dimension size -1 can be any value and is ambiguous The problem still exists in Trial3(SuPerV2-T1). I think this is caused by the “deform_verts” in super/deform_mesh.py: 286. When "deform_verts" contains "nan", an error will be reported. — Reply to this email directly, view it on GitHub <https://urldefense.com/v3/__https://github.com/ucsdarclab/Python-SuPer/issues/3*issuecomment-1774638729__;Iw!!Mih3wA!GJQjHcRrTlXh9E63rqJtDKXXK5Nw2Z6K-Im6TQC9sJh-HRrP3rjkm9K9gNswlGAtUWeEfqnMjd_3YC6pUXGLbXXdcKvU-0QBeMmq$>, or unsubscribe <https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AJVLE6KJKSWDLKUGORC7JYTYAYQGTAVCNFSM6AAAAAA4SH2WC6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZUGYZTQNZSHE__;!!Mih3wA!GJQjHcRrTlXh9E63rqJtDKXXK5Nw2Z6K-Im6TQC9sJh-HRrP3rjkm9K9gNswlGAtUWeEfqnMjd_3YC6pUXGLbXXdcKvU-xiGNvVn$>. You are receiving this because you commented.

[gkw0010]

Depth looks correct, but seg_render seems not to be rendering correctly.

[jinwoo777]

how did you visualized it?? I got events.out ~~~ files but have difficulties when use tensorboard.
Can you tell me the run code??

[imad151]

@jinwoo777, you can access this information from the TensorBoard library. These steps worked for me:

1. Run the following command as provided in the README:

   python run_super.py --model_name super_run --data_dir [path/to/data/] --tracking_gt_file rgb/left_pts.npy --load_depth --load_valid_mask --sf_point_plane --mesh_rot --mesh_arap

2. If you didn't change the model_name, you can simply navigate to the following directory:

   /Python-SuPer/results/super_run/
   

   Inside this folder, you'll find PNG files and some files named like events.out.tfevents.[some_numbers].[some_name].[some_numbers].0.

3. In your terminal (with TensorBoard installed), run the following command:

   tensorboard --logdir=/path/to/super/Python-SuPer/results/super_run/

4. If everything went smoothly, you should see an output like:

   TensorBoard 2.18.0 at http://localhost:6006/ (Press CTRL+C to quit)
   

   Open this URL in your browser, and you can find the data under the "Visualization" tab.

[jinwoo777]

@imad151 Thank you for your help! I successfully did it !