my segmentation model evaluation results are significantly lower than those reported in the paper

[sralvins]

the data evaluation part states:

For the segmentation module, a predicted bounding box is considered a true positive if its intersection over union with the ground truth is higher than 95%.

I downloaded the PDFs (some were missing), then used the ground truth and Decimer-segmentation predictions to evaluate the model. The P/R@IoU=0.95 values are significantly lower than those reported in the paper.

[lucas-morin]

Hi @sralvins ,

Did you ensure that the correct DECIMER-Segmentation model was used?
Additionally, when evaluating, did you tighten the predicted and ground-truth boxes or polygons before comparing them? By "tighten," I mean restricting the boxes/polygons to remove any white borders between the annotated regions and the chemical structures.