Inference Speed

[FrostyFridge]

From the paper:

NLF-S has a batched throughput of 410 fps and unbatched throughput of 79 fps on an Nvidia RTX 3090 GPU. For NLF-L these are 109 fps and 41 fps respectively.

What inference setup was used to achieve these metrics? On a 3090, using the demo code, my unbatched inference throughput is significantly lower.

[isarandi]

Hi, thanks for the interest. This measures the speed of the core model that takes image crops. The demo uses the more convenient wrapper model that takes care of homography reprojection, rebatching the crops, back-transforming, possibly running the person detector etc. I'll prepare this example code later.

[FrostyFridge]

I see, thanks! I look forward to it.