EfficientTAM

[bhack]

As we have already SAM2 in the repo now It could be nice to have an example with Meta new EfficientTAM:
https://github.com/yformer/EfficientTAM

[vkuzo]

cc @HDCharles , @drisspg on adding new models

[drisspg]

cc @cpuhrsch who has been doing alot of work on the sam front

[cpuhrsch]

Hm, are there particular changes you'd like to see us host @bhack?

[bhack]

We could see how it will perform e2e e.g. we could re-use something we have done already in VIT in the ao repo, then evaluate if we could do something in the cross attention. I don't know if we have other margins.

[cpuhrsch]

Oh I see, hm, maybe that's something we can use in https://github.com/yformer/EfficientTAM directly? For ViT we've been able to use autoquant for performance gain in some cases. cc @jerryzh168

[bhack]

We have also a WIP PR on attention in AO repo, twe could check see how it performs on his EfficientRopeAttention2.

Also you already have some inference tricks in Sam2 that we could reuse like your recent video predictor or batching stuffs.

[cpuhrsch]

@bhack - Makes sense. I don't think those are easily portable right now, but should hopefully become more portable soon. I can't give much guidance at the moment (pressed for time), but, of course, please feel free to copy-paste anything you like :)

[bhack]

/cc @yformer

[bhack]

facebookresearch/sam2#640