How to project text?

[xnought]

from the paper I see that you first embed text with text-embedding-3-large, then you use your trained projection network from the contrastive learning.

Can you also release the pretrained text project models?

I want to embed text in the joint embedding space and find similar proteins that way.

Any help would be appreciated! Thank you very much. This is a super cool project!

[xnought]

I made an interface too

Screen.Recording.2024-09-24.at.2.07.32.PM.mov

If you can help me with the text part I mentioned above, I can add natural language queries to the website.

[xnought]

update: https://ocular.cc.gatech.edu/DS569k/ deployed it in case anyone wanted to use it

[xnought]

Also made a Nomic 2d map of 250k proteins using protein clip + topic modeled based on function if you were interested https://atlas.nomic.ai/data/donnybertucci/swissprot-proteinclip/map

[young-su-ko]

Also made a Nomic 2d map of 250k proteins using protein clip + topic modeled based on function if you were interested https://atlas.nomic.ai/data/donnybertucci/swissprot-proteinclip/map

Which version of proteinclip did you use for this?

[xnought]

The smallest one. ESM2 6 layer one.

If you want the data, you can also download it here https://huggingface.co/datasets/donnyb/DS569k. I precomputed all the embeddings and other metadata just so I can reuse it later

[young-su-ko]

Thanks! Really fun to just look around the different regions of the 2d map

[nleroy917]

Hey I'm also quite intersted in projecting text. I think its a potentially missed opportunity to not be able to project text and search for proteins! Any chance we could get the pre-trained weights for the text-encoder projection MLPs?