[Feature] Qwen3 Reranker

[sigridjineth]

What does this PR do?

This PR adds support for Qwen3 reranker models to text-embeddings-inference. (Issue: #643)

These models function as binary classifiers that determine the relevance between a query and a document. They output a simple probability score, making them perfect for re-ranking search results.

Key Changes

• Introduced a new ListwiseReranker model type to properly distinguish these models from standard cross-encoder models.
• Enabled reranker functionality for both the standard and Flash Qwen3 models. This involved:
  • Implementing a predict method to extract the logits for "yes" and "no" tokens.
  • Calculating the final score based on the probability of the "yes" token.
• The router now automatically identifies reranker models by checking if "reranker" is in the model name or if the is_reranker flag is set in the model's config.

Technical Details

• The models use the hidden state of the final token to predict relevance.
• Scores are calculated from the logits for the "yes" (ID: 9693) and "no" (ID: 2152) tokens.
• The final output is a probability score between 0 and 1, representing how relevant a document is to the query.

Who can review?

Anyone in the community is welcome to review the PR once the tests have passed. Feel free to tag anyone who might be interested.

@OlivierDehaene or @Narsil

[sigridjineth]

The test has been passed on my local macbook pro

[vrdn-23]

@sigridjineth Thanks for the great work! Excited to see if this will get merged
Quick question: I think you may have accidentally deleted all the files for the Python backend? Is that intentional?

[vrdn-23]

I wonder if it would be a simpler code change to support the model as a SequenceClassificationModel as mentioned in this discussion?
https://huggingface.co/Qwen/Qwen3-Reranker-0.6B/discussions/3

[sigridjineth]

@vrdn-23 thanks for your comment! that would be nice one considering the changes. (discussion #3)

[sigridjineth]

I will try to work on #698

[qianjia-cn]

Mark

[fradav]

I tried this PR with a "converted-to-classifier" Qwen3-reranker-0.6B and it litteraly explodes at warming up, trying to allocate more than 80GB (tested on the Metal version on my mac). Something obvious I may have missed ?