feat: migrate discovery call sites to native registry (Phase 4)

[Paramveersingh-S]

Overview

This is Phase 4 of the staged rollout for Issue #2396. This PR successfully decouples the API server's model discovery logic from the legacy CGO bindings.

Key Changes

• API Model Info (apiserver/route_model_info_embeddings.go): Completely removed the hard-coded dependency on candle_binding.GetEmbeddingModelsInfo(). The API endpoint now correctly iterates over native.Registry.List() and queries adapter.Info() to assemble its response. Any new backend added to the registry automatically appears in the /models endpoint response.

Fixes #2396 (fixes part of it)

[netlify]

✅ Deploy Preview for vllm-semantic-router ready!

Name	Link
🔨 Latest commit	c34dc81
🔍 Latest deploy log	https://app.netlify.com/projects/vllm-semantic-router/deploys/6a57bbf93934ba000889e4e0
😎 Deploy Preview	https://deploy-preview-2534--vllm-semantic-router.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[github-actions]

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned when their GitHub accounts are assignable in this repository:

📁 candle-binding

Owners: @FAUST-BENCHOU, @shraderdm, @drivebyer, @ramkrishs, @WUKUNTAI-0211, @AayushSaini101, @siloteemu
Files changed:

• candle-binding/semantic-router_mock.go
• candle-binding/src/ffi/classify.rs
• candle-binding/src/ffi/embedding.rs
• candle-binding/src/ffi/generative_classifier.rs
• candle-binding/src/ffi/generative_guard.rs
• candle-binding/src/ffi/init.rs
• candle-binding/src/ffi/similarity.rs
• candle-binding/src/ffi/state_manager.rs
• candle-binding/src/ffi/tokenization.rs
• candle-binding/src/lib.rs
• candle-binding/src/model_architectures/traditional/bert.rs
• candle-binding/src/model_architectures/traditional/deberta_v3.rs
• candle-binding/src/model_architectures/traditional/modernbert.rs
• candle-binding/src/registry.rs

📁 src/semantic-router

Owners: @FAUST-BENCHOU, @shraderdm, @drivebyer, @ramkrishs, @WUKUNTAI-0211, @AayushSaini101, @siloteemu
Files changed:

• src/semantic-router/cmd/main.go
• src/semantic-router/pkg/apiserver/route_model_info_embeddings.go
• src/semantic-router/pkg/apiserver/route_model_info_test.go
• src/semantic-router/pkg/modelruntime/native/adapter.go
• src/semantic-router/pkg/modelruntime/native/candle/adapter.go
• src/semantic-router/pkg/modelruntime/native/onnx/adapter.go
• src/semantic-router/pkg/modelruntime/native/onnx/adapter_test.go
• src/semantic-router/pkg/modelruntime/native/openvino/adapter.go
• src/semantic-router/pkg/modelruntime/native/openvino/adapter_test.go
• src/semantic-router/pkg/modelruntime/native/registry.go
• src/semantic-router/pkg/modelruntime/native/registry_test.go
• src/semantic-router/pkg/modelruntime/native/request.go
• src/semantic-router/pkg/modelruntime/native/taxonomy.go

---

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

[github-actions]

✅ Supply Chain Security Report — All Clear

Scanner	Status	Findings
AST Codebase Scan (Py, Go, JS/TS, Rust)	✅	29 finding(s) — MEDIUM: 22 · LOW: 7
AST PR Diff Scan	✅	No issues detected
Regex Fallback Scan	✅	No issues detected

---

Scanned at 2026-07-15T17:11:51.967Z · View full workflow logs

[Xunzhuo]

Requesting changes because the Phase 4 migration currently breaks the existing /models embedding inventory. No production package imports native/candle, native/onnx, or native/openvino, so their init() functions never execute and the global registry is empty. This PR also does not compile (logging is now unused), adds no API parity/regression test, inherits the earlier phase blockers, is based on main rather than Phase 3, and prematurely says Fixes #2396. Please make this a real stacked PR, use Part of #2396, explicitly assemble the available adapters, and demonstrate parity with the legacy Candle inventory before removing the direct binding path.

[Xunzhuo]

This iterates an empty registry in production: none of the adapter packages are imported, so their init() registration is unreachable. Replacing GetEmbeddingModelsInfo() here therefore makes the endpoint return no embedding models. Please assemble/register adapters explicitly at the runtime composition seam and add an API test that loads a Candle embedding model and verifies the migrated response is non-empty and parity-compatible.

[Xunzhuo]

This import is left behind after removing the only warning path, so go test ./pkg/apiserver fails with logging imported and not used. More importantly, the old binding error is now silently represented as an empty inventory because Adapter.Info() cannot return an error. Please extend the contract to preserve discovery errors/status and retain observable warning behavior rather than treating failure as “zero models”.

[Xunzhuo]

matryoshka_2d is being read as a backend-wide property, but one backend can host models with different dimensional capabilities. This metadata must come from the individual ModelInfo (or equivalent per-model metadata), otherwise mixed inventories will report incorrect support.

[Paramveersingh-S]

1. Rebased: The PR is now properly stacked on Phase 3 and I'll update the description to "Part of Modularize native router model bindings and runtime capability contracts #2396".
2. Explicit Runtime Assembly: The adapters are now explicitly registered via native.Registry.Register() at the runtime composition seam during application startup (cmd/main.go).
3. API Parity Test: Added TestEmbeddingModelInfo_NativeRegistryParity which explicitly registers the mock Candle adapter and asserts the /models API response is populated correctly and matches the legacy structure.
4. Error Propagation: Updated the BackendAdapter contract so Info() returns ([]ModelInfo, error). Discovery errors are now caught and preserved via logging.Warnf instead of silently succeeding with 0 models. This also resolves the unused import compilation error.
5. Model-Level Features: Added a Features map to the ModelInfo struct, and the /models endpoint now accurately scopes the matryoshka_2d capability to the individual model rather than the entire backend.