feat(mlx-lm): add HuggingFace Hub model loader - #352
Open
fiorelorenzo wants to merge 1 commit into
Open
Conversation
Adds mlx-lm/src/loader.rs with a sync snapshot_download(model_id, &opts) that resolves config.json, tokenizer.json, tokenizer_config.json (optional), and either a single model.safetensors or the sharded set described by model.safetensors.index.json. Promotes WeightMap from models/qwen3.rs to the shared loader module; models/llama.rs and models/qwen3.rs import from there. Adds hf-hub = "0.4.3" (matching examples/mistral) and a hub-test feature gate for the live-network integration test (#[ignore] by default; CI keeps --test-threads=1 since MLX is not thread-safe). Refs oxiglade#274 (umbrella mlx_lm-equivalent), oxiglade#329 (OpenAI server).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a sync
snapshot_download(model_id, &opts) -> LoadedFilestomlx-lmso callers can fetch model files from the HuggingFace Hub instead of pointing at a hardcoded local path. Resolvesconfig.json,tokenizer.json,tokenizer_config.json(optional), and either a singlemodel.safetensorsor the sharded set described bymodel.safetensors.index.json.Promotes the existing
WeightMapstruct frommodels/qwen3.rsinto the newloadermodule;models/llama.rsandmodels/qwen3.rsimport from there. No behaviour change in the existing model loaders.Adds
hf-hub = "0.4.3"(matchingexamples/mistral/Cargo.toml) and ahub-testfeature gate so the live-network test stays#[ignore]by default in CI (which already runs--test-threads=1for MLX thread-safety).Tests:
WeightMapJSON parsing and value dedup.hub-testfeature,#[ignore]) that resolves themlx-community/Qwen3-0.6B-bf16config.Prerequisite for #274 (umbrella
mlx_lm-equivalent) and #329 (OpenAI-compatible API server). Does not yet update the example inexamples/lm/to use the new loader; that is a follow-up to keep this PR focused on the loader itself.