Skip to content

Port DeepSeek-MoE-16B-Base (deepseek) - #220

Open
DarthCeltic wants to merge 2 commits into
aifoundry-org:mainfrom
DarthCeltic:port-deepseek-moe
Open

Port DeepSeek-MoE-16B-Base (deepseek)#220
DarthCeltic wants to merge 2 commits into
aifoundry-org:mainfrom
DarthCeltic:port-deepseek-moe

Conversation

@DarthCeltic

Copy link
Copy Markdown
Contributor

Summary

  • Adds deepseek-ai/deepseek-moe-16b-base to the most_models_ported track (identity deepseek), introducing the DeepSeek MoE execution family to the board — the architecture that predates and informed the later DeepSeek-V2/V3 MoE designs.
  • Two layered real converter bugs found and fixed via a standalone monkeypatch wrapper (ported_models/deepseek_moe_16b_base/convert_wrapper.py, vendored submodule never touched, matching the established pythia410m precedent):
    1. Missing tokenizer chkhsh: this checkpoint's tokenizer produces a chkhsh (93105512fde79bc726022fe3cbfb7efef9738465b988d0600beba1296e3a91d8) entirely absent from get_vocab_base_pre()'s known-hash table, so conversion fails outright.
    2. The first fix attempt was itself incomplete: registering a made-up pre-tokenizer name ('deepseek-moe') let Python-side conversion succeed, but that name is NOT recognized by the separate, hardcoded C++ pre-tokenizer registry in llama-vocab.cpp (which only knows 'deepseek-llm'/'deepseek-coder'/'deepseek-v3'/'deepseek-r1-qwen') — a GGUF built that way loads fine in Python tooling but is silently wrong at real inference time. Caught this by actually loading the produced GGUF with llama-perplexity, not by trusting a clean Python conversion alone. Fixed by using the existing name 'deepseek-llm' instead, since DeepSeek's early dense and MoE models share the same base BPE tokenizer family.
  • Self-converted to Q8_0 GGUF (363 tensors, 17.4 GB, sha256=63eb27478a35ec36a3fd7c50e704ef6752ca235a59491d7673909da6fd8f0971), hosted on Hugging Face (darthceltic85/deepseek-moe-16b-base-gguf) since it far exceeds GitHub's 2 GB release-asset limit.
  • Verified live: model loads cleanly on the CPU backend (arch = deepseek, tokenizer.ggml.pre = deepseek-llm — confirming the C++ runtime recognizes the corrected name, 64 experts / 6 used) and real llama-perplexity against WikiText-2 raw gives PPL = 7.2909 +/- 1.30187 — a good, in-range result.

Test plan

  • benchmark_config in the claim JSON exactly matches .github/ci/benchmark_config.json's new deepseek_moe_16b_base entry
  • Source revision is a valid 40-char hex commit SHA
  • RECIPE.md exists and documents both real bugs found (including the trap in the first fix attempt)
  • All new/modified JSON files are valid JSON, no leftover placeholder values
  • Model loads and produces a real, in-range perplexity result (not fabricated)

Adds deepseek-ai/deepseek-moe-16b-base to the most_models_ported track.

Two layered converter bugs found and fixed via a standalone monkeypatch
wrapper (never touching the vendored submodule): (1) this checkpoints
tokenizer chkhsh was entirely missing from get_vocab_base_pre()s known-
hash table; (2) the first fix attempt registered a made-up pre-tokenizer
name (deepseek-moe) that passed Python-side conversion but was NOT
recognized by the separate hardcoded C++ pre-tokenizer registry in
llama-vocab.cpp -- caught by actually loading the produced GGUF, not
just trusting a clean conversion. Fixed by using the existing name
deepseek-llm instead, since DeepSeek's early dense/MoE models share the
same base BPE tokenizer family.

PPL = 7.2909 +/- 1.30187 on WikiText-2 raw, confirmed via a real
llama-perplexity run on CPU backend (tokenizer.ggml.pre = deepseek-llm
confirms the C++ runtime recognizes the corrected name). GGUF (17.4 GB)
hosted on Hugging Face (darthceltic85/deepseek-moe-16b-base-gguf) since
it far exceeds GitHub's 2GB release-asset limit.
@DarthCeltic
DarthCeltic requested a review from AFOliveira as a code owner July 26, 2026 07:13
@github-actions github-actions Bot added track: model-ports Global approved model ports prize track: community Community support and ecosystem contributions labels Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

track: community Community support and ecosystem contributions track: model-ports Global approved model ports prize

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant