Improve model-fit validation confidence

[i386]

This PR improves the local model-fit validation path so Mesh LLM can make more honest GGUF fit and throughput claims from metadata plus measured hardware facts. It keeps the fitter deterministic: no model-name shortcuts, no backend correction constants, and no validation-row-specific tuning.

What changed

• Adds source-shaped decode diagnostics for llama.cpp/GGML boundary misses.
• Adds Q5_K dense graph probe support so Q5 GGUFs can use direct probe evidence instead of weaker surrogate paths.
• Charges mixed residual tensor types as replacement deltas when a full-token graph already covers the operation slots.
• Keeps failed isolated native probes from aborting the validator when they still emitted usable probe JSON.
• Adds crates/model-fit/DESIGN.md documenting objectives, non-goals, evidence rules, memory/decode/prefill modeling, confidence, and validation strategy.

Why

The goal is a local fitter that can inspect arbitrary Hugging Face GGUF metadata, combine it with mesh-llm gpus benchmark output, and return explainable local fit recommendations. The current direction is to model llama.cpp execution boundaries rather than layer family multipliers or hidden magic constants.

Validation

• cargo fmt --all -- --check
• LLAMA_STAGE_BUILD_DIR=.deps/llama-build/build-stage-abi-metal cargo test -p model-fit dense_full_token_probe_charges_only_mixed_residual_replacement_delta --lib -- --nocapture
• LLAMA_STAGE_BUILD_DIR=.deps/llama-build/build-stage-abi-metal cargo test -p model-fit --bin model-fit-validate dense_probe_tensor_types_include_q5_k_when_metadata_contains_q5_weights -- --nocapture
• LLAMA_STAGE_BUILD_DIR=.deps/llama-build/build-stage-abi-metal cargo clippy -p model-fit --all-targets -- -D warnings
• LLAMA_STAGE_BUILD_DIR=.deps/llama-build/build-stage-abi-metal cargo clippy -p mesh-llm-gpu-bench --all-targets -- -D warnings
• just model-fit-release "" .deps/llama-build/build-stage-abi-metal

Note: the release build completed successfully. The build output still reported GGML decode probes disabled for one compile path, so native-probe linkage should be rechecked in the next validation pass before relying on the new Q5_K probe in fresh benchmark runs.

Current confidence

The strongest evidence is still dense quantized GGUFs with known tensor mappings on measured hardware. CUDA quant-ladder rows are near the ±10% target. Metal is improving but remains more guarded, especially for tiny models, MoE, and models where tensor types fall into unknown or surrogate buckets.

Next steps

• Fix broader GGUF tensor type mapping from model-artifact into model-fit.
• Rerun Metal and CUDA smoke validation with Q4/Q5/Q6/Q8 coverage.
• Keep high tok/s confidence restricted to rows with source-shaped evidence and low benchmark noise.

[github-actions]

This pull request is currently a draft. Reviews will not take place until the PR is marked as ready for review.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 38d374f8-575a-4f61-a7d3-6d8596865f9c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/skippy-tip-to-tip-placement

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[i386]

not sure if this will ever work or even if we want it anymore @michaelneale

[michaelneale]

this one needs a lot of work to bring up to date I think - and that is a LOT of new code - patch queue?