Skip to content

local mlx wrapper pattern

Jump to bottom
Nicolas Cravino edited this page Jul 10, 2026 · 2 revisions

id: local-mlx-wrapper-pattern title: Local MLX Wrapper Pattern tags: [mlx, apple-silicon, local-inference, cli, python] category: local-inference-mlx created: 2026-05-29 updated: 2026-07-09 freshness: fresh

Local MLX Wrapper Pattern

Summary

A recurring project template used across five sw30labs repos to wrap upstream HuggingFace/GitHub MLX models with a zero-build installer, CLI, and browser GUI — without vendoring model weights or modifying the upstream runtime. Each wrapper is a thin shell (install + launch) around a pinned runtime checkout, deployable with two commands on any Apple Silicon Mac.

Repos using this pattern

Canonical directory layout

<project-root>/
├── README.md
├── install.sh          # bootstraps uv venv, clones runtime at pinned ref, optional model download
├── gui.sh              # resolves venv python, exec gui/server.py
├── pyproject.toml      # project metadata; zero runtime deps (only dev: ruff)
├── uv.lock             # optional, present in newer wrappers
├── <package>/          # thin Python CLI / inference entrypoint
├── config/             # YAML or JSON model/runtime config (some wrappers)
├── gui/
│   ├── server.py       # stdlib ThreadingHTTPServer; shells out to scripts/, streams SSE/JSON
│   └── static/         # HTML + vanilla JS single-page UI
├── models/             # gitignored; populated by install.sh --download-model
├── output/             # inference artefacts written here (images, audio, video)
├── runtime/            # gitignored; upstream runtime cloned at pinned tag/commit
└── scripts/            # thin Python wrappers that call runtime APIs directly

Key contracts

install.sh

  • Accepts -y (non-interactive), --download-model, --update-runtime
  • Clones upstream runtime repo under runtime/ at a pinned $RUNTIME_REF (env-overridable)
  • Sets up a uv virtualenv inside the runtime dir; syncs deps from runtime's own lockfile
  • Model download is opt-in — never automatic — to avoid pulling multi-GB weights unexpectedly
  • Idempotent: safe to re-run; skips steps already complete

gui.sh

  • Resolves python from runtime/<runtime>/.venv/bin/python, falls back to python3
  • execs gui/server.py — single process, no daemon, Ctrl-C exits cleanly

gui/server.py

  • stdlib only: http.server.ThreadingHTTPServer + SimpleHTTPRequestHandler
  • Spawns inference as a subprocess (scripts/<model>_mlx.py), streams logs via polling
  • Job state held in-process dict (no DB); ports are project-specific (see table below)
  • Static frontend: single HTML file + vanilla JS; no build step

pyproject.toml

  • requires-python = ">=3.11"
  • Zero runtime dependencies at the wrapper level — all deps live in the runtime's own venv
  • Dev dependency: ruff for linting

Port assignments (known)

Repo Default port
sulphur-2-base 8786
supertonic-3-mlx 8795
ace-step-1-5-mlx 8789
ltx-2-3-mlx 8788
longcat-video-avatar-1-5-mlx 8796
lance-3b-video-bf16 (see gui.sh)
bonsai-image-ternary-4b-mlx-2bit (see gui.sh)

Variations

Variation Repos Notes
PrismML runtime bonsai Requires full Xcode Metal toolchain (xcrun metal), not just CLT
Gradio instead of stdlib server stable-audio-3 Uses run_gradio.py; same install.sh shell, different GUI layer
JSON graph + NPZ weights supertonic-3-mlx Non-standard model format; graph executor vendored from HF Space
Dual pipeline lance-3b-video-bf16 TextToVideoPipeline + UnderstandingPipeline exposed via same GUI
Prompt enhancer sulphur-2-base Optional second model (--download-prompt-enhancer) for prompt upscaling

When ingesting a new wrapper repo

If a new repo follows this pattern, the stub's Architecture section can reference this page instead of re-describing the shell structure:

## Architecture
Follows the [[local-mlx-wrapper-pattern]]. Runtime: `<upstream-repo>` pinned at `<ref>`.
Model: `<hf-org>/<model-id>`. GUI port: `<N>`.
Notable variation: <any deviation from canon>.

Related

  • local-inference-mlx — parent category page
  • tars-ai — older local MLX project; predates this pattern, uses LangGraph + mlx-lm directly
  • audiobook-generator — another MLX pipeline; LangGraph-orchestrated, not this shell pattern

Home | Index | Sitemap | Sitemap-Stacks | SCHEMA | Log

Clone this wiki locally