Lancer

A from-scratch Llama 3.2 inference engine, written in pure MLX for Apple Silicon.

Named after the lancer — the mounted knight who charged the lists at the joust — because this engine drives the model forward, token by token.

What's inside

The full forward pass is implemented by hand, no transformers model class:

• tokenizer.py — encoding/decoding and input embedding lookup
• attention.py — Grouped-Query Attention with Llama 3 RoPE frequency scaling
• MultiLayerPerceptron.py — the SwiGLU MLP block
• main.py — the inference loop: RMSNorm → attention → MLP across all 16 layers, then greedy decoding

Requirements

• Apple Silicon Mac
• Python 3.14+
• mlx, transformers

Model weights

The Llama 3.2 weights are not included in this repo (~4.6 GB). Download them from Hugging Face into a llama3.2/ directory at the repo root:

llama3.2/snapshots/<snapshot-hash>/model.safetensors

Run

python main.py