My sandbox for learning how AI works — hands-on experiments and projects as I go.
- induction-heads — reverse-engineering how GPT-2 small copies from its context (mechanistic interpretability): find the attention heads responsible, and prove it causally.
- gdm-ai-research-foundations — my completed labs from Google DeepMind's AI Research Foundations path: building a small language model from n-grams up, tokenization and embeddings, and attention implemented by hand.