Large Language Model Engineering (LLM Engineering) refers to the emerging best-practices and tools for pretraining, post-training, and optimizing LLMs prior to production deployment.
Pre- and post-training techniques include unsupervised pretraining, supervised fine-tuning, alignment, model merging, distillation, quantization. and others.
*LLM Engineering today is done with the GPT-style transformer architecture.
**Small Language Models (SLMs) of today can be as large as 70B parameters.
This course teaches you the fundamentals of LLMs, and will quickly onramp you up to the practical LLM Engineering edge. When you complete this course, you will understand how the latest Large and Small Language Models are built, and you'll be ready to build, ship, and share your very own.
🤖 The Transformer
🧐 Attention
🔠 Embeddings
🪙 Next-Token Prediction
🔡 Embedding Models
🚇 Pretraining
🚉 Fine-Tuning
🛤️ Alignment
🥪 Model Merging
⚗️ Distillation
and more from the LLM Edge!
We believe in the power of collaboration. Contributions, ideas, and feedback are highly encouraged! Let's build the ultimate resource for LLMEs together. 🤝
Feel free to reach out with any questions or suggestions. Happy coding! 🚀🔮