Name		Name	Last commit message	Last commit date
parent directory ..
homework		homework
README.md		README.md
lecture5.odp		lecture5.odp
lecture5.pdf		lecture5.pdf
practice.ipynb		practice.ipynb

README.md

Week 5: Data-parallel training

Lecture: slides, video
Seminar: link, video
Homework: see the homework folder

Further reading

Python multiprocessing docs (pay attention to fork vs spawn!)
PyTorch Distributed tutorial
Collective communication protocols in NCCL
There's a ton of links on the slides, please check the PDF.