Pytorch-Sample

This repo contains some sub projects that help understand advanced techniques used in PyTorch system. This repo contains following subprojects:

• Build first python package
• A simple pytorch demo project, from byrzhm, shows how to use pybind11 to use c++ as backend.
• A workshop to show features introduced in PyTorch2.0
• A small tutorial for pytest
• A naive Kernal Library in Triton, ref to FlagGems
• Triton Kernel integration example
• A tutoiral about how to implement flash attention from source
• A simple pybind project, support MLIR pass
• Stanford cs336 source codes
• Flash Attention topic

References

1. Build first python package
2. FlagGems
3. FlagTree
4. The simplest python project demo from byrzhm
5. A tour of PyTorch2.0
6. torch-compile with user define triton kernel
7. FlashAttention from source
8. Stanford cs336 source code
9. Flash Attention coarse