AnythingFromScratchV2

Synthesic in building, testing and try new model

Why we have AnythingFromScratchV2

Continuing the AnythingFromScratch series, you can see the previous version here. But since the current algorithms are quite large and require more than one file to implement, I decided to move them to this repo. This section will mainly be a link to my other repos, where I go into more depth on the algorithms.

Structure Overview

In terms of structure, I will divide it into different contents, for example Transformers, LLM, VLM, etc. If necessary, I will divide it into smaller parts (if any). The current structure will be [Link_Implementation_Repo] Paper_Name

List Of Paper

Transformer

• [RoPE] RoFORMER: ENHANCED TRANSFORMER WITH ROTARY POSITION EMBEDDING

• [NaViT] Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution

• [MLA] Multi-Head Latent Attention