You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Things would be easier after pytorch/torchtitan#814 is merged. This PR introduces ModelSpec to describe a model and how to parallelize a model. Here is an example of a ModelSpec for llama3
Proposal
apply_tp
&apply_cp
fns for each models as their layer definitions can be varied.Also see comments in fla-org/flame#4
The text was updated successfully, but these errors were encountered: