Code

Adapter small fix#356

Code

Open

Andrei-Aksionov wants to merge 4 commits intoLightning-AI:mainLightning-AI/lit-llama:mainfrom Andrei-Aksionov:adapter-small-fixAndrei-Aksionov/lit-llama:adapter-small-fixCopy head branch name to clipboard

Commits

Commits on Jun 2, 2023

• CausalSelfAttention.forward: n_embd --> C

  Andrei-Aksionov

  committed
  a1acde2

  Copy full SHA for a1acde2

• Adapter version of LLaMA class: vocab_size --> padded_vocab_size

  Andrei-Aksionov

  committed
  8e7501b

  Copy full SHA for 8e7501b

• Put aT=self.adapter_prompt_length above and use it instead of asking for shape of ak/av

  Andrei-Aksionov

  committed
  f1cc27b

  Copy full SHA for f1cc27b

• Adapter test: original model should also use padded_vocab_size

  Andrei-Aksionov

  committed
  cdd05dd

  Copy full SHA for cdd05dd