Skip to content

convert: add eagle2 draft arch #13908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
+28 −0
Open

convert: add eagle2 draft arch #13908

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
87daf40
convert: add eagle2 draft arch
May 29, 2025
4b4975c
fix: resolve code formatting issues
May 30, 2025
ec22e4b
refactor: make Eagle2DraftModel inherits from Qwen2Model instead of T…
Jun 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions convert_hf_to_gguf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2712,6 +2712,11 @@ def modify_tensors(self, data_torch: Tensor, name: str, bid: int | None) -> Iter
yield from super().modify_tensors(data_torch, name, bid)


@ModelBase.register("Eagle2DraftForCausalLM")
class Eagle2DraftModel(Qwen2Model):
model_arch = gguf.MODEL_ARCH.EAGLE2_DRAFT


@ModelBase.register(
"Qwen2VLModel",
"Qwen2VLForConditionalGeneration",
Expand Down
19 changes: 19 additions & 0 deletions gguf-py/gguf/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,7 @@ class MODEL_ARCH(IntEnum):
QWEN2VL = auto()
QWEN3 = auto()
QWEN3MOE = auto()
EAGLE2_DRAFT = auto()
PHI2 = auto()
PHI3 = auto()
PHIMOE = auto()
Expand Down Expand Up @@ -360,6 +361,7 @@ class MODEL_TENSOR(IntEnum):
TOKEN_EMBD_NORM = auto()
TOKEN_TYPES = auto()
POS_EMBD = auto()
FC = auto()
OUTPUT = auto()
OUTPUT_NORM = auto()
ROPE_FREQS = auto()
Expand Down Expand Up @@ -580,6 +582,7 @@ class MODEL_TENSOR(IntEnum):
MODEL_ARCH.QWEN2VL: "qwen2vl",
MODEL_ARCH.QWEN3: "qwen3",
MODEL_ARCH.QWEN3MOE: "qwen3moe",
MODEL_ARCH.EAGLE2_DRAFT: "eagle2-draft",
MODEL_ARCH.PHI2: "phi2",
MODEL_ARCH.PHI3: "phi3",
MODEL_ARCH.PHIMOE: "phimoe",
Expand Down Expand Up @@ -640,6 +643,7 @@ class MODEL_TENSOR(IntEnum):
MODEL_TENSOR.TOKEN_EMBD_NORM: "token_embd_norm",
MODEL_TENSOR.TOKEN_TYPES: "token_types",
MODEL_TENSOR.POS_EMBD: "position_embd",
MODEL_TENSOR.FC: "fc",
MODEL_TENSOR.OUTPUT_NORM: "output_norm",
MODEL_TENSOR.OUTPUT: "output",
MODEL_TENSOR.ROPE_FREQS: "rope_freqs",
Expand Down Expand Up @@ -1207,6 +1211,21 @@ class MODEL_TENSOR(IntEnum):
MODEL_TENSOR.FFN_DOWN,
MODEL_TENSOR.FFN_UP,
],
MODEL_ARCH.EAGLE2_DRAFT: [
MODEL_TENSOR.TOKEN_EMBD,
MODEL_TENSOR.FC,
MODEL_TENSOR.OUTPUT,
MODEL_TENSOR.ATTN_NORM,
MODEL_TENSOR.ATTN_Q,
MODEL_TENSOR.ATTN_K,
MODEL_TENSOR.ATTN_V,
MODEL_TENSOR.ATTN_OUT,
MODEL_TENSOR.FFN_NORM,
MODEL_TENSOR.FFN_GATE,
MODEL_TENSOR.FFN_DOWN,
MODEL_TENSOR.FFN_UP,

],
MODEL_ARCH.QWEN2MOE: [
MODEL_TENSOR.TOKEN_EMBD,
MODEL_TENSOR.OUTPUT_NORM,
Expand Down
4 changes: 4 additions & 0 deletions gguf-py/gguf/tensor_mapping.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,10 @@ class TensorNameMap:
"wpe", # gpt2
),

# eagle2 draft model
MODEL_TENSOR.FC: (
"model.fc",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about this name (also it appears to be just fc in your GGUF), what is the purpose of this tensor?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about this name (also it appears to be just fc in your GGUF), what is the purpose of this tensor?

The EAGLE-2 paper doesn't express this mechanism particularly clearly, but you can refer to Figure 6 in the EAGLE-1 paper at https://arxiv.org/html/2401.15077v3.
The FC layer concatenates the current hidden_states with the hidden_states passed from previous inference steps, forming a tensor of 2 * config.hidden_size dimensions. This FC layer then maps the 2 * config.hidden_size dimensional tensor back to a config.hidden_size dimensional tensor, as demonstrated in the code at https://github.com/SafeAILab/EAGLE/blob/main/eagle/model/cnets1.py#L524.

),
# Output
MODEL_TENSOR.OUTPUT: (
"embed_out", # gptneox
Expand Down