aifoundry-org · karabambus · Jul 25, 2026
@@ -315,6 +315,9 @@
     },
     "smolvlm_500m": {
       "config": "ported_models/llama_cpp_et/benchmarks/smolvlm_500m.json"
+    },
+    "granite_2b": {
+      "config": "ported_models/granite/benchmarks/granite.json"
     }
   }
 }
@@ -0,0 +1,49 @@
+# Granite-3.0-2B-Instruct Model Card
+
+- Reference family: **Granite 3.0** (IBM), decoder-only transformer. New `llama.cpp`
+  execution family `LLM_ARCH_GRANITE` — not a seed/registered family.
+- Hugging Face base (weights): `ibm-granite/granite-3.0-2b-instruct`, `apache-2.0`.
+- Pinned GGUF artifact: `lmstudio-community/granite-3.0-2b-instruct-GGUF` at
+  `0f35cb534c61d4f1ea9a8e266efc522db70dc2fa`, file
+  `granite-3.0-2b-instruct-Q8_0.gguf`
+  (sha256 `41f268169c7f0ab6758d0a51f497d9e55af0226bc71723e0a99a291b08e2ebda`,
+  2,801,068,896 bytes).
+- Benchmark model id: `granite_2b`. Runner: `llama_server` (shared `llama.cpp-et`).
+- Key docs: `docs/RECIPE.md`, `docs/HF_REFERENCES.md`,
+  `docs/proposed_identity_entry.json` + `docs/proposed_reference_contract.json`.
+
+Weights are not committed. The board CI downloads the pinned Q8_0 GGUF by url + sha256
+(`artifacts.json`).
+
+## Architecture (from `config.json`)
+
+`GraniteForCausalLM`: 40 layers, hidden 2048, FFN 8192, 32 attention heads, 8 KV heads
+(GQA, head_dim 64), vocab 49155, RoPE theta 10000, RMSNorm eps 1e-5, context 4096.
+
+Granite 3.0 is llama-shaped (RMSNorm + rotary attention + gated **SwiGLU** FFN + GQA) with
+four scalar multipliers applied as elementwise `SCALE`: embedding (×12.0), residual
+(×0.22), attention (×0.015625), and logits (÷8.0). Handled by `src/models/granite.cpp`. No
+LayerNorm, no bias, no ALiBi, no partial rotary.
+
+## Op coverage on ET (confidence: MEDIUM-HIGH)
+
+| Op | ET kernel | Status |
+|----|-----------|--------|
+| `RMS_NORM` | `et-kernels/src/rms_norm_f32.c` | seed-proven |
+| `ROPE` (NEOX, n_dims=64) | `et-kernels/src/rope_f32.c` | seed-proven |
+| `MUL_MAT` (Q8_0 × F32) | `et-kernels/src/mul_mat_*.c` | seed-proven |
+| `GLU` (**SwiGLU**) | `et-kernels/src/glu_f32.c` | seed-proven |
+| `SCALE` (4 scalar multipliers) | ggml-et | kernel present, **not seed-exercised** |
+| `SOFT_MAX`, `GET_ROWS`, `ADD`, `MUL`, `CONT` | ggml-et | seed-proven |
+
+Granite is stronger than the Gemma port: its FFN is **SwiGLU** (seed-proven), so the only
+non-seed-exercised op is the trivial elementwise `SCALE`. GQA (8 KV heads) and head_dim 64
+stay inside the seed-proven MatMul/RoPE paths. Fallback risk is low.
+
+## Model-ports track compliance
+
+- New standalone root `ported_models/granite/`; only regular files added beneath it.
+- Claim: `ported_models/submissions/model_ports/granite.json`.
+- New execution family `granite` — not in `baseline_port_roots`, not a variant of any
+  registered family.
+- One benchmark entry `granite_2b` added to `.github/ci/benchmark_config.json`.
@@ -0,0 +1,31 @@
+# Granite-3.0-2B-Instruct — ET-SoC1 model port
+
+Port of **Granite-3.0-2B-Instruct** (IBM, Apache-2.0) to ET-SoC1 through the shared
+`llama.cpp-et` runtime, for the "Most Models Ported by One Individual" track.
+
+Granite 3.0 is a new `llama.cpp` execution family (`LLM_ARCH_GRANITE`), already implemented
+in the committed framework submodule (`src/models/granite.cpp`). This port adds no framework
+code — it pins the Q8_0 GGUF, wires the `llama_server` benchmark, and files the track claim.
+
+Confidence is **MEDIUM-HIGH**: the FFN is SwiGLU (seed-proven), and the only op not already
+exercised by a seed model is the trivial elementwise `SCALE` (Granite's four scalar
+multipliers), which has a real ET kernel. See `MODEL.md` for the per-op mapping.
+
+## Layout
+
+```
+ported_models/granite/
+├── MODEL.md                              model card, architecture, op-coverage analysis
+├── README.md                            this file
+├── artifacts.json                       GGUF + shared framework artifact wiring
+├── benchmarks/granite.json              llama_server board benchmark config
+└── docs/
+    ├── RECIPE.md                        end-to-end reproduce recipe
+    ├── HF_REFERENCES.md                 pinned provenance
+    ├── proposed_identity_entry.json     stage-1 registry entry (maintainer adds)
+    └── proposed_reference_contract.json stage-1 correctness contract (maintainer adds)
+```
+
+Plus, outside this root:
+- `ported_models/submissions/model_ports/granite.json` — the track claim.
+- one `granite_2b` entry in `.github/ci/benchmark_config.json`.
@@ -0,0 +1,80 @@
+{
+  "artifacts": {
+    "llama_cpp_source": {
+      "kind": "framework_source",
+      "framework": "llama.cpp-et",
+      "submodule_path": "ported_models/llama_cpp_et/src/llama.cpp-et",
+      "license": "MIT",
+      "license_file": "LICENSE",
+      "upstream": {
+        "type": "git",
+        "url": "https://github.com/aifoundry-org/llama.cpp.git",
+        "branch": "et",
+        "revision": "cc4049d86b14e4ef72f827f3bb767b577f18fbcd",
+        "note": "Informational only. CI builds from the committed submodule_path; the granite port reuses the same shared framework source as the other llama.cpp-et models and does not repin it."
+      },
+      "note": "Framework source comes from the committed shared submodule. The granite port adds no framework code; Granite (LLM_ARCH_GRANITE, src/models/granite.cpp) is already implemented in this pinned revision."
+    },
+    "llama_cpp_build": {
+      "kind": "framework_workdir",
+      "framework": "llama.cpp-et",
+      "env": "LLAMA_CPP_ET_WORKDIR",
+      "local_cache": "local-artifacts/frameworks/llama.cpp-et/build-et",
+      "build": {
+        "cmake": "cmake",
+        "configure_args": [
+          "-DGGML_ET=ON",
+          "-DCMAKE_BUILD_TYPE=Release",
+          "-DCMAKE_CXX_STANDARD_LIBRARIES=-lglog"
+        ],
+        "build_args": ["--config", "Release"],
+        "targets": ["llama-server", "llama-perplexity", "llama-bench"]
+      },
+      "note": "Out-of-tree CMake build of the committed shared llama_cpp_source submodule."
+    },
+    "llama_server": {
+      "kind": "framework_binary",
+      "framework": "llama.cpp-et",
+      "env": "LLAMA_CPP_ET_SERVER",
+      "relative_to": "llama_cpp_build",
+      "relative_path": "bin/llama-server"
+    },
+    "llama_perplexity": {
+      "kind": "framework_binary",
+      "framework": "llama.cpp-et",
+      "env": "LLAMA_CPP_ET_PERPLEXITY",
+      "relative_to": "llama_cpp_build",
+      "relative_path": "bin/llama-perplexity"
+    },
+    "wikitext2_raw_test": {
+      "kind": "validation_dataset",
+      "dataset": "wikitext-2-raw-v1",
+      "split": "test",
+      "filename": "wiki.test.raw",
+      "env": "WIKITEXT_RAW_PATH",
+      "local_cache": "local-artifacts/datasets/wikitext-2-raw/wiki.test.raw",
+      "source_url": "https://cosmo.zip/pub/datasets/wikitext-2-raw/wiki.test.raw",
+      "sha256": "173c87a53759e0201f33e0ccf978e510c2042d7f2cb78229d9a50d79b9e7dd08",
+      "note": "WikiText-2 raw test corpus. Shared perplexity corpus, identical to the other llama.cpp-et models."
+    },
+    "granite_2b_q8_gguf": {
+      "kind": "model",
+      "framework": "llama.cpp-et",
+      "variant": "granite-3.0-2b-instruct-Q8_0",
+      "filename": "granite-3.0-2b-instruct-Q8_0.gguf",
+      "env": "GRANITE_2B_MODEL_PATH",
+      "source": {
+        "type": "huggingface",
+        "repo": "lmstudio-community/granite-3.0-2b-instruct-GGUF",
+        "revision": "0f35cb534c61d4f1ea9a8e266efc522db70dc2fa",
+        "filename": "granite-3.0-2b-instruct-Q8_0.gguf",
+        "url": "https://huggingface.co/lmstudio-community/granite-3.0-2b-instruct-GGUF/resolve/0f35cb534c61d4f1ea9a8e266efc522db70dc2fa/granite-3.0-2b-instruct-Q8_0.gguf"
+      },
+      "sha256": "41f268169c7f0ab6758d0a51f497d9e55af0226bc71723e0a99a291b08e2ebda",
+      "size_bytes": 2801068896,
+      "local_cache": "local-artifacts/models/granite_2b/granite-3.0-2b-instruct-Q8_0.gguf",
+      "board_path": "/data/models/granite-3.0-2b-instruct-Q8_0.gguf",
+      "note": "Weights are not committed. The board CI downloads the GGUF from the pinned Hugging Face revision by url + sha256."
+    }
+  }
+}
@@ -0,0 +1,53 @@
+{
+  "runner": "llama_server",
+  "board": true,
+  "framework": {
+    "name": "llama.cpp-et",
+    "runner": "llama_server",
+    "source_artifact": "llama_cpp_source"
+  },
+  "artifacts_file": "../artifacts.json",
+  "reference_contract": ".github/ci/reference/granite_2b.json",
+  "canonical_variant": "granite-3.0-2b-instruct-Q8_0",
+  "score": {
+    "metric": "tokens_per_second",
+    "label": "Decode tokens/s",
+    "higher_is_better": true
+  },
+  "llama_server": {
+    "source_artifact": "llama_cpp_source",
+    "model_artifact": "granite_2b_q8_gguf",
+    "server_artifact": "llama_server",
+    "workdir_artifact": "llama_cpp_build",
+    "host": "127.0.0.1",
+    "port": 18081,
+    "device": "ET",
+    "gpu_layers": 99,
+    "ctx_size": 2048,
+    "batch_size": 256,
+    "ubatch_size": 128,
+    "parallel": 1,
+    "cache_ram_mib": 0,
+    "ready_timeout_s": 180,
+    "request_timeout_s": 300,
+    "flash_attn": false,
+    "api": "completion",
+    "prompt": "Repeat this token sequence without commentary: OK OK OK OK OK OK OK OK OK OK",
+    "max_tokens": 96,
+    "temperature": 0,
+    "ignore_eos": true,
+    "min_completion_tokens": 32,
+    "perplexity": {
+      "enabled": true,
+      "perplexity_artifact": "llama_perplexity",
+      "corpus_artifact": "wikitext2_raw_test",
+      "ctx_size": 128,
+      "batch_size": 128,
+      "ubatch_size": 128,
+      "timeout_s": 300,
+      "min_ppl": 1.0,
+      "max_ppl": 1000.0,
+      "chunks": 4
+    }
+  }
+}
@@ -0,0 +1,26 @@
+# Granite-3.0-2B-Instruct — Hugging Face provenance
+
+Pinned so the model identity is reproducible (per `docs/HF_REFERENCES.md` submission rule).
+
+## Base model (weights)
+
+| Field | Value |
+|-------|-------|
+| Repo | `ibm-granite/granite-3.0-2b-instruct` |
+| License | `apache-2.0` |
+| Architecture | `GraniteForCausalLM` (40L, hidden 2048, FFN 8192, 32 heads / 8 KV GQA, vocab 49155, RoPE θ=10000, SwiGLU + scalar multipliers) |
+
+## Benchmark artifact (Q8_0 GGUF)
+
+| Field | Value |
+|-------|-------|
+| Repo | `lmstudio-community/granite-3.0-2b-instruct-GGUF` |
+| Revision | `0f35cb534c61d4f1ea9a8e266efc522db70dc2fa` |
+| Filename | `granite-3.0-2b-instruct-Q8_0.gguf` |
+| sha256 | `41f268169c7f0ab6758d0a51f497d9e55af0226bc71723e0a99a291b08e2ebda` |
+| Size | `2801068896` bytes |
+| URL | `https://huggingface.co/lmstudio-community/granite-3.0-2b-instruct-GGUF/resolve/0f35cb534c61d4f1ea9a8e266efc522db70dc2fa/granite-3.0-2b-instruct-Q8_0.gguf` |
+
+Direct Q8_0 conversion of Granite-3.0-2B-Instruct — no custom quantization, packing, or
+shape change. The sha256 above is the Hugging Face LFS object id of the pinned revision;
+verify with `sha256sum` after download.
@@ -0,0 +1,69 @@
+# `granite` (`granite_2b`) — model-ports reproduce recipe
+
+Ports **Granite-3.0-2B-Instruct** (IBM, Apache-2.0) to ET-SoC1 via the shared
+`llama.cpp-et` runtime. Granite is a new execution family (`LLM_ARCH_GRANITE`) already
+implemented in the committed framework — no framework change is made here. Two-stage
+`docs/SUBMISSION_GUIDE.md` flow.
+
+## 0. Provenance
+
+- Base weights: `ibm-granite/granite-3.0-2b-instruct`, Apache-2.0.
+- Benchmark GGUF: `lmstudio-community/granite-3.0-2b-instruct-GGUF` @
+  `0f35cb534c61d4f1ea9a8e266efc522db70dc2fa`, `granite-3.0-2b-instruct-Q8_0.gguf`,
+  sha256 `41f268169c7f0ab6758d0a51f497d9e55af0226bc71723e0a99a291b08e2ebda`,
+  2,801,068,896 bytes. See `docs/HF_REFERENCES.md` and `artifacts.json`.
+- No custom quantization/packing.
+
+## 1. Stage 1 — identity + contract approval (maintainer)
+
+- `docs/proposed_identity_entry.json` → add to `data/model-port-identities.json`.
+- `docs/proposed_reference_contract.json` → add as `.github/ci/reference/granite_2b.json`.
+
+Precomputed hashes in the identity entry:
+
+- `benchmark_config_sha256` = `canonical_sha256(effective granite model config)`:
+  ```
+  python3 - <<'PY'
+  import hashlib, json
+  cfg = json.load(open("ported_models/granite/benchmarks/granite.json"))
+  cfg["config"] = "ported_models/granite/benchmarks/granite.json"
+  print(hashlib.sha256(json.dumps(cfg, sort_keys=True, separators=(",",":")).encode()).hexdigest())
+  PY
+  ```
+- `validation_contract_sha256` = `sha256sum .github/ci/reference/granite_2b.json` after the
+  maintainer commits the contract (byte-identical to `docs/proposed_reference_contract.json`).
+
+## 2. Stage 2 — implementation PR (this port)
+
+- New standalone root `ported_models/granite/`.
+- Claim `ported_models/submissions/model_ports/granite.json`.
+- One `granite_2b` entry in `.github/ci/benchmark_config.json`:
+  ```json
+  "granite_2b": { "config": "ported_models/granite/benchmarks/granite.json" }
+  ```
+
+## 3. Build (CI, reproducible by hand)
+
+```
+cmake -S ported_models/llama_cpp_et/src/llama.cpp-et -B build-et \
+  -DGGML_ET=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_STANDARD_LIBRARIES=-lglog
+cmake --build build-et --config Release --target llama-server llama-perplexity
+
+build-et/bin/llama-server --model granite-3.0-2b-instruct-Q8_0.gguf \
+  --device ET --gpu-layers 99 --ctx-size 2048 --batch-size 256 --ubatch-size 128 \
+  --host 127.0.0.1 --port 18081
+```
+
+## 4. Board correctness + metric
+
+Main-owned `.github/ci/reference/granite_2b.json` contract: decode validation (≥32 tokens,
+temp 0, device ET, full offload, host-CPU agreement 1.0), zero op fallbacks, WikiText-2
+perplexity within bound (ET-vs-CPU relative diff ≤ 0.01), metric decode `tokens_per_second`.
+
+## 5. Op-coverage rationale (confidence MEDIUM-HIGH)
+
+Granite ops — `RMS_NORM`, `ROPE`-NEOX (n_dims=64), `MUL_MAT` (Q8_0×F32), `GLU` **SwiGLU**,
+`SCALE` (embedding ×12, residual ×0.22, attention ×0.015625, logits ÷8), `SOFT_MAX`,
+`GET_ROWS`, `ADD`, `MUL`, `CONT` — all have real ET kernels. The FFN is SwiGLU
+(seed-proven); the only non-seed-exercised op is the trivial elementwise `SCALE`. GQA (8 KV)
+and head_dim 64 reuse seed-proven paths. See `MODEL.md` for the per-op mapping.
@@ -0,0 +1,17 @@
+{
+  "_comment": "Stage-1 registry entry for the granite identity. A maintainer adds this object to the identities list in data/model-port-identities.json to make the identity eligible. The two sha256 values are precomputed against this port; recompute per docs/RECIPE.md if any file changes. Drop the _comment key before committing.",
+  "identity_id": "granite",
+  "execution_family": "granite",
+  "benchmark_models": ["granite_2b"],
+  "aliases": ["granite-3.0-2b-instruct"],
+  "eligible": true,
+  "canonical_source": {
+    "repo": "lmstudio-community/granite-3.0-2b-instruct-GGUF",
+    "revision": "0f35cb534c61d4f1ea9a8e266efc522db70dc2fa",
+    "license": "apache-2.0"
+  },
+  "approved_runner": "llama_server",
+  "benchmark_config_sha256": "bb18b01f4c55ac9c55ee812b0d2482f5e3ec5ec0b3881de3bad3ee81f6339a37",
+  "validation_contract": ".github/ci/reference/granite_2b.json",
+  "validation_contract_sha256": "a68dcc52a37d287008a374ce648b8616979c3d1dcad0ed4c8b792b7f3f8c5216"
+}