Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/NOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ Work: exact-chunks on main `1ce0d662b`; sm_120 measured at `3d2581551`.
| 35B mid-band | **canonical 0.918-0.972x** c1-c32 (@`348c265d`, first c16/c32) | Decode-only window, ONE tool |
| Qwen3.5-4B sm_120 | Exact chunks ON: 3.072x kernel / +2.272% run; tput 1.021x PASS; latency/VRAM OPEN | Spike 1.609x conv gap |
| RPi5 A76 CPU | **R5 asm GREEN; llama NOT MET**: 0.461x pf, 0.653x dec | W6: BF16 GEMM |
| MXFP4 parity | c1 1.020, c2-c8 0.962-0.969; #82 CLOSED, TERMINAL | — |
| SERVE-ASYNC-MIRROR | **#323 FIXED** (mitigation): graph declines while the mirror is live; 7/7 async gates | Graph read ids at REPLAY |
| CPU levers (`QUANT-GGUF-CIQ-GEMM`) | Profile DONE: decode **47% threadpool sync**, prefill **~39% paged attn** | Parakeet encoder; attn dtype hoist |
| `SERVE-METRICS` async (#277) | **`/metrics` was DEAD on the shipped server**: AsyncLLM folded nothing. Now live, ctest 366/366 | Config-gated families |
Expand All @@ -32,6 +31,7 @@ Work: exact-chunks on main `1ce0d662b`; sm_120 measured at `3d2581551`.
| `BACKEND-ROCM` | **(b) fix in; #140 gfx1201 hipBLAS + Gemma-4 MoE landed; W0 green** | compile + M2 ([spec](specs/rocm-unified-memory-b.md)) |
| TP spike #287 (PR #143) | **TP-W1 LANDED**: rank-group table + TP handle (6/6); DSR leak FIXED (unblocks #127/#154/#155) | TP-W2 (linears + loader) |
| Release | **ACTIVE; required W1-W11/W13 implemented in #196** | Finish hosted ten-SM proof; rebase/push; run full eight-tuple dry run |
| Containers `#170` | **cpu+vulkan CI green**; #312 fixed | W6: cuda/arm64; unpushed |
| `SAMPLE-PROMPT-LOGPROBS` (#223) | **LANDED** 21/21 | W2 `echo`; CUDA PENDING |
| `logprobs_mode` (#238) | **3 stubs -> all 4 work** | `logprob_token_ids` half |
| Surface coverage (`ARCH-ONE-SURFACE`) | ROW 8 + #139; **embeddings live (#137): ABI v15, endpoint, fold 4/4-231** | Real-checkpoint oracle cosine |
Expand Down
1 change: 1 addition & 0 deletions .agents/coordination.md
Original file line number Diff line number Diff line change
Expand Up @@ -1914,6 +1914,7 @@ this claim will meet. The tiled row is speed-gatable on dgx.
| `CLAIM-PARAKEET-MODEL-P4` | `MODEL-AUDIO-PARAKEET-ENCODER`, `MODEL-AUDIO-PARAKEET-TRANSDUCER` | Claude Opus 5 (1M context) | isolated worktree `/home/mudler/_git/vllm.cpp-parakeet` (CPU-only) | branch `row/MODEL-AUDIO-PARAKEET-ENCODER`, base `main` | The encoder, the CTC head with greedy collapse, the log-mel front end, an HF-safetensors loader and the RNN-T/TDT transducer. Owns `src/vllm/model_executor/models/parakeet_*`, `src/vllm/multimodal/parakeet_audio_processor.*`, `examples/parakeet_transcribe/`, the Parakeet tests and the two model-matrix rows. | `ACTIVE` | 2026-08-07 CPU-LANDED + gated against a HF `ParakeetForCTC` oracle with EXACT token ids; transcribes ctc-0.6b/1.1b, rnnt-0.6b, tdt-0.6b-v3. No CUDA, no aarch64, no speed number claimed. |
| `CLAIM-BACKEND-TENSTORRENT-SPIKE` | `BACKEND-TENSTORRENT` (`INVENTORIED`→`ACTIVE`; W0 skeleton, one op) | Claude Code (claude-sonnet-5) | isolated worktree `/home/lu_zero/Sources/vllmcpp-tenstorrent`; real Blackhole (P150) hardware available all session | branch `row/BACKEND-TENSTORRENT`, base `origin/main` `7534da65`, draft PR [#197](https://github.com/mudler/vllm.cpp/pull/197) | New extension platform (no upstream vLLM equivalent, same class as Metal/Vulkan): `DeviceType::kTENSTORRENT`, `vt::tenstorrent::Backend` (host-staged, discrete PCIe), `TenstorrentPlatform`, ONE op (`kMatmul`, F32/rank-2) via `ttnn::operations::matmul::matmul`. Owns `include/vt/device.h`, `src/vllm/platforms/{platform,tenstorrent}.cpp`, `src/vt/tenstorrent/`, `tests/vt/test_tenstorrent_backend.cpp`, the `VLLM_CPP_TENSTORRENT` CMake wiring, the backend-matrix/porting-inventory/spec, and this claim. | `ACTIVE` | 2026-08-09 W0 LANDED: 3/3 test cases, 8/8 assertions PASS on real hardware (registration, Platform/Backend mirror, kMatmul vs host F32 reference within bf16 tolerance, max_abs_diff 0.03375). Two bugs found+fixed in the same pass (nlohmann-json ABI collision between vllm.cpp's vendored copy and tt-metal's, isolated via a separate OBJECT library; a process-exit segfault from static MeshDevice teardown ordering, fixed by a deliberate leak) — both documented in the spec's Risks/decisions. Not reviewed. No model runs; one op only; host round-trip per call, no perf claim. |
| `CLAIM-ENG-LOAD-DIRECT-UPLOAD` | `ENG-LOAD-DIRECT-UPLOAD` (engine-matrix, NEW row -> `ACTIVE`) | Claude Code (opus-5) | isolated worktree `.claude/worktrees/agent-a122a5ddc3546bf31`; GB10 `dgx.casa` tree `~/work/load150/src` (Vulkan `build-vk` + CUDA `build-cuda`), every GPU stage under `flock $HOME/gpu.lock`, `local-ai-worker` parked and restored | branch `row/ENG-LOAD-DIRECT-UPLOAD`, base `origin/main` `375a471e` | Issue #150. Owns ONLY: the refcounted safetensors mapping + byte counters (`include/vllm/model_executor/model_loader/safetensors_reader.h`, `src/vllm/model_executor/model_loader/safetensors_reader.cpp`), `BorrowStTensorBytes` + the `AdoptDeviceBytesAsHost` mmap branch + `OwnedTensor::mmap_src` (`include/vllm/model_executor/models/qwen3_5_weights.h`, `src/vllm/model_executor/models/qwen3_5_weights.cpp`), the qualifying call sites in `include/vllm/model_executor/models/dense_weight_loaders.h` and `src/vllm/model_executor/models/qwen3_5_dense_weights.cpp`, the upload counter in `include/vllm/model_executor/models/dense_attn_block.h`, the `VT_LOAD_STATS` phase timing in `src/vllm/entrypoints/model_loader.cpp`, NEW `tests/vllm/test_load_direct_upload.cpp` + its `tests/CMakeLists.txt` line, NEW `.agents/specs/load-direct-upload.md`, the NEW engine-matrix row + its Loading/Total rollup, the `#150` roadmap intake cell, two `docs/ENVIRONMENT.md` rows, this claim, `.agents/NOW.md`, and the `docs/STATUS.md`/`docs/BENCHMARKS.md` one-liners. **NON-COLLISION:** no new kernel, no model registry change, no CMake target change beyond one test line; every non-verbatim loader helper is untouched and the lever is same-binary A/B-able (`VT_LOAD_DIRECT_UPLOAD=0`). MUST NOT touch README, Metal/SACRED/apex/darwin, or any model/kernel source | `ACTIVE` | 2026-08-09 — mechanism test 6/6 (77) and RED under two mutations; GB10 Vulkan gates on the changed tree green (`test_vulkan_backend` 35/35·2650, `test_backend_cross_device` 11/11·132, `test_opt_paged_engine` 6/6 prompts token-exact 96/96, 0 declines, device type 3); GB10 CUDA full `ctest` 383/393 with BOTH SACRED gates PASS and all 10 failures reproduced on a clean `origin/main` build; 27B bf16 load 1.54x warm / 1.61x cold, bytes moved 100.196 -> 81.260 GiB; `local-ai-worker` restored (`--restart=always`, Up). Fresh scoped review OWED |
| `CLAIM-ENG-RELEASE-CONTAINERS-W1-W7` | `ENG-RELEASE-CONTAINERS` (`INVENTORIED`->`READY`->`ACTIVE`; spike spec + W1-W5/W7) | Claude Code (opus-5), helper role | isolated worktree `/home/mudler/_git/vllm.cpp-containers`; local Docker only -- NO GPU, NO registry push, NO publication | `row/ENG-RELEASE-CONTAINERS`, base `origin/main` `24306364`, PR [#307](https://github.com/mudler/vllm.cpp/pull/307) | Container spike spec, `docker/Dockerfile` (cpu/vulkan/cuda calling the release scripts), container matrix + checker, image validator, workflow + least-privilege guard, and the SIGTERM fix the boot gate surfaced (#312). Excludes W6 matching-hardware runtime evidence and any GHCR publication | `ACTIVE` | 2026-08-10 -- cpu lane built and gated e2e (config+layout+boot, clean SIGTERM 0.25 s, RED 137 before the fix); mutation suites 31/31 and 22/22; cuda/vulkan gated statically but never built here; nothing published |
**CLOSED same-session claim — `CLAIM-TP-SPIKE-287` (task #287, 2026-08-08,
records-only, helper `row/SPIKE-TENSOR-PARALLELISM` draft PR #143, base
`b38f78a7`).** Not an active-claims row because every referenced row keeps its
Expand Down
Loading
Loading