Skip to content

mtmd-cli: reset roofline profiler after warmup prefill - #42

Merged
roberteg16 merged 1 commit into
gfx11from
rogarcia.llamacpp-roofline-for-vlm
Jul 9, 2026
Merged

mtmd-cli: reset roofline profiler after warmup prefill#42
roberteg16 merged 1 commit into
gfx11from
rogarcia.llamacpp-roofline-for-vlm

Conversation

@roberteg16

Copy link
Copy Markdown

Summary

This PR wires the existing per-op HIP roofline profiler into llama-mtmd-cli so VLM prefill can produce a clean, prefill-only artifact. After the multimodal warmup encode pass completes, ggml_cuda_roofline_reset() discards everything recorded during warmup; the subsequent measured prefill (vision encoder + text prompt) is what lands in the GGML_ROOFLINE_OUT report on exit. The change mirrors the post-warmup reset already used in llama-bench for text models.

Motivation

The roofline profiler added in #33 is already usable for text prefill through llama-bench (-p N -n 0 -r 1 with GGML_ROOFLINE_OUT set). VLM models are exercised through llama-mtmd-cli (or llama-server), and mtmd performs its own warmup encode during initialization. Without a reset at the prefill boundary, the atexit report would blend warmup vision-encoder ops with the measured prefill, making the artifact unusable for the prefill roofline KPI in rocm-scripts (tools/generate_llamacpp_ttft_from_artifacts.py expects a prefill-only slice).

What changed

tools/mtmd/mtmd-cli.cpp includes ggml-cuda-roofline.h and calls ggml_cuda_roofline_reset() immediately after the warmup message is evaluated and before the measured prefill eval_message runs. tools/mtmd/CMakeLists.txt defines GGML_HIP_ROOFLINE for the llama-mtmd-cli target and adds the ggml-cuda include path when the HIP roofline option is enabled at build time, matching the llama-bench wiring. Both the include and the reset call are behind #ifdef GGML_HIP_ROOFLINE, so non-roofline builds are unchanged.

Usage

Build with -DGGML_HIP=ON -DGGML_HIP_ROOFLINE=ON, then run: GGML_ROOFLINE_OUT=/tmp/vlm-roofline.json ./build/bin/llama-mtmd-cli -m model.gguf --mmproj mmproj.gguf --image img.jpg -p "describe this" -n 0. The JSON written on exit should cover the measured prefill only (vision encode + text prompt), with warmup ops excluded.

Co-authored-by: Cursor <cursoragent@cursor.com>
@roberteg16
roberteg16 requested a review from mgehre-amd July 8, 2026 16:06
@roberteg16
roberteg16 merged commit 482ed81 into gfx11 Jul 9, 2026
5 checks passed
@jimw567
jimw567 deleted the rogarcia.llamacpp-roofline-for-vlm branch August 16, 2026 23:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants