mtmd-cli: reset roofline profiler after warmup prefill - #42
Merged
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
mgehre-amd
approved these changes
Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR wires the existing per-op HIP roofline profiler into
llama-mtmd-cliso VLM prefill can produce a clean, prefill-only artifact. After the multimodal warmup encode pass completes,ggml_cuda_roofline_reset()discards everything recorded during warmup; the subsequent measured prefill (vision encoder + text prompt) is what lands in theGGML_ROOFLINE_OUTreport on exit. The change mirrors the post-warmup reset already used inllama-benchfor text models.Motivation
The roofline profiler added in #33 is already usable for text prefill through
llama-bench(-p N -n 0 -r 1withGGML_ROOFLINE_OUTset). VLM models are exercised throughllama-mtmd-cli(orllama-server), and mtmd performs its own warmup encode during initialization. Without a reset at the prefill boundary, the atexit report would blend warmup vision-encoder ops with the measured prefill, making the artifact unusable for the prefill roofline KPI in rocm-scripts (tools/generate_llamacpp_ttft_from_artifacts.pyexpects a prefill-only slice).What changed
tools/mtmd/mtmd-cli.cppincludesggml-cuda-roofline.hand callsggml_cuda_roofline_reset()immediately after the warmup message is evaluated and before the measured prefilleval_messageruns.tools/mtmd/CMakeLists.txtdefinesGGML_HIP_ROOFLINEfor thellama-mtmd-clitarget and adds theggml-cudainclude path when the HIP roofline option is enabled at build time, matching thellama-benchwiring. Both the include and the reset call are behind#ifdef GGML_HIP_ROOFLINE, so non-roofline builds are unchanged.Usage
Build with
-DGGML_HIP=ON -DGGML_HIP_ROOFLINE=ON, then run:GGML_ROOFLINE_OUT=/tmp/vlm-roofline.json ./build/bin/llama-mtmd-cli -m model.gguf --mmproj mmproj.gguf --image img.jpg -p "describe this" -n 0. The JSON written on exit should cover the measured prefill only (vision encode + text prompt), with warmup ops excluded.