perf(llama32_1b): log-depth tree release for the uberkernel barrier - #228
Open
karabambus wants to merge 1 commit into
Open
perf(llama32_1b): log-depth tree release for the uberkernel barrier#228karabambus wants to merge 1 commit into
karabambus wants to merge 1 commit into
Conversation
…experimental, board-untested)
karabambus
marked this pull request as ready for review
July 26, 2026 11:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Log-depth release for the uberkernel's chip-wide barrier.
Under batched dispatch the whole decode graph runs as one launch and
uber_barrier_global()executes before every instruction — ~299 times per token.Its release is serial: one hart issues two
fcc_sendstores per active shire (64stores at 32 shires) while ~2048 harts block in
fcc_consume.Change. Each active shire gets a rank = its index among the set bits of the
launch mask. The last arriver credits rank 0 only; each shire's designated hart
(local index 0) forwards to ranks
K*r+1 .. K*r+Kon waking, before doing anyother work.
KisUBER_TREE_FANOUT, default 4.Critical path at 32 shires: 8 stores best case, 26 worst case, versus 64 flat.
Properties.
in an idle shire's FCC counter and be consumed by the next kernel scheduled
there. The master shire is excluded structurally by the same rule.
exactly once, so every hart consumes exactly one credit per barrier.
finds zero.
K-agnostic; retuningUBER_TREE_FANOUTneeds no otherchange.
Verification (local).
mask, and full reachability, across K ∈ {2,3,4,5,8,16,64} —
tools/uber_tree_barrier_check.py.with forwarding disabled hangs where this passes.
csrw fcc..bssempty, barrier counter in.data, stack chain 3216 B against a 4160 B budget.mul_mat_Q8_0.candplatform.hbyte-identical to the validated revision.Board result. Decode 17.8109 tok/s at PPL 15.3098 (CPU 15.31, ET-vs-CPU
1.3e-05), samples
[17.8673, 17.7664, 17.8109], CV 0.23%. Prefill 79.5405 tok/s.Against the same build with the flat release — identical in every other respect —
this is +1.59% (17.5318 -> 17.8109).
Builds on the batched-dispatch mechanism from the upstream ET backend
(llama.cpp #24179). Developed with AI assistance.