clm-0044

Source: r/StrixHalo post by u/Look_0ver_There (GitHub stew675 — strongly implied same person, not confirmed), llama.cpp PR #26856 "bf16-tile-packed-q", head 9921e01, base dd1ea524 (2026-08-10) — AFTER both our builds (3653e6d, min-62bf73d), so none of this is in anything we run. Verified open/unmerged via GitHub API; only procedural comments so far. ## The lever llama.cpp silently converts BF16 KV to F16 before flash-attention on all backends. The PR adds a native BF16 tile path gated on `V_DOT2_F32_BF16_AVAILABLE` (RDNA3/3.5/4 only, gfx110x/115x/120x — includes our gfx1151), selected automatically when both cache types are bf16. No new flags. NVIDIA unaffected. Author's numbers (Qwen3.6-35B-A3B-Q8_0 unless noted): · F16 KV, both backends stock, same binary: Vulkan pp1024 717.93 vs ROCm 618.31 (+16%), tg256 46.86 vs 42.42 (+10.5%) — the CLEANEST of the three community comparisons. · BF16 KV, ROCm patched: ROCm pp1024 684.02 vs Vulkan 484.00 (~40% ahead); decode gap unchanged (Vulkan 47.59 vs 42.51). · PPL @32k (Qwen3.5-4B, wikitext-2): F32 8.6368, patched BF16 8.6403 (+0.04%), F16 9.1400 (+5.8%) — BF16 KV as a near-free QUALITY upgrade over F16 is the sleeper finding, directly relevant to our KV-quality thread (clm-0038/0041/0042). ## The decode-gap mechanism (unverified, no artifact) Author's per-op breakdown claims ROCm kernels match Vulkan's (20.93 vs 20.9 ms summed) but lose ~3.82 ms/token to inter-kernel dispatch gaps because HIP graphs never stabilise. UPDATE 2026-08-13: the author PUBLISHED the fix and RETRACTED that mechanism — "HIP_GRAPHS=ON was, in fact, working. It just wasn't really providing any real benefit." Corrected root cause: ROCm's HSA AQL dispatch floor (~2-3 us/kernel, ~974-1624 kernels/token on the 35B MoE) vs Vulkan's cheaper command-processor path — a library-level difference, mitigated by KERNEL FUSION, not graph repair. Public artifact: github.com/stew675/llama.cpp branch rdna-boosts (consolidated, includes the PR-26856 BF16-KV path, the fusion campaign, a gfx1151 mmvq table for Q8_0 decode, and a revert of upstream #24233 — which he identifies as the root of the Strix Halo async-race KV corruption, superseding the HIP_LAUNCH_BLOCKING PSA on his branch; mmap stays broken either way). Claimed results: ROCm-vs-Vulkan decode gap 12.2% -> 3.2% at d32768 (BF16 KV, Qwen3.6-35B) and ROCm prefill +36.5% over Vulkan; residual gap attributed to the dispatch floor itself. Now TESTABLE against clm-0050's stock matrix — queued. Source: r/StrixHalo thread (PR-26856 write-up, edits 11-12 Aug). ## What it does to clm-0031 Supports the direction (Vulkan genuinely ahead on decode-at-depth on this silicon), undercuts the magnitude (~10.5% clean vs 55% confounded). clm-0031's own hedge — "the backend, their patches, or the 43-build gap" — resolves as: mostly the latter two. ## Test plan (queued under the anchor-pair policy) One dual-backend binary (GGML_HIP=ON + GGML_VULKAN=ON) at base dd1ea524 with the PR branch cherry-picked; backend selection becomes the only variable. Anchor pair vs 3653e6d first to isolate the 3-day upstream drift. Cells: 35B (the post's own model, already screened here), pp512/1024 + tg128/256, shallow + d32768, f16 AND bf16 KV both backends, -fa 1, --parallel 1. Predictions on record: F16 Vulkan +16/+10.5%; BF16 ROCm +40% prefill, decode gap persists. ## ⚠ Side-findings from the same author's linked PSA (separate post, unverified) · `HIP_LAUNCH_BLOCKING=1` reportedly required on some recent ROCm versions to avoid SILENT KV-cache corruption on Strix Halo. Not in our protocol.json; our ROCm 7.1.0 may or may not be in the affected range — version labels ambiguous. · **MTP draft verification may pass corrupted tokens** — "not even a depth of 1 is truly safe". Production's 122B runs MTP (clm-0022). Nothing in our records addresses MTP output-correctness. Flagged for its own investigation before aibeast's production restore; a correctness question, not a performance one.