clm-0006

Quantized KV cache is mis-implemented for Strix Halo in stock llama.cpp: the code dequantizes to full precision repeatedly during inference, which a discrete GPU hides in cache and this box cannot. Community fixes report ROCm text generation +75% to +203% depending on depth, and q8_0 KV generating 23-53% FASTER than f16.

Source: independent 128GB verification (Nathanw1014) of a fork by another user, four controlled builds — patched and stock, Vulkan and ROCm — flash attention pinned, build hash and flags carried per row. Vulkan prefill +45%/+71%/+87% at 32k/64k/128k on Qwen3-Coder-30B. Full 262,144-token native context runs on both backends; at 262k the compressed cache generates 65% faster for a 2.6% prefill cost. One reported regression: MoE loses 0.3-2.4% ROCm prompt speed with the fix. DIRECTLY LOAD-BEARING FOR US: cfg-0002 runs q8_0/q8_0 KV on ROCm at 200K context, which is the deep end where the reported gain is largest — potentially a bigger win than MTP's +50%. NOT REPRODUCED HERE, and not upstream: no PR exists in ggml-org/llama.cpp, so adopting it means carrying a SECOND fork alongside con-0001. The author states plainly that none of it measures output quality.

Note — the record's own working

Cited by — computed at build time, never stored