clm-0051
measured-heremed ●●○
citable URL: https://halobench.com/records/clm-0051/ — this address never moves; the anchor /records/#clm-0051 keeps resolving
Quantised KV on gfx1151 splits three ways by build. Stock Vulkan q8_0 prefill COLLAPSES (-26% to -32% vs its own f16) even as its decode gains. Stock ROCm q8_0 decode CRATERS with depth (-16.8/-25.0/-35.1% at 32k/64k/131k) — the same unpatched-build failure clm-0022 measured and patched on the 122B, reproduced here on a build without the ce7689f kvfix. The community fork rescues the Vulkan collapse completely (q8_0 prefill within 3% of f16 at every depth) while keeping and extending the decode gain (+7.8/+14.1/+23.2% over its own f16). Fork + q8_0 is the best long-context configuration measured on this chip: 42.5 tok/s decode at 131,072 — +23% over the best f16 arm and 2.3x stock ROCm q8_0 — at roughly half the KV memory.
verified 2026-08-13 · volatility medium
Note — the record's own working
Same arms, model and methodology as clm-0050 (A = stock 3653e6d ROCm, B = stock 3653e6d Vulkan, C = fork v0.6.1 3be50ccc2 + bundled RADV 26.3.0-devel; fa on, pp1024/tg256, llama-bench defaults -b 2048 / -ub 512, median of 3 fresh-process reps, drop-caches; one benign outlier rep in the matrix). Deep cells only — q8_0 effects are depth effects.
q8_0 KV, pp/tg by depth:
| depth | A ROCm | B Vulkan | C fork | |---|---|---|---| | 32,768 | 595.7 / 35.2 | 517.8 / 53.8 | 701.1 / 55.0 | | 65,536 | 407.8 / 27.3 | 334.5 / 48.1 | 495.4 / 50.1 | | 131,072 | 251.8 / 18.5 | 197.4 / 40.2 | 330.9 / 42.5 |
q8_0-vs-f16 deltas, per arm (pp / tg at 32k, 64k, 131k): - A stock ROCm: pp ~0% at every depth; tg -16.8% / -25.0% / -35.1%. This is
clm-0022's stock-build decode crater — repeated KV dequantisation during
inference — reproduced on a second model, second chip-generation build. This
binary lacks the ce7689f kvfix; the fork lineage carries it.
- B stock Vulkan: pp -25.8% / -32.4% / -29.1% (the collapse); tg +6.7% / +10.6% /
+17.5%. On stock Vulkan, q8_0 trades a third of your prefill for a decode gain.
- C fork: pp -0.6% / -2.3% / +2.9% — collapse fully rescued, q8_0 prefill ≈ f16;
tg +7.8% / +14.1% / +23.2%. Quantised KV becomes free-or-better, echoing
clm-0022's patched-build result.
OPERATIONAL READING: fork + q8_0 at 131k gives 42.5 tok/s decode and 330.9 pp — both the best of any arm/KV combination at that depth — while halving KV memory.
CAVEATS: one model, and it is an hd256 MoE — fork docs say hd128 gains more, so fork deltas are a floor. Arm C bundles a newer Mesa with the fork kernels (not single-variable; the d0 prefill edge in clm-0050 is plausibly part-Mesa). No quality measurement anywhere in this matrix — this is throughput only, and clm-0006's source made the same disclaimer.
Cited by — computed at build time, never stored
docs coverage