clm-0021
measured-herehigh ●●●
citable URL: https://halobench.com/records/clm-0021/ — this address never moves; the anchor /records/#clm-0021 keeps resolving
The dense-model flash-attention prefill cliff reported in clm-0017 does NOT exist on ROCm. A stock ROCm build of a dense 27B reaches 214.66 t/s prefill at 32k and 153.63 at 65k — close to their PATCHED Vulkan numbers (237.4 / 180.4) and 5.2x their broken stock Vulkan at 65k (29.8). Flash attention is not the problem on HIP; it is what prevents one, worth 3.4x decode at 65k.
verified 2026-08-08 · volatility medium
Note — the record's own working
THIS IS THE REPLICATION clm-0017's AUTHOR ASKED FOR AND COULD NOT RUN. They dropped ROCm because Fedora 44 version locks blocked their release upgrade, leaving them Vulkan-only and unable to answer whether the collapse exists on the HIP path. It does not.
Qwen3.6-27B Q4_K_M (dense), aihydra, ROCm 7.1.0, **stock** llama.cpp 3653e6d, f16 KV, `--load-mode none`, 3 reps:
| depth | fa | pp512 | tg128 | |---|---|---|---| | 0 | 1 | 355.52 | 12.04 | | 0 | 0 | 353.93 | 11.95 | | 32768 | 1 | **214.66** | 10.86 | | 32768 | 0 | 175.87 | **4.76** | | 65536 | 1 | **153.63** | 9.90 | | 65536 | 0 | 119.08 | **2.92** |
SIDE BY SIDE WITH THE COMMUNITY REPORT (their dense 27B, f16 KV, FA on, pp512):
| | @32k | @65k | |---|---|---| | Vulkan STOCK (theirs, broken) | 94.5 | 29.8 | | Vulkan PATCHED (theirs) | 237.4 | 180.4 | | **ROCm STOCK (ours)** | **214.66** | **153.63** |
Our unpatched ROCm lands near their patched Vulkan and **5.2x their unpatched Vulkan at 65k**. Whatever the contiguize patch repairs, the HIP path does not suffer from it.
FLASH ATTENTION IS LOAD-BEARING FOR DECODE, NOT PREFILL. At empty context FA is irrelevant (355.52 vs 353.93 prefill, 12.04 vs 11.95 decode — both within noise). The divergence is entirely depth-driven, and decode is hit far harder than prefill:
· prefill at 65k: 1.29x with FA
· **decode at 65k: 3.4x with FA** (9.90 vs 2.92)
Without FA, decode falls 76% from empty context to 65k (11.95 → 2.92). With it, 18%. A benchmark that only reported prefill would have understated this badly.
SCATTER: every cell reproduced within **0.6% CV**. Per clm-0017 the broken path scatters 5-10% while a healthy one holds ~0.3% — our numbers carry the healthy signature, which is independent corroboration that we are not on the defective code path.
UNCONTROLLED: clm-0017 says "dense 27B" without naming the model; we used Qwen3.6-27B. If theirs was a different 27B, architecture is not perfectly matched — but the effect size (5.2x) is far larger than any plausible model-to-model difference.
CONSEQUENCE, COMBINED WITH clm-0020: leave `-fa on` everywhere on this hardware. It wins at every depth measured, on both a dense 27B and a 122B hybrid MoE, and its absence is catastrophic for decode at depth. The clm-0017 caution applies to Vulkan only.
WORTH REPORTING BACK. The author explicitly requested this test. Answering it costs us nothing and it resolves whether their patch is a general fix or a Vulkan-backend repair — which changes whether it belongs upstream as a backend fix or a cross-cutting one.