clm-0028

aihydra, ROCm 7.1.0, stock llama.cpp 3653e6d, `--parallel 1`, `--load-mode none`, 3 reps, page cache dropped between cells. Depths 0 / 32,768 / 131,072. Ingested as run-0050..run-0097. ## Flash attention: decode at 131,072 (f16 KV) | model | fa=1 | fa=0 | gain | |---|---|---|---| | Qwen3.6-35B-A3B | **28.46** | 5.91 | **4.8x** | | Nemotron-3-Super-120B-A12B | **16.21** | 6.41 | **2.5x** | | Qwen3.5-122B-A10B (at 32k) | 18.17 | 9.87 | 1.84x | | gpt-oss-120b | 24.53 | **failed to run at all** | n/a | **The absence of flash attention is not a slowdown, it is a collapse.** Qwen3.6-35B-A3B goes 50.88 → 5.91 tok/s from empty context to 131k without it, an 88% loss. With it, 51.12 → 28.46, a 44% loss. Nemotron: 17.46 → 6.41 without (−63%) against 17.51 → 16.21 with (−7%). **And it is invisible at shallow depth.** At depth 0 every model is within 1% either way (35B: 51.12 vs 50.88; Nemotron: 17.51 vs 17.46). A benchmark that only measures empty context would conclude flash attention does not matter. It is the single most important setting on this hardware and only depth reveals it. ## Quantised KV requires flash attention — 4 of 4 models Every `q8_0 + fa=0` cell failed outright: Qwen3.5-122B, Qwen3.6-35B, Nemotron, gpt-oss. llama-bench exits with "failed to create context" before any inference. This is not a performance finding but a hard constraint, and it means the two levers are not independent — you cannot sweep them as a clean 2x2. gpt-oss is stricter still: **both** its fa=0 arms failed, f16 as well as q8_0. It cannot run without flash attention at all. ## q8_0 penalty on a STOCK build varies by model Decode at 131k, fa=1, q8_0 against f16: | model | f16 | q8_0 | penalty | |---|---|---|---| | gpt-oss-120b | 24.53 | 10.70 | **−56%** | | Qwen3.6-35B-A3B | 28.46 | 18.48 | −35% | | Qwen3.5-122B-A10B | 12.03 | 7.81 | −35% | | Nemotron-3-Super | 16.21 | 12.58 | −22% | All four pay a penalty, spanning 22% to 56%. This is the defect clm-0006 describes and clm-0022 measured a fix for (+70.3% recovery at production depth on the 122B). **The patch has not been tested on the other three**, but gpt-oss's −56% suggests it has the most to gain of anything measured. ## Guards Qwen3.6-35B-A3B and Nemotron-3-Super both pass 4/4 — coherence, tool call with real arguments, needle recovered at 8,000 tokens. gpt-oss fails retrieval with the stale-answer defect (clm-0025) and its rows are recorded but not capability-cleared. ## Practical **Leave `-fa on` everywhere on this hardware, unconditionally.** It wins or ties at every depth on every model tested, it is the difference between a 44% and an 88% decode loss at 131k, and three of four models cannot use quantised KV without it.