clm-0029
measured-herehigh ●●●
citable URL: https://halobench.com/records/clm-0029/ — this address never moves; the anchor /records/#clm-0029 keeps resolving
DIAGNOSED: `llama-perplexity` produces garbage on this build — PPL 532 on deliberately repetitive English that should score 2-5, 3,654 on wikitext for a model that generates coherently at 51 tok/s and passes tool-calling guards. Not flash attention, not the corpus, not context size. **The KL-divergence route to measuring KV-quant QUALITY is therefore blocked**, and clm-0022's open half needs a different instrument.
verified 2026-08-08 · volatility high
Note — the record's own working
Qwen3.5-122B-A10B UD-Q4_K_M, aihydra, ROCm 7.1.0, stock llama.cpp 3653e6d, `-fa 1`, `-c 32768`, corpus `wiki.test.raw` sha256 `430983e5…` (bench/CORPUS.md), q8_0 arm against an f16 `--kl-divergence-base`.
**Core finding, by elimination across four controls (model, corpus, context, flash attention):** the defect is in `llama-perplexity` itself on this build (3653e6d) with these models — not the corpus (a 4-sentence repeating probe that should score 2-5 scored 532), not flash attention (disabling it moved 17,180 to 3,654, still absurd), not context size (4096 gave 56.2, no better than 32768), and not the model (the same 35B generates coherently and passes its tool-calling guard 4/4). Plausibly MoE or hybrid-architecture handling in the KL path, or the quant format — not established.
**Consequence:** `kv-quality.sh`, built on `llama-perplexity --kl-divergence`, is unusable on this build, so the KV-quant QUALITY question could not be answered this way. It was answered instead via τ²-bench task completion (`clm-0035`, `clm-0038`). `clm-0022` covers what quantised KV costs in SPEED, which this instrument failure never touched.
Full diagnostic method (the four controls, the two reasons the original KL numbers were implausible, and the alternatives considered) is written up as a standing methodology rule in `docs/methodology-lessons.md` §2 — read that before repeating this measurement.
Cited by — computed at build time, never stored
model pages qwen35-122b