Home › Evidence › Records › clm-0011

clm-0011

communitymed ●●○
citable URL: https://halobench.com/records/clm-0011/ — this address never moves; the anchor /records/#clm-0011 keeps resolving

Greedy decode is NOT run-to-run deterministic on the Vulkan backend — same config, same prompt, temperature 0, batch 1, no speculation, three different outputs.

verified 2026-08-05 · volatility medium

Note — the record's own working

Reported on Strix Halo/gfx1151 with outputs diverging mid-generation. Deep MTP (n>=4) amplifies it to first-token divergence via draft-length to batch-shape variance. The reporter is explicit that outputs stay coherent — this is variance, not corruption. DIRECT CORRECTION TO OUR PROTOCOL: benchmark-protocol.md adopted "temperature 0, fixed seed" as a determinism control, borrowed from homebench. On this backend that control does not hold, which has two consequences. First, N=1 is not defensible for anything output-dependent even where we assumed determinism. Second, and worse: **speculative losslessness cannot be verified by diffing outputs on Vulkan**, which is exactly how one would naturally check that MTP or ngram speculation is not changing results. UNVERIFIED HERE, and worth checking on ROCm rather than assuming it transfers — the report is Vulkan-specific and our production backend is ROCm. If it holds on ROCm too, quality comparisons need a distributional instrument (KL divergence) rather than output equality, which is the approach we had already chosen for KV quant.

Cited by — computed at build time, never stored