clm-0026

Qwen3.6-35B-A3B UD-Q4_K_XL, aihydra, ROCm 7.1.0, stock llama.cpp 3653e6d, `-fa on`, f16 KV, `--parallel 1`, ctx 16384, 3 reps x 5 varied prompts (n=15 per arm). | spec type | tok/s | stddev | ratio | |---|---|---|---| | none | **50.91** | 0.10 | 1.00x | | ngram-mod | 56.42 | **16.53** | 1.11x | | ngram-cache | 50.22 | 1.36 | 0.99x | | draft-mtp | FAILED | | | The floor agrees with llama-bench's independent 51.01 to 0.2%, so both harnesses are measuring the same thing. **draft-mtp failed cleanly and for a good reason:** `context type MTP requested but model doesn't contain MTP layers`. Unlike Qwen3.5-122B-A10B-MTP, this build of the 35B carries no NextN heads, so the 1.45x that MTP delivers on the 122B is simply unavailable here. A recorded failure against its fingerprint, not a retry candidate. **ngram-mod's 29% CV is the finding, not its 1.11x mean.** On the 122B it was 1.17x with 38% CV; here 1.11x with 29%. Draft-free speculation only pays when the output repeats text the n-gram pool has already seen, so on five deliberately dissimilar prompts some runs gain substantially and others gain nothing. The mean is not a number to plan with. This QUALIFIES clm-0016 rather than contradicting it: their 1.87x came from 500 IFEval prompts, which are short, structured and highly repetitive — the best case for the technique. On open-ended generation it largely evaporates. WHY WE WILL NOT REACH THEIR 121 tok/s BY CONFIGURATION. Decomposing the gap honestly: · **Their no-speculation floor is 64.8; ours is 50.91 — 27% apart.** A floor difference cannot be caused by speculation, parallelism or tuning. The remaining structural difference is the quant: they ran **ROCmFP4**, we ran UD-Q4_K_XL. Quant format sets bytes-per-weight and therefore the bandwidth ceiling itself, which is the one term that moves a floor. · **Their 121 is FOUR STREAMS POOLED, not single-stream.** Their own post labels the sustained figure as 4 slots across 500 prompts. Our 50.91 is one stream. Comparing the two directly overstates the gap; aggregate throughput and single-stream latency are different quantities and should never be set side by side. · Their trustworthy single-stream peak was 380 tok/s, but peak in a real agentic session is not a sustained rate either. So the honest reading: **ROCmFP4 is the lever worth chasing on this model, and it is worth roughly 27%.** Speculation is not — on varied prompts it buys ~11% with scatter three times larger than the gain. ⚠ AND WE CANNOT SIMPLY COPY THEIR PARALLEL-4 SETUP. clm-0019 measured that `--parallel 4` destroys long-context retrieval on this build (needle 0/8 against 8/8 at `--parallel 1`). Their 121 tok/s was measured at parallel 4. Either their build does not carry that defect, or their IFEval prompts were short enough never to expose it. Chasing their number by raising slot count would trade a capability we have verified for throughput we have not. NEXT, IF THE 35B MATTERS AS A REFLEX TIER: build the ROCmFPX fork and requantise. That is a third divergence to carry, on top of `con-0001` and the KV-dequant patch (clm-0022) — but unlike speculation it moves the floor, and the floor is what a reflex tier is for. ⚑ **CLM-0010 IS UNRECONCILED WITH THIS.** That report claims draft-free ngram alone takes a single stream from 71 to 216 tok/s — a 3x ratio, against our measured 1.11x on the same technique here. Neither their ROCmFPX-fork ROCmFP4 finetune nor their prompt regime (repetitive code-edit content, ngram-mod's best case) matches ours, and either could account for most of the gap — but it has not been tested, so the discrepancy stands rather than being averaged away.