clm-0027

Qwen3.5-122B-A10B UD-Q4_K_M, aihydra, ROCm 7.1.0, stock llama.cpp 3653e6d, `-fa on`, f16 KV, `-c 32768`, `--parallel 1`, `--load-mode none`. ## Cache behaviour — the ladder nobody publishes | probe | wall | prefill | meaning | |---|---|---|---| | cold | **25.99 s** | 317.5 t/s | first sight of this prefix | | warm | **2.65 s** | 269.2 t/s | same prefix, different question | | diverged | **26.26 s** | 315.9 t/s | three characters prepended | **9.8x from prefix reuse.** That is the whole warm-lane thesis, measured on this hardware rather than inferred: the difference between a turn that feels instant and one that does not is almost entirely whether the prefix was seen before. **And reuse is prefix-anchored, not similarity-based.** The `diverged` probe is 99.9% identical to `cold` — the same 8,000 tokens, with `ZZZ ` prepended. It gets **no reuse whatsoever** and costs 101% of cold. A changed head invalidates everything after it. That is the measured justification for design decisions already taken on faith: · why the prefix-relocation fix (`cache-maxing-prefix-fix`, upstreamed as openclaw #98267) mattered — moving two sections below the cache boundary took shared tokens from 1.46K to 15.5K, and this shows what each shared token is worth; · why heartbeats and crons that carry their own preamble **cannot** share a warm lane with iMessage traffic, and why slot pinning was the right answer rather than a bigger cache; · why anything that mutates the head of a prompt — a timestamp, a rotating greeting, a changing tool list — is far more expensive than its size suggests. GAP: this build does not populate `prompt_n_cached` in `timings`, so the ratio is measured by wall clock rather than reported token counts. The effect is large enough (9.8x) that this does not threaten the conclusion, but a cached-token figure would let us measure PARTIAL reuse rather than just its presence or absence. ## Fit — both KV arms load everywhere | KV | ctx | loaded | load_s | peak GiB | |---|---|---|---|---| | f16 | 32,768 | yes | 46 | 75 | | f16 | 131,072 | yes | 78 | 77 | | f16 | 204,800 | yes | 48 | **79** | | q8_0 | 32,768 | yes | 95 | 75 | | q8_0 | 131,072 | yes | 62 | 76 | | q8_0 | 204,800 | yes | 47 | **77** | **No fit cliff anywhere.** f16 KV reaches 204,800 with 79 GiB peak against the 104 GiB GTT ceiling — 25 GiB spare. The arms are therefore comparable at the top length, which protocol §9 warns is not to be assumed. ⚑ **THE f16/q8_0 DIFFERENCE IS 2 GiB, NOT 12.** `cfg-0002` records `kv_gb: 12.4` for q8_0 at 200,000 context, implying ~25 GiB for f16. The measurement says otherwise: 79 vs 77 GiB peak. The likely explanation is architecture — this is a hybrid model, so only a minority of layers hold a full attention cache, exactly as clm-0015 found for Ling-3.0-flash (7 of 42 layers). **If so, cfg-0002's memory record is a substantial overestimate and the fit calculus behind several decisions was too conservative.** It also means quantised KV saves far less memory than assumed — which, combined with clm-0022 showing it costs nothing in speed once patched, makes the whole q8_0-vs-f16 question much less consequential than it looked. Both figures come from `capability-probe.sh`, which only produced them after being given `--load-mode none`; before that both arms timed out at 673 s and were recorded as non-loading. A probe that cannot load the model reports a fit cliff that does not exist.