clm-0023
measured-herehigh ●●●
citable URL: https://halobench.com/records/clm-0023/ — this address never moves; the anchor /records/#clm-0023 keeps resolving
Qwen3.6-35B-A3B is 2.3x the 122B's decode on identical hardware (51.01 vs 21.90 tok/s at empty context) and holds 42.45 at 32k, with prefill above 1000 tok/s. It passes the capability guard 4/4.
verified 2026-08-08 · volatility medium
Note — the record's own working
QWEN3.6-35B-A3B (UD-Q4_K_XL, 21 GB), aihydra, ROCm 7.1.0, stock llama.cpp 3653e6d, `-fa 1`, f16 KV, `--parallel 1`, 3 reps. Ingested as cfg-0012 / run-0030..0032.
| depth | pp512 | tg128 | |---|---|---| | 0 | **1079.38** | **51.01** | | 4096 | 965.37 | 49.80 | | 32768 | 598.68 | 42.45 |
All cells ≤0.6% CV. Guard 4/4 at depth 8000.
THE ACTIVE-PARAMETER PREDICTION, TESTED. Decode is bandwidth-bound, so throughput should scale roughly inversely with ACTIVE parameters: A3B (3B) against A10B (10B) predicts ~3.3x. We measure **2.3x** (51.01 vs 21.90). The direction and rough magnitude hold, and the shortfall is expected — per-token overhead does not scale down with active parameters. Good enough to keep using active-parameter count as a first-order predictor, not good enough to quote as a rule.
It also retains speed at depth better than the 122B: 51.01 → 42.45 is a 17% loss to 32k, against the 122B's 21.90 → 18.16, also 17%. Proportionally identical, which suggests the depth penalty is an attention-cost property rather than a model-size one.
REFLEX-TIER VERDICT: at 42-51 tok/s with sub-second prefill on 21 GB, this is a credible fast lane. Combined with clm-0022's finding that quantised KV is free on a patched build, a 35B and the 122B could plausibly be co-resident within the 105 GiB ceiling — which is the council architecture's first concrete opportunity on this hardware.
⚑ GRAMMAR CEILING: see clm-0024, which re-measures this properly and finds the ceiling does not reproduce — the constraint is context size, not the grammar builder. Kept below as the process record of how the wrong number nearly got published.
NOT MEASURED HERE, AND THE FIRST ANSWER WAS WRONG. The probe reported "highest OK: 30 tools (127,415 B)" against the real 353 KB Home Assistant schemas. That is **not** the grammar ceiling. The 400 carried `{"type":"exceed_context_size_error", "n_prompt_tokens":16651, "n_ctx":16384}` — the server was started at `-c 16384` and 31 real tools need ~16.6k tokens. We measured the CONTEXT limit and nearly published it as a grammar limit, which would have understated the real ceiling badly and sent us hunting a regression that does not exist.
The script's own `grammar-related error text: no` line caught it — and then printed a ceiling anyway. **A caveat nobody acts on is not a safeguard.** Fixed 2026-08-08: the probe now exits 2 and refuses to report any number when the failure is a context overflow, naming the token count it needed.
Re-running at `-c 131072`. The real ceiling is somewhere above 30 tools and currently unknown. For scale: at ~4 KB per real MCP tool, probing 120 tools needs roughly 60k tokens of context — which is itself a useful fact, because production runs nowhere near that much context devoted to tool schemas.
ALSO PASSED TONIGHT: tau2-bench smoke test. LiteLLM reaches our endpoint and gets a correct native tool call (`get_booking {"reference":"ABC123"}`), so the adopted agentic-benchmark path works end to end and is no longer an untested assumption.