Home › Compare

The field

Every benched model's best measured configuration, side by side, on the same hardware. A row states its own fingerprint; a number you can't trace to a run is a bug. Click a column header to re-sort — the table is complete and sorted without it.

held constant across every row
aihydra · Strix Halo 128 GB · rocm · llama.cpp 3653e6d · fa on · f16 KV · --parallel 1 · 3 reps · page cache dropped between models · guard before every series
provisional — read before citing
Cross-model τ² scores ran self-play (agent = user simulator) and are confoundedclm-0043; re-runs under the pinned independent simulator are in progress clm-0048. Within-model deltas are unaffected.
modelactive / totalguarddec @0dec @32kpp @0τ² airlineturns med (all tasks)Wh / correctverified
Qwen3.6-35B-A3B
UD-Q4_K_XL · rocm · 3653e6d · benched 2026-08-09
35B / 3B active4/451.0142.451079.38not rununmeasured2026-08-08
gpt-oss-120b
UD-Q4_K_XL · rocm · 3653e6d · benched 2026-08-08
excluded — returns stale answers from previous requests clm-0025
117B / ~5B activestale answers55.4541.70450.83not run — guard failedunmeasured2026-08-08
Qwen3.5-122B-A10B (MTP)
UD-Q4_K_M · rocm · 3653e6d · benched 2026-08-08 · draft-mtp available (NextN heads in this GGUF)
122B / 10B active4/421.9118.17321.720.545 ±0.208 · n=22225.71 n=5 smoke2026-08-11
Nemotron-3-Super-120B-A12B
UD-IQ4_XS · rocm · 3653e6d · benched 2026-08-09
quant confound resolved — reward identical at UD-Q4_K_M, +11% total turns at IQ4_XS clm-0047
120B / ~12B active4/417.5117.16225.360.625 ±0.237 · n=1628unmeasured2026-08-11

decode/prefill: t/s, llama-bench, scatter published per cell on each model page · τ²-bench airline, thinking off; ± is a 95% binomial interval; n below 10 carries the smoke badge — clm-0036's noise finding made structural · turns = median conversation turns to a completed task across ALL tasks, failures included — this lab's headline efficiency metric · Wh / correct joins energy records to capability runs; the column ships even while sparse, because an honest gap in the differentiating unit beats hiding the unit · exclusions render muted with a red edge, but present: exclusions are data

the default sort, drawn — decode at working depth
01020304050decode t/s @ 32k · best config per model42.45 · qwen36-35b · CV 0.1% · N=3 · run-003542.45qwen36-35bUD-Q4_K_XL41.70 · gpt-oss-120b · CV 0% · N=3 · run-004941.70gpt-oss-120bguard failed — stale answers18.17 · qwen35-122b · CV 0.1% · N=3 · run-001918.17qwen35-122bUD-Q4_K_M17.16 · nemotron3-super · CV 0% · N=3 · run-004317.16nemotron3-superUD-IQ4_XS
3 reps per cell · max CV 0.1% · build 3653e6d · the guard-failed bar renders dashed, not deleted — exclusions are data · records: run-0035 run-0049 run-0019 run-0043
Not yet in the field:2 listed0 acquired13 screened0 blocked4 NPU-lane (separate artifacts — numbers never transfer)→ the candidates board, with every gate reason