clm-0042
measured-heremed ●●○
citable URL: https://halobench.com/records/clm-0042/ — this address never moves; the anchor /records/#clm-0042 keeps resolving
Per-task energy windows, matched to a common task set across arms, put a correct τ² answer at 6.81 Wh on the 122B with f16 KV, 9.48 Wh with q8_0, and 12.75 Wh on Nemotron. That is 0.21 p, 0.29 p and 0.39 p at 30.3 p/kWh. The q8_0 penalty is +39%, the same figure clm-0038 measured for turns, which is the mechanism. These supersede clm-0040's arm-level numbers, which were ~10x too high because ~90% of an arm's energy went on model loading and tasks that never scored.
Note — the record's own working
## Method — and why it took three attempts
Each τ² simulation records `start_time` and `end_time`. Energy is the cumulative wall counter differenced across each TASK's window and summed, so it excludes model load, inter-task gaps, and tasks that never scored. Restricted to the **9 tasks common to all three arms**, which removes the contiguous-prefix confound (`clm-0037`) — bound-limited arms reached different depths into the task list.
Three passes at this number, each wrong for a different reason:
1. **Hourly statistics** — too coarse; the operator flagged it. Fixed by using the 10-second raw
counter.
2. **Whole-arm energy ÷ correct answers** — the operator asked whether failed tasks were
included. They were, and worse: only 8-27% of each arm's wall time was inside scored
tasks at all, so the figures mostly measured loading and a 4.4-hour non-terminating
task that never scored.
3. **This.** Per-task windows, matched task set.
⚑ The transcripts carried `start_time`/`end_time` all along. I concluded per-task windows were unavailable and built new instrumentation for future runs — correct to do, but the data for these arms was already there and I had not looked. The operator asked "the tau2 transcripts don't contain timestamps?" and they did.
## The measurement
| arm | correct | in-task time | total Wh | Wh/task | **Wh per correct** | **pence per correct** | |---|---|---|---|---|---|---| | 122B f16 | **9 / 9** | 22.4 min | 61.30 | 6.81 | **6.81** | **0.21 p** | | 122B q8_0 | 8 / 9 | 27.5 min | 75.84 | 8.43 | **9.48** | **0.29 p** | | Nemotron off | 8 / 9 | 36.3 min | 102.00 | 11.33 | **12.75** | **0.39 p** |
## What it says
**The 122B with f16 KV is the cheapest per useful result, by a clear margin** — 6.81 Wh against Nemotron's 12.75, so Nemotron costs **87% more electricity per correct answer**. On the same nine tasks it also took 62% longer in-task (36.3 min against 22.4).
**q8_0 costs +39% per correct answer against f16 on the identical model.** `clm-0038` measured +39% TURNS on this same paired task set. Energy and turns agreeing to the percentage point is strong evidence that the extra turns *are* the mechanism — quantised KV does not draw more power, it does more work.
This also completes the picture with `clm-0041`, which found the dequant patch cuts energy 42% on fixed-token throughput runs and makes patched q8_0 cheaper than f16. Both hold: **quantised KV is cheaper per token and dearer per task**, and which dominates depends on whether the workload's length is fixed or emergent. The agentic measurement here is on the stock build; whether the patch also removes the per-task penalty is untested and is the obvious next run.
## ⚑ SELF-PLAY OVERHEAD — the user simulator is not useful work
The operator, 2026-08-10: *"the model is picking up both sides of the conversation, the actual useful work is the LLM side, not the 'user' side. Do you account for this already?"* I had flagged it in scope and NOT corrected for it. Correcting now.
τ² self-play runs the agent and the user simulator on the same model and the same box, so the measured energy includes generating the customer's half of the dialogue — scaffolding, not work anyone would pay for. Only `assistant` messages carry `generation_time_seconds`, but that bounds it:
| arm | agent generation | share of task wall time | remainder | |---|---|---|---| | 122B f16 | 1031.4 s | **76.8%** | 23.2% | | 122B q8_0 | 1353.4 s | **82.0%** | 18.0% | | Nemotron off | 1726.1 s | **79.3%** | 20.7% |
Apportioning energy by that share:
| arm | measured Wh/correct | **agent-only Wh/correct** | vs f16 | |---|---|---|---| | 122B f16 | 6.81 | **5.23** | — | | 122B q8_0 | 9.48 | **7.77** | **+49%** | | Nemotron off | 12.75 | **10.11** | **+93%** |
**The ranking is unchanged and the gaps widen.** So the conclusions hold, but the headline numbers were ~20-25% too high as a measure of useful work.
Treat the agent share as a LOWER bound on its share of energy. The non-agent remainder is user-simulator generation PLUS tool execution and framework overhead, and tool calls are local database operations that draw near-idle power. So the agent's share of energy ABOVE IDLE is higher than its share of wall time — the true correction is smaller than 20-25%, and the measured figures are conservative rather than optimistic.
A cleaner design would run the user simulator on a different host, or on a small model whose cost is separately accounted. That is a benchmark-harness change, not an analysis one, and worth doing before energy figures are quoted as model properties.
## Scope
· 9 matched tasks, one domain, self-play, deterministic scoring, read paths only. · Wall measurement, so whole-box: CPU, RAM, NVMe, fans, PSU losses. Idle floor 10.1 W is
included in these figures, not subtracted — at ~160 W active it is ~6% of the total.
· Tariff is the observed 30.3 p/kWh grid import. Solar and battery mean realised cost is
lower, sometimes zero.
· ⚠ The 122B-vs-Nemotron comparison carries the unpinned-simulator confound documented in
`clm-0043` and should be read as provisional; the f16-vs-q8_0 comparison does not, since
both arms ran the same model as its own simulator.
Cited by — computed at build time, never stored
supersedes clm-0040 — the withdrawn record keeps its URL
model pages qwen35-122b