clm-0040

Source: Home Assistant recorder, `sensor.hardware_ai_hydra_energy` — a CUMULATIVE kWh counter at the wall socket, sampled every ~10 seconds at full float precision. Measured at the SMART PLUG, so this is whole-box draw — CPU, 128 GB RAM, two NVMe, fans and PSU losses included — not GPU package power. ## Method: counter differences, not averaged power Per-arm energy is the counter's value at the end minus its value at the start. No integration of noisy power samples, no assumption about duty cycle. At ~150 W a 10-second boundary error is ~0.0004 kWh, i.e. negligible. I first computed these from HOURLY statistics, which the operator rightly flagged as too coarse — over a 6-hour arm the boundary error is tolerable, but it is meaningless for the minutes-long performance runs. Re-measured at 10-second resolution the values moved by at most 1.2%, so the conclusions were not wrong, merely imprecise. The method now works at any run length, which is what matters for the backfill. ⚠ **Raw 10-second history is retained ~10 days.** Long-term statistics persist forever but only hourly. So the Aug 7-8 performance runs must be extracted before roughly Aug 17-18 or they drop to hourly resolution permanently. ## ⚑ This field was empty for 99 runs before today The `energy` field has been in the run schema since the beginning and every one of the 99 recorded runs carries `energy: null`. The operator asked whether it was being captured; it was not. Wall-measured energy is the thing this lab has that published benchmarks almost universally lack, and it was designed in and then never populated. ## The power envelope | state | wall power | |---|---| | idle | **10.1 W** | | 122B under τ² load | ~154 W mean | | Nemotron under τ² load | ~167 W mean | | peak observed | **218 W** | Nemotron draws about **8.5% more** than the 122B for the same work — consistent with the 86 °C spike observed on it against the 122B's steady 68-74 °C. ## ⛔ WHAT "PER CORRECT ANSWER" ACTUALLY MEASURES HERE — read before quoting The operator, 2026-08-10: *"you had the energy delta for a full run, so including some failed tasks right? this would mean that the 'energy per correct' would be higher than that model?"* Correct, and the problem is larger than failed tasks. The figures below are **total arm energy divided by correct answers**. Two consequences: **1. Failed-task energy is included.** Deliberate — one useful result should carry the cost of the wrong ones. But it means this is NOT the model's per-task energy, and the label invites that reading. **2. Most of the energy was not spent on scored tasks at all.** Checking the summed task durations against the 360-minute arms: | arm | arm length | time inside scored tasks | fraction | |---|---|---|---| | 122B f16 | 360 min | 62.8 min | **17%** | | 122B q8_0 | 360 min | 27.5 min | **8%** | | Nemotron off | 360 min | 98.5 min | **27%** | So 73-92% of the measured energy went on model loading and on tasks that never scored. Arm 2's 4.4-hour non-terminating task is **not among its nine scored results** — 4.4 hours of electricity yielding no measurable outcome, then attributed to the eight answers that did land. **3. The arms cover DIFFERENT TASK SETS.** Bound-limited at 6h, arm 1 reached tasks 0-21, arm 2 only 0-8, arm 3 0-15. Comparing them compares different mixes of work — the same contiguous-prefix confound `clm-0037` names, which I flagged there and then ignored here. **So these numbers answer "run the box six hours; what does each correct answer that falls out cost?" — a real question, but not "how efficient is this model".** They should not be used to rank models. The sound comparison is the paired one in `clm-0038`, and a sound energy comparison needs matched task sets with per-task windows, which requires the `started_at`/`ended_at` instrumentation added on 2026-08-10 and does not exist for these arms. ## Energy per correct answer, as measured (see caveats above) Six-hour arms, so total energy is nearly identical across them; what differs is how many correct answers each bought. | arm | counter start -> end (kWh) | delta kWh | tasks | correct | Wh/task | **Wh per CORRECT answer** | marginal* | |---|---|---|---|---|---|---|---| | 122B f16 | 4.804997 -> 5.740740 | **0.93574** | 22 | 12 | 42.5 | **78.0** | 72.9 | | Nemotron off | 6.662776 -> 7.663805 | **1.00103** | 16 | 10 | 62.6 | **100.1** | 94.0 | | 122B q8_0 | 5.740740 -> 6.662776 | **0.92204** | 9 | 8 | 102.4 | **115.3** | 107.7 | *delta = total minus the 10.1 W idle floor, i.e. energy attributable to inference rather than to the machine merely being on. `docs/lab-site-design.md` calls this "the only figure that means anything", because a naked wattage reading mostly measures the idle floor. ## In money, which is the point of using kWh At the Grid Import Price observed today, **30.3 p/kWh**: | arm | Wh per correct answer | **pence per correct answer** | delta-only | |---|---|---|---| | 122B f16 | 78.0 | **2.36 p** | 2.21 p | | Nemotron off | 100.1 | **3.03 p** | 2.85 p | | 122B q8_0 | 115.3 | **3.49 p** | 3.26 p | So the q8_0 KV penalty is about **1.1 p per correct answer** — small per answer, and the kind of number that only becomes visible when denominated in something a person can price. ⚠ Tariff caveat: this uses the instantaneous import price. The house has solar and a battery, so the marginal cost of a run is lower — sometimes zero — when it lands in a solar or cheap-rate window. `docs/lab-site-design.md` reserves a `tariff_window` field for exactly this; these figures are grid-import-equivalent, not what was actually paid. ## ⚑ UNITS: kWh / Wh / pence, never joules `docs/lab-site-design.md` already specified this — *"mWh per task, Wh per 1,000 tasks, kWh per month. Joules is not a home-energy unit and doesn't map to /kWh tariffs."* I wrote "denominated in joules" and "J/token" anyway. Same failure as drifting from doc 14's time-to-correct headline: the project had decided, and I did not check. ## ⚑ Quantised KV is an energy REGRESSION Same model, same tasks, only `-ctk/-ctv` differing: **78.0 Wh per correct answer with f16 against 115.3 Wh with q8_0 — a 48% penalty**. Quantised KV saves memory and spends electricity. That is a second, independent instrument agreeing with `clm-0038`, which found q8_0 needs **+39% more turns** on paired tasks. Extra turns are extra work, and work is electricity. Two measurements of different quantities pointing the same direction is much stronger evidence than either alone — and neither was visible in the reward metric, which moved by a single task. It also reframes `clm-0022`. That claim established the dequant patch recovers +70.3% throughput, making quantised KV look free. At the wall it is not free: on the stock build it costs half again as much energy per useful result. Whether the patch removes the energy penalty along with the speed penalty is now a well-posed question with a way to answer it. ## Scope · Arms were bound-limited at 6h and are contiguous task prefixes, not random samples (clm-0037). The q8_0 arm completed only 9 tasks because one ran 4.4 h, which inflates its Wh/task — but that IS the cost, not an artefact. · Boundaries are resolved to ~10 s from the raw recorder, not to the hour. The residual error is ~0.0004 kWh per boundary, far below the effects being compared. · Whole-box measurement includes anything else the machine was doing. The arms ran with the box otherwise quiet, but model downloads overlapped part of the first arm's window. · The 10.1 W idle figure is the box powered but not serving. A model held resident in memory would sit higher, so marginal energy here slightly overstates the true increment for an always-warm server. · Wall measurement excludes nothing on the machine but does exclude network gear and the client side.