Lab Site — Design Notes

Structural design for publishing and maintaining the Warden stack findings. Captured 3 Aug 2026. Pre-implementation. Companion to openclaw_research.md and claude_model_landscape_aug2026.md.

1. Premise

1.1 The gap

The public benchmark commons is degrading at the moment local agents became viable:

• Official BFCL leaderboard last updated 12 Apr 2026
• BFCL v3 and v4 are not comparable, and are mixed silently by third-party leaderboards
• ACEBench effectively dead (two models listed)
• τ²-bench is the only consistent cross-vendor signal, and only because vendors self-report it

Meanwhile the wider field publishes tokens/second and almost nothing else.

1.2 What this site publishes that others don’t

Positioning: revealed preference over leaderboard scores. Not “this scored 73.4” but “this has been the daily driver for 62 days, here is what it displaced, here is the exact config, here is what it costs to run.”

1.3 Sustainability constraint

Living pages rot. Every homelab benchmark site dies at month four because updating it is a writing chore.

Therefore: build a dataset with a renderer, not a site.

• Benchmark results as structured records (YAML) in a git repo
• Site generated from records at build time
• Warden appends records from harness runs during cheap-tariff windows
• Build on push — Cloudflare Workers static assets, with wrangler.jsonc declaring the asset directory. (Cloudflare Pages is no longer the default path for new projects and has no build-output-directory field; any guide written against Pages is stale.)

The homelab becomes its own CI. Publishing stops being an act of will.

2. Record types

Eleven record types. Every page on the site is a view over these.

2.1 node — anything appearing on the topology

id: qwen36-27b-mtp              # stable, never reused
kind: model                      # host | model | service | agent | tier
title: Qwen3.6-27B dense @ Q8_0 + MTP
runs_on: [evo-x2]
engine: igpu                     # igpu | npu | cpu | discrete-gpu
tier: reflex
config: cfg-0087

lifecycle:                       # append-only. Warden writes, never edits.
  - state: candidate
    date: 2026-05-18
    evidence: [run-0142]
  - state: live
    date: 2026-06-02
    decision: dec-0031

rationale: rat-qwen36-27b        # → human-written "why"

States: candidate | live | retired | rejected.

lifecycle is load-bearing. Days-in-production, the historical topology renderer, and the changelog feed all derive from it — none are maintained by hand.

rejected is the rarest state on the internet and among the most useful. Negative findings (e.g. FastFlowLM v0.9.43: 1.55 s flat TTFT, revisit at v1.0+) are published, not discarded.

2.2 config — the reproducibility fingerprint

Separate record because runs are only comparable when configs are, and one config is reused across many runs.

id: cfg-0087
model_ref: unsloth/Qwen3.6-27B-GGUF
revision: 3f9a1c2                # HF commit, not tag
quant: Q8_0
host: evo-x2
engine: igpu

runtime:
  repo: ggml-org/llama.cpp
  commit: b6142
  backend: vulkan-amdvlk
  tree: fork                     # upstream | upstream+patches | fork — DERIVED
  fork_ref: <repo>@a91c2f
  divergence: [con-0007]         # → contribution records
  rebuild_required: true

flags: >
  -ngl 99 --spec-type draft-mtp --spec-draft-n-max 3
  --cache-type-k q8_0 --cache-type-v q8_0 --jinja --parallel 2
chat_template_sha256: 9c1f...
chat_template_kwargs: {preserve_thinking: true}
sampling: {temp: 1.0, top_p: 0.95, min_p: 0.01}

memory:
  weights_gb: 28.9
  ctx_per_slot: 65536
  slots: 2                       # --parallel
  ctx_total: 131072              # DERIVED
  kv_gb: 8.4
  kv_quant: q8_0/q8_0            # k/v — materially changes kv_gb
  overhead_gb: 1.2               # compute buffers, MTP draft heads
  total_gb: 38.5                 # DERIVED
  pool_gb: 96
  headroom_gb: 57.5              # DERIVED — concurrency answer

Conservative resolution is intentional, not a gap. Any non-empty divergence resolves to fork without checking merge state. That over-reports fork, failing in the safe direction: a replicator is told they may need more work than they do, never less. The alternative failure — trusting an upstream badge, building stock, getting different results — is the one that destroys trust in the dataset.

tree is derived, not hand-set. If every linked contribution is state: merged and merged at or before commit, it resolves to upstream automatically. Nothing to remember to flip back.

Three states, three messages to a replicator:

• upstream — checkout the commit, use the flags, done
• upstream+patches — stock build, but listed flags/template differ from defaults
• fork — build from a specific tree; PR and status linked

The middle state matters most. The common reproducibility risk isn’t a custom binary, it’s a patched Jinja template or non-default tool parser sitting quietly in config.

Memory block notes:

• slots changes everything — --parallel 2 doubles KV. Almost never stated publicly.
• kv_quant must sit next to kv_gb; q8_0 vs f16 is ~2× on the same context.
• On unified memory total_gb is a claim against a shared pool, not private VRAM.
• OOM risk is a warn line, not a fail line. Crossing > 0.8 requires an explicit acknowledgement record; it never makes a config unrecordable. A config record describes what was run — if it ran at 84%, the record must be able to say so. A schema that refuses to record reality is a schema that lies, and the first thing it would refuse is the actual daily driver. Three constraints:
  • A reason, not a boolean. An acknowledgement with no argument attached is a bypass with extra steps.
  • Warden must never set it. Risk acceptance is an argument, not a state (§5.1). Enforced by PATH: acknowledgements live in human/, outside Warden’s write path. Not by git identity — that would reject a legitimate acknowledgement the moment an agent touched anything else in the same file, and the obvious workaround trains you to launder agent commits.
  • Publish it. The badge renders with its reason. “Runs at 84% of pool, here is why that is tolerable” is more useful than any config sitting comfortably under the line.
• ⚑ total_gb is a static ESTIMATE and has never predicted an OOM (clm-0001). The 2026-07-21 panic happened at 83.8% estimated — and cfg-0002 sits at that same 83.8% today. What exhausted memory was runtime growth: context checkpoints defaulting to 32, cache-ram, fragmentation. None of it appears in the record. The mitigations worked because they targeted dynamic growth, which is exactly why they moved the ratio not at all. Split the quantities: total_gb stays the estimate; observed_peak_gb is the high-water mark under load and can only come from telemetry. The warn line keys on the observed peak when present (risk_basis: observed) and otherwise says so (risk_basis: estimate). Until that collector exists, 0.8 is a threshold on a number with no demonstrated predictive power, and the schema must not imply otherwise.
• That collector is a memory sensor, not the smart plug — a different job from the energy programme in §6, and arguably higher value for this specific risk. Peak RSS/GTT under load, exported from the node or the proxy.
• On Strix Halo an OOM takes the whole box down — headroom is a stability metric, not a comfort one, and recovery needs physical or out-of-band access.

2.3 run — immutable, append-only, Warden-generated

id: run-0142
date: 2026-05-18
config: cfg-0087
suite: tool-loop@v2.1
metrics:
  decode_tps: 18.1
  prefill_tps: 502
  ttft_ms: 180
  time_to_correct_answer_s: 14.2
  tool_call_success: 0.94
  turns_before_degradation: 10+
  empty_arg_calls: 0
energy: {ref: eng-0142}
cross_host_pair: null            # → same suite+config on another host

Runs are never edited. A wrong run is superseded, not corrected — visible history is the point.

2.4 energy — Wh-denominated, wall-metered

id: eng-0142
run: run-0142
method: wall-meter               # wall-meter | rapl | attributed | vendor-claim
meter: <device model>
sample_hz: 1
baseline_state: igpu-resident-quiet
baseline_w: 42                   # measured same session, same ambient
peak_w: 138
mean_w_active: 111
delta_w: 69                      # the only figure that means anything
mwh_per_task: 440
wh_per_1000_tasks: 440
pence_per_correct_answer: 0.011
tariff_window: agile-cheap
contention: false                # true if another engine was active

Units: mWh per task, Wh per 1,000 tasks, kWh per month. Joules is not a home-energy unit and doesn’t map to /kWh tariffs.

Delta over baseline is the honest figure. Strix Halo idles high; a naked wattage reading mostly measures the idle floor.

method does real work. A wall-meter badge sitting next to someone else’s vendor-claim is quietly decisive. Note the register: the “<2 W for a 3B” figure is a secondary source repeating a vendor claim, and no independent XDNA2 NPU power measurement exists.

2.5 idle_baseline — the silent cost of always-on

State is a vector, not a flat enum, because real states are combinations.

id: idle-evo-x2-2026-07
host: evo-x2
period: {from: 2026-07-01, to: 2026-07-31}
method: wall-meter

states:
  - state: {igpu: empty, npu: empty, cpu: idle}
    label: box-idle-all-empty         # the floor every delta subtracts from
    mean_w: 28
    hours: 0
  - state: {igpu: {resident_gb: 32.4, activity: quiet}, npu: empty}
    mean_w: 42
    hours: 611
  - state: {igpu: empty, npu: {resident_gb: 1.9, activity: quiet}}
    mean_w: 31
    hours: 0
  - state: {igpu: {activity: active}, npu: {activity: quiet}}
    mean_w: 118
    hours: 133

standing_kwh_month: 25.7          # idle hours only
standing_cost_gbp: 3.84           # at realised Agile average
duty_cycle: 0.18

Per engine, two independent axes — residency (empty / weights loaded) and activity (quiet / inferring) — giving {engine}-empty, {engine}-resident-quiet, {engine}-active, plus the box-level floor.

Three deltas that matter specifically here:

Headline framing: standing cost ÷ tasks completed. A box at 18% duty cycle carries most of its cost doing nothing — which reframes “buy a second box” from a capability question into a recurring-cost one.

2.6 workload — the drilldown dimension

id: wl-heartbeat-scan@v1
surface: openclaw                 # openclaw | claude-code | cron | interactive
session_type: isolated-light
character:
  prefill_tokens: {p50: 4200}
  decode_tokens: {p50: 180}
  tools_per_turn: {p50: 2}
  turns: {p50: 1}
  shape: prefill-heavy            # prefill-heavy | decode-heavy | tool-loop-heavy
latency_tolerance: minutes
correctness_bar: low

shape is the field that earns its keep. It predicts which hardware tier wins and is the quantitative form of the NPU tiering argument.

Versioned (@v1). Redefining a workload without a version bump silently compares different things — the same failure this site criticises in BFCL v3/v4 mixing.

2.7 telemetry — production, kept separate from benchmarks

Keyed on (config × workload × period). Benchmarks are controlled and reproducible; production is observed and workload-dependent. Mixing them destroys both.

id: tel-2026-07-qwen36-27b-heartbeat
config: cfg-0087
workload: wl-heartbeat-scan@v1
period: {from: 2026-07-01, to: 2026-07-31}
requests: 5131
prefill_tps: {p50: 471, p90: 502, mean: 468}
decode_tps:  {p50: 17.9, p90: 19.4, mean: 17.2}
ttft_ms:     {p50: 210, p90: 890, p99: 2100}
cache:
  prompt_prefix_reuse: 0.73       # llama.cpp — server metrics scrape
  proxy_response_hits: 0.09       # distinct mechanism — never merged
context_utilisation: {p50: 0.31, p90: 0.68}

Rules:

• Distributions, not means. p50/p90/p99 or it isn’t useful.
• Two cache numbers stay separate. Prefix reuse and proxy response hits are different mechanisms with different implications.
• Content never leaves. Counters and histograms only — no prompts, no session ids, no entity names. Hard schema constraint, since Warden generates these unattended.
• Minimum-n before a cell publishes. Drilldown multiplies cardinality fast; a cell with 11 requests must not render as a confident p90. Warden suppresses and shows n.

Known gaps in production instrumentation:

The energy/telemetry join is the only real build work in this design. Everything else is schema.

2.8 suite — versioned, executable, dependency-aware

id: tool-loop@v2.1
supersedes: tool-loop@v2.0
comparable_with: [tool-loop@v2.0]  # explicit, not inferred from semver
breaking: false
changed:
  - Added empty-arg-call detection
  - Fixed grader false-negative on tool ordering
frozen_inputs_sha256: a91c...
grader:
  kind: rubric-llm
  model: <ref>
  config: cfg-grader-004
harness_commit: 7f2ac91
runtime_tree: upstream             # same vocabulary as config.runtime.tree

• Patch = comparable · minor = comparable with a note · major = break the series. Stated in comparable_with, not inferred from the number.
• The grader is a dependency. If time-to-correct is judged by an LLM, changing that model silently re-baselines all history. A grader change is a major bump even if test content is identical.
• Frozen inputs must actually freeze. Anything touching live data (rail, weather, calendar) drifts the answer key. Fixture it, or mark non-deterministic and exclude from cross-date comparison.

2.9 decision — the browsable log

id: dec-0031
date: 2026-06-02
title: Promote Qwen3.6-27B Q8_0 + MTP to reflex tier
affects: [qwen36-27b-mtp, qwen36-35b-a3b]
supersedes: dec-0019               # makes reversals visible
evidence: [run-0142, run-0139]
alternatives_rejected:
  - node: glm-47-flash
    reason: MIT and best τ², but Ollama path unusable; deferred not rejected
outcome_review: 2026-09-02
outcome: null                      # filled later — did this hold up?

outcome is the field that makes this rare. Almost nobody publishes whether a decision survived three months of use.

2.10 claim — provenance as a primitive

id: clm-0402
text: FastFlowLM prefill ~5.2× vs iGPU
provenance: vendor                 # measured-here | vendor | community | inferred
evidence: [run-0142]
verified: 2026-07-28
volatility: medium                 # drives staleness alerts
sources:
  - url: https://...
    title: FastFlowLM docs
    accessed: 2026-07-28
    archived: https://web.archive.org/...
    excerpt: "..."                 # short, attributed
    excerpt_note: vendor claim, unverified on Strix Halo

Claims are embedded by id in prose and render as a badge; hovering shows evidence and age. The uncertainty register stops being an appendix and becomes a property of every sentence.

Two disciplines:

• Archive on capture. Vendor pages get quietly rewritten; half the value is proving what was claimed at the time.
• Short, attributed excerpts. This is a citation layer, not a mirror.

2.11 contribution — external work

id: con-0007
kind: pr                           # pr | fork | issue | patch | report
upstream: ggml-org/llama.cpp
url: https://github.com/.../pull/...
state: merged                      # open | merged | closed | carrying
opened: 2026-04-11
resolved: 2026-04-23
problem: clm-0388
affects_configs: [cfg-0087]
article: art-mtp-vulkan-fix

• state: carrying is a reproducibility hazard, not a badge of honour. A fork in use but not upstream must be visible on every config that depends on it.
• Closes the loop: finding → claim → contribution → merged → config re-verified on mainline → tree badge flips to upstream, with the whole arc dated.
• Site stat: “N configs currently depend on unmerged work” is a real reproducibility debt figure. Watching it drop is a good story about upstreaming rather than hoarding.

2.12 edge — coordination between nodes

id: edge-sentinel-escalation
from: lfm25-watcher
to: qwen36-27b-mtp
type: escalation
trigger: salience score > threshold
article: art-continuous-loop
state: planned                     # same lifecycle vocabulary as nodes

Edges carry the bespoke work — proxy and cache strategy, overnight reconciliation, tariff-aware tiering. Clicking a line on the topology is how a visitor reaches the most original material, which is otherwise the hardest thing to surface.

3. Site structure

3.1 The topology is the spine, not a feature

Every section is reachable through a node:

One structure maintained, not five.

3.2 Landing page

Device diagram as the hook — what is physically online, engine by engine, with live state. Then one click into the routing table: what each box is actually working on.

Headline strip: drawing now (W), standing cost (£/mo), duty cycle, models live.

Node states carry colour: active / resident-quiet / planned-dashed. The retired and rejected counter sits in plain sight.

3.3 Node record page

Order matters — “will this run on my box” is answered before the reader scrolls:

1. Title, state pill, days-in-production
2. Fit line: 64K × 2 slots · 38.5 GB of 96
3. Four headline stats: decode · time-to-correct · mWh/answer · pence/answer
4. Lifecycle timeline (what it displaced, and when)
5. Config fingerprint — monospace, unformatted, copy-pasteable, with the upstream/fork badge adjacent
6. In production (distributions, workload mix)
7. Claims on this node, with provenance badges — including unverified ones
8. Links: decision · suite definition · rejected candidates · contributions

3.4 Routing table

For each workload: which model is live, why, Wh and pence per task, what was rejected. Evidence-backed, drillable, machine-queryable.

The Claude Code axis adds something absent from the field — local vs cloud on identical work types, with cost attached to both. Everyone argues that abstractly.

3.5 Time travel

Records are dated, so the topology renders at any past date. Drag back to January and watch the Beast go dark and the Mac mini carry everything. Nearly free given the lifecycle records, and it turns a status board into a narrative device.

4. Machine-readable access

Decision: yes. llms.txt, JSON endpoints per record type, and an MCP server over the same data.

Rationale: the benchmark commons is degrading at exactly the moment coding agents became the main consumers of “which local model should I run.” Being the well-structured, provenance-tagged, queryable source makes this the default citation. Better position than competing for human search traffic.

Risk: numbers repeated without the config fingerprint — the exact failure this site criticises in others.

Mitigation is structural, not licence text:

• Every JSON record embeds its config id and provenance inline. A metric is never retrievable naked.
• retrieved_at and verified in every payload — staleness travels with the data.
• CC BY on data; attribution baked into the schema rather than requested in a footer.

MCP beats a static feed for queries like “what’s live on 96 GB unified memory right now, and what got rejected for that tier?” The rejected set is unique and absent from every model card in existence.

Public surface is merged-only. Warden’s staleness queue and draft decisions stay private. Working notes must not become quotable before they’re stood behind.

5. Warden’s role

5.1 Write boundary

Warden proposes decisions as drafts. A transition to live requires human merge.

Auto-generated prose would quietly poison the credibility being built. Warden owns state; the human owns argument.

5.2 Warden’s highest-value job is refusal and flagging

• Refuse to chart across a breaking: true suite boundary. Render a visible series break instead. This single rule prevents the ecosystem’s headline failure mode.
• Flag suite staleness when dependencies move — llama.cpp commit, grader model, harness commit — even if nobody edited the suite. A frozen test on a moving runtime isn’t frozen. This is the subtle one: the file looks unchanged in git while the thing it measures has shifted three releases underneath it.
• Raise staleness PRs on claims: “clm-0311 verified 87 days ago, volatility medium, llama.cpp has moved 6 releases. Re-run cfg-0087?” Per-claim verification dates, not per-page.
• Flag re-verification when a contribution merges — confirm the fork is no longer needed, flip tree to upstream.
• Flag OOM risk when total_gb / pool_gb > 0.8.
• Suppress under-n telemetry cells.

The staleness queue doubles as Warden’s own work queue.

5.3 Read-back verification — the silent no-op is the failure mode that matters

Warden’s entire premise is unattended writes. The specific way that premise fails is not a crash — it is an append that quietly does nothing and reports success. The site then goes stale while claiming to be current, and every freshness signal on it becomes a lie. That is strictly worse than a build failure, because a broken build is loud.

Three instances of this pattern occurred on 2026-08-03 alone (see inc-0003), all sharing one root: success inferred from the absence of an error rather than the presence of the intended effect. Fixtures reported pass because the build failed at all; scripted edits reported success while matching nothing.

Two controls, both required:

1. Warden’s write path must read back. After appending a record, re-read it and confirm the content is present and parses. A write is not complete when the call returns; it is complete when the read confirms it.
2. The site must publish its own freshness. Newest record date against expected cadence, rendered on the front page — implemented in src/lib/freshness.ts. A lab notebook that has quietly stopped taking notes should say so on its own front page rather than presenting stale data as current.

The second control exists because the first can itself fail silently. A freshness banner is checkable by a reader with no access to the write path at all.

6. Measurement programme

6.1 Prerequisite

Per-box smart plug. If both boxes plus networking sit behind one meter, every per-engine delta inherits noise from whatever else moved. Small purchase; determines whether the entire energy dataset is defensible.

6.2 Per-engine energy requires isolation

A wall meter gives one number for the whole box. Per-engine figures are only honestly obtained by running the workload with exactly one engine active, everything else at known idle, and taking the delta against that engine’s own quiet baseline.

Anything finer is inference and must be tagged method: attributed, not wall-meter.

Concurrent-residency runs get contention: true — a real-world number, but not a clean per-engine one, and carrying OOM risk on unified memory.

6.3 The headline experiment

Same silicon, same task, same meter. ⚑ Updated 2026-08-03: the contradiction is largely explained — the two claims measure different halves. An independent same-machine test (Sleeping Robots, Lemonade on Strix Halo) gives, at ~8–9B: NPU TTFT 2.22 s vs iGPU 5.12 s (NPU 2.3× on prefill) and iGPU decode 19.7 t/s vs NPU 8.2 t/s (iGPU 2.4× on decode). So “NPU is faster” and “GPU is faster” are both true, of prefill and decode respectively — which is exactly the tiering argument below. The same source ran them concurrently: iGPU decode −14.1%, NPU decode −5.8%, aggregate wall-clock 1.42× better — the shared-bandwidth tax quantified. What remains genuinely unmeasured is energy, so the table below is still the experiment worth running:

Expected outcome is not a single winner: NPU likely wins per-task energy on prefill-shaped classification, iGPU wins per-correct-answer wherever capability is required. That is the tiering argument in numbers rather than intuition.

6.4 Goliath arrival — run the overlap deliberately

Not just a re-baseline. Same suite, same config, same flags, both boxes:

id: run-0301
config: cfg-0087
host: goliath
cross_host_pair: run-0142

Isolates hardware as the only variable — the comparison people want when deciding whether a second box is worth it, and one almost never available because most people replace rather than overlap.

Same treatment on the Beast’s return: re-run its own historical suite versions to detect software-stack drift while it was offline.

7. Publication rules

1. Nothing publishes that hasn’t been cleared and decided. No draft state, no staleness queue, no working notes on the public surface.
2. Secrets scrub before commit one. Current working docs contain a Nabu Casa webhook URL, HA token references, an OAuth client ID and tailnet IPs. Redaction convention baked into the repo from the start, not retrofitted.
3. Every metric carries its config id. Never publish a naked number.
4. Provenance badge on every claim, including uncomfortable ones.
5. Publish the uncertainty register. Unusual, and it buys disproportionate credibility.
6. Negative results are first-class. Rejected models, failed approaches, and revisit-later verdicts get records.

8. Launch sequencing

Don’t launch all sections. Ship one artifact with the harness attached.

9. Open items

Decisions still to make:

1. Static site generator and hosting specifics — RESOLVED 2026-08-03 (§10)
2. Whether the routing table or the topology owns the landing page long-term (current position: topology hooks, routing table one click in)
3. Threshold values — minimum-n for telemetry cells, OOM-risk ratio
4. Licence split: CC BY on data, and what applies to prose and harness code

Build work, in dependency order:

1. Repo skeleton + redaction convention + secrets scrub — DONE 2026-08-03 (§10)
2. Schema definitions with validation rules
3. Per-box smart plug procured and installed
4. Energy ↔ telemetry timestamp join (the only non-trivial build)
5. Warden write path: append records, open staleness PRs, refuse cross-boundary charts
6. Renderer

Next session: write the schemas properly with validation rules. Warden needs them before it can append records rather than the work being written by hand.

10. Revision — 2026-08-03 (Fable, post-implementation of milestones 01/01b)

Written pre-implementation. This section reconciles it with what has since been built and measured. Inline ~~strikethrough~~ marks resolved open items; §6.3 carries an inline correction.

10.1 Resolved — §9 decision 1: stack and hosting

⚠️ Cloudflare Pages is no longer the default path. New projects route to the Workers flow, which has no “build output directory” field — the asset path comes from a required wrangler.jsonc in the repo root. Any guide written against Pages (including milestone 01 as originally drafted) is now stale.

§2.2’s schema-validation premise is proven, not assumed: removing a required field from a record fails the build with InvalidContentEntryDataError and exit code 1. That is the mechanism that will later stop Warden committing a malformed record, and it was verified on a deliberately broken two-field fixture before anything real was built on it.

10.2 Resolved — §9 build item 1, and §7 rule 2

gitleaks 8.30.1 with nine custom infra rules, an allowlist for legitimate config hex, a pre-commit hook, and a CI workflow. Deliberate-leak test rejected the commit; full ruleset validated (5/5 fire). Detail in milestone-01b-redaction-and-scan.md.

Two findings that change how §7 rule 2 should be read:

1. The scan caught real values in the redaction document itself — a live tailnet address and a live LAN address, written as illustrative examples. Review would not have caught them; the scanner did. Scan at the moment of copying, not at review.
2. Local and CI disagreed. Homebrew ships gitleaks 8.30.1; gitleaks-action@v2 pins 8.24.3, and entropy/stopword suppression differs between them. A document that passed the hook failed CI. Treat CI as authoritative — it is the only control that survives a fresh clone and cannot be bypassed with --no-verify. This matters directly for §5.1: an unattended Warden commit has no hook, only CI.

10.3 Naming — the doc and the infrastructure plans disagree

This document uses evo-x2, “Goliath” and “the Beast”; runbooks/aibeast-return-plan.md and runbooks/second-halo-integration-plan.md use aibeast and aiforge. Since node.id is explicitly “stable, never reused”, this needs settling before the first node record exists.

Recommendation: adopt aibeast / aihydra as node.id (they are the names used operationally and in every runbook), and keep evo-x2 as a hardware-model field rather than an identifier.

Renamed aiforge → aihydra, 2026-08-07 (operator decision). Done deliberately on the evening the box was unboxed, because this was the last free moment: node.id is “stable, never reused” and the host name enters config fingerprints, so a rename after the first run would mean rewriting records the schema treats as immutable. The name also appears in published rows, so it is semi-public. aiforge was a fine name in the wrong register — industrial where aibeast is mythic. aihydra keeps the register and the length, and says something true: many heads on one body is the council architecture (clm-0008), several specialist models sharing a single box.

10.4 Live examples now exist for three record types

• config.runtime.tree: fork + contribution.state: carrying — production runs a patched llama-server carrying our .ckpt sidecar fix for llama.cpp #25913. This is the design’s reproducibility hazard, live today, and the first config record should say so.
• contribution — real instances across all states: OpenClaw #98267 merged; #110571 closed (superseded by upstream work); #110097 open; llama.cpp #25913 open with a third-party PR (#26004) and our fork branch in independent community use.
• claim — FastFlowLM’s status changed while this doc was being written: it is now part of AMD (site states “The FastFlowLM team is now part of AMD”; repos still MIT and active). A textbook case for volatility and archive on capture.

10.5 Gap — there is no record type for incidents

PLAN.md §1 lists postmortems among the most valuable material (OOM panic and kernel deadlock, silent cloud fallback, decode collapse under memory pressure, a nine-day outage). Nothing in §2 can hold one: run is a benchmark, decision is a choice, claim is an assertion.

Proposed twelfth type, incident (schema per the operator, 2026-08-03):

id: inc-0003
date: 2026-08-03
title: Workers Builds queued but never executed
class: provider                 # provider | config | code | hardware | data
severity: blocked               # degraded | blocked | data-integrity
duration: {from: "13:52Z", to: "~15:00Z"}

detection:
  detected_by: manual-observation
  should_have_caught_it: null   # ← the sharp field
  detection_lag: 11m
  signature: three consecutive failures, correct config, no log output past line one

diagnosis:
  initial: repo access / clone failure
  actual: provider-side build fleet not picking up work
  false_path_cost: ~30m          # reconnecting the Git integration
  falsified_by: status page incident declared 14:20Z

blast_radius:
  runs_invalidated: []
  claims_falsified: []
  configs_affected: []

lesson: check the provider status page before re-doing setup

Why each field earns its place:

• should_have_caught_it is sharper than detected_by. That a smart plug found it is useful; which monitoring existed and stayed silent is the transferable finding. The field is a standing indictment of the project’s own observability, and watching it move from populated to null over time is a genuine maturity signal.
• blast_radius is the field nobody would think to add, and the strongest argument for the type existing at all. An incident during a benchmark window contaminates data. If the box thermally throttled or a competing process ran during run-0142, that run is suspect — and with no link, it silently poisons a comparison months later. This makes incidents a data-integrity mechanism rather than a war-stories section.
• class: provider must be first-class. Today produced two incidents, not one: the Cloudflare build-fleet outage (provider) and the local/CI gitleaks version divergence (config). Different classes, different lessons. A site publishing only self-inflicted failures reads as boastful or incomplete.
• false_path_cost with initial vs actual is the most useful part of any postmortem — what you thought it was, and what being wrong cost. That is what a reader recognises in their own situation. Nobody arrives already knowing the answer.

Structural requirement: incidents must be linkable from any record type, not only stand alone. A run carrying incidents: [inc-0007] makes contamination visible at the point of use rather than buried in a postmortem nobody reads. Warden can then refuse to chart a run with an unresolved data-integrity incident attached — the same refusal pattern as the breaking-suite rule in §5.2, and for the same reason.

That makes twelve record types. Milestone 02 still implements only the three-record spine (node, config, run); incident layers on cleanly afterwards, because its most valuable link points at run, which will already exist.

10.6 Corrections and cautions

• §1.3 “Build on push” is not yet true. Deploys are currently manual (npm run build && npx wrangler deploy). The Git integration is fully configured — Cloudflare resolves the commit hash, author and message — but builds hang, and a live Cloudflare incident (“Workers build failures”, 2026-08-03 14:20 UTC, Workers Builds degraded) explains it. Nothing to fix locally.
• §6.1 per-box smart plug — prerequisite partly met. A plug already exists on aibeast, and it proved its worth diagnostically before ever producing an energy figure: a steady 7–10 W with no inrush on power-cycle is what identified a dead box as not attempting to boot rather than failed hardware. Worth stating in the methodology: the meter is an availability instrument as well as an energy one.
• §6.4 “Goliath arrival” — the box is bought. Onboarding sequence, model placement and the parity gates are specified in runbooks/second-halo-integration-plan.md; the cross-host overlap run described here (cross_host_pair) is gate N2 in that plan and should reuse this doc’s run schema.
• §2.4 method: vendor-claim — independently confirmed 2026-08-03: no independent wattmeter measurement of XDNA2 NPU LLM inference exists. The “<2 W” figure remains a vendor claim repeated by secondary sources. The register in this document is correct and should not be softened.