Storage tiers and accelerator lanes

Two changes landing at once: a NAS arriving 2026-08-11, and the confirmation that aihydra’s NPU is live. Both alter what the bench queue can do, and both have a constraint that is easy to discover the expensive way.

The NPU is available now — and it is a second LANE, not a second setting

Confirmed on aihydra 2026-08-10:

c6:00.1  Signal processing controller [1180]: AMD Strix Halo Neural Processing Unit [1022:17f0]
/dev/accel0          present
amdxdna              loaded (172032, in use)

No driver work required. This is better than aibeast’s position, where the NPU was never tested and had thrown SMU errors on a wedged boot.

FastFlowLM pulls its own manifests. flm pull <model> fetches tuned builds with their own quantisation, context window and tokenizer — not our GGUFs. So an NPU entry is a different artifact, at a different quant, on a different runtime. That is why the candidate schema now refuses metrics_inherit_from on any lane: npu record, and refuses an NPU record that claims runtime: llama.cpp. Nothing measured on one lane describes the other, and the build enforces it rather than trusting the convention.

FLM’s catalog covers Qwen3.6 (Vision/MoE), Gemma 4 (omni), Qwen3.5 (Vision), Qwen 3, GPT-OSS (MoE), LFM2, EmbeddingGemma, Whisper, MedGemma, DeepSeek-R1 and LLaMA 3.x.

Sequencing: queued behind GPU work, never concurrent

The NPU shares the LPDDR5X pool and its bandwidth with the GPU. A concurrent NPU run would not merely be slow — it would perturb whatever GPU number was being measured beside it, silently corrupting a series rather than obviously breaking it. NPU work therefore runs only when the GPU queue is idle.

⚠ The standing cost nobody has priced

amd_iommu=off is worth +5–12% on the GPU and disables the NPU entirely. They are mutually exclusive. So the second lane is not free even when idle: keeping it costs single- digit GPU percent forever. That trade should be decided on measurements from both sides, which is an argument for getting NPU numbers rather than for assuming either answer.

⚑ What the lane is actually FOR — and why the first list missed it

The standing design intent (memory, 2026-08-02) is a continuous triage loop: the NPU watches inputs without interruption and decides what to escalate to the big model for analysis or action. That is a second lane past the single-slot bottleneck — heartbeat and triage — not a second place to run benchmarks.

My first four candidates did not serve that. They were a bring-up list — does the lane work at all — with use-case rationales retrofitted onto them afterwards. One of those rationales was simply wrong: I claimed EmbeddingGemma’s qmd re-embedding competes for Warden’s GPU slot. It does not — embedding runs on the Mac mini and was never contending for the inference box at all. The candidate is still worth testing; the reason I gave for it was invented rather than checked.

The distinction that matters: bring-up asks can the NPU run a model, and the triage loop asks can a small model decide well, continuously, for hours. Only the second is the product, and nothing in the original list measured it.

Revised list, ordered by what the loop needs

The measurement the loop needs does not exist yet

Sustained triage is not scored by τ²-bench, and that is the same hole clm-0034 opened when a 12B orchestrator turned out to route competently: we have no benchmark for routing. The escalation loop and the routing gap are the same missing instrument, so building it once serves both.

What it has to measure, none of which τ² reports:

1. Escalation precision and recall. A model that escalates everything provides no filtering; one that escalates nothing is dangerous. Both failure modes score identically badly and mean opposite things, so they must be reported separately — a single accuracy number would hide exactly the distinction the loop depends on.
2. Sustained residency. Hours of continuous operation, watching for thermal drift, memory growth, and NPU SMU errors of the class seen on aibeast’s wedged boot.
3. Non-interference. Whether a resident NPU loop measurably perturbs concurrent GPU work — the shared-bandwidth question, answered rather than assumed.
4. Per-decision latency, which is what makes a loop feel continuous or laggy.

Only item 1 needs a labelled corpus, and Warden’s own history is the obvious source: real inputs with known outcomes, where escalation either did or did not turn out to be warranted.

NAS: a library, not a working set

The endpoints are fast; the network gear is the bottleneck. aihydra carries an RTL8125 2.5GbE controller advertising 2500baseT/Full, and the NAS has 10GbE — but the router and switch are 1GbE, so eno1 negotiates down to 1000 Mb/s.

I originally recorded this as “aihydra’s link is 1 GbE”, which attributed the limit to the machine, and then recommended a USB 2.5GbE adapter — pointless, since the 2.5GbE is already built in and idle. The lever is the switch, not the hosts.

A 2.5GbE switch takes aihydra to its own ceiling and cuts staging roughly 2.5×. The NAS’s 10GbE cannot be reached by aihydra at all, so it only pays off for other clients or a future 10GbE host — worth knowing before sizing the switch.

Never benchmark from the NAS. Two reasons, and the second is the one that would bite quietly:

1. Load time is not incidental — cold boot is a metric we track (~8.7 min historically on the 122B). Serving weights over a 1 GbE link would fold ~11 minutes of network into a number we report as a property of the model.
2. Every run uses LLAMA_ARG_LOAD_MODE=none specifically to avoid mmap double-residency, which means the whole file is read at load. Over the network that read is the load, so NAS-hosted weights would inflate exactly the measurement the setting exists to protect.

So: NAS is the library, local NVMe is the working set. Stage → benchmark → evict.

What it unlocks — and it is the thing currently blocking us

Local disk holds 322 GB of 1.9 TB, so space is not yet the binding constraint — but it becomes one the moment we take quant equivalence seriously, because that means holding the SAME model at several quants at once:

• Nemotron-3-Super at Q4_K_M — it has only ever run at IQ4_XS, forced by an OOM on the 96 GB box, so every Nemotron-vs-peer comparison to date is confounded (clm-0036, candidates/nemotron3-super). This is the single highest-value item the NAS enables.
• 122B at Q5_K_M / Q6 — we have only ever seen this model at Q4.
• gpt-oss-120b at a higher quant — tests whether its blind-judging reversal was quant-limited.
• Qwen3.6-35B-A3B Q8 vs Q4 — the transferable number is the ratio.

It also satisfies G.1 of the aibeast return plan: archive the GGUFs and the chat templates we depend on. The templates are the fragile part — models get re-uploaded and re-quantised, and a template change is silent.

Worth checking on arrival

If the NAS has a 2.5 GbE or faster port, aihydra’s 1 GbE becomes the bottleneck rather than the disk. A USB 2.5 GbE adapter would roughly halve staging time — worth it only if staging turns out to be frequent, which the stage-and-evict pattern is designed to avoid. Measure before buying.