clm-0031

Source: a single operator (Reddit, u/Neuromacmd) on an ASUS PX13 — Strix Halo / Radeon 8060S, gfx1151, 128 GB unified, Fedora 44, kernel 7.1.5, Mesa 26.1.5. Model `unsloth/DeepSeek-V4-Flash-0731-GGUF` UD-IQ3_XXS, 97.05 GiB. Same silicon as aihydra, so it transfers more directly than most community reports. ## 1. INDEPENDENT CONFIRMATION of the double-residency hazard Their words: *"`-mmp 0` is not optional at this size on my box: with mmap the page cache gets counted against GTT on top of the weights and it never loads."* That is precisely the defect that bit us four separate times before being fixed properly (protocol §0, `LLAMA_ARG_LOAD_MODE=none`). A second operator hitting it independently, on the same silicon, confirms it is a property of unified memory rather than anything specific to our setup — and that the fix belongs at the environment level, not per script. ## 2. A 120 GiB GTT ceiling works in production They run `ttm.pages_limit=31457280` (**120 GiB**) with a **512 MB** VRAM carve-out, and load a 97 GiB model to a 99 GiB footprint. We run 104 GiB GTT with a 1 GiB carve-out. So our 104 GiB was conservative — deliberately, as a guardrail against the aibeast unkillable-OOM class (clm-0022 notes the reasoning). This is evidence the headroom above it is usable if a model ever needs it. It does not change the guardrail argument; it removes the worry that 120 GiB is unstable. ## 3. ⚑ VULKAN AHEAD OF ROCm ON 3 OF 4 — which contradicts our working assumption | test | Vulkan | ROCm | |---|---|---| | shallow pp2048 | **124.16** | 114.37 | | shallow tg64 | **18.15** | 13.27 | | d24576 pp2048 | 63.67 | **74.31** | | d24576 tg64 | **14.36** | 9.25 | **Decode at depth: Vulkan 14.36 against ROCm 9.25 — 55% faster.** We have been treating HIP as the default on the strength of two garbage-output reports (clm-0014) and clm-0021's finding that the FA prefill cliff is Vulkan-only. This says the picture is more mixed. ⚠ BUT IT IS NOT A CLEAN COMPARISON, and the author does not claim it is. The Vulkan side is revision `4a1fb6c` (build 867) carrying **three of their own unsubmitted patches**; the ROCm side is `cd0fa60` (build 824). **Different commits, different patch sets.** That is exactly the confound clm-0017's author avoided with five same-base builds and that we avoided by cherry-picking a single commit for clm-0022. Their Vulkan advantage may be the backend, their patches, or the 43-build gap. ⚠ CONFIDENCE LOWERED TO LOW ON THIS BASIS: a 43-build gap between the two revisions tested, compounded by three of the author's own unsubmitted patches present on the Vulkan side only, is a structural confound severe enough that the Vulkan-ahead finding cannot be attributed to the backend until it is reproduced here on matched commits. ACTION: this is worth testing ourselves and we are unusually well placed — we have both backends, a controlled harness, and the discipline to build both from one base. **A clean Vulkan-vs-HIP comparison on identical commits is now the highest-value untested backend question**, and it was already on the coverage matrix as not-started. ## 4. A failure mode we were not watching for *"Slow enough at depth that the 10s compute-ring watchdog fires and the driver kills the context."* With dmesg signature: amdgpu: ring comp_1.1.1 timeout, signaled seq=7906, emitted seq=7907 amdgpu: Starting comp_1.1.1 ring reset amdgpu: [drm] device wedged, but no recovery needed **We have zero occurrences of this on aihydra** (checked). But it is a plausible hazard for exactly the work we do — a single op slow enough at depth trips a 10-second GPU watchdog and the context dies. It would present as an inexplicable mid-run failure, and we would not currently know to look for it. Added to what our watchers check. Root cause in their case (traced by khimaros, llama.cpp #25664): with no Vulkan `LIGHTNING_INDEXER`, `resolve_fused_ops` disables `fused_lid`, DeepSeek-V4 takes a fallback path with a dim0/dim2 permute plus `CONT` that misses the tiled-transpose path and lands in a generic strided copy with consecutive elements 4 MB apart. ## 5. Worth noting on provenance The author states plainly: *"Claude wrote the shader and the ggml-vulkan.cpp integration under my direction. I ran the testing and the benchmarks. I can't defend the shader line by line to a reviewer or maintain it, which is why there's no PR."* That is an honest disclosure and the right call — and it mirrors our own constraint that upstream contributions must be defensible by the person submitting them. It also means the patches are unlikely to land upstream, so any advantage they confer is not something to plan around.