clm-0024

Qwen3.5-122B-A10B UD-Q4_K_M, aihydra, ROCm 7.1.0, stock llama.cpp 3653e6d, `--jinja`, `--parallel 1`, real Home Assistant MCP schemas (78 tools, 353 KB, cycled to reach higher counts). Fresh server, `-c 131072`: | tools | request bytes | result | |---|---|---| | 60 | 226,121 | HTTP 200 | | 80 | 304,532 | HTTP 200 | | 100 | 367,660 | HTTP 200 | | 120 | 464,680 | HTTP 200 | | 160 | 593,137 | HTTP 200 | | **200** | **752,383** | **HTTP 200** | No failure was found at any count tested. WHAT THE OLD CEILING ACTUALLY WAS. Tonight's first probe reported "30 tools" — that was the server at `-c 16384` returning `exceed_context_size_error`, pure context exhaustion. A second probe at `-c 131072` reported 59/60, and a fresh server then accepted 60 AND 61. Only the third run — fresh server, ladder rather than binary search — established that nothing fails up to 200. The consistent story across all three is **context**: 200 tools is ~94k tokens of schema, which fits in 131k and cannot fit in 16k. So the mental model changes. It was "llama.cpp cannot build a grammar above ~55-59 tools by schema SIZE". It should be "tool schemas are prompt tokens, and you need context to hold them". That is a far more tractable constraint — it trades against context budget rather than being a hard wall. ⚠ CONFIDENCE IS medium, NOT high, AND VOLATILITY IS high. Three reasons to hold this loosely: · **One unexplained failure.** n=60 genuinely returned 400 during the binary search at `-c 131072`, then passed twice on fresh servers. Something is state-dependent and we have not characterised it — the same flavour of instability as clm-0019. · **We only proved acceptance, not correctness.** HTTP 200 means the grammar compiled and inference started. It does NOT mean the model selects correctly among 200 tools. Tool-selection accuracy at high tool counts is a CAPABILITY question and completely untested here. · **Build-specific by construction.** The ceiling is a property of the build; ours is one day old. It may differ on the version production runs. WHAT THIS WOULD CHANGE IF IT HOLDS (do not act before re-verifying on the production build): the ~55-59 figure has directly shaped Warden's configuration — the Home Assistant `toolFilter` cut 42 tools to 9, apple-mail was curated, and the live count sits around 51 deliberately close to the limit. If the wall is really context, that curation buys tokens rather than avoiding a cliff, and the trade can be reasoned about instead of feared. The **silent cloud fallback** risk (HTTP 400 → OpenRouter, looking healthy) is the thing that made the ceiling frightening; a context error is at least loud. NEXT: re-run on the production llama.cpp build, and — more importantly — measure whether tool-SELECTION accuracy degrades as tool count rises. Acceptance was never the interesting question; it was just the one that used to fail first.