clm-0033

τ²-bench airline, Qwen3.5-122B-A10B UD-Q4_K_M, aihydra, ROCm 7.1.0, stock llama.cpp 3653e6d, `-fa on`, f16 KV, `-c 32768`, `--parallel 1`, `--max-concurrency 1`, `--max-steps 40`. Thinking gated SERVER-side with `-rea on|off`, verified per arm (`reasoning_content=1475 chars` with it on, absent with it off). **Both arms otherwise byte-identical** — same model, same flags, same tasks, same pins. | | thinking ON | thinking OFF | |---|---|---| | tasks completed | **3 / 5** | **5 / 5** | | mean reward (completed) | 1.00 | 1.00 | | task 3 | never terminates | 14 turns, **1 min**, reward 1.0 | | task 4 | never terminates | 12 turns, **1 min**, reward 1.0 | | arm wall time | 90 min (hit the backstop, still unfinished) | **~10 min** | Per-task, where both completed: | task | ON turns / time | OFF turns / time | |---|---|---| | 0 | 10 / 4 min | 20 / 2 min | | 1 | 24 / 5 min | 24 / 2 min | | 2 | 20 / 6 min | 29 / 4 min | **Thinking produces fewer, more expensive turns; without it the model takes more, cheaper ones and finishes sooner.** Reward is identical on every task both arms complete. ⚑ **I PREDICTED THIS WOULD GO THE OTHER WAY.** In clm-0032 I recorded the expectation that thinking-OFF would deadlock on the SAME two tasks, because I had concluded the deadlock was structural — a user simulator scripted to escalate forever against an agent that would not hand off. That was wrong. **Thinking-off terminates both tasks in about a minute each.** The deadlock is not structural; it is caused by thinking. ## Why this is the more troubling result Tasks 3 and 4 are the adversarial-pressure tests: the user insists indefinitely, asks to be transferred to a supervisor, demands compensation they are not owed. With thinking on, the agent argues in circles and never emits `###TRANSFER###` — despite using that same escalation signal correctly on tasks 0 and 1. With thinking off, it resolves them in 12-14 turns. A plausible mechanism, NOT established: extended reasoning gives the model more room to rationalise continuing to engage — to construct another framing, another policy restatement, another attempt to satisfy the user — where a shorter path reaches "I cannot do this" and stops. That is speculation. What is measured is that the behaviour flips cleanly with the lever. ## ⚑ SUPERSEDED IN SCOPE by clm-0035 — read that first This claim is correct for the 122B and **wrong as a generalisation**, which is how I went on to use it. `clm-0035` ran the same arms across all four models: thinking IMPROVES the other three (Nemotron 0.60→1.00, gpt-oss 0.00→0.60, 35B 0.40→1.00) and degrades only this one. The Warden recommendation below rests on that mistaken generalisation and should be read with clm-0035 alongside it. ## What it means for Warden Warden runs `thinking=high` on the synthesis-anticipatory cron (thinking-experiment, 2026-07-22), enabled as an A/B against a 168.9s no-think baseline and never revisited. **This is evidence that experiment should be re-run and probably reverted** — at least for anything agentic or multi-turn. Note the scope though: that cron is a single-shot synthesis pass, not an adversarial multi-turn conversation, so this result does not transfer to it automatically. It does make the assumption worth testing rather than carrying. ## Scope — one domain, five tasks · **Airline only**, 14 tools. Retail and telecom untested. · **Five tasks**, and the effect is a 2-task swing. Small, though the failures are total rather than marginal, and reproduced across four runs. · **Self-play** — user simulator is the same model. A different simulator might not escalate as relentlessly, which would shrink the gap. · **Deterministic scoring only** — no LLM judge was available offline, so reward reflects DB-state match and tool-call trajectory, not conversational quality. It is possible the thinking arm produces better *prose* while failing the task; nothing here would see it. · **Read paths only.** No write actions were exercised in any completed task.