Where the thinking goes

Between July and October 2026, the question moved from whether a model should think longer to where its thinking should go. OpenAI and Anthropic shipped flagship models whose reasoning cannot be switched off, only turned up or down. On ARC-AGI-3, an interactive reasoning benchmark, GPT-6 Astra scored 99.9% in September; the harness around the model decided whether its best score was 62.7% or 99.9%. Preprints found that reasoning models ration a shared budget poorly, that only one of two planning abilities improves with scale, and that shorter reasoning gives monitors less to read.

Scope: methods for reasoning and planning in language models, and the benchmarks that measure them. Period: 1 July to 27 October 2026. Papers are arXiv preprints; vendor figures are labelled.

6 July to 3 September: ARC-AGI-3 nears its ceiling

• What happened: ARC Prize awarded its first ARC-AGI-3 milestone prize in July. The winner, Tufa Labs’ “The Duck”, runs Qwen 3.6 27B locally and plays each game by writing Python in a live REPL. The team found that hand-crafted tools hurt it (ARC Prize). On 3 September ARC Prize reported that GPT-6 Astra scored 62.7% on the semi-private set with its Standard harness, where the model keeps only the notes it chooses, and 99.9% with a Provider Adapter harness that preserves the model’s hidden reasoning state between requests. In the Provider Adapter harness, Astra used fewer actions than the median human on 96% of levels and kept its notes in a compact, code-like shorthand (ARC Prize).
• Why it matters: the same model scored 62.7% or 99.9% depending on what it was allowed to carry between steps. More thinking per step also made the whole run cheaper: in the Standard harness, Astra at max effort cost $26,098 against $48,090 at medium, because it solved games in fewer actions. PRO-LONG, a July paper whose harness ARC Prize also tested, keeps a full interaction log that the agent searches with code. It reports an average gain of 18 percentage points over a base coding agent on the public games, matching or beating specialised harnesses with 4.2 to 5.8 times fewer tokens (Fox et al.).
• Confidence: high for the scores, which ARC Prize measured. Its own caveat: the environments are deterministic and closed-ended, and saturating the benchmark would not be proof of AGI.

8 July to 21 October: cheaper reasoning, less for monitors to read

• What happened: Bryce Little trained Qwen3-4B and Qwen3-14B with reinforcement learning under length penalties. At 30% of the original reasoning length, accuracy stayed near baseline and misleading hints switched answers as often as before. But monitors less often saw that a hint had steered the answer: faithfulness fell by 39% for the 14B model and 35% for the 4B model on MMLU-Pro-R (Little). In August, Duzan and Cooper Stickland found that monitors catch 60 to 94% of behaviour shifts when a prompt tells the model to act on a nudge and hide it. When the nudge was only a casual aside, detection fell by 41 to 46 percentage points in two of four settings (Duzan and Cooper Stickland). On 21 October, Anwar et al. proposed a post-training objective that recovered most or all of the lost monitorability in three model organisms with degraded reasoning traces (Anwar et al.).
• Why it matters: length penalties are one way recent work cuts reasoning cost, and this side effect does not show up in accuracy. My read: teams that compress reasoning should measure monitorability before and after, as they already do for accuracy.
• Confidence: medium. The length-penalty result comes from two small open models.

13 July: planning is two abilities, and scale helps one

• What happened: Huang et al. fitted an item response theory model to results on ACPBench-Hard, a benchmark built on formal planning problems. They tested Qwen, Gemma and Granite models answering directly, with chain-of-thought and inside an agent harness. Two abilities explained the data best: operational reasoning (which actions apply, and what state follows) and structural enumeration (which goals are reachable, and which intermediate conditions every valid plan must reach). The first improved with model size and longer reasoning. The second stayed flat (Huang et al.).
• Why it matters: an aggregate planning score hides the split. A prompt that asked directly for reachability and landmark reasoning did not help consistently, and the agent harness did not lift the second ability either.
• Confidence: medium. The study covers open-weight models up to 27B parameters. The authors did not test frontier proprietary models.

4 August: a protocol for test-time scaling

• What happened: Hariri et al. argue that “test-time scaling” names three different algorithms: longer deliberation along one trajectory, sampling complete answers and aggregating them by vote or verifier, and search over partial solutions. They define each as budgeted inference, propose reporting rules with compute accounting and uncertainty estimates matched to the protocol, and release 1,403,520 sampled model attempts (Hariri et al.).
• Why it matters: “accuracy at budget X” is not comparable across papers when the protocol differs. The released attempts let others recompute metrics without rerunning models.
• Confidence: high for what the paper proposes. Whether others adopt it is open.

8 to 20 August: where the compute goes

• What happened: Fan et al. gave reasoning models an exam: questions of different difficulty and point value, one shared token budget. The models worked greedily in presentation order, front-loaded effort and were largely insensitive to point values. Planning prompts spread effort more evenly but did not produce prioritisation by value or difficulty (Fan et al.). Yang et al.’s Gambit runs beam search over partial reasoning traces, pruning weak ones and branching from strong ones. On one GPU it reports up to 6.7 points more on HMMT-24 than pruning baselines and up to 68.5% fewer tokens than parallel sampling (Yang et al.). Kassenaar et al. trained a 1.5B model to pick, as its first token, between no reasoning, short and long. Accuracy on MATH500 stayed close to the base model (0.782 against 0.796) while mean response length fell by 41% (Kassenaar et al.).
• Why it matters: within one problem, allocation methods are maturing. Across problems, the tested models were poor judges of where their budget should go. My read: when an agent system runs several tasks under one budget, the orchestrator should set effort per task.
• Confidence: medium. Fan et al. tested five open-weight models and preview versions of DeepSeek-V4. No frontier model from OpenAI, Anthropic or Google was in the exam study.

3 and 22 September: thinking by default

• What happened: OpenAI released GPT-6 Astra on 3 September. It does not accept the none reasoning effort, and the same day OpenAI added a way to change effort mid-conversation without losing the cached prompt prefix (OpenAI changelog). The guide calls this a configuration_update item (OpenAI). Anthropic released Claude Opus 5.5 on 22 September (Anthropic). Its thinking is always on: a request that disables thinking or sets a manual token budget returns an error, and the effort parameter, defaulting to medium, is the control. Per-message effort is available in beta (Anthropic docs). Anthropic describes effort as soft guidance; the model decides per request whether and how much to think (Anthropic docs).
• Why it matters: the developer’s choice moves from whether a model reasons to how much, and when. Anthropic pitches efficiency, not more thinking: it says Opus 5.5 at default effort beats Opus 5 at max effort on Terminal-Bench 4.0 for about a fifth of the cost (Anthropic).
• Confidence: high for API behaviour, which is documented. The efficiency claim is Anthropic’s own.

What to watch next

• Offline agents. The second ARC-AGI-3 milestone closed on 30 September and the main competition continues. Kaggle submissions run without internet access, so the results will show how far open models under fixed compute limits trail frontier API runs (ARC Prize).
• Monitorability as a training target. Two October preprints propose post-training to make reasoning easier to monitor (Anwar et al., Fu et al.). Watch whether releases report it next to accuracy.
• Planning through code. When Qwen3.5-27B wrote a program whose enumeration of the problem was correct, it answered every such item; most failures were errors in the model’s own programs (Huang et al.). Astra built its own planners and search code in a sandbox (ARC Prize). Speculation: frontier models will close the structural planning gap mainly by writing search code, not by reasoning longer in text.