Thinking Inertia: Why LLMs Keep Reasoning When Told Not To

Short answer: telling a language model to stop thinking does not reliably make it stop. A paper accepted at NeurIPS 2026, "Thinking Inertia: LLMs Keep Thinking When Told Not To" (arXiv 2610.11765), measured what models actually print before their final answer. Even with a vendor's own thinking switch turned off, DeepSeek-V4-Flash (284B) still showed visible reasoning in 99.9% of open-ended responses. The authors include Dianqiao Lei, Kevin Qinghong Lin, Pan Lu, Philip Torr and James Zou, and they released their code.

If you pay per token or care about latency, this matters. Most teams treat "thinking off" or "answer directly" as a cost control. The paper says that assumption needs to be tested, not trusted.

TL;DR: what the paper found

What is "no-thinking," and why is it hard to define?

Reasoning models now ship with explicit controls: a thinking toggle, a reasoning-effort setting, or a prompt such as "answer directly." We have covered those controls from the user side, for example in our guide to the Claude effort parameter and in model versus effort in Claude Code. What nobody had pinned down is what "no thinking" should look like in the output.

The authors point out that earlier work relies on three surface proxies:

1. Prompt-side control. The model was run in a no-think mode or just not asked to think step by step.
2. Visible-format control. No <think> block or separate reasoning trace appears.
3. Length control. The output is short.

Each proxy has an easy failure case. A disabled thinking mode can still produce explanatory text in the normal answer channel. Dropping the <think> tags removes a format, not the reasoning. A short answer can still contain the one equation that does the work. The paper's claim is that these proxies confuse how you asked with what you got.

How the authors measure it

The paper splits every response into a pre-answer trace (all readable text before the final answer) and a final answer. It then scores the trace at three levels, each stricter than the last.

The distinction between levels is useful. A lead-in like "Let me think about this. The answer should be B" is not reasoning, but it breaks ETR. A single compact equation is reasoning, but it is short. The three metrics separate answer-only output, relevant but non-inferential text, and explicit inference.

The paper reports human validation: five annotators produced explicit-inference labels that matched the primary judge with 98.8% mean binary agreement, and results held up under an independent judge from a different model family. The authors also stress that this is a response-level observation. It makes no claim about hidden computation inside the model.

The headline results

The team tested six prompting interventions on six widely used models, both open and commercial, across Boolean, multiple-choice and open-ended questions. Benchmarks in the public code include BoolQ, StrategyQA, MMLU, MMLU-Pro, GSM8K and MATH.

Stepping stones across an answer space, showing how visible reasoning in LLM no-thinking mode grows from Boolean to open-ended questions

1. Switches are not guarantees

Even DeepSeek-V4-Flash, a 284B model, kept visible explicit inference in 99.9% of open-ended responses with thinking disabled. We covered that model family's pricing and agent behavior in our DeepSeek V4 Pro benchmarks post. The point is not that one vendor is at fault. It is that the switch and the visible behavior are different things.

2. The answer-space staircase

Visible inference follows the shape of the question. It is lowest on Boolean tasks, middle on multiple choice and highest on open-ended ones. Stricter controls lower the level inside each answer space but do not remove the staircase. That is the "inertia": the task's structure pulls the model toward working things out in text, and an instruction only pushes back so far.

3. The no-thinking versus performance trade-off

On open-ended tasks, stronger suppression has a price. Across the five models with thinking fully disabled, strict answer-only prompting lifted mean ETR from near zero to roughly 40%, and mean accuracy fell by about 15 percentage points. On Boolean and multiple-choice tasks accuracy stayed relatively stable.

4. Answer space beats difficulty

The authors rewrote the same questions into different formats. Fact-like and candidate-visible formats made answer-only replies much easier than open generation. So the effect is not just that open questions are harder or need longer answers. How the answer is requested changes how much the model writes first.

Why this matters for builders

Reasoning tokens are billed and they take time. We have written about how price per token can mislead and why tokens per second is only part of the speed story. This paper adds a third wrinkle: the number of tokens a model spends before answering may not drop when you flip a setting.

Practical implications, framed as our reading of the paper rather than the authors' recommendations:

• Measure, do not assume. If you turned off thinking to save cost, log the pre-answer token count. A "no-think" label may hide hundreds of tokens of visible work.
• Prefer closed answer spaces for cheap calls. Classification, routing and yes/no checks should be framed as Boolean or multiple choice. The paper found those formats were closest to answer-only.
• Expect an accuracy trade on open-ended prompts. If you force terse output for free-form work, test accuracy on your own data first.
• Do not equate "short" with "no reasoning." A one-line output can still carry the key derivation, which matters when you monitor or audit model behavior.
• Parsing needs care. If you rely on strict answer-only output for downstream parsing, the paper's strict check also fails responses that add substantive text after the answer.

For prompting basics, our zero-shot, few-shot and chain-of-thought guide covers the other direction, eliciting reasoning on purpose.

How it connects to other work on test-time compute

The paper sits in a growing line of work on spending reasoning effort adaptively. Our posts on recurrent-depth reasoning and reasoning-effort pricing claims for GPT-6 Astra show the other side of the market: vendors sell more thinking as a premium feature. Thinking Inertia asks the mirror question, whether you can buy less of it and verify you got less.

How to try it yourself

The repository at github.com/thinking-inertia/code (MIT license, 3 stars when we checked, so very early) provides the evaluator. According to its README it needs Python 3.10 or newer and talks to any OpenAI-compatible endpoint, so you can point it at a local server or a hosted API. No provider credentials are stored in the repo.

git clone https://github.com/thinking-inertia/code.git
cd code
python -m venv .venv
source .venv/bin/activate
python -m pip install -e .

python -m thinking_inertia.download_datasets --root-dir data --overwrite

python -m thinking_inertia.eval_thinking_spectrum \
  --root-dir data \
  --datasets boolq strategyqa mmlu mmlu_pro gsm8k math \
  --base-url http://127.0.0.1:8401/v1 \
  --model qwen3-4b \
  --api chat \
  --modes 1 2 3 4 5 6 \
  --max-samples 200 \
  --run-name qwen3_4b_spectrum_demo

The six modes span native think-off (called M2 in the README), reasoning re-elicitation, semantic suppression and structural prefix forcing, with M5 being the strict answer-only prompt. The README notes that upstream dataset licenses still apply to the normalized inputs under data/processed/.

Limits and open questions

• Behavior, not cognition. The authors explicitly limit claims to visible text. Latent computation inside the model is out of scope.
• Judge dependence. EIR uses an LLM judge (GPT-5.5). The paper reports high agreement with humans and with a second judge family, but any judge-based metric inherits some judge bias.
• Six models. The paper evaluates six LLMs. Newer releases and other vendors' controls may behave differently, and we have not independently re-run the experiments.
• Public benchmarks. Results come from standard QA and math sets. Your production prompts may sit elsewhere on the staircase.
• Early release. The code appeared in October 2026 and has little community testing yet.

What to do next

If you run reasoning models in production, pick one high-volume call, run it with thinking off, and count pre-answer tokens. If the count is not near zero, either restructure the prompt toward a closed answer space or budget for the tokens you are actually getting. If you operate agents where visible reasoning is audited, remember that agent behavior can change in ways a setting will not tell you. AgentBeam, from the explainx.ai team, watches what agents actually do before they take dangerous actions, which is the same measure-the-behavior principle applied to security.

Details reflect the arXiv v1 paper (submitted October 8, 2026) and the repository README as of October 13, 2026.

Related reading

• Claude effort parameter: model selection guide
• Model vs effort in Claude Code
• Zero-shot, few-shot and chain-of-thought prompting
• DeepSeek V4 Pro benchmarks and pricing
• Why price per token misleads
• What is tokens per second
• Recurrent-depth reasoning explained
• GPT-6 Astra xhigh quota claim