In 2022, researchers at Google published a finding that seemed almost too simple to matter: if you ask a large language model to show its reasoning step by step before giving a final answer, it gets meaningfully better at multi-step problems: arithmetic word problems, logic puzzles, tasks that require chaining several inferences together. That technique, chain-of-thought prompting, turned out to be one of the more consequential discoveries in how we get useful output from language models, and it's worth understanding why it works, not just that it does.

The problem it addressed

A language model generates its response one token at a time, and (critically) each token it produces becomes part of the context it uses to generate the next one. When a model is asked to jump straight to a final answer on a hard multi-step problem, it has to do all the necessary reasoning "in its head," compressed into the limited computation that goes into producing a single token, with no scratch space. That's a genuinely harder computational problem than it sounds, and it's why models that could recite complex information fluently would still fail embarrassingly on a moderately complex word problem when asked to answer directly.

Why "showing your work" helps a model, not just a student

Chain-of-thought prompting changes this by having the model generate its intermediate reasoning as visible text before the final answer. Because each generated token becomes part of the model's own context for producing the next one, writing out "first, we need to find X" and then "given X, we can calculate Y" gives the model access to its own prior reasoning steps as it works toward the answer. Effectively giving it scratch space it didn't have when jumping straight to a conclusion. The model isn't reasoning "better" in some deep sense; it's using its own generated text as external working memory, which is a genuinely different computational process than generating an answer in one shot.

What the original research actually showed

The 2022 paper demonstrated that this technique produced large accuracy gains on math word problems and symbolic reasoning tasks, and (notably) that the technique's benefit scaled with model size: larger models benefited more from chain-of-thought prompting than smaller ones, which was itself an interesting finding about how reasoning capability emerges with scale. It's part of the broader story of capabilities that weren't explicitly trained for but emerged as a byproduct of scaling up the transformer architecture we cover in our explainer on the paper that introduced transformers.

From a prompting trick to a training target

What started as a prompting technique (something a user does at inference time) evolved into something labs began training for directly. Later-generation "reasoning models" are trained specifically to generate extended chains of reasoning before answering, rather than relying on a user's prompt to elicit that behavior. That shift, from "trick a general-purpose model into showing its work" to "train a model whose default behavior is to reason at length," is one of the more significant changes in how frontier labs build models for hard problems, and it's a large part of why current reasoning-focused models handle multi-step tasks so much more reliably than earlier general-purpose models did.

The practical takeaway

For anyone actually using these systems, the lesson generalizes cleanly: for genuinely hard, multi-step problems, prompting a model to reason before answering tends to help; for simple, single-step tasks, it mostly just adds cost and latency without meaningfully improving accuracy. Our guide to writing better AI prompts covers when and how to apply this in practice.