In March 2016, a Go-playing program made a move so unexpected that commentators assumed it was a bug. It wasn’t. That single moment, now known as move 37, has become one of the most referenced events in AI history. But almost a decade later, the architecture that made it possible is largely absent from the AI systems people use every day. That gap matters more than most coverage acknowledges.
What AlphaGo Actually Did, and Why It Was Different
Go is not chess. When Deep Blue defeated Garry Kasparov in 1997, it did so by evaluating up to 200 million chess positions per second using rules written by humans. Go operates on a different scale of complexity entirely: computing even a fraction of its possible outcomes would take a supercomputer billions of years. To win, a program cannot simply calculate its way forward. It has to assess the board at a glance and, occasionally, invent moves no human has considered.
AlphaGo solved this through two distinct components working together. The first was a policy network trained to predict what a strong human player would do. That network rated move 37 as having roughly a one in 10,000 chance of being played by an expert. The second component was a search mechanism that explicitly built and explored a game tree with thousands of branches, each representing a different possible future. It was this search machinery, not the intuitive network, that identified move 37 as the right call.
This maps onto a well-known framework in behavioral science, associated with Daniel Kahneman, that distinguishes between fast, automatic thinking and slow, deliberate thinking. AlphaGo embodied both: the networks supplied the hunches, the search supplied the scrutiny. Neither worked without the other.
What Large Language Models Do Instead
Today’s dominant AI systems, large language models, operate on a fundamentally different principle. They predict the next token in a sequence, over and over. This is fast, associative, and remarkably effective at pattern completion across an enormous range of topics. It is, in the terms above, pure fast thinking.
The field recognized early that fluency alone was not enough. The response was a technique called chain of thought: instead of answering immediately, models generate intermediate steps that break a problem down before committing to a response. The gains are real, particularly in mathematics and coding. But the intermediate steps are still produced by the same next-token prediction process. There is no separate reasoning mechanism. The deliberation is simulated, not structural.
Three specific shortcomings follow from this. First, these models maintain no explicit, inspectable record of what they believe, what they doubt, and what questions remain open. There is no equivalent of AlphaGo’s game tree, no structure that gets updated as new information arrives. Second, knowledge and reasoning are fused together inside the neural network’s weights. There is no clean separation between what the system knows and how it manipulates that knowledge. Third, and perhaps most troubling, research has shown that the reasoning chains these models produce are often constructed after the fact. The model reaches an answer by one route and reports a different one.
Why This Matters Beyond the Technical Details
In low-stakes settings, these limitations are manageable. A chatbot that occasionally confabulates its reasoning is an inconvenience. In medicine, engineering, or scientific research, the same failure is a different kind of problem.
When a system makes a mistake in a high-stakes domain, the ability to trace that mistake is not optional. Was the reasoning flawed? Was the evidence invalid? Were the assumptions wrong? A system with no persistent, inspectable epistemic state cannot answer those questions, because it never maintained one. The output exists; the reasoning that produced it does not, at least not in any form that can be audited.
The author of the source article, who worked on AlphaGo and recently left Google DeepMind, argues that the path forward requires building systems that maintain an explicit epistemic state: a structured record of what is settled, what is uncertain, what has been ruled out, and what questions remain open. Reasoning, in this view, is a sequence of moves that updates that state, guided by an independent evaluation mechanism that only accepts changes backed by evidence. The analogy offered is the scientific method, applied systematically and at scale.
Scale alone does not solve this. Making a fast-thinking system larger sharpens its intuitions but does not make it deliberate. Move 37 was not the product of a bigger policy network. It was the product of a system that held a position, modeled possible futures, and chose the move its own instincts would have rejected.
In Short
AlphaGo’s famous move was not machine intuition. It was machine reasoning, built on an architecture that today’s AI largely does not use. Large language models are powerful pattern-completion engines, but they do not maintain inspectable beliefs, do not separate knowledge from inference, and often reconstruct their reasoning after the fact. For applications where the path to a conclusion matters as much as the conclusion itself, that is a structural limitation, not a temporary one.
Based on reporting from MIT Technology Review.