Reasoning as Pattern Matching: Shared Mechanisms in... | AI Research

Reasoning as Pattern Matching: Shared Mechanisms in Human and LLM Everyday Reasoning
This research investigates whether human reasoning and Large Language Model (LLM) performance rely on abstract "world models"—structured, logical frameworks for understanding reality—or if both systems are actually performing a sophisticated form of pattern matching. By comparing how humans and 25 different LLMs handle everyday causal scenarios, the authors explore why both systems often succeed at complex tasks but fail on simple, structurally identical problems when minor, seemingly irrelevant details are changed.

Testing Reasoning Through Everyday Scenarios

To compare human and machine cognition, the researchers designed a series of tasks involving simple causal relations, such as predicting the outcome of an action (e.g., what happens when a glass is bumped) or identifying the cause of an observed event. They specifically included scenarios that required inhibiting common associations to see if participants and models could move beyond simple word-co-occurrence. The study found that both humans and LLMs exhibit "graded generalization," where performance is highly sensitive to the specific content of a prompt. Small, surface-level modifications to a scenario often caused both humans and models to shift from correct reasoning to unpredictable errors.

The Pattern-Matching Mechanism

The authors argue that the observed errors suggest that neither humans nor LLMs are consistently using abstract, rule-based world models. Instead, they propose that reasoning is a process of relating inputs to learned patterns—a view where cognition is built from sequences of responses to cues. To test this, the researchers performed interpretability experiments on LLMs to isolate the specific "attention heads" (internal circuits) driving their responses. They discovered that these internal mechanisms are highly sensitive to specific content rather than abstract structure. Crucially, the activity of these pattern-matching circuits in the models was able to predict the specific, "inexplicable" errors made by human participants.

Why Human and Machine Errors Align

The study reveals a striking similarity between human and LLM performance. While larger, more advanced models often achieve higher overall accuracy, they are not necessarily more "human-like" in their reasoning patterns. Instead, models that align most closely with human behavior show the same specific strengths and weaknesses, such as finding geocentric relations (like cardinal directions) more difficult in the context of nearby objects compared to distant ones. Furthermore, the researchers found that human errors were not merely random guesses; participants were consistently likely to repeat the same mistakes when presented with the same prompts again, mirroring the brittle performance profiles observed in the language models.

Implications for Understanding Cognition

The findings suggest that the "fail states" often used to argue that LLMs are not truly reasoning may actually be a fundamental feature of how humans reason as well. Rather than relying on a perfectly consistent, abstract world model, both humans and LLMs appear to rely on contextual, content-laden configurations. This implies that what we perceive as "reasoning" may be more deeply rooted in pattern matching than previously assumed, with both biological and artificial systems settling on responses based on the specific cues provided in their environment.