Anthropic’s Claude Opus 5 Shatters Reasoning Benchmark Record, But Gains Raise Questions About Breadth

Anthropic’s Claude Opus 5 has set a new record on the ARC-AGI-3 benchmark, scoring 30.2 percent and nearly quadrupling the previous best of 7.8 percent set by OpenAI’s GPT-5.6 Sol (Max). The model solved five previously unsolved interactive environments, four of them at or above human level, according to results published by ARC Prize, the organisation behind the benchmark.

The jump is being attributed to stronger logical reasoning, which the ARC Prize team says enabled “more autonomous exploration, planning, and execution across unfamiliar environments.” Opus 5 also displayed a behaviour researchers had not seen from any model before: it began translating tasks into algebraic notation and independently formulating reflection equations. On the older ARC-AGI-2 benchmark, Opus 5 scores 90.4 percent, and it reaches 97.5 percent on ARC-AGI-1—both numbers matching previous top scores at slightly higher compute costs.

How the benchmark works

ARC-AGI-3 is designed to measure a model’s ability to reason about new tasks it did not encounter during training. Unlike standard knowledge tests, it presents an interactive environment where the model must infer hidden rules, plan a sequence of actions, and execute them step by step. The benchmark is intended to mimic the kind of general reasoning that humans apply to unfamiliar situations. Six of the 25 public demo environments have now been solved. The full results, replays and benchmarking code are publicly available.

Narrower gains on a different puzzle set

Tests on the Witness benchmark, a private set of interactive puzzle games, tell a more nuanced story. Opus 5 scored 43.4 on Witness, statistically tying with Kimi K3 and Anthropic’s own Fable 5. The improvement over Opus 4.8 was far smaller than the leap seen on ARC-AGI-3.

Witness creator Guanghan Ning noted that Opus 5 could state the hidden rules before making its first move when the puzzle used common mechanics. But when a game combined rules in an unfamiliar way, the model scored below Opus 4.8. Ning said this pattern is consistent with training on genre-specific data, though the Witness benchmark cannot determine what data Anthropic actually used.

What the researchers say

Greg Kamradt, one of the ARC-AGI-3 researchers, said the results do not rule out broader reasoning gains. The conventional puzzles in Witness may not test novelty well because they use familiar mechanics, while a single weak performance on an unusual game could be an isolated regression. He pointed out that Opus 4.8 also outperformed Opus 5 on some ARC-AGI-3 tasks despite trailing by a wide margin overall. Since Opus 5 improved across Witness as a whole, judging how broad the gain is would require detailed results from more unfamiliar tasks.

Ning later clarified that Opus 5 did show broader gains on Witness, but they were far smaller than on ARC-AGI-3. He compared the pattern to the evolution of coding benchmarks: as a major target for interactive reasoning, ARC-AGI-3 will likely receive the most training effort first. As researchers add more edge cases, models could learn to handle a wider range of abstract reasoning tasks.

Why this matters for India’s AI ecosystem

For Indian AI developers, startups and researchers tracking progress toward general reasoning, the Opus 5 results are significant but require careful interpretation. The model’s ability to formulate equations and reflect on its own actions points to a new capability that could be useful for autonomous agents, robotics, and complex problem-solving—all areas where Indian tech firms are investing heavily.

At the same time, the narrower gains on Witness highlight a recurring challenge: benchmarks can be gamed when models are developed after the test format becomes public. Anthropic likely used targeted data labeling and reinforcement learning to improve on ARC-AGI-3 puzzles, but it remains unclear whether the same reasoning strategies transfer to genuinely novel problems.

The results also underscore the importance of diverse benchmark suites. Indian researchers and companies evaluating AI models for local applications—such as multilingual customer support, agricultural planning, or financial analysis—should consider testing across multiple reasoning benchmarks rather than relying on a single leaderboard.

Datos clave

Model	ARC-AGI-3 Score	ARC-AGI-2 Score
Claude Opus 5	2%	4%
GPT-5.6 Sol (Max)	8%	–
Fable 5 (Anthropic)	~20%	–

Source: The Decoder, “Anthropic’s Opus 5 blows past Fable 5 and GPT-5.6 Sol on the benchmark designed to measure real intelligence”, https://the-decoder.com/anthropics-opus-5-blows-past-fable-5-and-gpt-5-6-sol-on-the-benchmark-designed-to-measure-real-intelligence/. Full results, replays and code available at ARC Prize.