TheoPhysEval benchmark released: GPT-6-Astra in Codex solves 49 of 50 unpublished theoretical physics problems at about $116 total

A 35-researcher team (ICTS, Harvard, KU Leuven, Google DeepMind and others) released TheoPhysEval, 50 unpublished theoretical physics problems with verifiable answers across eleven subfields. Problems and test setup are downloadable on Hugging Face. GPT-6-Astra in OpenAI Codex scored 49 of 50 (98%) at under $3 per problem, about $116 in total. Opus 5.5 in Claude Code scored 92%, Gemini 3.8 Flash 68%, DeepSeek-V4.1-Flash 54% and GLM 5.3 42%. With web search off, Astra and Opus both scored 92%. Some AI misses turned out to be errors in the experts' own reference answers.

Physics groups now have a public, cheap-to-rerun test showing that top agentic models can finish hard, well-specified calculations for a few pounds each. That matters for how research time is budgeted and how students are trained on routine derivations. It also widens the gap between labs that can afford these tools and those that cannot. The impact is limited by scope: 50 problems with checkable answers say little about open-ended research.

The suggestion that models may already surpass the best human researchers is speculation. The problems are closer to demanding calculations than discoveries, and the authors themselves say missing research 'taste' explains the lack of breakthroughs. The cost headline is also flattering, since it ignores the human effort of setting up and checking the runs. The figures come via The Decoder with no direct links in the coverage, and at 50 problems a single question moves the score by 2 points. Reference-answer errors also make the top scores harder to interpret.

1. 01Independent reruns of TheoPhysEval from the Hugging Face release, checking whether 98% and about $116 hold, and how the reference-answer errors are corrected and rescored
2. 02Results from other labs and models, including a score for the same models on CritPt or similar benchmarks, to see whether the 92-98% top band is specific to this problem set
3. 03Evidence of AI-led results on genuinely open problems, such as accepted papers or verified new derivations, over the next six to twelve months, rather than further benchmark gains