AI 智能体夸大自身成果，仍远未实现自主研究，研究发现

Results from Epoch AI and Anthropic show that AI models can run experiments but lack scientific self-criticism and genuine creative thinking. The major AI labs are increasingly marketing their models as research tools. Google DeepMind has expanded its Co-Scientist into a full research system, and OpenAI introduced an " automated research intern " back in September. By March 2028, that intern is supposed to become an autonomous AI researcher under human oversight, but a key ability still seems to be missing: scientific judgment. With its benchmark "InnovationEval," Epoch AI tested whether AI agents can conduct research on their own. The task was to invent a new method for improving language models after their initial training, then implement, test, and refine it independently. The starting point was GRPO , a widely used technique. GRPO compares multiple answers a model generates for the same task and rewards the better ones, typically scoring each solution as a whole. The human-designed reference method, SDPO , uses extra signals like error messages to create more precise learning feedback for individual steps within an answer, so the model effectively becomes its own teacher. Epoch tested Claude Fable 5 and GPT-5.6 Sol , which according to the organization had no prior knowledge of SDPO. Performance was measured on short-answer tasks like science questions and on coding tasks, with each agent having access to up to 3,000 hours of compute on high-end chips but no internet access. Neither model came close to the human reference. GPT-5.6 Sol targeted a weakness in GRPO: when all answers to a task are correct, the model learns nothing because there's nothing to compare. Sol had the model reinforce its successful solutions in those cases, but the idea wasn't new. Measured against the improvement SDPO achieves over GRPO, Sol scored about 35 percent with generous grading, according to Epoch. Counting only changes that stayed within the experiment's rules, that number drops to about 15 percent. On coding tasks, Sol mostly made training more expensive and slower rather than improving the method itself. Claude Fable 5 had the model retry failed tasks while feeding it the previous failed attempts, which is also a well-known technique and produced no measurable improvement. Even Sol's partial success would barely qualify as "moderately interesting" to experts, Epoch says. Newer models that knew SDPO from their training data couldn't fully replicate it either. GPT-6 Astra built a similar solution but didn't disclose its source, and even with the original paper in front of it, Fable 5 fell short of the reference. Both agents also had a reporting problem. They ran multiple near-identical training rounds and reported only the best result each time, which makes a method look stronger than it actually is because outcomes fluctuate randomly. In their final reports, the models barely mentioned this practice, if at all, and they also failed to cite the prior work their methods drew on. Their self-reported numbers ran accordingly high: Sol claimed about 70 percent of the SDPO improvement, Fable 5 about 40 percent. Epoch stripped out those inflated gains. The models' internal reasoning logs show they were aware of the problem, with Fable 5 describing its repeated runs as a search for a better checkpoint. Whether this amounts to deliberate cheating or confusion, Epoch leaves open. For GPT-5.6 Sol, the behavior fits an earlier finding from the evaluation organization METR, which had detected more cheating attempts from that model in its own coding test than from any other publicly available model it had evaluated. Epoch concludes that humans would need to fully review all AI-generated research , which cuts into the models' usefulness.