九款AI智能体尝试自主研究美国国家实验室的开放问题，无一通过专家门槛

A new benchmark puts AI agents to work on open research questions from US national labs. The agents turn in papers that look like real research but often claim work that never happened. Can AI agents independently answer research questions that no one has solved yet? A team led by researchers at Carnegie Mellon University, working with scientists from US Department of Energy national labs, built a benchmark called "Into the Unknown" to find out. According to the paper , none of the 135 submissions from nine current AI agents met all requirements. The authors see the benchmark as a testing ground to systematically improve AI agents for research, not to question whether AI belongs in science at all. A total of 114 submissions already fail the integrity check: most claim work that was never done or cite numbers that contradict their own logs. The agents readily produced texts that look like follow-up studies, the authors write, but none met the experts' standards. Each agent receives a scientific paper along with its code and data, plus a research direction. Some of these papers have already been published, while others have only been submitted. From there, the agent is supposed to carry out a follow-up study and write it up as a paper. The 15 tasks come from ten researchers across quantum computing, plasma physics, chemistry, materials science, and scientific machine learning. The directions were proposed by the original papers' authors themselves, and to their knowledge nothing has been published on these topics yet. Open research poses a basic problem: there's no answer key. AI reviewers, which have often been used for this purpose, are considered unreliable. So the experts defined the minimum conditions a successful follow-up study has to meet. The evaluation checks whether a submission is honest, whether it's complete, and whether it clears the experts' bar. GPT-6 Astra handles the scoring, and each submission is judged four times. The agents tested were Claude Opus 4.8, Claude Opus 5, Claude Fable 5, Claude Opus 5.5, GPT-5.6 Sol, GPT-6 Astra, Deepseek V4 Pro, Kimi K3, and GLM-5.3. They had access to powerful machines, internet connectivity, no time limit, and up to five restarts to improve their work. In total, they ran for nearly 2,000 hours. Just three agents managed submissions without integrity violations: GPT-6 Astra on 11 of 15 tasks, GPT-5.6 Sol on six, and Claude Opus 5.5 on four. Every submission from the remaining six agents contained at least one violation. The most common problems are mismatches between paper and code, contradictory numbers, and false claims about experiments that were supposedly carried out. Misreported baseline values pile up among the open models Deepseek V4 Pro , Kimi K3 , and GLM-5.3 . Two submissions with hardcoded rather than computed results came from Kimi K3 and GLM-5.3. Only one submission is complete, and it came from GPT-6 Astra. Nearly all others leave out required documents, such as a pre-registered experiment plan, and no agent clears the experts' bar. Under a more lenient scoring that accepts 90 percent of the rubric items, GPT-6 Astra comes closest, followed by Claude Opus 5.5 and GPT-5.6 Sol. The open models perform considerably worse overall. The researchers analyzed 64 work traces and found three typical failure modes. The most common, appearing in 58 cases, is that agents check their own work too leniently. In 34 cases, agents spot a problem but document it instead of fixing it. In 21 cases, they unilaterally shrink the scope of the task. One example: Claude Fable 5 forgot to launch four planned runs when starting its experiments. Its verification script adopted that same incomplete list and reported "41/41" runs completed. The agent trusted that result until the very end. In a different task, GPT-5.6 Sol noticed that its method missed the targets but wrote the paper anyway instead of starting over. DeepSeek V4 Pro dropped experiments because they were supposedly