趋势解读：Making AI chatbots helpful weakens their ability to，讨论数据集与基础模型

A large-scale study shows that the training process turning raw language models into helpful chatbots also weakens their ability to mimic human behavior. The effect gets worse with each new generation. Language models are increasingly used as stand-ins for human test subjects to predict reactions to policy measures , simulate clinical training for psychiatrists , or model how students learn. A new study from an international research consortium, including scientists from Helmholtz Munich, arrives at an inconvenient finding: the very training steps that turn language models into useful assistants make them worse at modeling human behavior. The study builds on Psych-201, a new dataset of transcripts from behavioral experiments. It covers about 208,000 participants and roughly 26 million individual responses from hundreds of experiments, several times larger than any previous collection of its kind. Each data point captures a participant's full run through an experiment, along with detailed metadata like age, nationality, questionnaire responses, and other traits. The dataset was assembled through an open research collaboration involving researchers from more than 35 institutions. The researchers compared models from the Qwen3 , Llama3 , and OLMo 3 families, testing both base models and their various post-trained variants. Base models are trained only to predict the next word in text. From there, extra training produces the versions tuned for instruction-following, step-by-step reasoning, or image processing. The metric: how well each model predicts the actual answers human participants gave. The result holds across all families and sizes. Base models predict human behavior better than their post-trained descendants. The effect shows up for every common training objective, hitting hardest with reasoning models, followed by instruction tuning and vision extensions. In nearly every head-to-head comparison, the base model outperforms its specialized variant. One obvious counter-explanation: maybe assistant models just answer more deterministically and fail to capture the natural spread of human behavior. The researchers tested this with an accuracy analysis on a subset of tasks with discrete answer options. Post-trained models still performed worse, making higher determinism unlikely as the sole explanation. While base models steadily improve from Qwen2 through Qwen2.5 to Qwen3, getting better at predicting human behavior with each generation, the gap to their derived assistant models keeps growing. Ongoing advances in post-training are making the divergence from human behavior worse. The biggest distortion shows up in language tasks and reasoning. The researchers offer a plausible explanation: base models are, at their core, models of human language and therefore well-calibrated for language processing tasks. Post-training techniques like reinforcement learning from human feedback push them away from that original objective toward more user-friendly or normatively correct answers. The same thing happens with reasoning. Human decisions are shaped by heuristics and systematic biases that base models apparently pick up. Reasoning training optimizes for logically correct answers instead, overwriting exactly the human quirks that matter for behavioral simulation.