中国Orca世界模型无需动作标签即可匹敌专业机器人系统

BAAI's Orca world model matches the performance of specialized systems across five robotics tasks, even though its base model was trained without a single action label. The approach could help solve robotics' chronic data shortage. The Beijing Academy of Artificial Intelligence (BAAI) has released Orca, a "world foundation model" that breaks from the dominant approach in today's AI. Instead of predicting the next token, the next video frame, or the next robot action, Orca models the next state of the world in an abstract internal representation. According to the technical report , intelligence shouldn't be defined by specialized prediction models like language models, video generators, or robot controllers. What's needed, the team argues, are models that build a general grasp of how the world changes and can use that as a base for many different tasks. Orca combines two learning modes. "Unconscious learning" uses raw videos without any captions. The model sees an image and predicts what the next one will look like, not at the pixel level, but in an abstract space, picking up motion patterns, occlusions, and typical scene dynamics along the way. "Conscious learning" adds verbal instructions. Videos are split into segments, each labeled with a description of the state change, so the model learns how a state shifts when a specific action happens. It also trains on classic video question-answering tasks so it can still respond to natural language queries. The pre-trained language-image model Qwen3.5 serves as the base, and after training, this core stays frozen. For each output type, the researchers attach a separate, smaller module that converts the internal state into the desired form. Text runs through Qwen3.5's existing language head. For images, Stable Diffusion 3.5 stays unchanged too, with only small upstream adapters trained to pass Orca's internal state to the image generator. Robot actions come from a control module trained from scratch called "Action Expert." The split is intentional because the team says that it isn't trying to chase top scores on single benchmarks. Instead, they want to show that a well-trained internal world state can serve as a shared base for very different tasks. For training, the researchers put together 125,000 hours of video footage, 160 million event descriptions, and 11.5 million question-answer pairs. The videos span four views, covering first-person shots of everyday interactions, third-person shots of object handling, robot recordings without action data, and naturally occurring scenes. Only one-tenth of the video data went into the current version. Orca was trained at two sizes, with 0.8 and 4 billion parameters. Training loss drops steadily with more data and larger models, and early tests also show that the better the internal world state gets during pre-training, the better results are across all three output modes. On text benchmarks, Orca-4B posts the best overall average of 51.8 percent across MVBench, TemporalBench, 3DSRBench, and SWITCH among the compared small VLMs and larger world models. It beats several VLM baselines like Qwen3.5-4B, Gemma 4-4B , and DeepSeek-VL2-3B on average, though it doesn't win every single test since Qwen3.5-4B leads on MVBench, for instance. Orca-4B also tops the much larger world models Emu3 (8B) and Emu3.5 (34B) on average. For image prediction, the researchers built their own benchmark called PRICE-V0.1, which uses real-world robot and first-person scenes and asks the model to generate an image showing the result of a command like "close the microwave." Orca-4B hits 59.8 percent on average, beating specialized image generators like FLUX.2 small (56.1 percent) , FLUX.1-context (40.9 percent) , and OmniGen2 (39.6 percent) . Orca keeps the robot's shape, its contact points with objects, and its link to the instruction intact better than pure image models, which often add irrelevant objects or hallucinated hands. In five manipulation tasks usi