谷歌DeepMind认为视频生成器已包含计算机视觉缺失的世界模型

Researchers at Google Deepmind developed GenCeption, a model that repurposes a pre-trained video generator for classic computer vision tasks like depth estimation and segmentation. The system builds on an open-source video model from Alibaba and delivers results in a single forward pass, guided by text prompts. It trained on just a small set of synthetic videos and needs far less data than competing approaches. In benchmarks, GenCeption matches established specialized models and transfers its abilities to real-world footage and untrained categories like animals. The authors say this supports the contested idea that video generators can serve as the basis for universal world models in computer vision. A new model from Google Deepmind called GenCeption uses a pre-trained video generation model as the basis for classic computer vision tasks. It achieves state-of-the-art performance in depth estimation, segmentation, and 3D pose estimation while needing very little training data. Language models became versatile processing systems almost as a byproduct of learning to predict the next word. That task appears to require models to absorb grammar, world knowledge, and contextual relationships during training. This is the prevailing explanation for the emergent capabilities of large language models. Computer vision still lacks an equivalent training method. Specialized models dominate the field, including "Segment Anything" for segmentation and "Depth Anything" for depth estimation. Each uses its own architecture. Ad In a new paper , Deepmind researchers argue that large text-to-video models could bridge this gap. Generating realistic videos requires understanding the spatial geometry of a scene, how objects move, and basic physics. These models also use text descriptions during training, which links language to visual content. They can train on vast datasets because labeling the data is relatively cheap. Ad DEC_D_Incontent-1 GenCeption builds on Alibaba's open source Wan2.1 video model . The main change is a simpler architecture. Diffusion models typically generate video from noise through many small steps. GenCeption instead produces a prediction in one forward pass, making it fast enough for practical computer vision tasks. The researchers use a simple method to make one model handle many tasks. GenCeption represents every output as a standard three channel RGB image, whether the result is a depth map, surface normal map, or segmentation mask. It also converts camera movement into an image representation. Ad A text prompt tells the model which task to perform, much like an instruction given to a chatbot. The team adds trainable modules for tasks such as 3D keypoint prediction, which don't produce images. Training uses one loss function across all tasks. Rather than changing the architecture for each task, the researchers process the training data to account for task specific requirements. Ad DEC_D_Incontent-2 Most of the training data came from a synthetic dataset containing just 7,500 videos. The team combined 800 digital human models with 200 motion sequences from a motion capture dataset. It then rendered the results in Blender with different backgrounds and camera angles. Real videos were used only for language guided segmentation. Ad