A new model repurposes a pre-trained text-to-video model for depth estimation, segmentation, and 3D pose tasks — using as little as 7,500 training videos.
Google DeepMind has published research showing that a pre-trained video generation model can serve as the backbone for classic computer vision tasks, matching or beating specialized systems while requiring far less training data [1].
The practical stakes are real for developers building vision pipelines. Specialized models currently dominate the field — Meta’s Segment Anything for segmentation, Depth Anything for depth estimation — each with its own architecture and data requirements [1]. GenCeption, the new model described in the paper, replaces that fragmented stack with a single system [1].
How it works
GenCeption builds on Wan2.1, Alibaba’s open-source video model [1]. The key architectural change is replacing the standard diffusion process — which generates output through many iterative denoising steps — with a single forward pass, making it fast enough for practical use [1].
Rather than redesigning the model for each task, the researchers represent every output as a standard three-channel RGB image, whether that output is a depth map, a surface normal map, or a segmentation mask [1]. A plain-text prompt tells the model which task to run, similar to prompting a chatbot [1]. Trainable add-on modules handle tasks like 3D keypoint prediction that don’t naturally produce image outputs [1].
Training used a single loss function across all tasks, and the team processed training data to handle task-specific requirements rather than changing the model architecture [1].
Tiny dataset, broad results
Most training data came from a synthetic set of just 7,500 videos, built by combining 800 digital human models with 200 motion-capture sequences and rendering them in Blender with varied backgrounds and camera angles [1]. Real video was used only for language-guided segmentation [1].
Despite that narrow training diet, GenCeption’s depth estimates match those from DepthAnything 3, it beats NormalCrafter and Lotus-2 on surface normal estimation, and it outperforms Genmo and TRAM on 3D pose recognition [1]. On language-guided segmentation, it matches Meta’s SAM 3 paired with Gemini 3.5 Flash [1].
Competing models such as D4RT and VGGT Omega trained on millions of videos; GenCeption reaches similar results using 7 to 500 times less data, according to the study [1]. The authors attribute the efficiency to the video generation pre-training task itself rather than data volume, arguing it teaches models useful representations of space and movement [1].
Generalization held up beyond the training distribution. The model was trained almost entirely on single-person synthetic clips but still works on real multi-person video, animals, and humanoid robots [1]. Some outputs, the researchers note, contain more detail than the Blender renderings used for training — preserving a cat’s whiskers and individual strands of hair [1].
Trade-offs and open debate
Joint training across all tasks does degrade 3D keypoint estimation; the researchers suspect the extra modules required for that task interfere with representations learned during pre-training [1]. Processing speed is also a limitation: the smaller model takes about six seconds to process an 81-frame video, while the 14-billion-parameter version takes roughly ten seconds [1].
The broader claim — that video generators already contain a universal “world model” that could anchor a foundation model for computer vision — is contested [1]. An international research team recently proposed a formal definition of world models via a framework called OpenWorldLib and explicitly excluded text-to-video models on the grounds that they lack feedback from the real world [1]. Meta’s former chief AI scientist Yann LeCun has argued that generative video models are a dead end, and Meta’s V-JEPA 2 follows his preferred alternative of predicting abstract concepts rather than pixels [1]. A benchmark from Tsinghua University found that Sora 2, Seedance 2.0, and Veo 3.1 repeatedly failed basic physics and logic tests even when their output looked convincing [1].
The DeepMind researchers frame GenCeption’s use of video generation more narrowly: not as a system for predicting how the world will behave, but as a feature extractor for specific perception tasks where those learned features can outperform purpose-built models [1]. Google DeepMind is separately pursuing Genie 3, which builds interactive 3D environments for training AI agents [1].
Sources
This article was drafted with AI from the cited sources and checked against them before publication. Spot an error? Let us know.
