Nvidia announced today that it is releasing a family of entry-level artificial intelligence models called Cosmos that can be used to train humanoids, industrial robots and autonomous cars. While language models learn how to generate text by learning from immense numbers of books, articles and social media posts, Cosmos is designed to generate images and 3D models of the physical world.
During a keynote presentation at the annual CES conference in Las Vegas, Nvidia CEO Jensen Huang showed examples of using Cosmos to simulate operations inside warehouses. Cosmos was trained on 20 million hours of real footage of “people walking, moving their hands and manipulating objects,” Jensen said. “It’s not about generating creative content, it’s about teaching AI to understand the physical world.”
Scientists and startups hope that these kinds of basic models can give robots used in factories and homes more sophisticated capabilities. For example, Space can generate realistic boxes with video footage of them falling off shelves in a warehouse, which can be used to train a robot to recognize accidents. Users can also fine-tune models using their own data.
Nvidia says many companies are already using Cosmos, including humanoid robot startups Agility and Figura AI, as well as autonomous car makers such as Uber, Waabi and Wayve.
Nvidia also announced software to assist various types of robots learn to perform modern tasks more efficiently. The modern feature is part of Nvidia’s existing Isaac robot simulation platform, which will enable robot designers to take a tiny number of examples of a desired task, such as grasping a specific object, and generate immense amounts of synthetic training data.
Nvidia hopes that Cosmos and Isaac will appeal to companies looking to build and exploit humanoid robots. Jensen was joined on stage at CES by life-size images of 14 different humanoid robots developed by companies including Tesla, Boston Dynamics, Agility and Figura.
Along with Cosmos, Nvidia also announced Project Digits, a $3,000 “personal AI supercomputer” that can run a immense language model with up to 200 billion parameters without the need for cloud services like AWS or Microsoft. It also announced the highly anticipated next-generation RTX Blackwell GPUs and upcoming software tools to assist create AI agents.