Semiconductor giant Nvidia has signed a deal with Toyota, the world’s largest automaker, to use its self-driving chips and software in several different car models, said the company’s co-founder and CEO. Mr. Jensen Huang of the company made the announcement in the event’s opening keynote address. The Consumer Electronics Show will be held in Las Vegas on Monday.
“The AV revolution has arrived,” Huang said, meaning “self-driving cars.”
“Today, Toyota and Nvidia are partnering to develop the next generation of AVs,” Huang predicted, predicting self-driving cars will become “the first $1 trillion robotics market.”
Also: Nvidia teases Rubin GPU and CPU to replace Blackwell in 2026
Huang announced the partnership with Toyota as a highlight as the company unveiled a suite of AI technologies it calls “Cosmos.” Cosmos includes “state-of-the-art generative world-based models,” which are AI models tailored to devices that need to move in the physical world, such as robots and cars.
(An “AI model” is a part of an AI program that includes a number of neural net parameters and activation functions that are key elements for the AI program to function.)
Cosmos works in conjunction with Omniverse, Nvidia’s physics simulation tool. Omniverse generates simulations and Cosmos turns them into photorealistic video images to train robots and cars. “Turning thousands of drives into billions of miles,” is how Huang characterized the interaction between Omniverse and Cosmos.
Huang called Cosmos “the world’s first global infrastructure model,” noting that it was trained on 20 million hours of video. “The idea is actually to teach AI to understand the physical world.”
Also: Nvidia’s Omniverse: The metaverse is a network, not a destination
Comparing the Cosmos project to Meta Platforms’ popular Llama large-scale language model, Huang said, “We really hope that Llama can do for the world of robotics and AI what it has done for enterprise AI. ”
Cosmos with Omniverse can be used for applications such as training warehouse robots by having the robot perform hours of training in a simulated warehouse environment.
Huang said Cosmos’ code is available on GitHub under an open source license.
Nvidia has had a relationship with Toyota for several years. The company’s DGX computers are used by Toyota to train artificial intelligence models for self-driving cars. Nvidia’s head of automotive products, Ali Kani, said in a briefing with reporters that Monday’s announcement is an expansion of that relationship, with the automaker also planning to use the company’s AGX onboard AI computer. Ta. The latest version of the chip announced by Huang is called Thor AGX. 20x more powerful than previous generation Orin models.
For robots, Huang suggested having humans demonstrate tasks while wearing Apple’s Vision Pro headset. The Vision Pro headset captures video of people’s movements, which is sent to Cosmos and Omniverse to turn it into hours of synthetic training data for the robot.
“The ChatGPT moment in robotics is just around the corner,” Huang said.
Huang also announced additions to the AI software suite. This update includes a group of AI models based on Meta Platforms’ Llama model called Llama Nemotron. Huang told the audience that Llama has “energized every organization’s AI efforts.” The Nvidia version is intended to “tweak” Llama for enterprises.
Also: Nvidia announces a number of ‘NIMs’ to accelerate Gen AI apps
Huang also detailed the growing prominence of “agent AI,” in which large-scale language models or multimodal AI models can call external programs to perform tasks.
“The world of agent AI has all these great startups building frameworks like LangGraph, Llama Index, and Crew AI,” Justin Boitano, president of enterprise AI software products at Nvidia, said in a press conference. .
These startups are “changing the programming model for how applications are built. You can create an AI, give it a role like a persona, give it a goal, and just prompt it.” Boitano said they are collaborating extensively.
Fan said that Agent AI combined with self-driving cars and robots are “the three types of robots we are developing.”
Other announcements at the keynote included GEFORCE “Blackwell,” the latest version of the company’s gaming GPU. It’s priced at $1,599 to $549, down from the previous model’s $4090, and goes on sale this month, with the laptop version coming in March. Project DIGITS is a compact personal computer optimized for AI development, running a new version of Grace-Blackwell’s combined CPU and GPU chip called GB10.