FX.co ★ ‘Big Bang’ of physical AI: Nvidia at GTC Taipei

Cosmos 3: all-seeing eye

Nvidia’s standout announcement was the unveiling of Cosmos 3, a revolutionary multimodal model capable of processing text, images, video, sound, and physical actions simultaneously. The company positions it as the first fully open system of its kind, designed specifically for the needs of physical AI. This development functions not merely as a text processor but as a comprehensive model of the world. The neural network recognizes the laws of gravity, motion, and interactions between objects. This serves as a fundamental foundation for training the next generation of robots and autonomous vehicles that are expected to coexist safely with humans in real spaces.

Super and Nano: superpower and superspeed

To tackle a range of practical applications, engineers have divided the Cosmos 3 architecture into several specialized versions. The Cosmos 3 Super modification is designed for heavy-duty robotics and autonomous transport, requiring maximum precision in reasoning and the processing of vast amounts of sensor data. Meanwhile, the lightweight Cosmos 3 Nano version is optimized for rapid inference and local deployment on end devices with limited computing power. This division allows AI agents to respond instantly to environmental changes, whether it is a maneuver by an autonomous truck on the highway or precise adjustments made by a robotic manipulator.

Unitree H2 enters labor market

Simultaneously, Nvidia unveiled an extensive suite of tools and libraries that facilitate the complete automation of synthetic data generation, modeling, and training of AI systems across industries, medicine, and computer vision. The most striking embodiment of this concept is the first open benchmark project for an android built on the AI platform Isaac GR00T. As the physical foundation, engineers selected the Unitree H2 humanoid robot (1.8 m, 68 kg). The project aims to accelerate the development of commercial robots worldwide.

Sharpa Wave: human-like agility

The physical capabilities of the benchmark Unitree H2 robot are impressive, showcasing remarkable flexibility. The metallic body of the android features 31 degrees of freedom, allowing it to mimic a natural human gait and maintain balance on uneven surfaces. A standout innovation is the Sharpa Wave, a set of five-fingered robotic hands. These high-precision manipulators increase the overall degrees of freedom of the entire system to an astonishing 75 units. With such a design, the robot is capable of performing delicate physical tasks with the same dexterity as a human.

Thor’s brain: world without pauses

Unitree H2’s intellectual abilities and movement coordination are powered by a robust embedded computing brain, the Nvidia Jetson AGX Thor AI platform. The robot continuously perceives its surroundings through built-in stereo cameras and advanced motion sensors. The visual and sensory information collected is instantly processed by local neural networks. In real time, the android recognizes obstacles, constructs a depth map of its environment, anticipates the movement trajectories of nearby individuals, and rapidly adjusts its physical actions to ensure overall safety.

Why robots need their own "matrix"

To train its robots, Nvidia has developed a comprehensive software environment within the Isaac GR00T platform. Researchers utilize simulators like Isaac Sim and Isaac Lab for virtual training of machines in digital worlds where robots can make millions of mistakes without incurring physical damage. The Isaac Teleop tool allows for the collection of demonstration data by remotely mimicking the movements of a human operator, while the Isaac ROS libraries transport the trained models onto actual hardware. Research centers are already implementing this system, laying the groundwork for the future of commercial robotics.

Secret alliance of silicon and neural networks

Nvidia’s physical AI is making inroads deep within chip production. The company has expanded its strategic partnership with TSMC, which specializes in complex computational lithography and precise modeling of transistor behavior. The company’s Metropolis and TAO Toolkit tools aid in automating process management and performing precision defect detection on silicon wafers. Neural networks are capable of instantly identifying nanometer-level deviations, significantly enhancing the efficiency of next-generation semiconductor production and reducing defects.

32 billion parameters powering autonomous transport revolution

For the autonomous transport industry, Nvidia has introduced the specialized Alpamayo 2 Super model, which boasts 32 billion parameters. This enormous neural network is specifically designed for integration into next-generation robo-taxi and autonomous truck systems. Physical AI enables vehicles not only to follow road markings but also to fully perceive, analyze, and understand real-time traffic situations. Alpamayo 2 Super evaluates hundreds of possible scenarios involving other road users, plans safe actions, and makes instantaneous decisions.

Silicon safe and Windows supercomputer

Completing this ecosystem is the DGX Station, a desktop supercomputer for AI running on Windows, built on the latest GB300 Grace Blackwell Ultra platform. This powerhouse can locally run models of up to 1 trillion parameters. The device is positioned as a secure enterprise platform that allows for the training and testing of autonomous AI agents without sending sensitive data to the cloud. The ultra-fast ConnectX-8 adapter provides a throughput of 800 Gbps, with initial deliveries of the supercomputer expected in the fourth quarter of 2026.