According to an official announcement published by Steven Hansen (Senior Staff Software Engineer) and Peng Xu (Staff Software Engineer) from Google DeepMind, Google has announced the launch of Gemini Robotics ER 2. This model is defined as the company's most advanced Embodied Reasoning model for robotics, developed to serve as a high-level "brain" for robots. It introduces a step change in robots' capabilities to operate in the physical world through video understanding, task orchestration, and multi-robot collaboration.
For robots assisting humans in everyday environments, accurate spatial reasoning is not enough on its own. Robots must think quickly, time their decisions, and execute reasoning processes at speeds that match the real-time pace of the physical world. Gemini Robotics ER 2 was developed for this purpose. The model enables robots to chat with humans, understand the physical world, and plan complex, multi-step tasks. After planning the task at a high level, the model hands off the actual physical motor execution to any given lower-level Vision-Language-Action (VLA) model.
The new model represents a significant upgrade over the company's previous model, Gemini Robotics ER 1.6. By watching continuous streaming video, robots can now track their own progress, adapt and correct their actions if something goes wrong, and know exactly when to transition to the next step of the task. Additionally, the model allows native calling of external tools; it is capable of naturally calling tools like Google Search to find information, or executing any other user-defined function. The model's architecture is designed so that the robot can "think" about its next step while simultaneously executing its current physical actions.
Improved Tool Orchestration and Latency-Free Integration
Most tasks in the physical world are complex and require multiple steps to complete. Gemini Robotics ER 2 acts as a physical agent that manages workflows for the robot, allowing it to self-correct and adapt to new, unfamiliar situations. To build an agentic setup, developers can declare low-level control interfaces—such as VLA models or navigation APIs—as tools, and stream multimodal video, audio, or text directly into the model.
The performance of Gemini Robotics ER 2 in tool orchestration was evaluated by the company in simulations, real-world physical robot control, and even in combination with a human operator controlling the robot remotely (human tele-operation). According to Google's testing data, the new model consistently outperforms the ER 1.6 model in tool orchestration across the three evaluated control modes: real VLA in the physical world, simulation VLA, and remote control by a human operator.
In robotics, high-level reasoning depends directly on execution speed. Gemini Robotics ER 2 integrates directly into the Gemini Live API, using a bidirectional streaming endpoint optimized specifically for latency-sensitive tasks. This integration enables smooth and fluid orchestration: the model passes commands to action models and the robot's APIs without needing prolonged "stop-and-think" pauses between steps.
To demonstrate this capability, Google built a demo together with its project partner Boston Dynamics, using the Spot robot. The developers used Gemini Robotics ER 2 to coordinate and manage Spot's APIs, including navigation and robot arm movement (manipulator movement). This integration created an interactive robot capable of fetching objects for the user; for example, the system prompted Spot to fetch a popcorn snack following a natural language voice command. The code for this demo and additional examples have been open-sourced on GitHub.
Task Progress Tracking and Precision Moment-Finding
One of the most complex challenges in robotics is the machine's ability to identify when a task has been fully completed. Gemini Robotics ER 2 presents significant progress in video understanding and progress tracking capabilities, allowing it to verify that complex tasks—such as tightening a light bulb or tying a trash bag—are completed exactly to the required specification before transitioning to the next task in line. Within this update, Google introduced a breakthrough in two foundational capabilities for task progress understanding:
First, Continuous progress classification: this refers to the robot's ability to evaluate and quantify its progress toward task completion. In the evaluations conducted by the developers, the model assigned each individual frame from the continuous video feed to one of five defined progress levels: 0-20%, 20-40%, 40-60%, 60-80%, and 80-100%. By quantifying task progress, the model provides robots with real-time situational awareness, enabling them to adjust actions on the fly or retry failed steps without having to restart the entire workflow from the beginning. In these tasks, Gemini Robotics ER 2 achieved an accuracy of 57.4%, a result that outperforms previous-generation models and leading competitor models in the market.
Second, Precision moment-finding: this metric evaluates the model's ability to identify the specific video frame where a critical event takes place (for example, detecting the exact moment when coffee pouring into a cup should stop). Gemini Robotics ER 2 demonstrates a significant improvement in performance on these tasks, with 91.3% accuracy and a mean absolute distance of only 0.96 seconds. The model manages to compete closely with much larger model categories, but delivers this high precision at a fraction of the compute cost and 4x the execution speed—enabling the sub-second latency required to safely operate robots in the physical world.
Multi-Robot Collaboration and Improved Spatial Intelligence
Since no single robot fits every task perfectly—a wheeled robot might excel at moving indoors, while a humanoid robot may excel at navigating uneven terrain—the new model enables collaboration between multiple different robots. Gemini Robotics ER 2 allows diverse machines to communicate with each other based on a shared semantic understanding, thereby handing off tasks and workflows from one to another to complete complex projects that a single robot could not perform alone. Google demonstrated this capability through a collaboration between Apptronik's Apollo 2 robot and the Franka F3 Duo robot, working together in a shared space.
In addition to collaboration, Gemini Robotics ER 2 features improvements in its core spatial intelligence, as measured by three defined benchmarks:
- Success or failure detection: The model now operates on raw video feeds instead of relying on static snapshots, thereby enabling it to detect failures that occur mid-execution, such as spills, slips, or misalignments and positional deviations.
- General instrument reading: The model's instrument reading capability has been expanded beyond circular dials and standard sight glasses, and now also includes digital displays, linear scales, rulers, and liquid thermometers. The company tested this capability across 10 different types of measuring instruments.
- Enhanced spatial visual question answering (VQA): A significant improvement in visual question answering capabilities, thanks to the general progress of Gemini models in multimodal understanding.
Advanced Safety and New Benchmarks
Another key aspect of the model's development is safety. Gemini Robotics ER 2 is defined as Google's safest model in this field to date, achieving significant improvements in Safety Instruction Following performance tests and Human Proximity benchmarks. These tests evaluate how the model adheres to defined physical constraints during reasoning tasks, and its spatial awareness for detecting human presence in its environment.
During testing, it was found that the model successfully halts a humanoid robot immediately and safely when a person approaches, and autonomously resumes work only once the area is completely cleared. To advance the field of safety for physical agents, Google is introducing a completely new evaluation benchmark. This benchmark evaluates a foundation model's ability to act as a safe VLA orchestrator by testing its capacity to enforce safety constraints, actively monitor the physical environment, assess the physical feasibility of actions, and request clarification from a human operator in case of uncertainty. Full details on these capabilities appear in Google's technical report on safety.
According to the announcement from Google's development team, future plans include the continued development of these models toward solving even more complex tasks, aiming to accelerate the development of useful assistant robots and broadly support the global robotics community.