Gemini Robotics-ER

Σχετικά με Google DeepMind

Ιδρύθηκε 2010 · London, UK / Mountain View, USA

Google DeepMind announced Gemini Robotics-ER (Embodied Reasoning) alongside Gemini Robotics in March 2025. While Gemini Robotics is the action-producing VLA, Gemini Robotics-ER is the reasoning-focused sibling: a Vision-Language Model variant of Gemini 2.0 specialised for spatial understanding, 3D grounding, point/box prediction, trajectory planning and code generation for robotics. It is designed to be combined with classical motion planners, low-level controllers or with the Gemini Robotics VLA itself. DeepMind positions Gemini Robotics-ER as a 'reasoning brain' that a robot stack can call with multimodal prompts to decompose tasks, locate objects in 2D / 3D, and emit waypoints or Python control code. As with Gemini Robotics, access is limited to research and partner programs.

Αρχιτεκτονική

Vision-Language Model for Embodied Reasoning (no end-to-end action head)

Gemini Robotics-ER is a fine-tuned variant of Gemini 2.0 specialised for embodied perception and planning rather than direct control. The architecture preserves the multimodal Transformer backbone of Gemini 2.0 (image, video, text, code) but is post-trained on a curated corpus of embodied tasks: object detection in 2D and 3D, point and bounding-box prediction, grasp prediction, motion-trajectory generation, and code-as-policy outputs that call robot APIs. It can accept egocentric robot camera streams and a natural-language task description, then produce structured outputs such as pixel-space points to grasp, 3D coordinates relative to the camera, planning steps, or Python snippets that drive a downstream controller. In combination with the Gemini Robotics VLA, Robotics-ER provides high-level reasoning while the VLA handles closed-loop low-level actions.

Παράμετροι

Undisclosed (Gemini 2.0-class)

Δυνατότητες

• Spatial reasoning over 2D / 3D scenes
• Point and bounding-box prediction for objects and grasps
• Trajectory waypoint generation
• Code-as-policy generation (Python that calls robot APIs)
• Compositional task planning from natural language
• Pair with motion planners or with Gemini Robotics VLA
• Multimodal context: images, video, text, robot state
• Improved zero-shot performance on embodied QA benchmarks
• Best for: planning and grounding modules in research robotics stacks.

Εκπαίδευση & άδεια

Gemini 2.0 multimodal pretraining plus embodied post-training on object-detection, 3D grounding, grasp prediction, trajectory planning, and code-generation tasks for robotic control. Draws on Google's internal robot datasets and curated public embodied datasets.

Άδεια: Research-only / partner access through Google DeepMind. Not publicly downloadable.

Δοκιμές ασφάλειας: Covered under Google DeepMind's Frontier Safety Framework and Responsible AI evaluations, with specific embodied-AI red-teaming for physical-harm and unsafe-action scenarios.