Survey on Physics Engines, Simulation Frameworks, and Benchmarks for Robot Learning

Alexandre Brown*1,8, Vivek Shankar Varadharajan*2,8, Nico Bohlinger*3, Elham Daneshmand*4,8, Maeva Guerrier*2,8, Reginald McLean*5, Michael Przystupa*6,7, Ozgur Aslan1,8, Artur Kuramshin1,8, Charlie Gauthier1,8, Miguel Saavedra-Ruiz1,8, Liam Paull1,8, Jan Peters9,10,11, Giovanni Beltrame2,8, Glen Berseth1,8
1Université de Montréal, 2 Polytechnique Montréal, 3 Technical University of Darmstadt, 4McGill University, 5Toronto Metropolitan University, 6University of Alberta, 7Alberta Machine Intelligence Institute (Amii), 8Mila - Québec AI Institute, 9German Research Center for AI (DFKI), 10Hessian.AI, 11Robotics Institute Germany (RIG).
*Indicates Equal Contribution
Paper Code

Abstract

Robot learning research is rapidly advancing and relies heavily on simulation tools---physics engines, simulation frameworks, and benchmarks---to develop and evaluate algorithms faster, safer, and more cost-effectively than using physical hardware alone. However, the recent explosion of such tools has created a fragmented landscape, making it difficult for researchers to navigate options or forcing them to constantly re-implement baselines. This challenge is amplified by blurry terminology, where terms like "simulator" are used ambiguously. To address this, our survey makes four key contributions: (1) We introduce a formal Robot Learning Problem definition to standardize terminology and categorize existing tools accordingly. (2) We provide a comprehensive overview of the features, strengths, and weaknesses of current simulation tools, based on literature and hands-on evaluation. (3) We offer guidelines for tool selection tailored to major robot learning domains: vision manipulation, locomotion, and navigation. (4) We introduce an accompanying open, living repository to track updates in the field. By systematically structuring the simulation landscape, we aim to lower the entry barrier for newcomers, reduce redundant engineering for experts, and accelerate progress toward reproducible, general-purpose robot learning systems.

Robot Learning Problem Diagram

Outline of Robot Learning
Outline of Robot Learning
The robot learning problem is complex and requires a variety of software tools to solve. The diagram above illustrates the different components involved in the robot learning problem, including the simulation environment, physics engine, and robot learning algorithms. The role of different components make it challenging to itemize software tools in mutually exclusive categories.

Simulation Frameworks/Benchmarks Gallery

IsaacSim

Move towards a direction with the MuJoCo ant robot. Insert a plug into its corresponding socket with the Franka robot. Insert and mesh gear into the base with other gears, using the Franka robot. Pick a cube and bring it to a sampled target position with the Franka robot. Stack three cubes (bottom to top: blue, red, green) with the Franka robot. Pick up and place an object in a basket with a GR-1 humanoid robot. Track a velocity command on rough terrain with the Unitree H1 robot. Passing an object from one hand over to the other hand. Navigate towards a target x-y position and heading with the ANYmal C robot. Grasp the handle of a cabinet's drawer and open it with the Franka robot.

Sapien/Maniskill3

Control the AnymalC robot to reach a target location in front of it. The robot must pick up one of the misplaced shapes on the board/kit and insert it into the correct empty slot. Control the humanoid unitree G1 robot to grab an apple with its right arm and place it in a bowl to the side. A G1 humanoid robot must find a box on a table and transport it to the other table and place it there.. Using the allegro hand with tactile sensors, rotate a random object. A simple task where the objective is to move a peg laying on the table to any upright position on the table. Use the Fetch mobile manipulation robot to rearrange objects in a complex scene. Use the Fetch mobile manipulation robot to move towards a target cabinet and open the target drawer out. A simple task where the objective is to grasp a red cube with the (WidowX/Franka/SO100) robot and move it to a target goal position. A simple task where the objective is to poke a red cube with a peg and push it to a target goal position.

Unity ML-Agents

A balance-ball task, where the agent balances the ball on it's head. A linear movement task where the agent must move left or right to rewarding states. A creature with 4 arms and 4 forearms. The agents must move its body toward the goal direction without falling. Agents are trapped in a dungeon with a dragon, and must work together to escape. A multi-goal version of the grid-world task. The agent must navigate the grid to the appropriate goal while avoiding the obstacles. Environment where the agent needs to find information in a room, remember it, and use it to move to the correct goal. A platforming environment where the agent can push a block around. The agent must push the block to the goal Environment where the agent needs to press a button to spawn a pyramid, then navigate to the pyramid, knock it over, and move to the gold brick at the top. Environment where four agents compete in a 2 vs 2 toy soccer game. Physics-based Humanoid agents with 26 degrees of freedom. The agents must move its body toward the goal direction without falling.

CoppeliaSim/RLBench

The agent needs to close a box. The agent needs to close a fridge. The agent needs to close a microwave. The agent needs to pick up the block and lift it up to the target. The agent needs to lift the charger up to the wall and plug it in. The agent needs to pick up the umbrella and drop it in its stand. The agent needs to touch the ball with the panda gripper. The agent needs to grip the handle and slide the door open. Grasping the umbrella by its handle, lift it up and out of the stand. Put the toilet seat down.

Gazebo

Robot manipulation in a complex factory setting. Full body humanoid robot definition. Preview of a complex scene that is part of the Gazebo SubT Challenge. Custom under-water vehicle in Gazebo. Humanoid model captured by complex sensors (thermal & depth). Robot locomotion in a complex factory setting. Complex office model in Gazebo. Outdoor scene with water. Robot manipulation as shown in the paper: An intelligent manufacturing cell based on human-robot collaboration of frequent task learning for flexible manufacturing. Robot manipulation as seen in the paper: Bimanual rope manipulation skill synthesis through context dependent correction policy learning from human demonstration.

CARLA

Habitat

Embodied Question Answering challenges an AI agent to spawn at a random location in a 3D environment, explore via egocentric vision to gather information, and then answer a posed question about the scene. Habitat 3.0 uses a motion and behavior generation policy, enabling the programmatic control of avatars for navigation, object interaction, and various other motions. This image shows an embodied agent navigating a photorealistic Habitat environment as it follows given instructions to locate and interact with a specified goal. This image shows an embodied agent exploring a photorealistic Habitat environment using egocentric vision to locate and navigate to a specified target object. The agent needs to methodically rearranging objects to match a specified goal configuration. The agent needs to methodically rearranging objects to match a specified goal configuration. High-fidelity Habitat Replica CAD environment reconstructed from detailed indoor scene scans. High-fidelity Habitat Replica CAD environment reconstructed from detailed indoor scene scans. Using egocentric vision the agent needs to navigate through connected rooms toward a designated destination. Agents need to collaboratively rearrange objects as part of a social interaction task.

AI2THOR

iTHOR scene example. iTHOR scene example where the agent needs to open a drawer to place objects in it. iTHOR scene example where the agent needs to pour liquid into a glass. iTHOR scene example where the state of the object changes as it is being cooked. ManipulaTHOR example showing a robot grabbing a fruit. ManipulaTHOR example showing a robot grabbing a fruit. ProcTHOR scene example. ProcTHOR scene example. RoboTHOR Real vs Sim scene comparison. RoboTHOR scene.

Robosuite

The robot arm must learn to turn the handle and open the door. The two robot arms must coordinate so that the arm closer to the hammer picks it up and hands it to the other arm. The robot arm must lift the cube above a certain height. The robot must fit the square nut onto the square peg and the round nut onto the round peg. The two robot arms must coordinate to insert the peg into the hole. The robot must place each object into its corresponding container. The robot must place one cube on top of the other cube. The robot must place one cube on top of the other cube. The two robot arms must each grab a handle and lift the pot together, above a certain height, while keeping the pot level. The robot arm must learn to wipe the whiteboard surface and clean all of the markings.

iGibson

MuJoCo Playground

A precise peg-insertion task in the ALOHA benchmark environment. Agent controlling a simulated robotic hand to perform precise hand manipulation tasks. Locomotion task. Locomotion task. Locomotion task. Locomotion task. Locomotion task. The agent needs to pick th cube. The agent needs to pick th cube. Agent controlling a simulated robotic hand to perform precise hand manipulation tasks to position the box in the right orientation and position.

PyBullet

Table tennis scene example. Table-top robot manipulation example Robot manipulation task where the robot must learn to pick and throw arbitrary objects into selected boxes. Table-top robotic manipulation pick & place task. Locomotion task. Robot assistance task where the agent needs to assist a patient. Robot assistance task where the agent needs to treat a patient.

BibTeX

BibTex Code Here