invisible fΦrce Knowledge hub
Learning

What is
reinforcement learning?

Reinforcement learning (RL) is learning by trial and error. The robot tries an action, gets a score, and adjusts so that it scores higher next time. Repeat millions of times and a robot that started by falling over ends up walking.

The loop

At each moment the robot observes its state: joint angles, body tilt, camera images. It picks an action: a set of motor commands. The world moves on, and the robot receives a reward, a number that says how well that went: positive for moving forward, negative for falling. The policy is the rule, usually a neural network, that maps states to actions. Training means adjusting the policy so that total reward goes up.[3] The standard software interface for this loop is called Gymnasium, and most simulators speak it.[6]

The field's founders, Andrew Barto and Richard Sutton, received the 2024 ACM Turing Award, computing's highest honour, for developing these ideas.[2] Their textbook is free online.[3]

Why simulation is essential

Learning by trial needs an enormous number of trials. A real robot can do one attempt at a time and breaks when it falls. A simulator can run thousands of copies at once: a standard NVIDIA Isaac Lab training task simulates 4,096 robots in one GPU scene.[4] NVIDIA's 2025 paper on Isaac Lab describes this as the point of the tool: GPU-parallel physics plus realistic rendering for robot learning at data-centre scale.[5] Without simulation, modern RL for robots would not exist.

The algorithms

Almost all robot locomotion today is trained with PPO (Proximal Policy Optimization), published by OpenAI in 2017. It is simple and stable, and it is still the main algorithm in rsl_rl, the ETH Zurich library used with Isaac Lab for legged robots.[7][9]

Model-based RL takes a different route. The robot first learns a "world model", a neural network that predicts what happens next, then practises inside its own imagination. DreamerV3, published in Nature in April 2025, did this across more than 150 tasks with a single configuration.[1] See What are world models and VLAs?

Does it work on real robots?

Yes, for many tasks. A 2025 paper trained a Unitree G1 humanoid to stand up from arbitrary positions and terrains entirely in simulation, then ran the policy on the real robot.[8] The difficulties arise when the simulator misses physics that matters in the real world, which is the subject of What is the sim-to-real gap?

References

  1. Hafner, Pasukonis, Ba and Lillicrap, Nature 640, 647-653. Mastering diverse control tasks through world models (DreamerV3). April 2025.
  2. US National Science Foundation. AI pioneers Andrew Barto and Richard Sutton win 2024 Turing Award. March 2025.
  3. Sutton and Barto, MIT Press. Reinforcement Learning: An Introduction, second edition (free online). 2018.
  4. NVIDIA. Isaac Lab documentation: Creating a direct-workflow RL environment. 2026.
  5. Mittal et al., NVIDIA, arXiv 2511.04831. Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning. November 2025.
  6. Towers et al., Farama Foundation, arXiv 2407.17032. Gymnasium: A Standard Interface for Reinforcement Learning Environments. July 2024, revised November 2025.
  7. Legged Robotics, ETH Zurich. rsl_rl (repository). 2025.
  8. Huang et al., Shanghai AI Lab, arXiv 2502.08378. Learning Humanoid Standing-up Control across Diverse Postures (HoST). February 2025.
  9. Schulman et al., OpenAI, arXiv 1707.06347. Proximal Policy Optimization Algorithms (origin). July 2017.

Every link was opened and checked in September 2026. Links open in a new tab.