Uncertainty-Aware Robotic World Model Makes Offline MBRL Work on Real Robots
Authors: Chenhao Li, Andreas Krause, Marco Hutter
Institution: ETH Zurich
arXiv: 2504.16680
Abstract
RWM-U extends autoregressive robotic world models with epistemic uncertainty estimation via ensembles, enabling temporally consistent multi-step rollouts with uncertainty propagated over long horizons. Combined with MOPO-PPO (uncertainty-penalized policy optimization), it enables training effective policies entirely from offline datasets on real quadruped and humanoid robots.
Key Method
- RWM-U: autoregressive world model with ensemble-based epistemic uncertainty
- Uncertainty signal penalizes high-risk imagined transitions during policy learning
- MOPO-PPO: adapts offline model-based RL with PPO for stable on-policy optimization
- Handles compounding errors and distribution shift in long-horizon rollouts
Safety Relevance
Uncertainty penalization is the core safety mechanism: by making the policy pessimistic about regions where the world model is unreliable, the agent avoids extrapolating into unsafe states not covered by offline data.
Results
- Evaluated on manipulation and locomotion tasks (simulation + real hardware)
- Outperforms model-free and uncertainty-unaware baselines
- Fusing real-world data with simulation further improves robustness
Code
GitHub: leggedrobotics/robotic_world_model