TL;DR This work explores building generative neural network models of popular reinforcement learning environments by using features extracted from the world model as inputs to an agent, and can train a very compact and simple policy that can solve the required task.