This week’s demo showcases my first foray into modern Reinforcement Learning (RL). Teaching a Quadcopter to “learn Gravity” is one of the simplest RL problems I could imagine — an excellent starting point for an RL journey. I quickly found out, however, that I actually have something like 5k to 10k hours of experience in this type of work (reward shaping, curriculum design, hyperparameter tuning, etc). We just used a different set of tools and libraries in 2010 (RL for stable legged locomotion via nonlinear optimization and Hybrid Zero Dynamics).