On the Road to Artificial General Intelligence • Danny Lange on game engines for AI/ML training

Great talk on the use of game engines (virtual worlds) for AI/ML agent training. Even if you are already knowledgeable about this topic, the examples he shows will be useful to guide your intuition as to what is possible, what is easy/hard with current technology and methods. Don't miss the puppies :-)

Conceptually, I would say there is not much new since the early successes with simpler (e.g., Atari) games. See papers/talks by Schmidhuber in this 2014 post. IIRC, the concept of curiosity: seeking "surprise" = large chunks of information = large model updates, was formulated already some time ago. 

One thing that is new is the use of physics engines in the virtual worlds - i.e., the AI has to deal with dynamics as in the real world. It seems to me that routine task automation, such as in manufacturing, is not that much harder than what is being done here in game worlds with good physics engines. (Note I'm not referring to the mechanical engineering or physical robotics challenges, which could be significant, just the ML part.) Replacement of humans in many routine tasks seems now a matter of economics tradeoffs and application of known technologies rather than big breakthroughs.

I've always thought we'd get to AGI after consuming a lot of FLOPS training agents in increasingly realistic virtual worlds. Of course, this makes one wonder whether we ourselves exist in a simulation ;-)