Coping with sample inefficiency of deep-reinforcement learning (DRL) for embodied AI

Abstract

We want to be efficient and reduce monotonous burden with AI tools like autonomous vehicle and home assistants. Learning has demonstrated great potential for applications like robotics, self-driving cars and IoT. But why learning algorithms haven’t been deployed on consumer robotics/AV platforms? For example, robot vacuum cleaners use currently rely on a few simple algorithms, such as spiral cleaning (spiraling), room crossing, wall-following and random walk, angle-changing after bumping into an object or wall. There are many reasons why robotic learning is challenging in real world - lack of safety assurances, reward specification, lack of progress on the continual learning front. One of the major focus is mainly on lack of sample efficiency. While we need to have good simulators to train, we also need better ways of acquiring experience in embodied AI and robots for real platforms.
In our breakout session, we discussed about some ways (either proven or promising) to make DRL feasible for embodied AI. We look into real embodied AI learning in order to fundamentally enhance the notion of intelligence by incorporating multi-modal interaction. We talk about two divergent approaches: algorithmic approaches to improve sample efficiency and alternatively, circumventing the sample efficiency problem by scaling up data collection for current state-of-the-art algorithms.