Driving by Dreaming: Offline Model-Based Reinforcement Learning for Autonomous Vehicles

Abstract

While there has been significant progress in deploying autonomous vehicles (AVs) in urban driving settings, there remains a long-tail of challenging motion planning scenarios that must be addressed before truly driverless operation is possible. The current paradigm for motion planner design is engineering intensive, making it challenging to scale to address the long-tail. In this work, we explore the use of offline model-based reinforcement learning as an alternative approach for designing AV motion planners. We propose an offline RL algorithm which make use of the structure in the AV domain to create dynamics models and policies that generalize successfully. Additionally, we propose an extension to this algorithm to a multi-agent training environment. We demonstrate that these algorithms can match state-of-the-art performance on simpler driving benchmarks and explore future works for scaling to the more challenging benchmarks.