For autonomous vehicles operating in environments where people or other vehicles may be sharing the road, it is important to detect and track moving objects. Merely detecting that an obstacle exists is useful, but it is even more useful to know whether the obstacle is moving, and, if so, what direction it is moving. The goal of this project is to detect, model, and track dynamic objects in the scene, taking into account spatio-temporal errors that arise from asynchronous data from different sensors.

Traditionally, moving object detection and tracking algorithms focus on either 2D imagery or 3D data, but do not effectively fuse the two sources of information. Methods that do utilize imagery and LIDAR typically use parallel, but disconnected, pipelines for processing the data from each source and then combine the information only as a final step. Our approach is to use the information from each sensing modality at each step in the process, thereby gaining the benefits of multi-modal sensing throughout the pipeline.

Moving object detection and tracking broadly consists of three steps: segmentation, in which the boundaries of objects are determined; modeling, in which objects are modeled by simple geometric shapes, such as boxes or poly-lines; and tracking, in which the modeled objects are tracked over time. We believe that use of fused imagery and LIDAR can be beneficial within each of these steps. In segmentation, we can use image edges to find object boundaries more reliably. In the modeling step, we can use the image for more accurate modeling and for modeling outside of the LIDAR plane. In the tracking step, we can use image features to help track object motion.