Abstract:
We address the problem of large-scale and long-term neural map representations. Maps, as our prior understanding toward the environment, provide valuable information for modern robotics applications such as autonomous driving and AR/VR. The size of maps largely affects the end task performance: usually a more detailed map can support better performance, but would cost a large system burden to store and process. Besides the map size, the real world is dynamic, adapting a pre-built map to fit the current environment is also very important.

In this talk, we will first introduce our previous work about compressing the maps image-to-SfM map registration, and then describe our plan about LiDAR-assisted neural map representation for large-scale and long-term environments. Recently, NeRF-inspired neural scene representations have shown very promising results in indoor and static scenes by compressing the rich scene information within a deep neural network. However, the existing approaches still struggle in large scenes due to high computation cost, dynamic objects, and lack of camera sample coverage. On the other hand, LiDAR data are widely available on city-scale and provide accurate 3D scene geometry. We aim to leverage the strength of LiDAR sensor to overcome the drawbacks in the current neural scene representation systems and to improve the performance of large-scale and long-term novel view synthesis. To achieve this, we will have to overcome multiple challenges, including the sparsity of LiDAR data, seasonal changes, temporal consistency, computational cost, and more. In short, given sparsely collected data, our goal is to interpolate/synthesize novel views not only spatially but also temporally with our LiDAR-assisted neural map representations.

Thesis Committee Members:
Michael Kaess, Chair
Simon Lucey, Co-chair
Matthew Johnson-Roberson
Ian Reid. The University of Adelaide

More Information