In the first part of the talk, I will first briefly give an overview of our new architecture for semantic segmentation that incorporates our proposed multiscale residual units, an efficient atrous spatial pyramid pooling module and a deep decoder with skip refinement stages. I will then describe our framework for probabilistically fusing features from multiple modality-specific streams adaptively based on the scene condition. This approach achieves state-of-the performance for multimodal segmentation on several benchmarks including Cityscapes, Synthia, SUN-RGBD, ScanNet and Freiburg Forest.

In the second part of the talk, I will describe our architecture for joint semantic motion segmentation where, in addition to the object category, our model also predicts the motion status of each pixel using consecutive monocular images. Our network fuses semantic features with learned motion features while suppressing any induced ego-flow to yield pixel-level semantic motion labels. This work is currently the state-of-the-art for semantic motion segmentation on the Cityscapes and KITTI datasets.

Speaker Bio
Abhinav Valada is a Ph.D. candidate in the Autonomous Intelligent Systems lab at the University of Freiburg in Germany, where he is advised by Prof. Wolfram Burgard. His research is focused on weakly-supervised/self-supervised deep learning algorithms for multimodal robot perception and state-estimation. His methods are currently the state-of-the-art on several scene understanding and pose estimation benchmarks. Abhinav received his M.S. in Robotics from Carnegie Mellon University. He was previously a research engineer at the Field Robotics Center and the National Robotics Engineering Center. His work has been published at major robotics conferences and journals including ICRA, IROS, RSS, FSR, ISRR, ISER and IJRR.