Abstract:
Autonomous mobile robots exploring in unfamiliar environments often need to detect target objects during exploration. Most prevalent approach is to use conventional object detection models, by training the object detector on large abundant image-annotation dataset, with a fixed and predefined categories of objects, and in advance of robot deployment. However, it lacks the capability to detect completely novel, unseen class categories of objects, which might appear during the robot exploration. We investigate how we can make robots detect novel objects and quickly learn the novel object categories during the exploration. We propose a human-in-the-loop framework to address this issue. Our framework combines (1) online interesting scene detection, which detects anomalous scenes in an online fashion and sends these scenes to a human operator for feedback, and (2) human-informed few-shot object detection module, which quickly trains the object detector with minimal data and time limits. We show that our framework shows competitive object detection performance along with state-of-the-art object detection baseline, and enables quick learning of novel objects during mobile robot exploration.

Committee:
Sebastian Scherer (Chair)
David Wettergreen
Deepak Pathak
Cherie Ho