Event Location: 1305 Newell Simon Hall
Bio: Kristen Grauman is a Clare Boothe Luce Assistant Professor in the Department of Computer Science at the University of Texas at Austin. Her research in computer vision and machine learning focuses on visual search and object recognition. Before joining UT-Austin in 2007, she received her Ph.D. in the EECS department at MIT, in the Computer Science and Artificial Intelligence Laboratory. She is a Microsoft Research New Faculty Fellow, and a recipient of an NSF CAREER award and the Krell Institute Howes Scholar Award in Computational Science.

Abstract: How should visual recognition algorithms solicit and exploit human knowledge? Existing approaches often manage human supervision in haphazard ways, and only allow a narrow, one-way channel of input from the annotator to the system. We propose learning algorithms that steer human insight towards where it will have the most impact, and expand the manner in which recognition methods can assimilate that insight.

I will present an approach to actively seek annotators’ input when training an object recognition system. Unlike traditional active learning methods, we target not only the example for which a label is most needed, but also the type of label itself (e.g., an image tag vs. full segmentation). Further, since annotations should be fielded by distributed, uncoordinated annotators, we develop selection algorithms to compute far-sighted predictions of which batches of data ought to be labeled next, and to rapidly identify uncertain points within massive unlabeled pools. Finally, beyond “asking” annotators the right questions, I will show how we can “listen” more deeply to what image-taggers unknowingly reveal in their annotations, by learning implied cues about object prominence from lists of ordered keywords. Using these techniques, we obtain state-of-the-art object detection and image retrieval results on benchmark datasets.

This talk describes work done with Sudheendra Vijayanarasimhan, Prateek Jain, and Sung Ju Hwang.