Serena Yeung
Ph.D. Student, Stanford University

Abstract
The quantity of video data is vast, yet our capabilities for visual recognition and understanding in videos lags significantly behind that for images. In this talk, I will discuss the challenges of scale in labeling, modeling, and inference behind this gap. I will then present three works addressing these challenges. The first is a method for efficient inference of action detection in videos. We formulate this method as a reinforcement learning-based agent that interacts with a video over time, and decides both where in the video to look next and when to emit a prediction, significantly reducing the total frames processed in the video. The second work pushes dense, detailed understanding of actions in video. We introduce a dataset of dense, multilabel action annotations to enable research in this direction, and a model that increases temporal modeling capacity from standard recurrent neural networks for action recognition to target this task. Finally, I will discuss an approach for leveraging noisy web videos to learn classifiers for new concepts without requiring manually labeling training videos. We propose a reinforcement learning-based formulation for selecting the right examples for training a classifier from noisy web search results. I will show that after learning a data labeling policy on a small labeled training set, we can then use this policy to automatically label noisy web data for new visual concepts.

Speaker Biography
Serena Yeung is a Ph.D. student in the Stanford Vision Lab, advised by Prof. Fei-Fei Li. Her research interests are in computer vision, machine learning, and deep learning. She is particularly interested in the areas of video understanding, human action recognition, and healthcare applications. Serena interned at Facebook AI Research in Summer 2016, and before starting her Ph.D., received a B.S. and M.S. in Electrical Engineering, both from Stanford.