Abstract:

As humans we can never fully observe the world around us and yet we are able to build remarkably useful models of it from our limited sensory data. Machine learning problems are often required to operate in a similar setup, that is the one of inferring unobserved information from the observed one. Partial observations entail data uncertainty, which may hinder the quality of the model predictions. In this talk, I will discuss two strategies to mitigate this problem: (1) leveraging the complementarity of different data modalities, and (2) actively acquiring additional information from the same data modality. In particular, I will give an overview of 3 recent contributions. The first part of the talk will be devoted to the problem of inverse cooking, where the goal is to infer the whole cooking process from a food image and its corresponding ingredients. The second part of the talk will show how fusing visual and tactile signals leads to better 3D understanding of object shapes. Finally, in the third part of the talk, I will discuss active signal acquisition in the context of MRI reconstruction.

BIO:

Adriana Romero-Soriano is a research scientist at Facebook AI Research and an adjunct professor at McGill University. Her research focuses on developing models and algorithms that are able to learn from multi-modal and real world data, understand and reason about conceptual relations, and recognize their uncertainties, while addressing impactful problems. She completed her postdoctoral studies at Mila, where she was advised by Prof. Yoshua Bengio. Her postdoctoral research revolved around deep learning techniques to tackle biomedical challenges, such as the ones posed by multi-modal data, high dimensional data and graph structured data. She received her Ph.D. from University of Barcelona with a thesis on assisting the training of deep neural networks with applications to computer vision, advised by Dr. Carlo Gatta. Adriana serves as AC for conferences such as ICLR, NeurIPS, ICML, and CVPR. She co-organized WiCV at CVPR 2018, RLGM at ICLR 2019, SEDL at NeurIPS 2019 and the Montreal AI Symposium 2019.

Homepage:  https://sites.google.com/site/adriromsor/home

Sponsored in part by:   Facebook Reality Labs Pittsburgh