Virtual VASC Seminar:  https://cmu.zoom.us/j/249106600

Abstract:  Deep Neural Networks (DNNs) are state-of-the-art models for many vision tasks. We propose an approach to assess the relationship between visual tasks and their task-specific models. Our method uses Representation Similarity Analysis (RSA), which is commonly used to find a correlation between neuronal responses from brain data and models. With RSA we obtain a similarity score among tasks by computing correlations between models trained on different tasks. We demonstrate the effectiveness and efficiency of our method to generating task taxonomy on the Taskonomy dataset. Also, results on PASCAL VOC suggest that initializing the models trained on tasks with higher similarity score show higher transfer learning performance. Finally, we explore the power of DNNs trained on 2D, 3D, and semantic visual tasks as a tool to gain insights into functions of visual brain areas (early visual cortex (EVC), OPA and PPA). We find that EVC representation is more similar to early layers of all DNNs and deeper layers of 2D-task DNNs. OPA representation is more similar to deeper layers of 3D DNNs, whereas PPA representation to deeper layers of semantic DNNs. We extend our study to performing searchlight analysis using such task specific DNN representations to generate task-specificity maps of the visual cortex. Our findings suggest that DNNs trained on a diverse set of visual tasks can be used to gain insights into functions of the visual cortex. This method has the potential to be applied beyond visual brain areas.

Bio:  Gemma Roig is a professor (W1 with tenure-track) at the computer science department in the Goethe University Frankfurt since January 2020. She also holds an honorary position as research affiliate at MIT. Before, she was an assistant professor at the Singapore University of Technology and Design. She conducted her postdoctoral work at MIT in the Center for Brains Minds and Machines, with Prof. Tomaso Poggio as her faculty host.  She was also affiliated at the Laboratory for Computational and Statistical Learning, which is a collaborative agreement between the Istituto Italiano di Tecnologia and the Massachusetts Institute of Technology.  She pursued her doctoral degree in Computer Vision at ETH Zurich. Previous to that, she was a research assistant at the Computer Vision Lab at EPFL in Lausanne, at the Department of Media Technologies at Ramon Llull University in Barcelona, and at the Robotics Institute at the Carnegie Mellon University in Pittsburgh. Her research aim is to build computational models of human vision to understand its underlying principles, and to use those models to build applications of artificial intelligence.
Homepage:  http://www.cvai.cs.uni-frankfurt.de/

Sponsored in part by:   Facebook Reality Labs Pittsburgh