VASC Seminar
Learning the hard way: From children and parents to ultrafast machines
Event Location: NSH 1507Bio: Neil Johnson heads up a new inter-disciplinary research group in Complexity at the University of Miami looking at collective behavior and emergent properties in a wide range of real-world Complex Systems: from physical, biological and medical domains through to social and financial domains. He is also a Professor of Physics. He [...]
Programmable Automotive Headlights
Event Location: NSH 1507Bio: Robert Tamburo is currently a project scientist at the Robotics Institute in the Illumination and Imaging laboratory. He received his Ph.D. degree from the Department of Bioengineering at the University of Pittsburgh in 2006. Afterwards, he was a post-doctoral fellow in the Department of Psychiatry at the University of Pittsburgh and [...]
Pose Machines: Articulated Pose Estimation via Inference Machines
Event Location: NSH 1507Bio: Varun Ramakrishna is a PhD student in the Robotics Institute, advised by Prof. Yaser Sheikh and Prof. Takeo Kanade. His research interests include structured prediction problems in computer vision with a focus on understanding human posture and motion from monocular images and image sequences. Varun was previously a master's student in [...]
Unfolding an Indoor Origami World
Event Location: NSH 1507Bio: David Fouhey is a Ph.D. student at the Robotics Institute, Carnegie Mellon University, where he is supervised by Abhinav Gupta and Martial Hebert. He holds an A.B. in Computer Science from Middlebury College. His research addresses a variety of scene understanding tasks in computer vision, with a particular interest in inferring [...]
Vision for cognitive robotics and some open meta problems
Event Location: NSH 1507Bio: Michael Zillich received his diploma in Mechatronics from Johannes Kepler University Linz in 1998 and his PhD in Electrical Engineering from Vienna University of Technology in 2007. Michael Zillich is (co)-author of over 80 publications and served on the program committees of a number of international conferences and as reviewer for [...]
Wave function representations in shape matching and indexing
Event Location: NSH 3305Bio: Anand Rangarajan has worked in shape analysis for over twenty years. After receiving the Ph.D. from the University of Southern California in 1991, he worked at Yale University in both the radiology and computer science departments until the advent of the new millennium. Since then, he has been in the computer [...]
Large Scale Visual Recognition in Real-World Images
Event Location: NSH 1507Bio: Jia Li is a research scientist at Yahoo! Research. She leads the Visual Computing and Learning Group. Her research interests are computer vision, machine learning,social network analysis and multimedia analysis. She received her Ph.D. degree from the Computer Science Department at Stanford University. She is the leader of the OPTIMOL team, [...]
Object Detection: from Structured Models to Deep ConvNets and Back Again
Event Location: NSH 1507Bio: Ross Girshick is a Researcher at Microsoft Research in Redmond, WA. He completed his Ph.D. in computer vision at The University of Chicago under the supervision of Pedro Felzenszwalb in 2012. Following his Ph.D., he spent two wonderful years as a postdoctoral fellow working with Jitendra Malik and Trevor Darrell at [...]
Improving visual category recognition by diversity and commonality
Event Location: NSH 1507Bio: Jonghyun Choi is a PhD candidate in the Computer Vision Lab at the University of Maryland, working with Prof. Larry Davis. He received the BS and MS degrees in electrical engineering and computer science from Seoul National University, Seoul Korea in 2003 and 2008 respectively. He has worked as an intern [...]
Head Pose Estimation and Facial Expression Recognition under a Wide Range of Head Poses
Event Location: NSH 1507Bio: Sergey Tulyakov is a PhD student advised by Prof. Nicu Sebe at the Department of Information and Communication Technologies, University of Trento, Italy. His research interest includes 2D and 3D computer vision with particular emphasis on real-time analysis of human faces. He received his MS and BS in computer science from [...]
Probabilistic Techniques for Mobile Robot Navigation
Event Location: NSH 1305Bio: I am a professor for computer science at the University of Freiburg and head of the research lab for Autonomous Intelligent Systems. My areas of interest lie in artificial intelligence and mobile robots. My research mainly focuses on the development of robust and adaptive techniques for state estimation and control. Over [...]
CANCELED
Event Location: NSH 1507
Computer vision for democratic and personalized access to sport video reports
Event Location: NSH 1507Bio: Christophe De Vleeschouwer is a Professor at ‘Université catholique de Louvain’ (UCL), and a Research Associate of the Belgian NSF. He received the M. Eng. and the Ph. D. degrees from UCL, in 1995 and 1999 respectively. As a young PhD, he was a senior research engineer with the IMEC Multimedia [...]
3D Object Understanding
Event Location: NSH 1507Bio: Joseph Lim is a PhD student in the Department of Electrical Engineering and Computer Science at Massachusetts Institute of Technology (MIT), where he is advised by Professor Antonio Torralba. His research interests are in computer vision and machine learning. He has been motivated by the goal of building computer systems that [...]
Decoupling Algorithms from the Organization of Computation for High-Performance Image Processing
Event Location: NSH 1507Bio: Jonathan Ragan-Kelley is a post-doctoral researcher at Stanford, where he works with Pat Hanrahan. He recently finished his PhD in computer science at MIT, where he worked with Frédo Durand and Saman Amarasinghe. His work focuses on high-efficiency computer graphics, at the intersection of graphics with systems, architecture, and compilers. He [...]
Visual Representation and Metrics for Face Classification with Image Sets
Event Location: NSH 1507Bio: Shiguang Shan is now a visiting scholar with Dr. Alex Hauptmann at CMU. He received Ph.D. degree in computer science from the Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS), Beijing, China, in 2004. He then joined ICT, CAS and has been a Professor since 2010. He is now [...]
Learning a Recurrent Visual Representation for Image Caption Generation
Event Location: NSH 1507Bio: Xinlei Chen is a PhD student in the Language Technologies Institute at Carnegie Mellon University, where he is supervised by Abhinav Gupta. He holds an Bachelor's degree in Computer Science from Zhejiang University, China. His research focuses on the intersection of computer vision and natural language processing and he is particularly [...]
Visual Parsing with Weak Supervision
Event Location: NSH 1507Bio: Jia Xu is a PhD candidate of the Computer Sciences Department at the University of Wisconsin-Madison, working with Prof. Vikas Singh, Prof. Jerry Zhu, and Prof. Chuck Dyer. He was a visiting student at University of Toronto in Summer 2014 and at TTI-Chicago during Summer 2013, both working with Prof. Raquel [...]
CANCELED 3D Vision in a Changing World
Event Location: NSH 1507Bio: Andrew Fitzgibbon is a principal researcher in the computer vision group at Microsoft Research Cambridge. He is best known for his work on 3D vision, having been a core contributor to the Emmy-award-winning 3D camera tracker "boujou" (www.boujou.com) and Kinect for Xbox 360, but his interests are broad, spanning computer vision, [...]
Commonsense, Vision and Language
Event Location: NSH 1507Bio: C. Lawrence Zitnick is a senior researcher in the Interactive Visual Media group at Microsoft Research, and is an affiliate associate professor at the University of Washington. He is interested in a broad range of topics related to object recognition, the semantic interpretation of visual scenes, and methods for gathering commonsense [...]
Toward BIGVISION
Event Location: NSH 1507Bio: Alex Berg's research concerns computational visual recognition. He has worked on general object recognition in images, action recognition in video, human pose identification in images, image parsing, face recognition, image search, and machine learning for computer and human vision. He co-organizes the ImageNet Large Scale Visual Recognition Challenge, and organized the [...]
Segmenting smoke, trees and waterfalls – Spatiotemporal filters and distance learning to segment dynamic textures in videos of natural scenes
Event Location: NSH 1507Bio: Damien Teney recently joined Carnegie Mellon University as a post-doctoral researcher with a fellowship of the Belgian-American Educational Foundation. His research interests are at the intersection of computer vision, machine learning, and robotics. He is currently working with Martial Hebert, focusing on visual scene understanding and semantic segmentation. He previously worked [...]
Language and Perceptual Categorization in Computer Vision
Event Location: NSH 1507Bio: Vicente Ordonez is a PhD student in the Department of Computer Science at the University of North Carolina at Chapel Hill. He holds an MS from Stony Brook University and an engineering degree from the Escuela Superior Politecnica del Litoral in Ecuador. His research interests are at the at the intersection [...]
Detection, segmentation and fine-grained localization
Event Location: NSH 1507Bio: I am a fifth year graduate student with Prof. Jitendra Malik in the Vision group at University of California Berkeley. I did my undergraduate at the Indian Institute of Technology, Delhi. My interests are in Computer Vision and Machine learning. I am funded by a Microsoft Research Fellowship.Abstract: Object recognition in [...]
Designing and Overcoming Challenges in Large-Scale Object Detection
Event Location: NSH 3305Bio: Olga Russakovsky (http://ai.stanford.edu/~olga) is a PhD student at Stanford University advised by Professor Fei-Fei Li. Her main research interests are in large-scale object detection and recognition. For the past two years she has been the lead organizer of the international ImageNet Large Scale Visual Recognition Challenge which has been featured in [...]
Words, Pictures, and Imagination
Event Location: NSH 1109Bio: Devi Parikh is an Assistant Professor in the Bradley Department of Electrical and Computer Engineering at Virginia Tech (VT) and an Allen Distinguished Investigator of Artificial Intelligence. She leads the Computer Vision Lab at VT, and is also a member of the Virginia Center for Autonomous Systems (VaCAS) and the VT [...]
Hedging Against Uncertainty via Multiple Diverse Predictions
Event Location: NSH 1109Bio: Dhruv Batra is an Assistant Professor at the Bradley Department of Electrical and Computer Engineering at Virginia Tech, where he leads the VT Machine Learning & Perception group. He is a member of the Virginia Center for Autonomous Systems (VaCAS) and the VT Discovery Analytic Center (DAC). Prior to joining VT, [...]
Towards Next Generation 3D Cameras
Event Location: NSH 1507Bio: Mohit Gupta will start as an assistant professor in the CS department at the University of Wisconsin-Madison in January ’16. He is currently a research scientist in the CAVE lab at Columbia University. He received a B.Tech. in computer science from Indian Institute of Technology Delhi in 2003, an M.S. from [...]
Inverting RANSAC: Global Model Detection via Inlier Rate Estimation
Event Location: NSH 1507Bio: Simon Korman is a Ph.D. student at the Department of Electrical Engineering at Tel-Aviv university, where he is advised by Professor Shai Avidan. Simon holds a B.Sc. in Mathematics and Computer Science from the Hebrew University of Jerusalem and an M.Sc. in Mathematics and Computer Science from the Weizmann Institute of [...]
From learning visual relationships to perceptual and cognitive organization
Event Location: NSH 1507Bio: Phillip Isola is a PhD student in Brain & Cognitive Sciences at MIT, under the supervision of Ted Adelson. He studies a mix of human and computer vision.Abstract: No pixel is an island. It is connected to the rest of the visual visual by a web of similarities, associations, and other [...]
Comprehensive Human State Modeling and Its Applications
Event Location: NSH 1507Bio: Ajay Divakaran, Ph.D., is a Program Director and leads the Vision and Multi-Sensor group in SRI International’s Vision and Learning Laboratory. Divakaran is currently the principal investigator for a number of SRI research projects. His work includes multimodal modeling and analysis of affective, cognitive, and physiological aspects of human behavior, interactive [...]
Grounding text, maps, and images to 3D geometry
Event Location: NSH 1305Bio: Bryan Russell is a Research Scientist in the Creative Technologies Lab at Adobe Research in San Francisco. He received his Ph.D. from MIT in the Computer Science and Artificial Intelligence Laboratory in 2008 under the supervision of Professors William T. Freeman and Antonio Torralba. He was a post-doctoral fellow from 2008-2010 [...]
Integrating Image Clustering and Codebook Learning
Event Location: NSH 1507Bio: Pengtao Xie is a graduate student in the Language Technologies Institute, working with Professor Eric Xing. His primary research interests lie in latent space models and large scale distributed machine learning. He received a M.E. from Tsinghua University in 2013 and a B.E. from Sichuan University in 2010. He is the [...]
Direct SLAM and 3D Reconstruction in real-time
Event Location: NSH 1507Bio: Jakob Engel received his Bachelor degree in Computer Science in 2009 and his Master degree in December 2011 at the Technical University of Munich (Germany). He received the SIEMENS Award for the best Master's Thesis 2012 for his work on Autonomous Camera-Based Navigation of a Quadrocopter. Since September 2012 he is [...]
Weakly supervised learning from images and videos
Event Location: NSH 1305Bio: Cordelia Schmid holds a M.S. degree in Computer Science from the University of Karlsruhe and a Doctorate, also in Computer Science, from the Institut National Polytechnique de Grenoble (INPG). Her doctoral thesis on "Local Greyvalue Invariants for Image Matching and Retrieval" received the best thesis award from INPG in 1996. She [...]
Weakly Supervised Video Understanding
Event Location: NSH 1305Abstract: This talk addresses the problem of understanding the visual content of videos using a weak form of supervision such as the textual information available in television or film scripts. I will discuss two instances of this problem, the joint localization and identification of movie characters and their actions, and the assignment [...]
Digital Humans
Event Location: NSH 1507Bio: Sofien Bouaziz is a PhD student in the Computer Graphics and Geometry Laboratory at École Polytechnique Fédérale de Lausanne (EPFL). He received an MSc degree in Computer Science from EPFL in 2009 and completed his master thesis at the Imaging Group of Mitsubishi Electric Research Laboratories, where he developed computer vision [...]
Photo Sequencing v.s. Feature Matching in CrowdCam Images
Event Location: NSH 1507Bio: Prof. Yael Moses, from the Efi Arazi School of Computer Science at the Interdisciplinary Center (IDC), Herzliya Israel, received a Ph.D. in computer science from the Weizmann Institute, Israel. Her early work concentrated on theoretical aspects of object recognition. Recently, she has been focusing on various aspects of multi-camera systems and [...]
Collaborative Representation for Person Re-identification
Event Location: NSH 1507Bio: Yang Wu received a BS degree and a Ph.D degree from Xi'an Jiaotong University in 2004 and 2010, respectively. From Sep. 2007 to Dec. 2008, he was a visiting student in the GRASP lab at University of Pennsylvania. From 2011 to 2014, he was a program specific researcher at the Academic [...]
Do Deep Nets Really Need To Be Deep?
Event Location: NSH 1507Bio: Rich Caruana is a Senior Researcher at Microsoft Research. Before joining Microsoft, Rich was on the faculty at the Computer Science Department at Cornell University, at UCLA’s Medical School, and at CMU’s Center for Learning and Discovery (CALD). Rich’s Ph.D. is from Carnegie Mellon University, where he worked with Tom Mitchell [...]
Towards context-ready tagging: Solutions and open challenges of implicit tagging in the real world
Event Location: NSH 1507Bio: Hayley Hung is an Assistant Professor and Delft Technology Fellow in the Pattern Recognition and Bioinformatics group at TU Delft, The Netherlands, since 2013. Between 2010-2013, she held a Marie Curie Intra-European Fellowship at the Intelligent Systems Lab at the University of Amsterdam. Between 2007-2010, she was a post-doctoral researcher at [...]
Unconstrained 3D Face Reconstruction
Event Location: NSH 3305Bio: Xiaoming Liu is an Assistant Professor at the Department of Computer Science and Engineering of Michigan State University. He received the Ph.D. degree in Electrical and Computer Engineering from Carnegie Mellon University in 2004. Before joining MSU in Fall 2012, he was a research scientist at General Electric (GE) Global Research. [...]
Removing Common Assumptions from Large Scale Structure-from-Motion
Event Location: NSH 1507Bio: Chris Sweeney is currently a PhD candidate in the 4 Eyes Lab at the University of California, Santa Barbara. His research interests include multi-view geometry, structure from motion, and augmented reality. His main research interest is in using unorganized photo collections from the internet to create a complete and up-to-date 3D [...]
The joint image variety
Event Location: NSH 1507Bio: Matthew Trager is a second year PhD student at Inria/ENS Paris. In 2014 he received a master's in applied mathematics (Degree in Mathematics, Machine Learning and Computer Vision "MVA") from École Normale Supérieure de Cachan. Previously, he received a master's in pure mathematics from Scuola Normale Superiore in Pisa. His research [...]
My Adventured with Bayes: In search of optimal solutions in machine learning, computer vision and beyond
Event Location: NSH 1507Bio: Aleix M. Martinez is a Professor in the Department of Electrical and Computer Engineering at The Ohio State University (OSU), where he is the founder and director of the Computational Biology and Cognitive Science Lab. He is also affiliated with the Department of Biomedical Engineering and to the Center for Cognitive [...]
OpenDR: An Approximate Differentiable Renderer
Event Location: NSH 1507Bio: Matthew Loper is a research engineer at Industrial Light and Magic, and a PhD candidate at the University of Tuebingen. He received his ScM from Brown University in 2008. His current research interests include differentiable rendering and statistical body shape modeling.Abstract: Inverse graphics attempts to take sensor data and infer 3D [...]
Context-sensitive Dynamic Ordinal Regression for Human Facial Behaviour Analysis
Event Location: NSH 1507Bio: Ognjen Rudovic received a PhD from Department of Computing, Imperial College London, UK, in 2014, an MSc degree in Computer Vision and Artificial Intelligence from Computer Vision Centre (CVC), Barcelona, Spain, in 2008, and BSc in Automatic Control Theory from Electrical Engineering Dept., University Of Belgrade, Serbia, in 2007. He is [...]
Towards Large-scale Video Understanding
Event Location: NSH 1507Bio: Chen Sun is a Ph.D. candidate in the Computer Vision group at University of Southern California, advised by Prof. Ram Nevatia. His research interest includes Computer Vision and Machine Learning, with a focus on large-scale video understanding. Chen got his bachelor degree in Computer Science at Tsinghua University, Beijing. He has [...]
Learning optimal seeds for diffusion based salient object detection
Event Location: NSH 1507Bio: Vijay Mahadevan is Research Scientist at Yahoo Labs in Sunnyvale, CA. He received the Ph.D degree in Electrical Engineering from UC San Diego. He also has an M.S. from Rensselaer, and a B.Tech from Indian Institute of Technology, Madras. His interests are in the areas of computer vision, machine learning and [...]
Using Motion to Understand Objects in the Real World
Event Location: NSH 1507Bio: David Held is a Computer Science Ph.D. student at Stanford doing research at the intersection of robotics, computer vision, and machine learning. He is co-advised by Sebastian Thrun and Silvio Savarese. David has also interned at Google, working on the self-driving car project. Before Stanford, he worked as a software developer [...]
Constrained Convolutional Neural Networks for Weakly Supervised Segmentation
Event Location: NSH 1507Bio: Philipp is a postdoctoral researcher at UC Berkeley. He received his PhD from Stanford University in 2014 under the supervision of Vladlen Koltun. His main research interests lie in Computer vision, Machine learning and Computer Graphics. He is particularly interested in image segmentation and deep learning. Abstract: In this talk, I'll [...]
Learning deconvolution network for semantic segmentation: DeconvNet and DecoupledNet
Event Location: NSH 1507Bio: Bohyung Han Associate Professor Dept. of Computer Science and Engineering POSTECH, Korea Bohyung Han received the B.S. and M.S. degrees from the Department of Computer Engineering at Seoul National University, Korea, in 1997 and 2000, respectively, and the Ph.D. degree from the Department of Computer Science at the University of Maryland, [...]
Novel Cues for Geocalibration: Cloudy Days, Rainbows, and More
Event Location: NSH 1507Bio: Nathan Jacobs earned a PhD in Computer Science at Washington University in St. Louis (2010). Since then, he has been an Assistant Professor of Computer Science at the University of Kentucky. Dr. Jacobs' research area is computer vision; his speciality is developing learning-based algorithms and systems for processing large-scale image collections. [...]
Unsupervised Deep Feature Learning from Video
Event Location: NSH 1507Bio: Ross Goroshin has recently obtained his PhD under Yann LeCun from the Department of Computer Science at New York University's Courant Institute. He received a masters in Electrical & Computer Engineering from Georgia Tech and his bachelors in Electrical Engineering from Concordia University in Montreal, Canada. Abstract: Many recent empirical successes [...]
Opportunities Building Visual Technology & Computer Vision Startups
Event Location: GHC 6115Bio: Evan Nisselson invests in early stage companies via LDV Capital with a focus in: Visual Technologies from “Capture to Smile.” He is a serial entrepreneur and digital media expert of over 20 years. He organizing the LDV Vision Summit and LDV Community dinners. He mentors at Seedcamp, 500 Startups, and Founders [...]
Design in context: bringing the physical world into CAD tools
Event Location: NSH 1507Bio: Ryan Schmidt is a Research Scientist and head of the Design & Fabrication Group at Autodesk Research in Toronto, Canada. He is the creator of several novel 3D design tools, including Meshmixer, which was acquired by Autodesk in 2011. At Autodesk he has evolved Meshmixer into one of the standard tools [...]
CANCELEDThe Depth of Our Understanding: Vision, Language, and Humor
Event Location: NSH 1507Bio: C. Lawrence Zitnick is a principal researcher in the Interactive Visual Media group at Microsoft Research, and is an affiliate associate professor at the University of Washington. He is interested in a broad range of topics related to visual object recognition. His current interests include object detection, semantically interpreting visual scenes, [...]
Rich Representations for Parsing Visual Scenes
Event Location: NSH 1507Bio: Greg is an Assistant Professor at TTI-Chicago, a philanthropically endowed academic computer science institute located on the University of Chicago campus, where he works on computer vision and machine learning. He also holds a part-time faculty appointment at the University of Chicago Department of Computer Science. Prior to coming to TTI-Chicago, [...]
Semantic Component Analysis
Event Location: NSH 1507Abstract: We propose a novel formulation for component analysis that allows for rich instance-level constraints that encourage semantic interpretability of the learned components. Even with simple features and intuitive spatial consistency priors, our method produces accurate, semantically-meaningful image segmentations both with and without supervision.
Webly supervised learning of convolutional networks
Event Location: NSH 1507Bio: I am a PhD student at Language Technology Institute, Carnegie Mellon University, from fall 2012. I am working with Prof. Abhinav Gupta on joint learning with language and vision and life-long learning. I am also working with Prof. Tom Mitchell in CMU. Recently, I just finished my internship in MSR with [...]
Unsupervised Visual Representation Learning by Context Prediction
Event Location: NSH 1507Bio: I'm a sixth year PhD Student in the Machine Learning Department at CMU, working with Alyosha Efros and Abhinav Gupta. I graduated from CMU in 2010 with a B.S. in computer science and cognitive science, with a minor in neural computation, completing an undergraduate thesis with Tai Sing Lee. I'm interested [...]
Panoptic Studio: A Massively Multiview System for Social Motion Capture
Event Location: NSH 1507Bio: I am a Ph.D. student in the Robotics Institute at Carnegie Mellon University, under the supervision of Yaser Sheikh. During summer 2015, I interned at Disney Research Zurich where I worked with Thabo Beeler and Derek Bradley. Before joining CMU, I spent three years as a researcher at ETRI, a government-funded [...]
Building models of cell organization, differentiation and perturbation directly from microscope images
Event Location: NSH 1507Bio: Dr. Robert F. Murphy is the Ray and Stephanie Lane Professor of Computational Biology and Head of the Computational Biology Department in the School of Computer Science at Carnegie Mellon University. He is also Professor of Biological Sciences, Biomedical Engineering, and Machine Learning at Carnegie Mellon, Honorary Professor of Biology at [...]
Human Activity Recognition from a Robot’s Viewpoint
Event Location: NSH 1507Bio: Michael S. Ryoo is an Assistant Professor of the School of Informatics and Computing at Indiana University. His research interest is within the areas of Computer Vision and Human-Robot Interaction, with a particular emphasis on human activity recognition, first-person vision, and wearable/ubiquitous cameras. Before joining IU, Dr. Ryoo was a staff [...]
The human side of computer vision
Event Location: NSH 1507Bio: Olga Russakovsky (http://cs.cmu.edu/~orussako) is a postdoctoral research fellow at Carnegie Mellon University. She recently completed a PhD in computer science at Stanford advised by Prof. Fei-Fei Li. Her research is in computer vision, closely integrated with machine learning and human-computer interaction. She led the ImageNet Large Scale Visual Recognition Challenge effort [...]
Brighter, Faster, Cheaper: Finding or Creating Light Fields for Visual Computing
Event Location: NSH 1507Bio: Robert Pless is a Professor of Computer Science and Engineering at Washington University in St. Louis, where he founded and directs the Media and Machines Lab. His research focus are big-data and geometric approaches to Visual Computing, with applications to social justice and environmental measurement. Dr. Pless has a Bachelors Degree [...]
Collective Insight: Crowd-driven Image Understanding
Event Location: NSH 1507Bio: Genevieve is a PhD Candidate in Computer Vision at Brown University. Her work on crowd-driven visual classification was recently awarded runner-up for Best Paper at the AAAI Conference on Human Computation (HCOMP). She built and maintains the SUN Attribute dataset, a widely used resource for scene understanding. Genevieve received her master's [...]
Towards A Physical and Human-Centric Understanding of Images
Event Location: Newell Simon Hall 1507Bio: David Fouhey is a Ph.D. student at the Robotics Institute of Carnegie Mellon University, where he is advised by Abhinav Gupta and Martial Hebert. His research interests include computer vision and machine learning with a particular focus on scene understanding. David's work has been supported by both NSF and [...]
3D Object Representations for Recognition
Event Location: NSH 1507Bio: Yu Xiang is a Postdoctoral Researcher in the Computer Science Department at Stanford University. His research focuses on understanding objects and scenes from images and videos, with emphasis on recognizing both semantic and 3D geometric properties of objects and scenes. His current work attempts to develop 3D object representation and recognition [...]
Computational Acoustic Design: From the Virtual to the Real
Event Location: Newell Simon Hall 1507Bio: Changxi Zheng is an Assistant Professor of Computer Science at Columbia University. Prior to joining Columbia, he received his M.S. and Ph.D. from Cornell University, and his B.S. from Shanghai Jiaotong University. His research spans computer graphics, physically-based simulation, computational design, computational acoustics, scientific computing and robotics, with a [...]
Scene Understanding from RGB-D Images
Event Location: 1507 Newell Simon HallBio: Saurabh Gupta is a Ph.D. student at UC Berkeley, where he is advised by Jitendra Malik. His research interests include computer vision and machine learning. During his PhD he has studied the problem of scene understanding from RGB-D images. His work has been supported by the Berkeley Fellowship and [...]
Inside-out: First Person Vision for Personalized Intelligence
Event Location: Gates 2109Bio: Jianbo Shi studied Computer Science and Mathematics as an undergraduate at Cornell University where he received his B.A. in 1994. He received his Ph.D. degree in Computer Science from University of California at Berkeley in 1998. He joined The Robotics Institute at Carnegie Mellon University in 1999 as a research faculty, [...]
Reconstruction and Understanding of Indoor Environments
Event Location: Gates 8102Bio: Matthias Niessner is a visiting assistant professor at Stanford University. Previous to his appointment at Stanford, he earned his PhD from the University of Erlangen-Nuremberg, Germany under the supervision of Günther Greiner. His research focuses on different fields of computer graphics and computer vision, including the reconstruction and semantic understanding of [...]
Towards Understanding Stories from Videos
Event Location: Newell Simon Hall 1507Bio: Sanja Fidler is an Assistant Professor at the Department of Computer Science, University of Toronto. Previously she was a Research Assistant Professor at TTI-Chicago, a philanthropically endowed academic institute located in the campus of the University of Chicago. She was a postdoctoral fellow at University of Toronto during 2011-2012. [...]
Sound provides supervision for visual learning
Event Location: Newell Simon Hall 1507Bio: Andrew Owens is a graduate student at the MIT Computer Science and Artificial Intelligence Laboratory, working under the supervision of Bill Freeman and Antonio Torralba. Before that, he obtained his B.A. in Computer Science at Cornell University in 2010. He is a recipient of a Microsoft Research PhD Fellowship, [...]
Human Manipulation Action Understanding for Cognitive Robots
Event Location: Newell Simon Hall 1507Bio: Dr. Yezhou Yang is a Postdoctoral Research Associate at the Computer Vision Lab and the Automation, Robotics and Cognition (ARC) Lab, with the University of Maryland Institute for Advanced Computer Studies, working with his PhD advisors: Prof. Yiannis Aloimonos and Dr. Cornelia Fermuller. His main interests lie in Cognitive [...]
Deep Learning for Robot Manipulation via Simulation
Event Location: Newell Simon Hall 1507Bio: Ed Johns is a Dyson Fellow at Imperial College London, working on computer vision, robotics and machine learning. He received a BA and MEng from Cambridge University, followed by a PhD in visual recognition and localisation from Imperial College London. After post-doctoral work at University College London, he then [...]
Matrix Completion: A vision-oriented perspective
Event Location: Newell Simon Hall 1507Bio: Xavier Alameda-Pineda received the M.Sc. degree in mathematics and telecommunications engineering from the Universitat Politècnica de Catalunya – BarcelonaTech in 2008 and 2009 respectively, the M.Sc. degree in computer science from the Université Joseph Fourier and Grenoble INP in 2010, and the Ph.D. degree in mathematics/computer science from the [...]
What can we do with motion cues?
Event Location: Newell Simon Hall 1507Bio: Karteek Alahari is an Inria permanent researcher (chargé de recherche) since October 2015. He has been at Inria since 2010, initially as a postdoctoral fellow in the WILLOW team in Paris, and then on a starting research position in Grenoble since September 2013. Dr. Alahari's PhD from Oxford Brookes [...]
Understanding Social and Physical Interactions from First Person Cameras
Event Location: Newell Simon Hall 1507Bio: Hyun Soo Park is an Assistant Professor at the Department of Computer Science and Engineering, the University of Minnesota. He is interested in understanding human visual sensorimotor behaviors from first person cameras. Prior to the UMN, he was a Postdoctoral Fellow working with Jianbo Shi at University of Pennsylvania. [...]
Instrumented and Connected: Designing Next-Generation Learning Experiences
Event Location: Newell Simon Hall 1507Bio: Tovi Grossman is a Distinguished Research Scientist at Autodesk Research, located in downtown Toronto. Dr. Grossman’s research is in HCI, focusing on input and interaction with new technologies. In particular, he has been exploring how emerging technologies, such as wearables, the Internet of Things, and gamification can be leveraged [...]
Simulation-guided Interactive Exploration of Functional Design
Event Location: Newell Simon Hall 1507Bio: Nobuyuki Umetani is a research scientist at Autodesk Research. Previously, he was a postdoctoral researcher in Autodesk Research and Disney Research Zurich. He received his Ph.D. degree in 2012 from The University of Tokyo under supervision of Takeo Igarashi. He also spent one year in Columbia University and in [...]
Molecular resolution structural pattern mining inside single cells
Event Location: Newell Simon Hall 1507Bio: Dr. Min Xu is an Assistant Research Professor of Computational Biology at the Computational Biology Department in the School of Computer Science at Carnegie Mellon University. He received degrees in Computational Biology, Computer Science, and Applied Mathematics. He has more than 16 years of research experience in various Computational [...]
Character Animator
Bio: Jovan Popovic is a Senior Principal Scientist at Adobe Systems. After receiving bachelor's degrees in mathematics and computer science in 1995, he attended the University of Washington and Carnegie Mellon University, where he earned a doctoral degree for his work in computer animation and geometric modeling. He was on the faculty at the Massachusetts [...]
Exploring and Modifying Spatial Variations in a Single Image
Event Location: Gates Hillman 5222Bio: Tali Dekel is currently a Research Scientist at Google, working on developing computer vision and computer graphics algorithms. Before Google, she was a Postdoctoral Associate at the Computer Science and Artificial Intelligence Lab (CSAIL) at MIT, working with Prof. William T. Freeman. Tali completed her Ph.D studies at the school [...]
Reasoning About Our Visual World
Event Location: Newell Simon Hall 1507Bio: C. Lawrence Zitnick is a research manager at Facebook AI Research, and an affiliate associate professor at the University of Washington. He is interested in a broad range of topics related to artificial intelligence including object recognition, the relation of language and imagery, and methods for gathering common sense [...]
Fine-Scale Structure Design for 3D Printing
Event Location: Newell Simon Hall 1507Bio: Julian Panetta is a PhD candidate at NYU's Courant Institute, where he is advised by Denis Zorin. Julian is interested in simulation and optimal designproblems, specifically focusing on applications for 3D printing. Before joining NYU, he received his BS in computer science from Caltech and did research at NASA's [...]
Visual Correspondences in the Big Data Era
Event Location: Newell Simon Hall 1507Bio: Qixing Huang is an assistant professor at the University of Texas at Austin. He obtained his PhD in Computer Science from Stanford University and his MS and BSin Computer Science from Tsinghua University. He was a research assistant professor at Toyota Technological Institute at Chicago before joining UT Austin. [...]
Computational Perception of Geometric and Physical Object Properties
Event Location: Newell Simon Hall 1507Bio: Jiajun Wu is a third-year Ph.D. student at Massachusetts Institute of Technology, advised by Professor Bill Freeman and Professor Josh Tenenbaum. His research interests lie on the intersection of computer vision, machine learning, and computational cognitive science. Before coming to MIT, he received his B.Eng. from Tsinghua University, China, [...]
Me Car, You Human: Understanding Human Activity for Intelligent Collaborative Robotic Vehicles
Eshed Ohn-Bar Postdoctoral Researcher, University of California, San Diego Abstract The goal of my research is to develop human-centered algorithms for intelligent and autonomous systems. The research emphasizes modeling the perception, intent, and behavior of humans inside and around a vehicle. Over a decade has passed since the DARPA Grand Challenges, and the way in [...]
Attention and Activities in First Person Vision
Event Location: Newell Simon Hall 1507Bio: Yin Li is currently a doctoral candidate in the School of Interactive Computing at the Georgia Institute of Technology. His research interests lie at the intersection of computer vision and mobile health. Specifically, he creates methods and systems to automatically analyze first person videos, known as First Person Vision [...]
TBA: Yin Li
Embodied learning for visual recognition
Event Location: Gates 7101Bio: Dinesh Jayaraman is a PhD candidate in Kristen Grauman's group at UT Austin. His research interests are broadly in visual recognition and machine learning. In the last few years, Dinesh has worked on visual learning and active recognition in embodied agents, unsupervised representation learning from unlabeled video, visual attribute prediction, and [...]
Assistive technology for wayfinding, information access, and public transit
Event Location: Newell Simon Hall 1507Bio: Roberto Manduchi is a Professor of Computer Engineering at the University of California, Santa Cruz, where he conducts research in the areas of computer vision and sensor processing with applications to assistive technology. Prior to joining UCSC in 2001, he worked at the NASA Jet Propulsion Laboratory and at [...]
Intuitive Physics & Intuitive Behavior
Event Location: Newell Simon Hall 1507Bio: Pulkit is a PhD Student in the department of Computer Science at UC Berkeley. His research focuses on computer vision, robotics and computational neuroscience. He is advised by Dr. Jitendra Malik. Pulkit completed his bachelors in Electrical Engineering from IIT Kanpur and was awarded the Director’s Gold Medal. He is a recipient of Fulbright Science [...]
The lifetime of an object – an object’s perspective onto interactions
Event Location: Newell Simon Hall 1507Bio: Lecturer (Assistant Professor) in Computer Vision at the University of Bristol. Received her PhD from the University of Leeds (2009). Dima's research interests are in the automatic understanding of object interactions, actions and activities using static and wearable visual (and depth) sensors. Dima co-chaired BMVC 2013, is area chair [...]
The lifetime of an object – an object’s perspective onto interactions
Dima Damen Assistant Professor, University of Bristol, United Kingdom April 10, 2017, 3:00-4:00 p.m., Newell Simon Hall 1507 Abstract As opposed to the traditional notion of actions and activities in computer vision, where the motion (e.g. jumping) or the goal (e.g. cooking) is the focus, I will argue for an object-centred perspective onto actions and [...]
Computer Vision @ Scale
Manohar Paluri Research Lead, Facebook Abstract Over the past 5 years the community has made significant strides in the field of Computer Vision. Thanks to large scale datasets, specialized computing in form of GPUs and many breakthroughs in modeling better convnet architectures Computer Vision systems in the wild at scale are becoming a reality. At [...]
Towards scaling video understanding
Serena Yeung Ph.D. Student, Stanford University Abstract The quantity of video data is vast, yet our capabilities for visual recognition and understanding in videos lags significantly behind that for images. In this talk, I will discuss the challenges of scale in labeling, modeling, and inference behind this gap. I will then present three works addressing [...]
Visual Analysis of Dense Crowds
Event Location: Newell Simon Hall 1507Bio: Haroon Idrees is a postdoctoral researcher in the Center for Research in Computer Vision (CRCV) at the University of Central Florida (UCF). He is interested in machine vision and learning, with focus on crowd analysis, action recognition, multi-camera and airborne surveillance, as well as deep learning and multimedia content [...]
Haroon Idrees: Visual Analysis of Dense Crowds
Haroon Idrees Post Doc Associate, Center for Research in Computer Vision, University of Central Florida (UCF) Abstract Automated analysis of dense crowds is a challenging problem with far-reaching applications in crowd safety and management, as well as gauging political significance of protests and demonstrations. In this talk, I will first describe a counting approach which [...]
Toward Deep Geometric Image Understanding
Event Location: Newell Simon Hall 1507Bio: Jia Deng is an Assistant Professor of Computer Science and Engineering at the University of Michigan. His research focus is on computer vision and machine learning, in particular, achieving human-level visual understanding by integrating perception, cognition, and learning. He received his Ph.D. from Princeton University and his B.Eng. from [...]
Interactive Scene Understanding
Abstract Despite recent progress, AI is still far from understanding the physics of the world, and there is a large gap between the abilities of humans and the state-of-the-art AI methods. In this talk, I will focus on physics-based scene understanding and interactive visual reasoning, which are crucial next steps in computer vision and AI. [...]
Multi-class Boosting in Computer Vision
Abstract: Boosting classifiers have been extensively used for learning multi-view single objects detectors (e.g. faces, cars or pedestrians) or in multiple object categories detectors. Object detection has been evolving from being specific for a given object category to multi-view or even being able to detect multiple categories at the same time. The usual framework for [...]
Fun with Fluids
Abstract: Visual simulation of fluids has become an indispensable tool for computer graphics. Many fluid phenomena can be simulated by solving Navier-Stokes equations. In computer graphics, the NS equations are mostly used for simulating smoke, water and fire. However, it is useful for other different purposes. In this talk, we show our usage of [...]
Challenges Facing Computational Face
Abstract: Recent advances in computational face research make possible a growing range of scientific, behavioral, and commercial applications. Many companies are focusing on the future of computational face products and services, but number of critical research questions remain to be solved. These include 3D face alignment from 2D image, face analysis under extreme pose variation [...]
Bayesian Eigenobjects: A Unified Framework for 3D Robot Perception
Abstract: Robot-object interaction requires several key perceptual building blocks including object pose estimation, object classification, and partial-object completion. These tasks form the perceptual foundation for many higher level operations including object manipulation and world-state estimation. Most existing approaches to these problems in the context of 3D robot perception assume an existing database of objects [...]
Two Tales about Image Classification
Abstract: This talk tells two tales about image-classification systems, both of which are motivated by the real-world deployment of such systems. The first tale introduces a new convolutional neural network architecture, called multi-scale DenseNets, with the ability to adapt dynamically to computational resource limits at inference time. The network uses progressively growing multi-scale convolutions, dense [...]
Dense 3D Shape Reconstruction of Complex Dynamic Scene with a Single Monocular Camera
Abstract: In this talk, I will describe our recent work (presented at ICCV 2017) on monocular camera based 3D geometry reconstruction of a non-rigid dynamic scene. We aim to answer an open question in multi-view geometry, namely, "Is it possible to recover the 3D structure of a complex dynamic environment from two image frames captured by [...]
Learning to Visually Reason
Abstract: Visual reasoning is a core capability of artificial intelligence. It is a necessity for effective communication, planning, and for question/answering tasks. In this talk, I discuss some recent explorations into visual reasoning for question/answering, game playing and dialog. I also describe our new reinforcement learning platform ELF; an Extensive, Lightweight and Flexible research platform [...]
Towards Lifelong Robot Learning
Abstract: Google Brain Robotics vision is to leverage learning to push the field of robotics forward. As such, we have engaged in research ranging in application from navigation to grasping and approach from deep RL to learning from demonstration. Fundamentally, our research is built around the core idea of lifelong learning. Our long term goal [...]
Comparing apples and oranges: Off-road pedestrian detection on the NREC agricultural person-detection dataset
Abstract: Person detection from vehicles has made rapid progress recently with the advent of multiple high-quality datasets of urban and highway driving, yet no large-scale benchmark has been available for the same problem in off-road or agricultural environments. In this talk, we present the NREC Agricultural Person-Detection Dataset to spur research in these environments. It [...]
Adaptive Information Gathering via Imitation Learning
Abstract: In the adaptive information gathering problem, a robot is required to select an informative sensing location using the history of measurements acquired thus far. While there is an extensive amount of prior work investigating effective practical approximations using variants of Shannon’s entropy, the efficacy of such policies heavily depends on the geometric distribution of [...]
Learning Single-view 3D Reconstruction of Objects and Scenes
Abstract: In this talk, I will discuss the task of inferring 3D structure underlying an image, in particular focusing on two questions - a) how we can plausibly obtain supervisory signal for this task, and b) what forms of representation should we pursue. I will first show that we can leverage image-based supervision to learn [...]
Neuromorphic Event-based time oriented vision and Computation
Abstract: There has been significant research over the past two decades in developing new systems for spiking neural computation. The impact of neuromorphic concepts on recent developments in optical sensing, display and artificial vision is presented. State-of-the-art image sensors suffer from severe limitations imposed by their very principle of operation. These sensors acquire the visual [...]
Towards Goal-Driven Visually Grounded Dialog Agents
Abstract: Communication between human users and artificial intelligences is essential for human-AI cooperative tasks. For these collaborations to extend into real environments, artificial agents must be able to perceive their environment (visually, aurally, tactilely, etc.) and to communicate with humans about it in order to accomplish mutual goals. For example, a user might talk with [...]
On challenges in image generation
Abstract: Recent work has shown impressive success in automatically synthesizing new images with desired properties such as transferring painterly style, modifying facial expressions, increasing image resolution or manipulating the center of attention of the image. In this talk I will discuss two of the standing challenges in image synthesis and how we tackle them: - [...]
Learning Common Sense: a Grand Challenge for Academic AI Research
Abstract: In a world where Google, Facebook, and others possess massive proprietary data sets, and unprecedented computational power---how is a graduate student to make a dent in the universe? I’ll address this conundrum by re-visiting one of the holy grails of AI: acquiring, representing, and utilizing common-sense knowledge. Can we leverage modern methods including deep [...]
Multimodal, multilevel analysis of human behavior
Abstract: Computer analysis of human behavior is an interdisciplinary endeavor combining sensing technology, theoretical and empirical models of human behavior, pattern recognition and machine learning algorithms, and interaction sciences. The applications in this area range widely, from robotics to healthcare, from smart environments to multimedia, from security to humanitarian response. While human behaviors span different [...]
Object Detection and Tracking on Low Resolution Aerial Images
Abstract: Object tracking from an aerial platform poses a number of unique challenges including the small number of pixels representing the objects, large camera motion, and low temporal resolution. Because of these unique reasons, low resolution aerial image analysis needs to be tackled differently than the traditional image analysis both in terms of the sensors, [...]
Data-Driven Learning Towards Perceptual Organization
Abstract: Computer vision has advanced rapidly with deep learning, achieving above human performance on some classification benchmarks. At the core of the state-of-the-art approaches for image classification, object detection, and semantic/instance segmentation is sliding window classification, engineered for computational efficiency. Such piecemeal analysis of visual perception often has trouble getting details right and fails miserably [...]
Deep Representation Learning with Induced Structural Priors
Abstract: With the support of big-data and big-compute, deep learning has reshaped the landscape of research and applications in artificial intelligence. Whilst traditional hand-guided feature engineering in many cases is simplified, the deep network architectures become increasingly more complex. A central question is if we can distill the minimal set of structural priors that can [...]
Lifelong Learning via Curiosity and Self-supervision
Abstract: Humans demonstrate remarkable ability to generalize their knowledge and skills to new unseen scenarios. One of the primary reasons is that they are continually learning by acting in the environment and adapting to novel circumstances. This is in sharp contrast to our current machine learning algorithms which are incredibly narrow in only performing the [...]
Capturing and Learning Digital Humans
Abstract: The world is shifting towards a digitization of everything -- music, books, movies and news in digital form are common in our everyday lives. Digitizing human beings would redefine the way we think and communicate (with other humans and with machines), and it is necessary for many applications; for example, to transport people into virtual and augmented reality, [...]
Deformable models meet deep learning: supervised and unsupervised approaches
Abstract: In this talk I will be presenting recent work on combining ideas from deformable models with deep learning. I will start by describing DenseReg and DensePose, two recently introduced systems for establishing dense correspondences between 2D images and 3D surface models ``in the wild'', namely in the presence of background, occlusions, and multiple objects. [...]
Building Scalable Framework and Environment of Reinforcement Learning
Abstract: Deep Reinforcement Learning (DRL) has made strong progress in many tasks that are traditionally considered to be difficult, such as complete information games, navigation, architecture search, etc. Although the basic principle of DRL is quite simple and straightforward, to make it work often requires substantially more samples with more computational resource, compared to traditional [...]
Scene Understanding
Abstract: Accurate and efficient scene understanding is a fundamental task in a variety of computer vision applications including autonomous driving, human-machine interaction, and robot navigation. Reducing computational complexity and memory use is important to minimize response time and power consumption for portable devices such as robots and virtual/augmented devices. Also, it is beneficial for vehicles [...]
Relating First-person and Third-person Videos
Abstract: Thanks to the availability and increasing popularity of wearable devices such as GoPro cameras, smart phones and glasses, we have access to a plethora of videos captured from the first person perspective. Capturing the world from the perspective of one's self, egocentric videos bear characteristics distinct from the more traditional third-person (exocentric) videos. In [...]
Towards better methods of video generation
Abstract: Learning to generate future frames of a video sequence is a challenging research problem with great relevance to reinforcement learning, planning and robotics. Existing approaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this talk I will address two important aspects of video generations: (i) what [...]
Acquiring and Transferring Generalizable Vision-based Robot Skills
Abstract: In recent years, there have been great advances in policy learning for goal-oriented agents. However, there are still major challenges brought by real-world constraints for teaching highly generalizable and versatile robot policies in a cost efficient and safe manner. In this talk, I will argue that instead of aiming to teach large motion repertoires [...]
Learning to localize and anonymize objects with indirect supervision
Abstract: Computer vision has made great strides for problems that can be learned with direct supervision, in which the goal can be precisely defined (e.g., drawing a box that tightly-fits an object). However, direct supervision is often not only costly, but also challenging to obtain when the goal is more ambiguous. In this talk, I [...]
Video Compression for Recognition & Video Recognition for Compression
Abstract: Training robust deep video representations has proven to be much more challenging than learning deep image representations. One reason is: videos are huge and highly redundant. The 'true' and interesting signal often drowns in too much irrelevant data. In the first part of the talk, I will show how to train a deep network [...]
Tracking Beyond Detection
Abstract: The majority of existing vision-based methods perform multi-object tracking in the image domain. Yet, in mobile robotics and autonomous driving scenarios, pixel-precise object localization and trajectory estimation in 3D space are of fundamental importance. Furthermore, the leading paradigms for vision-based multi-object tracking and trajectory prediction heavily rely on object detectors and effectively limit tracking [...]
Exploiting Deviations from Ideal Visual Recurrence
Abstract: Visual repetitions are abundant in our surrounding physical world: small image patches tend to reoccur within a natural image, and across different rescaled versions thereof. Similarly, semantic repetitions appear naturally inside an object class within image datasets, as a result of different views and scales of the same object. We studied deviations from these [...]
Attending to Pixels, Embedding Pixels, Predicting Pixels
Abstract: Nowadays splashy applications heavily depend on meticulously annotated datasets, data-driven and learning-based methods, among which pixel labeling plays an important role yet often lacks interpretability. In this talk, I will discuss how we deal with pixels with better interpretability. Firstly, I'll introduce the pixel embedding framework that allows for clustering pixels into discrete groups [...]
Automatically Supervised Learning: Two more steps on a long journey
Abstract: I will talk about two recent pieces of work that attempt to move towards learning with less reliance on labeled data. In the first, part, I will talk about how the surrogate task of predicting the motion of objects can induce complex representations in neural networks without any labeled data. In the second part of [...]
Geometric Deep Learning for Perceiving and Modeling Humans
Abstract: Perceiving and modeling shape and appearance of the human body from single images is a severely under-constrained problem that not only requires large volumes of data, but also prior knowledge. In this talk I will present recent solutions on how deep learning can leverage on geometric reasoning to address tasks like 3D estimation of [...]
Human-Level Learning of Driving Primitives through Bayesian Nonparametric Statistics
Abstract: Understanding and imitating human driver behavior has benefited for autonomous driving in terms of perception, control, and decision-making. However, the complexity of multi-vehicle interaction behavior is far messier than human beings can cope with because of the limited prior knowledge and capability of dealing with high-dimensional and large-scale sequential data. In this talk, I [...]
Knowledge Transfer Graph for Deep Collaborative Learning
Abstract: In this talk I will present our latest research about knowledge transfer graph for Deep Collaborative Learning (DCL), which is a method that incorporates Knowledge Distillation and Deep Mutual Learning. DCL is represented by a directional graph where each model is represented by a node, and the propagation of knowledge from the source node to the [...]
Some New Designs of Convolutional and Recurrent Networks
Abstract: Convolutional networks (CNNs) and recurrent networks have driven the great engineering success of deep learning in recent years. However, as academics, we still wonder whether they are indeed the ultimate models of choice. Especially, CNNs seem unable to characterize predictive uncertainty, and they are highly dependent on small filters on small, rectangular neighborhoods. On [...]
Language and Interaction in Minecraft
Abstract: I will discuss a research program aimed at building a Minecraft assistant, in order to facilitate the study of agents that can complete tasks specified by dialogue, and eventually, to learn from dialogue interactions. I will describe the tools and platform we have built allowing players to interact with the agents and to record those interactions, and [...]
Attentive Human Action Recognition
Abstract: Enabling computers to recognize human actions in video has the potential to revolutionize many areas that benefit society such as clinical diagnosis, human-computer interaction, and social robotics. Human action recognition, however, is tremendously challenging for computers due to the subtlety of human actions and the complexity of video data. Critical to the success of [...]
Temporal Modeling and Data Synthesis for Visual Understanding
Abstract: In this talk, I will present two recent pieces of work on leveraging temporal information and synthetic data to enhance video and image understanding. In the first part, I will introduce a progressive learning framework, Spatio-TEmporalProgressive (STEP), for action detection in videos. STEP is able to more effectively make use of longer temporal information, [...]
VR facial animation via multiview image translation
Abstract: A key promise of Virtual Reality (VR) is the possibility of remote social interaction that is more immersive than any prior telecommunication media. However, existing social VR experiences are mediated by inauthentic digital representations of the user (i.e., stylized avatars). These stylized representations have limited the adoption of social VR applications in precisely those [...]
Neural Volumes: Learning Dynamic Renderable Volumes from Images
Abstract: Modeling and rendering of dynamic scenes is challenging, as natural scenes often contain complex phenomena such as thin structures, evolving topology, translucency, scattering, occlusion, and biological motion. Mesh-based reconstruction and tracking often fail in these cases, and other approaches (e.g., light field video) typically rely on constrained viewing conditions, which limit interactivity. We [...]
Towards Lightweight Real-time Hand Reconstruction in Challenging
Abstract: Humans naturally use their hands to interact and communicate with their surroundings. Reconstructing these complex and dexterous hand interactions enables sign-language recognition and translation, better assistive robots, and more immersive human-computer interaction (e.g. for AR and VR). To make hand reconstruction usable for the aforementioned applications and to a wide set of users, the [...]
Hybrid Methods for the Integration of Heterogeneous Multimodal Biomedical Data
Abstract: The prevalence of smartphones and wearable devices for health monitoring and widespread use of electronic health records have led to a surge in heterogeneous multimodal healthcare data, collected at an unprecedented scale. My research focuses on developing machine learning techniques that learn salient representations of multimodal, heterogeneous data for biomedical predictive models. The first [...]
Self-Driving Cars & AI: Transforming our Cities and our Lives
Abstract: Recent algorithmic and hardware improvements resulted in several success stories in the field of Artificial Intelligence (AI) which impact our daily lives. However, despite its ubiquity, AI is only just starting to make advances in what may arguably have the largest societal impact thus far, the nascent field of autonomous driving. At Uber ATG, [...]
Go, fastMRI, and Minecraft: Exploring the limits of AI
Abstract: The application of AI across various domains demonstrates both the promise of existing techniques but also their limitations. In this talk, I explore three recent projects and how they shed light on the progress of AI and the challenges to come. These projects include ELF OpenGo a reimplementation of AlphaZero, fastMRI for reducing the time [...]
Towards Weakly-Supervised Visual Understanding
Abstract: Learning with weak and self-supervisions recently emerged as compelling tools towards leveraging vast amounts of unlabeled or partially-labeled data. In this talk, I will present some of the latest advances in weakly-supervised visual scene understanding from NVIDIA. Specifically, I will summarize and discuss some challenges and potential solutions in weakly-supervised learning, and introduce our [...]
Imaging without focusing: A computational approach to miniaturizing cameras
Abstract: Miniaturization of cameras is key to enabling new applications in areas such as connected devices, wearables, implantable medical devices, in vivo microscopy, and micro-robotics. Recently, lenses were identified as the main bottleneck in miniaturization of cameras. Standard smaller lens-system camera modules have a thickness of about 10 mm or higher, and reducing the size [...]
Towards photo-realistic face digitization from monocular videos
Abstract: Recent advances in face capture now enable digitizing high-quality 3D faces for the entertainment industry. Standardized digitization solutions, however, require tailor-made capture systems and extensive manual work, making them expensive and hard to deploy. With the advent of commodity sensors, new lightweight approaches that push the boundaries of human digitization have been introduced, slowly [...]
Reconstructing 3D Human Avatars from Monocular Images
Abstract: Statistical 3D human body models have helped us to better understand human shape and motion and already enabled exciting new applications. However, if we want to learn detailed, personalized, and clothed models of human shape, motion, and dynamics, we require new approaches that learn from ubiquitous data such as plain RGB-images and video. I [...]
Reasoning about complex media from weak multi-modal supervision
Abstract: In a world of abundant information targeting multiple senses, and increasingly powerful media, we need new mechanisms to model content. Techniques for representing individual channels, such as visual data or textual data, have greatly improved, and some techniques exist to model the relationship between channels that are “mirror images” of each other and contain [...]
Building Trust in Real World Applications of Vision Based Machine Learning
Abstract: In all machine learning problems, there is an explicit trade off between cost and benefit. In real world vision problems, this optimization becomes increasingly difficult since those trade offs directly impact technology and product development as well as business strategy. For any successful business case, it is critical that the cost/benefit trade offs in [...]
Knowledge Infused Deep Learning
Abstract: This talk is motivated by the following thesis: Background knowledge is key to intelligent decision making. While deep learning methods have made significant strides over the last few years, they often lack the context in which they operate. Knowledge Graphs (and more generally multi-relational graphs) provide a flexible framework to capture and represent knowledge [...]
Learning to Reconstruct 3D Humans
Abstract: Recent advances in 2D perception have led to very successful systems, able to estimate the 2D pose of humans with impressive robustness. However, our interactions with the world are fundamentally 3D, so to be able to understand, explain and predict these interactions, it is crucial to reconstruct people in 3D. In this talk, I [...]
Deep Learning for Understanding Dynamic Visual Data
Abstract: Perceiving dynamic environments from visual inputs allows autonomous agents to understand and interact with the world and is a core topic in Artificial Intelligence. The success of deep learning motivates us to apply deep learning techniques to the perception of dynamic visual data. However, how to design and apply deep neural networks to effectively [...]
Analyzing Grasp Contact via Thermal Imaging
Abstract: Grasping and manipulating objects is an important human skill. Because contact between hand and object is fundamental to grasping, measuring it can lead to important insights. However, observing contact through external sensors is challenging because of occlusion and the complexity of the human hand. I will discuss the use of thermal cameras to capture [...]
Fast Foveation for LIDARs, Projectors and Cameras
Abstract: Most cameras today capture images without considering scene content. In contrast, animal eyes have fast mechanical movements that control how the scene is imaged in detail by the fovea, where visual acuity is highest. This concentrates computational (i.e. neuronal) resources in places where they are most needed. The prevalence of foveation, and the wide [...]
Learning to See Through Occlusions and Obstructions
Virtual VASC: https://cmu.zoom.us/j/249106600 Abstract: Photography allows us to capture and share memorable moments of our lives. However, 2D images appear flat due to the lack of depth perception and may suffer from poor imaging conditions such as taking photos through reflecting or occluding elements. In this talk, I will present our recent efforts to [...]
Detectron2 in Object Detection Research
Virtual VASC: https://cmu.zoom.us/j/249106600 Abstract: Detectron2 is Facebook's library for object detection and segmentation. It has been used widely in FAIR's research and Facebook's products. This talk will introduce detectron2 with a focus on its use in object detection research, including the lessons we learned from building it, as well as the new research enabled [...]
Fairness in visual recognition
Virtual VASC Seminar: https://cmu.zoom.us/j/249106600 Abstract: Computer vision models trained on unparalleled amounts of data hold promise for making impartial, well-informed decisions in a variety of applications. However, more and more historical societal biases are making their way into these seemingly innocuous systems. Visual recognition models have exhibited bias by inappropriately correlating age, gender, sexual [...]
Bio-inspired depth sensing using computational optics
Virtual Seminar: https://cmu.zoom.us/j/249106600 Abstract: Jumping spiders rely on accurate depth perception for predation and navigation. They accomplish depth perception, despite their tiny brains, by using specialized optics. Each principal eye includes a multitiered retina that simultaneously receives multiple images with different amounts of defocus, and distance is decoded from these images with seemingly little [...]
Task-specific Vision DNN Models and Their Relation for Explaining Different Areas of the Visual Cortex
Virtual VASC Seminar: https://cmu.zoom.us/j/249106600 Abstract: Deep Neural Networks (DNNs) are state-of-the-art models for many vision tasks. We propose an approach to assess the relationship between visual tasks and their task-specific models. Our method uses Representation Similarity Analysis (RSA), which is commonly used to find a correlation between neuronal responses from brain data and models. [...]
End-to-end Generative 3D Human Shape and Pose Models and Active Human Sensing
Virtual VASC Seminar: https://cmu.zoom.us/j/249106600 Title: End-to-end Generative 3D Human Shape and Pose Models and Active Human Sensing Abstract: I will review some of our recent work in 3d human modeling, synthesis, and active vision. I will present our new, end-to-end trainable nonlinear statistical 3d human shape and pose models of different resolutions (GHUM and GHUMLite) as [...]
Telling Left from Right: Learning Spatial Correspondence Between Sight and Sound
Virtual VASC Seminar: https://cmu.zoom.us/j/92741882813?pwd=R1R0eGRaeXFHTEF2VWNwY2VIZmU5Zz09 Abstract: Self-supervised audio-visual learning aims to capture useful representations of video by leveraging correspondences between visual and audio inputs. Existing approaches have focused primarily on matching semantic information between the sensory streams. In my talk, I’ll describe a novel self-supervised task to leverage an orthogonal principle: matching spatial information in the [...]
The Topology of Learning
Zoom Virtual Meeting: https://cmu.zoom.us/j/92178295543?pwd=L2dwZU5SbDY5NzZZNzZ4ZmFUclRqQT09 Abstract: Deep Neural Networks (DNNs) have revolutionized computer vision. We now have DNNs that achieve top results in many computer vision problems, including object recognition, facial expression analysis, and semantic segmentation, to name but a few. Unfortunately, the rise in performance has come with a cost. DNNs have become so [...]
Implicit Neural Scene Representations
Virtual Zoom Seminar: https://cmu.zoom.us/j/92178295543?pwd=L2dwZU5SbDY5NzZZNzZ4ZmFUclRqQT09 Abstract How we represent signals has major implications for the algorithms we build to analyze them. Today, most signals are represented discretely: Images as grids of pixels, shapes as point clouds, audio as grids of amplitudes, etc. If images weren't pixel grids - would we be using convolutional neural networks [...]
Computational Imaging: Beyond the Limits Imposed by Lenses
Virtual VASC Seminar: https://cmu.zoom.us/j/92587238250?pwd=S0paYUVBUXozQkFTclMwRUg0MzBNZz09 Abstract: The lens has long been a central element of cameras, since its early use in the mid-nineteenth century by Niepce, Talbot, and Daguerre. The role of the lens, from the Daguerrotype to modern digital cameras, is to refract light to achieve a one-to-one mapping between a point in the scene and a point on the sensor. This effect enables the sensor to compute a particular two-dimensional (2D) [...]
Learning 3D Reconstruction in Function Space
Virtual VASC Seminar: https://cmu.zoom.us/j/96635002737?pwd=RkxGVlJaUTlhcDdGeVBPcnpTS015dz09 Abstract: In this talk, I will show several recent results of my group on learning neural implicit 3D representations, departing from the traditional paradigm of representing 3D shapes explicitly using voxels, point clouds or meshes. Implicit representations have a small memory footprint and allow for modeling arbitrary 3D toplogies at [...]
Compositional Representations for Visual Recognition
Virtual VASC - https://cmu.zoom.us/j/99437689110?pwd=cWxuQkIwWlFFZEk0QkVDUVFiN0lTdz09 Abstract: Compositionality is the ability for a model to recognize a concept based on its parts or constituents. This ability is essential to use language effectively as there exists a very large combination of plausible objects, attributes, and actions in the world. We posit that visual recognition models should be [...]
Making 3D Predictions with 2D Supervision
Abstract: Building computer vision systems that understand 3D shape are important for applications including autonomous vehicles, graphics, and VR / AR. If we assume 3D shape supervision, we can now build systems that do a reasonable job at predicting 3D shapes from images. However, 3D supervision is difficult to obtain at scale; therefore we should [...]
Perceiving 3D Human-Object Spatial Arrangements from a Single Image In-the-wild
Abstract: We live in a 3D world that is dynamic—it is full of life, with inhabitants like people and animals who interact with their environment through moving their bodies. Capturing this complex world in 3D from images has a huge potential for many applications such as compelling mixed reality applications that can interact with people [...]
Detection of Photo Manipulation with Media Forensics
Abstract: Rapid progress in machine learning, computer vision and graphics leads to successive democratization of media manipulation capabilities. While convincing photo and video manipulation used to require substantial time and skill, modern editors bring (semi-) automated tools that can be used by everyone. Some of the most recent examples include manipulation of human faces, e.g., [...]
Advancing the State of the Art of Computer Vision for Billions of Users
Abstract: At Google, advancing the state of the art of computer vision is very impactful as there are billions of users of Google products, many of which require high-quality, artifact-free images. I will share what we learned from successfully launching core computer vision techniques for various Google products, including PhotoScan (Photos), seamless Google Street View [...]
Learning-based 6D Object Pose Estimation in Real-world Conditions
Abstract: Estimating the 6D pose, i.e., 3D rotation and 3D translation, of objects relative to the camera from a single input image has attracted great interest in the computer vision community. Recent works typically address this task by training a deep network to predict the 6D pose given an image as input. While effective on [...]
Deep Learning: (still) Not Robust
Abstract: One of the key limitations of deep learning is its inability to generalize to new domains. This talk studies recent attempts at increasing neural network robustness to both natural and adversarial distribution shifts. Robustness to adversarial examples, inputs crafted specifically to fool machine learning models, are arguably the most difficult type of domain shift. [...]
End-to-End ‘One Networks’: Learning Regularizers for Least Squares via Deep Neural Networks
Abstract: Linear Restoration Problems (or Linear Inverse Problems) involve reconstructing images or videos from noisy measurement vectors. Notable examples include denoising, inpainting, super-resolution, compressive sensing, deblurring and frame prediction. Often, multiple such tasks should be solved simultaneously, e.g., through Regularized Least Squares, where each individual problem is underdetermined (overcomplete) with infinitely many solutions from which [...]
Detecting Image Synthesis — Shallow and Deep
Abstract: The proliferation of synthetic media are subject to malicious usages such as disinformation campaigns, posing potential threats to media integrity and democracy. A way to combat this is developing forensics algorithms to identify manipulated media. In the beginning of the talk, I will discuss how one can train a model to detect photos manipulated [...]
Deep Learning to Distinguish Recalled but Benign Mammography Images in Breast Cancer Screening
Abstract: Breast cancer screening using the standard mammography exam currently exhibits a high false recall rate (11.6% for women in the U.S.). Only a low proportion (0.5%) of women who were recalled for additional workup were actually found to have breast cancer. As a result of the unnecessary stress and follow-up work from these false [...]
The Plenoptic Camera
Abstract: Imagine a futuristic version of Google Street View that could dial up any possible place in the world, at any possible time. Effectively, such a service would be a recording of the plenoptic function—the hypothetical function described by Adelson and Bergen that captures all light rays passing through space at all times. While the plenoptic function [...]
Photorealistic Reconstruction of Landmarks and People using Implicit Scene Representation
Abstract: Reconstructing scenes to synthesize novel views is a long standing problem in Computer Vision and Graphics. Recently, implicit scene representations have shown novel view synthesis results of unprecedented quality, like the ones of Neural Radiance Fields (NeRF), which use the weights of a multi-layer perceptron to model the volumetric density and color of a [...]
Towards Discriminative and Domain-Invariant Feature Learning
Abstract: Deep neural networks have achieved great success in various visual applications, when trained with large amounts of labeled in-domain data. However, the networks usually suffer from a heavy performance drop on the data whose distribution is quite different from the training one. Domain adaptation methods aim to deal with such performance gap caused by [...]
Learning Efficient Visual Representation on Model, Data, Label and Beyond
Abstract: Efficient deep learning is a broad concept that we aim to learn compressed deep models and develop training algorithms to improve the efficiency of model representations, data and label utilization, etc. In recent years, deep neural networks have been recognized as one of the most effective techniques for many learning tasks, also, in the [...]
Self-supervised Learning and Generalization
Abstract: Contrastive self-supervised learning is a highly effective way of learning representations that are useful for, i.e. generalise, to a wide range of downstream vision tasks and datasets. In the first part of the talk, I will present MoCHi, our recently published contrastive self-supervised learning approach (NeurIPS 2020) that is able to learn transferable representations [...]
Learning to see from few labels
Abstract: Computer vision systems today exhibit a rich and accurate understanding of the visual world, but increasingly rely on learning on large labeled datasets to do so. This reliance on large labeled datasets is a problem especially when one considers difficult perception tasks, or novel domains where annotations might require effort or expertise. We thus [...]
Seeing the unseen: inferring unobserved information from multi-modal data
Abstract: As humans we can never fully observe the world around us and yet we are able to build remarkably useful models of it from our limited sensory data. Machine learning problems are often required to operate in a similar setup, that is the one of inferring unobserved information from the observed one. Partial observations [...]
Towards AI for 3D Content Creation
Abstract: 3D content is key in several domains such as architecture, film, gaming, and robotics. However, creating 3D content can be very time consuming -- the artists need to sculpt high quality 3d assets, compose them into large worlds, and bring these worlds to life by writing behaviour models that "drives" the characters around in [...]
Understanding the Placenta: Towards an Objective Pregnancy Screening
Abstract: My research focusses on the development of a pregnancy screening tool, that will be: (i) system and user-independent; and (ii) provides a quantifi able measure of placental health. With this end, I am working towards the design of a multiparametric quantitative ultrasound (QUS) based placental tissue characterization method. The method would potentially identify the [...]
Relational Reasoning for Multi-Agent Systems
Abstract: Multi-agent interacting systems are prevalent in the world, from purely physical systems to complicated social dynamics systems. The interactions between entities / components can give rise to very complex behavior patterns at the level of both individuals and the whole system. In many real-world multi-agent interacting systems (e.g., traffic participants, mobile robots, sports players), [...]
Self-supervised learning for visual recognition
Abstract: We are interested in learning visual representations that are discriminative for semantic image understanding tasks such as object classification, detection, and segmentation in images/videos. A common approach to obtain such features is to use supervised learning. However, this requires manual annotation of images, which is costly, ambiguous, and prone to errors. In contrast, self-supervised [...]
Reasoning over Text in Images for VQA and Captioning
Abstract: Text in images carries essential information for multimodal reasoning, such as VQA or image captioning. To enable machines to perceive and understand scene text and reason jointly with other modalities, 1) we collect the TextCaps dataset, which requires models to read and reason over text and visual content in the image to generate image [...]
Point Cloud Registration with or without Learning
Abstract: I will be presenting two of our recent works on 3D point cloud registration: A scene flow method for non-rigid registration: I will discuss our current method to recover scene flow from point clouds. Scene flow is the three-dimensional (3D) motion field of a scene, and it provides information about the spatial arrangement [...]
Propelling Robot Manipulation of Unknown Objects using Learned Object Centric Models
Abstract: There is a growing interest in using data-driven methods to scale up manipulation capabilities of robots for handling a large variety of objects. Many of these methods are oblivious to the notion of objects and they learn monolithic policies from the whole scene in image space. As a result, they don’t generalize well to [...]
When and Why Does Contrastive Learning Work?
Abstract: Contrastive learning organizes data by pulling together related items and pushing apart everything else. These methods have become very popular but it's still not entirely clear when and why they work. I will share two ideas from our recent work. First, I will argue that contrastive learning is really about learning to forget. Different [...]
Anticipating the Future: forecasting the dynamics in multiple levels of abstraction
Abstract: A key navigational capability for autonomous agents is to predict the future locations, actions, and behaviors of other agents in the environment. This is particularly crucial for safety in the realm of autonomous vehicles and robots. However, many current approaches to navigation and control assume perfect perception and knowledge of the environment, even though [...]
Learning to Perceive Videos for Embodiment
Abstract: Video understanding has achieved tremendous success in computer vision tasks, such as action recognition, visual tracking, and visual representation learning. Recently, this success has gradually been converted into facilitating robots and embodied agents to interact with the environments. In this talk, I am going to introduce our recent efforts on extracting self-supervisory signals and [...]
Open Challenges in Sign Language Translation & Production
Abstract: Machine translation and computer vision have greatly benefited of the advances in deep learning. The large and diverse amount of textual and visual data have been used to train neural networks whether in a supervised or self-supervised manner. Nevertheless, the convergence of the two field in sign language translation and production is still poses [...]
3D Recognition with self-supervised learning and generic architectures
Abstract: Supervised learning relies on manual labeling which scales poorly with the number of tasks and data. Manual labeling is especially cumbersome for 3D recognition tasks such as detection and segmentation and thus most 3D datasets are surprisingly small compared to image or video datasets. 3D recognition methods are also fragmented based on the type [...]
Rapid Adaptation for Robot Learning
Abstract: How can we train a robot to generalize to diverse environments? This question underscores the holy grail of robot learning research because it is difficult to supervise an agent for all possible situations it can encounter in the future. We posit that the only way to guarantee such a generalization is to continually learn and [...]
Humans, hands, and horses: 3D reconstruction of articulated object categories using strong, weak, and self-supervision
Abstract: Reconstructing 3D objects from a single 2D image is a task that humans perform effortlessly, yet computer vision so far has only robustly solved 3D face reconstruction. In this talk we will see how we can extend the scope of monocular 3D reconstruction to more challenging, articulated categories such as human bodies, hands and [...]
Looking behind the Seen in Order to Anticipate
Abstract: Despite significant recent progress in computer vision and machine learning, personalized autonomous agents often still don’t participate robustly and safely across tasks in our environment. We think this is largely because they lack an ability to anticipate, which in turn is due to a missing understanding about what is happening behind the seen, i.e., [...]
The Clinician’s AI Partner: Augmenting Clinician Capabilities Across the Spectrum of Healthcare
Abstract: Clinicians often work under highly demanding conditions to deliver complex care to patients. As our aging population grows and care becomes increasingly complex, physicians and nurses are now also experiencing feelings of burnout at unprecedented levels. In this talk, I will discuss possibilities for computer vision to function as a partner to clinicians, and to augment their capabilities, across [...]
Reliable and Accessible Visual Recognition
Abstract: As visual recognition models are developed across diverse applications; we need the ability to reliably deploy our systems in a variety of environments. At the same time, visual models tend to be trained and evaluated on a static set of curated and annotated data which only represents a subset of the world. In this [...]
Fake It Till You Make It: Face analysis in the wild using synthetic data alone
Abstract: In this seminar I will demonstrate how synthetic data alone can be used to perform face-related computer vision in the wild. The community has long enjoyed the benefits of synthesizing training data with graphics, but the domain gap between real and synthetic data has remained a problem, especially for human faces. Researchers have tried [...]
Leveraging StyleGAN for Image Editing and Manipulation
Abstract: StyleGAN has recently been established as the state-of-the-art unconditional generator, synthesizing images of phenomenal realism and fidelity, particularly for human faces. With its rich semantic space, many works have attempted to understand and control StyleGAN’s latent representations with the goal of performing image manipulations. To perform manipulations on real images, however, one must learn to [...]
Next-Gen Video Communication
Abstract: Video communication connects our world. It is necessary in conducting business, educational and personal activities across different geographical locations. However, the quality of an average user’s video communication is dramatically worse than that of professionally created videos in news broadcasts, talk shows, and on YouTube. This is because professionally created videos are often captured with [...]
Activity Understanding of Scripted Performances
Abstract: The PSU Taichi for Smart Health project has been doing a deep-dive into vision-based analysis of 24-form Yang-style Taichi (TaijiQuan). A key property of Taichi, shared by martial arts katas and prearranged form exercises in other sports, is practice of a scripted routine to build both mental and physical competence. The scripted nature of routines [...]
Domain adaptive object detection
Abstract: Recent advances in deep learning have led to the development of accurate and efficient models for object detection. However, learning highly accurate models relies on the availability of large-scale annotated datasets. Due to this, model performance drops drastically when evaluated on label-scarce datasets having visually distinct images. Domain adaptation tries to mitigate this degradation. In [...]
Visual Understanding across Semantic Groups, Domains and Devices
Abstract: Deep neural networks often lack generalization capabilities to accommodate changes in the input/output domain distributions and, therefore, are inherently limited by the restricted visual and semantic information contained in the original training set. In this talk, we argue the importance of the versatility of deep neural architectures and we explore it from various perspectives. [...]
Topology-Driven Learning for Biomedical Imaging Informatics
Abstract: Thanks to decades of technology development, we are now able to visualize in high quality complex biomedical structures such as neurons, vessels, trabeculae and breast tissues. We need innovative methods to fully exploit these structures, which encode important information about underlying biological mechanisms. In this talk, we explain how topology, i.e., connected components, handles, loops, [...]
Learning generative representations for image distributions
Abstract: Autoencoder neural networks are an unsupervised technique for learning representations, which have been used effectively in many data domains. While capable of generating data, autoencoders have been inferior to other models like Generative Adversarial Networks (GAN’s) in their ability to generate image data. We will describe a general autoencoder architecture that addresses this limitation, and [...]
Building Intelligent and Visceral Machines: From Sensing to Application
Abstract: Humans have evolved to have highly adaptive behaviors that help us survive and thrive. As AI prompts a move from computing interfaces that are explicit and procedural to those that are implicit and intelligent, we are presented with extraordinary opportunities. In this talk, I will argue that understanding affective and behavioral signals presents many opportunities [...]
GANcraft – an unsupervised 3D neural method for world-to-world translation
Abstract: Advances in 2D image-to-image translation methods, such as SPADE/GauGAN, have enabled users to paint photorealistic images by drawing simple sketches similar to those created in Microsoft Paint. Despite these innovations, creating a realistic 3D scene remains a painstaking task, out of the reach of most people. It requires years of expertise, professional software, a library [...]
Learning Optical Flow: Model, Data, and Applications
Abstract: Optical flow provides important information about the dynamic world and is of fundamental importance to many tasks. In this talk, I will present my work on different aspects of learning optical flow. I will start with the background and talk about PWC-Net, a compact and effective model built using classical principles for optical flow. Next, [...]
Do Vision-Language Pretrained Models Learn Spatiotemporal Primitive Concepts?
Abstract: Vision-language models pretrained on web-scale data have revolutionized deep learning in the last few years. They have demonstrated strong transfer learning performance on a wide range of tasks, even under the "zero-shot" setup, where text "prompts" serve as a natural interface for humans to specify a task, as opposed to collecting labeled data. These models are [...]
Max-Affine Spline Insights into Deep Learning
Abstract: We build a rigorous bridge between deep networks (DNs) and approximation theory via spline functions and operators. Our key result is that a large class of DNs can be written as a composition of max-affine spline operators (MASOs) that provide a powerful portal through which we view and analyze their inner workings. For instance, [...]
Understanding 3D Scenes and Interacting Hands
Abstract: Abstract: The long-term goal of my research is to help computers understand the physical world from images, including both 3D properties and how humans or robots could interact with things. This talk will summarize two recent directions aimed at enabling this goal. I will begin with learning to reconstruct full 3D scenes, including [...]