Abstract:  Enabling computers to recognize human actions in video has the potential to revolutionize many areas that benefit society such as clinical diagnosis, human-computer interaction, and social robotics. Human action recognition, however, is tremendously challenging for computers due to the subtlety of human actions and the complexity of video data. Critical to the success of [...]

Abstract: In this talk, I will present two recent pieces of work on leveraging temporal information and synthetic data to enhance video and image understanding. In the first part, I will introduce a progressive learning framework, Spatio-TEmporalProgressive (STEP), for action detection in videos. STEP is able to more effectively make use of longer temporal information, [...]

Abstract:  A key promise of Virtual Reality (VR) is the possibility of remote social interaction that is more immersive than any prior telecommunication media. However, existing social VR experiences are mediated by inauthentic digital representations of the user (i.e., stylized avatars). These stylized representations have limited the adoption of social VR applications in precisely those [...]

Abstract:   Modeling and rendering of dynamic scenes is challenging, as natural scenes often contain complex phenomena such as thin structures, evolving topology, translucency, scattering, occlusion, and biological motion. Mesh-based reconstruction and tracking often fail in these cases, and other approaches (e.g., light field video) typically rely on constrained viewing conditions, which limit interactivity. We [...]

Abstract: Humans naturally use their hands to interact and communicate with their surroundings. Reconstructing these complex and dexterous hand interactions enables sign-language recognition and translation, better assistive robots, and more immersive human-computer interaction (e.g. for AR and VR). To make hand reconstruction usable for the aforementioned applications and to a wide set of users, the [...]

Abstract:  The prevalence of smartphones and wearable devices for health monitoring and widespread use of electronic health records have led to a surge in heterogeneous multimodal healthcare data, collected at an unprecedented scale. My research focuses on developing machine learning techniques that learn salient representations of multimodal, heterogeneous data for biomedical predictive models. The first [...]

Abstract:  Recent algorithmic and hardware improvements resulted in several success stories in the field of Artificial Intelligence (AI) which impact our daily lives. However, despite its ubiquity, AI is only just starting to make advances in what may arguably have the largest societal impact thus far, the nascent field of autonomous driving. At Uber ATG, [...]

Abstract: The application of AI across various domains demonstrates both the promise of existing techniques but also their limitations. In this talk, I explore three recent projects and how they shed light on the progress of AI and the challenges to come. These projects include ELF OpenGo a reimplementation of AlphaZero, fastMRI for reducing the time [...]

Abstract:  Learning with weak and self-supervisions recently emerged as compelling tools towards leveraging vast amounts of unlabeled or partially-labeled data. In this talk, I will present some of the latest advances in weakly-supervised visual scene understanding from NVIDIA. Specifically, I will summarize and discuss some challenges and potential solutions in weakly-supervised learning, and introduce our [...]

Abstract:  Miniaturization of cameras is key to enabling new applications in areas such as connected devices, wearables, implantable medical devices, in vivo microscopy, and micro-robotics. Recently, lenses were identified as the main bottleneck in miniaturization of cameras. Standard smaller lens-system camera modules have a thickness of about 10 mm or higher, and reducing the size [...]

Abstract:  Recent advances in face capture now enable digitizing high-quality 3D faces for the entertainment industry. Standardized digitization solutions, however, require tailor-made capture systems and extensive manual work, making them expensive and hard to deploy. With the advent of commodity sensors, new lightweight approaches that push the boundaries of human digitization have been introduced, slowly [...]

Abstract:  Statistical 3D human body models have helped us to better understand human shape and motion and already enabled exciting new applications. However, if we want to learn detailed, personalized, and clothed models of human shape, motion, and dynamics, we require new approaches that learn from ubiquitous data such as plain RGB-images and video. I [...]

Abstract:  In a world of abundant information targeting multiple senses, and increasingly powerful media, we need new mechanisms to model content. Techniques for representing individual channels, such as visual data or textual data, have greatly improved, and some techniques exist to model the relationship between channels that are “mirror images” of each other and contain [...]

Abstract:  In all machine learning problems, there is an explicit trade off between cost and benefit. In real world vision problems, this optimization becomes increasingly difficult since those trade offs directly impact technology and product development as well as business strategy. For any successful business case, it is critical that the cost/benefit trade offs in [...]

Abstract:  This talk is motivated by the following thesis: Background knowledge is key to intelligent decision making. While deep learning methods have made significant strides over the last few years, they often lack the context in which they operate. Knowledge Graphs (and more generally multi-relational graphs) provide a flexible framework to capture and represent knowledge [...]

Abstract:  Recent advances in 2D perception have led to very successful systems, able to estimate the 2D pose of humans with impressive robustness. However, our interactions with the world are fundamentally 3D, so to be able to understand, explain and predict these interactions, it is crucial to reconstruct people in 3D. In this talk, I [...]

Abstract:  Perceiving dynamic environments from visual inputs allows autonomous agents to understand and interact with the world and is a core topic in Artificial Intelligence. The success of deep learning motivates us to apply deep learning techniques to the perception of dynamic visual data. However, how to design and apply deep neural networks to effectively [...]

Abstract:  Grasping and manipulating objects is an important human skill. Because contact between hand and object is fundamental to grasping, measuring it can lead to important insights. However, observing contact through external sensors is challenging because of occlusion and the complexity of the human hand. I will discuss the use of thermal cameras to capture [...]

Abstract:  Most cameras today capture images without considering scene content. In contrast, animal eyes have fast mechanical movements that control how the scene is imaged in detail by the fovea, where visual acuity is highest. This concentrates computational (i.e. neuronal) resources in places where they are most needed. The prevalence of foveation, and the wide [...]