Student Talks
Multi-Human 3D Reconstruction from Monocular Videos
Abstract: We study the problem of multi-human 3D reconstruction from videos captured in the wild. Human movements are dynamic, and accurately reconstructing them in various settings is crucial for developing immersive social telepresence, assistive humanoid robots, and augmented reality systems. However, creating such a system requires addressing fundamental issues with previous works regarding the data [...]
Language-Conditioned Object Detection and Manipulation
Abstract: Traditional object detection methods are often confined to predefined object vocabularies, limiting their versatility in real-world scenarios where robots need to understand and execute diverse household tasks. Additionally, the 2D and 3D perception communities have typically pursued separate approaches tailored to their respective domains. In this thesis, we present a language-conditioned object detector with [...]
How I Learned to Love Blobs: The Power of Gaussian Representations in Differentiable Rendering and Optimization
Abstract: In this thesis, we explore the use of Gaussian Representations in multiple application areas of computer vision and robotics. In particular, we design a ray-based differentiable renderer for 3D Gaussians that can be used to solve multiple classic computer vision problems in a unified manner. For example, we can reconstruct 3D shapes from color, [...]
Watch, Practice, Improve: Towards In-the-wild Manipulation
Abstract: The longstanding dream of many roboticists is to see robots perform diverse tasks in diverse environments. To build such a robot that can operate anywhere, many methods train on robotic interaction data. While these approaches have led to significant advances, they rely on heavily engineered setups or high amounts of supervision, neither of which [...]
Towards Photorealistic Dynamic Capture and Animation of Human Hair and Head
Abstract: Realistic human avatars play a key role in immersive virtual telepresence. To reach a high level of realism, a human avatar needs to faithfully reflect human appearance. A human avatar should also be drivable and express natural motions. Existing works have made significant progress in building drivable realistic face avatars, but they rarely include [...]
Modeling Dynamic Clothing for Data-Driven Photorealistic Avatars
Abstract: In this thesis, we aim to build photorealistic animatable avatars of humans wearing complex clothing in a data-driven manner. Such avatars will be a critical technology to enable future applications such as immersive telepresence in Virtual Reality (VR) and Augmented Reality (AR). Existing full-body avatars that jointly model geometry and view-dependent texture using Variational [...]
Manipulation Among Movable Objects for Pick-and-Place Tasks in Cluttered 3D Workspaces
Abstract: In cluttered real-world workspaces, simple pick-and-place tasks for robot manipulators can be quite challenging to solve. Often there is no collision-free trajectory that allows the robot to grasp and extract a desired object from the scene. This requires motion planning algorithms to reason about rearranging some of the “movable” clutter in the scene so [...]
Generalizable Dexterity with Reinforcement Learning
Abstract: Dexterity, the ability to perform complex interactions with the physical world, is at the core of robotics. However, existing research in robot manipulation has been focused on tasks that involve limited dexterity, such as pick-and-place. The motor skills of the robots are often quasi-static, have a predefined or limited sequence of contact events, and [...]
Preference Based Optimization of Multi-Objective Robot Performance
Abstract: Robotic systems often require that tradeoffs be made--for example, between performance and robustness, power and longevity, or efficiency and safety. While roboticists can design cost functions with hand-picked weights for different metrics, it is not always a straightforward task, particularly when some aspects of performance are not easily quantified. This can occur especially when [...]
Ensuring safety for uncertain high-dimensional robotic systems
Abstract: Two major obstacles for safe control and planning are (1) scaling to high-dimensional systems and (2) handling uncertain systems. This is problematic because such systems are ubiquitous in practice: e.g. drones with unknown drag, manipulators carrying unknown packages. In this proposal, we aim to address both challenges. At the control level, we have synthesized [...]