PhD Speaking Qualifier
Carnegie Mellon University
Online Kinodynamic Planning for Teams of Aerial Robots in 3-D Workspaces
Abstract: An efficient online planning or replanning methodology is a critical requirement for scalable and responsive real world multi-robot deployments. The need to replan typically stems from the invalidation of existing plans due to incomplete knowledge of the environment, or, from scenarios that necessitate changing goal locations in response to evolving application requirements. In this [...]
Carnegie Mellon University
Open-world 3D Object Detection
Abstract: Perception for autonomous robots presents a set of unique challenges: finding the right representation for 3D signals, adapting to an open-world setting, and exploiting geometric priors. Successfully detecting objects regardless of their labels lays a solid foundation for safe navigation. I will present two of my recent works in this line. First, I will [...]
When to use CNNs for Inverse Problems in Vision
Abstract: Reconstruction tasks in computer vision aim fundamentally to recover an undetermined signal from a set of noisy measurements. Examples include super-resolution, image denoising, and non-rigid structure from motion\cite{Kong_2019}, all of which have seen recent advancements through deep learning. However, earlier work made extensive use of sparse signal reconstruction frameworks (e.g. convolutional sparse coding). While [...]
Tendon Driven Foam Hands
Abstract: There has been great progress in soft robot design, manufacture, and control in recent years, and soft robots are a tool of choice for safe and robust handling of objects in conditions of uncertainty. Still, dexterous in-hand manipulation using soft robots remains a challenge. This talk introduces a novel class of soft robots in [...]
Carnegie Mellon University
Towards a Good Representation For Reinforcement Learning
Abstract: Deep reinforcement learning has achieved many successes over the recent years. However, its high sample complexity and the difficulty in specifying a reward function have limited its application. In this talk, I will take a representation learning perspective towards these issues. Is it possible to map from the raw observation, potentially in high dimension, [...]
Carnegie Mellon University
Resource-constrained learning and inference for visual perception
Zoom Link Abstract Real-world applications usually require computer vision algorithms to meet certain resource constraints. In this talk, I will present evaluation methods and principled solutions for both cases of training and testing. First, I will talk about a formal setting for studying training under the non-asymptotic, resource-constrained regime, i.e., budgeted training. We analyze the [...]
Carnegie Mellon University
Planning and Execution using Inaccurate Models with Provable Guarantees
Zoom Link Abstract: Models used in modern planning problems to simulate outcomes of real world action executions are becoming increasingly complex, ranging from simulators that do physics-based reasoning to precomputed analytical motion primitives. However, robots operating in the real world often face situations not modeled by these models before execution. This imperfect modeling can lead [...]
Carnegie Mellon University
The Effect of Locomotion Configuration on Discrete Obstacle Traversal for a Small Tracked Vehicle
Zoom Link Abstract: As mobile robots are being designed for increasingly rugged and unknown terrain, mechanical reconfigurability presents one possibility for improving vehicle efficiency and mobility. To validate this idea, we created an 18.5-kg modular tracked vehicle with adjustable track tension, track width, track length, and sprocket diameter. In this talk, I will explain the [...]
Carnegie Mellon University
Task-Driven Modular Networks for Zero-Shot Compositional Learning
Zoom Link Abstract: One of the hallmarks of human intelligence is the ability to compose learned knowledge into novel concepts which can be recognized without a single training example. In contrast, current state-of-the-art methods require hundreds of training examples for each possible category to build reliable and accurate classifiers. To alleviate this striking difference in [...]
Image to LiDAR Map Registration using Late Feature Projection
Zoom Link Abstract: Accurate localization is essential for autonomous operation in many problem domains. This is most often performed by comparing LiDAR scans collected in real-time to a HD point cloud based map. While this enables centimeter-level accuracy, it depends on an expensive LiDAR sensor at run time. Recently, efforts have been underway to reduce [...]
Carnegie Mellon University
A Theory of Fermat Paths for Non-line-of-sight Shape Reconstruction
Zoom Link Abstract: Traditionally, computer vision systems and algorithms, such as stereo vision, and shape from shading, have been developed to mimic human vision. As a consequence, a lot of these systems operate under constraints that we take for granted in human vision. An example of such a constraint is that the scene of interest [...]
Learning Contextual Actions for Heuristic Search-Based Motion Planning
Zoom Link Abstract: Heuristic search-based motion planning can be computationally costly in large state and action spaces. In this work we explore the use of generative models to learn contextual actions for successor generation in heuristic search. We focus on cases where the robot operates in similar environments, i.e. environments drawn from some underlying distribution. [...]
Carnegie Mellon University
Interactive Weak Supervision – Learning Useful Heuristics for Data Labeling
Zoom Link Abstract: Obtaining large annotated datasets is critical for training successful machine learning models and it is frequently a bottleneck in practice. Weak supervision offers a promising alternative for producing labeled datasets without ground truth annotations by generating probabilistic labels using multiple noisy heuristics. This process can scale to large amounts of data and [...]
Carnegie Mellon University
Learning Active Task-Oriented Exploration Policies for Bridging the Sim-to-Real Gap
Zoom Link Abstract: Training robotic policies in simulation suffers from the sim-to-real gap, as simulated dynamics can be different from real-world dynamics. Past works tackled this problem through domain randomization and online system-identification. The former is sensitive to the manually-specified training distribution of dynamics parameters and can result in behaviors that are overly conservative. The [...]
Interferometric light transmission probing with coded mutual intensity
Zoom Link Abstract: We introduce a new interferometric imaging methodology that we term interferometry with coded mutual intensity, which allows selectively imaging photon paths based on attributes such as their length and endpoints. At the core of our methodology is a new technical result that shows that manipulating the spatial coherence properties of the light [...]
Sparse Spatial Hashing for Dense 3D Reconstruction
Abstract: Real-world 3D data is locally dense but globally sparse. Therefore, efficient sparse data structures are an essential component of dense 3D perception for computer vision and robotics. We manifest the power of spatial hashing by two typical tasks: dense scene reconstruction and global registration. In the first task, we accelerate volumetric integration and surface [...]
Carnegie Mellon University
3D Multi-Object Tracking for Autonomous Driving
Abstract: 3D multi-object tracking (MOT) is a key component of a perception system for autonomous driving. Due to recent progress in 3D object detection in the context of autonomous driving, recent work in 3D MOT primarily focuses on online tracking with the use of a tracking-by-detection pipeline. In this talk, we introduce a new 3D [...]
Carnegie Mellon University
Ergodic Trajectory Optimization for Information Gathering
Abstract: Planetary robots currently rely on significant guidance from expert human operators. Science autonomy adds algorithms and methods for autonomous scientific exploration to improve efficiency of discovery and overcome limited communication bandwidth and delay bottlenecks. This research focuses on planning trajectories for information gathering and choosing sampling locations that have the most informative samples. We [...]
Carnegie Mellon University
Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
Abstract: Reinforcement learning has shown great promise for synthesizing realistic human behaviors by learning humanoid control policies from motion capture data. However, it is still very challenging to reproduce sophisticated human skills like ballet dance, or to stably imitate long-term human behaviors with complex transitions. The main difficulty lies in the dynamics mismatch between the [...]
Studying the Evolution of Pedestrian Group Space
Abstract: Imagine walking along a busy sidewalk, do you track the movement of every single individual? Or do you simply group pedestrians with similar moving patterns and then track the movement of this group? Grouping is a common behavior in pedestrian navigation and it is typically inappropriate for a robot to cut through the social [...]
Carnegie Mellon University
Soft actuators by electrochemical oxidation of liquid metal surfaces
Abstract: Soft robotic systems typically operate through the use of soft actuators constructed from highly deformable materials or liquids. Because of their intrinsic compliance, these actuators can achieve elastic resilience and adaptability similar to their biological counterparts. One challenge with engineering these artificial muscles is the selection of soft materials and activation methods while maintaining [...]
A Graph-Based Method for Joint Instance Segmentation of Point Clouds and Image Sequences
Abstract: While learning-based semantic instance segmentation methods have achieved impressive progress, their use is limited in robotics applications due to reliance on expensive training data annotations and assumptions of single sensor modality or known object classes. We propose a novel graph-based instance segmentation approach that combines information from a 2D image sequence and a 3D [...]
Continual Reinforcement Learning using Self-Activating Neural Ensembles
Abstract: The ability for an agent to continuously learn new skills without catastrophically forgetting existing knowledge is of critical importance for the development of generally intelligent agents. Most methods devised to address this problem depend heavily on well-defined task boundaries which simplify the problem considerably. Our task-agnostic method, Self-Activating Neural Ensembles (SANE), uses a hierarchical [...]
Carnegie Mellon University
Unsupervised 2D-3D Lifting with Deep Structure Priors
Abstract: Learning to estimate non-rigid 3D structures from 2D imaged observations is bottle-necked by the availability of abundant 3D annotated data. Learning methods that reduce the amount of required annotation is of high practical value. In this regard, Non-Rigid Structure from Motion (NRSfM) methods offer the opportunity to infer 3D structures solely from 2D annotations. [...]
Model Adaptation for Compliant Parallel Robot with Nonstationary Dynamics
Abstract: Soft robots can be constructed with few parts and from a wide variety of materials. This makes them a potentially appealing choice for applications where there are resource constraints on system fabrication. However, soft robot dynamics are difficult to accurately model analytically, due to a multiphysics coupling between shape, forces, temperature, and history of [...]
Adaptive Safety Margins for Safe Replanning under Time-Varying Disturbances
Abstract: Safe real-time navigation is a considerable challenge because engineers often need to work with uncertain vehicle dynamics, variable external disturbances, and imperfect controllers. A common strategy used to address safety is to employ hand-defined margins for obstacle inflation. However, arbitrary static margins often fail in more dynamic scenarios, and using worst-case assumptions proves to [...]
HyperDynamics: Generating Expert Dynamics Models by Observation
Abstract: We propose HyperDynamics, a framework that conditions on an agent’s interactions with the environment and optionally its visual observations, and generates the parameters of neural dynamics models based on inferred properties of the dynamical system. Physical and visual properties of the environment that are not part of the low-dimensional state yet affect its temporal [...]
Direct Fitting of Mixture Models
Abstract: There exist many choices of 3D shape representation. Some recent work has advocated for the use of Gaussian Mixture Models as a compact representation for 3D shapes and scenes. These models are typically fit to point clouds, even when the shapes were obtained as 3D meshes. Here we present a formulation for fitting Gaussian [...]
Carnegie Mellon University
Terrain Perception using Structured Light for Micro-Rovers
Abstract: With continuing advancement in technology, the future of planetary exploration is likely to be dominated by robotic missions. Yet rovers capable of science investigations are slow and bulky with very limited computing which prohibits demonstrating full autonomy. These rovers are also risk averse due to their huge mission cost. However there is a new [...]
Carnegie Mellon University
Analysis of Deadlock in Multirobot Systems
Abstract: Collision avoidance for multirobot systems is a well-studied problem. Recently, control barrier functions (CBFs) have been proposed for synthesizing controllers that guarantee safety while simultaneously encouraging goal stabilization for multiple robots. However, it has been noted that reactive control synthesis methods (such as CBFs) are prone to deadlock, an equilibrium of system dynamics that [...]
Interleaving Graph Search and Trajectory Optimization for Aggressive Quadrotor Flight
Abstract: Quadrotors can achieve aggressive flight by tracking complex maneuvers and rapidly changing directions. Planning for aggressive flight with trajectory optimization could be incredibly fast, even in higher dimensions, and can account for dynamics of the quadrotor, however, only provides a locally optimal solution. On the other hand, planning with discrete graph search can handle [...]
See, Hear, Explore: Curiosity via Audio-Visual Association
Abstract: Exploration is one of the core challenges in reinforcement learning. A common formulation of curiosity-driven exploration uses the difference between the real future and the future predicted by a learned model. However, predicting the future is an inherently difficult task which can be ill-posed in the face of stochasticity. In this work, we introduce [...]
MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video
Abstract: We present a method to capture temporally coherent dynamic clothing deformation from a monocular RGB video input. In contrast to the existing literature, our method does not require a pre-scanned personalized mesh template, and thus can be applied to in-the-wild videos. To constrain the output to a valid deformation space, we build statistical deformation [...]
Policy Decomposition : Approximate Optimal Control with Suboptimality Estimates
Abstract: Owing to the curse of dimensionality, numerically computing global policies to optimal control problems for complex dynamical systems quickly becomes intractable. In consequence, a number of approximation methods have been developed. However, none of the current methods can quantify by how much the resulting control underperforms the elusive globally optimal solution. We propose Policy [...]
Inverse Reinforcement Learning with Explicit Policy Estimates
Abstract: Various methods for solving the inverse reinforcement learning (IRL) problem have been developed independently in machine learning and economics. In particular, the method of Maximum Causal Entropy IRL is based on the perspective of entropy maximization, while related advances in the field of economics instead assume the existence of unobserved action shocks to explain [...]
Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation
Abstract: To perform manipulation tasks in the real world, robots need to operate on objects with various shapes, sizes and without access to geometric models. It is often infeasible to train monolithic neural network policies across such large variance in object properties. Towards this generalization challenge, we propose task-axis controllers, which are defined relative to [...]
Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies
Abstract: Real-world environments, such as homes, hospitals, and restaurants, often contain many objects that a robot could possibly manipulate. However, for a given manipulation task, only a small number of objects and object properties may actually be relevant. This talk presents CREST (Causal Reasoning for Efficient Structure Transfer), our approach to learn the relevant state [...]
Grasping Transparent, Specular, and Deformable Objects
Abstract: A large body of research exists on grasping for objects with ideal properties like Lambertian reflectance and rigidity. On the other hand, real-world environments contain many objects for which such properties do not hold, such as transparent, specular, and deformable objects. For such objects, new approaches are required to achieve the same level of [...]
PoseIt: A Visual-Tactile Dataset of Holding Poses for Grasp Stability Analysis
Abstract: When humans grasp objects in the real world, we often move our arm to hold the object in a different pose where we can use it. In contrast, typical lab settings only study the stability of the grasp immediately after lifting, without any subsequent re-positioning of the arm. However, an object’s stability could vary [...]
Planning to Minimize Human and Robot Efforts Over Tasks
Abstract: It is not feasible to pre-program robots a priori for every possible task they may encounter in unstructured domains. Upon encountering a task that a robot can't solve, one common strategy is to teach it new skills via demonstrations. However, demonstrating a task can often be more cumbersome than performing the task directly. This [...]
Fine-Tuning Offline Reinforcement Learning with Model-Based Policy Optimization
Abstract: In offline reinforcement learning (RL), we attempt to learn a control policy from a fixed dataset of environment interactions. This setting has the potential benefit of allowing us to learn effective policies without needing to collect additional interactive data, which can be expensive or dangerous in real-world systems. However, traditional off-policy RL methods tend [...]
Modeling Coupled Human-Robot Motion for Provable Safety
Abstract: Guide robots that help users who are blind or low vision navigate through crowds and complex environments show promise for improving accessibility in public spaces. These robots must provide real-time safety guarantees for the users, which requires accurate modeling of their behavior in the context of closely coupled human-robot motion. This model must also [...]
Diminished Reality for Close Quarters Robotic Telemanipulation
Abstract: In robot telemanipulation tasks, the robot itself can sometimes occlude a target object from the user's view. We investigate the potential of diminished reality to address this problem. Our method uses an optical see-through head-mounted display to create a diminished reality illusion that the robot is transparent, allowing users to see occluded areas behind [...]
Learning Compositional Radiance Fields of Dynamic Human Heads
Meeting ID: 942 4671 0665 Passcode: jkhzoom Abstract: Photorealistic rendering of dynamic humans is an important capability for telepresence systems. Recently, neural rendering methods have been developed to create high-fidelity models of humans and objects. Some of these methods do not produce results with high-enough fidelity for driveable human models (Neural Volumes) whereas others have [...]
An Experimental Design Perspective on Model-Based Reinforcement Learning
Abstract: In many practical applications of RL, it is expensive to observe state transitions from the environment. For example, in the problem of plasma control for nuclear fusion, computing the next state for a given state-action pair requires querying an expensive transition function which can lead to many hours of computer simulation or dollars of [...]
Learning Model Preconditions for Planning with Multiple Models
Abstract: Different models can provide differing levels of fidelity when a robot is planning. Analytical models are often fast to evaluate but only work in limited ranges of conditions. Meanwhile, physics simulators are effective at modeling complex interactions between objects but are typically more computationally expensive. Learning when to switch between the various models can [...]
Reconstructing common objects to interact with
Abstract: We humans are able to understand 3D shapes of common daily objects and interact with them from a wide range of categories. We understand cups are usually cylinder-like and we can easily predict the shape of one particular cup, both in isolation or even when it is held by a human. We aim to [...]
A causal framework to diagnose and fix issues with doors
Abstract: Many animals, such as ravens, (and a fortiori humans) exhibit a great deal of physical intelligence that allows them to solve complex multi-step physical puzzles. This ability indicates an understanding or a faculty to represent causality and mechanisms, understand when something goes wrong, and figure out how to deal with it. As a step [...]