Carnegie Mellon University
Robust Manipulation with Active Compliance
Abstract: Human manipulation skills are filled with creative use of physical contacts to move the object about the hand and in the environment. However, it is difficult for robot manipulators to enjoy this dexterity since contacts may cause the manipulation task to fail by introducing huge forces or unexpected change of constraints, especially when modeling [...]
Carnegie Mellon University
Open-world Object Detection and Tracking
Abstract: Computer vision today excels at recognizing narrow slices of the real world: our models seem to accurately detect objects like cats, cars, or chairs in benchmark datasets. However, deploying models requires that they work in the open world, which includes arbitrary objects in diverse settings. Current methods struggle on both axes: they recognize only [...]
Self-supervised learning for visual recognition
Abstract: We are interested in learning visual representations that are discriminative for semantic image understanding tasks such as object classification, detection, and segmentation in images/videos. A common approach to obtain such features is to use supervised learning. However, this requires manual annotation of images, which is costly, ambiguous, and prone to errors. In contrast, self-supervised [...]
Policy Decomposition : Approximate Optimal Control with Suboptimality Estimates
Abstract: Owing to the curse of dimensionality, numerically computing global policies to optimal control problems for complex dynamical systems quickly becomes intractable. In consequence, a number of approximation methods have been developed. However, none of the current methods can quantify by how much the resulting control underperforms the elusive globally optimal solution. We propose Policy [...]
Inverse Reinforcement Learning with Explicit Policy Estimates
Abstract: Various methods for solving the inverse reinforcement learning (IRL) problem have been developed independently in machine learning and economics. In particular, the method of Maximum Causal Entropy IRL is based on the perspective of entropy maximization, while related advances in the field of economics instead assume the existence of unobserved action shocks to explain [...]
Carnegie Mellon University
GANs for Everyone
Abstract: The power and promise of deep generative models such as StyleGAN, CycleGAN, and GauGAN lie in their ability to synthesize endless realistic, diverse, and novel content with user controls. Unfortunately, the creation and deployment of these large-scale models demand high-performance computing platforms, large-scale annotated datasets, and sophisticated knowledge of deep learning methods. This makes [...]
Carnegie Mellon University
MSR Thesis Talk – Hans Kumar
Title: Multi-Session Periodic SLAM for Legged Robots Abstract: Methods for state estimation that rely on visual information are challenging on dynamic robots because of rapid changes in the viewing angle of onboard cameras. In this thesis, we show that by leveraging structure in the way that dynamic robots locomote, we can increase the feasibility [...]
Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation
Abstract: To perform manipulation tasks in the real world, robots need to operate on objects with various shapes, sizes and without access to geometric models. It is often infeasible to train monolithic neural network policies across such large variance in object properties. Towards this generalization challenge, we propose task-axis controllers, which are defined relative to [...]
Reasoning over Text in Images for VQA and Captioning
Abstract: Text in images carries essential information for multimodal reasoning, such as VQA or image captioning. To enable machines to perceive and understand scene text and reason jointly with other modalities, 1) we collect the TextCaps dataset, which requires models to read and reason over text and visual content in the image to generate image [...]
Carnegie Mellon University
MSR Thesis Talk: Eagle Dapeng Zhao
Title: Predicting Human Trajectories by Learning and Matching Patterns Zoom Link: https://cmu.zoom.us/j/93356993095?pwd=Nzd3a09PbG9mVkV5blFVaU5nRk1GQT09 Abstract: As more and more robots are envisioned to cooperate with humans sharing the same space, it is desired for robots to be able to predict others' trajectories to navigate in a safe and self-explanatory way. We propose a Convolutional Neural Network-based approach [...]
Design and control of insect-scale bees and dog-scale quadrupeds
Abstract: Enhanced robot autonomy---whether it be in the context of extended tether-free flight of a 100mg insect-scale flapping-wing micro aerial vehicle (FWMAV), or long inspection routes for a quadrupedal robot---is hindered by fundamental constraints in power and computation. With this motivation, I will discuss a few projects I have worked on to circumvent these issues in [...]
Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies
Abstract: Real-world environments, such as homes, hospitals, and restaurants, often contain many objects that a robot could possibly manipulate. However, for a given manipulation task, only a small number of objects and object properties may actually be relevant. This talk presents CREST (Causal Reasoning for Efficient Structure Transfer), our approach to learn the relevant state [...]
Grasping Transparent, Specular, and Deformable Objects
Abstract: A large body of research exists on grasping for objects with ideal properties like Lambertian reflectance and rigidity. On the other hand, real-world environments contain many objects for which such properties do not hold, such as transparent, specular, and deformable objects. For such objects, new approaches are required to achieve the same level of [...]
Carnegie Mellon University
Auto-generated Manipulation Primitives
Abstract: The central theme in robotic manipulation is that of the robot interacting with the world through physical contact. We tend to describe that physical contact using specific words that capture the nature of the contact and the action, such as grasp, roll, pivot, push, pull, tilt, close, open etc. We refer to these situation-specific [...]
Carnegie Mellon University
Learning 3D Registration and Reconstruction from the Visual World
Abstract: Humans learn to develop strong senses for 3D geometry by looking around in the visual world. Through pure visual perception, not only can we recover a mental 3D representation of what we are looking at, but meanwhile we can also recognize where we are looking at the scene from. Finding the 3D scene representation [...]
Point Cloud Registration with or without Learning
Abstract: I will be presenting two of our recent works on 3D point cloud registration: A scene flow method for non-rigid registration: I will discuss our current method to recover scene flow from point clouds. Scene flow is the three-dimensional (3D) motion field of a scene, and it provides information about the spatial arrangement [...]
Carnegie Mellon University
Development of an Agile and Dexterous Balancing Mobile Manipulator Robot
Abstract: The proposed thesis work focuses on the design and control of a new unique agile and dexterous mobile manipulator, the Carnegie Mellon University (CMU) ballbot. The CMU ballbot is a human-sized dynamically stable mobile robot that balances on a single ball. We present the development and integration of a new pair of seven-degree-of-freedom (7-DOF) [...]
Carnegie Mellon University
MSR Thesis Talk: Shengcao Cao
Title: Efficient Model Performance Estimation via Feature Histories Abstract: An important step in the task of neural network design, such as hyper-parameter optimization (HPO) or neural architecture search (NAS), is the evaluation of a candidate model's performance. Given fixed computational resources, one can either invest more time training each model to obtain more accurate estimates [...]