Carnegie Mellon University
Data-Driven Robotic Grasping in the Wild
Zoom Link Abstract: Humans can effortlessly grasp a wide variety of objects in diverse environments. On the other hand, robotic grasping has been extremely challenging in practice and is far from matching human dexterity. Despite recent progress in the community, most research is still largely focused on constrained environments like picking individual objects on a [...]
Scaling Probabilistically Safe Learning to Robotics
Abstract: Before learning robots can be deployed in the real world, it is critical that probabilistic guarantees can be made about the safety and performance of such systems. In recent years, safe reinforcement learning algorithms have enjoyed success in application areas with high-quality models and plentiful data, but robotics remains a challenging domain for [...]
Carnegie Mellon University
Routing for Persistent Exploration in Dynamic Environments with Teams of Energy-Constrained Robots
Abstract: Disaster relief scenarios require rapid and persistent situational awareness to inform first-responders of safe and viable routes through a constantly shifting environment. Knowing what roads have become flooded or are suddenly obstructed by debris can significantly improve response time and ease the distribution of resources. In a sufficiently large environment, deploying and maintaining fixed [...]
Compositional Representations for Visual Recognition
Virtual VASC - https://cmu.zoom.us/j/99437689110?pwd=cWxuQkIwWlFFZEk0QkVDUVFiN0lTdz09 Abstract: Compositionality is the ability for a model to recognize a concept based on its parts or constituents. This ability is essential to use language effectively as there exists a very large combination of plausible objects, attributes, and actions in the world. We posit that visual recognition models should be [...]
Sparse Spatial Hashing for Dense 3D Reconstruction
Abstract: Real-world 3D data is locally dense but globally sparse. Therefore, efficient sparse data structures are an essential component of dense 3D perception for computer vision and robotics. We manifest the power of spatial hashing by two typical tasks: dense scene reconstruction and global registration. In the first task, we accelerate volumetric integration and surface [...]
Carnegie Mellon University
Coordinated online multi-robot planning
Abstract: Multi-robot applications frequently seek to employ human operators to direct robot actions online because fully automated planners struggle to encode human expertise or handle the extenuating circumstances that occur during real world operations. However, it is extremely challenging for a human to direct multi-robot teams, especially online, i.e., in real-time. From entertainment to defense, [...]
From kinematic to energetic design and control of wearable robots for agile human locomotion
Abstract: Even with the help of modern prosthetic and orthotic (P&O) devices, lower-limb amputees and stroke survivors often struggle to walk in the home and community. Emerging powered P&O devices could actively assist patients to enable greater mobility, but these devices are currently designed to produce a small set of pre-defined motions. Finite state machines [...]
Sensor Planning for Large Numbers of Robots
Abstract: In the wake of a natural disaster, locating and extracting victims quickly is critical because mortality rises rapidly after the first forty-eight hours. In order to assist search and rescue teams and improve response times, teams of aerial robots equipped with sensors and cameras can engage in sensing tasks such as mapping buildings, assessing [...]
Making 3D Predictions with 2D Supervision
Abstract: Building computer vision systems that understand 3D shape are important for applications including autonomous vehicles, graphics, and VR / AR. If we assume 3D shape supervision, we can now build systems that do a reasonable job at predicting 3D shapes from images. However, 3D supervision is difficult to obtain at scale; therefore we should [...]
Carnegie Mellon University
3D Multi-Object Tracking for Autonomous Driving
Abstract: 3D multi-object tracking (MOT) is a key component of a perception system for autonomous driving. Due to recent progress in 3D object detection in the context of autonomous driving, recent work in 3D MOT primarily focuses on online tracking with the use of a tracking-by-detection pipeline. In this talk, we introduce a new 3D [...]
The World’s Tiniest Space Program
Abstract: The aerospace industry has experienced a dramatic shift over the last decade: Flying a spacecraft has gone from something only national governments and large defense contractors could afford to something a small startup can accomplish on a shoestring budget. A virtuous cycle has developed where lower costs have led to more launches and the [...]
Perceiving 3D Human-Object Spatial Arrangements from a Single Image In-the-wild
Abstract: We live in a 3D world that is dynamic—it is full of life, with inhabitants like people and animals who interact with their environment through moving their bodies. Capturing this complex world in 3D from images has a huge potential for many applications such as compelling mixed reality applications that can interact with people [...]
A future with affordable Self-driving vehicles
(Video to appear once approved) Abstract: We are on the verge of a new era in which robotics and artificial intelligence will play an important role in our daily lives. Self-driving vehicles have the potential to redefine transportation as we understand it today. Our roads will become safer and less congested, while parking spots will be repurposed as leisure [...]
Detection of Photo Manipulation with Media Forensics
Abstract: Rapid progress in machine learning, computer vision and graphics leads to successive democratization of media manipulation capabilities. While convincing photo and video manipulation used to require substantial time and skill, modern editors bring (semi-) automated tools that can be used by everyone. Some of the most recent examples include manipulation of human faces, e.g., [...]
Robotics and Biosystems
Abstract: Research at the Center for Robotics and Biosystems at Northwestern University encompasses bio-inspiration, neuromechanics, human-machine systems, and swarm robotics, among other topics. In this talk I will give an overview of some of our recent work on in-hand manipulation, robot locomotion on yielding ground, and human-robot systems. Biography: Kevin Lynch received the B.S.E. degree [...]
Advancing the State of the Art of Computer Vision for Billions of Users
Abstract: At Google, advancing the state of the art of computer vision is very impactful as there are billions of users of Google products, many of which require high-quality, artifact-free images. I will share what we learned from successfully launching core computer vision techniques for various Google products, including PhotoScan (Photos), seamless Google Street View [...]
Carnegie Mellon University
Ergodic Trajectory Optimization for Information Gathering
Abstract: Planetary robots currently rely on significant guidance from expert human operators. Science autonomy adds algorithms and methods for autonomous scientific exploration to improve efficiency of discovery and overcome limited communication bandwidth and delay bottlenecks. This research focuses on planning trajectories for information gathering and choosing sampling locations that have the most informative samples. We [...]
Learning-based 6D Object Pose Estimation in Real-world Conditions
Abstract: Estimating the 6D pose, i.e., 3D rotation and 3D translation, of objects relative to the camera from a single input image has attracted great interest in the computer vision community. Recent works typically address this task by training a deep network to predict the 6D pose given an image as input. While effective on [...]
SubT Fall Update Webinar Led by CMU’s Robotics Institute faculty members Sebastian Scherer and Matt Travers, as well as OSU’s Geoff Hollinger
We invite you to meet members of the award-winning Team Explorer, the CMU DARPA Subterranean Challenge team, and learn more about this groundbreaking competition. Some of the world's top universities have entered the DARPA Subterranean Challenge, developing technologies to map, navigate, and search underground environments. Led by CMU's Robotics Institute faculty members Sebastian Scherer and Matt [...]
Deep Learning: (still) Not Robust
Abstract: One of the key limitations of deep learning is its inability to generalize to new domains. This talk studies recent attempts at increasing neural network robustness to both natural and adversarial distribution shifts. Robustness to adversarial examples, inputs crafted specifically to fool machine learning models, are arguably the most difficult type of domain shift. [...]
Drones in Public: distancing and communication with all users
Abstract: This talk will focus on the role of human-robot interaction with drones in public spaces and be focused on two individual research areas: proximal interactions in shared spaces and improved communication with both end-users and bystanders. Prior work on human-interaction with aerial robots has focused on communication from the users or about the intended direction [...]
End-to-End ‘One Networks’: Learning Regularizers for Least Squares via Deep Neural Networks
Abstract: Linear Restoration Problems (or Linear Inverse Problems) involve reconstructing images or videos from noisy measurement vectors. Notable examples include denoising, inpainting, super-resolution, compressive sensing, deblurring and frame prediction. Often, multiple such tasks should be solved simultaneously, e.g., through Regularized Least Squares, where each individual problem is underdetermined (overcomplete) with infinitely many solutions from which [...]
Data Scalability for Robot Learning
Abstract: Recent progress in robot learning has demonstrated how robots can acquire complex manipulation skills from perceptual inputs through trial and error, particularly with the use of deep neural networks. Despite these successes, the generalization and versatility of robots across environment conditions, tasks, and objects remains a major challenge. And, unfortunately, our existing algorithms and [...]
Carnegie Mellon University
Learning to Generalize beyond Training
Abstract: Generalization, i.e., the ability to adapt to novel scenarios, is the hallmark of human intelligence. While we have systems that excel at cleaning floors, playing complex games, and occasionally beating humans, they are incredibly specific in that they only perform the tasks they are trained for and are miserable at generalization. One of the [...]
Carnegie Mellon University
Planning and Execution using Inaccurate Models with Provable Guarantees on Task Completeness
Abstract: Modern planning methods are effective in computing feasible and optimal plans for robotic tasks when given access to accurate dynamical models. However, robots operating in the real world often face situations that cannot be modeled perfectly before execution. Thus, we only have access to simplified but potentially inaccurate models. This imperfect modeling can lead [...]
Detecting Image Synthesis — Shallow and Deep
Abstract: The proliferation of synthetic media are subject to malicious usages such as disinformation campaigns, posing potential threats to media integrity and democracy. A way to combat this is developing forensics algorithms to identify manipulated media. In the beginning of the talk, I will discuss how one can train a model to detect photos manipulated [...]
Carnegie Mellon University
Shreyas Srivatchan – MSR Thesis Talk
Zoom link: https://cmu.zoom.us/j/92767964421?pwd=N0NqRXZ5M04zQUhObklyZ3ZTL29jZz09 Meeting ID: 927 6796 4421 Password: password Title: Development of a balancing robot as an indoor service agent Abstract: This work presents a robotic system that can navigate human environments, respond to speech commands, and perform simple tasks. To achieve this, a ballbot-type robot that balances and navigates on a single spherical [...]
Carnegie Mellon University
Constraint-Based Coverage Path Planning: A Novel Approach to Achieving Energy-Efficient Coverage
Abstract: Despite substantial technological progress that has driven the proliferation of robots across various industries and aspects of our lives, the lack of a decisive breakthrough in energy storage capabilities has restrained this trend, particularly with respect to mobile robots designed for use in unstructured and unknown field environments. The fact that these domains are [...]
Deep Learning to Distinguish Recalled but Benign Mammography Images in Breast Cancer Screening
Abstract: Breast cancer screening using the standard mammography exam currently exhibits a high false recall rate (11.6% for women in the U.S.). Only a low proportion (0.5%) of women who were recalled for additional workup were actually found to have breast cancer. As a result of the unnecessary stress and follow-up work from these false [...]
Carnegie Mellon University
Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
Abstract: Reinforcement learning has shown great promise for synthesizing realistic human behaviors by learning humanoid control policies from motion capture data. However, it is still very challenging to reproduce sophisticated human skills like ballet dance, or to stably imitate long-term human behaviors with complex transitions. The main difficulty lies in the dynamics mismatch between the [...]
Studying the Evolution of Pedestrian Group Space
Abstract: Imagine walking along a busy sidewalk, do you track the movement of every single individual? Or do you simply group pedestrians with similar moving patterns and then track the movement of this group? Grouping is a common behavior in pedestrian navigation and it is typically inappropriate for a robot to cut through the social [...]
Carnegie Mellon University
Unsupervised Learning of the 4D Audio-Visual World from Sparse Unconstrained Real-World Samples
Abstract: We, humans, can easily observe, explore, and analyze the world we live in. We, however, struggle to share our observation, exploration, and analysis with others. This thesis introduce Computational Studio, computational machinery that can understand, explore, and create the four-dimensional audio-visual world. This allows: (1) humans to communicate with other humans without any loss [...]
Carnegie Mellon University
Xueting Li – MSR Thesis Talk
Title: Multi-agent Deception in Attack-Defense Stochastic Game Abstract: In adversarial scenarios, defending oneself by using deception has recently been studied. A popular direction is to design deceptive defense strategies when the defender has complete information of the game and the attacker doesn't. The work on deception so far models the games as a signal game [...]
Carnegie Mellon University
Physical Interaction and Manipulation of the Environment using Aerial Robots
Abstract: There has been an increasing demand for applications that include aerial robots' physical interactions with their environment, such as contact inspection, package pickup, and drilling. The demand has pushed the research groups towards new robot architectures and methods, but only limited research has been done to enable real-world applications. Fully-actuated multirotors were developed to [...]
Carnegie Mellon University
Visual Recognition Towards Autonomy
Abstract: Perception for autonomy presents a collection of compelling challenges for visual recognition. We focus on three key challenges in this thesis. The first key challenge is learning representations for 2D data such as RGB images. 2D sensing brings unique challenges in scale variance and occlusion. Intuitively, the cues for recognizing a 3px tall object [...]
Carnegie Mellon University
Rich Models and Maps in Factor Graphs with Applications to Tactile Sensing
Abstract: Factor graphs offer a flexible and powerful framework for solving large-scale, nonlinear inference problems as encountered in robot perception. Typically these methods rely on simple models that are efficient to optimize. However, robots often perceive the world through complex, high-dimensional observations. They must in turn infer states that are used downstream by planning and [...]
Carnegie Mellon University
Shubhankar Deshpande – MSR Thesis Talk
Where: https://cmu.zoom.us/j/92520469322?pwd=SjlpTVI5MGdtN1VBakFkRG82bStYQT09 Meeting ID: 925 2046 9322 Passcode: 323696 Calendar Invite: https://tinyurl.com/shuby-msr-thesis-talk-invite Title: Towards Interpretable RL — Interactive Visualizations to Increase Insight Abstract: Visualization tools for supervised learning (SL) allow users to interpret, introspect, and gain an intuition for the successes and failures of their models. While reinforcement learning (RL) practitioners ask many of [...]
Carnegie Mellon University
Distributed Navigation of Quadrotor Teams in Uncertain 3D Workspaces
Abstract: A fundamental requirement for realizing scalable and responsive real-world multi-robot systems for time-sensitive critical applications such as search and rescue or building clearance is a motion-planning and coordination framework that exhibits two essential properties. The first property is safety which encompasses aspects relating to kinodynamic feasibility and collision-avoidance. The second property is reliability which [...]
Carnegie Mellon University
Soft actuators by electrochemical oxidation of liquid metal surfaces
Abstract: Soft robotic systems typically operate through the use of soft actuators constructed from highly deformable materials or liquids. Because of their intrinsic compliance, these actuators can achieve elastic resilience and adaptability similar to their biological counterparts. One challenge with engineering these artificial muscles is the selection of soft materials and activation methods while maintaining [...]
The Plenoptic Camera
Abstract: Imagine a futuristic version of Google Street View that could dial up any possible place in the world, at any possible time. Effectively, such a service would be a recording of the plenoptic function—the hypothetical function described by Adelson and Bergen that captures all light rays passing through space at all times. While the plenoptic function [...]
Carnegie Mellon University
Rohith Pillai – MSR Thesis Talk
ZOOM Link: https://cmu.zoom.us/j/95344974779?pwd=aXlmbktDMFZIUjhyeTRuNWxmeXcwdz09 Meeting ID: 953 4497 4779 Passcode: 783497 Title: 3D Face Reconstruction from Monocular Video and its Applications In the Wild Abstract: 3D face reconstruction is a very popular field of computer vision due to its applications in social media, entertainment and health. However, ever since the introduction of 3D morphable models as [...]
A Graph-Based Method for Joint Instance Segmentation of Point Clouds and Image Sequences
Abstract: While learning-based semantic instance segmentation methods have achieved impressive progress, their use is limited in robotics applications due to reliance on expensive training data annotations and assumptions of single sensor modality or known object classes. We propose a novel graph-based instance segmentation approach that combines information from a 2D image sequence and a 3D [...]
Carnegie Mellon University
Mayra Melendez – MSR Thesis Talk
Continual Reinforcement Learning using Self-Activating Neural Ensembles
Abstract: The ability for an agent to continuously learn new skills without catastrophically forgetting existing knowledge is of critical importance for the development of generally intelligent agents. Most methods devised to address this problem depend heavily on well-defined task boundaries which simplify the problem considerably. Our task-agnostic method, Self-Activating Neural Ensembles (SANE), uses a hierarchical [...]
Carnegie Mellon University
Unsupervised 2D-3D Lifting with Deep Structure Priors
Abstract: Learning to estimate non-rigid 3D structures from 2D imaged observations is bottle-necked by the availability of abundant 3D annotated data. Learning methods that reduce the amount of required annotation is of high practical value. In this regard, Non-Rigid Structure from Motion (NRSfM) methods offer the opportunity to infer 3D structures solely from 2D annotations. [...]
Model Adaptation for Compliant Parallel Robot with Nonstationary Dynamics
Abstract: Soft robots can be constructed with few parts and from a wide variety of materials. This makes them a potentially appealing choice for applications where there are resource constraints on system fabrication. However, soft robot dynamics are difficult to accurately model analytically, due to a multiphysics coupling between shape, forces, temperature, and history of [...]
Adaptive Safety Margins for Safe Replanning under Time-Varying Disturbances
Abstract: Safe real-time navigation is a considerable challenge because engineers often need to work with uncertain vehicle dynamics, variable external disturbances, and imperfect controllers. A common strategy used to address safety is to employ hand-defined margins for obstacle inflation. However, arbitrary static margins often fail in more dynamic scenarios, and using worst-case assumptions proves to [...]
HyperDynamics: Generating Expert Dynamics Models by Observation
Abstract: We propose HyperDynamics, a framework that conditions on an agent’s interactions with the environment and optionally its visual observations, and generates the parameters of neural dynamics models based on inferred properties of the dynamical system. Physical and visual properties of the environment that are not part of the low-dimensional state yet affect its temporal [...]
Direct Fitting of Mixture Models
Abstract: There exist many choices of 3D shape representation. Some recent work has advocated for the use of Gaussian Mixture Models as a compact representation for 3D shapes and scenes. These models are typically fit to point clouds, even when the shapes were obtained as 3D meshes. Here we present a formulation for fitting Gaussian [...]
Carnegie Mellon University
Terrain Perception using Structured Light for Micro-Rovers
Abstract: With continuing advancement in technology, the future of planetary exploration is likely to be dominated by robotic missions. Yet rovers capable of science investigations are slow and bulky with very limited computing which prohibits demonstrating full autonomy. These rovers are also risk averse due to their huge mission cost. However there is a new [...]
Photorealistic Reconstruction of Landmarks and People using Implicit Scene Representation
Abstract: Reconstructing scenes to synthesize novel views is a long standing problem in Computer Vision and Graphics. Recently, implicit scene representations have shown novel view synthesis results of unprecedented quality, like the ones of Neural Radiance Fields (NeRF), which use the weights of a multi-layer perceptron to model the volumetric density and color of a [...]
Carnegie Mellon University
Bayesian Models for Science-Driven Robotic Exploration
Abstract: Planetary rovers have traversed many kilometers and made major scientific discoveries. However, they spend a considerable amount of time awaiting instructions from ground operators. The reason is that they are designed for automated science data collection, not for autonomous exploration. The exploration of more distant worlds with stronger communication constraints will require a new [...]
Carnegie Mellon University
Expressive Real-time Intersection Scheduling: New Methods for Adaptive Traffic Signal Control
Abstract: Traffic congestion is a widespread problem throughout global metropolitan areas. In this thesis, we consider methods to optimize the performance of traffic signals to reduce congestion. We begin by presenting Expressive Real-time Intersection Scheduling (ERIS), a schedule-driven intersection control strategy that runs independently on each intersection in a traffic network. For each intersection, ERIS [...]
Verification and Accreditation of Artificial Intelligence
Abstract: This work involves formally verifying a trained model's adherence to important design specifications for the purpose of model accreditation. Accreditation of a trained model requires enumeration of the explicit operational conditions under which the model is certified to meet all necessary specifications. By verifying model adherence to specifications set by developers, we increase the [...]
Carnegie Mellon University
Direct-drive Hands: Making Robot Hands Transparent and Reactive to Contacts
Abstract: Industrial manipulators and end-effectors are a vital driver of the automation revolution. These robot hands, designed to reject disturbances with stiffness and strength, are inferior to their human counterparts. Human hands are dexterous and nimble effectors capable of a variety of interactions with the environment. Through this thesis we wish to answer a question: [...]
Towards Discriminative and Domain-Invariant Feature Learning
Abstract: Deep neural networks have achieved great success in various visual applications, when trained with large amounts of labeled in-domain data. However, the networks usually suffer from a heavy performance drop on the data whose distribution is quite different from the training one. Domain adaptation methods aim to deal with such performance gap caused by [...]
Learning Efficient Visual Representation on Model, Data, Label and Beyond
Abstract: Efficient deep learning is a broad concept that we aim to learn compressed deep models and develop training algorithms to improve the efficiency of model representations, data and label utilization, etc. In recent years, deep neural networks have been recognized as one of the most effective techniques for many learning tasks, also, in the [...]
Self-supervised Learning and Generalization
Abstract: Contrastive self-supervised learning is a highly effective way of learning representations that are useful for, i.e. generalise, to a wide range of downstream vision tasks and datasets. In the first part of the talk, I will present MoCHi, our recently published contrastive self-supervised learning approach (NeurIPS 2020) that is able to learn transferable representations [...]
Carnegie Mellon University
Teleoperation via Intuition: Safe and Intent Oriented Navigation
Abstract: This thesis aims to enable seamless teleoperation of a mobile robot by a human operator, such that the robot navigates in unstructured environments following the operator’s intent intuitively, safely, and efficiently. The roles of the human and robot are disproportionate in traditional teleoperation: The human is responsible for most of the autonomy of the [...]
Enabling Robots to Cooperate & Compete: Distributed Optimization & Game Theoretic Methods for Multiple Interacting Robots
Abstract: For robots to effectively operate in our world, they must master the skills of dynamic interaction. Autonomous cars must safely negotiate their trajectories with other vehicles and pedestrians as they drive to their destinations. UAVs must avoid collisions with other aircraft, as well as dynamic obstacles on the ground. Disaster response robots must coordinate [...]
Learning to see from few labels
Abstract: Computer vision systems today exhibit a rich and accurate understanding of the visual world, but increasingly rely on learning on large labeled datasets to do so. This reliance on large labeled datasets is a problem especially when one considers difficult perception tasks, or novel domains where annotations might require effort or expertise. We thus [...]
Carnegie Mellon University
Towards the Automated Design of Neural Networks
Abstract: Neural architecture search (NAS) is recently proposed to automate the process of designing network architectures. Instead of manually designing network architectures, NAS automatically finds the optimal architecture in a data-driven way. Despite its impressive progress, NAS is still far from being widely adopted as a common paradigm for architecture design in practice. This thesis [...]
The Role of Manipulation Primitives in Building Dexterous Robotic Systems
Abstract: I will start this talk by illustrating four different perspectives that we as a community have embraced to study robotic manipulation: 1) controlling a simplified model of the mechanics of interaction with an object; 2) using haptic feedback such as force or tactile to control the interaction with an environment; 3) planning sequences or [...]
Seeing the unseen: inferring unobserved information from multi-modal data
Abstract: As humans we can never fully observe the world around us and yet we are able to build remarkably useful models of it from our limited sensory data. Machine learning problems are often required to operate in a similar setup, that is the one of inferring unobserved information from the observed one. Partial observations [...]
Carnegie Mellon University
Analysis of Deadlock in Multirobot Systems
Abstract: Collision avoidance for multirobot systems is a well-studied problem. Recently, control barrier functions (CBFs) have been proposed for synthesizing controllers that guarantee safety while simultaneously encouraging goal stabilization for multiple robots. However, it has been noted that reactive control synthesis methods (such as CBFs) are prone to deadlock, an equilibrium of system dynamics that [...]
Design and Analysis of Open-Source Educational Haptic Devices
Abstract: The sense of touch (haptics) is an active perceptual system used from our earliest days to discover the world around us. However, formal education is not designed to take advantage of this sensory modality. As a result, very little is known about the effects of using haptics in K-12 and higher education or the [...]
Towards AI for 3D Content Creation
Abstract: 3D content is key in several domains such as architecture, film, gaming, and robotics. However, creating 3D content can be very time consuming -- the artists need to sculpt high quality 3d assets, compose them into large worlds, and bring these worlds to life by writing behaviour models that "drives" the characters around in [...]
Move over, MSE! – New probabilistic models of motion
Abstract: Data-driven character animation holds great promise for games, film, virtual avatars and social robots. A "virtual AI actor" that moves in response to intuitive, high-level input could turn 3D animators into directors, instead of requiring them to laboriously pose the character for each frame of animation, as is the case today. However, the high [...]
Understanding the Placenta: Towards an Objective Pregnancy Screening
Abstract: My research focusses on the development of a pregnancy screening tool, that will be: (i) system and user-independent; and (ii) provides a quantifi able measure of placental health. With this end, I am working towards the design of a multiparametric quantitative ultrasound (QUS) based placental tissue characterization method. The method would potentially identify the [...]
Human-Robot Interactive Collaboration & Communication
Abstract: Autonomous and anthropomorphic robots are poised to play a critical role in manufacturing, healthcare and the services industry in the near future. However, for this vision to become a reality, robots need to efficiently communicate and interact with their human partners. Rather than traditional remote controls and programming languages, adaptive and transparent techniques for [...]
Interleaving Graph Search and Trajectory Optimization for Aggressive Quadrotor Flight
Abstract: Quadrotors can achieve aggressive flight by tracking complex maneuvers and rapidly changing directions. Planning for aggressive flight with trajectory optimization could be incredibly fast, even in higher dimensions, and can account for dynamics of the quadrotor, however, only provides a locally optimal solution. On the other hand, planning with discrete graph search can handle [...]
See, Hear, Explore: Curiosity via Audio-Visual Association
Abstract: Exploration is one of the core challenges in reinforcement learning. A common formulation of curiosity-driven exploration uses the difference between the real future and the future predicted by a learned model. However, predicting the future is an inherently difficult task which can be ill-posed in the face of stochasticity. In this work, we introduce [...]
Carnegie Mellon University
Robots “R” Us: 25 years of Robotics Technology Development and Commercialization at NREC
Abstract: Since its founding in 1979, the Robotics Institute (RI) at Carnegie Mellon University has been leading the world in robotics research and education. In the mid 1990s, RI created NREC as the applied R&D center within the Institute with a specific mission to apply robotics technology in an impactful way on real-world applications. In this talk, I will go over [...]
Relational Reasoning for Multi-Agent Systems
Abstract: Multi-agent interacting systems are prevalent in the world, from purely physical systems to complicated social dynamics systems. The interactions between entities / components can give rise to very complex behavior patterns at the level of both individuals and the whole system. In many real-world multi-agent interacting systems (e.g., traffic participants, mobile robots, sports players), [...]
Carnegie Mellon University
Dynamical Model Learning and Inversion for Aggressive Quadrotor Flight
Quadrotor applications have seen a surge recently and many tasks require precise and accurate controls. Flying fast is critical in many applications and the limited onboard power source makes completing tasks quickly even more important. Staying on a desired course while traveling at high speeds and high accelerations is difficult due to complex and stochastic [...]
MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video
Abstract: We present a method to capture temporally coherent dynamic clothing deformation from a monocular RGB video input. In contrast to the existing literature, our method does not require a pre-scanned personalized mesh template, and thus can be applied to in-the-wild videos. To constrain the output to a valid deformation space, we build statistical deformation [...]
Towards an Intelligence Architecture for Human-Robot Teaming
Abstract: Advances in autonomy are enabling intelligent robotic systems to enter human-centric environments like factories, homes and workplaces. To be effective as a teammate, we expect robots to accomplish more than performing simplistic repetitive tasks; they must perceive, reason, perform semantic tasks in a human-like way. A robot's ability to act intelligently is fundamentally tied [...]
Carnegie Mellon University
Robust Manipulation with Active Compliance
Abstract: Human manipulation skills are filled with creative use of physical contacts to move the object about the hand and in the environment. However, it is difficult for robot manipulators to enjoy this dexterity since contacts may cause the manipulation task to fail by introducing huge forces or unexpected change of constraints, especially when modeling [...]
Carnegie Mellon University
Open-world Object Detection and Tracking
Abstract: Computer vision today excels at recognizing narrow slices of the real world: our models seem to accurately detect objects like cats, cars, or chairs in benchmark datasets. However, deploying models requires that they work in the open world, which includes arbitrary objects in diverse settings. Current methods struggle on both axes: they recognize only [...]
Self-supervised learning for visual recognition
Abstract: We are interested in learning visual representations that are discriminative for semantic image understanding tasks such as object classification, detection, and segmentation in images/videos. A common approach to obtain such features is to use supervised learning. However, this requires manual annotation of images, which is costly, ambiguous, and prone to errors. In contrast, self-supervised [...]
Policy Decomposition : Approximate Optimal Control with Suboptimality Estimates
Abstract: Owing to the curse of dimensionality, numerically computing global policies to optimal control problems for complex dynamical systems quickly becomes intractable. In consequence, a number of approximation methods have been developed. However, none of the current methods can quantify by how much the resulting control underperforms the elusive globally optimal solution. We propose Policy [...]