PhD Speaking Qualifier
Carnegie Mellon University
Expressive Real-time Intersection Scheduling
Abstract: Traffic congestion is a major annoyance throughout global metropolitan areas. This talk will present Expressive Real-time Intersection Scheduling (ERIS), a schedule-driven control strategy for adaptive intersection control to reduce traffic congestion. ERIS maintains separate estimates for each lane approaching a traffic intersection allowing it to more accurately estimate the effects of scheduling decisions than [...]
Carnegie Mellon University
Scaling up Self Supervised Robot Learning
Abstract Robot learning holds promise in alleviating several real world problems, by performing complex behaviors in complex environments. But what is the right way to train these robots? Our methods on self supervision shows encouraging results on several tasks like grasping objects, pushing objects and even flying drones. One key challenge with these methods is [...]
Carnegie Mellon University
Data Collection for Screwdriving
Abstract: As the use of robotic manipulation in manufacturing continues to increase, the robustness requirements for fastening operations such as screwdriving increase as well. To investigate the reliability of screwdriving and the diverse failure categories that can arise, we collected a dataset of screwdriving operations and manually classified them into stages and result categories. I [...]
Carnegie Mellon University
Predictive Corrective Networks for Action Detection
Abstract: Although computer vision has seen significant advances in static image analysis, the relatively slow advances in video tasks such as action detection suggest we're struggling to build effective temporal models. In this talk, I will present a few main ideas that drive contemporary approaches, such as "two-stream networks" and "3D" convolutional networks. I'll also [...]
Characterization of Anchoring in Granular Soils
Abstract: I will present the results of tests conducted to characterize the pullout force of an anchor buried in cohesionless soils. Sensitivity analyses were conducted to understand how key measures of fin geometry affect an anchor's pullout force. To generalize the data collected, I propose a dimensionless model for predicting the performance of arbitrary fin [...]
Carnegie Mellon University
Liquid Metal-Microelectronics Integration for a Sensorized Soft Robot Skin
Abstract: Progress in the emerging field of soft robotics depends on the integration of sensors that are capable of sensing, power regulation, and signal processing. Commercially available microelectronics are well suited for these needs, as well as small enough to preserve the natural mechanics of a host system. Here, we present a method for integrating [...]
Model Predictive Path Following for Wheeled Mobile Robots
Abstract: The navigation success of a wheeled mobile robotic mission is directly correlated to the degree of accuracy to which the robot can follow a given path. This, in turn, is largely affected by two factors: a) the environment and b) the intrinsic properties of the robot – its design, actuation mechanism etc. In the [...]
Carnegie Mellon University
Generative Models of Orbital and In Situ Data for Autonomous Science
Abstract: The mapping and characterization of planetary bodies relies on the analysis of data collected by spacecraft and orbiters. For example, the instruments carried by the Mars Reconnaissance Orbiter have been crucial in the mapping of landforms, stratigraphy, minerals, and ice of Mars. These instruments provide extensive contextual information, but factors such as sparsity, resolution, [...]
Design with Interpretability in Mind: An Alternate Ethos for Data Science
Abstract: The fields of Machine Learning and Data Science generally follow the paradigm that “the ends justify the means”, where improving predictive power of an algorithm is considered of paramount value, even when implemented at the expense of model intelligibility. While accuracy is an important performance metric, interpretability should be a major consideration for many [...]
Carnegie Mellon University
Multi-Robot Routing and Scheduling with Spatio-Temporal And Ordering Constraints
Abstract We consider the problem of allocation and routing a fleet of robots to service a given set of locations while minimizing makespan. The service start times for the locations are subject to AND/OR type precedence constraints. Spatio-temporal constraints prohibit certain states from all feasible schedules where a state is defined as a tuple of [...]
Carnegie Mellon University
Robot Learning in Homes – Improving Generalization and Reducing Dataset Bias
Abstract: Data-driven approaches to solving robotic tasks have gained a lot of traction in recent years. However, most existing policies are trained on large-scale datasets collected in curated lab settings. If we aim to deploy these models in unstructured visual environments like people’s homes, they will be unable to cope with the mismatch in data [...]
Carnegie Mellon University
Online, Interactive User Guidance for High-dimensional, Constrained Motion Planning
Abstract: We consider the problem of planning a collision-free path for a high-dimensional robot. Specifically, we suggest a planning framework where a motion-planning algorithm can obtain guidance from a user. In contrast to existing approaches that try to speed up planning by incorporating experiences or demonstrations ahead of planning, we suggest to seek user guidance [...]
Carnegie Mellon University
Robot Task Execution by Policy Adaptation and Switching Among Multiple Tasks
Abstract: While mobile robots reliably perform service tasks by accurately localizing and safely navigating while avoiding obstacles, they do not respond in any other way to their surroundings. In this work, we introduce two methods that enable the robots to be more responsive to their environment, including humans and other robots. The first algorithm enables [...]
Carnegie Mellon University
Persistent Multi-Robot Mapping in an Uncertain Environment
Abstract: We present a system that addresses the challenge of concurrently mapping, scheduling, and deploying a team of energy-constrained robots to persistently cover an unknown and potentially dynamic environment. This system can passively maintain an accurate representation of occupied space, allowing robots reliable access for monitoring, study, or search and rescue. Current state-of-the-art algorithms only [...]
Carnegie Mellon University
Direct Drive Hands: Force-Motion Transparency in Gripper Design
Abstract: The Direct Drive Hand (DDHand) project is exploring a new design philosophy for grippers. The conventional approach is to prioritize clamping force, leading to high gear ratios, slow motion, and poor transmission of force/motion signals. Instead, the DDHand prioritizes transparency: we view the gripper as a signal transmission channel, and seek high-bandwidth, high-fidelity transmission [...]
Carnegie Mellon University
Learning to Align without Geometric Supervision
Abstract: Extracting geometric information from image data is a highly nonlinear problem that exhibits in a number of visual recognition tasks such as object localization, facial landmark tracking and human pose estimation. Successful alignment across image data often serves as a crucial component in making them possible. In this talk, I will present how one [...]
Carnegie Mellon University
Towards Safe and Robust Behavior Mixing for Multi-Robot Systems
Abstract: Multi-robot systems have been widely studied for extending its capability of accomplishing complex tasks through cooperative behaviors. In large-scale multi-robot behavior mixing, the heterogeneous robotic team executes simultaneously multiple behaviors or sequences of behaviors with various task-prescribed controllers in real time to increase efficiency in parallel tasks. Key to the success of behavior mixing [...]
Carnegie Mellon University
Speeding Up Search-based Motion Planning Via Conservative Heuristics
Abstract: Weighted A* search (wA*) is a popular tool for robot motion-planning. Its efficiency however depends on the quality of heuristic function used. In fact, it has been shown that the correlation between the heuristic function and the true cost-to-goal significantly affects the efficiency of the search, when used with a large weight on the [...]
Carnegie Mellon University
Toward a New Type of Agile and Dexterous Mobile Manipulator
Abstract: Mobile robot bases have been developed over many decades, but only recently have researchers added arms to these bases, opening up the rich field of mobile manipulation. Most of these robots either need wide, heavy, statically-stable bases that may or may not be omnidirectional to support the arms and provide stability. Such robot bases, [...]
Carnegie Mellon University
Dexterous Manipulation via Simple Robot Hands
Abstract: Most of the industrial robotic applications nowadays can only deal with pick-and-place manipulation, in which fixed graspings are the only interactions between the object and the robot hand. Simple hands, such as pinch grippers and suction cups, suffice to accomplish such tasks. However, there exist many unsolved automation problems where more dexterous manipulations are [...]
Carnegie Mellon University
Contrastive View Predictive Learning with 3D-Bottlenecked RNNs
Abstract: In this talk, I will describe our recent work on neural architectures for visual recognition, which use 3D not as input nor as the desired output space, but rather as the bottleneck of the learned representations. We consider embodied agents moving in otherwise static worlds equipped with these architectures; they learn 3D visual feature [...]
Toward Intent Recognition through Nonverbal Behaviors in Assistive Co-Manipulation
Abstract: Robots are becoming more versatile, increasing the available opportunities to use them in situations that aid people in everyday tasks. For example, recent research has investigated robot manipulators for assisting people with motor impairments in activities of daily living such as eating a meal. To form successful collaborations in these interactions, researchers need to [...]
Carnegie Mellon University
Rotational Distributions for Pose Estimation
Abstract: For robots to operate robustly in the real world, they should be aware of their uncertainty, particularly when estimating the position and orientation, or pose, of objects. This uncertainty can be caused by many factors, such as occlusions, poor lighting, or object symmetry. These factors can naturally induce an inherent ambiguity in terms of [...]
Carnegie Mellon University
Manipulation Planning using Pushing or Pulling Primitives
Abstract: Humans manipulate objects using a wide range of actions, such as grasping, pushing, pulling, in-hand rolling, and more. This observation has lead to much research about modeling and learning individual manipulation actions. To better understand the impact of action models on planning and executing manipulation actions, we applied manipulation planning with pushing and pulling [...]
Carnegie Mellon University
Scaling Up Deep Learning with Model and Algorithm Awareness
Abstract: In recent years, the pace of innovations in the fields of deep learning has accelerated. To cope with the sheer computational complexity of training large ML models on large datasets, researchers in the systems and ML communities have created software systems that parallelize training algorithms over multiple CPUs or GPUs (multi-device parallelism), or even [...]
Carnegie Mellon University
Online and Consistent Occupancy Grid Mapping
Abstract: Actively exploring and mapping an unknown environment requires integration of both simultaneous localization and mapping (SLAM) and path planning methods. Path planning relies on a map that contains free and occupied space information and is efficient to query, while the role of SLAM is to keep the map consistent as new measurements are continuously [...]
Carnegie Mellon University
A Planning Framework for Persistent, Multi-UAV Coverage with Global Deconfliction
Abstract: Planning for multi-robot coverage seeks to determine collision-free paths for a fleet of robots, enabling them to collectively observe points of interest in an environment. Persistent coverage is a variant of traditional coverage where coverage-levels in the environment decay over time. Thus, robots have to continuously revisit parts of the environment to maintain a [...]
Carnegie Mellon University
Online Kinodynamic Planning for Teams of Aerial Robots in 3-D Workspaces
Abstract: An efficient online planning or replanning methodology is a critical requirement for scalable and responsive real world multi-robot deployments. The need to replan typically stems from the invalidation of existing plans due to incomplete knowledge of the environment, or, from scenarios that necessitate changing goal locations in response to evolving application requirements. In this [...]
Carnegie Mellon University
Open-world 3D Object Detection
Abstract: Perception for autonomous robots presents a set of unique challenges: finding the right representation for 3D signals, adapting to an open-world setting, and exploiting geometric priors. Successfully detecting objects regardless of their labels lays a solid foundation for safe navigation. I will present two of my recent works in this line. First, I will [...]
When to use CNNs for Inverse Problems in Vision
Abstract: Reconstruction tasks in computer vision aim fundamentally to recover an undetermined signal from a set of noisy measurements. Examples include super-resolution, image denoising, and non-rigid structure from motion\cite{Kong_2019}, all of which have seen recent advancements through deep learning. However, earlier work made extensive use of sparse signal reconstruction frameworks (e.g. convolutional sparse coding). While [...]
Tendon Driven Foam Hands
Abstract: There has been great progress in soft robot design, manufacture, and control in recent years, and soft robots are a tool of choice for safe and robust handling of objects in conditions of uncertainty. Still, dexterous in-hand manipulation using soft robots remains a challenge. This talk introduces a novel class of soft robots in [...]
Carnegie Mellon University
Towards a Good Representation For Reinforcement Learning
Abstract: Deep reinforcement learning has achieved many successes over the recent years. However, its high sample complexity and the difficulty in specifying a reward function have limited its application. In this talk, I will take a representation learning perspective towards these issues. Is it possible to map from the raw observation, potentially in high dimension, [...]
Carnegie Mellon University
Resource-constrained learning and inference for visual perception
Zoom Link Abstract Real-world applications usually require computer vision algorithms to meet certain resource constraints. In this talk, I will present evaluation methods and principled solutions for both cases of training and testing. First, I will talk about a formal setting for studying training under the non-asymptotic, resource-constrained regime, i.e., budgeted training. We analyze the [...]
Carnegie Mellon University
Planning and Execution using Inaccurate Models with Provable Guarantees
Zoom Link Abstract: Models used in modern planning problems to simulate outcomes of real world action executions are becoming increasingly complex, ranging from simulators that do physics-based reasoning to precomputed analytical motion primitives. However, robots operating in the real world often face situations not modeled by these models before execution. This imperfect modeling can lead [...]
Carnegie Mellon University
The Effect of Locomotion Configuration on Discrete Obstacle Traversal for a Small Tracked Vehicle
Zoom Link Abstract: As mobile robots are being designed for increasingly rugged and unknown terrain, mechanical reconfigurability presents one possibility for improving vehicle efficiency and mobility. To validate this idea, we created an 18.5-kg modular tracked vehicle with adjustable track tension, track width, track length, and sprocket diameter. In this talk, I will explain the [...]
Carnegie Mellon University
Task-Driven Modular Networks for Zero-Shot Compositional Learning
Zoom Link Abstract: One of the hallmarks of human intelligence is the ability to compose learned knowledge into novel concepts which can be recognized without a single training example. In contrast, current state-of-the-art methods require hundreds of training examples for each possible category to build reliable and accurate classifiers. To alleviate this striking difference in [...]
Image to LiDAR Map Registration using Late Feature Projection
Zoom Link Abstract: Accurate localization is essential for autonomous operation in many problem domains. This is most often performed by comparing LiDAR scans collected in real-time to a HD point cloud based map. While this enables centimeter-level accuracy, it depends on an expensive LiDAR sensor at run time. Recently, efforts have been underway to reduce [...]
Carnegie Mellon University
A Theory of Fermat Paths for Non-line-of-sight Shape Reconstruction
Zoom Link Abstract: Traditionally, computer vision systems and algorithms, such as stereo vision, and shape from shading, have been developed to mimic human vision. As a consequence, a lot of these systems operate under constraints that we take for granted in human vision. An example of such a constraint is that the scene of interest [...]
Learning Contextual Actions for Heuristic Search-Based Motion Planning
Zoom Link Abstract: Heuristic search-based motion planning can be computationally costly in large state and action spaces. In this work we explore the use of generative models to learn contextual actions for successor generation in heuristic search. We focus on cases where the robot operates in similar environments, i.e. environments drawn from some underlying distribution. [...]
Carnegie Mellon University
Interactive Weak Supervision – Learning Useful Heuristics for Data Labeling
Zoom Link Abstract: Obtaining large annotated datasets is critical for training successful machine learning models and it is frequently a bottleneck in practice. Weak supervision offers a promising alternative for producing labeled datasets without ground truth annotations by generating probabilistic labels using multiple noisy heuristics. This process can scale to large amounts of data and [...]
Carnegie Mellon University
Learning Active Task-Oriented Exploration Policies for Bridging the Sim-to-Real Gap
Zoom Link Abstract: Training robotic policies in simulation suffers from the sim-to-real gap, as simulated dynamics can be different from real-world dynamics. Past works tackled this problem through domain randomization and online system-identification. The former is sensitive to the manually-specified training distribution of dynamics parameters and can result in behaviors that are overly conservative. The [...]
Interferometric light transmission probing with coded mutual intensity
Zoom Link Abstract: We introduce a new interferometric imaging methodology that we term interferometry with coded mutual intensity, which allows selectively imaging photon paths based on attributes such as their length and endpoints. At the core of our methodology is a new technical result that shows that manipulating the spatial coherence properties of the light [...]
Sparse Spatial Hashing for Dense 3D Reconstruction
Abstract: Real-world 3D data is locally dense but globally sparse. Therefore, efficient sparse data structures are an essential component of dense 3D perception for computer vision and robotics. We manifest the power of spatial hashing by two typical tasks: dense scene reconstruction and global registration. In the first task, we accelerate volumetric integration and surface [...]
Carnegie Mellon University
3D Multi-Object Tracking for Autonomous Driving
Abstract: 3D multi-object tracking (MOT) is a key component of a perception system for autonomous driving. Due to recent progress in 3D object detection in the context of autonomous driving, recent work in 3D MOT primarily focuses on online tracking with the use of a tracking-by-detection pipeline. In this talk, we introduce a new 3D [...]
Carnegie Mellon University
Ergodic Trajectory Optimization for Information Gathering
Abstract: Planetary robots currently rely on significant guidance from expert human operators. Science autonomy adds algorithms and methods for autonomous scientific exploration to improve efficiency of discovery and overcome limited communication bandwidth and delay bottlenecks. This research focuses on planning trajectories for information gathering and choosing sampling locations that have the most informative samples. We [...]
Carnegie Mellon University
Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
Abstract: Reinforcement learning has shown great promise for synthesizing realistic human behaviors by learning humanoid control policies from motion capture data. However, it is still very challenging to reproduce sophisticated human skills like ballet dance, or to stably imitate long-term human behaviors with complex transitions. The main difficulty lies in the dynamics mismatch between the [...]
Studying the Evolution of Pedestrian Group Space
Abstract: Imagine walking along a busy sidewalk, do you track the movement of every single individual? Or do you simply group pedestrians with similar moving patterns and then track the movement of this group? Grouping is a common behavior in pedestrian navigation and it is typically inappropriate for a robot to cut through the social [...]
Carnegie Mellon University
Soft actuators by electrochemical oxidation of liquid metal surfaces
Abstract: Soft robotic systems typically operate through the use of soft actuators constructed from highly deformable materials or liquids. Because of their intrinsic compliance, these actuators can achieve elastic resilience and adaptability similar to their biological counterparts. One challenge with engineering these artificial muscles is the selection of soft materials and activation methods while maintaining [...]
A Graph-Based Method for Joint Instance Segmentation of Point Clouds and Image Sequences
Abstract: While learning-based semantic instance segmentation methods have achieved impressive progress, their use is limited in robotics applications due to reliance on expensive training data annotations and assumptions of single sensor modality or known object classes. We propose a novel graph-based instance segmentation approach that combines information from a 2D image sequence and a 3D [...]
Continual Reinforcement Learning using Self-Activating Neural Ensembles
Abstract: The ability for an agent to continuously learn new skills without catastrophically forgetting existing knowledge is of critical importance for the development of generally intelligent agents. Most methods devised to address this problem depend heavily on well-defined task boundaries which simplify the problem considerably. Our task-agnostic method, Self-Activating Neural Ensembles (SANE), uses a hierarchical [...]
Carnegie Mellon University
Unsupervised 2D-3D Lifting with Deep Structure Priors
Abstract: Learning to estimate non-rigid 3D structures from 2D imaged observations is bottle-necked by the availability of abundant 3D annotated data. Learning methods that reduce the amount of required annotation is of high practical value. In this regard, Non-Rigid Structure from Motion (NRSfM) methods offer the opportunity to infer 3D structures solely from 2D annotations. [...]
Model Adaptation for Compliant Parallel Robot with Nonstationary Dynamics
Abstract: Soft robots can be constructed with few parts and from a wide variety of materials. This makes them a potentially appealing choice for applications where there are resource constraints on system fabrication. However, soft robot dynamics are difficult to accurately model analytically, due to a multiphysics coupling between shape, forces, temperature, and history of [...]
Adaptive Safety Margins for Safe Replanning under Time-Varying Disturbances
Abstract: Safe real-time navigation is a considerable challenge because engineers often need to work with uncertain vehicle dynamics, variable external disturbances, and imperfect controllers. A common strategy used to address safety is to employ hand-defined margins for obstacle inflation. However, arbitrary static margins often fail in more dynamic scenarios, and using worst-case assumptions proves to [...]
HyperDynamics: Generating Expert Dynamics Models by Observation
Abstract: We propose HyperDynamics, a framework that conditions on an agent’s interactions with the environment and optionally its visual observations, and generates the parameters of neural dynamics models based on inferred properties of the dynamical system. Physical and visual properties of the environment that are not part of the low-dimensional state yet affect its temporal [...]
Direct Fitting of Mixture Models
Abstract: There exist many choices of 3D shape representation. Some recent work has advocated for the use of Gaussian Mixture Models as a compact representation for 3D shapes and scenes. These models are typically fit to point clouds, even when the shapes were obtained as 3D meshes. Here we present a formulation for fitting Gaussian [...]
Carnegie Mellon University
Terrain Perception using Structured Light for Micro-Rovers
Abstract: With continuing advancement in technology, the future of planetary exploration is likely to be dominated by robotic missions. Yet rovers capable of science investigations are slow and bulky with very limited computing which prohibits demonstrating full autonomy. These rovers are also risk averse due to their huge mission cost. However there is a new [...]
Carnegie Mellon University
Analysis of Deadlock in Multirobot Systems
Abstract: Collision avoidance for multirobot systems is a well-studied problem. Recently, control barrier functions (CBFs) have been proposed for synthesizing controllers that guarantee safety while simultaneously encouraging goal stabilization for multiple robots. However, it has been noted that reactive control synthesis methods (such as CBFs) are prone to deadlock, an equilibrium of system dynamics that [...]
Interleaving Graph Search and Trajectory Optimization for Aggressive Quadrotor Flight
Abstract: Quadrotors can achieve aggressive flight by tracking complex maneuvers and rapidly changing directions. Planning for aggressive flight with trajectory optimization could be incredibly fast, even in higher dimensions, and can account for dynamics of the quadrotor, however, only provides a locally optimal solution. On the other hand, planning with discrete graph search can handle [...]
See, Hear, Explore: Curiosity via Audio-Visual Association
Abstract: Exploration is one of the core challenges in reinforcement learning. A common formulation of curiosity-driven exploration uses the difference between the real future and the future predicted by a learned model. However, predicting the future is an inherently difficult task which can be ill-posed in the face of stochasticity. In this work, we introduce [...]
MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video
Abstract: We present a method to capture temporally coherent dynamic clothing deformation from a monocular RGB video input. In contrast to the existing literature, our method does not require a pre-scanned personalized mesh template, and thus can be applied to in-the-wild videos. To constrain the output to a valid deformation space, we build statistical deformation [...]
Policy Decomposition : Approximate Optimal Control with Suboptimality Estimates
Abstract: Owing to the curse of dimensionality, numerically computing global policies to optimal control problems for complex dynamical systems quickly becomes intractable. In consequence, a number of approximation methods have been developed. However, none of the current methods can quantify by how much the resulting control underperforms the elusive globally optimal solution. We propose Policy [...]
Inverse Reinforcement Learning with Explicit Policy Estimates
Abstract: Various methods for solving the inverse reinforcement learning (IRL) problem have been developed independently in machine learning and economics. In particular, the method of Maximum Causal Entropy IRL is based on the perspective of entropy maximization, while related advances in the field of economics instead assume the existence of unobserved action shocks to explain [...]
Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation
Abstract: To perform manipulation tasks in the real world, robots need to operate on objects with various shapes, sizes and without access to geometric models. It is often infeasible to train monolithic neural network policies across such large variance in object properties. Towards this generalization challenge, we propose task-axis controllers, which are defined relative to [...]
Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies
Abstract: Real-world environments, such as homes, hospitals, and restaurants, often contain many objects that a robot could possibly manipulate. However, for a given manipulation task, only a small number of objects and object properties may actually be relevant. This talk presents CREST (Causal Reasoning for Efficient Structure Transfer), our approach to learn the relevant state [...]
Grasping Transparent, Specular, and Deformable Objects
Abstract: A large body of research exists on grasping for objects with ideal properties like Lambertian reflectance and rigidity. On the other hand, real-world environments contain many objects for which such properties do not hold, such as transparent, specular, and deformable objects. For such objects, new approaches are required to achieve the same level of [...]
PoseIt: A Visual-Tactile Dataset of Holding Poses for Grasp Stability Analysis
Abstract: When humans grasp objects in the real world, we often move our arm to hold the object in a different pose where we can use it. In contrast, typical lab settings only study the stability of the grasp immediately after lifting, without any subsequent re-positioning of the arm. However, an object’s stability could vary [...]
Planning to Minimize Human and Robot Efforts Over Tasks
Abstract: It is not feasible to pre-program robots a priori for every possible task they may encounter in unstructured domains. Upon encountering a task that a robot can't solve, one common strategy is to teach it new skills via demonstrations. However, demonstrating a task can often be more cumbersome than performing the task directly. This [...]
Fine-Tuning Offline Reinforcement Learning with Model-Based Policy Optimization
Abstract: In offline reinforcement learning (RL), we attempt to learn a control policy from a fixed dataset of environment interactions. This setting has the potential benefit of allowing us to learn effective policies without needing to collect additional interactive data, which can be expensive or dangerous in real-world systems. However, traditional off-policy RL methods tend [...]
Modeling Coupled Human-Robot Motion for Provable Safety
Abstract: Guide robots that help users who are blind or low vision navigate through crowds and complex environments show promise for improving accessibility in public spaces. These robots must provide real-time safety guarantees for the users, which requires accurate modeling of their behavior in the context of closely coupled human-robot motion. This model must also [...]
Diminished Reality for Close Quarters Robotic Telemanipulation
Abstract: In robot telemanipulation tasks, the robot itself can sometimes occlude a target object from the user's view. We investigate the potential of diminished reality to address this problem. Our method uses an optical see-through head-mounted display to create a diminished reality illusion that the robot is transparent, allowing users to see occluded areas behind [...]
Learning Compositional Radiance Fields of Dynamic Human Heads
Meeting ID: 942 4671 0665 Passcode: jkhzoom Abstract: Photorealistic rendering of dynamic humans is an important capability for telepresence systems. Recently, neural rendering methods have been developed to create high-fidelity models of humans and objects. Some of these methods do not produce results with high-enough fidelity for driveable human models (Neural Volumes) whereas others have [...]
An Experimental Design Perspective on Model-Based Reinforcement Learning
Abstract: In many practical applications of RL, it is expensive to observe state transitions from the environment. For example, in the problem of plasma control for nuclear fusion, computing the next state for a given state-action pair requires querying an expensive transition function which can lead to many hours of computer simulation or dollars of [...]
Learning Model Preconditions for Planning with Multiple Models
Abstract: Different models can provide differing levels of fidelity when a robot is planning. Analytical models are often fast to evaluate but only work in limited ranges of conditions. Meanwhile, physics simulators are effective at modeling complex interactions between objects but are typically more computationally expensive. Learning when to switch between the various models can [...]
Reconstructing common objects to interact with
Abstract: We humans are able to understand 3D shapes of common daily objects and interact with them from a wide range of categories. We understand cups are usually cylinder-like and we can easily predict the shape of one particular cup, both in isolation or even when it is held by a human. We aim to [...]
A causal framework to diagnose and fix issues with doors
Abstract: Many animals, such as ravens, (and a fortiori humans) exhibit a great deal of physical intelligence that allows them to solve complex multi-step physical puzzles. This ability indicates an understanding or a faculty to represent causality and mechanisms, understand when something goes wrong, and figure out how to deal with it. As a step [...]