PhD Thesis Proposal
Spatial, Temporal and Spatio-temporal Correspondence for Computer Vision Problems
Event Location: NSH 1507Abstract: Many computer vision problems, such as object classification, motion estimation or shape registration rely on solving the correspondence problem. Existing algorithms to solve the correspondence problems are usually NP-hard, difficult to approximate and lack mechanism for feature selection. This proposal addresses the correspondence problem in computer vision, and proposes two new [...]
Characteristic Prosody and Meaning in Speech Synthesis
Event Location: NSH 3305Abstract: A new trainable model and method for generating prosody for speech synthesis is proposed, in which a relationship between pragmatics, intonational phonology, and performance is made explicit in a language-neutral manner. The literature of pragmatics and intonation is broad, but produces only coarse descriptions and few actionable models (Steedman; Pierrehumbert and [...]
3D Manipulation of Objects in Photographs
Event Location: NSH 1305Abstract: This thesis provides a system that allows users to interact with objects and human subjects in photographs in three dimensions. Current photo-editing tools provide a diverse array of operations in photographs. However, they are limited to the 2D plane of the image. The key contribution of this thesis is to leverage [...]
View Trajectory Planning for Modeling Planetary Features under Transient Illumination
Event Location: GHC 8102Abstract: This research addresses the problem of planning views for modeling large, local, substantially 3D terrain features, such as skylights, at long range from surface rovers. Skylights, which are vertical cave entrances, and other pits have been identified in recent high-resolution images of the Moon and Mars. Planetary pits are interesting scientific [...]
Mapping & Localizing on the Beating Heart
Event Location: NSH 1507Abstract: In the United States the annual cost of cardiovascular disease exceeds $475 billion. Medical imaging costs more than $100 billion annually, with cardiovascular imaging representing almost one-third. Any approach that can reduce the need for imaging in cardiovascular care has the potential to provide significant savings. Without direct visualization of the [...]
Experience Graphs: Leveraging Experience in Planning
Event Location: NSH 1305Abstract: Motion planning is a central problem in robotics and is crucial to finding paths to navigate and manipulate safely and efficiently. Ideally, we want planners which find paths quickly and of good quality. Additionally, planners should generate predictable motions, which are safer when operating in the presence of humans. While the [...]
Subdimensional Expansion: An Approach to Computationally Tractable Multirobot Path Planning
Event Location: NSH 1305Abstract: Planning optimal paths for large numbers of robots is computationally expensive. In our research, we developed a new framework for multirobot path planning called subdimensional expansion, which initially plans for each robot individually, and then coordinates motion among the robots as needed. More specifically, subdimensional expansion initially creates a one-dimensional search [...]
Multiple Prediction Learning: Learning to Optimize Sequences via Submodular Maximization
Event Location: NSH 1305Abstract: Increasingly, real world problems require multiple predictions. For instance in advertisement placement on the web, a list of advertisements is placed on a page with the objective of maximizing click- through rate. Traditionally machine learning has focused on producing a single best prediction. More generally, in this thesis, we build an [...]
Pose Inference Machines: Efficient and Accurate Human Pose Estimation from Images
Event Location: GHC 8102Abstract: Fast and accurate human pose estimation enables a wide spectrum of applications from interactive control, markerless motion capture and gesture recognition to providing rich semantic information for higher level vision tasks such as scene understanding. The goal of this work is to develop an accurate and real-time human pose estimation system [...]
Handheld Micromanipulator for Robot-Assisted Microsurgery
Event Location: NSH 3305Abstract: Robot-assisted surgery has been increasingly adopted in a wide variety of surgical applications because it offers fine manipulation with high precision and dexterity. Despite the commercial success of robotic platforms, practical use in microsurgery is still challenging due to a considerable level of accuracy required at sub-millimeter scales. Limited visualization and [...]
State Estimation for Humanoid Robots
Event Location: GHC 8102Abstract: This proposal focuses on dynamic model based state estimation for hydraulic humanoid robots. The goal is to produce state estimates that are robust and achieve good performance when combined with the controller. Three issues are addressed in this proposal. 1. How to handle modelling error using state estimation? 2. How to [...]
Bayesian Aggregation of Evidence for Detection and Characterization of Patterns in Multiple Noisy Observations
Event Location: NSH 3305Abstract: Effective use of Machine Learning to support extracting maximal information from limited sensor data is one of the important research challenges in robotic sensing. This thesis develops techniques for detecting and characterizing patterns in noisy sensor data. Our Bayesian Aggregation (BA) algorithmic framework can leverage data fusion from multiple low Signal-To-Noise [...]
Factoring Scenes into 3D Structure an Style
Event Location: GHC 4405Abstract: Given a single image of a scene, humans have few issues answering questions about the 3D structure like “is this facing upwards?” even though mathematically speaking this should be impossible. We have similarly have few issues accounting for this 3D structure in answering viewpoint independent questions like “is this the same [...]
Evaluation of Decentralized Reactive Swing-Leg Controllers on Powered Robotic Legs
Event Location: NSH 1507Abstract: We present work to transfer decentralized neuromuscular control strategies of human locomotion to powered segmented robotic legs. State-of-the-art robotic locomotion control approaches, like centralized planning and tracking in fully robotic systems and predefined motion pattern replay in prosthetic systems, do not enable the dynamism and reactiveness of able-bodied humans. Animals largely [...]
A Team Planning Language with Markup for Usage Adaptation
Event Location: NSH 1507Abstract: As robots become more reliable and user interfaces (UI) become more powerful, human-robot-agent teams are being applied to an increasing number of real world problems. Common areas of interest for these teams include scientific investigation, surveillance, disaster response, and search and rescue. These systems often employ a set of high level [...]
Creative Robotic Systems for Talent-Based Learning
Event Location: GHC 4405Abstract: In recent years, the U.S. educational system has fallen short in training the technology innovators of the future. To do so, we must give students the experience of designing and creating technological artifacts, rather than relegating students to the role of technology consumers, and must provide students with engineering activities that [...]
Online Hierarchical Optimization for Humanoid Control
Event Location: NSH 3305Abstract: This thesis proposal presents an online approach that is based on hierarchical optimization for controlling humanoid robots. While our primary focus is on developing a fast and robust walking system that is able to follow desired footsteps, full body manipulation capability is also achieved. The proposed hierarchical system consists of three [...]
Expressive Motion for Low Degree of Freedom Robots
Event Location: NSH 3305Abstract: As social and collaborative robots move into everyday life, the need for algorithms enabling their acceptance becomes critical. The proposed work will create a framework for Expressive Motion generation that allows a robot to use modified task motions to communicate its state naturally and efficiently, including mental state, task state, and [...]
Planning for Robot Recovery with Varying Model Fidelity
Event Location: GHC 4405Abstract: For robots to be deployable, robot systems need to operate reliably for long periods. Reliable operation requires robots to reason about events that impede task progression. Robots that consider failure recovery strategies are more dependable and robust to unforeseen events. Recovery strategies allow the robot to continue consistent task operation even [...]
Learning, Prediction, and Optimization for Hybrid Vehicle Energy Management
Event Location: GHC 4405Abstract: Each year, hybrid vehicles command a larger portion of total vehicles on the road. These vehicles combine multiple sources of energy, such as batteries and gasoline, which have different strengths and weaknesses. Active management of these energy sources can increase vehicle efficiency, longevity, or performance. Optimizing energy management is highly sensitive [...]
Online Lidar and Vision based Ego-motion Estimation and Mapping
Event Location: NSH 1507Abstract: In many real-world applications, motion estimation and mapping must be conducted online and in real-time. The real-time motion estimates are important for controlling and maneuvering autonomous vehicles, and the maps generated online are critical for obstacle avoidance and path planning. Further, the final map can be used as a representation of [...]
The Optical Coherence Tomography Microsurgical Augmented Reality System (OCT-MARS): a novel device for microsurgeries
Event Location: GHC 4405Abstract: I am proposing the development and testing of a novel augmented reality system for stereomicroscopic surgeries. This system will allow surgeons to view real-time medical image data as an in-situ overlay within the surgical field. Our application for the device is in-situ intraoperative OCT during surgeries on the anterior segment of [...]
Efficient Manipulation Task Planning via Reuse-Informed Optimization of Planning Effort
Event Location: GHC 6501Abstract: In order to assist humans with dangerous or menial tasks, autonomous robots will need to act under significant time and energy constraints. At task time, the amount of effort a robot spends planning directly detracts from its total performance. Manipulation tasks, however, present challenges to efficient motion planning. They are often [...]
Supervised Descent Method
Event Location: GHC 2109Abstract: In this thesis, I aim to present a new perspective on looking at classical problems in Computer Vision. Instead of spending efforts on finding novel objective functions and improving existing optimization algorithms, I aim to demonstrate that formulating those problems as policy learning yields more flexible and efficient algorithms. Our recently [...]
Correcting Regions of Inaccurate Modeling for Online Performance Improvement
Event Location: NSH 3002Abstract: To make intelligent decisions, robots often use models of the effects of their actions on the world. Unfortunately, in complex environments, it is often infeasible to create models that are accurate in every plausible situation, which can lead to suboptimal performance. In these cases, online correction of model inaccuracies can lead [...]
Forecasting Human Motion in Social Situations
Event Location: NSH 1507Abstract: The goal of this thesis is to further our understanding of nonverbal communication by developing methods and representations to quantify and predict human body motion during social interactions. We design a data collection protocol and capture system to obtain 3D body pose and facial annotations for groups of interacting people engaged [...]
Modeling Closed-Loop Adaptive Systems
Event Location: NSH 1305Abstract: Brain-machine interfaces (BMIs) are a powerful class of assistive devices that may one day restore movement ability to paralyzed individuals. These devices act by creating a direct mapping between recorded neural activity and the movement of an external actuator, like a computer cursor or a robotic arm, bypassing defective neural transmission [...]
Robust Manipulation via Contact Sensing
Event Location: NSH 3305Abstract: Humans effortlessly manipulate objects in cluttered and uncertain environments. In contrast, most robotic manipulators are limited to carefully engineered environments, e.g. factories, to circumvent the difficulty of manipulation under uncertainty. Contact sensors can provide robots with with the feedback vital to addressing this limitation. However, there are three principal challenges to [...]
Efficiently Sampling from Underlying Physical Models
Event Location: NSH 1305Abstract: Robots today have the capability to collect terabytes of data about their environment and travel kilometers in a single day, yet they are still constrained by one fundamental resource: time. Time limits the number of samples a robot can collect, sites it can analyze, and data it can return for review, [...]
Articulated 3D SLAM
Event Location: NSH 3305Abstract: Consider a robot arm with a hand-mounted sensor. In order to interact with and understand the world, the robot must be able to reconstruct it using its sensor by moving its joints to scan the scene. If the robot's kinematics are known with absolute certainty, the problem is the simple 3D mapping problem. [...]
Navigation for Balancing Robots in Contact with People
Event Location: NSH 1305Abstract: This work describes methods for advancing the state of the art in mobile robot navigation and physical Human-Robot Interaction (pHRI). An enabling technology in this effort is the ballbot, a person-sized mobile robot that balances on a ball. This underactuated robot presents unique challenges in planning, navigation, and control; however, it also has significant advantages over [...]
Robust Rearrangement Planning using Nonprehensile Interaction
Event Location: GHC 6501Abstract: As we work to move robots out of factories and into human environments, we must empower robots to interact freely in unstructured, cluttered spaces. Humans do this easily, using diverse, whole-arm, nonprehensile actions such as pushing or pulling in everyday tasks. These interaction strategies make difficult tasks easier and impossible tasks [...]
Learning Policies for Shared Autonomy
Event Location: NSH 3305Abstract: In shared autonomy, user input and robot autonomy are combined to control a robot to achieve a goal. Most prior work accomplishes this by augmenting user input with some autonomous strategy for that goal. We take a different viewpoint, treating the user as a policy minimizing some cost function. Our aim [...]
Direct Multiple View Visual Simultaneous Localization And Mapping
Event Location: NSH 1305Abstract: We propose a direct, featureless, Lucas-Kanade-based method as a reliable Visual Simultaneous Localization And Mapping (VSLAM) solution in challenging environments, where feature detection and precise subpixel localization may be unreliable. Current state-of-the-art direct methods have been shown to perform well on a range of challenging datasets. Nonetheless, they have been limited [...]
A Reflex-based Neuromuscular Control Model of Human Locomotion
Event Location: GHC 4405Abstract: The neural controls of human and animal locomotion have been studied over centuries. However, much of our knowledge about the locomotion control of complex species, especially humans, still relies on extrapolating from what is known in simpler animals. One barrier for better understanding the control of human locomotion is that we [...]
Moment-based Algorithms for Structured Prediction
Event Location: GHC 4405Abstract: Latent variable models constitute a compact representation for complex, high dimensional data. This is useful in many applications in Robotics and Natural Language Processing (NLP). Latent models are however very hard to learn, in part because of the non-convexity of the likelihood function. The main difficult is associated with estimating the [...]
Reasoning About Spatial Patterns of Human Behavior During Group Conversations with Robots
Event Location: NSH 3305Abstract: Human conversations are a type of jointly focused encounter in which the participants gather together and openly cooperate to sustain a single focus of attention. When the participants are standing, their cooperation can be observed in their spatial organization. People position and orient themselves to maximize their opportunity to monitor one [...]
Low-Cost Fine Particulate Monitors and their Applications
Event Location: NSH 3305Abstract: Air quality has long been a major health concern for citizens around the world, and increased levels of exposure to fine particulate matter (PM2.5) has been definitively linked to serious health effects such as cardiovascular disease, respiratory illness, and increased mortality. PM2.5 is one of six attainment criteria pollutants used by [...]
Active Illumination for the Real World
Event Location: GHC 4405Abstract: Active illumination systems use a controllable light source and a light sensor to measure properties of a scene. For such a system to work reliably it must be able to handle the effects of global light transport, bright ambient light, defocus and scene motion. The goal of this thesis is to [...]
Planning for a Small Team of Heterogeneous Robots: from Collaborative Exploration to Collaborative Localization
Event Location: GHC 8102Abstract: Robots have become increasingly adept at performing a wide variety of tasks in the world. However, many of these tasks can benefit tremendously from having more than a single robot simultaneously working on the problem. Multiple robots can aid in a search and rescue mission each scouting a subsection of the [...]
The theory and implementation of spring mass running on ATRIAS, a bipedal robot
Event Location: GHC 8102Abstract: We expect legged robots to be highly mobile. Human walking and running can execute quick changes in speed and direction, even on non-flat ground. Indeed, analysis of simplified models shows that these quantities can be tightly controlled by adjusting the leg placement between steps and that the leg placement can also [...]
Performing Self-Scheduled Services in the Spare Time of a Mobile Autonomous Service Robot
Event Location: NSH 3305Abstract: Mobile autonomous service robots perform services requested by users at specific times. The goal of this thesis is to explore how a service robot can make use of the spare time between user requests. We propose the robot performs self-scheduled services for which it receives reward from the users. Our proposed [...]
Compact Generative Models of Point Cloud Data for 3D Perception
Event Location: GHC 8102Abstract: One of the most fundamental tasks for any robotics application is the ability to adequately assimilate and respond to incoming sensor data. The goal of this thesis is to explore how statistical models for point cloud data can facilitate, accelerate, and unify many common tasks in the area of 3D perception. [...]
Rover Traverse Planning for Prolonged Solar-Powered Polar Exploration
Event Location: GHC 4405Abstract: Future in-situ resource utilization promises to enable affordable exploration of space and extend human presence in the Solar System by minimizing the materials that must be carried from Earth. This is predicated on the existence of economic quantities of native materials that can be converted into consumable resources, such as water, [...]
From Supervised to Unsupervised Facial Action Discovery
Event Location: NSH 1507Abstract: Facial actions speak louder than words. Facial actions can reveal a person's emotion, intention, and physical state; and make possible a range of applications that include market research, human-robot interaction, drowsiness detection, and clinical and developmental psychology research. In this proposal, we investigate both supervised and unsupervised approaches to facial action [...]
Semantic Mapping for Robotic Navigation and Exploration
Event Location: GHC 2109Abstract: The last decade has seen remarkable advances in 3D perception for robotics. Advances in range sensing and SLAM now allow robots to easily acquire detailed 3D maps of their environment in real-time. However, adaptive robot behavior requires an understanding the environment that goes beyond pure geometry. A step above purely geometric [...]
Traversability Prediction for Planetary Rovers in Granular Terrain
Event Location: GHC 2109Abstract: Loose, granular terrain can cause rovers to slip and sink, inhibiting mobility and sometimes even permanently entrapping a vehicle. Traversability of granular terrain is difficult to foresee using traditional, non-contact sensing methods, such as cameras and LIDAR. This inability to detect loose terrain hazards has caused significant delays for rovers on [...]
In-Field Parameter Selection for Perception Context Adaptation
Event Location: NSH 1305Abstract: Perception and state estimation are critical robot competencies that remain difficult to harden and generalize. This is due in part to the incredible complexity of modern perception systems which are commonly comprised of dozens of components with hundreds of parameters overall. Selecting a configuration of parameters relies on a human's understanding [...]
Safe, Efficient, and Robust Predictive Control of Constrained Nonlinear Systems
Event Location: NSH 1305Abstract: As autonomous systems are deployed in increasingly complex and uncertain environments, safe, accurate, and robust feedback control techniques are required to ensure reliable operation. Accurate trajectory tracking is essential to complete a variety of tasks, but this may be difficult if the system’s dynamics change online, e.g., due to environmental effects [...]
Algorithms, Implementation, and Studies on Eating with a Shared Control Robot Arm
Event Location: NSH 1305Abstract: People with upper extremity disabilities are gaining increased independence through the use of assisted devices such as wheelchair-mounted robotic arms. However, the increased capability and dexterity of these robotic arms also makes them challenging to control through accessible interfaces like joysticks, sip-and-puff, and buttons that are lower-dimensional than the control space [...]
Decision-Theoretic User Modeling for Human-Robot Mutual Adaptation
Event Location: GHC 4405Abstract: In many application domains, robots co-exist in the same physical space with humans and aim to become trustworthy partners. We particularly envision personal robots arranging furniture with a human partner, manufacturing robots performing spar assembly with human co-workers, or rehabilitation robots assisting spinal cord injury patients. In such collaborative settings, humans [...]
Data-Driven Statistical Models of Robotic Manipulation
Event Location: NSH 3305Abstract: Improving robotic manipulation is critical for robots to be actively useful in real-world factories and homes. While some success has been shown in simulation and controlled environments, robots are slow, clumsy, and not general or robust enough when interacting with their environment. By contrast, humans effortlessly manipulate objects. One possible reason [...]
Training Strategies for Time Series: Learning for Filtering and Reinforcement Learning
Event Location: GHC 4405Abstract: Data driven approaches to modeling time-series are important in a variety of applications from market prediction in economics to the simulation of robotic systems. However, traditional supervised machine learning techniques designed for i.i.d. data often perform poorly on these sequential problems. This thesis proposes that time series and sequential prediction, whether [...]
Deliberative Perception
Event Location: NSH 1507Abstract: A recurrent and elementary machine perception task is to localize objects of interest in the physical world, be it objects on a warehouse shelf or cars on a road. In many real-world examples, this task entails localizing specific object instances with known 3D models. For example, a warehouse robot equipped with [...]
Designing Data Visualization and Crowdsourcing Systems in Community-based Citizen Science
Event Location: GHC 4405Abstract: Citizen science forges partnerships between experts and citizens through collaboration and has become a trend in public participation in scientific research over the past decade. While public participation has been applied to science education, researchers recently noticed that this strategy can contribute to participatory democracy, which empowers citizens to advocate for [...]
Discovering and Leveraging Visual Structure for Large-scale Recognition
Event Location: GHC 4405Abstract: Visual Recognition has seen tremendous advances in the last decade. This progress is primarily due to learning algorithms trained with two key ingredients: large amounts of data and extensive supervision. While acquiring visual data is cheap, getting it labeled is far more expensive. So how do we enable learning algorithms to [...]
Adaptive Motion Planning
Event Location: NSH 1305Abstract: Mobile robots are increasingly being deployed in the real world in response to a heightened demand for applications such as transportation, delivery and inspection. The motion planning systems for these robots are expected to have consistent performance across the wide range of scenarios that they encounter. While state-of-the art planners can [...]
Robust and Natural Gait via Neuromuscular Control for Transfemoral Prostheses
Nitish Thatte Carnegie Mellon University February 03, 2017, Robust and Natural Gait via Neuromuscular Control for Transfemoral Prostheses, Porter Hall A19C Abstract We present work towards developing a control method for powered knee and ankle prostheses based on a neuromuscular model of human locomotion. Previous research applying neuromuscular control to simulated biped models and to [...]
Data-Driven Visual Forecasting
Event Location: GHC 4405Abstract: Understanding the temporal dimension of images is a fundamental part of computer vision. Humans are able to interpret how the entities in an image will change over time. However, it has only been relatively recently that researchers have focused on visual forecasting—getting machines to anticipate events in the visual world before [...]
Flexible and High-Fidelity Off-Road Lidar Scene Simulation
Event Location: NSH 3305Abstract: As the target scale of robot operations grows, so too does the challenge of developing software for such systems. It may be difficult, unsafe, or expensive to develop software on enough real-world conditions. Similarly, as the target applications of learning algorithms grow, so too do the challenges of gathering adequate training [...]
Extensions of the Principal Fiber Bundle Model for Locomoting Robots
Event Location: NSH 1507Abstract: Our goal is to establish a rigorous formulation for modeling the locomotion of a broad class of robotic systems. Recent research has identified a number of systems with the structure of a principal fiber bundle. This framework has led to a number of tools for analysis and motion planning applicable to [...]
Learning to Learn and Structure Learning in Model Spaces for Small Sample Visual Recognition
Yuxiong Wang Carnegie Mellon University Abstract Understanding how to recognize novel categories from few examples for both humans and machines remains a fundamental challenge. Humans are remarkably able to grasp a new category and make meaningful generalization to novel instances from just few examples. By contrast, state-of-the-art machine learning techniques and visual recognition systems typically [...]
Harnessing Task Mechanics for Robotic Manipulation: Modeling, Uncertainty Reduction and Control
Jiaji Zhou Carnegie Mellon University Abstract A high-fidelity and tractable mechanics model of the physical interaction is essential for autonomous robotic manipulation in complex and uncertain environments. Nonetheless, task mechanics are often ignored or nullified in most robotic manipulation systems. This thesis proposal addresses three aspects of harnessing task mechanics: mechanics model learning, uncertainty reduction [...]
Sankalp Arora: Safe, Efficient Data Gathering in Physical Spaces
Sankalp Arora Ph.D. Thesis Proposal Abstract: Reliable and efficient acquisition of data from physical spaces will have countless applications in industry, policy and defense. The capability of gaining information at different scales makes Micro-Aerial Vehicles (MAVs) excellent for aforementioned applications. However, reasoning about information gathering at multiple resolution is NP-Hard and the state of the [...]
Sasanka Nagavalli: Behavior Composition in Human Interaction with Robotic Swarms
Sasanka Nagavalli Ph.D. Thesis Proposal Abstract: Robotic swarms are multi-robot systems whose global behavior emerges from local interactions between individual robots and spatially proximal neighboring robots. Each robot can be programmed with several local control laws that can be activated depending on an operator's choice of global swarm behavior (e.g. flocking, aggregation, formation control, area [...]
Dynamic 3D Reconstruction from the Crowd
Abstract: With the advent of affordable and high-quality smartphone cameras, any significant event, such as a wedding ceremony, a surprised birthday party, or a concert, can be easily captured from multiple of cameras. Automatically organizing such large scale visual data and creating a comprehensive 3D scene model for event browsing is an unsolved problem. State [...]
Measuring and Modeling Kinesic Signals in Social Communication
Abstract: Humans use subtle and elaborate body signals to convey their thoughts, emotions, and intentions. "Kinesics" is a term that refers to the study of such body movements used in social communication, including facial expressions and hand gestures. Understanding kinesic signals is fundamental to understanding human communication; it is among the key technical barriers to [...]
Probabilistic Approaches for Pose Estimation
Abstract: Pose estimation is central to several robotics applications such as registration, manipulation, SLAM, etc. In this thesis, we develop probabilistic approaches for fast and accurate pose estimation. A fundamental contribution of this thesis is formulating pose estimation in a parameter space in which the problem is truly linear and thus globally optimal solutions can [...]
Ada J. Zhang: Personalized Human Motion Classification
Abstract: Algorithms for human motion understanding have a wide variety of applications, including health monitoring, performance assessment, and user interfaces. However, differences between individual styles make it difficult to achieve robust performance, particularly for individuals who were not in the training population. We believe that adapting algorithms to individual behaviors is essential for effective human [...]
Carnegie Mellon University
Learning to learn from simulation: Using simulations to expedite learning on robots
Abstract: Robot controllers, including locomotion controllers, often consist of expert-designed heuristics. These heuristics can be hard to tune, particularly in higher dimensions. It is typical to use simulation to tune or learn these parameters and test on hardware. However, controllers learned in simulation often don't transfer to hardware due to model mismatch. This necessitates controller [...]
Carnegie Mellon University
Visual Learning without Exhaustive Supervision
Abstract Machine learning models have led to remarkable progress in visual recognition. A key driving factor for this progress is the abundance of labeled data. Over the years, researchers have spent a lot of effort curating visual data and carefully labeling it. However, moving forward, it seems impossible to annotate the vast amounts of visual [...]
Carnegie Mellon University
Learning with Clusters
Abstract As machine learning becomes more ubiquitous, clustering has evolved from primarily a data analysis tool into an integrated component of complex machine learning systems, including those involving dimensionality reduction, anomaly detection, network analysis, image segmentation and classifying groups of data. With this integration into multi-stage systems comes a need to better understand interactions between [...]
Carnegie Mellon University
Intra-Robot Replanning and Learning for Multi-Robot Teams in Complex Dynamic Domains
Abstract: In complex dynamic multi-robot domains, we have a set of individual robots that must coordinate together through a team planner that inevitably makes assumptions based on probabilities about the state of world and the actions of the individuals. Eventually, the individuals may encounter failures, because the team planner’s models of the states and actions [...]
Carnegie Mellon University
Automated Collaborations Among Neighborhood-based Search Heuristics
Abstract: For this thesis, we propose to study how to automatically combine multiple neighborhood-based heuristics. For most computationally challenging problems, there exists multiple heuristics, and it is generally the case that any such heuristic exploits only a limited number of aspects among all the possible problem characteristics that we can think of. As a result, [...]
Carnegie Mellon University
Computational Design Tools for Accessible Robotics
Abstract: A grand vision in robotics is that of a future wherein robots are integrated in daily human life just as smart phones are today. Such pervasive integration of robots would greatly benefit from faster design and manufacturing of robots that cater to individual needs. However, robots of today often take years to be created [...]
Carnegie Mellon University
Soft-Matter Robotic Materials
Abstract: Soft machines and electronics are key components for emerging applications in wearable biomonitoring, human-machine interaction, and soft robotics. In contrast to conventional machines and electronics, soft-matter technologies provide a method for replicating these traditionally rigid devices using intrinsically soft materials that exhibit properties similar to soft biological tissue. This provides a path forward for [...]
Carnegie Mellon University
Exploiting Redundancy for Learning Visual Representations
Abstract: Our visual world is highly structured and the visual data is highly redundant. In recent years, the computer vision field has been transformed by the success of Convolutional Neural Networks (ConvNets). However, the structure and redundancy in visual data has not been well explored in deep learning. The benefits of exploring data redundancy are [...]
Carnegie Mellon University
Towards Generalization and Efficiency of Reinforcement Learning
Abstract In classic supervised machine learning, a learning agent behaves as a passive observer: it receives examples from some external environment which it has no control over and then makes predictions. The predictions the agent made will not affect any future examples it will see (i.e., examples are identically and independently sampled from some unknown [...]
Carnegie Mellon University
Depth Imaging for Navigation in Challenging Environments
Abstract: Depth sensors for robust navigation must measure scenes in darkness, bright light, and in scattering media. Scanning LIDAR devices are the most robust to these conditions, but capture sparse measurements, are slow, and expensive. Consumer depth cameras, on the other hand, are inexpensive and produce dense, high rate depth measurements, but fail in bright [...]
Carnegie Mellon University
Observing Humans In Their Natural Habitat: Data, Algorithms, and Analysis
Abstract: Computer vision has a great potential to help our daily lives by searching for lost keys, watering flowers or reminding us to take a pill. To succeed with such tasks, computer vision methods need to be trained from real and diverse examples of our daily dynamic scenes. First, we need to give computers insight [...]
Carnegie Mellon University
Analysis of Spatio-Temporally Varying Features in Optical Coherence Tomographic (OCT) and Ultrasound (US) Image Sequences
Abstract: Optical Coherence Tomography (OCT) and Ultrasound (US) are non-ionizing and non-invasive imaging modalities that are clinically used to visualize anatomical structures in the body. OCT has been widely adopted in clinical practice due to its micron-scale resolution to visualize in-vivo structures of the eye. Ultra-High Frequency Ultrasound (UHFUS) can capture images at a depth [...]
Carnegie Mellon University
Planning for Energy-Efficient Coverage and Exploratory Deviation by Robots in Rivers
Abstract: Manual collection of environmental data over a large area can be a time-consuming, costly, and even dangerous process, making it a perfect candidate for automation with mobile robots. Despite this clear suitability and numerous advances in robotics resulting in decreased costs, improved reliability, and increased ease of use, the problem of powering autonomous robots [...]
Carnegie Mellon University
Learning to Forecast Egocentric and Allocentric Behavior in Diverse Domains
Abstract: Reasoning about the future is fundamental to intelligence. In this work, I consider the problem of reasoning about the future actions of an intelligent agent. This poses two key questions. How can we build learning-based systems to forecast the behavior of observed agents (third-person, "allocentric forecasting")? More challenging is the question: how should we [...]
Carnegie Mellon University
Search-based Robust Motion Planning under Uncertainty Guided by Multiple Heuristics
Abstract: Motion planning has achieved a great success in many robotic applications but still suffers in the real world under ample uncertainty. For example, manipulation involves interaction with unstructured and stochastic environments, which results in motion uncertainty. Perception that provides understanding of the environment is also not perfect, which in turn leads to sensing uncertainty. [...]
Learning Multi-Modal Navigation for Unmanned Ground Vehicles
Abstract: A robot that operates efficiently in a team with a human in an unstructured outdoor environment must be able to translate commands from a modality that is intuitive to its operator into actions. This capability is especially important as robots become ubiquitous and interact with untrained users. For this to happen, the robot must [...]
Carnegie Mellon University
Scaling up Self-Supervision for Robot Learning
Abstract: A general purpose robot will need to interact with objects in cluttered environments with minimal supervision. Machine learning provides methods that can deal with these complex tasks without explicitly modelling the environment. More recently, deep learning techniques combined with large scale data has revolutionized the fields of computer vision, language processing and reinforcement learning. [...]
Carnegie Mellon University
MRFMaps: A Representation for Multi-Hypothesis Dense Volumetric SLAM
Abstract: Robust robotic flight requires tightly coupled perception and control. Conventional approaches employ a SLAM algorithm to infer the most likely trajectory and then generate an occupancy grid map using dense sensor data for planning purposes. In such approaches all the robustness and accuracy costs are offset to the SLAM algorithm; if there are any [...]
Carnegie Mellon University
Sparse and Dense Methods for Underwater Localization and Mapping with Imaging Sonar
Abstract: Imaging sonars have been used for a variety of tasks geared towards increasing autonomy of underwater vehicles: image registration and mosaicing, vehicle localization, object recognition, mapping, and path planning, to name a few. However, the complexity of the image formation has led many algorithms to make the restrictive assumption that the scene geometry is [...]
Carnegie Mellon University
Deep Interpretable Non-rigid Structure from Motion
Abstract: Current non-rigid structure from motion (NRSfM) algorithms are limited with respect to: (i) the number of images, and (ii) the type of shape variability they can handle. This has hampered the practical utility of NRSfM for many applications within vision. Deep Neural Networks (DNNs) are an obvious candidate to help with such issue. However, [...]
Carnegie Mellon University
Vision with Small Baselines
Abstract: Portable camera sensor systems are becoming more and more popular in computer vision applications such as autonomous driving, virtual reality, robotics manipulation and surveillance, due to the decreasing expense and size of RGB camera. Despite the compactness and portability of the small baseline vision systems, it is well-known that the uncertainty in range finding [...]
Carnegie Mellon University
Machine Imagination: Data-driven User Controllable Visual Content Creation
Abstract: Humans have the remarkable ability to create visual worlds far beyond what could be seen by human eye, including inferring the state of unobserved, imagining the unknown, and thinking about diverse possibilities about what lies in the future. Machines lack this inquisitive ability despite the current revolution in machine learning and computer vision. We [...]
Carnegie Mellon University
Spatiotemporal Understanding of People Using Scenes, Objects, and Poses
Abstract: Humans are arguably one of the most important entities that AI systems would need to understand to be useful and ubiquitous. From autonomous cars observing pedestrians to assistive robots helping the elderly, a large part of this understanding is focused on recognizing human actions, and potentially, their intentions. Humans themselves are quite good at [...]
Carnegie Mellon University
Robust Active SLAM for Real-time Large-scale Indoor Dense 3D Reconstruction
Abstract: Real-time dense 3D reconstruction of indoor environments has been a popular research field due to its wide application, such as inspection, virtual / augmented reality (VR/AR), and service robotics. While many state-of-the-art algorithms are capable of reconstructing accurate 3D dense models in general indoor scenes, robustness is still an unsolved problem for all of [...]
Carnegie Mellon University
Terrain Relative Navigation for Lunar Polar Roving: Exploiting Geometry, Shadows, and Planning
Abstract: Water ice at the lunar poles could be the most valuable resource beyond planet Earth. However, that value is not foregone, and can only be determined by rovers that evaluate the distributions of abundance, concentration, and characteristics of this ice. The near-term explorations will be solar and unlikely to endure night, and hence are [...]
Carnegie Mellon University
Hybrid Soft Sensing in Robotic Systems
Abstract: The increasing prevalence of wearable technology in our daily lives has created a demand for safe and robust sensing skins. Largely inspired by human skin, the ultimate goal of electronic skins is to measure diverse sensory information, conform to surfaces, and avoid interfering with the natural mechanics of the host or user. These demands [...]
Carnegie Mellon University
Deep Reinforcement Learning Representations for Robotics
Abstract: A long standing goal of robotics research is to create algorithms that can automatically learn complex control strategies from scratch. Part of the challenge of applying such algorithms to robots is the choice of representation. While RL algorithms have been successfully applied to many robotics tasks such as Ball-in-a-Cup and various RoboCup soccer domains, [...]
Carnegie Mellon University
Learning multi-robot behaviors for online control
Abstract: Finding dynamically feasible and safe global plans for multi-agent teams in real world applications is enormously difficult because the decision branching factor, when considering all possible interactions across agents and an environment, is usually intractable. Humans, however, have great success in the multi-agent planning domain by using behaviors: practiced, coordinated responses for groups of [...]
Carnegie Mellon University
Routing for Persistent Exploration in Dynamic Environments with Teams of Energy-Constrained Robots
Abstract: In domains requiring effective situational awareness with limited resources, prioritizing focus is critical. Search and rescue tasks require fast identification of safe avenues for rescuers to traverse the area. Inspection tasks must realize trends over long durations to identify issues caused by the confluence of high-stress modes that compound into catastrophic failure. Deploying robots [...]
Carnegie Mellon University
Enabling Role-Reversible Human-Robot Interaction by Leveraging Standardized Tools for Provider-Receiver Interactions
Abstract: Developing 'social intelligence' for assistive robots to seamlessly interact with humans remains an open research challenge. However, socially assistive robots typically engage in types of interactions that already exist between humans, which makes models of human-human interactions useful to inform the design of robot social behaviors. In particular, in applications such as healthcare, therapy [...]
Carnegie Mellon University
Towards Understanding and Mitigating Biases
Abstract: There are many problems in real life that involve collecting and aggregating evaluation from people, such as conference peer review and peer grading. In this thesis, we consider multiple sources of biases that may arise in this process: (1) human bias -- the data collected from people are noisy and reflect people's calibration criteria [...]
Sensor Planning for Large Numbers of Robots
Abstract: Teams of aerial sensing robots can provide valuable situational awareness to first responders during disasters and emergencies by locating victims, mapping environments, identifying hazards, and locating gas sources. While individual sensing tasks may feature a variety of goals, many tasks will be time-sensitive such as after a widespread disaster or due to imminent danger [...]
Carnegie Mellon University
Resource-Constrained State Estimation with Multi-Modal Sensing
Abstract: Accurate and reliable state estimation is essential for safe mobile robot operation in real-world environments because ego-motion estimates are required by many critical autonomy functions such as control, planning, and mapping. Computing accurate state estimates depends on the physical characteristics of the environment, the selection of suitable sensors to capture that information, and the [...]
Carnegie Mellon University
Open-world Object Detection and Tracking
Abstract: Computer vision today excels at recognition in narrow slices of the real world. Our systems seem to accurately detect cats, cars, or chairs, but largely ignore the vast diversity of objects in the world that are absent from our training datasets. Perception in the open world, however, requires detecting and tracking any object, regardless [...]
Carnegie Mellon University
‘Unboxing’ anomaly detection and panoptic segmentation
Abstract: Panoptic segmentation is a recent problem in computer vision that attempts to classify each pixel in an image according to its semantic and instance label (accomplishing both semantic segmentation and instance segmentation respectively). Most existing panoptic and instance segmentation methods run a detection-first pipeline, where a bounding box is placed around an object and [...]
Carnegie Mellon University
Transfers Between Multiple Service Robots
Abstract: With the deployment of more robots, human-robot interaction will no longer be limited to a one-to-one interaction between a user and a robot. Instead, users will likely have to interact with multiple robots, simultaneously or sequentially, throughout their day to receive services and complete different tasks. In this thesis proposal, I am proposing joint [...]
Carnegie Mellon University
Machine Learning Parallelism Could Be Adaptive, Composable and Automated
Abstract: In recent years, the pace of innovations in the fields of machine learning has accelerated. To cope with the sheer computational complexity of training large ML models on large datasets, researchers in SysML have created algorithms and systems that parallelize ML training and inference over multiple CPUs or GPUs, or even multiple computing nodes [...]
Carnegie Mellon University
Expressive Real-time Intersection Scheduling: New Methods for Adaptive Traffic Signal Control
Abstract: Traffic congestion is a widespread problem throughout global metropolitan areas. In this thesis, we consider methods to optimize the performance of traffic signals to reduce congestion. We begin by presenting Expressive Real-time Intersection Scheduling (ERIS), a schedule-driven intersection control strategy that runs independently on each intersection in a traffic network. For each intersection, ERIS [...]
Adaptive Planning and Control of Wheeled Mobile Robots in Challenging Environments
Abstract: Over the last two decades, we have seen driverless cars conquer the Mojave desert, drive on mars and operate on our streets and warehouses. One of the most fundamental requirements of such robots is their ability to navigate their environment with minimal human oversight. As more robots graduate from the confines of laboratories to [...]
Carnegie Mellon University
Learning Dense 3D Object Reconstruction without Geometric Supervision
Abstract: Geometric alignment across visual data has been the fundamental issue for effective and efficient computer vision algorithms. The established pixel correspondences between images indirectly infer the underlying 3D geometry, physically or semantically. While this builds the foundation of classical multi-view 3D reconstruction algorithms such as Structure from Motion (SfM) and Simultaneous Localization and Mapping [...]
Carnegie Mellon University
Eye Gaze for Assistive Manipulation
Abstract: Full robot autonomy is the traditional goal of robotics research. To work in a human-inhabited world, however, robots will often need to collaborate with humans. Many scenarios require human users to teleoperate robots to perform tasks, a paradigm that appears everywhere from space exploration, to disaster recovery, to assistive robotics. This collaboration enables tasks [...]
Carnegie Mellon University
Stability-Centric Mechanics for Rigid Body Manipulation
Abstract: The repertoire of human manipulation is filled with creative use of contacts to move the object about the hand and the environment. It’s the combination of these skills that makes human manipulation dexterous. However, in most robotic applications the robot just fix all contact points on the object and do grasping. Reliable robot manipulation [...]
Carnegie Mellon University
Robotic Grasping in the Wild
Zoom Link Abstract Robotics and artificial intelligence have witnessed tremendous progress in the past decade. Yet, we are still far from building the general purpose robot butler that can autonomously operate in homes and help with manipulation tasks like household chores. Grasping is an important action primitive for manipulation and needs to generalize to unstructured [...]
Carnegie Mellon University
Active Vision: Autonomous Aerial Cinematography with Learned Artistic Decision-Making
Zoom Link Abstract: Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. Fundamentally, it is a tool with immense potential to improve human creativity, expressiveness, and sharing of experiences. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely [...]
Carnegie Mellon University
Efficient Robot Decision-Making for Achieving Multiple Independent Tasks
Zoom Link Abstract: We focus on robotics applications where a robot is required to accomplish a set of tasks that are partially observable and evolve independently of each other according to their dynamics. One such domain that we target in this work is decision-making for a robot waiter waiting tables at a restaurant. The robot [...]
Carnegie Mellon University
Provably Constant-time Motion Planning
Zoom Link Abstract: In manufacturing and warehouse scenarios, robots often perform recurring manipulation tasks in structured environments. Fast and reliable motion planning is one of the key elements that ensure efficient operations in such environments. A very common example scenario is of manipulators working at conveyor belts, where they have limited time to pick moving [...]
Carnegie Mellon University
Safe and Resilient Multi-Robot Systems: Heterogeneity and Human Presence
Zoom Link Abstract: In the mission of a multi-robot team, the large number of robots behave like a system that relies on networking to enable smooth information propagation and inter-robot interaction as the mission evolves in a collective fashion. Key to the success of mission operation demands for safe and reliable robot interactions within the [...]
Carnegie Mellon University
Heuristics for routing and scheduling of Spatio-Temporal type problems in industrial environments
Zoom Link Abstract: Spatio-temporal problems are fairly common in industrial environments. In practice, these problems come with different characteristics and are often very hard to solve optimally. So practitioners prefer to develop heuristics that exploit mathematical structure specific to the problem for obtaining good performance. In this proposal, I will present work on heuristics for [...]
Carnegie Mellon University
Understanding, Exploiting and Improving Inter-view Relationships
Zoom Link Abstract: Multi-view machine learning has received substantial attention in various applications over recent years. These applications typically involve learning on data obtained from multiple sources of information, such as, for example, in multi-sensor systems such as self-driving cars and patient bed-side monitoring. Learning models for such applications can often benefit from leveraging not [...]
Carnegie Mellon University
Computational Contact Modes for Robotics
Zoom Link Abstract: A central theme in robotics is that of robots interacting with the world through physical contact. Whether it is a walking robot or robotic manipulator picking up an object, such as a spoon, we desire robots that physically interact with their environments. One significant challenge in physical robot interactions involves dealing with [...]
Carnegie Mellon University
Planning and Execution using Inaccurate Models with Provable Guarantees on Task Completeness
Abstract: Modern planning methods are effective in computing feasible and optimal plans for robotic tasks when given access to accurate dynamical models. However, robots operating in the real world often face situations that cannot be modeled perfectly before execution. Thus, we only have access to simplified but potentially inaccurate models. This imperfect modeling can lead [...]
Carnegie Mellon University
Physical Interaction and Manipulation of the Environment using Aerial Robots
Abstract: There has been an increasing demand for applications that include aerial robots' physical interactions with their environment, such as contact inspection, package pickup, and drilling. The demand has pushed the research groups towards new robot architectures and methods, but only limited research has been done to enable real-world applications. Fully-actuated multirotors were developed to [...]
Carnegie Mellon University
Visual Recognition Towards Autonomy
Abstract: Perception for autonomy presents a collection of compelling challenges for visual recognition. We focus on three key challenges in this thesis. The first key challenge is learning representations for 2D data such as RGB images. 2D sensing brings unique challenges in scale variance and occlusion. Intuitively, the cues for recognizing a 3px tall object [...]
Carnegie Mellon University
Rich Models and Maps in Factor Graphs with Applications to Tactile Sensing
Abstract: Factor graphs offer a flexible and powerful framework for solving large-scale, nonlinear inference problems as encountered in robot perception. Typically these methods rely on simple models that are efficient to optimize. However, robots often perceive the world through complex, high-dimensional observations. They must in turn infer states that are used downstream by planning and [...]
Carnegie Mellon University
Distributed Navigation of Quadrotor Teams in Uncertain 3D Workspaces
Abstract: A fundamental requirement for realizing scalable and responsive real-world multi-robot systems for time-sensitive critical applications such as search and rescue or building clearance is a motion-planning and coordination framework that exhibits two essential properties. The first property is safety which encompasses aspects relating to kinodynamic feasibility and collision-avoidance. The second property is reliability which [...]
Carnegie Mellon University
Bayesian Models for Science-Driven Robotic Exploration
Abstract: Planetary rovers have traversed many kilometers and made major scientific discoveries. However, they spend a considerable amount of time awaiting instructions from ground operators. The reason is that they are designed for automated science data collection, not for autonomous exploration. The exploration of more distant worlds with stronger communication constraints will require a new [...]
Verification and Accreditation of Artificial Intelligence
Abstract: This work involves formally verifying a trained model's adherence to important design specifications for the purpose of model accreditation. Accreditation of a trained model requires enumeration of the explicit operational conditions under which the model is certified to meet all necessary specifications. By verifying model adherence to specifications set by developers, we increase the [...]
Carnegie Mellon University
Direct-drive Hands: Making Robot Hands Transparent and Reactive to Contacts
Abstract: Industrial manipulators and end-effectors are a vital driver of the automation revolution. These robot hands, designed to reject disturbances with stiffness and strength, are inferior to their human counterparts. Human hands are dexterous and nimble effectors capable of a variety of interactions with the environment. Through this thesis we wish to answer a question: [...]
Carnegie Mellon University
Teleoperation via Intuition: Safe and Intent Oriented Navigation
Abstract: This thesis aims to enable seamless teleoperation of a mobile robot by a human operator, such that the robot navigates in unstructured environments following the operator’s intent intuitively, safely, and efficiently. The roles of the human and robot are disproportionate in traditional teleoperation: The human is responsible for most of the autonomy of the [...]
Carnegie Mellon University
Towards the Automated Design of Neural Networks
Abstract: Neural architecture search (NAS) is recently proposed to automate the process of designing network architectures. Instead of manually designing network architectures, NAS automatically finds the optimal architecture in a data-driven way. Despite its impressive progress, NAS is still far from being widely adopted as a common paradigm for architecture design in practice. This thesis [...]
Carnegie Mellon University
Dynamical Model Learning and Inversion for Aggressive Quadrotor Flight
Quadrotor applications have seen a surge recently and many tasks require precise and accurate controls. Flying fast is critical in many applications and the limited onboard power source makes completing tasks quickly even more important. Staying on a desired course while traveling at high speeds and high accelerations is difficult due to complex and stochastic [...]
Carnegie Mellon University
Development of an Agile and Dexterous Balancing Mobile Manipulator Robot
Abstract: The proposed thesis work focuses on the design and control of a new unique agile and dexterous mobile manipulator, the Carnegie Mellon University (CMU) ballbot. The CMU ballbot is a human-sized dynamically stable mobile robot that balances on a single ball. We present the development and integration of a new pair of seven-degree-of-freedom (7-DOF) [...]
Carnegie Mellon University
Accelerating Numerical Methods for Optimal Control
Abstract: Many modern control methods, such as model-predictive control, rely heavily on solving optimization problems in real time. In particular, the ability to efficiently solve optimal control problems has enabled many of the recent breakthroughs in achieving highly dynamic behaviors for complex robotic systems. The high computational requirements of these algorithms demand novel algorithms tailor-suited [...]
Carnegie Mellon University
Robust Object Representations for Robot Manipulation
Abstract: As robots become more common in our daily lives, they will need to interact with many different environments and countless types of objects. While we, as humans, can easily understand an object after seeing it only once, this task is not trivial for robots. Researchers have, for the most part, been left with two [...]
Carnegie Mellon University
Visual Representation and Recognition without Human Supervision
Abstract: Visual recognition models have seen great advancements by relying on large-scale, carefully curated datasets with human annotations. Most computer vision models leverage human supervision to either construct strong initial representations (e.g. using the ImageNet dataset) or for modeling the visual concepts relevant for downstream tasks (e.g. MS-COCO for object detection). In this thesis, we [...]
Carnegie Mellon University
Heuristic Search Based Planning by Minimizing Anticipated Search Efforts
Abstract: Robot planning problems in dynamic environments, such as navigation among pedestrians, driving at high-speed on densely populated roads, and manipulation for collaborative tasks alongside humans, necessitate efficient planning. Bounded-suboptimal heuristic search algorithms are a popular alternative to optimal heuristic search algorithms that compromise solution quality for computation speed. Specifically, these searches aim to find [...]
Carnegie Mellon University
Liquid Metal Actuators
Abstract: Bioinspired robotic actuators arise from the advances in soft materials and activation methods to achieve desired performance. Because of their intrinsic compliance, actuators built from soft materials and liquids can achieve elastic resilience and adaptability similar to their biological counterparts. Liquid metals provide great opportunities for creating an artificial muscle that generates forces at [...]
Carnegie Mellon University
Self-Learning of Structured Visual Representations
Abstract: Most computer vision models in deployment today are not learning. Instead, they are in a "test" mode, where they will behave the same way perpetually, until they are replaced by newer models. This is a problem, because it means the models may perform poorly as soon as their "test" environment becomes different from their [...]
Carnegie Mellon University
Resource-Constrained Learning and Inference for Visual Perception
Abstract: Real-world applications usually require computer vision algorithms to meet certain resource constraints. In this talk, I will present evaluation methods and principled solutions for both training and testing. First, I will talk about a formal setting for studying training under the non-asymptotic, resource-constrained regime, i.e., budgeted training. We analyze the following problem: "given a [...]
Social Navigation with Pedestrian Groups
Abstract: Autonomous navigation in human crowds (i.e., social navigation) presents several challenges: The robot often needs to rely on its noisy sensors to identify and localize the pedestrians in human crowds; The robot needs plan efficient paths to reach its goals; The robot needs to do so in a safe and socially appropriate manner. In [...]
Control of Robots with Nonstationary Dynamics
Abstract: Robots can be constructed with fewer resources and less strict constraints than is generally believed. Soft robots can be constructed with very few parts and from a wide variety of materials. This makes them a potentially appealing choice for applications where there are resource constraints on system fabrication. However, soft robot dynamics are difficult [...]
Meshlet Primitives for Dense RGB-D SLAM in Dynamic Environments
Abstract: Dense RGB-D SLAM has been well established as a method for achieving robust localization while providing high quality dense surface reconstruction. However, despite significant progress, dense RGB-D SLAM has remained difficult to achieve on computationally constrained platforms, such as those used on autonomous aerial vehicles. A significant limiting factor in the current state of [...]
Computational Light Transport with Interferometry
Abstract: Optical interferometry is the measurement of small, sub-wavelength distances by exploiting the wave nature of light. Due to its capability to resolve micron-scale displacements, it has found widespread applications in biomedical imaging, industrial fabrication, physics, and astrophysics. In this thesis, we introduce a set of techniques we call computational interferometry, that bring the benefits [...]
Carnegie Mellon University
3D Reconstruction using Differential Imaging
Abstract: 3D reconstruction has been at the core of many computer vision applications, including autonomous driving, visual inspection in manufacturing, and augmented and virtual reality (AR/VR). Despite the tremendous progress made over the years, there remain challenging open-research problems. This thesis addresses three such problems in 3D reconstruction. First, we address the problem of defocus [...]
Carnegie Mellon University
Beyond rigid objects: Data-driven Methods for Manipulation of Deformable Objects
Abstract: Manipulation of deformable objects challenges common assumptions made for rigid objects. Deformable objects have high intrinsic state representation and complex dynamics with high degrees of freedom, making it difficult for state estimation and planning. The completed work can be divided into two parts. In the first part, we explore reinforcement learning (RL) as a [...]
Carnegie Mellon University
Simulation, Perception, and Generation of Human Behavior
Abstract: Understanding and modeling human behavior is fundamental to almost any computer vision and robotics applications that involve humans. In this thesis, we take a holistic approach to human behavior modeling and tackle its three essential aspects --- simulation, perception, and generation. Throughout this thesis, we show how the three aspects are deeply connected and [...]
Carnegie Mellon University
Structured Learning for Robust Robot Manipulation
Abstract: Robust and generalizable robots that can autonomously manipulate objects in semi-structured environments can bring material benefits to society. Data-driven learning approaches are crucial for enabling such systems by identifying and exploiting patterns in semi-structured environments, allowing robots to adapt to novel scenarios with minimal human supervision. However, despite significant prior work in learning for [...]
Simulation-based Planning for Pick-and-Place in Heavy Clutter using Non-prehensile Manipulation
Abstract: Robot manipulation in domestic households, industrial manufacturing and warehouses might require contact-rich interactions with objects in the environment. For pick-and-place style grasping tasks in cluttered scenes, it can be more economical for the robot to rely on non-prehensile actions vis-à-vis deliberate prehensile rearrangement. Non-prehensile actions also let the robot manipulate large and bulky objects [...]
Carnegie Mellon University
Learning with Diverse Forms of Imperfect and Indirect Supervision
Abstract: High capacity Machine Learning (ML) models trained on large, annotated datasets have driven impressive advances in several fields including natural language processing and computer vision, in turn leading to impactful applications of ML in areas such as healthcare, e-commerce, and predictive maintenance. However, obtaining annotated datasets at the scale required for training such models [...]
Carnegie Mellon University
3D Representation Learning for Perception and Prediction: A Modular Yet Highly Integrated Approach
Abstract: Modularized and cascaded autonomy stacks (object detection, then tracking and then trajectory prediction) have been widely adopted in many autonomous systems such as self-driving cars due to its interpretability. In this talk, I advocate the use of such a modular approach but improve its accuracy and robustness by developing different 3D representations for each [...]
Carnegie Mellon University
Understanding Unbalanced Datasets Through Simple Models and Dataset Exploration
Abstract: Computer vision models have proven to be tremendously capable of recognizing and detecting several classes and objects. They succeed in classes widely ranging in type and scale from humans to cans to pens. However, the best performing classes have abundant examples in large-scale datasets today. In unbalanced datasets, where some categories are seen in [...]
Carnegie Mellon University
Self-Supervising Occlusions for Vision
Abstract: Virtually every scene has occlusions. Even a scene with a single object exhibits self-occlusions - a camera can only view one side of an object (left or right, front or back), or part of the object is outside the field of view. More complex occlusions occur when one or more objects block part(s) of [...]
Carnegie Mellon University
Massively Parallelized Lazy Planning Algorithms
Abstract: Search-based planning algorithms enable autonomous agents like robots to come up with well-reasoned long horizon plans to achieve a given task objective. They do so by optimizing a task-specific cost function while respecting the constraints on either the agent (e.g. motion constraints) or the environment (e.g. obstacles). In robotics, such as in motion planning [...]
Run-Time Optimization in the Deep Learning Age
Abstract: In a recovery task one seeks to obtain an estimate of an unknown signal from a set of incomplete measurements. These problems arise in a number of computer vision applications, from image based tasks such as super-resolution and in-painting to 3D reconstruction tasks such as Non-Rigid Structure from Motion and scene flow estimation. Early [...]
Carnegie Mellon University
System Identification and Control of Multiagent Systems Through Interactions
Abstract: This thesis investigates the problem of identifying dynamics models of individual agents of a multiagent system (MAS) and exploiting these models to shape their behavior using robots extrinsic to the MAS. While task-based control of a MAS using onboard controllers of its agents is well studied, we investigate (a) how easy it is for [...]
Carnegie Mellon University
Driving Reconfigurable Unmanned Vehicle Design for Mobility Performance
Abstract: Unmanned ground vehicles are being deployed in increasingly diverse and complex environments. Advances in the field of robotics, including perception technology, computing power, and machine learning, have brought robots from the lab to the real world. Remote and autonomous vehicles are now used to explore volcanoes, caves, pipes, war zones, disaster sites, and even [...]
Towards Large-scale and Long-term Neural Map Representations
Abstract: We address the problem of large-scale and long-term neural map representations. Maps, as our prior understanding toward the environment, provide valuable information for modern robotics applications such as autonomous driving and AR/VR. The size of maps largely affects the end task performance: usually a more detailed map can support better performance, but would cost [...]
Manipulating Objects with Challenging Visual and Geometric Properties
Abstract: Object manipulation is a well-studied domain in robotics, yet manipulation remains difficult for objects with visually and geometrically challenging properties. Visually challenging properties, such as transparency and specularity, break assumptions of Lambertian reflectance that existing methods rely on for grasp estimation. On the other hand, deformable objects such as cloth pose both visual and [...]
Large Scale Dense 3D Reconstruction via Sparse Representations
Abstract: Scene reconstruction systems take in (3D) videos as input, and output 3D models with associated poses for inputs. With the demand of 3D content generation, the technique has been drastically evolving in recent years. For professionals equipped with depth sensors, efficient dense reconstruction systems have become available to efficiently recover scene geometry. For ordinary [...]
Teaching Agent Reward Functions via Demonstrations for Human Inverse Reinforcement Learning
Abstract: For intelligent agents (e.g. robots) to be seamlessly integrated into human society, humans must be able to understand their decision making. For example, the decision making of autonomous cars must be clear to the engineers certifying their safety, passengers riding them, and nearby drivers negotiating the road simultaneously. As an agent's decision making can [...]
Policy Decomposition: Approximate Optimal Control with Suboptimality Estimates
Abstract: Optimal Control is a formulation for designing controllers for dynamical systems by posing it as an optimization problem, whereby the desired long-term behavior of the system is expressed using a cost function. The objective is to compute a policy, i.e. a mapping from the state of the system to its control inputs, that minimizes [...]
Audience-Aware Legibility for Social Navigation
Abstract: Robots often need to communicate their goals to humans when navigating in a shared space to assist observers in anticipating the robot’s future actions. These human observers are often scattered throughout the environment, and each observer only has a partial view of the robot and its movements. A path that non-verbally communicates with multiple [...]
On Sample-Efficient Reinforcement Learning for Nuclear Fusion
Abstract: In many practical applications of reinforcement learning (RL), it is expensive to observe state transitions from the environment. For example, in the problem of plasma control for nuclear fusion, determining the next state for a given state-action pair requires querying an expensive transition function which can lead to many hours of computer simulation or [...]
Carnegie Mellon University
Towards reconstructing non-rigidity from single camera
Abstract: In this proposal, we study how to infer 3D from images captured by a single camera, without assuming the target scenes / objects being static. The non-static setting makes our problem ill-posed and challenging to solve, but is vital in practical applications where target-of-interest is non-static. To solve ill-posed problems, the current trend in [...]
Efficient 3D Representations: Algebraic Surfaces for Differentiable Rendering
Abstract: In this proposal, we show how some classic computer vision tasks can robustly be solved via optimization techniques by using an object representation that is compact and interpretable. Specifically, we explore the applications and benefits of representing 3D objects with an analytical, algebraic function by building an approximate, ray-based differentiable renderer. Our approximate formulation [...]
Continual Robot Learning: Benchmarks and Modular Methods
Zoom Meeting Passcode: 841755 Abstract: The earliest reinforcement learning models were designed to learn one task, specified up-front. However, an agent operating freely in the real world will not in general be granted this luxury, as the demands placed on the agent may change as environments or goals change. We refer to this ever-shifting scenario [...]
Improving Robotic Exploration with Self-Supervision and Diverse Data
Abstract: Reinforcement learning (RL) holds great promise for improving robotics, as it allows systems to move beyond passive learning and interact with the world while learning from these interactions. A key aspect of this interaction is exploration: which actions should an RL agent take to best learn about the world? Prior work on exploration is typically [...]