PhD Thesis Proposal
Trustworthy Learning using Uncertain Interpretation of Data
Abstract: Non-parametric models are popular in real-world applications of machine learning. However, many modern ML methods that ensure that models are pragmatic, safe, robust, fair, and otherwise trustworthy in increasingly critical applications, assume parametric, differentiable models. We show that, by interpreting data as locally uncertain, we can achieve many of these without being limited to [...]
Allocation, Planning, and Control in Off-road Automated Convoy Operations
Abstract: The lack of structure in off-road terrains makes off-road operations of automated platforms difficult. The difficulty arises from uncertainty in the optimality and safety of the actions (e.g., planning and control) taken by the automated platform. When multiple automated platforms are required to act in a coordinated manner (e.g., a convoy) in complex cluttered [...]
Learning Generalizable Robot Skills for Dynamic and Interactive Tasks
Abstract: Enabling robots to perform complex dynamic tasks such as picking up an object in one sweeping motion or pushing off a wall to quickly turn a corner is a challenging problem. The dynamic interactions implicit in these tasks are critical for successful task execution. Furthermore, given the interactive nature of such tasks, safety, in [...]
Low-Cost Multimodal Sensing and Dexterity for Deformable Object Manipulation
Abstract: To integrate robots seamlessly into daily life, they must be able to handle a variety of tasks in diverse environments, like assisting in hospitals or cooking in kitchens. Many of the items in these environments are deformable such as bedding in hospitals or vegetables in kitchens, and a certain level of dexterity is necessary [...]
Learning Safe Human-Robot Interactions for a Seamlessly Shared Airspace
Abstract: The growing need for fully autonomous aerial operations in shared spaces, necessitates the development of reliable agents capable of navigating safely and seamlessly alongside uncertain human agents. In response, we advocate endowing autonomous agents with the ability to predict human actions, comprehend and ground abstract rules in the action space, and embrace the uncertainty [...]
Informative Path Planning Toward Autonomous Real-World Applications
Abstract: Gathering information from the physical world plays a crucial role in many applications—whether it be scientific research, environmental monitoring, search and rescue, defense, or disaster response. The utilization of robots for information gathering allows for the leveraging of intelligent algorithms to efficiently collect data, providing critical insights and facilitating informed decision-making. These autonomous robots [...]
Efficient Sensor Coverage in Complex Environments
Abstract: This thesis develops sensor coverage algorithms for mobile robots that are scalable to large and complex environments. The core challenge is computing the shortest paths that can direct one or more robots to sweep onboard sensors over all accessible surfaces within an environment. This problem resembles the watchman route problem that is known to [...]
Combining Physics-Based Light Transport and Neural Fields for Robust Inverse Rendering
Abstract: Inverse rendering — the process of recovering shape, material, and/or lighting of an object or environment from a set of images — is essential for applications in robotics and elsewhere, from AR/VR to perception on self-driving vehicles. While it is possible to perform inverse rendering from color images alone, it is often far easier [...]
Robotic Climbing for Extreme Terrain Exploration
Abstract: Climbing robots can operate in steep and unstructured environments that are inaccessible to other ground robots, with applications ranging from the inspection of artificial structures on Earth to the exploration of natural terrain features throughout the solar system. Climbing robots for planetary exploration face many challenges to deployment, including mass restrictions, irregular surface features, [...]
Exploration for Continually Improving Robots
Abstract: General purpose robots should be able to perform arbitrary manipulation tasks, and get better at performing new ones as they obtain more experience. The current paradigm in robot learning involves imitation or simulation. Scaling these approaches to learn from more data for various tasks is bottle-necked by human labor required either in collecting demonstrations [...]
Deep 3D Geometric Reasoning for Robot Manipulation
Abstract: To solve general manipulation tasks in real-world environments, robots must be able to perceive and condition their manipulation policies on the 3D world. These agents will need to understand various common-sense spatial/geometric concepts about manipulation tasks: that local geometry can suggest potential manipulation strategies, that policies should be invariant across choice of reference frame, [...]
Towards diverse zero-shot manipulation via actualizing visual plans
Abstract: In this thesis, we seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation — interacting with unseen objects in novel scenes without test-time adaptation. Robots that can be reliably deployed out-of-the-box in new scenarios have the potential for helping humans in everyday tasks. Not requiring any test-time training through demonstrations or [...]
Deep Learning for Sensors: Development to Deployment
Abstract: Robots rely heavily on sensing to reason about physical interactions, and recent advancements in rapid prototyping, MEMS sensing, and machine learning have led to a plethora of sensing alternatives. However, few of these sensors have gained widespread use among roboticists. This thesis proposes a framework for incorporating sensors into a robot learning paradigm, from [...]
Towards Influence-Aware Safe Human-Robot Interaction
Abstract: In recent years, we have seen through recommender systems on social media how influential (and potentially harmful) algorithms can be in our lives, sometimes creating polarization and conspiracies that lead to unsafe behavior. Now that robots are also growing more common in the real world, we must be very careful to ensure that they [...]
Design Principles for Robotics Systems that Support Human-Human Collaborative Learning
Abstract: Robots possess unique affordances granted by combining software and hardware. Most existing research focuses on the impact of these affordances on human-robot collaboration, but the theory of how robots can facilitate human-human collaboration is underdeveloped. Such theory would be beneficial in education. An educational device can afford collaboration in both assembly and use. This [...]
Robust Incremental Distributed Collaborative Simultaneous Localization and Mapping
Abstract: Multi-robot teams show exceptional promise across applications like Search-and-Rescue, disaster-response, agriculture, forestry, and scientific exploration due to their ability to go where humans cannot, parallelize activity, operate robustly to failures, and expand capabilities beyond that of an individual robot. Collaborative Simultaneous Localization and Mapping (C-SLAM) is a fundamental capability for these multi-robot teams as [...]
A Unified Control Framework for Robust Aerial Manipulation
Abstract: Aerial robots are now widely employed in diverse applications, such as delivery, environmental monitoring, and especially aerial manipulation—the focus of this thesis. Aerial manipulation involves integrating robotic arms with drones to perform physical tasks remotely. This capability is particularly crucial for operations that are either too dangerous or inaccessible for humans, such as high-altitude [...]
Beyond Robot Safety: Adaptability and Interactivity
Abstract: The deployment of autonomous robots in various areas, including transportation and human-robot collaboration, requires strong safety measures for effective interaction with the physical world. Traditional safe control algorithms work well in controlled settings but struggle to adapt to more interactive and unpredictable real-world scenarios. This thesis emphasizes the need to explore beyond traditional robot [...]
Towards Pragmatic Time Series Intelligence
Abstract: The widespread adoption of time series machine learning (ML) models faces multiple challenges involving data, modeling and evaluation. Data. Modern ML models depend on copious amounts of cohesive and reliably annotated data for training and evaluation. However, labeled data is not always available and reliable, and can also be dispersed across different locations. We [...]
Improved Surface Estimation for use in Virtual Fixtures during Retinal Surgery
Abstract: Retinal surgery procedures require surgeons to manipulate very delicate tissues with little room for error. During epiretinal membrane surgery, to reduce chances of recurrence, surgeons may have to remove the 10 µm thick internal limiting membrane from the retinal surface. An experimental procedure to treat retinal vein occlusion is retinal vein cannulation. During this [...]
3D Perception In-The-Wild
Abstract: State estimation is a fundamental component of embodied perception. Contemporary autonomous vehicle (AV) benchmarks have advanced techniques for training 3D detectors, particularly on large-scale data. Notably, although prior work has nearly solved 3D object detection for a few common classes (e.g., pedestrian and car), detecting many rare classes in-the-tail (e.g., debris and stroller) remains [...]
Differentiable Convex Modeling for Robotic Planning and Control
Abstract: Robotic simulation, planning, estimation, and control, have all been built on top of numerical optimization. In this same time, modern convex optimization has matured into a robust technology delivering globally optimal solutions in polynomial time. With advances in differentiable optimization and custom solvers capable of producing smooth derivatives, convex modeling has become fast, reliable, [...]
Information-Based Adaptive Allocation of Heterogeneous Multi-Agent Teams for Search and Coverage
Abstract: Information-based search and coverage are important in planetary exploration and disaster response applications. Efficient information acquisition can help with increasing geological understanding or situational awareness. Heterogeneous robots, each with different sensing and motion modalities, can be coordinated to optimize search and coverage in a target region. Information maps, which estimate the importance of visiting [...]
Dynamic Multi-Objective Trajectory Planning for Mobile Robots
Abstract: Robotic explorers play a crucial role in acquiring data from areas that are difficult or impossible for humans to reach. Whether for planetary exploration, search and rescue missions, agriculture, or other scientific exploration tasks, these robots can utilize pre-existing knowledge of the terrain to navigate effectively. In search- and coverage-oriented scenarios, robots must consider [...]
Motion planning for manipulation under pose uncertainty using contacts
Abstract: Numerous manipulation tasks, such as plug insertion and pipe assembly, demand an extremely high level of precision in pose estimation. Even minor errors, on the order of 2mm, can lead to task failure. While robots often rely on vision for object detection and localization, achieving consistent, high-precision localization using visual methods is not always [...]
Optimal Modular Robot Design for Mobile Manipulation in Agriculture
Abstract: Although agriculture is a highly mechanized industry, numerous sectors like horticulture and floriculture heavily depend on manual labor because they require safe handling of plants and produce that can only be left to humans. However, many research and commercial robots have succeeded in several challenging dexterous manipulation tasks like harvesting, pruning, and plant health [...]
Aligning Robot Task and Interaction Policies to Human Values
Abstract: The value alignment problem considers how robots can learn to behave in accordance with human values. Today, robot learning paradigms enable humans to provide data (e.g., preference labels or demonstrations), which the robot uses to update its behavior (e.g., reward model or policy) to be closer to the human’s values. However, the current paradigm [...]
Accelerating Robot Task Learning with Large Pretrained Models and Internet Data
Abstract: Large pre-trained models and internet data sources are key to general and efficient robot task learning. However, learning contact-rich behaviors, semantic task constraints, and robust task planning from internet data sources remains an open challenge. This proposal seeks to make progress towards a general robot task learning system leveraging pre-trained models and internet data. [...]
Unlocking Generalization for Robotics via Modularity and Scale
Abstract: How can we build generalist robot systems? Looking at fields such as vision and language, the common theme has been large scale end-to-end learning with massive, curated datasets. In robotics, on the other hand, scale alone may not be enough due to the significant multimodality of robotics tasks, lack of easily accessible data and [...]