Analysis by Synthesis for Modern Computer Vision
Abstract: Image denoising, depth completion, scene flow, and dynamic 3D reconstruction are all examples of recovery problems: the estimation of multidimensional signals from corrupted or partial measurements. This thesis examines these problems from the classic analysis-by-synthesis perspective, where a signal model is used to propose hypotheses, which are then compared to observations. This paradigm has [...]
Reinforcement Learning with Spatial Reasoning for Dexterous Robotic Manipulation
Abstract: Robotic manipulation in unstructured environments requires adaptability and the ability to handle a wide variety of objects and tasks. This thesis presents novel approaches for learning robotic manipulation skills using reinforcement learning (RL) with spatially-grounded action spaces, addressing the challenges of high-dimensional, continuous action spaces and alleviating the need for extensive training data. Our [...]
Leveraging Vision, Force Sensing, and Language Feedback for Deformable Object Manipulation
Deformable object manipulation represents a significant challenge in robotics due to its complex dynamics, lack of low-dimensional state representations, and severe self-occlusions. This challenge is particularly critical in assistive tasks, where safe and effective manipulation of various deformable materials can significantly improve the quality of life for individuals with disabilities and address the growing needs [...]
CBGT-Net: A Neuromimetic Architecture for Robust Classification of Streaming Data
Abstract: This research introduces CBGT-Net, a neural network model inspired by the cortico-basal ganglia-thalamic (CBGT) circuits in mammalian brains, which are crucial for critical thinking and decision-making. Unlike traditional neural network models that generate an output for each input or after a fixed sequence of inputs, CBGT-Net learns to produce an output once sufficient evidence [...]
Information-Based Adaptive Allocation of Heterogeneous Multi-Agent Teams for Search and Coverage
Abstract: Information-based search and coverage are important in planetary exploration and disaster response applications. Efficient information acquisition can help with increasing geological understanding or situational awareness. Heterogeneous robots, each with different sensing and motion modalities, can be coordinated to optimize search and coverage in a target region. Information maps, which estimate the importance of visiting [...]
Enhancing Robot Perception and Interaction Through Structured Domain Knowledge
Abstract: Despite the advancements in deep learning driven by increased computational power and large datasets, significant challenges remain. These include difficulty in handling novel entities, limited mechanisms for human experts to update knowledge, and lack of interpretability, all of which are crucial for human-centric applications like assistive robotics. To address these issues, we propose leveraging [...]
Dynamic Multi-Objective Trajectory Planning for Mobile Robots
Abstract: Robotic explorers play a crucial role in acquiring data from areas that are difficult or impossible for humans to reach. Whether for planetary exploration, search and rescue missions, agriculture, or other scientific exploration tasks, these robots can utilize pre-existing knowledge of the terrain to navigate effectively. In search- and coverage-oriented scenarios, robots must consider [...]
From Understanding to Interacting with the 3D World
Abstract: Understanding the 3D structure of real-world environments is a fundamental challenge in machine perception, critical for applications spanning robotic navigation, content creation, and mixed reality scenarios. In recent years, machine learning has undergone rapid advancements; however, in the 3D domain, such data-driven learning is often very challenging under limited 3D/4D data availability. In this talk, [...]
Motion planning for manipulation under pose uncertainty using contacts
Abstract: Numerous manipulation tasks, such as plug insertion and pipe assembly, demand an extremely high level of precision in pose estimation. Even minor errors, on the order of 2mm, can lead to task failure. While robots often rely on vision for object detection and localization, achieving consistent, high-precision localization using visual methods is not always [...]
Robust Off-road Wheel Odometry with Slip Estimation
Abstract: Wheel odometry is not often used in state estimation for off-road vehicles due to frequent wheel slippage, varying wheel radii, and the 3D motion of the vehicle not fitting with the 2D nature of integrated wheel odometry. This paper proposes a novel 3D preintegration of wheel encoder measurements on manifold. Our method additionally estimates [...]