Student Talks
Incorporating Robustness into Learning-Based Aircraft Detection and Tracking Systems
Abstract: In the field of aviation, the Detect and Avoid (DAA) problem deals with incorporating collision avoidance capabilities into current autopilot navigation systems. In order to standardize DAA capabilities, ASTM has published performance requirements to define safe DAA operations of unmanned aircraft systems (UAS). However, the performance of DAA models are entirely dependent on the [...]
Carnegie Mellon University
MSR Thesis Talk: Siddarth Venkatraman
Title: Latent Skill Models for Offline Reinforcement Learning Abstract: Offline reinforcement learning (RL) holds promise as a means to learn high-value policies from a static dataset, without the need for further environment interactions. However, a key challenge in offline RL lies in effectively stitching portions of suboptimal trajectories from the static dataset while avoiding extrapolation [...]
Long-Tailed 3D Detection via Multi-Modal Fusion
Abstract: Contemporary autonomous vehicle (AV) benchmarks have advanced techniques for training 3D detectors, particularly on large-scale LiDAR data. Surprisingly, although semantic class labels naturally follow a long-tailed distribution, these benchmarks focus on only a few common classes (e.g., pedestrian and car) and neglect many rare classes in-the-tail (e.g., debris and stroller). However, in the real [...]
MSR Thesis Talk: Eric Schneider
Title: Phenotyping and Skeletonization for Agricultural Robotics Abstract: Scientific phenotyping of plants is a crucial aspect of experimental plant breeding. By accurately measuring plant characteristics, phenotyping plays a vital role in the development of new plant varieties that are better adapted to specific environments and have improved yield, quality, and resistance to stress and disease. In [...]
MSR Thesis Talk: Shivesh Khaitan
Zoom Link: https://cmu.zoom.us/j/95273358670?pwd=Z09Jc3g1aDV1dTdTMEVUWUwxcUZPQT09 Meeting ID: 952 7335 8670 Passcode: 050721 Title: Exploring Reinforcement Learning approaches for Safety Critical EnvironmentsAbstract: Reinforcement Learning (RL) has emerged as a powerful paradigm for addressing challenging decision-making and robotic control tasks. By leveraging the principles of trial-and-error learning, RL algorithms enable agents to learn optimal strategies through interactions with an environment. However, [...]
MSR Thesis Talk: Ravi Tej Akella
Title: Distributional Distance Classifiers for Goal-Conditioned Reinforcement Learning Abstract: Autonomous systems are increasingly being deployed in stochastic real-world environments. Often, these agents are trying to find the shortest path to a commanded goal. But what does it mean to find the shortest path in stochastic environments, where every strategy has a non-zero probability of failing? At [...]
MSR Thesis Talk: Seth Karten
Title: Emergent Communication and Decision-Making in Multi-Agent Teams Abstract: Explicit communication among humans is key to coordinating and learning. In multi-agent reinforcement learning for partially-observable environments, agents may convey information to others via learned communication, allowing the team to complete its task. However, agents need to be able to communicate more than simply referential messages [...]
MSR Thesis Talk: Sashank Tirumala
Title: Tactile Sensing applied to deformable object manipulation Abstract: The application of robotic manipulation of deformable materials, such as cloth, spans various sectors including fabric manufacturing and domestic laundry management. Historically, most methodologies have employed vision-based sensors as the proprioceptive input to robot policies. However, this study aims to explore an alternate route by leveraging [...]
MSR Thesis Talk: Zhizhu Zhao
Title: Distilling View-conditioned Diffusion for 3D Reconstruction Abstract: We propose a 3D neural mode-seeking formulation that combines probabilistic generation of unseen regions with faithful reprojection of seen regions in a consistent 3D representation. Feature reprojection methods (NerFormer, PixelNeRF) are 3D consistent, but fail to hallucinate unseen regions. Image generation methods (ViewFormer) generate plausible hallucinations, but generated [...]
Differentiable Fluid-Structure Interaction for Robotics
Abstract: We present Aquarium, a differentiable fluid-structure interaction solver for robotics that offers stable simulation, accurately coupled fluid-robot physics in two dimensions, and full differentiability with respect to fluid and robot states and parameters. Aquarium achieves stable simulation with accurate flow physics by directly integrating over the incompressible Navier-Stokes equations using a fully implicit Crank-Nicolson [...]
MSR Thesis Talk: Khiem Vuong
Title: Scaling up Camera Calibration and Amodal 3D Object Reconstruction for Smart Cities Abstract: Smart cities integrate thousands of outdoor cameras to enhance urban infrastructure, but their automated analysis potential remains untapped due to various challenges. Firstly, the lack of accurate camera calibration information, such as its intrinsics parameters and external orientation, restricts the measurement [...]
From Reinforcement Learning to Robot Learning: Leveraging Prior Data and Shared Evaluation
Abstract: Unlike most machine learning applications, robotics involves physical constraints that make off-the-shelf learning challenging. Difficulties in large-scale data collection and training present a major roadblock to applying today’s data-intensive algorithms. Robot learning has an additional roadblock in evaluation: every physical space is different, making results across labs inconsistent. Two common assumptions of the robot [...]
MSR Thesis Talk: Tianyuan Zhang
Title: Surface Ripples: Analyzing Transient Vibrations on Object's Surfaces Abstract: The subtle vibrations on an object's surface contain information about its physical properties and interaction with the environment. Prior works imaged surface vibration to recover the object's material properties via modal analysis, which discards the transient vibrations propagating immediately after the object is disturbed. In this [...]
MSR Thesis Talk: Anurag Ghosh
Title: Learned Two-Plane Perspective Prior based Image Resampling for Efficient Object Detection Abstract: Real-time efficient perception is critical for autonomous navigation and city scale sensing. Orthogonal to architectural improvements, streaming perception approaches have exploited adaptive sampling improving real-time detection performance. In this work, we propose a learnable geometry-guided prior that incorporates rough geometry of the [...]
MSR Thesis Talk: David Russell
Title: Using Drones and Remote Sensing to Understand Forests with Limited Labeled Data Abstract: Drones and remote sensing can provide observations of forests at scale, but this raw data needs to be interpreted to further scientific understanding and inform effective management decisions. This thesis studies two problems under the realistic constraint of limited domain-specific training [...]
MSR Thesis TallK: Aarrushi Shandilya
Title: Lights, Camera, Render: Neural Fields for Structured Lighting Abstract: 3D scene reconstruction from 2D image supervision alone is an under-constrained problem. Recent neural rendering frameworks have made great strides in learning 3D scene representations to enable novel view synthesis, but they struggle to reconstruct geometry of low-texture regions or from sparse views. The prevalence of active [...]
Building 4D Models of Objects and Scenes from Monocular Videos
Abstract: We explore how to infer the time-varying 3D structures of generic, deformable objects, and dynamic scenes from monocular videos. A solution to this problem is essential for virtual reality and robotics applications. However, inferring 4D structures given 2D observations is challenging due to its under-constrained nature. In a casual setup where there is neither [...]
MSR Thesis Talk: Anirudha Ramesh
Title: Learning to See in the Dark and Beyond Abstract: Robotic Perception in diverse domains such as low-light scenarios remains a challenge, even upon the employment of new sensing modalities like thermal imaging and specialized night-vision sensors. This is largely due to the high difficulty in obtaining labeled data for certain tasks. In this work, [...]
MSR Thesis Talk: Mateo Guaman Castro
Title: Self-Supervised Costmap Learning for Off-Road Vehicle Traversability Abstract: Estimating terrain traversability in off-road environments requires reasoning about complex interaction dynamics between the robot and these terrains. However, it is challenging to build an accurate physics model, or create informative labels to learn a model in a supervised manner, for these interactions. We propose a method [...]
Learning to Manipulate Using Diverse Datasets
Abstract: Manipulation is a key challenge in the robotic fields that impedes the deployment of robots in real-world scenarios. While notable advancements have been made in solving high/mid level planning problems, such as decomposing tasks (e.g. "bring me a bottle") into primitives (e.g. "pick up bottle"), the acquisition of fundamental manipulation primitives remains a difficult [...]
MSR Thesis Talk: Gaoyue Zhou
Title: On Generalization and Benchmarking on Physical Robots Abstract: Robotics research has seen significant advancements; however, the field remains predominantly demo-driven, making direct comparisons between methods difficult without replicating them on individual setups. While many simulation benchmarks exist, they usually feature contrived datasets and do not accurately reflect real-world performance. In my thesis, we [...]
An Effective Learning Framework for Active Perception and a Case Study on Liquid Property Estimation
Abstract: Active perception refers to a perception process where robot actions are taken to improve perception. To do this, the robot needs an observation model that knows what it will observe based on the actions it takes. However, existing approaches struggle to learn a good observation model since it needs to account for all possible [...]
MSR Thesis Talk: Heng Yu
Title: Towards Real-time Controllable Neural Face Avatars Abstract: Neural Radiance Fields (NeRF) are compelling techniques for modeling dynamic 3D scenes from 2D image collections. These volumetric representations would be well suited for synthesizing novel facial expressions but for three problems. First, deformable NeRFs are object agnostic and model holistic movement of the scene: they can [...]
MSR Thesis Talk: Winnie Kuang
Title: Design and Integration of Semantic Mapping System for Forest Fire Mitigation Abstract: Remote sensing technologies can provide an automated approach to monitor and analyze conditions in the forest environment over a period of time for forest maintenance and wildfire mitigation efforts. In particular, unmanned aerial vehicles (UAVs) are a promising remote sensing modality since they [...]
MSR Thesis Talk: Jinqi Luo
Title: Vision Model Diagnosis: A Generative Perspective Abstract: In the evolving landscape of computer vision, deep learning has emerged as a transformative force, enhancing a myriad of societal facets. The real-world deployment of such a deep vision model requires a reliable evaluation, particularly when the model can have different sensitivities across various semantic attributes and concepts. [...]
MSR Thesis Talk: Daphne Chen
Title: Learning Task Preferences from Real-World Data Abstract: In order to provide personalized assistance that is capable of adapting to the needs of unique individuals, it is necessary to understand peoples’ preferences for different tasks. Robot assistance often assumes a static model of the individual, while in the real world, people have different capabilities and needs [...]
Unified Control for Over and Fully-Actuated Aerial Vehicles
Abstract: The growing domain of aerial robotics necessitates advancements in the control strategies and robustness of over-actuated and fully-actuated aerial vehicles. This thesis proposal makes contributions to this endeavor by providing in-depth analysis and methodologies concerning these vehicles, control allocation strategies during actuator failures, high-fidelity simulations, and a unified control framework. Our completed work has [...]
MSR Thesis Talk: Prasanna Kettavarapalyam Sriganesh
Title: Fast Staircase Detection and Estimation with Multi-View Merging for Multi-Robot Systems Abstract: When robotic systems are deployed in the real world, they demand advanced mobility capabilities to operate in complex, three-dimensional environments designed for human use, e.g., multi-level buildings. Staircases have been an integral part of facilitating vertical movement in these three-dimensional environments. This work [...]
MSR Thesis Talk: Aarush Gupta
Title: LightSpeed: Light and Fast Neural Light Fields on Mobile Devices Abstract: Real-time novel-view image synthesis on mobile devices is prohibitive due to limited on-device computational power and storage. Using volumetric rendering methods, such as NeRF and its derivatives, on mobile devices is not suitable due to the high computational cost of volumetric rendering. On the [...]
MSR Thesis Talk: Akshaya Kesarimangalam Srinivasan
Title: Multi-agent Multi-objective Ergodic Search Abstract: In order to find points of interest in a given domain, many planners use a priori information to guide the search to expedite the detection of targets. We present an approach to direct multiple agents (MA) to search a given domain subject to multiple objectives (MO), each characterized by its own information [...]
MSR Thesis Talk: Joshua Spisak
Title: Stochastic Optimization for Autonomous Navigation, Leveraging Parallel Computation Abstract: Stochastic Optimal Control (SOC) is a framework that allows disturbances and uncertainty in system models to be accounted for in its optimization framework. Despite accounting for this uncertainty, many first and second order methods for solving SOC problems are subject to local minima and are [...]
MSR Thesis Talk: Xuxin Cheng
Title: Learning Legged Robot Agility: Sim-to-Real and Beyond Abstract: Legged robotics has seen significant advancements in both manipulation and locomotion. However, there remain significant gaps compared to their biological counterparts, particularly in energy efficiency, natural motion, and the capacity for agile skills. This thesis primarily focuses on two aspects: the unified control of legged manipulators [...]
MSR Thesis Talk: Harry Freeman
Title: Computer Vision-Based Phenotyping in Agriculture: Leveraging Semantic Information for Non-Destructive Small Crop Analysis Abstract: Fast and reliable non-destructive phenotyping of plants plays an important role in precision agriculture, as the information enables farmers to make real-time crop management decisions without affecting yield. To non-destructively phenotype crops, computer and stereo-vision based methods are commonly used, [...]
MSR Thesis Talk: Nishant Mohanty
Title: Multi-Robot Control using Control Barrier Functions: Theory and Application Abstract: Control Barrier Functions (CBFs) have emerged as a powerful theoretical tool for designing controllers with provable safety guarantees. This work presents a novel methodology that leverages CBFs to synthesize controllers for multi-robot coordination. Two multi-agent use cases are explored, i.e., a) Non-Cooperative Herding and [...]
MSR Thesis Talk: Yuyao Shi
Title: A Learning Approach to Understand How Spinal Cord Learns Multiple Behaviors Abstract: The spinal cord plays a crucial role in the control of human locomotion, generating motor patterns and coordinating reflex responses to sensory signals. Although this spinal control is traditionally viewed as a simple relay system, more recent neurophysiological evidence points to a [...]
Carnegie Mellon University
MSR Thesis Talk: FNU Abhimanyu
Title: Improving Robotic Ultrasound AI Using Optical Flow Abstract: Ultrasound is an important modality for medical intervention such as vascular access because it is safe, portable, and low-cost. However, ultrasound scanning requires trained sonographers who are scarce, and it can be challenging to perform ultrasound examinations in disaster or battlefield scenarios. This motivates us to automate [...]
Vision-based Proprioceptive and Tactile Sensing for Soft Robots
Abstract: Soft robotic manipulators present many unique advantages in difficult manipulation tasks. The inherent compliance of soft robots' constituent deformable material makes them safe and reliable in delicate tasks such as harvesting fruit and assisting in household work. To address challenges in proprioceptive and tactile sensing for soft robots, we present a family of vision-based [...]
MSR Thesis Talk: Lucas Casanova De Oliveira Nogueira
Title: SuperLoop: a LIDAR-based SLAM Back-end for Underground Exploration Abstract: Robots deployed in underground scenarios require a SLAM system that can handle a variety of challenges, such as the absence of GPS, large scale maps, bad illumination, and geometrically degenerate environments. It is nearly impossible for any SLAM solution to handle all these challenges perfectly, specially [...]
Learning via Visual-Tactile Interaction
Abstract: Humans learn by interacting with their surroundings using all of their senses. The first of these senses to develop is touch, and it is the first way that young humans explore their environment, learn about objects, and tune their cost functions (via pain or treats). Yet, robots are often denied this highly informative and [...]
MSR Thesis Talk: Neil Khera
Title: PyCubed-Mini: A Low-Cost, Open-Source Satellite Research Platform Abstract: Satellite development has become more accessible with decreasing launch costs and shrinking hardware. However, the expenses associated with pre-built satellite kits remain high, making it difficult for student and hobbyist teams to participate. The lack of standardized satellite hardware and software further adds to the challenge, [...]