MSR Thesis Defense
MSR Thesis Talk: Himangi Mittal
Title: Audio-Visual State-Aware Representation Learning from Interaction-Rich Data Abstract In robotics and augmented reality, the input to the agent is a long stream of video from the first-person or egocentric point of view. Recently, there have been significant efforts to capture humans from their first-person/egocentric view interacting with their own environment as they go about [...]
MSR Thesis Talk: Ken Liu
Title: On Privacy and Personalization in Federated Learning: Analyses and Applications Abstract: Recent advances in machine learning often rely on large and centralized datasets. However, curating such data can be challenging when they hold private information, and policies/regulations may mandate that they remain distributed across data silos (e.g. mobile devices or hospitals). Federated learning (FL) [...]
Carnegie Mellon University
MSR Thesis Talk: Haolun Zhang
Title: Seeing in 3D: Towards Generalizable 3D Visual Representations for Robotic Manipulation Abstract: Despite the recent progress in computer vision and deep learning, robot perception remains a tremendous challenge due to the variations of the objects and the scenes in manipulation tasks. Ideally, a robot trying to manipulate a new object should be able to [...]
MSR Thesis Talk: Muyang Li
Title: Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models Abstract: During image editing, existing deep generative models tend to re-synthesize the entire output from scratch, including the unedited regions. This leads to a significant waste of computation, especially for minor editing operations. In this work, we present Spatially Sparse Inference (SSI), a general-purpose technique [...]
MSR Thesis Talk: Rohan Zeng
Title: Spectral Unmixing and Mapping of Coral Reef Benthic Cover Abstract: Coral reefs are important to the global ecosystem and the local communities and wildlife that rely on the habitat they create. However, coral reefs are also in critical and rapid decline: reefs have degraded over recent decades and what remains is at increasing risk [...]
MSR Thesis Talk: Ashwin Misra
Title: Learn2Plan: Learning variable ordering heuristics for scalable task planning Abstract: Traditional approaches to planning attempt to transform a system into a goal state by applying specific actions in a specific order. In these methods, there is an exponential search space due to considering many possible actions at every decision point. Hierarchical Task Networks use incremental [...]
MSR Thesis Talk: Andrew Jong
Title: Robot Information Gathering for Dynamic Systems in Wildfire Scenarios Abstract: The monitoring of complex dynamic systems, such as those encountered in disaster response, search and rescue, wildlife conservation, and environmental monitoring, presents the fundamental challenge of how to track efficiently with limited resources and partial observability. This thesis presents algorithms and techniques for robotic [...]
MSR Thesis Talk: Erin Wong
Title: Edge Detection by Centimeter Scale Low-Cost Mobile Robots Abstract: In Search and Rescue (SaR) efforts after natural disasters like earthquakes, the primary focus is to find and rescue people in building rubble. These rescue efforts could put first responders at risk and are slow due to the unstable nature of the environment. Robotic solutions [...]
MSR Thesis Talk: Sarvesh Patil
Title: Soft Delta Robots for Dexterous Manipulation Abstract: Dexterous manipulation capabilities of end-effectors afford us a wide range of strategies for fine-grained manipulation tasks. Recent utilization of readily available materials like soft filaments and silicone elastomers has enabled the development of low-cost mechanically intelligent robotic manipulators. This is important for democratizing robot manipulation and increasing [...]
MSR Thesis Talk: Fan Yang
Title: Exploring Safe Reinforcement Learning for Sequential Decision Making Abstract: Safe Reinforcement Learning (RL) focuses on the problem of training a policy to maximize the reward while ensuring safety. It is an important step towards applying RL to safety-critical real-world applications. However, safe RL is challenging due to the trade-off between the two objectives [...]
Long-Tailed 3D Detection via Multi-Modal Fusion
Abstract: Contemporary autonomous vehicle (AV) benchmarks have advanced techniques for training 3D detectors, particularly on large-scale LiDAR data. Surprisingly, although semantic class labels naturally follow a long-tailed distribution, these benchmarks focus on only a few common classes (e.g., pedestrian and car) and neglect many rare classes in-the-tail (e.g., debris and stroller). However, in the real [...]
MSR Thesis Talk: Eric Schneider
Title: Phenotyping and Skeletonization for Agricultural Robotics Abstract: Scientific phenotyping of plants is a crucial aspect of experimental plant breeding. By accurately measuring plant characteristics, phenotyping plays a vital role in the development of new plant varieties that are better adapted to specific environments and have improved yield, quality, and resistance to stress and disease. In [...]
MSR Thesis Talk: Shivesh Khaitan
Zoom Link: https://cmu.zoom.us/j/95273358670?pwd=Z09Jc3g1aDV1dTdTMEVUWUwxcUZPQT09 Meeting ID: 952 7335 8670 Passcode: 050721 Title: Exploring Reinforcement Learning approaches for Safety Critical EnvironmentsAbstract: Reinforcement Learning (RL) has emerged as a powerful paradigm for addressing challenging decision-making and robotic control tasks. By leveraging the principles of trial-and-error learning, RL algorithms enable agents to learn optimal strategies through interactions with an environment. However, [...]
MSR Thesis Talk: Ravi Tej Akella
Title: Distributional Distance Classifiers for Goal-Conditioned Reinforcement Learning Abstract: Autonomous systems are increasingly being deployed in stochastic real-world environments. Often, these agents are trying to find the shortest path to a commanded goal. But what does it mean to find the shortest path in stochastic environments, where every strategy has a non-zero probability of failing? At [...]
MSR Thesis Talk: Seth Karten
Title: Emergent Communication and Decision-Making in Multi-Agent Teams Abstract: Explicit communication among humans is key to coordinating and learning. In multi-agent reinforcement learning for partially-observable environments, agents may convey information to others via learned communication, allowing the team to complete its task. However, agents need to be able to communicate more than simply referential messages [...]
MSR Thesis Talk: Sashank Tirumala
Title: Tactile Sensing applied to deformable object manipulation Abstract: The application of robotic manipulation of deformable materials, such as cloth, spans various sectors including fabric manufacturing and domestic laundry management. Historically, most methodologies have employed vision-based sensors as the proprioceptive input to robot policies. However, this study aims to explore an alternate route by leveraging [...]
MSR Thesis Talk: Zhizhu Zhao
Title: Distilling View-conditioned Diffusion for 3D Reconstruction Abstract: We propose a 3D neural mode-seeking formulation that combines probabilistic generation of unseen regions with faithful reprojection of seen regions in a consistent 3D representation. Feature reprojection methods (NerFormer, PixelNeRF) are 3D consistent, but fail to hallucinate unseen regions. Image generation methods (ViewFormer) generate plausible hallucinations, but generated [...]
MSR Thesis Talk: Khiem Vuong
Title: Scaling up Camera Calibration and Amodal 3D Object Reconstruction for Smart Cities Abstract: Smart cities integrate thousands of outdoor cameras to enhance urban infrastructure, but their automated analysis potential remains untapped due to various challenges. Firstly, the lack of accurate camera calibration information, such as its intrinsics parameters and external orientation, restricts the measurement [...]
MSR Thesis Talk: Tianyuan Zhang
Title: Surface Ripples: Analyzing Transient Vibrations on Object's Surfaces Abstract: The subtle vibrations on an object's surface contain information about its physical properties and interaction with the environment. Prior works imaged surface vibration to recover the object's material properties via modal analysis, which discards the transient vibrations propagating immediately after the object is disturbed. In this [...]
MSR Thesis Talk: Anurag Ghosh
Title: Learned Two-Plane Perspective Prior based Image Resampling for Efficient Object Detection Abstract: Real-time efficient perception is critical for autonomous navigation and city scale sensing. Orthogonal to architectural improvements, streaming perception approaches have exploited adaptive sampling improving real-time detection performance. In this work, we propose a learnable geometry-guided prior that incorporates rough geometry of the [...]
MSR Thesis Talk: David Russell
Title: Using Drones and Remote Sensing to Understand Forests with Limited Labeled Data Abstract: Drones and remote sensing can provide observations of forests at scale, but this raw data needs to be interpreted to further scientific understanding and inform effective management decisions. This thesis studies two problems under the realistic constraint of limited domain-specific training [...]
MSR Thesis TallK: Aarrushi Shandilya
Title: Lights, Camera, Render: Neural Fields for Structured Lighting Abstract: 3D scene reconstruction from 2D image supervision alone is an under-constrained problem. Recent neural rendering frameworks have made great strides in learning 3D scene representations to enable novel view synthesis, but they struggle to reconstruct geometry of low-texture regions or from sparse views. The prevalence of active [...]
MSR Thesis Talk: Anirudha Ramesh
Title: Learning to See in the Dark and Beyond Abstract: Robotic Perception in diverse domains such as low-light scenarios remains a challenge, even upon the employment of new sensing modalities like thermal imaging and specialized night-vision sensors. This is largely due to the high difficulty in obtaining labeled data for certain tasks. In this work, [...]
MSR Thesis Talk: Mateo Guaman Castro
Title: Self-Supervised Costmap Learning for Off-Road Vehicle Traversability Abstract: Estimating terrain traversability in off-road environments requires reasoning about complex interaction dynamics between the robot and these terrains. However, it is challenging to build an accurate physics model, or create informative labels to learn a model in a supervised manner, for these interactions. We propose a method [...]
MSR Thesis Talk: Gaoyue Zhou
Title: On Generalization and Benchmarking on Physical Robots Abstract: Robotics research has seen significant advancements; however, the field remains predominantly demo-driven, making direct comparisons between methods difficult without replicating them on individual setups. While many simulation benchmarks exist, they usually feature contrived datasets and do not accurately reflect real-world performance. In my thesis, we [...]
MSR Thesis Talk: Heng Yu
Title: Towards Real-time Controllable Neural Face Avatars Abstract: Neural Radiance Fields (NeRF) are compelling techniques for modeling dynamic 3D scenes from 2D image collections. These volumetric representations would be well suited for synthesizing novel facial expressions but for three problems. First, deformable NeRFs are object agnostic and model holistic movement of the scene: they can [...]
MSR Thesis Talk: Winnie Kuang
Title: Design and Integration of Semantic Mapping System for Forest Fire Mitigation Abstract: Remote sensing technologies can provide an automated approach to monitor and analyze conditions in the forest environment over a period of time for forest maintenance and wildfire mitigation efforts. In particular, unmanned aerial vehicles (UAVs) are a promising remote sensing modality since they [...]
MSR Thesis Talk: Jinqi Luo
Title: Vision Model Diagnosis: A Generative Perspective Abstract: In the evolving landscape of computer vision, deep learning has emerged as a transformative force, enhancing a myriad of societal facets. The real-world deployment of such a deep vision model requires a reliable evaluation, particularly when the model can have different sensitivities across various semantic attributes and concepts. [...]
MSR Thesis Talk: Daphne Chen
Title: Learning Task Preferences from Real-World Data Abstract: In order to provide personalized assistance that is capable of adapting to the needs of unique individuals, it is necessary to understand peoples’ preferences for different tasks. Robot assistance often assumes a static model of the individual, while in the real world, people have different capabilities and needs [...]
MSR Thesis Talk: Prasanna Kettavarapalyam Sriganesh
Title: Fast Staircase Detection and Estimation with Multi-View Merging for Multi-Robot Systems Abstract: When robotic systems are deployed in the real world, they demand advanced mobility capabilities to operate in complex, three-dimensional environments designed for human use, e.g., multi-level buildings. Staircases have been an integral part of facilitating vertical movement in these three-dimensional environments. This work [...]
MSR Thesis Talk: Aarush Gupta
Title: LightSpeed: Light and Fast Neural Light Fields on Mobile Devices Abstract: Real-time novel-view image synthesis on mobile devices is prohibitive due to limited on-device computational power and storage. Using volumetric rendering methods, such as NeRF and its derivatives, on mobile devices is not suitable due to the high computational cost of volumetric rendering. On the [...]
MSR Thesis Talk: Akshaya Kesarimangalam Srinivasan
Title: Multi-agent Multi-objective Ergodic Search Abstract: In order to find points of interest in a given domain, many planners use a priori information to guide the search to expedite the detection of targets. We present an approach to direct multiple agents (MA) to search a given domain subject to multiple objectives (MO), each characterized by its own information [...]
MSR Thesis Talk: Joshua Spisak
Title: Stochastic Optimization for Autonomous Navigation, Leveraging Parallel Computation Abstract: Stochastic Optimal Control (SOC) is a framework that allows disturbances and uncertainty in system models to be accounted for in its optimization framework. Despite accounting for this uncertainty, many first and second order methods for solving SOC problems are subject to local minima and are [...]
MSR Thesis Talk: Xuxin Cheng
Title: Learning Legged Robot Agility: Sim-to-Real and Beyond Abstract: Legged robotics has seen significant advancements in both manipulation and locomotion. However, there remain significant gaps compared to their biological counterparts, particularly in energy efficiency, natural motion, and the capacity for agile skills. This thesis primarily focuses on two aspects: the unified control of legged manipulators [...]
MSR Thesis Talk: Nishant Mohanty
Title: Multi-Robot Control using Control Barrier Functions: Theory and Application Abstract: Control Barrier Functions (CBFs) have emerged as a powerful theoretical tool for designing controllers with provable safety guarantees. This work presents a novel methodology that leverages CBFs to synthesize controllers for multi-robot coordination. Two multi-agent use cases are explored, i.e., a) Non-Cooperative Herding and [...]
MSR Thesis Talk: Yuyao Shi
Title: A Learning Approach to Understand How Spinal Cord Learns Multiple Behaviors Abstract: The spinal cord plays a crucial role in the control of human locomotion, generating motor patterns and coordinating reflex responses to sensory signals. Although this spinal control is traditionally viewed as a simple relay system, more recent neurophysiological evidence points to a [...]
Carnegie Mellon University
MSR Thesis Talk: FNU Abhimanyu
Title: Improving Robotic Ultrasound AI Using Optical Flow Abstract: Ultrasound is an important modality for medical intervention such as vascular access because it is safe, portable, and low-cost. However, ultrasound scanning requires trained sonographers who are scarce, and it can be challenging to perform ultrasound examinations in disaster or battlefield scenarios. This motivates us to automate [...]
MSR Thesis Talk: Lucas Casanova De Oliveira Nogueira
Title: SuperLoop: a LIDAR-based SLAM Back-end for Underground Exploration Abstract: Robots deployed in underground scenarios require a SLAM system that can handle a variety of challenges, such as the absence of GPS, large scale maps, bad illumination, and geometrically degenerate environments. It is nearly impossible for any SLAM solution to handle all these challenges perfectly, specially [...]
MSR Thesis Talk: Neil Khera
Title: PyCubed-Mini: A Low-Cost, Open-Source Satellite Research Platform Abstract: Satellite development has become more accessible with decreasing launch costs and shrinking hardware. However, the expenses associated with pre-built satellite kits remain high, making it difficult for student and hobbyist teams to participate. The lack of standardized satellite hardware and software further adds to the challenge, [...]
Strategy assessment for solving rich physical problems
Abstract: We present a framework that acts as an "intuitive physics reasoner" which takes in strategies expressed in natural language (whether from a human or LLM), and assesses their validity based on a physics knowledge library. We believe the ability to quickly determine whether a strategy is worth considering and allocating further resources to planning [...]
MSR Thesis Talk: Siva Kailas
Title: Multi-Robot Information Gathering for Spatiotemporal Environment Modelling Abstract: Learning to predict or forecast spatiotemporal (ST) environmental processes from a sparse set of samples collected autonomously is a difficult task from both a sampling perspective (collecting the best sparse samples) and from a learning perspective (predicting unseen locations or forecasting the next timestep). We investigate [...]
MRS Thesis Talk: Ruijie Fu
Title: Towards Mechanical Communication in Multi-Agent Locomotive Systems: Principally Kinematic Robots on a Shared Platform Abstract: Many biological multi-agent systems exhibit a mechanism for information exchange among individuals known as mechanical communication, which leads to the emergence of collective behavior within the group. One such example is the swarming behavior of bacteria, where they form rafts [...]
Architecture and Algorithms for Space-Based Global Wildlife Tracking
Abstract: Accurate satellite based positioning revolutionized several industries over the past two decades from agriculture to transportation. However, conventional GNSS receivers consume significant amounts of energy and are too large for many applications, including wildlife-tracking which is critical for conservation efforts and improving our understanding of the global climate. To address this capability gap, we [...]
Language-Conditioned Object Detection and Manipulation
Abstract: Traditional object detection methods are often confined to predefined object vocabularies, limiting their versatility in real-world scenarios where robots need to understand and execute diverse household tasks. Additionally, the 2D and 3D perception communities have typically pursued separate approaches tailored to their respective domains. In this thesis, we present a language-conditioned object detector with [...]
Exploring Diverse Interaction Types for Human in the Loop Robot Learning
Abstract: Teaching sessions between humans and robots will need to be maximally informative for optimal robot learning and to ease the human’s teaching burden. However, the bulk of prior work considers one or two modalities through which a human can convey information to a robot—namely, kinesthetic demonstrations and preference queries. Moreover, people will teach robots [...]