MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

Enhancing Reinforcement Learning with Error-Prone Language Models

GHC 6501

The correct specification of reward models is a well-known challenge in reinforcement learning. Hand-crafted reward functions, which are usually sparse, often lead to inefficient or suboptimal policies, misalignment with user values, or difficulties in attributing credit or blame within multi-agent systems. Reinforcement learning from human feedback is a successful technique that can mitigate such issues [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Efficient Multi-Agent Motion Planning using Local Policies

WEH 4625

Abstract: Teams of multiple robots working together can achieve challenging tasks like warehouse automation, search and rescue, and cooperative construction. However, finding efficient collision-free motions for all agents is extremely challenging as the complexity of the multi-agent motion planning (MAMP) problem grows exponentially with the number of agents. Multi-Agent Path Finding (MAPF) is a subset [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

Foundation Control Model for General Embodied Intelligence

3305 Newell-Simon Hall

Abstract: With the growing accessibility of humanoid hardware and rapid advances in foundation models, we are entering an era where achieving general embodied intelligence is within reach—enabling humanoid robots to perform a wide range of tasks in human-centric environments. Despite significant progress in language and vision foundation models, controlling humanoids with high degrees of freedom [...]

MSR Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning Humanoid Robots from Simulation to Real to Simulation

Gates Hillman Center 6115

Abstract: How do we teach humanoid robots to move like humans—and do so reliably in the real world? In this talk, I’ll share my journey in building a learning-based pipeline that closes the loop between simulation and reality for humanoid whole-body control. Starting from real-time teleoperation (H2O), to scalable data humanoid collection (OmniH2O), to learning [...]

Faculty Candidate
Jason Ma
University of Pennsylvania

Faculty Candidate Talk: Jason Ma

Newell-Simon Hall 4305

Title: Internet Supervision for Robot Learning Abstract: The availability of internet-scale data has led to impressive large-scale AI models in various domains, such as vision and language. For learning robot skills, despite recent efforts in crowd-sourcing robot data, robot-specific datasets remain orders of magnitude smaller. Rather than focusing on scaling robot data, my research takes the alternative path of directly [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

Experience-Based Action Advising for Multi-Agent Teaming

GHC 6115

Abstract: We study how to improve coordination efficiency for multi-agent teams with heterogeneously experienced agents. In such a setting, experienced agents can transfer their knowledge to less experienced agents to accelerate their learning, while leveraging the students' initial expertise to inform what knowledge to transfer. Inspired by this idea, this work specifically assumes one teacher [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

Towards Controllable Sampling and Diverse Score Distillation in Diffusion Models

GHC 6115

Abstract: Denoising diffusion models have emerged as a powerful paradigm for generative modeling, which has been widely used for perception, generation, and action. These models can be utilized through sampling or score distillation; however, existing methods lack controllability in sampling and suffer from limited diversity in score distillation. In this thesis, we propose two complementary mechanisms to enhance the [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

RESCUE Rollers: A Platform for Collaborative, Multi-robot Exploration in Search and Rescue

GHC 4405

Abstract: The use of robotic platforms for search and rescue remains a significant challenge for many roboticist. While human and animal first responders play critical roles, their effectiveness can be limited by biological constraints. Robotic systems offer the potential to overcome these limitations, especially in environments inaccessible to humans and animals due to size or [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Scaling, Automating and Adapting Sim-to-real Policy Learning

GHC 6121

Abstract: Building a generalist robot capable of performing diverse tasks in unstructured environments remains a longstanding challenge. A recent trend in robot learning aims to address this by scaling up demonstration datasets for imitation learning. However, most large-scale robotics datasets are collected in the real-world, often via manual teleoperation. This process is labor-intensive, slow, hardware-dependent, [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

Generative 3D Garment Modeling with Sparse Visual Cues

Gates Hillman Center 4405

Abstract: As digital apparel becomes increasingly vital to virtual environments and personalized experiences, there is a growing need for intuitive tools that enable non-experts to create and interact with 3D garments. To broaden accessibility, these tools must function effectively with minimal input - raising the key question: How can we achieve high-quality 3D garment modeling [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Towards Pragmatic Time Series Intelligence

CIC LL06

Abstract: This thesis aims to democratize time series intelligence by making advanced modeling capabilities accessible to users without specialized machine learning knowledge. We pursue this goal through three complementary contributions that build foundation models, improve our understanding of them, and address challenges emerging in their practical use. We start by introducing MOMENT, the first family [...]

RI Event
PhD Student
Robotics Institute,
Carnegie Mellon University

Underwater 3D Visual Perception and Generation

NSH 4305

Abstract: With modern robotic technologies, seafloor imagery has become more accessible to researchers and the public. This thesis leverages deep learning and 3D vision techniques to deliver valuable information from seafloor image observations collected by robotic platforms. Despite the widespread use of deep learning and 3D vision algorithms across various fields, underwater imaging presents unique [...]

Faculty Events
Ji Zhang
Systems Scientist
Robotics Institute, Carnegie Mellon University

Autonomous Exploration and Navigation, Full Autonomy System, and Beyond

Newell-Simon Hall 4305

Abstract: In this talk, I will present work on autonomous exploration and introduce our full autonomy system. The work started several years ago from lidar-based state estimation. Building upon the state estimation module, the autonomy system now contains multiple fundamental modules, e.g. collision avoidance, terrain traversability analysis, and waypoint following. At the high level of [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

Towards Efficient and Accurate Neural Geometry and Appearance Representations

GHC 6501

Abstract: Neural scene representations have transformed the way we model and understand the visual world, enabling stunningly realistic reconstructions from image data. However, these advances often come at a significant computational cost, particularly due to the inefficiencies in volume rendering. In this talk, I’ll present GL-NeRF, a new approach that tackles this challenge from a [...]

RI Seminar
Charlie Kemp
Chief Technology Officer
Hello Robot Inc.

RI Seminar with Charlie Kemp

1403 Tepper School Building

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Advancing Spacecraft Autonomy: Optimal GNC, Vision-Based Estimation, and Systems Integration for Small Spacecraft

GHC 6501

Abstract: Optimization and machine learning-based methods are increasingly critical in enhancing the autonomy, efficiency, and overall return on investment (ROI) of small, resource-constrained spacecraft. By enabling more effective decision-making, adaptive control, and robust state estimation, these techniques expand mission capabilities while operating within strict mass, power, and computational limitations. This thesis builds on previous contributions [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Multimodal Robot Learning for Contact-Rich Manipulation

CIC LL06

Abstract: Robots operating in the real world can leverage intentional contacts with objects to understand and manipulate them effectively—especially in cluttered, partially observable environments where vision alone is insufficient. This thesis explores how intentional physical interactions, combined with haptic sensing, can provide rich spatial, temporal, and physical cues that enhance a robot’s perception and decision-making. [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Rethinking the Safety Case for Risk-Aware Social Embodied Intelligence

WEH 6403

Abstract: Achieving real-world robot safety requires more than avoiding risk—it demands embracing and managing it effectively. This thesis presents a safety case for risk-aware decision-making and behavior modeling in complex, multi-agent environments such as aviation and autonomous driving. We argue that safety arises from an agent’s ability to anticipate uncertainty, reason about intent, and act [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Human-System Communications for Expectation Mismatch

NSH 4305

Abstract: Robots, and autonomous systems in general, are becoming increasingly more advanced beyond traditional functions. This can potentially widen the mismatch between human expectations of system behaviors during interaction, especially when the systems behave unexpectedly. Unexpected system behaviors could induce negative emotional responses in humans, which not all systems have the capability of recognizing and [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Integrating Safety Across the Learning-Based Perception Pipeline: From Training to Deployment

CIC LL06

Abstract: Robots operating in safety-critical environments must reason under uncertainty and novel situations. However, recent advances in data-driven perception have made it challenging to provide formal safety guarantees, particularly when systems encounter out-of-distribution or previously unseen inputs. For such systems to be safely deployed in the real world, we need to incorporate safety considerations alongside [...]

RI Event

Robotics Institute Picnic

Please mark your calendars and plan to join us for the 2025 Robotics Institute Picnic! More information and RSVP e-vite to follow as we get closer to the event.