PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Bridging Generative and Discriminative Learning with Diffusion Models

GHC 4405

Abstract: Generative models have advanced significantly, synthesizing photorealistic images, videos, and text. Building on this progress, our work explores the potential of diffusion models to bridge generative and discriminative learning, uncovering new pathways for leveraging their strengths in visual perception tasks. In the first part, we propose Diff-2-in-1, a unified framework for multi-modal data generation [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Bring Hand to The Air: Towards Universal Aerial Manipulation

NSH 4305

Abstract: Uncrewed Aerial Vehicles (UAVs) have attracted the interest of researchers, industry, and the general public in many applications. Noticing that high-altitude tasks sometimes require active interaction with the environment, there have been more and more works focusing on aerial manipulation recently. Each of them has demonstrated the ability to use a specific aerial manipulator [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Robust Reinforcement Learning for Safety Critical Applications via Curricular Learning

NSH 4305

Abstract:  Reinforcement Learning (RL) presents great promises for autonomous agents. However, when using robots in a safety critical domain, a system has to be robust enough to be deployed in real life. For example, the robot should be able to perform across different scenarios it will encounter. The robot should avoid entering undesirable and irreversible [...]

VASC Seminar
Jiaqi Ma
Assistant Professor
University of Illinois Urbana-Champaign

Practical Challenges and Recent Advances in Data Attribution

3305 Newell-Simon Hall

Abstract: Data plays an increasingly crucial role in both the performance and the safety of AI models. Data attribution is an emerging family of techniques aimed at quantifying the impact of individual training data points on a model trained on them, which has found data-centric applications such as training data curation, instance-based explanation, and copyright [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Spatial Reasoning and Semantic Representations for Intelligent Multi-Robot Exploration and Navigation

NSH 4305

Abstract: Autonomous robot exploration is widely applied in areas such as search and rescue, environmental monitoring, and structural inspection. Multi-robot exploration has garnered significant attention in the robotics research community, as it enables faster task completion and greater coverage than a single robot can achieve. However, it presents unique challenges: behavior coordination is complex, communication [...]

MSR Thesis Defense
MSR Student / MSR Student
Robotics Institute,
Carnegie Mellon University

Autonomous Sensor Insertion and Exchange for Cornstalk Monitoring Robot

Newell-Simon Hall 4305

Abstract: Interactive sensors are an important component of robotic systems but often require manual replacement due to wear and tear. Automating this process can enhance system autonomy and facilitate long-term deployment. We developed an autonomous sensor exchange and maintenance system for an agriculture crop monitoring robot that inserts a nitrate sensor into cornstalks. A novel [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Leveraging Sense of Agency to Improve the Experience of Control Over Assistive Robots

GHC 6121

Abstract: In an age of autonomous driving and robotics, we are increasingly engaging with robots that deploy autonomous assistance. Cognitive science and human-computer interaction literature tells us that, when we apply autonomy in assistive settings, we are often augmenting the user's sense of agency over the system. Sense of agency is a phenomenon from cognitive [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Artificial Intelligence in Support of Emergency Care in the Field

GHC 6115

Abstract: Medical emergencies demand rapid and accurate interventions to save lives. Severe injuries often require surgical care within the first 60 minutes when timely action significantly improves survival rates. However, limited resources, remote locations, and unpredictable conditions often obstruct access to advanced medical care during this critical period. This thesis focuses on developing a medical [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Efficient Synthetic Data Generation and Utilization for Action Recognition and Universal Avatar Generation

NSH 3305

Abstract: Human-centered computer vision technology relies heavily on large, diverse datasets, but collecting data from human subjects is time-consuming, labor-intensive, and raises privacy concerns. To address these challenges, researchers are increasingly using synthetic data to augment real-world datasets. This thesis explores efficient methods for generating and utilizing synthetic data to train human-based computer vision models. [...]

MSR Thesis Defense
MSR Student / Research Associate I
Robotics Institute,
Carnegie Mellon University

Multi-Resolution Informative Path Planning for Small Teams of Robots

GHC 4405

Abstract: Unmanned aerial vehicles can increase the efficiency of information gathering applications . A key challenge is balancing the search across multiple locations of varying importance while determining the best sensing altitude, given each agent's finite operation time. In this work, we present a multi-resolution informative path planning approach for small teams of unmanned aerial [...]

PhD Thesis Defense
Postdoctoral Fellow
Robotics Institute,
Carnegie Mellon University

Communication-Efficient Active Reconstruction using Self-Organizing Gaussian Mixture Models

GHC 4405

Abstract: For the multi-robot active reconstruction task, this thesis proposes using Gaussian mixture models (GMMs) as the map representation that enables multiple downstream tasks: high-fidelity static scene reconstruction, communication-efficient map sharing, and safe informative planning. A new method called Self-Organizing Gaussian mixture modeling (SOGMM) is proposed that estimates the model complexity (i.e., number of Gaussian [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Agenda was sent via a calendar invite.

Field Robotics Center Seminar
Nathan Michael
Chief Technology Officer
Shield AI

From Lab to Launch

CIC CIC Buuilding Conference Room 1, LL Level

Bio: Nathan Michael is Shield AI’s Chief Technology Officer and a former Associate Research Professor in the Robotics Institute of Carnegie Mellon University (CMU). At CMU, Nathan was the Director of the Resilient Intelligent Systems Lab, a research lab dedicated to improving the performance and reliability of artificially intelligent and autonomous systems that operate in [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

Vision-Language Models for Hand-Object Interaction Prediction

Rashid Auditorium - 4401 Gates and Hillman Centers

Abstract: How can we predict future interaction trajectories of human hands in a scene given high-level colloquial task specifications in the form of natural language? In this paper, we extend the classic hand trajectory prediction task to two tasks involving explicit or implicit language queries. Our proposed tasks require extensive understanding of human daily activities [...]

Special Events

Robotics Institute Winter Party

All Robotics Institute Faculty. Staff, Students, and Visitors are invited to attend this event. Please join us for food, beverages, and casual conversation with colleagues. A calendar invite including details will be sent closer to the event.

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Agenda was sent via a calendar invite.

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Universal Semantic-Geometric Priors for Zero-Shot Robotic Manipulation

NSH 3305

Abstract: Visual imitation learning has shown promising results in robotic manipulation in recent years. However, its generalization to unseen objects is often limited by the size and diversity of training data. Although more large-scale robotic datasets are available, they remain significantly smaller than image and text datasets. Additionally, scaling these datasets is time-consuming and labor-intensive, [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Personalized Context-aware Multimodal Robot Feedback

GHC 4405

Abstract: In the field of human-robot interaction (HRI), integration of robots into social settings, such as healthcare and education, is gaining traction. Robots that provide individualized support to improve human performance and subjective experience will generally be more successful in these domains. Robots should personalize their interactions, be aware of the contextual nuances surrounding their [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Sensorized Soft Materials Systems with Integrated Electronics and Computing

NSH 3305

Abstract: The integration of soft and multifunctional materials in emerging technologies is becoming more widespread due to their ability to enhance or improve functionality in ways not possible using typical rigid alternatives. This trend is evident in various fields. For example, wearable technologies are increasingly designed using soft materials to improve modulus compatibility with biological [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Enabling Reliable Model-Based Planning with Inaccurate Models

GHC 8102

Abstract: This thesis aims to provide a framework for combining complementary tools that enable robots to manipulate objects in the world using diverse forms of knowledge. We consider heterogeneous types of knowledge, such as physics-based models, learned dynamics models, and model-free skills learned from human demonstrations. Each form of knowledge comes with its own assumptions [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Unlocking Generalization for Robotics via Scale and Modularity

GHC 4405

Abstract: How can we build generalist robot systems? Looking at fields such as vision and language, the common theme has been large scale end-to-end learning with massive, curated datasets. In robotics, on the other hand, scale alone may not be enough due to the significant multimodality of robotics tasks, lack of easily accessible data and [...]

RI Seminar
Assistant Professor
Robotics Institute,
Carnegie Mellon University

Towards Open World Robot Safety

1403 Tepper School Building

Abstract: Robot safety is a nuanced concept. We commonly equate safety with collision-avoidance, but in complex, real-world environments (i.e., the “open world’’) it can be much more: for example, a mobile manipulator should understand when it is not confident about a requested task, that areas roped off by caution tape should never be breached, and [...]

RI Seminar
Alfred Rizzi
Chief Technology Officer
Boston Dynamics AI Institute

RI Seminar with Alfred Rizzi

1403 Tepper School Building

RI Seminar
Jacob Andreas
Associate Professor
EECS and CSAIL, Massachusetts Institute of Technology

Good Old-Fashioned LLMs (or, Autoformalizing the World)

1403 Tepper School Building

Abstract: Classical formal approaches to artificial intelligence, based on manipulation of symbolic structures, have a number of appealing properties---they generalize (and fail) in predictable ways, provide interpretable traces of behavior, and can be formally verified or manually audited for correctness. Why are they so rarely used in the modern era? One of the major challenges [...]

RI Seminar
Ken Goldberg
Professor
Electrical Engineering & Computer Sciences, University of California, Berkeley

Unfamiliar Intelligence: Art, AI, and Robots

1403 Tepper School Building

Abstract: Shortly after the 1918 pandemic, the word "robot" was coined in a play about mechanical workers organizing a rebellion to defeat their human overlords. A century later, emerging advances in Artificial Intelligence and robotics, fueled by venture capital and governments, are disrupting labor, trade, and political stability. Claims about “superintelligence” and existential threats to [...]

RI Seminar
Nima Fazeli
Assistant Professor
Robotics and Mechanical Engineering, University of Michigan

RI Seminar with Nima Fazeli

1403 Tepper School Building

RI Seminar
Nikolay Atanasov
Associate Professor
Electrical and Computer Engineering, University of California, San Diego

RI Seminar with Nikolay Atanasov

1403 Tepper School Building

RI Seminar
Sangbae Kim
Professor
Mechanical Engineering, Massachusetts Institute of Technology

Physical Intelligence and Cognitive Biases Toward AI

1403 Tepper School Building

Abstract: When will robots be able to clean my house, dishes, and take care of laundry? While we source labor primarily from automated machines in factories, the penetration of physical robots in our daily lives has been slow. What are the challenges in realizing these intelligent machines capable of human level skill? Isn’t AI advanced [...]

RI Seminar
Charlie Kemp
Chief Technology Officer
Hello Robot Inc.

RI Seminar with Charlie Kemp

1403 Tepper School Building

RI Event

Robotics Institute Picnic

Please mark your calendars and plan to join us for the 2025 Robotics Institute Picnic! More information and RSVP e-vite to follow as we get closer to the event.