Efficient Synthetic Data Generation and Utilization for Action Recognition and Universal Avatar Generation
Abstract: Human-centered computer vision technology relies heavily on large, diverse datasets, but collecting data from human subjects is time-consuming, labor-intensive, and raises privacy concerns. To address these challenges, researchers are increasingly using synthetic data to augment real-world datasets. This thesis explores efficient methods for generating and utilizing synthetic data to train human-based computer vision models. [...]
Multi-Resolution Informative Path Planning for Small Teams of Robots
Abstract: Unmanned aerial vehicles can increase the efficiency of information gathering applications . A key challenge is balancing the search across multiple locations of varying importance while determining the best sensing altitude, given each agent's finite operation time. In this work, we present a multi-resolution informative path planning approach for small teams of unmanned aerial [...]
Communication-Efficient Active Reconstruction using Self-Organizing Gaussian Mixture Models
Abstract: For the multi-robot active reconstruction task, this thesis proposes using Gaussian mixture models (GMMs) as the map representation that enables multiple downstream tasks: high-fidelity static scene reconstruction, communication-efficient map sharing, and safe informative planning. A new method called Self-Organizing Gaussian mixture modeling (SOGMM) is proposed that estimates the model complexity (i.e., number of Gaussian [...]
RI Faculty Business Meeting
Meeting for RI Faculty. Agenda was sent via a calendar invite.
From Lab to Launch
Bio: Nathan Michael is Shield AI’s Chief Technology Officer and a former Associate Research Professor in the Robotics Institute of Carnegie Mellon University (CMU). At CMU, Nathan was the Director of the Resilient Intelligent Systems Lab, a research lab dedicated to improving the performance and reliability of artificially intelligent and autonomous systems that operate in [...]
Vision-Language Models for Hand-Object Interaction Prediction
Abstract: How can we predict future interaction trajectories of human hands in a scene given high-level colloquial task specifications in the form of natural language? In this paper, we extend the classic hand trajectory prediction task to two tasks involving explicit or implicit language queries. Our proposed tasks require extensive understanding of human daily activities [...]
Robotics Institute Winter Party
All Robotics Institute Faculty. Staff, Students, and Visitors are invited to attend this event. Please join us for food, beverages, and casual conversation with colleagues. A calendar invite including details will be sent closer to the event.
In Search of Shmoo: The Quest to Build a Capable Low-cost Ballbot for Research and Education
RI Faculty Business Meeting
Meeting for RI Faculty. Agenda was sent via a calendar invite.
Universal Semantic-Geometric Priors for Zero-Shot Robotic Manipulation
Abstract: Visual imitation learning has shown promising results in robotic manipulation in recent years. However, its generalization to unseen objects is often limited by the size and diversity of training data. Although more large-scale robotic datasets are available, they remain significantly smaller than image and text datasets. Additionally, scaling these datasets is time-consuming and labor-intensive, [...]
RI Faculty Business Meeting
Meeting for RI Faculty. Agenda was sent via a calendar invite.
Personalized Context-aware Multimodal Robot Feedback
Abstract: In the field of human-robot interaction (HRI), integration of robots into social settings, such as healthcare and education, is gaining traction. Robots that provide individualized support to improve human performance and subjective experience will generally be more successful in these domains. Robots should personalize their interactions, be aware of the contextual nuances surrounding their [...]
Sensorized Soft Materials Systems with Integrated Electronics and Computing
Abstract: The integration of soft and multifunctional materials in emerging technologies is becoming more widespread due to their ability to enhance or improve functionality in ways not possible using typical rigid alternatives. This trend is evident in various fields. For example, wearable technologies are increasingly designed using soft materials to improve modulus compatibility with biological [...]
Enabling Reliable Model-Based Planning with Inaccurate Models
Abstract: This thesis aims to provide a framework for combining complementary tools that enable robots to manipulate objects in the world using diverse forms of knowledge. We consider heterogeneous types of knowledge, such as physics-based models, learned dynamics models, and model-free skills learned from human demonstrations. Each form of knowledge comes with its own assumptions [...]
Unlocking Generalization for Robotics via Scale and Modularity
Abstract: How can we build generalist robot systems? Looking at fields such as vision and language, the common theme has been large scale end-to-end learning with massive, curated datasets. In robotics, on the other hand, scale alone may not be enough due to the significant multimodality of robotics tasks, lack of easily accessible data and [...]
Advancing Multimodal Sensing and Robotic Interfaces for Chronic Care
Abstract: The healthcare system prioritizes reactive care for acute illnesses, often overlooking the ongoing needs of individuals with chronic conditions that require long-term management and personalized care. Addressing this gap through technology can empower patients to better manage their conditions, enhancing independence and quality of life. Multimodal sensing, incorporating inertial, acoustic, and vision-based sensors, within [...]
RI Seminar with Aaron Johnson
RI Faculty Business Meeting
Meeting for RI Faculty. Agenda was sent via a calendar invite.
Towards Open World Robot Safety
Abstract: Robot safety is a nuanced concept. We commonly equate safety with collision-avoidance, but in complex, real-world environments (i.e., the “open world’’) it can be much more: for example, a mobile manipulator should understand when it is not confident about a requested task, that areas roped off by caution tape should never be breached, and [...]
RI Seminar with Alfred Rizzi
Good Old-Fashioned LLMs (or, Autoformalizing the World)
Abstract: Classical formal approaches to artificial intelligence, based on manipulation of symbolic structures, have a number of appealing properties---they generalize (and fail) in predictable ways, provide interpretable traces of behavior, and can be formally verified or manually audited for correctness. Why are they so rarely used in the modern era? One of the major challenges [...]
Unfamiliar Intelligence: Art, AI, and Robots
Abstract: Shortly after the 1918 pandemic, the word "robot" was coined in a play about mechanical workers organizing a rebellion to defeat their human overlords. A century later, emerging advances in Artificial Intelligence and robotics, fueled by venture capital and governments, are disrupting labor, trade, and political stability. Claims about “superintelligence” and existential threats to [...]
RI Seminar with Nima Fazeli
RI Seminar with Nikolay Atanasov
Physical Intelligence and Cognitive Biases Toward AI
Abstract: When will robots be able to clean my house, dishes, and take care of laundry? While we source labor primarily from automated machines in factories, the penetration of physical robots in our daily lives has been slow. What are the challenges in realizing these intelligent machines capable of human level skill? Isn’t AI advanced [...]
