PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Unified Control for Over and Fully-Actuated Aerial Vehicles

NSH 3002

Abstract:  The growing domain of aerial robotics necessitates advancements in the control strategies and robustness of over-actuated and fully-actuated aerial vehicles. This thesis proposal makes contributions to this endeavor by providing in-depth analysis and methodologies concerning these vehicles, control allocation strategies during actuator failures, high-fidelity simulations, and a unified control framework. Our completed work has [...]

MSR Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Prasanna Kettavarapalyam Sriganesh

NSH 4305

Title:  Fast Staircase Detection and Estimation with Multi-View Merging for Multi-Robot Systems Abstract: When robotic systems are deployed in the real world, they demand advanced mobility capabilities to operate in complex, three-dimensional environments designed for human use, e.g., multi-level buildings. Staircases have been an integral part of facilitating vertical movement in these three-dimensional environments. This work [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Aarush Gupta

NSH 3305

Title: LightSpeed: Light and Fast Neural Light Fields on Mobile Devices Abstract: Real-time novel-view image synthesis on mobile devices is prohibitive due to limited on-device computational power and storage. Using volumetric rendering methods, such as NeRF and its derivatives, on mobile devices is not suitable due to the high computational cost of volumetric rendering. On the [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Akshaya Kesarimangalam Srinivasan

NSH 4305

Title: Multi-agent Multi-objective Ergodic Search Abstract:  In order to find points of interest in a given domain, many planners use a priori information to guide the search to expedite the detection of targets. We present an approach to direct multiple agents (MA) to search a given domain subject to multiple objectives (MO), each characterized by its own information [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Joshua Spisak

NSH 3305

Title: Stochastic Optimization for Autonomous Navigation, Leveraging Parallel Computation   Abstract: Stochastic Optimal Control (SOC) is a framework that allows disturbances and uncertainty in system models to be accounted for in its optimization framework. Despite accounting for this uncertainty, many first and second order methods for solving SOC problems are subject to local minima and are [...]

RI Event
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Yimin Tang

NSH 3305

Title: Solving Multi-Agent Target Assignment and Path Finding with a Single Constraint Tree Abstract: Multi-Agent Path Finding (MAPF) and Combined Target-Assignment and Path-Finding problem (TAPF) arise in many applications such as robotics, computer gaming, warehouse automation and traffic management at road intersections. Combined Target-Assignment and Path-Finding problem (TAPF) requires simultaneously assigning targets to agents and [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Xuxin Cheng

NSH 3305

Title: Learning Legged Robot Agility: Sim-to-Real and Beyond Abstract: Legged robotics has seen significant advancements in both manipulation and locomotion. However, there remain significant gaps compared to their biological counterparts, particularly in energy efficiency, natural motion, and the capacity for agile skills. This thesis primarily focuses on two aspects: the unified control of legged manipulators [...]

MSR Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Harry Freeman

NSH 4305

Title: Computer Vision-Based Phenotyping in Agriculture: Leveraging Semantic Information for Non-Destructive Small Crop Analysis Abstract: Fast and reliable non-destructive phenotyping of plants plays an important role in precision agriculture, as the information enables farmers to make real-time crop management decisions without affecting yield. To non-destructively phenotype crops, computer and stereo-vision based methods are commonly used, [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Nishant Mohanty

Newell-Simon Hall 1305

Title: Multi-Robot Control using Control Barrier Functions: Theory and Application Abstract: Control Barrier Functions (CBFs) have emerged as a powerful theoretical tool for designing controllers with provable safety guarantees. This work presents a novel methodology that leverages CBFs to synthesize controllers for multi-robot coordination. Two multi-agent use cases are explored, i.e., a) Non-Cooperative Herding and [...]

MSR Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Yuyao Shi

Newell-Simon Hall 4305

Title: A Learning Approach to Understand How Spinal Cord Learns Multiple Behaviors Abstract: The spinal cord plays a crucial role in the control of human locomotion, generating motor patterns and coordinating reflex responses to sensory signals. Although this spinal control is traditionally viewed as a simple relay system, more recent neurophysiological evidence points to a [...]

MSR Thesis Defense
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: FNU Abhimanyu

3305 Newell-Simon Hall

Title: Improving Robotic Ultrasound AI Using Optical Flow Abstract:  Ultrasound is an important modality for medical intervention such as vascular access because it is safe, portable, and low-cost. However, ultrasound scanning requires trained sonographers who are scarce, and it can be challenging to perform ultrasound examinations in disaster or battlefield scenarios. This motivates us to automate [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Vision-based Proprioceptive and Tactile Sensing for Soft Robots

Abstract: Soft robotic manipulators present many unique advantages in difficult manipulation tasks. The inherent compliance of soft robots' constituent deformable material makes them safe and reliable in delicate tasks such as harvesting fruit and assisting in household work. To address challenges in proprioceptive and tactile sensing for soft robots, we present a family of vision-based [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Lucas Casanova De Oliveira Nogueira

Newell-Simon Hall 4305

Title: SuperLoop: a LIDAR-based SLAM Back-end for Underground Exploration Abstract: Robots deployed in underground scenarios require a SLAM system that can handle a variety of challenges, such as the absence of GPS, large scale maps, bad illumination, and geometrically degenerate environments. It is nearly impossible for any SLAM solution to handle all these challenges perfectly, specially [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning via Visual-Tactile Interaction

NSH 1305

Abstract: Humans learn by interacting with their surroundings using all of their senses. The first of these senses to develop is touch, and it is the first way that young humans explore their environment, learn about objects, and tune their cost functions (via pain or treats). Yet, robots are often denied this highly informative and [...]

MSR Thesis Defense
Engineer II
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Neil Khera

TBD

Title: PyCubed-Mini: A Low-Cost, Open-Source Satellite Research Platform   Abstract: Satellite development has become more accessible with decreasing launch costs and shrinking hardware. However, the expenses associated with pre-built satellite kits remain high, making it difficult for student and hobbyist teams to participate. The lack of standardized satellite hardware and software further adds to the challenge, [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

Strategy assessment for solving rich physical problems

NSH 4305

Abstract: We present a framework that acts as an "intuitive physics reasoner" which takes in strategies expressed in natural language (whether from a human or LLM), and assesses their validity based on a physics knowledge library. We believe the ability to quickly determine whether a strategy is worth considering and allocating further resources to planning [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Siva Kailas

1305 Newell Simon Hall

Title: Multi-Robot Information Gathering for Spatiotemporal Environment Modelling Abstract: Learning to predict or forecast spatiotemporal (ST) environmental processes from a sparse set of samples collected autonomously is a difficult task from both a sampling perspective (collecting the best sparse samples) and from a learning perspective (predicting unseen locations or forecasting the next timestep). We investigate [...]

MSR Thesis Defense
MSR Alumna
Robotics Institute,
Carnegie Mellon University

MRS Thesis Talk: Ruijie Fu

Newell-Simon Hall 3305

Title: Towards Mechanical Communication in Multi-Agent Locomotive Systems: Principally Kinematic Robots on a Shared Platform Abstract: Many biological multi-agent systems exhibit a mechanism for information exchange among individuals known as mechanical communication, which leads to the emergence of collective behavior within the group. One such example is the swarming behavior of bacteria, where they form rafts [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

Faculty Events
Assistant Professor
Robotics Institute,
Carnegie Mellon University

Robot Learning, Wearable Sensing, and Teleoperation in Pursuit of Robotic Caregivers

Abstract Designing safe and reliable robotic assistance for caregiving is a grand challenge in robotics. A sixth of the United States population is over the age of 65 and in 2014 more than a quarter of the population had a disability. Robotic caregivers could positively benefit society; yet, physical robotic assistance presents several challenges and [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Personalized Context-aware Affective Nonverbal Robot Feedback

NSH 1305

Abstract:  We first consider the problem of estimating context, specifically key features of the human state. We predict engagement-related events in an educational activity before the end of that activity, which could allow the robot to provide feedback early enough to improve the human's experience. We then explore generating nonverbal affective robot behavior by correlating [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Redefining the Perception-Action Interface: Visual Action Representations for Contact-Centric Manipulation

GHC 6501

Abstract:  In robotics, understanding the link between perception and action is pivotal. Typically, perception systems process sensory data into state representations like  segmentations and bounding boxes, which a planner uses to plan actions. However, this state estimation approach can fail in environments with partial observability, and in cases with challenging object properties like transparency and deformability.  [...]

Special Events

RI Picnic

Schenley Park Vietnam Veteran's Pavilion , United States

The RI Picnic will be held at the Vietnam Veteran's Pavilion @ Schenley Park on Overlook Drive, Tuesday, August 29, 1-7pm. SOCIALIZE, EAT, DRINK & BE MERRY! Receive this year's RI giveaway item; witness the exciting final rounds of the annual RI croquet tournament; enjoy lawn games right at our own pavilion area. Plan to spend some time at the [...]

RI Event
PhD Student
Robotics Institute,
Carnegie Mellon University

Continual Robot Learning: Benchmarks and Modular Methods

GHC 6501

Abstract: Humans adapt continuously to the world around us, allowing us to acquire new skills and explore diverse environments seamlessly. Current AI methods, however, cannot attain this versatility. Instead, they are typically trained with vast datasets, and learn all tasks simultaneously. However, the trained models have limited ability to adapt to changing contexts, and are [...]

MSR Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Architecture and Algorithms for Space-Based Global Wildlife Tracking

GHC 6501

Abstract: Accurate satellite based positioning revolutionized several industries over the past two decades from agriculture to transportation. However, conventional GNSS receivers consume significant amounts of energy and are too large for many applications, including wildlife-tracking which is critical for conservation efforts and improving our understanding of the global climate. To address this capability gap, we [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Multi-Human 3D Reconstruction from Monocular Videos

NSH 4305

Abstract: We study the problem of multi-human 3D reconstruction from videos captured in the wild. Human movements are dynamic, and accurately reconstructing them in various settings is crucial for developing immersive social telepresence, assistive humanoid robots, and augmented reality systems. However, creating such a system requires addressing fundamental issues with previous works regarding the data [...]

MSR Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Language-Conditioned Object Detection and Manipulation

NSH 4305

Abstract: Traditional object detection methods are often confined to predefined object vocabularies, limiting their versatility in real-world scenarios where robots need to understand and execute diverse household tasks. Additionally, the 2D and 3D perception communities have typically pursued separate approaches tailored to their respective domains. In this thesis, we present a language-conditioned object detector with [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

How I Learned to Love Blobs: The Power of Gaussian Representations in Differentiable Rendering and Optimization

NSH 3305

Abstract: In this thesis, we explore the use of Gaussian Representations in multiple application areas of computer vision and robotics. In particular, we design a ray-based differentiable renderer for 3D Gaussians that can be used to solve multiple classic computer vision problems in a unified manner. For example, we can reconstruct 3D shapes from color, [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Watch, Practice, Improve: Towards In-the-wild Manipulation

NSH 3305

Abstract: The longstanding dream of many roboticists is to see robots perform diverse tasks in diverse environments. To build such a robot that can operate anywhere, many methods train on robotic interaction data. While these approaches have led to significant advances, they rely on heavily engineered setups or high amounts of supervision, neither of which [...]

VASC Seminar
Aayush Bansal
Startup

Generating Beautiful Pixels

Newell-Simon Hall 3305

Abstract: In this talk, I will present three experiments that use low-level image statistics to generate high-resolution detailed outputs. In the first experiment, I will use 2D pixels to efficiently mine hard examples for better learning. Simply biasing ray sampling towards hard ray examples enables learning of neural fields with more accurate high-frequency detail in less [...]

VASC Seminar
Viraj Prabhu
CS PhD Student
Georgia Institute of Technology

Towards Reliable Computer Vision Systems

Newell-Simon Hall 3305

Abstract:  The real world has infinite visual variation – across viewpoints, time, space, and curation. As deep visual models become ubiquitous in high-stakes applications, their ability to generalize across such variation becomes increasingly important. In this talk, I will present opportunities to improve such generalization at different stages of the ML lifecycle: first, I will [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Towards Photorealistic Dynamic Capture and Animation of Human Hair and Head

NSH 4305

Abstract: Realistic human avatars play a key role in immersive virtual telepresence. To reach a high level of realism, a human avatar needs to faithfully reflect human appearance. A human avatar should also be drivable and express natural motions. Existing works have made significant progress in building drivable realistic face avatars, but they rarely include [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Modeling Dynamic Clothing for Data-Driven Photorealistic Avatars

NSH 3305

Abstract: In this thesis, we aim to build photorealistic animatable avatars of humans wearing complex clothing in a data-driven manner. Such avatars will be a critical technology to enable future applications such as immersive telepresence in Virtual Reality (VR) and Augmented Reality (AR). Existing full-body avatars that jointly model geometry and view-dependent texture using Variational [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Manipulation Among Movable Objects for Pick-and-Place Tasks in Cluttered 3D Workspaces

NSH 1305

Abstract: In cluttered real-world workspaces, simple pick-and-place tasks for robot manipulators can be quite challenging to solve. Often there is no collision-free trajectory that allows the robot to grasp and extract a desired object from the scene. This requires motion planning algorithms to reason about rearranging some of the “movable” clutter in the scene so [...]

RI Seminar
Paul Debevec
Chief Research Officer
Eyeline Studios

Transforming Hollywood Visual Effects with Graphics and Vision

3305 Newell-Simon Hall

Abstract: Paul will describe his path to developing visual effects technology used in hundreds of movies, including The Matrix, Spider-Man 2, Benjamin Button, Avatar, Maleficent, Furious 7, and Blade Runner: 2049. These techniques include image-based modeling and rendering, high dynamic range imaging, image-based lighting, and high-resolution facial scanning for photoreal digital actors. Paul will also [...]

VASC Seminar
Bharath Hariharan
Assistant Professor
Cornell University

Vision without labels

3305 Newell-Simon Hall

Abstract: Deep learning has revolutionized all aspects of computer vision, but its successes have come from supervised learning at scale: large models trained on ever larger labeled datasets. However this reliance on labels makes these systems fragile when it comes to new scenarios or new tasks where labels are unavailable. This is in stark contrast to [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

RI Seminar
Shuran Song
Assistant Professor
Robotics and Embodied AI Lab, Stanford University

Learning Meets Gravity: Robots that Learn to Embrace Dynamics from Data

1305 Newell Simon Hall

Abstract: Despite the incredible capabilities (speed and repeatability) of our hardware today, many robot manipulators are deliberately programmed to avoid dynamics – moving slow enough so they can adhere to quasi-static assumptions of the world. In contrast, people frequently (and subconsciously) make use of dynamic phenomena to manipulate everyday objects – from unfurling blankets, to [...]

VASC Seminar
Yong Jae Lee
Associate Professor
Department of Computer Sciences , University of Wisconsin-Madison

Large Multimodal (Vision-Language) Models for Image Generation and Understanding

Newell-Simon Hall 3305

Abstract: Large Language Models and Large Vision Models, also known as Foundation Models, have led to unprecedented advances in language understanding, visual understanding, and AI. In particular, many computer vision problems including image classification, object detection, and image generation have benefited from the capabilities of such models trained on internet-scale text and visual data. In [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

RI Seminar
Fei Miao
Associate Professor
Department of Computer Science & Engineering, University of Connecticut

Learning and Control for Safety, Efficiency, and Resiliency of Embodied AI

1305 Newell Simon Hall

Abstract: The rapid evolution of ubiquitous sensing, communication, and computation technologies has revolutionized of cyber-physical systems (CPS) across virous domains like robotics, smart grids, aerospace, and smart cities. Integrating learning into dynamic systems control presents significant Embodied AI opportunities. However, current decision-making frameworks lack comprehensive understanding of the tridirectional relationship among communication, learning and control, [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Generalizable Dexterity with Reinforcement Learning

GHC 4405

Abstract: Dexterity, the ability to perform complex interactions with the physical world, is at the core of robotics. However, existing research in robot manipulation has been focused on tasks that involve limited dexterity, such as pick-and-place. The motor skills of the robots are often quasi-static, have a predefined or limited sequence of contact events, and [...]

VASC Seminar
Mohamed Elhoseiny
Assistant Professor
Computer Science, KAUST

Imaginative Vision Language Models: Towards human-level imaginative AI skills transforming species discovery, content creation, self-driving cars, and emotional health

3305 Newell-Simon Hall

Abstract:   Most existing AI learning methods can be categorized into supervised, semi-supervised, and unsupervised methods. These approaches rely on defining empirical risks or losses on the provided labeled and/or unlabeled data. Beyond extracting learning signals from labeled/unlabeled training data, we will reflect in this talk on a class of methods that can learn beyond the vocabulary [...]

VASC Seminar
Kenneth Marino
Research Scientist
Google DeepMind

World Knowledge in the Time of Large Models

Newell-Simon Hall 3305

Abstract:  This talk will discuss the massive shift that has come about in the vision and ML community as a result of the large pre-trained language and language and vision models such as Flamingo, GPT-4, and other models. We begin by looking at the work on knowledge-based systems in CV and robotics before the large model [...]