MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Yves Georgy Daoud

NSH 4305

Title: Spatial Tasking in Human-Robot Collaborative Exploration Abstract: This work develops a methodology for collaborative human-robot exploration that leverages implicit coordination. Most autonomous single- and multi-robot exploration systems require a remote operator to provide explicit guidance to the robot team. Few works consider how to integrate the human partner alongside robots to provide guidance in the [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Ambareesh Revanur

Gates Hillman Center 4405

Title: Towards Video-based Physiology Estimation Abstract: RGB-video based human physiology estimation has a wide range of practical applications in telehealth, sports and deep fake detection. Therefore, researchers in the community have collected several video datasets and have advanced new methods over the years. In this dissertation, we study these methods extensively and aim to address the [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Raghavv Goel

Wean Hall 2302

Title: Automating Ultrasound Based Vascular Access Abstract: Timely care of trauma patients is important to prevent casualties in resource-limited regions such as the battlefield. In order to treat such trauma using point of care diagnosis, medical practitioners typically use an ultrasound for vascular access or detection of subcutaneous splinters for providing critical care. The problem here is two-fold: [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Mayank Singh

3305 Newell-Simon Hall

Title: Analogical Networks: Memory-Modulated In-Context 3D Parsing Abstract: Recent advances in the applications of deep neural networks to numerous visual perception tasks have shown excellent performance. However, this generally requires access to large amount of training samples and hence one persistent challenge is the setting of few-shot learning. In most existing works, a separate parametric neural [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Learning with Diverse Forms of Imperfect and Indirect Supervision

NSH 4305

Abstract: Powerful Machine Learning (ML) models trained on large, annotated datasets have driven impressive advances in fields including natural language processing and computer vision. In turn, such developments have led to impactful applications of ML in areas such as healthcare, e-commerce, and predictive maintenance. However, obtaining annotated datasets at the scale required for training high [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Yutian Lei

Title: ARC: AdveRsarial Calibration between Modalities Abstract: Advances in computer vision and machine learning techniques have led to flourishing success in RGB-input perception tasks, which has also opened unbounded possibilities for non-RGB-input perception tasks, such as object detection from wireless signals, point clouds, and infrared light. However, compared to the matured development pipeline of RGB-input [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

FRIDA: Supporting Artistic Communication in Real-World Image Synthesis Through Diverse Input Modalities

NSH 4305

Abstract: FRIDA, a Framework and Robotics Initiative for Developing Arts, is a robot painting system designed to translate an artist's high-level intentions into real world paintings. FRIDA can paint from combinations of input images, text, style examples, sounds, and sketches. Planning is performed in a differentiable, simulated environment created using real data from the robot [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Perception for High-Speed Off-Road Driving

GHC 4405

Abstract: On-road autonomous driving has seen rapid progress in recent years with driverless vehicles being tested in various cities worldwide. However, this progress is limited to cities with well-established infrastructure and has yet to transfer to off-road regimes with unstructured environments and few paved roads. Advances in high-speed and reliable autonomous off-road driving can unlock [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Continual Learning of Compositional Skills for Robust Robot Manipulation

NSH 4305

Abstract: Real world robots need to continuously learn new manipulation tasks in a lifelong learning manner. These new tasks often share sub-structures (in the form of sub-tasks, controllers) with previously learned tasks. To utilize these shared sub-structures, we explore a compositional and object-centric approach to learn manipulation tasks. While compositionality in robot manipulation can manifest [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Samuel Ong

Title: Data-Driven Slip Model for Improved Localization and Path Following applied to Lunar Micro-Rovers Abstract Micro-lunar rovers need to solve a slew of challenges on the Moon, with no human intervention. One such challenge is the need to know their location in order to navigate and build maps. However, localization is challenging on the moon due [...]

PhD Thesis Defense
Postdoctoral Fellow
Robotics Institute,
Carnegie Mellon University

Computational Interferometric Imaging

NSH 4305

Abstract: Imaging systems typically accumulate photons that, as they travel from a light source to a camera, follow multiple different paths and interact with several scene objects. This multi-path accumulation process confounds the information that is available in captured images about the scene and makes using these images to infer properties of scene objects, such [...]

Special Talk
Senior Systems Scientist
Robotics Institute,
Carnegie Mellon University

Making AI trustworthy and understandable by clinicians

Newell-Simon Hall 4305

Abstract:  Understandable-AI techniques facilitate to use of AI as a tool by human experts, giving humans insight into how AI decisions are made thereby helping experts discern which AI predictions should or shouldn’t be trusted.  Understandable techniques may be especially useful for applications with insufficient validation data for regulatory approval, for which human experts must remain the final decision [...]

VASC Seminar
Christoph Lassner
Senior Research Scientist
Epic Games

Towards Interactive Radiance Fields

Newell-Simon Hall 3305

Abstract:  Over the last years, the fields of computer vision and computer graphics have increasingly converged. Using the exact same processes to model appearance during 3D reconstruction and rendering has shown tremendous benefits, especially when combined with machine learning techniques to model otherwise hard-to-capture or -simulate optical effects. In this talk, I will give an [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Robust and Context-Aware Real-Time Collaborative Robot Handling with Dynamic Gesture Commands

GHC 6501

Abstract: Real-time collaborative robot (cobot) handling is a task where the cobot maneuvers an object under human dynamic gesture commands. Enabling dynamic gesture commands is useful when the human needs to avoid direct contact with the robot or the object handled by the robot. However, the key challenge lies in the heterogeneity in human behaviors [...]

RI Seminar
Dorsa Sadigh
Assistant Professor
Computer Science Electrical Engineering, Stanford University

Learning Representations for Interactive Robotics

Newell-Simon Hall 1305

In this talk, I will be discussing the role of learning representations for robots that interact with humans and robots that interactively learn from humans through a few different vignettes. I will first discuss how bounded rationality of humans guided us towards developing learned latent action spaces for shared autonomy. It turns out this “bounded rationality” is not a [...]

RI Seminar
Russ Tedrake
Professor
Electrical Engineering & Computer Science, MIT

Motion Planning Around Obstacles with Graphs of Convex Sets

1305 Newell Simon Hall

Abstract: In this talk, I'll describe a new approach to planning that strongly leverages both continuous and discrete/combinatorial optimization. The framework is fairly general, but I will focus on a particular application of the framework to planning continuous curves around obstacles. Traditionally, these sort of motion planning problems have either been solved by trajectory optimization [...]

RI Seminar
Jorgen Pedersen
Chief Operating Officer
Sarcos Technology and Robotics Corporation

RE2 Robotics: from RI spinout to Acquisition

1305 Newell Simon Hall

Abstract: It was July 2001.  Jorgen Pedersen founded RE2 Robotics.  It was supposed to be a temporary venture while he figured out his next career move.  But the journey took an unexpected course.  RE2 became a leading developer of mobile manipulation systems.  Fast forward to 2022, RE2 Robotics exited via an acquisition to Sarcos Technology and [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Equivalent Policy Sets for Learning Aligned Models and Abstractions

GHC 4405

Abstract: Recent successes in model-based reinforcement learning (MBRL) have demonstrated the enormous value that learned representations of environmental dynamics (i.e., models) can impart to autonomous decision making. While a learned model can never perfectly represent the dynamics of complex environments, models that are accurate in the "right” ways may still be highly useful for decision [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Dynamic Route Guidance in Vehicle Networks by Simulating Future Traffic Patterns

GHC 4405

Abstract: Roadway congestion leads to wasted time and money and environmental damage. Since adding more roadway capacity is often not possible in urban environments, it is becoming more important to use existing road networks more efficiently. Toward this goal, recent research in real-time, schedule-driven intersection control has shown an ability to significantly reduce the delays [...]

VASC Seminar
Rika Antonova
Postdoctoral Scholar
Stanford University

Enabling Self-sufficient Robot Learning

3305 Newell-Simon Hall

Abstract:  Autonomous exploration and data-efficient learning are important ingredients for helping machine learning handle the complexity and variety of real-world interactions. In this talk, I will describe methods that provide these ingredients and serve as building blocks for enabling self-sufficient robot learning. First, I will outline a family of methods that facilitate active global exploration. [...]

PhD Thesis Proposal
Extern
Robotics Institute,
Carnegie Mellon University

Adaptive Robotic Assistance through Observations of Human Behavior

GHC 6501

Abstract: Assistive robots should take actions that support people's goals. This is especially true as robots enter into environments where personal agency is paramount, such as a person's home. Home environments have a wide variety of "optimal' solutions that depend on personal preference, making it difficult for a robot to know the goal it should [...]

Special Talk
Assistant Professor
Robotics Institute,
Carnegie Mellon University

Perceiving Objects and Interactions in 3D

Newell-Simon Hall 4305

Abstract: We observe and interact with myriad of objects in our everyday lives, from cups and bottles to hammers and tennis rackets. In this talk, I will outline our group’s efforts towards understanding these objects and our everyday interactions with them in 3D. I will first focus on scaling 3D prediction for isolated objects across [...]

RI Seminar
David Fouhey
Assistant Professor
University of Michigan

Understanding the Physical World from Images

1305 Newell Simon Hall

If I show you a photo of a place you have never been to, you can easily imagine what you could do in that picture. Your understanding goes from the surfaces you see to the ones you know are there but cannot see, and can even include reasoning about how interaction would change the scene. [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Beyond Pick-and-Place: Towards Dynamic and Contact-rich Motor Skills with Reinforcement Learning

NSH 3305

Abstract: Interactions with the physical world are at the core of robotics. However, robotics research, especially in manipulation, has been mainly focused on tasks with limited interactions with the physical world such as pick-and-place or pushing objects on the table top. These interactions are often quasi-static, have predefined or limited sequence of contact events and [...]

VASC Seminar
Vasudevan (Vasu) Sundarababu
SVP & Head of Digital Engineering
Centific

How Computer Vision Helps – from Research to Scale

3305 Newell-Simon Hall

Abstract:  Vasudevan (Vasu) Sundarababu, SVP and Head of Digital Engineering, will cover the topic: ‘How Computer Vision Helps – from Research to Scale’. During his time, Vasu will explore how Computer Vision technology can be leveraged in-market today, the key projects he is currently leading that leverage CV, and the end-to-end lifecycle of a CV initiative - [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Adaptive-Anytime Planning and Mapping for Multi-Robot Exploration in Large Environments

GHC 6501

Abstract: Robotic systems are being leveraged to explore environments too hazardous for humans to enter. Robot sensing, compute, and kinodynamic (SCK) capabilities are inextricably tied to the size, weight, and power (SWaP) constraints of the vehicle. When designing a robot team for exploration, the diversity and types of robots used must be carefully considered because [...]

PhD Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Neural Radiance Fields with LiDAR Maps

Abstract: Maps, as our prior understanding of the environment, play an essential role for many modern robotic applications. The design of maps, in fact, is a non-trivial art of balance between storage and richness. In this thesis, we explored map compression for image-to-LiDAR registration, LiDAR-to-LiDAR map registration, and image-to-SfM map registration, and finally, inspired by [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Enabling Data-Efficient Real-World Model-Based Manipulation by Estimating Preconditions for Inaccurate Models

NSH 3305

Abstract: This thesis explores estimating and reasoning about model deviation in robot learning for manipulation to improve data efficiency and reliability to enable real-robot manipulation in a world where models are inaccurate but still useful. Existing strategies are presented for improving planning robustness with low amounts of real-world data by an empirically estimated model precondition to guide [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Robust Adaptive Reinforcement Learning for Safety Critical Applications via Curricular Learning

GHC 6115

Abstract: Reinforcement Learning (RL) presents great promises for autonomous agents. However, when using robots in a safety critical domain, a system has to be robust enough to be deployed in real life. For example, the robot should be able to perform across different scenarios it will encounter. The robot should avoid entering undesirable and irreversible [...]

VASC Seminar
Rachel McDonnell
Associate Professor
Creative Technologies, Trinity College Dublin, Ireland

Motion Matters in the Metaverse

3305 Newell-Simon Hall

Abstract:  Abstract: In the early 1970s, Psychologists investigated biological motion perception by attaching point-lights to the joints of the human body, known as ‘point light walkers’. These early experiments showed biological motion perception to be an extreme example of sophisticated pattern analysis in the brain, capable of easily differentiating human motions with reduced motion cues. Further [...]

MSR Thesis Proposal
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Yichen Li

NSH 3305

Title: Simulation-guided Design for Vision-based Tactile Sensing on a Soft Robot Finger Abstract: Soft pneumatic robot manipulators have garnered widespread interest due to their compliance and flexibility, which enable soft, non-destructive grasping and strong adaptability to complex working environments. Tactile sensing is crucial for these manipulators to provide real-time contact information for control and manipulation. [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Controllable Visual-Tactile Synthesis

GHC 6501

Abstract: Deep generative models have various content creation applications such as graphic design, e-commerce, and virtual Try-on. However, current works mainly focus on synthesizing realistic visual outputs, often ignoring other sensory modalities, such as touch, which limits physical interaction with users. The main challenges for multi-modal synthesis lie in the significant scale discrepancy between vision [...]

Faculty Events
Associate Professor
Robotics Institute,
Carnegie Mellon University

Geometry Processing and Differential Geometry

Newell-Simon Hall 3305

Abstract: Basic representations for three-dimensional geometry have a profound effect on what can be achieved downstream, in a variety of disciplines (physical simulation, computational design, geometric learning, etc.).  In this talk I will discuss recent efforts in our group to revisit fundamental choices in the way we represent digital geometry, and solve geometric equations.  The guiding [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Perceiving Particles Inside a Container using Dynamic Touch Sensing

GHC 6501

Abstract: Dynamic touch sensing has shown potential for multiple tasks. In this talk, I will present how we utilize dynamic touch sensing to perceive particles inside a container with two tasks: classification of the particles inside a container and property estimation of the particles inside a container. First, we try to recognize what is inside [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Towards Photorealistic Dynamic Capture and Animation of Human Hair and Head

GHC 4405

Abstract: Realistic human avatars play a key role in immersive virtual telepresence. To reach a high level of realism, a human avatar needs to faithfully reflect human appearance. A human avatar should also be drivable and express natural motions. Existing works have made significant progress on building drivable realistic face avatars, but they rarely include [...]

VASC Seminar
Anand Bhattad
PhD candidate
University of Illinois Urbana-Champaign

What do generative models know about geometry and illumination?

3305 Newell-Simon Hall

Abstract: Generative models can produce compelling pictures of realistic scenes. Objects are in sensible places, surfaces have rich textures, illumination effects appear accurate, and the models are controllable. These models, such as StyleGAN, can also generate semantically meaningful edits of scenes by modifying internal parameters. But do these models manipulate a purely abstract representation of the [...]

Seminar
Matthew Johnson-Roberson, Zeynep Temel, Kris Kitani, Deva Ramanan, Henny Admoni
Singleton Room, Roberts Engineering Hall
Robotics Institute, Carnegie Mellon University

Life as a Professor Seminar

Singleton Room, Roberts Engineering Hall

Have you ever wondered what life is like as a professor? What do professors do on a daily basis? What makes the faculty career challenging and rewarding? Maybe you have even thought about becoming a faculty member yourself? Join us on March 22nd from 2:00 - 3:30 PM, where a panel of CMU faculty will [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

System Identification and Control of Multiagent Systems Through Interactions

NSH 3305

Abstract: This thesis investigates the problem of inferring the underlying dynamic model of individual agents of a multiagent system (MAS) and using these models to shape the MAS's behavior using robots extrinsic to the MAS. We investigate (a) how an observer can infer the latent task and inter-agent interaction constraints from the agents' motion and [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Examining the Role of Adaptation in Human-Robot Collaboration

GHC 4405

Abstract: Human and AI partners increasingly need to work together to perform tasks as a team. In order to act effectively as teammates, collaborative AI should reason about how their behaviors interplay with the strategies and skills of human team members as they coordinate on achieving joint goals. This talk will discuss a formalism for [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

A Multi-view Synthetic and Real-world Human Activity Recognition Dataset

NSH 3305

Abstract: Advancements in Human Activity Recognition (HAR) partially relies on the creation of datasets that cover a broad range of activities under various conditions. Unfortunately, obtaining and labeling datasets containing human activity is complex, laborious, and costly. One way to mitigate these difficulties with sufficient generality to provide robust activity recognition on unseen data is [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

RI Seminar
Lerrel Pinto
Assistant Professor of Computer Science
Robotics and Machine Learning, New York University

A Constructivist’s Guide to Robot Learning

1305 Newell Simon Hall

Over the last decade, a variety of paradigms have sought to teach robots complex and dexterous behaviors in real-world environments. On one end of the spectrum we have nativist approaches that bake in fundamental human knowledge through physics models, simulators and knowledge graphs. While on the other end of the spectrum we have tabula-rasa approaches [...]

VASC Seminar
Saurabh Gupta
Assistant Professor
University of Illinois Urbana-Champaign

Robot Learning by Understanding Egocentric Videos

GHC 8102

Abstract: True gains of machine learning in AI sub-fields such as computer vision and natural language processing have come about from the use of large-scale diverse datasets for learning. In this talk, I will discuss if and how we can leverage large-scale diverse data in the form of egocentric videos (first-person videos of humans conducting [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Eye Gaze for Intelligent Driving

NSH 4305

Abstract: Intelligent vehicles have been proposed as one path to increasing vehicular safety and reduce on-road crashes. Driving intelligence has taken many forms, ranging from simple blind spot occupancy or forward collision warnings to lane keeping and all the way to full driving autonomy in certain situations. Primarily, these methods are outward-facing and operate on [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Dense 3D Representation Learning for Geometric Reasoning in Manipulation Tasks

NSH 3001

Abstract: When solving a manipulation task like "put away the groceries" in real environments, robots must understand what *can* happen in these environments, as well as what *should* happen in order to accomplish the task. This knowledge can enable downstream robot policies to directly reason about which actions they should execute, and rule out behaviors [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

RI Seminar
Luca Carlone
Leonardo Career Development Associate Professor
Department of Aeronautics and Astronautics, Massachusetts Institute of Technology

Next-Generation Robot Perception: Hierarchical Representations, Certifiable Algorithms, and Self-Supervised Learning

1305 Newell Simon Hall

Spatial perception —the robot’s ability to sense and understand the surrounding environment— is a key enabler for robot navigation, manipulation, and human-robot interaction. Recent advances in perception algorithms and systems have enabled robots to create large-scale geometric maps of unknown environments and detect objects of interest. Despite these advances, a large gap still separates robot [...]

Field Robotics Center Seminar
Dr. Larry Matthies
Technology Coordinator
Mars Exploration Program Office, Jet Propulsion Laboratory, California

Autonomous mobility in Mars exploration: recent achievements and future prospects

NSH 4305

Abstract: This talk will summarize key recent advances in autonomous surface and aerial mobility for Mars exploration, then discuss potential future missions and technology needs for Mars and other planetary bodies. Among recent advances, the Perseverance rover that is now operating on Mars includes new autonomous navigation capability that dramatically increases its traverse speed over [...]

PhD Thesis Proposal
Extern
Robotics Institute,
Carnegie Mellon University

Passive Coupling in Robot Swarms

NSH 4305

Abstract: In unstructured environments, ant colonies demonstrate remarkable abilities to adaptively form functional structures in response to various obstacles, such as stairs, gaps, and holes. Drawing inspiration from these creatures, robot swarms can collectively exhibit complex behaviors and achieve tasks that individual robots cannot accomplish. Existing modular robot platforms that employ dynamic coupling and decoupling [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

RI Seminar
Phillip Isola
Associate Professor
Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology

Structures and Environments for Generalist Agents

1305 Newell Simon Hall

Abstract: We are entering an era of highly general AI, enabled by supervised models of the Internet. However, it remains an open question how intelligence emerged in the first place, before there was an Internet to imitate. Understanding the emergence of skillful behavior, without expert data to imitate, has been a longstanding goal of reinforcement [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning novel objects during robot exploration via human-informed few-shot detection

NSH 1109

Abstract: Autonomous mobile robots exploring in unfamiliar environments often need to detect target objects during exploration. Most prevalent approach is to use conventional object detection models, by training the object detector on large abundant image-annotation dataset, with a fixed and predefined categories of objects, and in advance of robot deployment. However, it lacks the capability [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning to Perceive and Predict Everyday Interactions

GHC 6121

Abstract: This thesis aims to develop a computer vision system that can understand everyday human interactions with rich spatial information. Such systems can benefit VR/AR to perceive the reality and modify its virtual twin, and robotics to learn manipulation by watching human. Previous methods have been limited to constrained lab environment or pre-selected objects with [...]

Faculty Candidate
Systems Scientist
Robotics Institute,
Carnegie Mellon University

Faculty Candidate: Wenshan Wang

Newell-Simon Hall 4305

Title: Towards General Autonomy: Learning from Simulation, Interaction, and Demonstration Abstract: Today's autonomous systems are still brittle in challenging environments or rely on designers to anticipate all possible scenarios to respond appropriately. On the other hand, leveraging machine learning techniques, robot systems are trained in simulation or the real world for various tasks. Due to [...]

VASC Seminar
Angjoo Kanazawa
Assistant Professor of the Department of Electrical Engineering and Computer Science
, University of California at Berkeley

From Videos to 4D Worlds and Beyond

Newell-Simon Hall 3305

Abstract:  Abstract: The world underlying images and videos is 3-dimensional and dynamic, i.e. 4D, with people interacting with each other, objects, and the underlying scene. Even in videos of a static scene, there is always the camera moving about in the 4D world. Accurately recovering this information is essential for building systems that can reason [...]

MSR Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning Models and Cost Functions from Unlabeled Data for Off-Road Driving

GHC 8102

Abstract: Off-road driving is an important instance of navigation in unstructured environments, which is a key robotics problem with many applications, such as exploration, agriculture, disaster response and defense. The key challenge in off-road driving is to be able to take in high dimensional, multi-modal sensing data and use it to make intelligent decisions on [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Active Vision for Manipulation

GHC 4405

Abstract: Decades of research on computer vision has highlighted the importance of active sensing -- where the agent actively controls parameters of the sensor to improve perception. Research on active perception the context of robotic manipulation has demonstrated many novel and robust sensing strategies involving a multitude of sensors like RGB and RGBD cameras, a [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

RI Seminar
Vandi Verma
Deputy Manager
Mobility and Robotics Systems, NASA Jet Propulsion Laboratory

Mars Robots and Robotics at NASA JPL

Newell-Simon Hall 4305

Abstract: In this seminar I’ll discuss Mars robots, the unprecedented results we’re seeing with the latest Mars mission, and how we got here. Perseverance’s manipulation and sampling systems have collected samples from unique locations at twice the rate of any prior mission. 88% of all driving has been autonomous. This has enabled the mission to [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Continually Improving Robots

GHC 8102

Abstract: General purpose robots should be able to perform arbitrary manipulation tasks, and get better at performing new ones as they obtain more experience. The current paradigm in robot learning involves training a policy, in simulation or directly in the real world, with engineered rewards or demonstrations. However, for robots that need to keep learning [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Parallelized Search on Graphs with Expensive-to-Compute Edges

GHC 6115

Abstract: Search-based planning algorithms enable robots to come up with well-reasoned long-horizon plans to achieve a given task objective. They formulate the problem as a shortest path problem on a graph embedded in the state space of the domain. Much research has been dedicated to achieving greater planning speeds to enable robots to respond quickly [...]

VASC Seminar
Sergey Tulyakov
Principal Research Scientist
Snap Inc.

Generative and Animatable Radiance Fields

Newell-Simon Hall 3305

Abstract:  Generating and transforming content requires both creativity and skill. Creativity defines what is being created and why, while skill answers the question of how. While creativity is believed to be abundant, skill can often be a barrier to creativity. In our team, we aim to substantially reduce this barrier. Recent Generative AI methods have simplified the problem for 2D [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Chonghyuk Song

GHC 6501

Title: Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesis Abstract: We explore the task of embodied view synthesis from monocular videos of deformable scenes. Given a minute-long RGBD video of people interacting with their pets, we render the scene from novel camera trajectories derived from in-scene motion of actors: (1) egocentric cameras that simulate the point [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Design Iteration of Dexterous Compliant Robotic Manipulators

GHC 4405

Abstract: One goal of personal robotics is to have robots in homes performing everyday tasks efficiently to improve our quality of life. Towards this end, manipulators are needed which are low cost, safe around humans, and approach human-level dexterity. However, existing off-the-shelf manipulators are expensive both in cost and manufacturing time, difficult to repair, and [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Shivam Duggal

NSH 1109

Title: Learning Single Image 3D Reconstruction from Single-View Image Collections Abstract We present a framework for learning 3D object shapes and dense cross-object 3D correspondences from just an unaligned category-specific image collection. The 3D shapes are generated implicitly as deformations to a category-specific signed distance field and are learned in an unsupervised manner solely from unaligned [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Whisker Sensors for Unstructured Environments

NSH 3305

Abstract: As robot applications expand from controllable factory settings to unknown environments, the robots will need a larger breadth of sensors to perceive these complex environments. In this thesis, I focus on developing whisker sensors for robot perception. The inspiration for whisker sensors comes from the biological world, where whiskers serve as tactile and flow [...]

MSR Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Himangi Mittal

GHC 6115

Title: Audio-Visual State-Aware Representation Learning from Interaction-Rich Data Abstract In robotics and augmented reality, the input to the agent is a long stream of video from the first-person or egocentric point of view. Recently, there have been significant efforts to capture humans from their first-person/egocentric view interacting with their own environment as they go about [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Ken Liu

NSH 3305

Title: On Privacy and Personalization in Federated Learning: Analyses and Applications Abstract: Recent advances in machine learning often rely on large and centralized datasets. However, curating such data can be challenging when they hold private information, and policies/regulations may mandate that they remain distributed across data silos (e.g. mobile devices or hospitals). Federated learning (FL) [...]

MSR Thesis Defense
MSR Student / Teaching Assistant
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Haolun Zhang

NSH 3305

Title: Seeing in 3D: Towards Generalizable 3D Visual Representations for Robotic Manipulation Abstract: Despite the recent progress in computer vision and deep learning, robot perception remains a tremendous challenge due to the variations of the objects and the scenes in manipulation tasks. Ideally, a robot trying to manipulate a new object should be able to [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Muyang Li

NSH 4305

Title: Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models Abstract: During image editing, existing deep generative models tend to re-synthesize the entire output from scratch, including the unedited regions. This leads to a significant waste of computation, especially for minor editing operations. In this work, we present Spatially Sparse Inference (SSI), a general-purpose technique [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

3D-aware Conditional Image Synthesis

NSH 3002

Abstract: We propose pix2pix3D, a 3D-aware conditional generative model for controllable photorealistic image synthesis. Given a 2D label map, such as a segmentation or edge map, our model learns to synthesize a corresponding image from different viewpoints. To enable explicit 3D user control, we extend conditional generative models with neural radiance fields. Given widely-available posed [...]

Faculty Events
Associate Research Professor
Robotics Institute,
Carnegie Mellon University

Promoting Human Creativity with FRIDA: Framework and Robotics Initiative for Developing Arts

Newell-Simon Hall 4305

ABSTRACT: FRIDA, a reference to the vibrant painter Frida Kahlo, stands for a Framework and Robotics Initiative for Developing Arts to promote human creativity. FRIDA supports intuitive ways for people to collaboratively create artworks including natural language, images, and sounds. Because FRIDA is for real-world arts, our work is uniquely different from digital art tools [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Robotic Climbing for Extreme Terrain Exploration

WEH 4623

Abstract: Climbing robots can investigate scientifically valuable sites that are inaccessible to conventional rovers due to steep terrain features. Robots equipped with microspine grippers are particularly well-suited to ascending rocky cliff faces, but existing designs are either large and slow, or limited to relatively flat surfaces such as buildings. We have developed a novel free-climbing [...]

MSR Thesis Defense
MSR Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Rohan Zeng

NSH 3305

Title: Spectral Unmixing and Mapping of Coral Reef Benthic Cover Abstract: Coral reefs are important to the global ecosystem and the local communities and wildlife that rely on the habitat they create. However, coral reefs are also in critical and rapid decline: reefs have degraded over recent decades and what remains is at increasing risk [...]