Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Multi-Object Tracking in the Crowd

NSH 4305

Abstract: In this talk, I will focus on the problem of multi-object tracking in crowded scenes. Tracking within crowds is particularly challenging due to heavy occlusion and frequent crossover between tracking targets. The problem becomes more difficult when we only have noisy bounding boxes due to background and neighboring objects. Existing tracking methods try to [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Utilizing Panoptic Segmentation and a Locally-Conditioned Neural Representation to Build Richer 3D Maps

NSH 4305

Abstract: Advances in deep-learning based perception and maturation of volumetric RGB-D mapping algorithms have allowed autonomous robots to be deployed in increasingly complex environments. For robust operation in open-world conditions however, perceptual capabilities are still lacking. Limitations of commodity depth sensors mean that complex geometries and textures cannot be reconstructed accurately. Semantic understanding is still [...]

Faculty Events
Senior Commercialization Specialist
Robotics Institute,
Carnegie Mellon University

NREC Study Group & Recent Projects

Newell-Simon Hall 4305

This talk will describe the NREC study process that has been developed as a lower cost of entry work product for potential partners. This is a process that is available for anyone on campus that wants to help their sponsors create viable system concepts and potential development costs before committing to a full development program. [...]

RI Seminar
Byron Boots
Amazon Professor
Machine Learning in the Paul G. Allen School of Computer Science, University of Washington

Machine Learning and Model Predictive Control for Adaptive Robotic Systems

1305 Newell Simon Hall

Abstract: In this talk I will discuss several different ways in which ideas from machine learning and model predictive control (MPC) can be combined to build intelligent, adaptive robotic systems. I’ll begin by showing how to learn models for MPC that perform well on a given control task. Next, I’ll introduce an online learning perspective on [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Magnification-invariant retinal distance estimation using a laser aiming beam

NSH 1109

Abstract: Retinal surgery procedures like epiretinal membrane peeling and retinal vein cannulation require surgeons to manipulate very delicate structures in the eye with little room for error. Many robotic surgery systems have been developed to help surgeons and enforce safeguards during these demanding procedures. One essential piece of information that is required to create and [...]

Field Robotics Center Seminar
José Luís Silva
Assistant Professor
Science and Technology Department, University Institute of Lisbon

Towards more effective remote execution of exploration operations using multimodal interfaces

1305 Newell Simon Hall

Abstract: Remote robots enable humans to explore and interact with environments while keeping them safe from existing harsh conditions (e.g., in search and rescue, deep sea or planetary exploration scenarios). However, the gap between the control station and the remote robot presents several challenges (e.g., situation awareness, cognitive load, perception, latency) for effective teleoperation. Multimodal [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Bridging Humans and Generative Models

NSH 4305

Abstract: Deep generative models make visual content creation more accessible to novice and professional users alike by automating the synthesis of diverse, realistic content based on a collected dataset. People often use generative models as data-driven sources, making it challenging to personalize a model easily. Currently, personalizing a model requires careful data curation, which is [...]

VASC Seminar
Andrew Owens
Assistant Professor
Electrical Engineering & Computer Science , University of Michigan

Learning Visual, Audio, and Cross-Modal Correspondences

Newell-Simon Hall 3305

Abstract:  Today's machine perception systems rely heavily on supervision provided by humans, such as labels and natural language. I will talk about our efforts to make systems that, instead, learn from two ubiquitous sources of unlabeled data: visual motion and cross-modal sensory associations. I will begin by discussing our work on creating unified models for [...]

PhD Speaking Qualifier
MSR Student
Robotics Institute,
Carnegie Mellon University

Impulse considerations for reasoning about intermittent contacts

NSH 4305

Abstract: Many of our interactions with the environment involve making and breaking contacts. However, it is not always obvious how one should reason about these intermittent contacts (sequence, timings, locations) in an online and adaptive way. This is particularly relevant in gait generation for legged locomotion control, where it is standard to simply predefine and [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Multi-Human 3D Reconstruction from Monocular RGB Videos

NSH 3305

Abstract: We study the problem of multi-human 3D reconstruction from RGB videos captured in the wild. Humans have dynamic motion, and reconstructing them in arbitrary settings is key to building immersive social telepresence, assistive humanoid robots, and augmented reality systems. However, creating such a system requires addressing fundamental issues with previous works regarding the data [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning and Translating Temporal Abstractions across Humans and Robots

NSH 3305

Abstract: Humans possess a remarkable ability to learn to perform tasks from a variety of different sources-from language, instructions, demonstration, etc. In each case, they are able to easily extract the high-level strategy to solve the task, such as the recipe of cooking a dish, whilst ignoring irrelevant details, such as the precise shape of [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Robust Incremental Smoothing and Mapping

NSH 3001

Abstract: In this work we present a method for robust optimization for online incremental Simultaneous Localization and Mapping (SLAM). Due to the NP-Hardness of data association in the presence of perceptual aliasing, tractable (approximate) approaches to data association will produce erroneous measurements. We require SLAM back-ends that can converge to accurate solutions in the presence [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

3D Reconstruction using Differential Imaging

GHC 4405

Abstract: 3D reconstruction has been at the core of many computer vision applications, including autonomous driving, visual inspection in manufacturing, and augmented and virtual reality (AR/VR). Because monocular 3D sensing is fundamentally ill-posed, many techniques aiming for accurate reconstruction use multiple captures to solve the inverse problem. Depending on the amount of change in these [...]

PhD Thesis Defense
Jacky Liang
PhD Student
Robotics Institute, Carnegie Mellon University

Learning with Structured Priors for Robust Robot Manipulation

NSH 4305

Abstract: Robust and generalizable robots that can autonomously manipulate objects in semi-structured environments can bring material benefits to society. Data-driven learning approaches are crucial for enabling such systems by identifying and exploiting patterns in semi-structured environments, allowing robots to adapt to novel scenarios with minimal human supervision. However, despite significant prior work in learning for [...]

MSR Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning Parameter-Efficient Quadrotor Dynamics Models

NSH 4305

Abstract: Operation of quadrotors through high-speed, high-acceleration maneuvers remains a challenging problem due to the complex aerodynamics in this regime. While standard physical models suffice for control in near-hover conditions, the primary challenge in executing aggressive trajectories is obtaining a model for the quadrotor dynamics that adequately models the aerodynamic effects present, including lift, drag, [...]

Faculty Events

RI Faculty Business Meeting

Newell-Simon Hall 4305

Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Self-Supervising Occlusions For Vision

GHC 4405

Abstract: Virtually every scene has occlusions. Even a scene with a single object exhibits self-occlusions - a camera can only view one side of an object (left or right, front or back), or part of the object is outside the field of view. More complex occlusions occur when one or more objects block part(s) of [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Predicting The Future and Linking the Past: Learning and Constructing Structured Models for Robotic Manipulation

GHC 4405

Abstract: Intelligent robotic agents need to reason about the dynamics of their surrounding world, and use such dynamics reasoning to make future predictions for efficient task planning. In addition, it is also desirable for robots to associate past experience in their memories to their current observation, and conduct analogical reasoning to complete tasks at their [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Tushar Kusnur

NSH 4305

Title: Search-based Planning for Sensor-based Coverage Abstract: Robots are excellent candidates for the dull, dirty, and dangerous jobs we do not want humans to perform. Today, these include inspection of large areas or structures, post-disaster assessment, and surveillance. Assessing the aftermath of the recent Fern Hollow bridge collapse in Pittsburgh is one such example. Many [...]

MSR Thesis Defense
PhD Student
Robotics Institute,
Carnegie Mellon University

Human-in-the-loop Model Creation

GHC 7101

Abstract: Deep generative models make visual content creation more accessible to novice users by automating the synthesis of diverse, realistic content based on a collected dataset. However, the current machine learning approaches miss several elements of the creative process -- the ability to synthesize things that go far beyond the data distribution and everyday experience, [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Robotic Interestingness via Human-Informed Few-Shot Object Detection

NSH 1109

Abstract: Interestingness recognition is crucial for decision making in autonomous exploration for mobile robots. Previous methods proposed an unsupervised online learning approach that can adapt to environments and detect interesting scenes quickly, but lack the ability to adapt to human-informed interesting objects. To solve this problem, we introduce a human-interactive framework, AirInteraction, that can detect [...]

VASC Seminar
Lachlan MacDonald
Postdoc
Australian Institute for Machine Learning, University of Adelaide

Towards a formal theory of deep optimisation

Newell-Simon Hall 3305

Abstract:  Precise understanding of the training of deep neural networks is largely restricted to architectures such as MLPs and cost functions such as the square cost, which is insufficient to cover many practical settings.  In this talk, I will argue for the necessity of a formal theory of deep optimisation.  I will describe such a [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Nikhil Angad Bakshi

NSH 4305

Title: See But Don't Be Seen: Towards Stealthy Active Search in Heterogeneous Multi-Robot Systems Abstract: Robotic solutions for quick disaster response are essential to ensure minimal loss of life, especially when the search area is too dangerous or too vast for human rescuers. We model this problem as an asynchronous multi-agent active-search task where each robot aims [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Yves Georgy Daoud

NSH 4305

Title: Spatial Tasking in Human-Robot Collaborative Exploration Abstract: This work develops a methodology for collaborative human-robot exploration that leverages implicit coordination. Most autonomous single- and multi-robot exploration systems require a remote operator to provide explicit guidance to the robot team. Few works consider how to integrate the human partner alongside robots to provide guidance in the [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Ambareesh Revanur

Gates Hillman Center 4405

Title: Towards Video-based Physiology Estimation Abstract: RGB-video based human physiology estimation has a wide range of practical applications in telehealth, sports and deep fake detection. Therefore, researchers in the community have collected several video datasets and have advanced new methods over the years. In this dissertation, we study these methods extensively and aim to address the [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Raghavv Goel

Wean Hall 2302

Title: Automating Ultrasound Based Vascular Access Abstract: Timely care of trauma patients is important to prevent casualties in resource-limited regions such as the battlefield. In order to treat such trauma using point of care diagnosis, medical practitioners typically use an ultrasound for vascular access or detection of subcutaneous splinters for providing critical care. The problem here is two-fold: [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Mayank Singh

3305 Newell-Simon Hall

Title: Analogical Networks: Memory-Modulated In-Context 3D Parsing Abstract: Recent advances in the applications of deep neural networks to numerous visual perception tasks have shown excellent performance. However, this generally requires access to large amount of training samples and hence one persistent challenge is the setting of few-shot learning. In most existing works, a separate parametric neural [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Learning with Diverse Forms of Imperfect and Indirect Supervision

NSH 4305

Abstract: Powerful Machine Learning (ML) models trained on large, annotated datasets have driven impressive advances in fields including natural language processing and computer vision. In turn, such developments have led to impactful applications of ML in areas such as healthcare, e-commerce, and predictive maintenance. However, obtaining annotated datasets at the scale required for training high [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Yutian Lei

Title: ARC: AdveRsarial Calibration between Modalities Abstract: Advances in computer vision and machine learning techniques have led to flourishing success in RGB-input perception tasks, which has also opened unbounded possibilities for non-RGB-input perception tasks, such as object detection from wireless signals, point clouds, and infrared light. However, compared to the matured development pipeline of RGB-input [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

FRIDA: Supporting Artistic Communication in Real-World Image Synthesis Through Diverse Input Modalities

NSH 4305

Abstract: FRIDA, a Framework and Robotics Initiative for Developing Arts, is a robot painting system designed to translate an artist's high-level intentions into real world paintings. FRIDA can paint from combinations of input images, text, style examples, sounds, and sketches. Planning is performed in a differentiable, simulated environment created using real data from the robot [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Perception for High-Speed Off-Road Driving

GHC 4405

Abstract: On-road autonomous driving has seen rapid progress in recent years with driverless vehicles being tested in various cities worldwide. However, this progress is limited to cities with well-established infrastructure and has yet to transfer to off-road regimes with unstructured environments and few paved roads. Advances in high-speed and reliable autonomous off-road driving can unlock [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Continual Learning of Compositional Skills for Robust Robot Manipulation

NSH 4305

Abstract: Real world robots need to continuously learn new manipulation tasks in a lifelong learning manner. These new tasks often share sub-structures (in the form of sub-tasks, controllers) with previously learned tasks. To utilize these shared sub-structures, we explore a compositional and object-centric approach to learn manipulation tasks. While compositionality in robot manipulation can manifest [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Samuel Ong

Title: Data-Driven Slip Model for Improved Localization and Path Following applied to Lunar Micro-Rovers Abstract Micro-lunar rovers need to solve a slew of challenges on the Moon, with no human intervention. One such challenge is the need to know their location in order to navigate and build maps. However, localization is challenging on the moon due [...]

PhD Thesis Defense
Postdoctoral Fellow
Robotics Institute,
Carnegie Mellon University

Computational Interferometric Imaging

NSH 4305

Abstract: Imaging systems typically accumulate photons that, as they travel from a light source to a camera, follow multiple different paths and interact with several scene objects. This multi-path accumulation process confounds the information that is available in captured images about the scene and makes using these images to infer properties of scene objects, such [...]

Special Talk
Senior Systems Scientist
Robotics Institute,
Carnegie Mellon University

Making AI trustworthy and understandable by clinicians

Newell-Simon Hall 4305

Abstract:  Understandable-AI techniques facilitate to use of AI as a tool by human experts, giving humans insight into how AI decisions are made thereby helping experts discern which AI predictions should or shouldn’t be trusted.  Understandable techniques may be especially useful for applications with insufficient validation data for regulatory approval, for which human experts must remain the final decision [...]

VASC Seminar
Christoph Lassner
Senior Research Scientist
Epic Games

Towards Interactive Radiance Fields

Newell-Simon Hall 3305

Abstract:  Over the last years, the fields of computer vision and computer graphics have increasingly converged. Using the exact same processes to model appearance during 3D reconstruction and rendering has shown tremendous benefits, especially when combined with machine learning techniques to model otherwise hard-to-capture or -simulate optical effects. In this talk, I will give an [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Robust and Context-Aware Real-Time Collaborative Robot Handling with Dynamic Gesture Commands

GHC 6501

Abstract: Real-time collaborative robot (cobot) handling is a task where the cobot maneuvers an object under human dynamic gesture commands. Enabling dynamic gesture commands is useful when the human needs to avoid direct contact with the robot or the object handled by the robot. However, the key challenge lies in the heterogeneity in human behaviors [...]