Understanding the Placenta: Towards an Objective Pregnancy Screening
Abstract: My research focusses on the development of a pregnancy screening tool, that will be: (i) system and user-independent; and (ii) provides a quantifi able measure of placental health. With this end, I am working towards the design of a multiparametric quantitative ultrasound (QUS) based placental tissue characterization method. The method would potentially identify the [...]
Human-Robot Interactive Collaboration & Communication
Abstract: Autonomous and anthropomorphic robots are poised to play a critical role in manufacturing, healthcare and the services industry in the near future. However, for this vision to become a reality, robots need to efficiently communicate and interact with their human partners. Rather than traditional remote controls and programming languages, adaptive and transparent techniques for [...]
Interleaving Graph Search and Trajectory Optimization for Aggressive Quadrotor Flight
Abstract: Quadrotors can achieve aggressive flight by tracking complex maneuvers and rapidly changing directions. Planning for aggressive flight with trajectory optimization could be incredibly fast, even in higher dimensions, and can account for dynamics of the quadrotor, however, only provides a locally optimal solution. On the other hand, planning with discrete graph search can handle [...]
See, Hear, Explore: Curiosity via Audio-Visual Association
Abstract: Exploration is one of the core challenges in reinforcement learning. A common formulation of curiosity-driven exploration uses the difference between the real future and the future predicted by a learned model. However, predicting the future is an inherently difficult task which can be ill-posed in the face of stochasticity. In this work, we introduce [...]
Carnegie Mellon University
Robots “R” Us: 25 years of Robotics Technology Development and Commercialization at NREC
Abstract: Since its founding in 1979, the Robotics Institute (RI) at Carnegie Mellon University has been leading the world in robotics research and education. In the mid 1990s, RI created NREC as the applied R&D center within the Institute with a specific mission to apply robotics technology in an impactful way on real-world applications. In this talk, I will go over [...]
Relational Reasoning for Multi-Agent Systems
Abstract: Multi-agent interacting systems are prevalent in the world, from purely physical systems to complicated social dynamics systems. The interactions between entities / components can give rise to very complex behavior patterns at the level of both individuals and the whole system. In many real-world multi-agent interacting systems (e.g., traffic participants, mobile robots, sports players), [...]
Carnegie Mellon University
Dynamical Model Learning and Inversion for Aggressive Quadrotor Flight
Quadrotor applications have seen a surge recently and many tasks require precise and accurate controls. Flying fast is critical in many applications and the limited onboard power source makes completing tasks quickly even more important. Staying on a desired course while traveling at high speeds and high accelerations is difficult due to complex and stochastic [...]
MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video
Abstract: We present a method to capture temporally coherent dynamic clothing deformation from a monocular RGB video input. In contrast to the existing literature, our method does not require a pre-scanned personalized mesh template, and thus can be applied to in-the-wild videos. To constrain the output to a valid deformation space, we build statistical deformation [...]
Towards an Intelligence Architecture for Human-Robot Teaming
Abstract: Advances in autonomy are enabling intelligent robotic systems to enter human-centric environments like factories, homes and workplaces. To be effective as a teammate, we expect robots to accomplish more than performing simplistic repetitive tasks; they must perceive, reason, perform semantic tasks in a human-like way. A robot's ability to act intelligently is fundamentally tied [...]
Carnegie Mellon University
Robust Manipulation with Active Compliance
Abstract: Human manipulation skills are filled with creative use of physical contacts to move the object about the hand and in the environment. However, it is difficult for robot manipulators to enjoy this dexterity since contacts may cause the manipulation task to fail by introducing huge forces or unexpected change of constraints, especially when modeling [...]
Carnegie Mellon University
Open-world Object Detection and Tracking
Abstract: Computer vision today excels at recognizing narrow slices of the real world: our models seem to accurately detect objects like cats, cars, or chairs in benchmark datasets. However, deploying models requires that they work in the open world, which includes arbitrary objects in diverse settings. Current methods struggle on both axes: they recognize only [...]
Self-supervised learning for visual recognition
Abstract: We are interested in learning visual representations that are discriminative for semantic image understanding tasks such as object classification, detection, and segmentation in images/videos. A common approach to obtain such features is to use supervised learning. However, this requires manual annotation of images, which is costly, ambiguous, and prone to errors. In contrast, self-supervised [...]
Policy Decomposition : Approximate Optimal Control with Suboptimality Estimates
Abstract: Owing to the curse of dimensionality, numerically computing global policies to optimal control problems for complex dynamical systems quickly becomes intractable. In consequence, a number of approximation methods have been developed. However, none of the current methods can quantify by how much the resulting control underperforms the elusive globally optimal solution. We propose Policy [...]
Inverse Reinforcement Learning with Explicit Policy Estimates
Abstract: Various methods for solving the inverse reinforcement learning (IRL) problem have been developed independently in machine learning and economics. In particular, the method of Maximum Causal Entropy IRL is based on the perspective of entropy maximization, while related advances in the field of economics instead assume the existence of unobserved action shocks to explain [...]
GANs for Everyone
Abstract: The power and promise of deep generative models such as StyleGAN, CycleGAN, and GauGAN lie in their ability to synthesize endless realistic, diverse, and novel content with user controls. Unfortunately, the creation and deployment of these large-scale models demand high-performance computing platforms, large-scale annotated datasets, and sophisticated knowledge of deep learning methods. This makes [...]
Carnegie Mellon University
MSR Thesis Talk – Hans Kumar
Title: Multi-Session Periodic SLAM for Legged Robots Abstract: Methods for state estimation that rely on visual information are challenging on dynamic robots because of rapid changes in the viewing angle of onboard cameras. In this thesis, we show that by leveraging structure in the way that dynamic robots locomote, we can increase the feasibility [...]
Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation
Abstract: To perform manipulation tasks in the real world, robots need to operate on objects with various shapes, sizes and without access to geometric models. It is often infeasible to train monolithic neural network policies across such large variance in object properties. Towards this generalization challenge, we propose task-axis controllers, which are defined relative to [...]
Reasoning over Text in Images for VQA and Captioning
Abstract: Text in images carries essential information for multimodal reasoning, such as VQA or image captioning. To enable machines to perceive and understand scene text and reason jointly with other modalities, 1) we collect the TextCaps dataset, which requires models to read and reason over text and visual content in the image to generate image [...]
Carnegie Mellon University
MSR Thesis Talk: Eagle Dapeng Zhao
Title: Predicting Human Trajectories by Learning and Matching Patterns Zoom Link: https://cmu.zoom.us/j/93356993095?pwd=Nzd3a09PbG9mVkV5blFVaU5nRk1GQT09 Abstract: As more and more robots are envisioned to cooperate with humans sharing the same space, it is desired for robots to be able to predict others' trajectories to navigate in a safe and self-explanatory way. We propose a Convolutional Neural Network-based approach [...]
Design and control of insect-scale bees and dog-scale quadrupeds
Abstract: Enhanced robot autonomy---whether it be in the context of extended tether-free flight of a 100mg insect-scale flapping-wing micro aerial vehicle (FWMAV), or long inspection routes for a quadrupedal robot---is hindered by fundamental constraints in power and computation. With this motivation, I will discuss a few projects I have worked on to circumvent these issues in [...]
Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies
Abstract: Real-world environments, such as homes, hospitals, and restaurants, often contain many objects that a robot could possibly manipulate. However, for a given manipulation task, only a small number of objects and object properties may actually be relevant. This talk presents CREST (Causal Reasoning for Efficient Structure Transfer), our approach to learn the relevant state [...]
Grasping Transparent, Specular, and Deformable Objects
Abstract: A large body of research exists on grasping for objects with ideal properties like Lambertian reflectance and rigidity. On the other hand, real-world environments contain many objects for which such properties do not hold, such as transparent, specular, and deformable objects. For such objects, new approaches are required to achieve the same level of [...]
Carnegie Mellon University
Auto-generated Manipulation Primitives
Abstract: The central theme in robotic manipulation is that of the robot interacting with the world through physical contact. We tend to describe that physical contact using specific words that capture the nature of the contact and the action, such as grasp, roll, pivot, push, pull, tilt, close, open etc. We refer to these situation-specific [...]
Carnegie Mellon University
Learning 3D Registration and Reconstruction from the Visual World
Abstract: Humans learn to develop strong senses for 3D geometry by looking around in the visual world. Through pure visual perception, not only can we recover a mental 3D representation of what we are looking at, but meanwhile we can also recognize where we are looking at the scene from. Finding the 3D scene representation [...]
Point Cloud Registration with or without Learning
Abstract: I will be presenting two of our recent works on 3D point cloud registration: A scene flow method for non-rigid registration: I will discuss our current method to recover scene flow from point clouds. Scene flow is the three-dimensional (3D) motion field of a scene, and it provides information about the spatial arrangement [...]
Carnegie Mellon University
Development of an Agile and Dexterous Balancing Mobile Manipulator Robot
Abstract: The proposed thesis work focuses on the design and control of a new unique agile and dexterous mobile manipulator, the Carnegie Mellon University (CMU) ballbot. The CMU ballbot is a human-sized dynamically stable mobile robot that balances on a single ball. We present the development and integration of a new pair of seven-degree-of-freedom (7-DOF) [...]
Carnegie Mellon University
MSR Thesis Talk: Shengcao Cao
Title: Efficient Model Performance Estimation via Feature Histories Abstract: An important step in the task of neural network design, such as hyper-parameter optimization (HPO) or neural architecture search (NAS), is the evaluation of a candidate model's performance. Given fixed computational resources, one can either invest more time training each model to obtain more accurate estimates [...]
Dynamical Robots via Origami-Inspired Design
Abstract: Origami-inspired engineering produces structures with high strength-to-weight ratios and simultaneously lower manufacturing complexity. This reliable, customizable, cheap fabrication and component assembly technology is ideal for robotics applications in remote, rapid deployment scenarios that require platforms to be quickly produced, reconfigured, and deployed. Unfortunately, most examples of folded robots are appropriate only for small-scale, low-load [...]
Carnegie Mellon University
MSR Thesis Talk: Steven Lee
Title: Learning to Represent and Accurately Arrange Food Items Abstract: Arrangements of objects are commonplace in a myriad of everyday scenarios, such as decorations at one’s home, displays at museums, and plates of food at restaurants. An efficient personal robot should be able to learn how to robustly recreate an arrangement using only a few [...]
Carnegie Mellon University
MSR Thesis Talk: Amrita Sawhney
Title: Learning to Perceive and Manipulate Diverse Food Materials Through Interaction Abstract: The home kitchen environment presents many challenges for an autonomous cooking robot, such as the deformability of food items, the wide range of material properties of food, and the complex interaction dynamics involved in food manipulation tasks. Material properties are important when interacting [...]
Carnegie Mellon University
MSR Thesis Talk: Haidar Jamal
Title: Localization for Lunar Micro-Rovers Abstract: This talk presents an avionics and localization system that enables a lunar micro-rover to navigate autonomously. This system is important for the latest class of small, low-powered, and fast robots going to the Moon in search of polar ice. The first component of the system is an Extended [...]
PoseIt: A Visual-Tactile Dataset of Holding Poses for Grasp Stability Analysis
Abstract: When humans grasp objects in the real world, we often move our arm to hold the object in a different pose where we can use it. In contrast, typical lab settings only study the stability of the grasp immediately after lifting, without any subsequent re-positioning of the arm. However, an object’s stability could vary [...]
Planning to Minimize Human and Robot Efforts Over Tasks
Abstract: It is not feasible to pre-program robots a priori for every possible task they may encounter in unstructured domains. Upon encountering a task that a robot can't solve, one common strategy is to teach it new skills via demonstrations. However, demonstrating a task can often be more cumbersome than performing the task directly. This [...]
MSR Thesis Talk: Akash Sharma
Title: Incorporating Semantic Structure in SLAM Abstract: For robots to understand the environment they interact with, a combination of geometric information and semantic information is imperative. In this talk, I propose a fast and scalable Simultaneous Localization and Mapping (SLAM) system that represents indoor scenes as a graph of semantic objects. Leveraging the observation that [...]
Propelling Robot Manipulation of Unknown Objects using Learned Object Centric Models
Abstract: There is a growing interest in using data-driven methods to scale up manipulation capabilities of robots for handling a large variety of objects. Many of these methods are oblivious to the notion of objects and they learn monolithic policies from the whole scene in image space. As a result, they don’t generalize well to [...]
Carnegie Mellon University
Yaadhav Raaj MSR Thesis Talk
Title: Exploiting Uncertainty in Triangulation Light Curtains for Object Tracking and Depth Estimation Abstract: Active sensing through the use of Adaptive Depth Sensors is a nascent field, with potential in areas such as Advanced driver-assistance systems (ADAS). One such class of sensor is the Triangulation Light Curtain, which was developed in the Illumination and Imaging [...]
Carnegie Mellon University
Active Vision: Autonomous Aerial Cinematography with Learned Artistic Decision-Making
Abstract: Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. Fundamentally, it is a tool with immense potential to improve human creativity, expressiveness, and sharing of experiences. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often [...]
Fine-Tuning Offline Reinforcement Learning with Model-Based Policy Optimization
Abstract: In offline reinforcement learning (RL), we attempt to learn a control policy from a fixed dataset of environment interactions. This setting has the potential benefit of allowing us to learn effective policies without needing to collect additional interactive data, which can be expensive or dangerous in real-world systems. However, traditional off-policy RL methods tend [...]
MSR Thesis Talk: Zhipeng Bao
Title: Introducing Generative Models to Facilitate Multi-Task Visual Learning Abstract: Motivated by multi-task learning of shared feature representations, this talk considers a novel problem of learning a shared generative model that can facilitate multi-task learning. We present two systems to utilize generative modeling for other visual tasks. The first system focuses on learning a generative [...]
Carnegie Mellon University
MSR Thesis Talk: Shanshan Jessy Xie
Title: GPU based perception via search for object pose estimation with RGB data Abstract: Known object pose estimation is essential for a robot to interact with the real world. It is the first and fundamental task if the robot wants to manipulate the object. This problem is particularly challenging when the environment is complicated [...]
Carnegie Mellon University
Accelerating Numerical Methods for Optimal Control
Abstract: Many modern control methods, such as model-predictive control, rely heavily on solving optimization problems in real time. In particular, the ability to efficiently solve optimal control problems has enabled many of the recent breakthroughs in achieving highly dynamic behaviors for complex robotic systems. The high computational requirements of these algorithms demand novel algorithms tailor-suited [...]
Modeling Coupled Human-Robot Motion for Provable Safety
Abstract: Guide robots that help users who are blind or low vision navigate through crowds and complex environments show promise for improving accessibility in public spaces. These robots must provide real-time safety guarantees for the users, which requires accurate modeling of their behavior in the context of closely coupled human-robot motion. This model must also [...]
MSR Thesis Talk – Mosam Dabhi
Title: Multi-view NRSfM: Affordable setup for high-fidelity 3D reconstruction Abstract: Triangulating a point in 3D space should only require two corresponding camera projections. However in practice, expensive multi-view setups -- involving tens sometimes hundreds of cameras -- are required to obtain the high fidelity 3D reconstructions necessary for many modern applications. In this talk, we argue [...]
Carnegie Mellon University
Robust Object Representations for Robot Manipulation
Abstract: As robots become more common in our daily lives, they will need to interact with many different environments and countless types of objects. While we, as humans, can easily understand an object after seeing it only once, this task is not trivial for robots. Researchers have, for the most part, been left with two [...]
Diminished Reality for Close Quarters Robotic Telemanipulation
Abstract: In robot telemanipulation tasks, the robot itself can sometimes occlude a target object from the user's view. We investigate the potential of diminished reality to address this problem. Our method uses an optical see-through head-mounted display to create a diminished reality illusion that the robot is transparent, allowing users to see occluded areas behind [...]
Carnegie Mellon University
Visual Representation and Recognition without Human Supervision
Abstract: Visual recognition models have seen great advancements by relying on large-scale, carefully curated datasets with human annotations. Most computer vision models leverage human supervision to either construct strong initial representations (e.g. using the ImageNet dataset) or for modeling the visual concepts relevant for downstream tasks (e.g. MS-COCO for object detection). In this thesis, we [...]
Learning Compositional Radiance Fields of Dynamic Human Heads
Meeting ID: 942 4671 0665 Passcode: jkhzoom Abstract: Photorealistic rendering of dynamic humans is an important capability for telepresence systems. Recently, neural rendering methods have been developed to create high-fidelity models of humans and objects. Some of these methods do not produce results with high-enough fidelity for driveable human models (Neural Volumes) whereas others have [...]
When and Why Does Contrastive Learning Work?
Abstract: Contrastive learning organizes data by pulling together related items and pushing apart everything else. These methods have become very popular but it's still not entirely clear when and why they work. I will share two ideas from our recent work. First, I will argue that contrastive learning is really about learning to forget. Different [...]
Carnegie Mellon University
Heuristic Search Based Planning by Minimizing Anticipated Search Efforts
Abstract: Robot planning problems in dynamic environments, such as navigation among pedestrians, driving at high-speed on densely populated roads, and manipulation for collaborative tasks alongside humans, necessitate efficient planning. Bounded-suboptimal heuristic search algorithms are a popular alternative to optimal heuristic search algorithms that compromise solution quality for computation speed. Specifically, these searches aim to find [...]
Carnegie Mellon University
Liquid Metal Actuators
Abstract: Bioinspired robotic actuators arise from the advances in soft materials and activation methods to achieve desired performance. Because of their intrinsic compliance, actuators built from soft materials and liquids can achieve elastic resilience and adaptability similar to their biological counterparts. Liquid metals provide great opportunities for creating an artificial muscle that generates forces at [...]
Anticipating the Future: forecasting the dynamics in multiple levels of abstraction
Abstract: A key navigational capability for autonomous agents is to predict the future locations, actions, and behaviors of other agents in the environment. This is particularly crucial for safety in the realm of autonomous vehicles and robots. However, many current approaches to navigation and control assume perfect perception and knowledge of the environment, even though [...]
Carnegie Mellon University
Understanding and Mitigating Biases in Evaluation
Abstract: There are many problems in real life that involve collecting and aggregating evaluation from people, such as hiring, peer grading and conference peer review. In this thesis, we focus on three sources of biases that arise in such problems, and propose methods to mitigate them. First, we study human bias, that is, the bias [...]
Carnegie Mellon University
MSR Thesis Talk: Manan Shah
ZOOM Link: https://www.google.com/url?q=https://cmu.zoom.us/j/93845075967?pwd%3DbndGc3NvaUVDVFFTTDZvektrNWJqdz09&sa=D&source=calendar&ust=1623592142330000&usg=AOvVaw1xfNPT5c59CQGKzR2bw5sO ID: 93845075967 Passcode: 159459 Title: 3D SLAM for Powered Lower Limb Prosthesis Abstract: During locomotion, humans use visual feedback to adjust their leg movement when navigating the environment. This natural behavior is lost, however, for lower-limb amputees, as current control strategies of prosthetic legs do not typically consider environment perception. With [...]
Carnegie Mellon University
MSR Thesis Talk: Dennis Melamed
Title: Learnable Spatio-Temporal Map Embeddings for Deep Inertial Localization Abstract: Pedestrian localization systems often fuse inertial odometry with map information via hand-defined methods to reduce odometry drift, but such methods are sensitive to noise and struggle to generalize across odometry sources. To address the robustness problem in map utilization, we propose a system that forms a [...]
Learning to Perceive Videos for Embodiment
Abstract: Video understanding has achieved tremendous success in computer vision tasks, such as action recognition, visual tracking, and visual representation learning. Recently, this success has gradually been converted into facilitating robots and embodied agents to interact with the environments. In this talk, I am going to introduce our recent efforts on extracting self-supervisory signals and [...]
Carnegie Mellon University
Self-Learning of Structured Visual Representations
Abstract: Most computer vision models in deployment today are not learning. Instead, they are in a "test" mode, where they will behave the same way perpetually, until they are replaced by newer models. This is a problem, because it means the models may perform poorly as soon as their "test" environment becomes different from their [...]
Carnegie Mellon University
Resource-Constrained Learning and Inference for Visual Perception
Abstract: Real-world applications usually require computer vision algorithms to meet certain resource constraints. In this talk, I will present evaluation methods and principled solutions for both training and testing. First, I will talk about a formal setting for studying training under the non-asymptotic, resource-constrained regime, i.e., budgeted training. We analyze the following problem: "given a [...]
Carnegie Mellon University
Towards Safe and Resilient Autonomy in Multi-Robot Systems
Abstract: Autonomous systems such as robotic systems are envisioned to co-exist with humans in our daily lives, from household service to large-scale warehouse logistics, agricultural monitoring, and smart city. Reliable interactions among robots and humans require provably correct guarantees about safety and performance when designing robot behaviors. While traditional approaches for safety and performance analysis [...]
Carnegie Mellon University
Provably Constant-Time Motion Planning
Abstract: In many robotic applications, including logistics and manufacturing, robots often operate in semi-structured environments and perform highly repetitive manipulation tasks. Additionally, large parts of these environments are static most of the time. Fast and reliable motion planning is one of the key elements that ensure efficient operations in such environments. A very common example [...]
MSR Thesis Talk: Kshitij Goel
Title: Rapid Subsurface Exploration with Multiple Aerial Robots Abstract: This thesis develops a robotic exploration framework that allows for rapid and communication-efficient mapping of unknown environments with a team of aerial robots. Aerial robots can provide rapid and agile mobility in diverse environments where ground mobility is either severely constrained or impossible. However, high-speed flight [...]
Carnegie Mellon University
MSR Thesis Talk: Albert Ye
Title: BusEdge: Efficient Live Video Analytics for Transit Buses via Edge Computing Abstract: Many vehicles like transit buses are now routinely fitted with cameras. These live visual data are invaluable to achieve real-time traffic monitoring, but it is intractable to handle such a gigantic amount of data either locally or in the cloud due to [...]
Carnegie Mellon University
Planning to Optimize and Learn Reward in Navigation Tasks in Structured Environments with Time Constraints
Abstract: Planning problems in which an agent must perform tasks for reward by navigating its environment while constrained by time and location have a wide variety of applications in robotics. Many real-world environments in which such planning problems apply, such as office buildings or city streets, are very structured. They consist of passages with notable [...]
Carnegie Mellon University
MSR Thesis Talk: Yunze Man
Title: Multi-Echo 3D Object Detection Abstract: LiDAR sensors can be used to obtain a wide range of measurement signals other than a simple 3D point cloud, and those signals can be leveraged to improve perception tasks like 3D object detection. A single laser pulse can be partially reflected by multiple objects along its path, resulting [...]
Carnegie Mellon University
Krishna Uppala – MSR Thesis Talk
Title: Exemplar free video retrieval. Abstract: Video retrieval of activities has a wide range of applications. In the traditional mode of operation, a collection of example videos describing the activities are given and the retrieval technique identifies other samples in a dataset that semantically match the examples provided. However, retrieval using a collection of example [...]
Ruixuan Liu – MSR Thesis Talk
Title: Data-efficient Behavior Prediction for Safe Human-Robot Collaboration. Abstract: Predicting human behavior is critical to facilitate safe and efficient human-robot collaboration (HRC). However, human behavior is difficult to predict due to the scarcity of human motion data. This work explores using online adaptation, an online approach, and data augmentation, an offline approach, to deal with the [...]
MSR Thesis Talk: Vidhi Jain
Title: Explainability in navigation policies Abstract: Today's autonomous agents have improved performance with learning and planning algorithms, but the applicability of such agents in the human-inhabited world is confined. Humans find it hard to explain the model's decision-making and thus, may not trust it as a teammate. While working with a machine learning model that [...]
Carnegie Mellon University
MSR Thesis Talk: Aditya Sripada
Title: Turning Behavior of Running Systems induced by Leg Placement Abstract: Compared to legged robots, animals and humans can perform much faster and larger turns, even when they run at high speeds. Such rapid turns require the body of a runner to reorient dynamically and in synchrony with its redirection during stance. While it is [...]
Carnegie Mellon University
MSR Thesis Talk: Rohit Jena
Title: Learning Mental Models of Experts in a Simulated Search and Rescue Scenario Abstract: Search and Rescue is a task where the rescuers need to be cognitively agile, strategically consistent, and efficient to save as many trapped victims as possible. In a team scenario, the rescuers must additionally coordinate with each other based on [...]
MSR Thesis Talk: Mohamad Qadri
Title: Robotic Vision for 3D Modeling and Sizing in Agriculture Abstract: Obtaining accurate perceptual information is a critical component in agricultural robotics since there is a heavy need for interaction with the environment to perform tasks such as pruning, harvesting, and phenotyping. In this thesis, we tackle the problem of perception and 3D modeling in [...]
Carnegie Mellon University
MSR Thesis Talk: Alex Baikovitz
Title: Underground Representations for Robot Localization and Mapping Abstract: There has been exciting recent progress in using radar as a sensor for robot navigation given its increased robustness to varying environmental conditions. However, within these different radar perception systems, ground penetrating radar (GPR) remains under-explored. By measuring structures beneath the ground, GPR can provide stable features that [...]
Carnegie Mellon University
MSR Thesis Talk: Ankur Deka
Title: On combining Reinforcement Learning & Adversarial Training Abstract: Reinforcement Learning (RL) allows us to train an agent to excel at a given sequential decision-making task by optimizing for a reward signal. Adversarial training involves a joint optimization scheme where an agent and an adversary compete against each other. In this work, we explore some [...]
MSR Thesis Talk: Zhengyi Luo
Title: Physics-based object-aware ego-centric human pose estimation Abstract: We investigate the roles of body kinematics, dynamics, and objects for 3D human pose estimation using a head-mounted camera. Human kinematics models play a key role in encoding the natural range of human motion, while dynamics models can react to the spatial arrangement between humans and objects [...]
MSR Thesis Talk: Tarasha Khurana
Title: Detecting Invisible People Abstract: Monocular object detection and tracking have improved drastically in recent years, but rely on a key assumption: that objects are visible to the camera. Many offline tracking approaches reason about occluded objects post-hoc, by linking together tracklets after the object re-appears, making use of reidentification (ReID). However, online tracking [...]
Carnegie Mellon University
MSR Thesis Talk: Sujay Bajracharya
work, we address the problem of goal-directed cloth manipulation, a challenging task due to the deformability of cloth. Our insight is that optical flow, a technique normally used for motion estimation in video, can also provide an effective representation for corresponding cloth poses across observation and goal images. We introduce FabricFlowNet (FFN), a cloth manipulation [...]
Carnegie Mellon University
MSR Thesis Talk: Edward Chen
Title: Towards Practical Ultrasound AI Across Real-World Patient Diversity Abstract: In the case of high-tempo, traumatic scenarios on the battlefield, real-time ultrasound (US) imaging serves as an enabler for countless possible robotic interventions. Having the ability to automatically segment anatomical landmarks in the body, such as arteries, veins, ligaments, and veins, for percutaneous procedures remains [...]
Carnegie Mellon University
MSR Thesis Talk: Haochen Wang
Title: Audiovisual ontology and robust representations via cross-modal fusion Abstract: The shrill of an ambulance siren and flashing lights, the hum of an accelerating car — important events often come to us simultaneously through sight and sound. We first consider the problem of identifying these events from raw, unlabeled audiovisual data of agents interacting with [...]
Social Navigation with Pedestrian Groups
Abstract: Autonomous navigation in human crowds (i.e., social navigation) presents several challenges: The robot often needs to rely on its noisy sensors to identify and localize the pedestrians in human crowds; The robot needs plan efficient paths to reach its goals; The robot needs to do so in a safe and socially appropriate manner. In [...]
Carnegie Mellon University
MSR Thesis Talk: Viraj Parimi
Title: T-HTN: Timeline Based HTN Planning for Multi-Agent Robots Abstract: Planning in mission-critical systems like deep-space habitats with onboard robotic systems must be robust to unforeseen circumstances. Such systems are expected to complete a set of goals with different deadlines each day for routine maintenance while also accounting for emergencies. With the presence of [...]
Carnegie Mellon University
Robust and Scalable Perception For Autonomy
Abstract: Autonomous mobile robots have the potential to drastically improve the quality of our daily life. For example, self-driving vehicles could make transportation safer and more affordable. To navigate complex environments, such robots need a perception system that translates raw sensory data to high-level understanding. This thesis focuses on two fundamental challenges in learning such [...]
Carnegie Mellon University
MSR Thesis Talk: Yiming Zuo
Title: Towards Self-supervised Object Discovery and Tracking Abstract: Object discovery and multiple object tracking (MOT) are two highly interrelated tasks that are known to be fundamental problems in computer vision, and are crucial for video understanding. Most existing methods rely on supervised training with human annotations, which is laborious and expensive. In this thesis, [...]
Carnegie Mellon University
MSR Thesis Talk: Qiao Gu
Title: Towards Object-generic 6D Pose Estimation Abstract: Pose estimation is a basic module in many robot manipulation pipelines. Estimating the pose of objects in the environment can be useful for grasping, motion planning, or manipulation. However, current state-of-the-art methods for pose estimation either rely on large annotated training sets or simulated data. Further, the long [...]
Carnegie Mellon University
MSR Thesis Talk: Divam Gupta
Title: End-to-End Deep Stereo Layout Estimation Abstract: Accurate layout estimation is crucial for planning and navigation in robotics applications, such as self-driving. In this paper, we introduce the Stereo Bird's Eye ViewNetwork (SBEVNet), a novel supervised end-to-end framework for estimation of bird's eye view layout from a pair of stereo images. Although our network [...]
Carnegie Mellon University
Michael Tasota – MSR Thesis Talk
Title: Design of a Multimodal System for Social Emotional Learning in Early Childhood Classrooms Abstract: As the prevalence of mobile and touch-based devices continues to expand in society, so too does its impact on young children. With educational technologies also on the rise, young children benefit most from those technologies that are designed to [...]
Carnegie Mellon University
MSR Thesis Talk: Aaron Huang
Title: End-to-End Methods for Autonomous Driving in Simulation Abstract: Fully autonomous driving is considered one of the grand challenges of modern technology and a variety of approaches have emerged for creating and evaluating autonomous driving agents. The self-driving industry typically adopts a modular software architecture and uses large fleets of autonomous vehicles for data [...]
Carnegie Mellon University
MSR Thesis Talk
Title: Retrieval-based Novel Activity Detection in Untrimmed Videos Abstract: Accurately detecting activities in untrimmed videos is a challenging task as systems need to handle variance in object scales, multiple viewpoints, and multiple types of activities. Furthermore, in a real-world scenario, activity detectors are often required to detect novel kinds of activities when the need [...]
Carnegie Mellon University
Semantic Mapping for Autonomous Navigation and Exploration
Abstract: The last two decades have seen enormous progress in the sensors and algorithms for 3D perception, giving robots the means to build accurate spatial maps and localize themselves in them in real time. The geometric information in these maps is invaluable for navigation while avoiding obstacles, but insufficient, by itself, for robots to robustly [...]
Carnegie Mellon University
MSR Thesis Talk: Howard Coffin
Title: Mutual Information Maps for Single and Multi-Target Ergodic Search Abstract: This thesis addresses use of multi-agent systems to perform autonomous search for moving targets. Target search has many applications, including search and rescue and surveillance, but most robotic systems used in these situations require human operators. Recent works have used ergodic search methods to [...]
Carnegie Mellon University
MSR Thesis Talk: Wanwen Chen
Title: Ultrasound-based Needle Tracking and Lateral Manipulation Planning for Common Needle Steering Abstract: Many medical interventions require inserting a needle towards the desired site. Robotic needle insertion can potentially provide reliable needle access independent of practitioner experience and availability. However, when the needle is inserted into heterogeneous tissue, the complicated needle-tissue interaction introduces multiple challenges [...]
Carnegie Mellon University
MSR Thesis Talk: Rohit Garg
Title: Coordinated Aerial Exploration of Subterranean Voids Abstract: Subterranean exploration has been thrust into the spotlight by the recent DARPA Subterranean Challenge. Teams are tasked with developing a multi-agent system that can rapidly navigate through unknown underground environments while sending back maps and other semantic information. A system with such capabilities has a wide [...]
Carnegie Mellon University
MSR Thesis Talk: Harvey Shi
Title: The design of a compact laser scanner and an integrated simulation environment for smart manufacturing Abstract: Advanced manufacturing is on the rise with the notable progress in the additive and hybrid manufacturing field. However, most of the existing solutions are open loop without sensory feedback because of the lack of a small form factor [...]
Carnegie Mellon University
MSR Thesis Talk: Daqian Cheng
Zoom link: https://cmu.zoom.us/j/99726348928?pwd=azlicW9qZk1Nc3BkQXBLT2d6djVBQT09 Meeting ID: 997 2634 8928 Passcode: 136122 Title: SLAM with Laser Profilers for High Definition Mapping in Confined Spaces Abstract: Three-dimensional reconstruction in confined spaces is important for the manufacturing of aircraft wings, the inspection of narrow pipes, the examination of turbine blades, etc. It is also challenging because confined spaces tend [...]
Carnegie Mellon University
MSR Thesis Talk: Quint Hester
Title: Graph Neural Networks for Wind Farm Yaw Optimization Abstract: In order to mitigate the effects of climate change, it is important to maximize the power output of our wind energy resources. Wind farm control is typically greedy, with all turbines facing directly into the wind. Due to wake interactions, this leads to reductions [...]
Carnegie Mellon University
MSR Thesis Talk: Henry Zhang
Title: On Visual-Aided LiDAR-Inertial Odometry System in Challenging Subterranean Environments Abstract: Simultaneous Localization and Mapping (SLAM) is one of the fundamental components for autonomous robotic exploration. The goal of SLAM is to create an accurate map for the environment and provide robust state estimation for planning, control, and perception tasks. However, due to the [...]
Carnegie Mellon University
Efficient Robot Decision-Making for Achieving Multiple Independent Tasks
Abstract: We focus on robotics applications where a robot is required to accomplish a set of tasks that are partially observable and evolve independently of each other according to their dynamics. One such domain that we target in this work is decision-making for a robot waiter waiting tables at a restaurant. The robot waiter should [...]
MSR Thesis Talk: Ryan Darnley
Title: Communication Network Flow Control With LQR and Factor Graphs Abstract: Advances in the estimation and more recently the controls communities have leveraged the use of factor graphs to overcome problems of scalability and runtime in dynamical systems. In continuation of that work, we look to use factor graphs to perform LQR control on [...]
Carnegie Mellon University
MSR Thesis Talk: Shirsendu Halder
Title: Robust 3D reconstruction in noisy environments Abstract: Automated inspection in industrial manufacturing can minimize the total production cost of a part. Current inspection solutions often involve measuring a part manually, which interrupts the machining process. We present two non-contact real-time systems which integrate visual inspection in-line with CNC (computer numerical control) machines and ensures [...]
MSR Thesis Talk: Tejus Gupta
ZOOM Link: https://cmu.zoom.us/j/2388465851 Meeting ID: 238 846 5851 Google calendar invite link Title: Adaptive and Efficient Models for Intent Recognition Abstract: Assistive robots should have the ability to understand the intent of humans, predict their behavior, and plan to provide anticipatory assistance in complex real-life environments. In this thesis, we present adaptive and efficient algorithms for recognizing [...]
Carnegie Mellon University
MSR Thesis Talk: Siddharth Agrawal
Title: Learning to Imitate, Adapt and Communicate Abstract: For AI agents to co-exist with humans, they need to be able to learn from us humans, adapt to any perceived changes in our behavior, and communicate in a manner that is easily interpretable. In this work, we investigate the following 3 subproblems: Imitation Learning, Adaptation in Human-Agent [...]
Carnegie Mellon University
MSR Thesis Talk: Danny Vedova
Link: https://cmu.zoom.us/j/92810207130?pwd=blphbkovYVA2b1VoSmhuZ0h2bFRzZz09 Meeting ID: 928 1020 7130 Passcode: 721620 Title: Design and Analysis of a Control Pipeline for a Manipulator in Space Abstract: Many satellites are rapidly reaching the end of their lifespans, and risk de-orbiting if no action is taken. One common problem satellites face towards the end of their lifespans is that they are running [...]
Carnegie Mellon University
MSR Thesis Talk: Fan Jia
Zoom Link: https://cmu.zoom.us/j/9793644214?pwd=enpab3dPc0FBWVdqWDNrcVlabUJCZz09 Meeting ID: 979 364 4214 Passcode: 304453 Title: Trust-Aware Behavior Reflection for Robot Swarm Self-Healing Abstract: The deployment of robot swarms is influenced by real-world factors, such as motor issues, sensor failure, and wind disturbances. These factors cause the appearance of faulty robots. In a decentralized swarm, sharing incorrect information from faulty robots will lead to [...]
Carnegie Mellon University
MSR Thesis Talk: Tejas Zodage
Meeting Link: https://cmu.zoom.us/j/2427727031 Title: Point Cloud Registration as a Classification Problem Abstract: Point Cloud Registration(PCR) is an important step in fields such as robotic manipulation, augmented and virtual reality, SLAM, etc. In the context of computer vision, registration, in general, refers to the process of aligning data obtained from different frames, and as the [...]
Carnegie Mellon University
Decentralized Navigation of Quadrotor Teams in Uncertain Workspaces
Abstract: A fundamental requirement for realizing scalable and responsive real-world multi-robot systems for time-sensitive critical applications such as search and rescue or building clearance is a motion-planning and coordination framework that exhibits three essential properties. The first property is safety that encompasses aspects relating to kinodynamic feasibility and collision-avoidance. The second property is reliability that [...]