Title: Learning to Perceive and Manipulate Diverse Food Materials Through Interaction Abstract: The home kitchen environment presents many challenges for an autonomous cooking robot, such as the deformability of food items, the wide range of material properties of food, and the complex interaction dynamics involved in food manipulation tasks. Material properties are important when interacting [...]

Title: Localization for Lunar Micro-Rovers   Abstract: This talk presents an avionics and localization system that enables a lunar micro-rover to navigate autonomously. This system is important for the latest class of small, low-powered, and fast robots going to the Moon in search of polar ice. The first component of the system is an Extended [...]

Title: Incorporating Semantic Structure in SLAM Abstract: For robots to understand the environment they interact with, a combination of geometric information and semantic information is imperative. In this talk, I propose a fast and scalable Simultaneous Localization and Mapping (SLAM) system that represents indoor scenes as a graph of semantic objects. Leveraging the observation that [...]

Title: Exploiting Uncertainty in Triangulation Light Curtains for Object Tracking and Depth Estimation   Abstract: Active sensing through the use of Adaptive Depth Sensors is a nascent field, with potential in areas such as Advanced driver-assistance systems (ADAS). One such class of sensor is the Triangulation Light Curtain, which was developed in the Illumination and Imaging [...]

Title: Introducing Generative Models to Facilitate Multi-Task Visual Learning Abstract: Motivated by multi-task learning of shared feature representations, this talk considers a novel problem of learning a shared generative model that can facilitate multi-task learning. We present two systems to utilize generative modeling for other visual tasks. The first system focuses on learning a generative [...]

Title: GPU based perception via search for object pose estimation with RGB data   Abstract: Known object pose estimation is essential for a robot to interact with the real world.  It is the first and fundamental task if the robot wants to manipulate the object.  This problem is particularly challenging when the environment is complicated [...]

Title: Multi-view NRSfM: Affordable setup for high-fidelity 3D reconstruction   Abstract: Triangulating a point in 3D space should only require two corresponding camera projections. However in practice, expensive multi-view setups -- involving tens sometimes hundreds of cameras -- are required to obtain the high fidelity 3D reconstructions necessary for many modern applications. In this talk, we argue [...]

ZOOM Link: https://www.google.com/url?q=https://cmu.zoom.us/j/93845075967?pwd%3DbndGc3NvaUVDVFFTTDZvektrNWJqdz09&sa=D&source=calendar&ust=1623592142330000&usg=AOvVaw1xfNPT5c59CQGKzR2bw5sO   ID: 93845075967 Passcode: 159459 Title: 3D SLAM for Powered Lower Limb Prosthesis Abstract: During locomotion, humans use visual feedback to adjust their leg movement when navigating the environment. This natural behavior is lost, however, for lower-limb amputees, as current control strategies of prosthetic legs do not typically consider environment perception. With [...]

Title: Learnable Spatio-Temporal Map Embeddings for Deep Inertial Localization Abstract: Pedestrian localization systems often fuse inertial odometry with map information via hand-defined methods to reduce odometry drift, but such methods are sensitive to noise and struggle to generalize across odometry sources. To address the robustness problem in map utilization, we propose a system that forms a [...]

Title: Rapid Subsurface Exploration with Multiple Aerial Robots Abstract: This thesis develops a robotic exploration framework that allows for rapid and communication-efficient mapping of unknown environments with a team of aerial robots. Aerial robots can provide rapid and agile mobility in diverse environments where ground mobility is either severely constrained or impossible. However, high-speed flight [...]

Title: BusEdge: Efficient Live Video Analytics for Transit Buses via Edge Computing Abstract: Many vehicles like transit buses are now routinely fitted with cameras. These live visual data are invaluable to achieve real-time traffic monitoring, but it is intractable to handle such a gigantic amount of data either locally or in the cloud due to [...]

Title: Multi-Echo 3D Object Detection Abstract: LiDAR sensors can be used to obtain a wide range of measurement signals other than a simple 3D point cloud, and those signals can be leveraged to improve perception tasks like 3D object detection. A single laser pulse can be partially reflected by multiple objects along its path, resulting [...]

Title: Exemplar free video retrieval.   Abstract: Video retrieval of activities has a wide range of applications. In the traditional mode of operation,  a collection of example videos describing the activities are given and the retrieval technique identifies other samples in a dataset that semantically match the examples provided. However, retrieval using a collection of example [...]

Title: Data-efficient Behavior Prediction for Safe Human-Robot Collaboration.   Abstract: Predicting human behavior is critical to facilitate safe and efficient human-robot collaboration (HRC). However, human behavior is difficult to predict due to the scarcity of human motion data. This work explores using online adaptation, an online approach, and data augmentation, an offline approach, to deal with the [...]

Title: Explainability in navigation policies Abstract: Today's autonomous agents have improved performance with learning and planning algorithms, but the applicability of such agents in the human-inhabited world is confined. Humans find it hard to explain the model's decision-making and thus, may not trust it as a teammate. While working with a machine learning model that [...]

Title: Turning Behavior of Running Systems induced by Leg Placement Abstract: Compared to legged robots, animals and humans can perform much faster and larger turns, even when they run at high speeds. Such rapid turns require the body of a runner to reorient dynamically and in synchrony with its redirection during stance. While it is [...]

Title: Learning Mental Models of Experts in a Simulated Search and Rescue Scenario   Abstract: Search and Rescue is a task where the rescuers need to be cognitively agile, strategically consistent, and efficient to save as many trapped victims as possible. In a team scenario, the rescuers must additionally coordinate with each other based on [...]

Title: Robotic Vision for 3D Modeling and Sizing in Agriculture Abstract: Obtaining accurate perceptual information is a critical component in agricultural robotics since there is a heavy need for interaction with the environment to perform tasks such as pruning, harvesting, and phenotyping. In this thesis, we tackle the problem of perception and 3D modeling in [...]

Title: Underground Representations for Robot Localization and Mapping   Abstract: There has been exciting recent progress in using radar as a sensor for robot navigation given its increased robustness to varying environmental conditions. However, within these different radar perception systems, ground penetrating radar (GPR) remains under-explored. By measuring structures beneath the ground, GPR can provide stable features that [...]

Title: On combining Reinforcement Learning & Adversarial Training Abstract: Reinforcement Learning (RL) allows us to train an agent to excel at a given sequential decision-making task by optimizing for a reward signal. Adversarial training involves a joint optimization scheme where an agent and an adversary compete against each other. In this work, we explore some [...]

Title: Physics-based object-aware ego-centric human pose estimation Abstract: We investigate the roles of body kinematics, dynamics, and objects for 3D human pose estimation using a head-mounted camera. Human kinematics models play a key role in encoding the natural range of human motion, while dynamics models can react to the spatial arrangement between humans and objects [...]

Title: Detecting Invisible People   Abstract: Monocular object detection and tracking have improved drastically in recent years, but rely on a key assumption: that objects are visible to the camera. Many offline tracking approaches reason about occluded objects post-hoc, by linking together tracklets after the object re-appears, making use of reidentification (ReID). However, online tracking [...]

work, we address the problem of goal-directed cloth manipulation, a challenging task due to the deformability of cloth. Our insight is that optical flow, a technique normally used for motion estimation in video, can also provide an effective representation for corresponding cloth poses across observation and goal images. We introduce FabricFlowNet (FFN), a cloth manipulation [...]

Title: Towards Practical Ultrasound AI Across Real-World Patient Diversity Abstract: In the case of high-tempo, traumatic scenarios on the battlefield, real-time ultrasound (US) imaging serves as an enabler for countless possible robotic interventions. Having the ability to automatically segment anatomical landmarks in the body, such as arteries, veins, ligaments, and veins, for percutaneous procedures remains [...]

Title: Audiovisual ontology and robust representations via cross-modal fusion   Abstract: The shrill of an ambulance siren and flashing lights, the hum of an accelerating car — important events often come to us simultaneously through sight and sound. We first consider the problem of identifying these events from raw, unlabeled audiovisual data of agents interacting with [...]

Title: T-HTN: Timeline Based HTN Planning for Multi-Agent Robots   Abstract: Planning in mission-critical systems like deep-space habitats with onboard robotic systems must be robust to unforeseen circumstances. Such systems are expected to complete a set of goals with different deadlines each day for routine maintenance while also accounting for emergencies. With the presence of [...]

Title: Towards Self-supervised Object Discovery and Tracking   Abstract: Object discovery and multiple object tracking (MOT) are two highly interrelated tasks that are known to be fundamental problems in computer vision, and are crucial for video understanding. Most existing methods rely on supervised training with human annotations, which is laborious and expensive. In this thesis, [...]

Title: Towards Object-generic 6D Pose Estimation Abstract: Pose estimation is a basic module in many robot manipulation pipelines. Estimating the pose of objects in the environment can be useful for grasping, motion planning, or manipulation. However, current state-of-the-art methods for pose estimation either rely on large annotated training sets or simulated data. Further, the long [...]

Title: End-to-End Deep Stereo Layout Estimation   Abstract: Accurate layout estimation is crucial for planning and navigation in robotics applications, such as self-driving. In this paper, we introduce the Stereo Bird's Eye ViewNetwork (SBEVNet), a novel supervised end-to-end framework for estimation of bird's eye view layout from a pair of stereo images. Although our network [...]

Title: Design of a Multimodal System for Social Emotional Learning in Early Childhood Classrooms   Abstract: As the prevalence of mobile and touch-based devices continues to expand in society, so too does its impact on young children. With educational technologies also on the rise, young children benefit most from those technologies that are designed to [...]

Title: End-to-End Methods for Autonomous Driving in Simulation   Abstract: Fully autonomous driving is considered one of the grand challenges of modern technology and a variety of approaches have emerged for creating and evaluating autonomous driving agents. The self-driving industry typically adopts a modular software architecture and uses large fleets of autonomous vehicles for data [...]

Title: Retrieval-based Novel Activity Detection in Untrimmed Videos   Abstract: Accurately detecting activities in untrimmed videos is a challenging task as systems need to handle variance in object scales, multiple viewpoints, and multiple types of activities. Furthermore, in a real-world scenario, activity detectors are often required to detect novel kinds of activities when the need [...]

Title: Mutual Information Maps for Single and Multi-Target Ergodic Search   Abstract: This thesis addresses use of multi-agent systems to perform autonomous search for moving targets. Target search has many applications, including search and rescue and surveillance, but most robotic systems used in these situations require human operators. Recent works have used ergodic search methods to [...]

Title: Ultrasound-based Needle Tracking and Lateral Manipulation Planning for Common Needle Steering Abstract: Many medical interventions require inserting a needle towards the desired site. Robotic needle insertion can potentially provide reliable needle access independent of practitioner experience and availability. However, when the needle is inserted into heterogeneous tissue, the complicated needle-tissue interaction introduces multiple challenges [...]

Title: Coordinated Aerial Exploration of Subterranean Voids   Abstract: Subterranean exploration has been thrust into the spotlight by the recent DARPA Subterranean Challenge. Teams are tasked with developing a multi-agent system that can rapidly navigate through unknown underground environments while sending back maps and other semantic information. A system with such capabilities has a wide [...]

Title: The design of a compact laser scanner and an integrated simulation environment for smart manufacturing Abstract: Advanced manufacturing is on the rise with the notable progress in the additive and hybrid manufacturing field. However, most of the existing solutions are open loop without sensory feedback because of the lack of a small form factor [...]

Zoom link: https://cmu.zoom.us/j/99726348928?pwd=azlicW9qZk1Nc3BkQXBLT2d6djVBQT09 Meeting ID: 997 2634 8928 Passcode: 136122   Title: SLAM with Laser Profilers for High Definition Mapping in Confined Spaces   Abstract: Three-dimensional reconstruction in confined spaces is important for the manufacturing of aircraft wings, the inspection of narrow pipes, the examination of turbine blades, etc. It is also challenging because confined spaces tend [...]

Title: Graph Neural Networks for Wind Farm Yaw Optimization   Abstract: In order to mitigate the effects of climate change, it is important to maximize the power output of our wind energy resources. Wind farm control is typically greedy, with all turbines facing directly into the wind. Due to wake interactions, this leads to reductions [...]

Title: On Visual-Aided LiDAR-Inertial Odometry System in Challenging Subterranean Environments   Abstract: Simultaneous Localization and Mapping (SLAM) is one of the fundamental components for autonomous robotic exploration. The goal of SLAM is to create an accurate map for the environment and provide robust state estimation for planning, control, and perception tasks. However, due to the [...]

Title: Communication Network Flow Control With LQR and Factor Graphs   Abstract: Advances in the estimation and more recently the controls communities have leveraged the use of factor graphs to overcome problems of scalability and runtime in dynamical systems. In continuation of that work, we look to use factor graphs to perform LQR control on [...]

Title: Robust 3D reconstruction in noisy environments Abstract: Automated inspection in industrial manufacturing can minimize the total production cost of a part. Current inspection solutions often involve measuring a part manually, which interrupts the machining process. We present two non-contact real-time systems which integrate visual inspection in-line with CNC (computer numerical control) machines and ensures [...]

ZOOM Link: https://cmu.zoom.us/j/2388465851 Meeting ID: 238 846 5851 Google calendar invite link Title: Adaptive and Efficient Models for Intent Recognition Abstract: Assistive robots should have the ability to understand the intent of humans, predict their behavior, and plan to provide anticipatory assistance in complex real-life environments. In this thesis, we present adaptive and efficient algorithms for recognizing [...]

Title: Learning to Imitate, Adapt and Communicate Abstract: For AI agents to co-exist with humans, they need to be able to learn from us humans, adapt to any perceived changes in our behavior, and communicate in a manner that is easily interpretable. In this work, we investigate the following 3 subproblems: Imitation Learning, Adaptation in Human-Agent [...]

Link: https://cmu.zoom.us/j/92810207130?pwd=blphbkovYVA2b1VoSmhuZ0h2bFRzZz09 Meeting ID: 928 1020 7130 Passcode: 721620 Title: Design and Analysis of a Control Pipeline for a Manipulator in Space Abstract: Many satellites are rapidly reaching the end of their lifespans, and risk de-orbiting if no action is taken. One common problem satellites face towards the end of their lifespans is that they are running [...]

Zoom Link: https://cmu.zoom.us/j/9793644214?pwd=enpab3dPc0FBWVdqWDNrcVlabUJCZz09 Meeting ID: 979 364 4214 Passcode: 304453 Title: Trust-Aware Behavior Reflection for Robot Swarm Self-Healing Abstract: The deployment of robot swarms is influenced by real-world factors, such as motor issues, sensor failure, and wind disturbances. These factors cause the appearance of faulty robots. In a decentralized swarm, sharing incorrect information from faulty robots will lead to [...]

Meeting Link: https://cmu.zoom.us/j/2427727031   Title: Point Cloud Registration as a Classification Problem Abstract: Point Cloud Registration(PCR) is an important step in fields such as robotic manipulation, augmented and virtual reality, SLAM, etc. In the context of computer vision, registration, in general, refers to the process of aligning data obtained from different frames, and as the [...]