MSR Speaking Qualifier

Title: Towards Self-supervised Object Discovery and Tracking   Abstract: Object discovery and multiple object tracking (MOT) are two highly interrelated tasks that are known to be fundamental problems in computer vision, and are crucial for video understanding. Most existing methods rely on supervised training with human annotations, which is laborious and expensive. In this thesis, [...]

Title: Towards Object-generic 6D Pose Estimation Abstract: Pose estimation is a basic module in many robot manipulation pipelines. Estimating the pose of objects in the environment can be useful for grasping, motion planning, or manipulation. However, current state-of-the-art methods for pose estimation either rely on large annotated training sets or simulated data. Further, the long [...]

Title: End-to-End Deep Stereo Layout Estimation   Abstract: Accurate layout estimation is crucial for planning and navigation in robotics applications, such as self-driving. In this paper, we introduce the Stereo Bird's Eye ViewNetwork (SBEVNet), a novel supervised end-to-end framework for estimation of bird's eye view layout from a pair of stereo images. Although our network [...]

Title: Design of a Multimodal System for Social Emotional Learning in Early Childhood Classrooms   Abstract: As the prevalence of mobile and touch-based devices continues to expand in society, so too does its impact on young children. With educational technologies also on the rise, young children benefit most from those technologies that are designed to [...]

Title: End-to-End Methods for Autonomous Driving in Simulation   Abstract: Fully autonomous driving is considered one of the grand challenges of modern technology and a variety of approaches have emerged for creating and evaluating autonomous driving agents. The self-driving industry typically adopts a modular software architecture and uses large fleets of autonomous vehicles for data [...]

Title: Retrieval-based Novel Activity Detection in Untrimmed Videos   Abstract: Accurately detecting activities in untrimmed videos is a challenging task as systems need to handle variance in object scales, multiple viewpoints, and multiple types of activities. Furthermore, in a real-world scenario, activity detectors are often required to detect novel kinds of activities when the need [...]

Title: Mutual Information Maps for Single and Multi-Target Ergodic Search   Abstract: This thesis addresses use of multi-agent systems to perform autonomous search for moving targets. Target search has many applications, including search and rescue and surveillance, but most robotic systems used in these situations require human operators. Recent works have used ergodic search methods to [...]

Title: Ultrasound-based Needle Tracking and Lateral Manipulation Planning for Common Needle Steering Abstract: Many medical interventions require inserting a needle towards the desired site. Robotic needle insertion can potentially provide reliable needle access independent of practitioner experience and availability. However, when the needle is inserted into heterogeneous tissue, the complicated needle-tissue interaction introduces multiple challenges [...]

Title: Coordinated Aerial Exploration of Subterranean Voids   Abstract: Subterranean exploration has been thrust into the spotlight by the recent DARPA Subterranean Challenge. Teams are tasked with developing a multi-agent system that can rapidly navigate through unknown underground environments while sending back maps and other semantic information. A system with such capabilities has a wide [...]

Title: The design of a compact laser scanner and an integrated simulation environment for smart manufacturing Abstract: Advanced manufacturing is on the rise with the notable progress in the additive and hybrid manufacturing field. However, most of the existing solutions are open loop without sensory feedback because of the lack of a small form factor [...]

Zoom link: https://cmu.zoom.us/j/99726348928?pwd=azlicW9qZk1Nc3BkQXBLT2d6djVBQT09 Meeting ID: 997 2634 8928 Passcode: 136122   Title: SLAM with Laser Profilers for High Definition Mapping in Confined Spaces   Abstract: Three-dimensional reconstruction in confined spaces is important for the manufacturing of aircraft wings, the inspection of narrow pipes, the examination of turbine blades, etc. It is also challenging because confined spaces tend [...]

Title: Graph Neural Networks for Wind Farm Yaw Optimization   Abstract: In order to mitigate the effects of climate change, it is important to maximize the power output of our wind energy resources. Wind farm control is typically greedy, with all turbines facing directly into the wind. Due to wake interactions, this leads to reductions [...]

Title: On Visual-Aided LiDAR-Inertial Odometry System in Challenging Subterranean Environments   Abstract: Simultaneous Localization and Mapping (SLAM) is one of the fundamental components for autonomous robotic exploration. The goal of SLAM is to create an accurate map for the environment and provide robust state estimation for planning, control, and perception tasks. However, due to the [...]

Title: Communication Network Flow Control With LQR and Factor Graphs   Abstract: Advances in the estimation and more recently the controls communities have leveraged the use of factor graphs to overcome problems of scalability and runtime in dynamical systems. In continuation of that work, we look to use factor graphs to perform LQR control on [...]

Title: Robust 3D reconstruction in noisy environments Abstract: Automated inspection in industrial manufacturing can minimize the total production cost of a part. Current inspection solutions often involve measuring a part manually, which interrupts the machining process. We present two non-contact real-time systems which integrate visual inspection in-line with CNC (computer numerical control) machines and ensures [...]

ZOOM Link: https://cmu.zoom.us/j/2388465851 Meeting ID: 238 846 5851 Google calendar invite link Title: Adaptive and Efficient Models for Intent Recognition Abstract: Assistive robots should have the ability to understand the intent of humans, predict their behavior, and plan to provide anticipatory assistance in complex real-life environments. In this thesis, we present adaptive and efficient algorithms for recognizing [...]

Title: Learning to Imitate, Adapt and Communicate Abstract: For AI agents to co-exist with humans, they need to be able to learn from us humans, adapt to any perceived changes in our behavior, and communicate in a manner that is easily interpretable. In this work, we investigate the following 3 subproblems: Imitation Learning, Adaptation in Human-Agent [...]

Link: https://cmu.zoom.us/j/92810207130?pwd=blphbkovYVA2b1VoSmhuZ0h2bFRzZz09 Meeting ID: 928 1020 7130 Passcode: 721620 Title: Design and Analysis of a Control Pipeline for a Manipulator in Space Abstract: Many satellites are rapidly reaching the end of their lifespans, and risk de-orbiting if no action is taken. One common problem satellites face towards the end of their lifespans is that they are running [...]

Zoom Link: https://cmu.zoom.us/j/9793644214?pwd=enpab3dPc0FBWVdqWDNrcVlabUJCZz09 Meeting ID: 979 364 4214 Passcode: 304453 Title: Trust-Aware Behavior Reflection for Robot Swarm Self-Healing Abstract: The deployment of robot swarms is influenced by real-world factors, such as motor issues, sensor failure, and wind disturbances. These factors cause the appearance of faulty robots. In a decentralized swarm, sharing incorrect information from faulty robots will lead to [...]

Meeting Link: https://cmu.zoom.us/j/2427727031   Title: Point Cloud Registration as a Classification Problem Abstract: Point Cloud Registration(PCR) is an important step in fields such as robotic manipulation, augmented and virtual reality, SLAM, etc. In the context of computer vision, registration, in general, refers to the process of aligning data obtained from different frames, and as the [...]

Title: Stiffness Mapping of Deformable Objects Through Supervised Embedding and Gaussian Process Regression   Abstract: The stiffness map of a deformable object stores information about that object's surface compliance. Thus, through a stiffness map, we gain insight into the physical properties of that object. Depending on the object, an understanding of stiffness has applications ranging [...]

Title: Spatially-Adaptive Multilayer GAN Inversion   Abstract: Existing GAN inversion and editing methods are well suited for only a target images that contain aligned objects with a clean background, such as portraits and animal faces, but often struggle for more difficult categories with complex scene layouts and object occlusions, such as cars, animals, and outdoor images. [...]

Title: Model-Accuracy Aware Anytime Planning with Simulation Verification for Navigating Complex Terrains Abstract: Off-road and unstructured environments often contain complex patches of various types of terrain, rough elevation changes, deformable objects, etc. An autonomous ground vehicle traversing such environments experiences physical interactions that are extremely hard to model at scale and thus very hard to [...]

Title: Stabilizing the Training Dynamics of Generative Models using Self-Supervision   Abstract: Generative Models have been shown to be adept in mimicking the behavior of an unknown distribution solely from bootstrapped data. However, deep learning models have been shown to overfit in either the minimization or maximization stage of the two player min-max game, resulting [...]

Title: Smartphone localization for Indoor Pedestrian Navigation Abstract: Global positioning system (GPS) interfacing with applications such as Google Maps has proven very useful for navigation in outdoor open settings. However in crowded metropolitan environments with high rise buildings or in indoor settings, GPS quickly becomes unreliable. Using sensors found on commodity smartphones to perform accurate [...]

Title: Distributed Reinforcement Learning for Autonomous Driving Abstract: Due to the complex and safety-critical nature of autonomous driving, recent works typically test their ideas on simulators designed for the very purpose of advancing self-driving research. Despite the convenience of modeling autonomous driving as a trajectory optimization problem, few of these methods resort to online reinforcement [...]

Title: Ride Comfort-Aware Visual Navigation via Self-Supervised Learning Abstract: Under shared autonomy, wheelchair users expect vehicles to provide safe and comfortable rides while following users’ high-level navigation plans. To find such a path, vehicles negotiate with different terrains and assess their traversal difficulty. Most prior works model surroundings either through geometric representations or semantic classifications, [...]

Title: Fast 6D Object Pose Refinement via Deep Texture Rendering   Abstract: We present RePOSE, a fast iterative refinement method for 6D object pose estimation. Prior methods perform refinement by feeding zoomed-in input and rendered RGB images into a CNN and directly regressing an update of a refined pose. Their runtime is slow due to the [...]

Abstract: Detect-and-Avoid (DAA) capabilities are critical for autonomous operations of small unmanned aircraft systems (sUAS). Traditionally DAA systems for large aircraft have been ground and radar-based. Due to the size, weight, and power (SWaP) constraints of sUAS, current DAA systems rely mainly on vision-based sensors and ADS-B (Automatic Dependent Surveillance-Broadcast) transponders. However, not all flying [...]

Title: Preprocessing-based Methods for Robotic Manipulation Abstract: Robotic manipulation is a key problem for several applications such as welding, pick-and-place, and automated assembly. However, motion planning for manipulation can be computationally expensive as it requires planning in the high-dimensional configuration space of the manipulator. Additionally, task-specific constraints such as strict time limits or constraints on end-effector [...]

Title: Socially-Aware Trajectory Prediction Guided by Motion Patterns Abstract: As intelligent robots across domains start collaborating with humans in shared environments, e.g., urban settings and terminal airspace, algorithms that enable them to reason over human motion and intent are important to enable seamless and safe interplay. In our work, we study human intent by focusing on the [...]

Date: Tuesday, July 19, 2022 Time: 9:00 AM - 10:00 AM ET Location: Newell-Simon Hall (NSH) 3305 Title: Detect Active Object in a Sequential Voting Process Abstract: A key component of understanding hand-object interactions is the ability to identify the active object -- the object that is being manipulated by the human hand. In order to accurately localize the [...]

Title: Real-Time Visual Localization System in Changing and Challenging Environments via Visual Place Recognition   Abstract: Localization is one of the fundamental capabilities to guarantee reliable robot autonomy. Many excellent Visual-Inertial and LiDAR-based algorithms have been developed to solve the localization problem. However, deploying these methods on a real-time portable device is challenging due to high [...]

Title: Seeing the Unseen: Closed-loop Occlusion Reasoning for Cloth Manipulation Robotic manipulation of cloth remains challenging due to the complex dynamics of cloth, lack of a low-dimensional state representation, and self-occlusions. Particularly, self-occlusion makes it difficult to estimate the full state of the cloth, which poses significant challenges to policy learning and dynamics modeling. Ideally, [...]

Title: Features in Extra Dimensions: Spatial and Temporal Scene Representations Abstract: Computer vision models have made great progress in featurizing pixels of images. However, an image is only a projection of the actual 3D scene: occlusions and perspective distortions exist. To arrive at a better representation of the scene itself, extra dimensions are needed to [...]

Title: Library of behaviors and tools for robot manipulation Abstract: Learned policies often fail to generalize across environment variations, such as, different objects, object arrangements, or camera viewpoints. Moreover, most policies are trained and tested in simulation environments, and the sim2real gap remains large under weak visual representations that do not disentangle the scene from [...]

Title: 6D Object Pose Estimation for Manipulation via Weak Supervision Abstract: 6D object pose estimation is essential for robotic manipulation tasks. Existing learning-based pose estimators often rely on training from labeled absolute poses with fixed object canonical frames, which (1) requires datasets with annotations of object absolute pose that are resource-intensive to collect; (2) is hard [...]

Title: Using 3D Imaging Radar for Indoor Localization and Mapping Zoom: https://cmu.zoom.us/j/95090884062?pwd=dVZDVHJDTGVUWW9iSlJLTWtidThBUT09 Meeting ID: 950 9088 4062 Passcode: 411959 Abstract: 3D Imaging Radars offer robust perception capability through visually demanding environments due to the unique penetrative and reflective properties of millimeter waves. However, the utilization of Imaging radar for robot navigation and mapping remains under-explored due [...]

Title: Programmable light curtains for Safety Envelopes, SLAM and Navigation Abstract: Conventional robot perception and navigation pipelines are built using traditional sensors such as RGB cameras, stereo depth sensors and LiDARs.These sensors scan the entire scene in a fixed and uniform way. In contrast, programmable light curtains are a recently-invented, resource-efficient sensor that measure the [...]

Title: Lidar-Visual-Inertial Odometry via Modifications and Improvements to Super Odometry Abstract:     The main focus of this thesis involves improvements and extensions to Super Odometry, a preexisting method for lidar-inertial odometry. This was done in the context of the DARPA RACER program as a member of Carnegie Mellon's DEAD Fast team, aiming to provide reliable [...]

Title: Boundary-Aware Demons Algorithm with Applications in Electronic Waste Recycling Abstract Electronic waste (e-waste) refers to electronic devices that are nearing the end of their useful life, and are discarded, donated, or given away. Valuable metallic and plastic components in e-waste (gold, silver, platinum) is estimated to value upwards of $60 billion and although e-waste represents [...]

Title: Introspective Perception through Identifying Blur, Light Direction, and Angle-of-View Abstract Robotic perception tasks have achieved great performance, especially in autonomous vehicles and robot assistance. However, we still often do not understand how and when perception tasks fail. Researchers have achieved some success in creating introspective perception systems that detect when perception tasks will fail, [...]

Title: Image Synthesis with Appearance Decomposition Abstract: Our visual world is compositional and its appearance can be decomposed into various components. Leveraging these components can be beneficial for challenging image synthesis tasks. To this end, this thesis focuses on studying how appearance decomposition can improve image synthesis methods using two examples. (1) Structural decomposition: we introduce [...]

Title: A VPR-Based Technique for UAV Localization In Unseen Environments Abstract: Unmanned Aerial Vehicles (UAVs) primarily rely on GPS-assisted localization and navigation due to the accessibility and ubiquity of such systems. However, this presents a potentially catastrophic single point of failure that may prevent autonomous UAVs from becoming truly reliable, as GPS is prone to dropout, [...]

Title: GPS-Denied Global Visual-Inertial Ground Vehicle State Estimation via Image Registration Abstract: Robotic systems such as unmanned ground vehicles (UGVs) often depend on GPS for navigation in outdoor environments. In GPS-denied environments, one approach to maintain a global state estimate is localizing based on preexisting georeferenced aerial or satellite imagery. However, this is inherently challenged [...]

Title: Learning to Distill Datasets by Matching Expert Training Trajectories Project Page: https://georgecazenavette.github.io/mtt-distillation/ Abstract: Dataset distillation is the task of synthesizing a small dataset such that a model trained on the synthetic set will match the test accuracy of the model trained on the full dataset. In this talk, we review 3 several of our recent [...]

Title: Coordinating Heterogeneous Teams for Urban Search and Rescue Abstract: The mission of Urban Search and Rescue (USAR) has drawn significant interest in robotics. Autonomous entities must be able to share knowledge efficiently to address visibility and collaboration challenges in a complex environment shortly after structural collapse catastrophes. In this thesis, we present methods to coordinate [...]

Title: Composition Learning in “Modular” Robot Systems Abstract: Modular robot and multi-robot systems share a concept in common: composition,  i.e. the study of how parts can be combined so they can be used to achieve certain objectives. Our vision is to enable robotic systems to configure and reconfigure themselves during field deployment, either autonomously or [...]

Title: Towards HD Map Updates With Crosswalk Change Detection From Vehicle-mounted Cameras Zoom: https://cmu.zoom.us/j/4452379705   Abstract: Many autonomous vehicles rely on high-definition maps that contain road layout and road semantics as priors for perception, planning and prediction. However, these maps can become stale over time as the road environment changes. This thesis develops a road monitoring framework [...]

Title: Taxim: An Example-based Simulation Model for GelSight Tactile Sensors and its Sim-to-Real Applications Location: NSH 4305 or Zoom https://cmu.zoom.us/j/91769761787?pwd=cGZ2RElKMVJaQ1NVNG5BdFQ0Ny9uQT09 Abstract: Simulation is widely used in robotics for system verification and large-scale data collection. However, simulating a robot system efficiently and with high fidelity, from sensing, perception to manipulation, has been a long-standing challenge. Tactile sensing, as [...]

Title: A Low-Cost Attitude Determination and Control System and Hardware-in-the-Loop Testbed for CubeSats Zoom: https://cmu.zoom.us/j/92654622790?pwd=d0pYcTJ4K0xzdmYvUHFYWC9lMDBhQT09   Abstract: Since their initial development in the late 1990s, CubeSats have quickly grown popular due to their relatively low cost and short development period. However, CubeSat launches are prone to failure, with less than half of CubeSats completely fulfilling their [...]

Title: Driving by Dreaming: Offline Model-Based Reinforcement Learning for Motion Planning for Autonomous Vehicles Abstract: While there has been significant progress in deploying autonomous vehicles (AVs) in urban driving settings, there remains a long-tail of challenging motion planning scenarios that must be addressed before truly driverless operation is possible. The current paradigm for motion planner [...]

Title: Learning from Physical Human Feedback: An Object-Centric One-Shot Adaptation Method Abstract: For robots to be effectively deployed in novel environments and tasks, they must be able to understand the feedback expressed by humans during intervention. This can either correct undesirable behavior or indicate additional preferences. Existing methods either require repeated episodes of interactions or [...]

Title: Dense Human Pose Estimation From WiFi Abstract: Advances in computer vision and machine learning techniques have led to significant development in 2D and 3D human pose estimation from RGB cameras, LiDAR, and radars. However, human pose estimation from images is adversely affected by occlusion and lighting, which are common in many scenarios of interest. [...]

Title: An efficient approach for sequential shape human performance capture from monocular video Abstract: Human performance capture from RGB videos in unconstrained environments has become very popular for applications to generate virtual avatars or digital actors. Modern approaches rely on neural network algorithms to estimate geometry directly from images, resulting in a coarse representation of [...]

Title: Efficient Methods for Model Performance Inference Abstract: A key challenge in neural architecture search (NAS) is quickly inferring the predictive performance of a broad spectrum of neural networks to discover statistically accurate and computationally efficient ones. We refer to this task as model performance inference (MPI). The current practice for efficient MPI is gradient-based methods [...]

Title: Addressing Time-series Signal Quality in Healthcare Data Abstract: Healthcare data time-series signal quality assessment (SQA) plays a vital role in the accuracy and reliability of machine learning algorithms to analyze health metrics. However, these signals are often corrupted with different kinds of noises and artifacts, including Baseline Wander, Muscle Artifacts, Powerline Interference, and Equipment Failure. This [...]

Title: Search-based Planning for Sensor-based Coverage Abstract: Robots are excellent candidates for the dull, dirty, and dangerous jobs we do not want humans to perform. Today, these include inspection of large areas or structures, post-disaster assessment, and surveillance. Assessing the aftermath of the recent Fern Hollow bridge collapse in Pittsburgh is one such example. Many [...]

Title: See But Don't Be Seen: Towards Stealthy Active Search in Heterogeneous Multi-Robot Systems Abstract: Robotic solutions for quick disaster response are essential to ensure minimal loss of life, especially when the search area is too dangerous or too vast for human rescuers. We model this problem as an asynchronous multi-agent active-search task where each robot aims [...]

Title: Spatial Tasking in Human-Robot Collaborative Exploration Abstract: This work develops a methodology for collaborative human-robot exploration that leverages implicit coordination. Most autonomous single- and multi-robot exploration systems require a remote operator to provide explicit guidance to the robot team. Few works consider how to integrate the human partner alongside robots to provide guidance in the [...]

Title: Towards Video-based Physiology Estimation Abstract: RGB-video based human physiology estimation has a wide range of practical applications in telehealth, sports and deep fake detection. Therefore, researchers in the community have collected several video datasets and have advanced new methods over the years. In this dissertation, we study these methods extensively and aim to address the [...]

Title: Automating Ultrasound Based Vascular Access Abstract: Timely care of trauma patients is important to prevent casualties in resource-limited regions such as the battlefield. In order to treat such trauma using point of care diagnosis, medical practitioners typically use an ultrasound for vascular access or detection of subcutaneous splinters for providing critical care. The problem here is two-fold: [...]

Title: Analogical Networks: Memory-Modulated In-Context 3D Parsing Abstract: Recent advances in the applications of deep neural networks to numerous visual perception tasks have shown excellent performance. However, this generally requires access to large amount of training samples and hence one persistent challenge is the setting of few-shot learning. In most existing works, a separate parametric neural [...]

Title: ARC: AdveRsarial Calibration between Modalities Abstract: Advances in computer vision and machine learning techniques have led to flourishing success in RGB-input perception tasks, which has also opened unbounded possibilities for non-RGB-input perception tasks, such as object detection from wireless signals, point clouds, and infrared light. However, compared to the matured development pipeline of RGB-input [...]

Title: Data-Driven Slip Model for Improved Localization and Path Following applied to Lunar Micro-Rovers Abstract Micro-lunar rovers need to solve a slew of challenges on the Moon, with no human intervention. One such challenge is the need to know their location in order to navigate and build maps. However, localization is challenging on the moon due [...]