Carnegie Mellon University
‘Unboxing’ anomaly detection and panoptic segmentation
Abstract: Panoptic segmentation is a recent problem in computer vision that attempts to classify each pixel in an image according to its semantic and instance label (accomplishing both semantic segmentation and instance segmentation respectively). Most existing panoptic and instance segmentation methods run a detection-first pipeline, where a bounding box is placed around an object and [...]
Carnegie Mellon University
Self-Supervised Learning on Mobile Robots Using Acoustics, Vibration, and Visual Models to Build Rich Semantic Terrain Maps
Abstract: Humans and robots would benefit from having rich semantic maps of the terrain in which they operate. Mobile robots equipped with sensors and perception software could build such maps as they navigate through a new environment. This information could then be used by humans or robots for better localization and path planning, as well [...]
Self-Driving Cars & AI: Transforming our Cities and our Lives
Abstract: Recent algorithmic and hardware improvements resulted in several success stories in the field of Artificial Intelligence (AI) which impact our daily lives. However, despite its ubiquity, AI is only just starting to make advances in what may arguably have the largest societal impact thus far, the nascent field of autonomous driving. At Uber ATG, [...]
Carnegie Mellon University
Transfers Between Multiple Service Robots
Abstract: With the deployment of more robots, human-robot interaction will no longer be limited to a one-to-one interaction between a user and a robot. Instead, users will likely have to interact with multiple robots, simultaneously or sequentially, throughout their day to receive services and complete different tasks. In this thesis proposal, I am proposing joint [...]
Carnegie Mellon University
Machine Learning Parallelism Could Be Adaptive, Composable and Automated
Abstract: In recent years, the pace of innovations in the fields of machine learning has accelerated. To cope with the sheer computational complexity of training large ML models on large datasets, researchers in SysML have created algorithms and systems that parallelize ML training and inference over multiple CPUs or GPUs, or even multiple computing nodes [...]
MSR Thesis Talk – Matt Martone
Title: Design and Control of a Large Modular Hexapod Abstract: Legged robotic systems have made great strides in recent years, but unlike wheeled robots, limbed locomotion does not scale well. Long legs demand huge torques, driving up actuator size and onboard battery mass. This relationship results in massive structures that lack the safety, portability, [...]
Improving Robot and Deep Reinforcement Learning via Quality Diversity and Open-Ended Algorithms
Abstract: Quality Diversity (QD) algorithms are those that seek to produce a diverse set of high-performing solutions to problems. I will describe them and a number of their positive attributes. I will then summarize our Nature paper on how they, when combined with Bayesian Optimization, produce a learning algorithm that enables robots, after being damaged, to adapt in 1-2 minutes [...]
Carnegie Mellon University
Online Kinodynamic Planning for Teams of Aerial Robots in 3-D Workspaces
Abstract: An efficient online planning or replanning methodology is a critical requirement for scalable and responsive real world multi-robot deployments. The need to replan typically stems from the invalidation of existing plans due to incomplete knowledge of the environment, or, from scenarios that necessitate changing goal locations in response to evolving application requirements. In this [...]
Go, fastMRI, and Minecraft: Exploring the limits of AI
Abstract: The application of AI across various domains demonstrates both the promise of existing techniques but also their limitations. In this talk, I explore three recent projects and how they shed light on the progress of AI and the challenges to come. These projects include ELF OpenGo a reimplementation of AlphaZero, fastMRI for reducing the time [...]
Robotics Institute Administrative Staff Winter Tree Lunch
Please join us for our annual Robotics Institute Administrative Staff Winter Tree Decorating Lunch. A light lunch will be provided but staff-created treats will always be welcomed.
Carnegie Mellon University
Expressive Real-time Intersection Scheduling: New Methods for Adaptive Traffic Signal Control
Abstract: Traffic congestion is a widespread problem throughout global metropolitan areas. In this thesis, we consider methods to optimize the performance of traffic signals to reduce congestion. We begin by presenting Expressive Real-time Intersection Scheduling (ERIS), a schedule-driven intersection control strategy that runs independently on each intersection in a traffic network. For each intersection, ERIS [...]
Carnegie Mellon University
Open-world 3D Object Detection
Abstract: Perception for autonomous robots presents a set of unique challenges: finding the right representation for 3D signals, adapting to an open-world setting, and exploiting geometric priors. Successfully detecting objects regardless of their labels lays a solid foundation for safe navigation. I will present two of my recent works in this line. First, I will [...]
Towards Weakly-Supervised Visual Understanding
Abstract: Learning with weak and self-supervisions recently emerged as compelling tools towards leveraging vast amounts of unlabeled or partially-labeled data. In this talk, I will present some of the latest advances in weakly-supervised visual scene understanding from NVIDIA. Specifically, I will summarize and discuss some challenges and potential solutions in weakly-supervised learning, and introduce our [...]
MSR Thesis Talk: Jenny Nan
Title: Combining Deep Learning and Verification for Precise Object Instance Detection Abstract: Deep learning based object detectors often return false positives with very high confidence. Although they optimize generic detection performance, such as mean average precision (mAP), they are not designed for reliability. For a reliable detection system, if a high confidence detection is [...]
Carnegie Mellon University
MSR Thesis talk – Vasu Agrawal
Title: Ground Up Design of a Multi-modal Object Localization System Abstract: Rapid situational awareness is the key to enabling a successful response from first responders during an emergency, where time is of the essence. Emergency personnel are often sent into incident scenes to gather information, but this is often a dangerous and slow process. Subterranean environments [...]
MSR Thesis Talk – Swaminathan Gurumurthy
Title: Improving generalization in data-driven models with task-specific knowledge Abstract: With the rise of the over-parameterized deep learning models and massive datasets, many have started advocating towards minimizing the amount of prior knowledge added to a learning model. Ironically, the traditional machine learning community advocated for exactly the opposite. Whereas the latter assumes knowledge of [...]
RI Winter Party
Robotics Institute Winter Party Please join us for some fun, food, beverages and conversation! All RI faculty, staff, students and visitors are invited to the Robotics Institute Winter Party! We apologize but due to space limitations in the Atrium we regretfully cannot include family or other non-RI guests.
Imaging without focusing: A computational approach to miniaturizing cameras
Abstract: Miniaturization of cameras is key to enabling new applications in areas such as connected devices, wearables, implantable medical devices, in vivo microscopy, and micro-robotics. Recently, lenses were identified as the main bottleneck in miniaturization of cameras. Standard smaller lens-system camera modules have a thickness of about 10 mm or higher, and reducing the size [...]
MRSD Annual Poster Presentation
Nine RI MRSD program student teams will use posters, videos, and hardware to show their project work on truck crash avoidance, balance recovery, construction via UAVs, room tidying, crop disease detection, beach cleaning, temperature field sensing, UGV-UAV firefighting, and multirover moon pit modeling. Website: https://mrsd.ri.cmu.edu/project-examples/student-project-websites/spring-2019-fall-2019/
Adaptive Planning and Control of Wheeled Mobile Robots in Challenging Environments
Abstract: Over the last two decades, we have seen driverless cars conquer the Mojave desert, drive on mars and operate on our streets and warehouses. One of the most fundamental requirements of such robots is their ability to navigate their environment with minimal human oversight. As more robots graduate from the confines of laboratories to [...]
Towards photo-realistic face digitization from monocular videos
Abstract: Recent advances in face capture now enable digitizing high-quality 3D faces for the entertainment industry. Standardized digitization solutions, however, require tailor-made capture systems and extensive manual work, making them expensive and hard to deploy. With the advent of commodity sensors, new lightweight approaches that push the boundaries of human digitization have been introduced, slowly [...]
When to use CNNs for Inverse Problems in Vision
Abstract: Reconstruction tasks in computer vision aim fundamentally to recover an undetermined signal from a set of noisy measurements. Examples include super-resolution, image denoising, and non-rigid structure from motion\cite{Kong_2019}, all of which have seen recent advancements through deep learning. However, earlier work made extensive use of sparse signal reconstruction frameworks (e.g. convolutional sparse coding). While [...]
Toward telelocomotion: human sensorimotor control of contact-rich robot dynamics
Abstract: Human interaction with the physical world is increasingly mediated by automation -- planes assist pilots, cars assist drivers, and robots assist surgeons. Such semi-autonomous machines will eventually pervade our world, doing dull and dirty work, assisting the elderly and disabled, and responding to disasters. Recent results (e.g. from the DARPA Robotics Challenge) demonstrate that, [...]
2020 RI Faculty Dinner
Invitation with information will be emailed to invitees.
Formal Synthesis for Robots
Abstract: In this talk I will describe how formal methods such as synthesis – automatically creating a system from a formal specification – can be leveraged to design robots, explain and provide guarantees for their behavior, and even identify skills they might be missing. I will discuss the benefits and challenges of synthesis techniques and [...]
Reconstructing 3D Human Avatars from Monocular Images
Abstract: Statistical 3D human body models have helped us to better understand human shape and motion and already enabled exciting new applications. However, if we want to learn detailed, personalized, and clothed models of human shape, motion, and dynamics, we require new approaches that learn from ubiquitous data such as plain RGB-images and video. I [...]
Carnegie Mellon University
Learning Dense 3D Object Reconstruction without Geometric Supervision
Abstract: Geometric alignment across visual data has been the fundamental issue for effective and efficient computer vision algorithms. The established pixel correspondences between images indirectly infer the underlying 3D geometry, physically or semantically. While this builds the foundation of classical multi-view 3D reconstruction algorithms such as Structure from Motion (SfM) and Simultaneous Localization and Mapping [...]
Extreme Motions in Biological and Engineered Systems
Abstract: Dr. Temel’s work mainly focuses on understanding the dynamics and energetics of extreme motions in small-scale natural and synthetic systems. Small-scale biological systems achieve extraordinary accelerations, speeds, and forces that can be repeated with minimal costs throughout the life of the organism. Zeynep uses analytical and computational models as well as physical prototypes to learn about these systems, test [...]
Reasoning about complex media from weak multi-modal supervision
Abstract: In a world of abundant information targeting multiple senses, and increasingly powerful media, we need new mechanisms to model content. Techniques for representing individual channels, such as visual data or textual data, have greatly improved, and some techniques exist to model the relationship between channels that are “mirror images” of each other and contain [...]
Spring 2020 RI Reappointment & Promotion Preview Meeting – For RI Voting Faculty Only
Vision Tool Seminar: Hydra
Abstract: Hydra is an open-source Python framework developed at FAIR that aims to reduce the amount of boilerplate code in research and other complex applications. The key feature is the ability to dynamically create a hierarchical configuration by composition and override it through config files and the command line. The name Hydra comes from its [...]
CANCELLED
Building Trust in Real World Applications of Vision Based Machine Learning
Abstract: In all machine learning problems, there is an explicit trade off between cost and benefit. In real world vision problems, this optimization becomes increasingly difficult since those trade offs directly impact technology and product development as well as business strategy. For any successful business case, it is critical that the cost/benefit trade offs in [...]
Tendon Driven Foam Hands
Abstract: There has been great progress in soft robot design, manufacture, and control in recent years, and soft robots are a tool of choice for safe and robust handling of objects in conditions of uncertainty. Still, dexterous in-hand manipulation using soft robots remains a challenge. This talk introduces a novel class of soft robots in [...]
Knowledge Infused Deep Learning
Abstract: This talk is motivated by the following thesis: Background knowledge is key to intelligent decision making. While deep learning methods have made significant strides over the last few years, they often lack the context in which they operate. Knowledge Graphs (and more generally multi-relational graphs) provide a flexible framework to capture and represent knowledge [...]
Carnegie Mellon University
Towards a Good Representation For Reinforcement Learning
Abstract: Deep reinforcement learning has achieved many successes over the recent years. However, its high sample complexity and the difficulty in specifying a reward function have limited its application. In this talk, I will take a representation learning perspective towards these issues. Is it possible to map from the raw observation, potentially in high dimension, [...]
Yes, That’s a Robot in Your Grocery Store. Now what?
Abstract: Retail stores are becoming ground zero for indoor robotics. Fleet of different robots have to coexist with each others and humans every day, navigating safely, coordinating missions, and interacting appropriately with people, all at large scale. For us roboticists, stores are giant labs where we're learning what doesn't work and iterating. If we get [...]
Learning to Reconstruct 3D Humans
Abstract: Recent advances in 2D perception have led to very successful systems, able to estimate the 2D pose of humans with impressive robustness. However, our interactions with the world are fundamentally 3D, so to be able to understand, explain and predict these interactions, it is crucial to reconstruct people in 3D. In this talk, I [...]
Carnegie Mellon University
Eye Gaze for Assistive Manipulation
Abstract: Full robot autonomy is the traditional goal of robotics research. To work in a human-inhabited world, however, robots will often need to collaborate with humans. Many scenarios require human users to teleoperate robots to perform tasks, a paradigm that appears everywhere from space exploration, to disaster recovery, to assistive robotics. This collaboration enables tasks [...]
CANCELLED
Abstract: Before learning robots can be deployed in the real world, it is critical that probabilistic guarantees can be made about the safety and performance of such systems. In recent years, safe reinforcement learning algorithms have enjoyed success in application areas with high-quality models and plentiful data, but robotics remains a challenging domain for scaling [...]
Deep Learning for Understanding Dynamic Visual Data
Abstract: Perceiving dynamic environments from visual inputs allows autonomous agents to understand and interact with the world and is a core topic in Artificial Intelligence. The success of deep learning motivates us to apply deep learning techniques to the perception of dynamic visual data. However, how to design and apply deep neural networks to effectively [...]
Carnegie Mellon University
Stability-Centric Mechanics for Rigid Body Manipulation
Abstract: The repertoire of human manipulation is filled with creative use of contacts to move the object about the hand and the environment. It’s the combination of these skills that makes human manipulation dexterous. However, in most robotic applications the robot just fix all contact points on the object and do grasping. Reliable robot manipulation [...]
Optimizing for coordination with people
https://youtu.be/AQ-w5o2oGI8 Abstract: From autonomous cars to quadrotors to mobile manipulators, robots need to co-exist and even collaborate with humans. In this talk, we will explore how our formalism for decision making needs to change to account for this interaction, and dig our heels into the subtleties of modeling human behavior -- sometimes strategic, often irrational, [...]
Analyzing Grasp Contact via Thermal Imaging
Abstract: Grasping and manipulating objects is an important human skill. Because contact between hand and object is fundamental to grasping, measuring it can lead to important insights. However, observing contact through external sensors is challenging because of occlusion and the complexity of the human hand. I will discuss the use of thermal cameras to capture [...]
Numerical Methods for Things That Move: From Quadrupeds to Starships
Abstract: Recent advances in motion planning and control have led to dramatic successes like SpaceX’s rocket landings and Boston Dynamics’ humanoid robot acrobatics. However, the underlying numerical methods used in these applications are typically decades old, not tuned for high performance on planning and control problems, and are often unable to cope with the types [...]
Carnegie Mellon University
Combining Multiple Heuristics: Studies on Neighborhood-base Heuristics and Sampling-based Heuristics
Abstract: This thesis centers on the topic of how to automatically combine multiple heuristics. For most computationally challenging problems, there exist multiple heuristics, and it is generally the case that any such heuristic exploits only a limited number of aspects among all the possible problem characteristics that we can think of, and by definition, is [...]
Fast Foveation for LIDARs, Projectors and Cameras
Abstract: Most cameras today capture images without considering scene content. In contrast, animal eyes have fast mechanical movements that control how the scene is imaged in detail by the fovea, where visual acuity is highest. This concentrates computational (i.e. neuronal) resources in places where they are most needed. The prevalence of foveation, and the wide [...]