Faculty Events

2020 RI Faculty Dinner

Pittsburgh Golf Club 5280 Northumberland Street, Pittsburgh, PA, United States

Invitation with information will be emailed to invitees.

RI Seminar
Hadas Kress-Gazit
Associate Professor
College of Engineering, Cornell University

Formal Synthesis for Robots

Abstract: In this talk I will describe how formal methods such as synthesis – automatically creating a system from a formal specification – can be leveraged to design robots, explain and provide guarantees for their behavior, and even identify skills they might be missing. I will discuss the benefits and challenges of synthesis techniques and [...]

VASC Seminar
Thiemo Alldieck
PhD Candidate
Facebook Reality Labs

Reconstructing 3D Human Avatars from Monocular Images

GHC 6501

Abstract:  Statistical 3D human body models have helped us to better understand human shape and motion and already enabled exciting new applications. However, if we want to learn detailed, personalized, and clothed models of human shape, motion, and dynamics, we require new approaches that learn from ubiquitous data such as plain RGB-images and video. I [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Learning Dense 3D Object Reconstruction without Geometric Supervision

GHC 6501

Abstract: Geometric alignment across visual data has been the fundamental issue for effective and efficient computer vision algorithms. The established pixel correspondences between images indirectly infer the underlying 3D geometry, physically or semantically. While this builds the foundation of classical multi-view 3D reconstruction algorithms such as Structure from Motion (SfM) and Simultaneous Localization and Mapping [...]

RI Seminar
Assistant Professor
Robotics Institute,
Carnegie Mellon University

Extreme Motions in Biological and Engineered Systems

1305 Newell Simon Hall

Abstract: Dr. Temel’s work mainly focuses on understanding the dynamics and energetics of extreme motions in small-scale natural and synthetic systems. Small-scale biological systems achieve extraordinary accelerations, speeds, and forces that can be repeated with minimal costs throughout the life of the organism. Zeynep uses analytical and computational models as well as physical prototypes to learn about these systems, test [...]

VASC Seminar
Adriana Kovashka
Assistant Professor
University of Pittsburgh

Reasoning about complex media from weak multi-modal supervision

GHC 6501

Abstract:  In a world of abundant information targeting multiple senses, and increasingly powerful media, we need new mechanisms to model content. Techniques for representing individual channels, such as visual data or textual data, have greatly improved, and some techniques exist to model the relationship between channels that are “mirror images” of each other and contain [...]

RI Event
Omry Yadan
Research Engineer
Facebook AI Research

Vision Tool Seminar: Hydra

3305 Newell-Simon Hall

Abstract: Hydra is an open-source Python framework developed at FAIR that aims to reduce the amount of boilerplate code in research and other complex applications. The key feature is the ability to dynamically create a hierarchical configuration by composition and override it through config files and the command line. The name Hydra comes from its [...]

RI Seminar
Raquel Urtasun
Chief Scientist & Head
Uber Advanced Technologies Group Toronto

CANCELLED

CIC Building Room 1201

VASC Seminar
Benjamin Schmidt
President and Co-Founder
RoadBotics

Building Trust in Real World Applications of Vision Based Machine Learning

GHC 6501

Abstract:  In all machine learning problems, there is an explicit trade off between cost and benefit. In real world vision problems, this optimization becomes increasingly difficult since those trade offs directly impact technology and product development as well as business strategy. For any successful business case, it is critical that the cost/benefit trade offs in [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Tendon Driven Foam Hands

GHC 6501

Abstract: There has been great progress in soft robot design, manufacture, and control in recent years, and soft robots are a tool of choice for safe and robust handling of objects in conditions of uncertainty. Still, dexterous in-hand manipulation using soft robots remains a challenge. This talk introduces a novel class of soft robots in [...]

VASC Seminar
Partha Pratim Talukdar
Associate Professor
IIScBangalore / Founder, KENOME

Knowledge Infused Deep Learning

Newell-Simon Hall 4305

Abstract:  This talk is motivated by the following thesis: Background knowledge is key to intelligent decision making. While deep learning methods have made significant strides over the last few years, they often lack the context in which they operate. Knowledge Graphs (and more generally multi-relational graphs) provide a flexible framework to capture and represent knowledge [...]

PhD Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Towards a Good Representation For Reinforcement Learning

WEH 5421

Abstract: Deep reinforcement learning has achieved many successes over the recent years. However, its high sample complexity and the difficulty in specifying a reward function have limited its application. In this talk, I will take a representation learning perspective towards these issues. Is it possible to map from the raw observation, potentially in high dimension, [...]

RI Seminar
Sarjoun Skaff
Co-Founder & CTO
Bossa Nova Robotics

Yes, That’s a Robot in Your Grocery Store. Now what?

CIC Building Room 1201

Abstract: Retail stores are becoming ground zero for indoor robotics. Fleet of different robots have to coexist with each others and humans every day, navigating safely, coordinating missions, and interacting appropriately with people, all at large scale. For us roboticists, stores are giant labs where we're learning what doesn't work and iterating. If we get [...]

VASC Seminar
Georgios Pavlakos
PhD Student
University of Pennsylvania

Learning to Reconstruct 3D Humans

GHC 6501

Abstract:  Recent advances in 2D perception have led to very successful systems, able to estimate the 2D pose of humans with impressive robustness. However, our interactions with the world are fundamentally 3D, so to be able to understand, explain and predict these interactions, it is crucial to reconstruct people in 3D. In this talk, I [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Eye Gaze for Assistive Manipulation

NSH 4305

Abstract: Full robot autonomy is the traditional goal of robotics research. To work in a human-inhabited world, however, robots will often need to collaborate with humans. Many scenarios require human users to teleoperate robots to perform tasks, a paradigm that appears everywhere from space exploration, to disaster recovery, to assistive robotics. This collaboration enables tasks [...]

RI Seminar
Scott Niekum
Assistant Professor
Department of Computer Science, University of Texas at Austin

CANCELLED

Abstract: Before learning robots can be deployed in the real world, it is critical that probabilistic guarantees can be made about the safety and performance of such systems.  In recent years, safe reinforcement learning algorithms have enjoyed success in application areas with high-quality models and plentiful data, but robotics remains a challenging domain for scaling [...]

VASC Seminar
Xingyu Liu
PhD Student
Stanford University

Deep Learning for Understanding Dynamic Visual Data

GHC 6501

Abstract:  Perceiving dynamic environments from visual inputs allows autonomous agents to understand and interact with the world and is a core topic in Artificial Intelligence. The success of deep learning motivates us to apply deep learning techniques to the perception of dynamic visual data. However, how to design and apply deep neural networks to effectively [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Stability-Centric Mechanics for Rigid Body Manipulation

NSH 4305

Abstract: The repertoire of human manipulation is filled with creative use of contacts to move the object about the hand and the environment. It’s the combination of these skills that makes human manipulation dexterous. However, in most robotic applications the robot just fix all contact points on the object and do grasping. Reliable robot manipulation [...]

RI Seminar
Anca Dragan
Assistant Professor
EECS Department, University of California Berkeley

Optimizing for coordination with people

1305 Newell Simon Hall

https://youtu.be/AQ-w5o2oGI8 Abstract: From autonomous cars to quadrotors to mobile manipulators, robots need to co-exist and even collaborate with humans. In this talk, we will explore how our formalism for decision making needs to change to account for this interaction, and dig our heels into the subtleties of modeling human behavior -- sometimes strategic, often irrational, [...]

VASC Seminar
James Hays
Associate Professor
Georgia Institute of Technology

Analyzing Grasp Contact via Thermal Imaging

GHC 6501

Abstract:  Grasping and manipulating objects is an important human skill. Because contact between hand and object is fundamental to grasping, measuring it can lead to important insights. However, observing contact through external sensors is challenging because of occlusion and the complexity of the human hand. I will discuss the use of thermal cameras to capture [...]

Special Talk
Zac Manchester
Assistant Professor
Stanford University

Numerical Methods for Things That Move: From Quadrupeds to Starships

Newell-Simon Hall 4305

Abstract:  Recent advances in motion planning and control have led to dramatic successes like SpaceX’s rocket landings and Boston Dynamics’ humanoid robot acrobatics. However, the underlying numerical methods used in these applications are typically decades old, not tuned for high performance on planning and control problems, and are often unable to cope with the types [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Combining Multiple Heuristics: Studies on Neighborhood-base Heuristics and Sampling-based Heuristics

NSH 4305

Abstract: This thesis centers on the topic of how to automatically combine multiple heuristics. For most computationally challenging problems, there exist multiple heuristics, and it is generally the case that any such heuristic exploits only a limited number of aspects among all the possible problem characteristics that we can think of, and by definition, is [...]

VASC Seminar
Sanjeev J. Koppal
Assistant Professor
University of Florida

Fast Foveation for LIDARs, Projectors and Cameras

GHC 6501

Abstract:  Most cameras today capture images without considering scene content. In contrast, animal eyes have fast mechanical movements that control how the scene is imaged in detail by the fovea, where visual acuity is highest. This concentrates computational (i.e. neuronal) resources in places where they are most needed. The prevalence of foveation, and the wide [...]

Special Events

POSTPONED – 2020 Robotics Institute Semi-formal

Pittsburgh Golf Club 5280 Northumberland Street, Pittsburgh, PA, United States

Due to uncertainty and developing conditions of the coronavirus (COVID-19) outbreak, the Robotics Institute at Carnegie Mellon has determined it best to postpone our planned RI Semi-formal activity on March 20, 2020. More information will be sent out once a new date has been selected. __________________POSTPONED _____________________ By invitation only: The 2020 Robotics Institute Semi-formal [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Robotic Grasping in the Wild

Zoom Link Abstract Robotics and artificial intelligence have witnessed tremendous progress in the past decade. Yet, we are still far from building the general purpose robot butler that can autonomously operate in homes and help with manipulation tasks like household chores. Grasping is an important action primitive for manipulation and needs to generalize to unstructured [...]

PhD Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Resource-constrained learning and inference for visual perception

Zoom Link Abstract Real-world applications usually require computer vision algorithms to meet certain resource constraints. In this talk, I will present evaluation methods and principled solutions for both cases of training and testing. First, I will talk about a formal setting for studying training under the non-asymptotic, resource-constrained regime, i.e., budgeted training. We analyze the [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Multi-hypothesis iSAM2 for Ambiguity-aware Passive and Active SLAM

Archived video Abstract Simultaneous localization and mapping (SLAM) is the problem of estimating the state of a moving agent with sensors on it while simultaneously reconstructing a map of its surrounding environment, which has been a popular research field due to its wide applications. As many state-of-the-art SLAM algorithms can already achieve high accuracy in [...]

VASC Seminar
Jia-Bin Huang
Assistant Professor
Virginia Tech

Learning to See Through Occlusions and Obstructions

Virtual VASC:  https://cmu.zoom.us/j/249106600   Abstract:  Photography allows us to capture and share memorable moments of our lives. However, 2D images appear flat due to the lack of depth perception and may suffer from poor imaging conditions such as taking photos through reflecting or occluding elements. In this talk, I will present our recent efforts to [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Terrain Relative Navigation for Lunar Polar Roving: Exploiting Geometry, Shadows, and Planning

Archived Zoom Video Abstract Water ice at the lunar poles could be the most valuable resource beyond planet Earth. However, that value is not foregone, and can only be determined by rovers that evaluate the distributions of abundance, concentration, and characteristics of this ice. The near-term explorations will be solar and unlikely to endure night, [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Resource-Constrained State Estimation with Multi-Modal Sensing

Zoom Link Accurate and reliable state estimation is essential for safe mobile robot operation in real-world environments because ego-motion estimates are required by many critical autonomy functions such as control, planning, and mapping. Computing accurate state estimates depends on the physical characteristics of the environment, the selection of suitable sensors to capture that information, and [...]

VASC Seminar
Yuxin Wu
Research Engineer
Facebook AI Research

Detectron2 in Object Detection Research

Virtual VASC:  https://cmu.zoom.us/j/249106600   Abstract:  Detectron2 is Facebook's library for object detection and segmentation. It has been used widely in FAIR's research and Facebook's products. This talk will introduce detectron2 with a focus on its use in object detection research, including the lessons we learned from building it, as well as the new research enabled [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Aditya Dhawale – MSR Thesis Talk

TBA

Title: Hierarchical Gaussian Distributions for Real-Time SLAM ZOOM Link: (Virtual Presentation) https://cmu.zoom.us/j/7210519673 Abstract: We present Gaussian distributions as structure primitives in a hierarchical multi-fidelity framework to enable accurate real-time Simultaneous Localization and Mapping (SLAM) using uncertain depth data. Real-time mapping and localization capabilities on a mobile robot can enable deployment of robots in real-world scenarios. An autonomous system must [...]

VASC Seminar
Olga Russakovsky
Assistant Professor
Department of Computer Science, Princeton University

Fairness in visual recognition

Virtual VASC Seminar:  https://cmu.zoom.us/j/249106600   Abstract: Computer vision models trained on unparalleled amounts of data hold promise for making impartial, well-informed decisions in a variety of applications. However, more and more historical societal biases are making their way into these seemingly innocuous systems. Visual recognition models have exhibited bias by inappropriately correlating age, gender, sexual [...]

PhD Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Planning and Execution using Inaccurate Models with Provable Guarantees

Zoom Link Abstract: Models used in modern planning problems to simulate outcomes of real world action executions are becoming increasingly complex, ranging from simulators that do physics-based reasoning to precomputed analytical motion primitives. However, robots operating in the real world often face situations not modeled by these models before execution. This imperfect modeling can lead [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Hybrid Soft Sensing in Robotic Systems

Zoom Link Abstract: The desire to operate robots in unstructured environments, side-by-side with humans, has created a demand for safe and robust sensing skins. Largely inspired by human skin, the ultimate goal of electronic skins is to measure diverse sensory information, conform to surfaces, and avoid interfering with the natural mechanics of the host or [...]

PhD Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

The Effect of Locomotion Configuration on Discrete Obstacle Traversal for a Small Tracked Vehicle

Zoom Link Abstract: As mobile robots are being designed for increasingly rugged and unknown terrain, mechanical reconfigurability presents one possibility for improving vehicle efficiency and mobility. To validate this idea, we created an 18.5-kg modular tracked vehicle with adjustable track tension, track width, track length, and sprocket diameter. In this talk, I will explain the [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Qian Long – MSR Thesis Talk

TBA

ZOOM Link: https://cmu.zoom.us/j/7263914910   Title: Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning Abstract: In multi-agent games, the complexity of the environment can grow exponentially as the number of agents increases, so it is particularly challenging to learn good policies when the agent population is large. We introduce Evolutionary Population Curriculum (EPC), a curriculum learning [...]

MSR Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Wen-Hsuan Chu – MSR Thesis Talk

TBA

ZOOM Link: https://cmu.zoom.us/j/4417558334 Title: Neural Batch Sampling with Reinforcement Learning for Semi-Supervised Anomaly Detection Abstract: We are interested in the detection and segmentation of anomalies in images where the anomalies are typically small (i.e., a small tear in woven fabric, bro-ken pin of an IC chip). From a statistical learning point of view, anomalies have [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Chendi Lin – MSR Thesis Talk

TBA

Zoom Link: https://cmu.zoom.us/j/95571441174   Title: Online Connectivity-aware Dynamic Distribution for Heterogeneous Multi-Robot Systems   Abstract: In many multi-robot applications the robot team needs to execute multiple tasks simultaneously with different task-related controllers. To ensure effective coordination and at the same time avoid collisions, the robots have to stay connected. In this work, we consider the problem where a [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Robert Li – MSR Thesis Talk

TBA

Zoom Link: https://cmu.zoom.us/j/91465601940   Title: Solving Puzzles Like A Human With Two Stage Random Search   Abstract: Humans are remarkably good at solving novel physical puzzles and tasks, with only a basic understanding of abstract concepts like kinematics, gravity, mass, friction, and inertia. We wanted to replicate how a human would explore the search space of a problem. [...]

MSR Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Samantha Speer – MSR Thesis Talk

TBA

Zoom Link: https://cmu.zoom.us/j/98546775449   Title: Grounding Abstract Concepts With Robotic Manipulatives   Abstract: Technology in education has been on the rise for a long time, developing from computer manipulatives to mobile apps and finally into robotics. Robotics has the unique affordances of the classic physical manipulatives and virtual manipulative, providing both a physical aspect along with [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Vaibhav (Vai) Viswanathan – MSR Thesis Talk

TBA

Zoom Link: https://cmu.zoom.us/j/2112607862   Title: Bitwise Trajectory Elimination: An Efficient Method for Filtering Trajectory Libraries for Quadrotor Navigation   Abstract: Quadrotor flight in unknown environments is challenging due to the limited range of perception sensors, state estimation drift, and limited onboard computation. In this work, we tackle these challenges by proposing an efficient, reactive planning approach. We introduce [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Xi (Sandy) Sun – MSR Thesis Talk

TBA

Zoom link: https://cmu.zoom.us/j/94541819048   Title: Visual-Inertial Source Localization for Co-Robot Rendezvous   Abstract: We aim to enable robots to visually localize a target person through the aid of an additional sensing modality -- the target person's 3D inertial measurements. The need for such technology may arise when a robot is to meet a person in [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Harsh Agarwal – MSR Thesis Talk

Zoom Link: https://cmu.zoom.us/j/99544484313   Title   DeepBLE - Generalizing RSSI based Localization Across Different Devices   Abstract   Accurate smartphone localization ( < 1-meter error) for indoor navigation using only RSSI received from a set of BLE beacons remains a challenging problem, due to the inherent noise of RSSI measurements. To overcome the large variance in [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Xia Chen – MSR Thesis Talks

ZOOM Link: https://cmu.zoom.us/j/93785335144   Title: Combining Semantic and Geometric Understanding for Modern Visual Recognition Tasks Abstract: For autonomous driving perception, visual data, such as camera image and LiDAR point cloud, consists of two aspects: semantic feature and geometric structure. While usually studied separately, these two properties can be combined and jointly used by a unified framework. [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Himanshi Yadav – MSR Thesis Talk

TBA

Zoom Link: https://cmu.zoom.us/j/96397153508 Title: A Comprehensive Study of Unsupervised Classification Techniques for Hyperspectral Datasets Abstract: Unsupervised learning and in this specific research, clustering regional composition in hyperspectral images, poses significant challenges in the fields of machine learning and remote sensing. Hyperspectral images capture the spectral information in many wavelengths, as opposed to typical images that [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Alan Zhao – MSR Thesis Talk

Zoom Link: https://cmu.zoom.us/j/98144379626   Title   Learning Precise and Task-oriented Grasps for Robotic Assembly   Abstract   Robust, precise, and task-oriented grasp planning is vital for autonomous robotic assembly. It requires reasoning about the object geometry and preconditions of a task so as to properly grasp an object and complete the down-stream tasks. However, achieving such [...]

PhD Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Task-Driven Modular Networks for Zero-Shot Compositional Learning

Zoom Link Abstract: One of the hallmarks of human intelligence is the ability to compose learned knowledge into novel concepts which can be recognized without a single training example. In contrast, current state-of-the-art methods require hundreds of training examples for each possible category to build reliable and accurate classifiers. To alleviate this striking difference in [...]

VASC Seminar
Qi Guo
PhD Student
Harvard University

Bio-inspired depth sensing using computational optics

Virtual Seminar:  https://cmu.zoom.us/j/249106600   Abstract:  Jumping spiders rely on accurate depth perception for predation and navigation. They accomplish depth perception, despite their tiny brains, by using specialized optics. Each principal eye includes a multitiered retina that simultaneously receives multiple images with different amounts of defocus, and distance is decoded from these images with seemingly little [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Active Vision: Autonomous Aerial Cinematography with Learned Artistic Decision-Making

Zoom Link Abstract: Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. Fundamentally, it is a tool with immense potential to improve human creativity, expressiveness, and sharing of experiences. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Efficient Robot Decision-Making for Achieving Multiple Independent Tasks

Zoom Link Abstract: We focus on robotics applications where a robot is required to accomplish a set of tasks that are partially observable and evolve independently of each other according to their dynamics. One such domain that we target in this work is decision-making for a robot waiter waiting tables at a restaurant. The robot [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Image to LiDAR Map Registration using Late Feature Projection

Zoom Link Abstract: Accurate localization is essential for autonomous operation in many problem domains. This is most often performed by comparing LiDAR scans collected in real-time to a HD point cloud based map. While this enables centimeter-level accuracy, it depends on an expensive LiDAR sensor at run time. Recently, efforts have been underway to reduce [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Vision with Small Baselines

Zoom Link Abstract: 3D sensing with portable imaging systems is becoming more and more popular in computer vision applications such as autonomous driving, virtual reality, robotics manipulation and surveillance, due to the decreasing expense and size of RGB cameras. Despite the compactness and portability of the small baseline vision systems, it is well-known that the [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Provably Constant-time Motion Planning

Zoom Link Abstract: In manufacturing and warehouse scenarios, robots often perform recurring manipulation tasks in structured environments. Fast and reliable motion planning is one of the key elements that ensure efficient operations in such environments. A very common example scenario is of manipulators working at conveyor belts, where they have limited time to pick moving [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Humans In Their Natural Habitat: Training AI to Understand People

Zoom Link Abstract: Computer vision has a great potential to help our daily lives by searching for lost keys, watering flowers or reminding us to take a pill. To succeed with such tasks, computer vision methods need to be trained from real and diverse examples of our daily dynamic scenes. First, we need to give [...]

VASC Seminar
Gemma Roig
Assistant Professor
Department of Computer Science, Goethe University Frankfurt

Task-specific Vision DNN Models and Their Relation for Explaining Different Areas of the Visual Cortex

Virtual VASC Seminar:  https://cmu.zoom.us/j/249106600   Abstract:  Deep Neural Networks (DNNs) are state-of-the-art models for many vision tasks. We propose an approach to assess the relationship between visual tasks and their task-specific models. Our method uses Representation Similarity Analysis (RSA), which is commonly used to find a correlation between neuronal responses from brain data and models. [...]

PhD Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

A Theory of Fermat Paths for Non-line-of-sight Shape Reconstruction

Zoom Link Abstract: Traditionally, computer vision systems and algorithms, such as stereo vision, and shape from shading, have been developed to mimic human vision. As a consequence, a lot of these systems operate under constraints that we take for granted in human vision. An example of such a constraint is that the scene of interest [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning Contextual Actions for Heuristic Search-Based Motion Planning

Zoom Link Abstract: Heuristic search-based motion planning can be computationally costly in large state and action spaces. In this work we explore the use of generative models to learn contextual actions for successor generation in heuristic search. We focus on cases where the robot operates in similar environments, i.e. environments drawn from some underlying distribution. [...]

VASC Seminar
Cristian Sminchisescu
Research Scientist / Professor
Google / Lund University

End-to-end Generative 3D Human Shape and Pose Models and Active Human Sensing

Virtual VASC Seminar:  https://cmu.zoom.us/j/249106600 Title:  End-to-end Generative 3D Human Shape and Pose Models and Active Human Sensing Abstract:  I will review some of our recent work in 3d human modeling, synthesis, and active vision. I will present our new, end-to-end trainable nonlinear statistical 3d human shape and pose models of different resolutions (GHUM and GHUMLite) as [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Safe and Resilient Multi-Robot Systems: Heterogeneity and Human Presence

Zoom Link Abstract: In the mission of a multi-robot team, the large number of robots behave like a system that relies on networking to enable smooth information propagation and inter-robot interaction as the mission evolves in a collective fashion. Key to the success of mission operation demands for safe and reliable robot interactions within the [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Michael Tatum – MSR Thesis Talk

Archived Zoom Video Password: 1u%i4YO%   Title: Communications Coverage in Unknown Underground Environments   Abstract:In field robotics, maintaining communications between the user at a stationary basestation and all deployed robots is essential.  This task's difficulty increases when the test environment is underground and the environment is unknown to the operator and robots.  A common approach [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Brendan Miller – MSR Thesis Talk

NSH 4305

Zoom Link: https://cmu.zoom.us/j/96617143856 Title: IBB-Net: Fast Iterative Bounding Box Regression for Point Clouds Abstract: Currently, most point cloud based detection pipelines are focused on producing high accuracy results while requiring significant computational resources and a high-end GPU. Our research explores how to reduce the computational overhead by improving a key element of detection: bounding box regression. We [...]

PhD Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Interactive Weak Supervision – Learning Useful Heuristics for Data Labeling

Zoom Link Abstract: Obtaining large annotated datasets is critical for training successful machine learning models and it is frequently a bottleneck in practice. Weak supervision offers a promising alternative for producing labeled datasets without ground truth annotations by generating probabilistic labels using multiple noisy heuristics. This process can scale to large amounts of data and [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Automated Action Selection and Embodied Simulation for Socially Assistive Robots using Standardized Interactions

Zoom Link Abstract: Robots have the tremendous potential of assisting people in their lives, allowing them to achieve goals that they would not be able to achieve by themselves. In particular, socially assistive robots provide assistance primarily through social interaction, in healthcare, therapy, and education contexts. Despite their potential, current socially assistive robots still lack [...]

VASC Seminar
Bryan Russell
Senior Research Scientist
Adobe Research

Telling Left from Right: Learning Spatial Correspondence Between Sight and Sound

Virtual VASC Seminar:  https://cmu.zoom.us/j/92741882813?pwd=R1R0eGRaeXFHTEF2VWNwY2VIZmU5Zz09 Abstract:  Self-supervised audio-visual learning aims to capture useful representations of video by leveraging correspondences between visual and audio inputs. Existing approaches have focused primarily on matching semantic information between the sensory streams. In my talk, I’ll describe a novel self-supervised task to leverage an orthogonal principle: matching spatial information in the [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Heuristics for routing and scheduling of Spatio-Temporal type problems in industrial environments

Zoom Link Abstract: Spatio-temporal problems are fairly common in industrial environments. In practice, these problems come with different characteristics and are often very hard to solve optimally. So practitioners prefer to develop heuristics that exploit mathematical structure specific to the problem for obtaining good performance. In this proposal, I will present work on heuristics for [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Aditya Agarwal – MSR Thesis Talk

Zoom Link:  https://cmu.zoom.us/j/3276236755   Title: Fast and High-Quality GPU-based Deliberative Perception for Object Pose Estimation Abstract:  Pose estimation of known objects is fundamental to tasks such as robotic grasping and manipulation. The need for reliable grasping imposes stringent accuracy requirements on pose estimation in cluttered, occluded scenes in dynamic environments. Existing methods either require large sets of [...]

PhD Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Learning Active Task-Oriented Exploration Policies for Bridging the Sim-to-Real Gap

Zoom Link Abstract: Training robotic policies in simulation suffers from the sim-to-real gap, as simulated dynamics can be different from real-world dynamics. Past works tackled this problem through domain randomization and online system-identification. The former is sensitive to the manually-specified training distribution of dynamics parameters and can result in behaviors that are overly conservative. The [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Understanding, Exploiting and Improving Inter-view Relationships

 Zoom Link Abstract: Multi-view machine learning has received substantial attention in various applications over recent years. These applications typically involve learning on data obtained from multiple sources of information, such as, for example, in multi-sensor systems such as self-driving cars and patient bed-side monitoring. Learning models for such applications can often benefit from leveraging not [...]

PhD Speaking Qualifier
Postdoctoral Fellow
Robotics Institute,
Carnegie Mellon University

Interferometric light transmission probing with coded mutual intensity

Zoom Link Abstract: We introduce a new interferometric imaging methodology that we term interferometry with coded mutual intensity, which allows selectively imaging photon paths based on attributes such as their length and endpoints. At the core of our methodology is a new technical result that shows that manipulating the spatial coherence properties of the light [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Josh Jaekel – MSR Thesis Talk

Zoom Link: https://cmu.zoom.us/j/97161117200?pwd=QlpkS0hFOFVLRDlKVlFqby9JbWZTUT09 Title: Towards Robust Multi-Camera Visual Inertial Odometry Abstract: Visual inertial odometry (VIO) has become an increasingly popular method of obtaining a state estimate on board smaller robots like micro aerial vehicles (MAVs). While VIO has demonstrated impressive results in certain environments, there is still work to be done in improving the robustness of [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

John Mai – MSR Thesis Talk

Zoom Link: https://cmu.zoom.us/j/7518832261 Title: System Design, Modelling, and Control for an Off-Road Autonomous Ground Vehicle Abstract: Autonomy in passenger road vehicles has long been a goal for many research groups and companies, and there has been a significant amount of focus on achieving this endeavour. A lesser focused upon area is the task of precise autonomous [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Zoom Link: https://cmu.zoom.us/j/5523238059   Title: Robust Instance Tracking via Uncertainty Flow   Abstract: Current state-of-the-art trackers often fail due to distractors and large object appearance changes. In this work, we explore the use of dense optical flow to improve tracking robustness. Our main insight is that, because flow estimation can also have errors, we need [...]

VASC Seminar
Ciprian Corneanu
Research Assistant
Tawny GmbH, University of Barcelona

The Topology of Learning

Zoom Virtual Meeting:  https://cmu.zoom.us/j/92178295543?pwd=L2dwZU5SbDY5NzZZNzZ4ZmFUclRqQT09   Abstract: Deep Neural Networks (DNNs) have revolutionized computer vision. We now have DNNs that achieve top results in many computer vision problems, including object recognition, facial expression analysis, and semantic segmentation, to name but a few. Unfortunately, the rise in performance has come with a cost.  DNNs have become so [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Sarthak Ahuja – MSR Thesis Talk

NSH 4305

Zoom Link: https://cmu.zoom.us/j/8978517404 Title: Visual Assessment for Non-Disruptive Object Extraction Abstract: Robots operating in human environments need to perform a variety of dexterous manipulation tasks on object arrangements that have complex physical support relationships, e.g. procuring utensils from a large pile of dishes, grabbing a bottle from a stuffed fridge, or fetching a book from a loaded [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Eric Dexheimer – MSR Thesis Talk

Location: https://cmu.zoom.us/j/98262481359?pwd=dnN4UERmQlF6dVVROTQ1czYrU215UT09 Title: Efficient Multiresolution Scrolling Grid for MAV Obstacle Avoidance Abstract: In this talk, we propose the use of an efficient, structured multiresolution representation for robot mapping and planning.  We focus on expanding the sensor range of dense local grids for memory-constrained platforms.  While multiresolution data structures have been proposed previously, we avoid processing [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Tanmay Agarwal – MSR Thesis Talk

Zoom link: https://cmu.zoom.us/j/3388909661 Title: On-Policy Reinforcement Learning for Learning to Drive in Urban Settings Abstract: Traditional autonomous vehicle pipelines that follow a modular approach have been very successful in the past both in academia and industry, which has led to autonomy deployed on road. Though this approach provides ease of interpretation, its generalizability to unseen [...]

MSR Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Jay Patrikar – MSR Thesis Talk

Zoom link: https://cmu.zoom.us/j/93391276533?pwd=RTM4NTc0cTJETmRudGcwenNCSVgzdz09 Title: Wind-Field Estimation and Curvature Continuous Path Planning for Low Altitude Urban Aerial Mobility Abstract: Unmanned Aerial Vehicles (UAVs) operating in dense urban areas need the ability to generate wind-aware collision-free, smooth, dynamically feasible trajectories between two locations. The complex and high-rise structure of the modern urban landscape affects the wind flow [...]

MSR Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Benjamin Freed – MSR Thesis Talk

Where?:https://cmu.zoom.us/j/96355036481?pwd=OCtBeWZpMnZsZzFlRkJWc2dkZW5qUT09   Title: Discrete Communication Learning via Backpropagation for Distributed Computing on Bandwidth-Limited Communication Networks   Abstract: Efficient inter-agent communication is an important requirement for both cooperative multi-agent robotics tasks, as well as distributed computing.  In both of these domains, the rate at which information can be transferred between robots or computing nodes is often [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Tanvir Parhar – MSR Thesis Talk

Zoom Link:https://cmu.zoom.us/j/3399055387 Title: Applications of Deep Learning for Robotic Agriculture.   Abstract: Agricultural automation is a varied and challenging field, with tasks ranging from detection to sizing and from manipulation to navigation. These are also precursors to effective plant breeding and management. Making plant measurements by manually scouting is labor-intensive and intractable at large scale. [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Hitesh Arora – MSR Thesis Talk

Zoom link: https://cmu.zoom.us/j/4937138807   Title: Off-Policy Reinforcement Learning for Autonomous Driving   Abstract: Modern autonomous driving systems continue to face the challenges of handling complex and variable multi-agent real-world scenarios. Some subsystems, such as perception, use deep learning-based approaches to leverage large amounts of data to generalize to novel scenes. Other subsystems, such as planning [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Scott Sun – MSR Thesis Talk

Zoom link: https://cmu.zoom.us/j/93097644031?pwd=RzZVSXEvdE5zZ0RDaU9FdmRUMU1vQT09 Title: Accurate Orientation Estimates for Deep Inertial Odometry Abstract: Many smartphone applications use inertial measurement units (IMUs) to sense movement, but the use of these sensors for pedestrian localization can be challenging due to their noise characteristics. Recent deep inertial odometry approaches to pedestrian navigation have demonstrated the increasing feasibility of inertial navigation. However, [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Allen Cheng – MSR Thesis Talk

Zoom link: https://cmu.zoom.us/j/6056258382   Title: Search-Based Planning with Extend Operator   Abstract: Sampling-based approaches are often favored in robotics for high-dimensional motion planning for their fast coverage of the search space. However, at best they offer asymptotic guarantees on completeness and solution quality, and returned paths are typically unpredictable due to their inherent stochasticity. By [...]

VASC Seminar
Vincent Sitzmann
Postdoc
MIT CSAIL

Implicit Neural Scene Representations

Virtual Zoom Seminar:  https://cmu.zoom.us/j/92178295543?pwd=L2dwZU5SbDY5NzZZNzZ4ZmFUclRqQT09   Abstract How we represent signals has major implications for the algorithms we build to analyze them. Today, most signals are represented discretely: Images as grids of pixels, shapes as point clouds, audio as grids of amplitudes, etc. If images weren't pixel grids - would we be using convolutional neural networks [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Chenfeng Tu – MSR Thesis Talk

Location: https://cmu.zoom.us/j/96696044200?pwd=MVl4aUpiZlYvYlRwRmF1SVBUeGx6Zz09   Title: On-the-fly Targetless Extrinsics Calibration For Multi-Stereo Systems Without Field-of-View Overlap Abstract: In this talk, we propose an on-the-fly extrinsics calibration method for stereo pairs lacking overlapping field of view that is robust to visual odometry errors. Multi-stereo systems are becoming increasingly popular because of their large field of view (FoV) that benefits [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Shuoqi Chen – MSR Thesis Talk

Zoom link: https://cmu.zoom.us/j/9608506704   Title: Towards Geometric Motion Planning for 3-link Kinematic Systems   Abstract: Geometric mechanics offers a powerful mathematical framework for studying locomotion for mobile systems. Despite the well-established literature, challenges remain when using geometric mechanics to design gaits for robots made of multi-link chain; in this thesis, we look at two of them. First, [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Robot Deep Reinforcement Learning: Tensor State-Action Spaces and Auxiliary Task Learning with Multiple State Representations

Zoom Link Abstract: A long standing goal of robotics research is to create algorithms that can automatically learn complex control strategies from scratch. Part of the challenge of applying such algorithms to robots is the choice of representation. Reinforcement Learning (RL) algorithms have been successfully applied to many different robotic tasks such as the Ball-in-a-Cup [...]

MSR Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Raunaq Bhirangi – MSR Thesis Talk

Zoom link: https://cmu.zoom.us/j/93803046130?pwd=dE5LU21lakcxNjBmZ0EvVDdNOWswdz09   Title: Learning Families of Behaviors for Legged Locomotion using Model-Free Deep Reinforcement Learning   Abstract: Conventional planning and control of highly articulated legged robots is challenging because of the high dimensionality of the state space, and such conventional techniques normally produce a single point solution. In this work, we present a [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

William Qi – MSR Thesis Talk

Location: https://cmu.zoom.us/j/96923127678?pwd=TWt3Zk5neFUzSlJWUjZEN2F6UVhudz09 Title: Representation Learning for Safe Autonomous Movement Abstract: Mobile robots have become an increasingly common presence in our homes and on our roads. To move safely within these shared spaces, autonomous agents must understand how other dynamic actors behave and how such behavior influences the navigability of the surrounding scene. Towards this goal, we [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Ryan Coulson – MSR Thesis Talk

Zoom link: https://cmu.zoom.us/j/91138367616 Title: Soft Materials Architectures for Robot Manipulation Abstract: Robot manipulation has been a prolific subject of academic research for several decades - however, today's robotic manipulators have yet to demonstrate an ability to perform robust and versatile dexterous manipulation. This challenge can largely be attributed to a tradeoff between complexity and capability [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Anish Bhattacharya – MSR Thesis Talk

Zoom link: https://cmu.zoom.us/j/4413360562 Title: Toward Increased Airspace Safety: Quadrotor Guidance for Targeting Aerial Objects Abstract: As the market for commercially available unmanned aerial vehicles (UAVs) booms, there is an increasing number of small, teleoperated or autonomous aircraft found in protected or sensitive airspace. Existing solutions for removal of these aircraft are either military-grade and too [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Aaron Miller – MSR Thesis Talk

Zoom link: https://cmu.zoom.us/j/95386019509?pwd=cmNnTm9lWWlNbTh1SmQ0RU5PVTBmQT09 Title: Cooperative Perception for Pairs of Self-Driving Cars   Abstract: Fully autonomous vehicles are expected to share the road with less advanced vehicles for a significant period of time. Furthermore, an increasing number of vehicles on the road are equipped with a variety of low-fidelity sensors which provide some perception and localization [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Sara Misra – MSR Thesis Talk

Zoom link: https://cmu.zoom.us/j/3216213856   Title: Learning-based modular framework for environment-adaptive planning in exploration tasks   Abstract: Search-based path planning has spawned a number of different solutions using different paradigms and strategies, both generalized and specific to certain problems, representations, and environments. Split into heuristic and non-heuristic based approaches, where heuristic-based approaches, embedded within these approaches [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Blake Buchanan – MSR Thesis Talk

Location: https://cmu.zoom.us/j/99874277969?pwd=Q1MvczNhWTB4UmF3UXFOMEFtVG1uZz09 Title: Mechanics and Control of Coupled Interactions in Ambient Media Abstract: Many multi-agent systems in nature comprise agents that interact with, and respond to, the dynamics of their environment. For example, fish school based on the fundamental fluid phenomena of vortex shedding, birds shed leading-edge vortices in formation for flocking, and E. coli bacteria [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Online Inference of Joint Occupancy using Forward Sensor Models and Trajectory Posteriors for Deliberate Robot Navigation

Zoom Link Abstract: Robotic navigation algorithms for real-world robots require dense and accurate probabilistic volumetric representations of the environment in order to traverse efficiently. Sensor data in a Simultaneous Localisation And Mapping (SLAM) context, however, always has associated acquisition noise and pose uncertainty, and encoding this within the map representation while still maintaining computational tractability [...]