RI Seminar
Assistant Professor
Robotics Institute,
Carnegie Mellon University

GANs for Everyone

Abstract:  The power and promise of deep generative models such as StyleGAN, CycleGAN, and GauGAN lie in their ability to synthesize endless realistic, diverse, and novel content with user controls. Unfortunately, the creation and deployment of these large-scale models demand high-performance computing platforms, large-scale annotated datasets, and sophisticated knowledge of deep learning methods. This makes [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk – Hans Kumar

Title: Multi-Session Periodic SLAM for Legged Robots   Abstract: Methods for state estimation that rely on visual information are challenging on dynamic robots because of rapid changes in the viewing angle of onboard cameras. In this thesis, we show that by leveraging structure in the way that dynamic robots locomote, we can increase the feasibility [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation

Abstract: To perform manipulation tasks in the real world, robots need to operate on objects with various shapes, sizes and without access to geometric models. It is often infeasible to train monolithic neural network policies across such large variance in object properties. Towards this generalization challenge, we propose task-axis controllers, which are defined relative to [...]

VASC Seminar
Ronghang Hu
Research Scientist
Facebook Inc.

Reasoning over Text in Images for VQA and Captioning

Abstract: Text in images carries essential information for multimodal reasoning, such as VQA or image captioning. To enable machines to perceive and understand scene text and reason jointly with other modalities, 1) we collect the TextCaps dataset, which requires models to read and reason over text and visual content in the image to generate image [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Eagle Dapeng Zhao

Title: Predicting Human Trajectories by Learning and Matching Patterns Zoom Link: https://cmu.zoom.us/j/93356993095?pwd=Nzd3a09PbG9mVkV5blFVaU5nRk1GQT09 Abstract: As more and more robots are envisioned to cooperate with humans sharing the same space, it is desired for robots to be able to predict others' trajectories to navigate in a safe and self-explanatory way. We propose a Convolutional Neural Network-based approach [...]

RI Seminar
Avik De
Co-founder & CTO
Ghost Robotics

Design and control of insect-scale bees and dog-scale quadrupeds

Abstract: Enhanced robot autonomy---whether it be in the context of extended tether-free flight of a 100mg insect-scale flapping-wing micro aerial vehicle (FWMAV), or long inspection routes for a quadrupedal robot---is hindered by fundamental constraints in power and computation. With this motivation, I will discuss a few projects I have worked on to circumvent these issues in [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies

Abstract: Real-world environments, such as homes, hospitals, and restaurants, often contain many objects that a robot could possibly manipulate. However, for a given manipulation task, only a small number of objects and object properties may actually be relevant. This talk presents CREST (Causal Reasoning for Efficient Structure Transfer), our approach to learn the relevant state [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Grasping Transparent, Specular, and Deformable Objects

Abstract: A large body of research exists on grasping for objects with ideal properties like Lambertian reflectance and rigidity. On the other hand, real-world environments contain many objects for which such properties do not hold, such as transparent, specular, and deformable objects. For such objects, new approaches are required to achieve the same level of [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Auto-generated Manipulation Primitives

Abstract: The central theme in robotic manipulation is that of the robot interacting with the world through physical contact. We tend to describe that physical contact using specific words that capture the nature of the contact and the action, such as grasp, roll, pivot, push, pull, tilt, close, open etc. We refer to these situation-specific [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Learning 3D Registration and Reconstruction from the Visual World

Abstract: Humans learn to develop strong senses for 3D geometry by looking around in the visual world. Through pure visual perception, not only can we recover a mental 3D representation of what we are looking at, but meanwhile we can also recognize where we are looking at the scene from. Finding the 3D scene representation [...]

VASC Seminar
Jhony Kaesemodel Pontes
Research Scientist
Argo AI

Point Cloud Registration with or without Learning

Abstract: I will be presenting two of our recent works on 3D point cloud registration:   A scene flow method for non-rigid registration: I will discuss our current method to recover scene flow from point clouds. Scene flow is the three-dimensional (3D) motion field of a scene, and it provides information about the spatial arrangement [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Development of an Agile and Dexterous Balancing Mobile Manipulator Robot

Abstract: The proposed thesis work focuses on the design and control of a new unique agile and dexterous mobile manipulator, the Carnegie Mellon University (CMU) ballbot. The CMU ballbot is a human-sized dynamically stable mobile robot that balances on a single ball. We present the development and integration of a new pair of seven-degree-of-freedom (7-DOF) [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Shengcao Cao

Title: Efficient Model Performance Estimation via Feature Histories Abstract: An important step in the task of neural network design, such as hyper-parameter optimization (HPO) or neural architecture search (NAS), is the evaluation of a candidate model's performance. Given fixed computational resources, one can either invest more time training each model to obtain more accurate estimates [...]

RI Seminar
Cynthia Sung
Assistant Professor
Mechanical Engineering & Applied Mechanics, University of Pennsylvania

Dynamical Robots via Origami-Inspired Design

Abstract: Origami-inspired engineering produces structures with high strength-to-weight ratios and simultaneously lower manufacturing complexity. This reliable, customizable, cheap fabrication and component assembly technology is ideal for robotics applications in remote, rapid deployment scenarios that require platforms to be quickly produced, reconfigured, and deployed. Unfortunately, most examples of folded robots are appropriate only for small-scale, low-load [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Steven Lee

Title: Learning to Represent and Accurately Arrange Food Items   Abstract: Arrangements of objects are commonplace in a myriad of everyday scenarios, such as decorations at one’s home, displays at museums, and plates of food at restaurants. An efficient personal robot should be able to learn how to robustly recreate an arrangement using only a few [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Amrita Sawhney

Title: Learning to Perceive and Manipulate Diverse Food Materials Through Interaction Abstract: The home kitchen environment presents many challenges for an autonomous cooking robot, such as the deformability of food items, the wide range of material properties of food, and the complex interaction dynamics involved in food manipulation tasks. Material properties are important when interacting [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Haidar Jamal

Zoom

Title: Localization for Lunar Micro-Rovers   Abstract: This talk presents an avionics and localization system that enables a lunar micro-rover to navigate autonomously. This system is important for the latest class of small, low-powered, and fast robots going to the Moon in search of polar ice. The first component of the system is an Extended [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

PoseIt: A Visual-Tactile Dataset of Holding Poses for Grasp Stability Analysis

Abstract: When humans grasp objects in the real world, we often move our arm to hold the object in a different pose where we can use it. In contrast, typical lab settings only study the stability of the grasp immediately after lifting, without any subsequent re-positioning of the arm. However, an object’s stability could vary [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Planning to Minimize Human and Robot Efforts Over Tasks

Abstract: It is not feasible to pre-program robots a priori for every possible task they may encounter in unstructured domains. Upon encountering a task that a robot can't solve, one common strategy is to teach it new skills via demonstrations. However, demonstrating a task can often be more cumbersome than performing the task directly. This [...]

MSR Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Akash Sharma

Title: Incorporating Semantic Structure in SLAM Abstract: For robots to understand the environment they interact with, a combination of geometric information and semantic information is imperative. In this talk, I propose a fast and scalable Simultaneous Localization and Mapping (SLAM) system that represents indoor scenes as a graph of semantic objects. Leveraging the observation that [...]

VASC Seminar
Arsalan Mousavian
Senior Robotics Research Scientist
NVIDIA

Propelling Robot Manipulation of Unknown Objects using Learned Object Centric Models

Abstract: There is a growing interest in using data-driven methods to scale up manipulation capabilities of robots for handling a large variety of objects. Many of these methods are oblivious to the notion of objects and they learn monolithic policies from the whole scene in image space. As a result, they don’t generalize well to [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

Yaadhav Raaj MSR Thesis Talk

Title: Exploiting Uncertainty in Triangulation Light Curtains for Object Tracking and Depth Estimation   Abstract: Active sensing through the use of Adaptive Depth Sensors is a nascent field, with potential in areas such as Advanced driver-assistance systems (ADAS). One such class of sensor is the Triangulation Light Curtain, which was developed in the Illumination and Imaging [...]

PhD Thesis Defense
Robotics Institute,
Carnegie Mellon University

Active Vision: Autonomous Aerial Cinematography with Learned Artistic Decision-Making

Abstract: Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. Fundamentally, it is a tool with immense potential to improve human creativity, expressiveness, and sharing of experiences. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Fine-Tuning Offline Reinforcement Learning with Model-Based Policy Optimization

Abstract: In offline reinforcement learning (RL), we attempt to learn a control policy from a fixed dataset of environment interactions. This setting has the potential benefit of allowing us to learn effective policies without needing to collect additional interactive data, which can be expensive or dangerous in real-world systems. However, traditional off-policy RL methods tend [...]

MSR Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Zhipeng Bao

Title: Introducing Generative Models to Facilitate Multi-Task Visual Learning Abstract: Motivated by multi-task learning of shared feature representations, this talk considers a novel problem of learning a shared generative model that can facilitate multi-task learning. We present two systems to utilize generative modeling for other visual tasks. The first system focuses on learning a generative [...]

MSR Speaking Qualifier
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk: Shanshan Jessy Xie

TBA

Title: GPU based perception via search for object pose estimation with RGB data   Abstract: Known object pose estimation is essential for a robot to interact with the real world.  It is the first and fundamental task if the robot wants to manipulate the object.  This problem is particularly challenging when the environment is complicated [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Accelerating Numerical Methods for Optimal Control

Abstract: Many modern control methods, such as model-predictive control, rely heavily on solving optimization problems in real time. In particular, the ability to efficiently solve optimal control problems has enabled many of the recent breakthroughs in achieving highly dynamic behaviors for complex robotic systems. The high computational requirements of these algorithms demand novel algorithms tailor-suited [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Modeling Coupled Human-Robot Motion for Provable Safety

Abstract: Guide robots that help users who are blind or low vision navigate through crowds and complex environments show promise for improving accessibility in public spaces. These robots must provide real-time safety guarantees for the users, which requires accurate modeling of their behavior in the context of closely coupled human-robot motion. This model must also [...]

MSR Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

MSR Thesis Talk – Mosam Dabhi

Title: Multi-view NRSfM: Affordable setup for high-fidelity 3D reconstruction   Abstract: Triangulating a point in 3D space should only require two corresponding camera projections. However in practice, expensive multi-view setups -- involving tens sometimes hundreds of cameras -- are required to obtain the high fidelity 3D reconstructions necessary for many modern applications. In this talk, we argue [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Robust Object Representations for Robot Manipulation

Abstract: As robots become more common in our daily lives, they will need to interact with many different environments and countless types of objects. While we, as humans, can easily understand an object after seeing it only once, this task is not trivial for robots. Researchers have, for the most part, been left with two [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Diminished Reality for Close Quarters Robotic Telemanipulation

Abstract: In robot telemanipulation tasks, the robot itself can sometimes occlude a target object from the user's view. We investigate the potential of diminished reality to address this problem. Our method uses an optical see-through head-mounted display to create a diminished reality illusion that the robot is transparent, allowing users to see occluded areas behind [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Visual Representation and Recognition without Human Supervision

Abstract: Visual recognition models have seen great advancements by relying on large-scale, carefully curated datasets with human annotations. Most computer vision models leverage human supervision to either construct strong initial representations (e.g. using the ImageNet dataset) or for modeling the visual concepts relevant for downstream tasks (e.g. MS-COCO for object detection). In this thesis, we [...]

PhD Speaking Qualifier
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning Compositional Radiance Fields of Dynamic Human Heads

Meeting ID: 942 4671 0665 Passcode: jkhzoom Abstract: Photorealistic rendering of dynamic humans is an important capability for telepresence systems. Recently, neural rendering methods have been developed to create high-fidelity models of humans and objects. Some of these methods do not produce results with high-enough fidelity for driveable human models (Neural Volumes) whereas others have [...]

VASC Seminar
Phillip Isola
Assistant Professor
EECS, MIT

When and Why Does Contrastive Learning Work?

Abstract: Contrastive learning organizes data by pulling together related items and pushing apart everything else. These methods have become very popular but it's still not entirely clear when and why they work. I will share two ideas from our recent work. First, I will argue that contrastive learning is really about learning to forget. Different [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Heuristic Search Based Planning by Minimizing Anticipated Search Efforts

Abstract: Robot planning problems in dynamic environments, such as navigation among pedestrians, driving at high-speed on densely populated roads, and manipulation for collaborative tasks alongside humans, necessitate efficient planning. Bounded-suboptimal heuristic search algorithms are a popular alternative to optimal heuristic search algorithms that compromise solution quality for computation speed. Specifically, these searches aim to find [...]

PhD Thesis Proposal
Robotics Institute,
Carnegie Mellon University

Liquid Metal Actuators

Abstract: Bioinspired robotic actuators arise from the advances in soft materials and activation methods to achieve desired performance. Because of their intrinsic compliance, actuators built from soft materials and liquids can achieve elastic resilience and adaptability similar to their biological counterparts. Liquid metals provide great opportunities for creating an artificial muscle that generates forces at [...]

VASC Seminar
Ehsan Adeli
Clinical Assistant Professor
Stanford University

Anticipating the Future: forecasting the dynamics in multiple levels of abstraction

Abstract: A key navigational capability for autonomous agents is to predict the future locations, actions, and behaviors of other agents in the environment. This is particularly crucial for safety in the realm of autonomous vehicles and robots. However, many current approaches to navigation and control assume perfect perception and knowledge of the environment, even though [...]