RI Seminar
Algorithms and challenges in scaling up autonomous flight
Event Location: NSH 1305Bio: Adam Bry is co-founder and CEO of Skydio, a venture backed drone startup based in the bay area. Prior to Skydio he helped start Project Wing at Google[x] where he worked on the flight algorithms and software. He holds a SM in Aero/Astro from MIT and a BS in Mechanical Engineering [...]
Learning Models for Robot Decision Making
Event Location: NSH 1305Bio: Thomas Howard is an assistant professor in the Department of Electrical and Computer Engineering and the Department of Computer Science. He is also a member of the Institute for Data Science and holds a secondary appointment in the Department of Biomedical Engineering. Previously he held appointments as a research scientist and [...]
Exploiting the Environment to Improve Autonomy: Robots in Geophysical Flows
Event Location: NSH 1305Bio: M. Ani Hsieh is an Associate Professor in the Mechanical Engineering & Mechanics Department at Drexel University. She received a B.S. in Engineering and B.A. in Economics from Swarthmore College in 1999 and a PhD in Mechanical Engineering from the University of Pennsylvania in 2007. Her current work focuses on developing [...]
The Paradox of Human Performance
Event Location: NSH 1305Bio: Neville Hogan is Professor of Mechanical Engineering, Professor of Brain and Cognitive Sciences and Director of the Newman Laboratory for Biomechanics and Human Rehabilitation at the Massachusetts Institute of Technology. He is a co-founder of Interactive Motion Technologies, Inc., and a board member of Advanced Mechanical Technologies, Inc. Born in Dublin, [...]
Data-driven Social Informatics
Event Location: NSH 1305Bio: Dr. Gita Sukthankar is an Associate Professor and Charles N. Millican Faculty Fellow in the Department of Computer Science at the University of Central Florida, and an affiliate faculty member at UCF’s Institute for Simulation and Training. She received her Ph.D. from the Robotics Institute at Carnegie Mellon and an A.B. [...]
Designing Robots to Walk and Run
Event Location: NSH 1305Bio: Jonathan W. Hurst is the College of Engineering Dean's Professor of Robotics in the School of Mechanical, Industrial, and Manufacturing Engineering at Oregon State University, and the co-founder and Chief Technology Officer of Agility Robotics. He holds a B.S. in Mechanical Engineering, and both an M.S. and Ph.D. in Robotics, all [...]
Robots at Sea
Event Location: NSH 1305Bio: Gaurav S. Sukhatme is Dean’s Professor of Computer Science and Electrical Engineering at the University of Southern California (USC). He currently serves as the Chairman of the Computer Science department. His research is in networked robots with applications to aquatic robots and on-body networks. Sukhatme has published extensively in these areas [...]
Humanitarian Robotics and Automation Technologies: Improving the Quality of Life for Humanity
Event Location: NSH 1507Bio: Raj Madhavan is the Founder & CEO of Humanitarian Robotics Technologies, LLC, Maryland, U.S.A. and a Distinguished Visiting Professor of Robotics with AMMACHI Labs at Amrita University, Kerala, India. He has held appointments with the Oak Ridge National Laboratory (March 2001-January 2010) as an R&D staff member based at the National [...]
Human versus Machine Perception of Visual Regularity or Are you a human or a robot?
Event Location: NSH 1305Bio: Yanxi Liu received her B.S. degree in physics/electrical engineering (Beijing, China), her Ph.D. degree in computer science for group theory applications in robotics (University of Massachusetts, Amherst, US), and her postdoctoral training in the robotics lab of LIFIA/IMAG (Grenoble, France). Before joining the Robotics Institute of Carnegie Mellon Institute in 1996 [...]
Reasoning in Deep Learning
Event Location: Newell Simon Hall 1507Bio: Yuandong Tian is a Research Scientist in Facebook AI Research, working on Deep Learning and Computer Vision. Prior to that, he was a Software Engineer in Google Self-driving Car team in 2013-2014. He received Ph.D in Robotics Institute, Carnegie Mellon University on 2013, Bachelor and Master degree of Computer [...]
The Journey to Consumer Robotics
Event Location: GHC 6115Bio: Boris is co-founder and CEO of Anki, an artificial intelligence and robotics company focused on using these technologies to reinvent everyday consumer experiences. With an initial focus on entertainment, Anki's first product line, Overdrive, is a battle-racing game that allowed a level of physical gameplay and interaction previously not possible outside [...]
The Journey to Consumer Robotics
Event Location: GHC 6115Bio: Hanns is co-founder and President of Anki, an artificial intelligence and robotics company focused on creating groundbreaking consumer products. Anki's first product line, Overdrive, is a battle-racing game that allowed a level of physical gameplay and interaction previously not possible outside of video games and was one of the top selling [...]
Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours
Event Location: NSH 1305Bio: Lerrel recently graduated as a Masters student from CMU RI where he was advised by Professor Abhinav Gupta. His research interests revolve around big data, computer vision and robotics. He is currently a PhD student at CMU RI.Abstract: Current learning-based robot grasping approaches exploit human-labeled datasets for training the models. However, [...]
A Convex Polynomial Force-Motion Model for Planar Sliding: Identification and Application
Event Location: NSH 1305Bio: Jiaji Zhou is a PhD student in the Robotics Institute of Carnegie Mellon University advised by Professor Matt Mason and Professor Drew Bagnell. His research interests lie in the intersection of manipulation, machine learning and control, with a focus on system identification, stability condition and planning for contact-rich manipulation. Jiaji has [...]
Human Autonomy through Robotics Autonomy
Event Location: NSH 1305Bio: Brenna Argall is the June and Donald Brewer Junior Professor of Electrical Engineering & Computer Science at Northwestern University, and also an assistant professor in the Department of Mechanical Engineering and the Department of Physical Medicine & Rehabilitation. Her research lies at the intersection of robotics, machine learning and human rehabilitation. [...]
Safe and Optimal Path Planning in Uncertain Skies
Event Location: NSH 1305Bio: Ashish Kapoor is a senior researcher at Microsoft Research, Redmond. Currently, his research focuses on Aerial Informatics and Robotics with an emphasis on building intelligent and autonomous flying agents that are safe and enable applications that can positively influence our society. The research builds upon cutting edge research in machine intelligence, [...]
Surgical Robotics- past, present and future
Event Location: NSH 1305Bio: Umamaheswar Duvvuri, MD, PhD, is a graduate of the University of Pennsylvania obtaining his Medical Degree in 2000 and his PhD in Biophysics in 2002. He completed an internship in General Surgery in 2003 and residency training in Otolaryngology in 2007 at the University of Pittsburgh Medical Center. He completed fellowship [...]
Learning Models of Language, Action and Perception for Human-Robot Collaboration
Event Location: NSH 1305Bio: Stefanie Tellex is an Assistant Professor of Computer Science and Assistant Professor of Engineering at Brown University. Her group, the Humans To Robots Lab, creates robots that seamlessly collaborate with people to meet their needs using language, gesture, and probabilistic inference, aiming to empower every person with a collaborative robot. She [...]
Autonomous Exploration and Inspection using Aerial Robots
Event Location: NSH 1305Bio: Kostas Alexis obtained his Ph.D. in the field of aerial robotics control and collaboration from the University of Patras, Greece in 2011. His Ph.D. research was supported by the Greek national-European Commission Excellence scholarship. After successfully defending his Ph.D. thesis, he was a awarded a Swiss Government fellowship and moved to [...]
Robotic Data Gathering in the Wild
Event Location: NSH 1305Bio: Volkan Isler is an Associate Professor in the Computer Science Department at the University of Minnesota. He is a 2009-2012 resident fellow at the Institute on Environment and 2010-2012 McKnight Land-Grant Professor. Previously, he was an Assistant Professor at Rensselaer Polytechnic Institute, and a post-doctoral researcher at CITRIS at UC Berkeley. [...]
Machine Learning For Modeling Real-World Dynamical Systems
Event Location: NSH 3305Bio: Byron Boots is an Assistant Professor in the School of Interactive Computing and the College of Computing at Georgia Tech. He directs the Georgia Tech Robot Learning Lab, which is affiliated with the Center for Machine Learning, the Institute for Data Engineering and Science, and the Institute for Robotics and Intelligent [...]
The Next Frontier in AI: Unsupervised Learning
Event Location: McConomy Auditorium, CUCBio: Yann LeCun is Director of AI Research at Facebook, and Silver Professor of Data Science, Computer Science, Neural Science, and Electrical Engineering at New York University, affiliated with the NYU Center for Data Science, the Courant Institute of Mathematical Science, the Center for Neural Science, and the Electrical and Computer [...]
Autonomous Intelligent Service Robots: Learning and Explanations in Human-Robot Interaction
Manuela Veloso Herbert A Simon University Professor, Carnegie Mellon Abstract We research on autonomous mobile robots with a seamless integration of perception, cognition, and action. In this talk, I will first introduce our CoBot service robots and their novel localization and symbiotic autonomy, which enable them to consistently move in our buildings, now for more [...]
Beyond Geometric Path Planning: Paradigms and algorithms for modern robotics
Kris Hauser Associate Professor, Duke University Abstract The development of fast randomized algorithms for geometric path planning – computing collision-free paths for high dimensional systems – was a major achievement in the field of motion planning in the 2000's. But since then, recent advances in affordable robot sensors, actuators, and systems have changed the robotics [...]
Pathway Toward Vision Restoration, Artificial Vision, Artificial Retina, Optogenetics
José Alain Sahel, MD Professor & Chairman, Department of Ophthalmology, University of Pittsburgh, School of Medicine Abstract Progress in ophthalmology over the past decade moved preclinical data to clinical proof-of-concept studies bringing innovative therapeutic strategies to the market. Diseases such as retinitis pigmentosa (RP) and age-related macular degeneration (AMD) destroy photoreceptors but leave intact and [...]
Towards Agile Flight of Vision-controlled Micro Flying Robots: from Active Vision to Event-based Vision
Davide Scaramuzza Assistant Professor of Robotics, University of Zurich Abstract Autonomous quadrotors will soon play a major role in search-and-rescue and remote-inspection missions, where a fast response is crucial. Quadrotors have the potential to navigate quickly through unstructured environments, enter and exit buildings through narrow gaps, and fly through collapsed buildings. However, their speed and [...]
e-Intangible Heritage
Event Location: NSH 1305Bio: Dr. Katsushi Ikeuchi is a Principal Researcher of Microsoft Research Asia, stationed at Microsoft Redmond campus. He received a Ph.D. degree in Information Engineering from the University of Tokyo in 1978. After working at Artificial Intelligence Lab of Massachusetts Institute of Technology as a pos-doc fellows for three years, Electrotechnical Lab [...]
Katsushi Ikeuchi : e-Intangible Heritage
Katsushi Ikeuchi Principal Researcher, Microsoft Research Asia Abstract Tangible heritage, such as temples and statues, is disappearing day-by-day due to human and natural disaster. In e-tangible heritage, such as folk dances, local songs, and dialects, has the same story due to lack of inheritors and mixing cultures. We have been developing methods to preserve such [...]
From Drones To Robots, The Road To Make Technologies More Accessible
Shuo Yang Director of Intelligent Navigation Technologies, DJI Abstract Over the past decade, DJI has developed several world-leading drone products, turning cutting-edge technologies such as high resolution image transmission, visual odometry, and learning-based object tracking into affordable commercial products. Along with all these technological successes, DJI is exploring innovative ways to make them more accessible. [...]
Stabilizing the Unstable Brain
Noah Cowan Associate Professor of Mechanical Engineering, Johns Hopkins University Abstract The nervous system is arguably the most sophisticated control system in the known universe, riding at the helm of an equally sophisticated plant. Understanding how the nervous system encodes and processes sensory information, and then computes motor action, therefore, involves understanding a closed loop. [...]
Robot Skill Learning: From the Real World to Simulation and Back
Event Location: NSH 1305Bio: Dr. Peter Stone is the David Bruton, Jr. Centennial Professor and Associate Chair of Computer Science, as well as Chair of the Robotics Portfolio Program, at the University of Texas at Austin. In 2013 he was awarded the University of Texas System Regents' Outstanding Teaching Award and in 2014 he was [...]
Robot Skill Learning: From the Real World to Simulation and Back
Peter Stone David Bruton, Jr. Centennial Professor, The University of Texas at Austin Abstract For autonomous robots to operate in the open, dynamically changing world, they will need to be able to learn a robust set of interacting skills. This talk begins by introducing "Overlapping Layered Learning" as a novel hierarchical machine learning paradigm for [...]
Deep Robotic Learning
Sergey Levine Assistant Professor, UC Berkeley Abstract Deep learning methods have provided us with remarkably powerful, flexible, and robust solutions in a wide range of passive perception areas: computer vision, speech recognition, and natural language processing. However, active decision making domains such as robotic control present a number of additional challenges, standard supervised learning methods [...]
Robots for the social good: Identifying and addressing organizational and societal factors in the design and use of robots
Event Location: NSH 1305Bio: I am an Associate Professor of Informatics and Cognitive Science at Indiana University, Bloomington, where I founded and direct the R-House Human-Robot Interaction Lab. My work combines the social studies of computing, focusing particularly on the design, use, and consequences of socially interactive and assistive robots in different social and cultural [...]
Selma Sabanovic: Robots for the social good: Identifying and addressing organizational and societal factors in the design and use of robots
Selma Sabanovic Associate Professor of Informatics and Cognitive Science, Indiana University Bloomington Additional Information Host: Aaron Steinfeld Appointments: Stephanie Matvey Abstract Robots are expected to become ubiquitous in the near future, working alongside and with people in everyday environments to provide various societal benefits. In contrast to this broad ranging social vision for robotics applications, [...]
Robotic Manipulation under clutter and uncertainty with and around people
Abstract Robots manipulate with super-human speed and dexterity on factory floors. But yet they fail even under moderate amounts of clutter or uncertainty. However, human teleoperators perform remarkable acts of manipulation with the same hardware. My research goal is to bridge the gap between what robotic manipulators can do now and what they are capable [...]
Sven Koenig: Progress on Multi-Robot Path Finding
Abstract Teams of robots often have to assign target locations among themselves and then plan collision-free paths to their target locations. Examples include autonomous aircraft towing vehicles and automated warehouse systems. For example, in the near future, autonomous aircraft towing vehicles might tow aircraft all the way from the runways to their gates (and vice [...]
David Held: Robots Learning to Understand Environmental Changes
Abstract Robots today are typically confined to operate in relatively simple, controlled environments. One reason for these limitation is that current methods for robotic perception and control tend to break down when faced with occlusions, viewpoint changes, poor lighting, unmodeled dynamics, and other challenging but common situations that occur when robots are placed in the [...]
Toward Natural Interactions With Assistive Robots
Abstract Robots can help people live better lives by assisting them with the complex tasks involved in everyday activities. This is especially impactful for people with disabilities, who can benefit from robotic assistance to increase their independence. For example, physically assistive robots can collaborate with people in preparing a meal, enabling people with motor impairments [...]
On-Demand Machine Knitting for Everyone
Abstract: Knitting machines are general-purpose fabrication devices that can robustly create intricate 3D surfaces from yarn by cleverly actuating thousands of mechanical needles. Knitting machines are an established feature of the textiles production landscape, in use today to make everything from socks to sweaters. However, the current design tools for machine knitting are sorely lacking [...]
AI, Robotics, and Autonomous Vehicle Development at Ford Motor Company
Notice: The Location for these event has changed! The event will now take place in 6115 Gates Hillman Center. Education: Ph.D. in Physics, University of Michigan M.S. in Physics, Michigan State University B.S. in Physics & Mathematics, University of Wisconsin – River Falls Abstract: This presentation will highlight the history of autonomous vehicle development at [...]
Modeling Human Movements for Robotics
Abstract: Creating realistic virtual humans has traditionally been considered a research problem in Computer Animation primarily for entertainment applications. With the recent breakthrough in collaborative robots and deep reinforcement learning, accurately modeling human movements and behaviors has become a common challenge faced by researchers in robotics, artificial intelligence, as well as Computer Animation. In this [...]
What Matters for Deformable Object Manipulation
Abstract: Deformable objects such as cables and clothes are ubiquitous in factories, hospitals, and homes. While a great deal of work has investigated the manipulation of rigid objects in these settings, manipulation of deformable objects remains under-explored. The problem is indeed challenging, as these objects are not straightforward to model and have infinite-dimensional configuration spaces, [...]
Optimizing ankle prostheses to improve walking in transtibial amputees
Abstract: With a prosthetic device, people with a lower limb amputation can remain physically active, but most do not achieve medically recommended physical activity standards and are therefore at a greater risk of obesity and cardiovascular disease. Their reduced activity may be attributed to the 10 - 30% increase in energetic cost during walking compared [...]
Exploring Human-Robot Trust during Emergencies
Abstract: This talk presents our experimental results related to human-robot trust involving more than 2000 paid subjects exploring topics such as how and why people trust a robot too much and how broken trust in a robot might be repaired. From our perspective, a person trusts a robot when they rely on and accept the [...]
Deep Structured Models for Human Activity Recognition
Abstract: Visual recognition involves reasoning about structured relations at multiple levels of detail. For example, human behaviour analysis requires a comprehensive labeling covering individual low-level actions to pair-wise interactions through to high-level events. Scene understanding can benefit from considering labels and their inter-relations. In this talk I will present recent work by our group building [...]
Level Set Models for Computer Graphics
ABSTRACT A level set model is a deformable implicit model that has a regularly-sampled representation. It is defined as an iso-contour, i.e. a level set, of some implicit function f. The contour is deformed by solving a partial differential equation on a sampling of f, an image in 2D and a volume dataset in 3D. [...]
“Does it look right? – Why capture and reconstruction quality really matter.”
Special RI Seminar Please Note Different Day and Time Abstract: At first sight, 3D reconstruction can be considered a solved problem. The principles are well understood and we can reconstruct a wide range of objects and scenes using active as well as passive reconstruction approached. However, most of these reconstructions are not convincing when really [...]
Factor Graphs and Automatic Differentiation for Flexible Inference in Robotics and Vision
PLEASE NOTE: THIS SEMINAR WILL NOT BE RECORDED Abstract: Simultaneous Localization and Mapping (SLAM) and Structure from Motion (SFM) are important and closely related problems in robotics and vision. I will review how SLAM, SFM and other problems in robotics and vision can be posed in terms of factor graphs, which provide a graphical language [...]
Long Duration Autonomy With Applications to Persistent Environmental Monitoring
Abstract: By now, we have a fairly good understanding of how to design coordinated control strategies for making teams of mobile robots achieve geometric objectives in a distributed manner, such as assembling shapes or covering areas. But, the mapping from high-level tasks to geometric objectives is not well understood. In this talk, we investigate this [...]
Marine Robotics: Planning, Decision Making, and Learning
Abstract: Underwater gliders, propeller-driven submersibles, and other marine robots are increasingly being tasked with gathering information (e.g., in environmental monitoring, offshore inspection, and coastal surveillance scenarios). However, in most of these scenarios, human operators must carefully plan the mission to ensure completion of the task. Strict human oversight not only makes such deployments expensive and [...]
Signal Processing – From Images to Surfaces
Abstract: In this talk we will revisit some classical techniques from image processing and explore what is involved in translating them to the context of surfaces. We will show that by leveraging existing methodology from discrete differential geometry, it is often easy to extend the image-based techniques so that they can be used to edit [...]
Bio-inspired dynamics for multi-agent decision-making
Abstract: I will present distributed decision-making dynamics for multi-agent systems, motivated by studies of animal groups, such as house-hunting honeybees, and their extraordinary ability to make collective decisions that are both robust to disturbance and adaptable to change. The dynamics derive from principles of symmetry, consensus, and bifurcation in networked systems, exploiting instability as a [...]
Robotics-Inspired Implantable Passive Mechanisms to Surgically Re-Engineer the Human Body
Abstract: Tendon-transfer surgeries are performed for a variety of conditions such as stroke, palsies, trauma, and congenital defects. The surgery involves re-routing a tendon from a nonfunctioning muscle to a functioning muscle to partially restore lost function. However, a fundamental aspect of the current surgery, namely the suture that attaches the tendon(s) to the muscles, [...]
Rendering Material Properties through Touch
Abstract: Humans haptically perceive the material properties of objects, such as roughness and compliance, through signals from sensory receptors in skin, muscles, tendons, and joints. Approaches to haptic rendering of material properties operate by stimulating, or attempting to stimulate, some or all of these receptor populations. My talk will describe research on haptic perception of [...]
From Automation to Autonomy and the Ubiquity of Moral Decision Making
Abstract: I argue that there is an important sense in which all decisions are moral decisions and I explore some implications of this insight (and its denial) for the design and human impacts of increasingly complex automated systems and emerging autonomous systems. This insight is obscured when we think about automated systems by the social [...]
Learning to Drive
Abstract: Why is our understanding of sensorimotor control behind our understanding of perception? I will talk about structural differences between perception and control, and how these differences can be mitigated to help advance sensorimotor control systems. Judicious use of simulation can play an important role and I will describe some simulation tools that we have [...]
Imaging the World One Photon at a Time
Abstract: The heart of a camera and one of the pillars for computer vision is the digital photodetector, a device that forms images by collecting billions of photons traveling through the physical world and into the lens of a camera. While the photodetectors used by cellphones or professional DSLR cameras are designed to aggregate as [...]
Carnegie Mellon University
Lesson Learned from Two Decades of Robotics Development and Thoughts on Where We Go from Here
Abstract: In this talk, Herman Herman will offer various lessons learned from developing various robots for the last 2 decades at the National Robotics Engineering Center. He will also offer his perspective on the future of autonomous robots in various industries, including self-driving cars, material handling and consumer robotics. Bio: Dr. Herman Herman is the [...]
Factor Graphs for Robot Perception
Abstract: Factor graphs have become a popular tool for modeling robot perception problems. Not only can they model the bipartite relationship between sensor measurements and variables of interest for inference, but they have also been instrumental in devising novel inference algorithms that exploit the spatial and temporal structure inherent in these problems. I will overview [...]
Light-Sensitive Displays
Abstract: Nobel prize winner M. G. Lippmann described his dream of an ideal display as a “window into the world.” “While the current most perfect photographic print only shows one aspect of reality, reduced to a single image fixed in a plane, the direct view of reality offers, as we know, infinitely more variety.” Changing [...]
What People See in a Robot: A New Look at Human-Like Appearance
Abstract: A long-standing question in HRI is what effects a robot’s human-like appearance has on various psychological responses. A substantial literature has demonstrated such effects on liking, trust, ascribed intelligence, and so on. Much of this work has relied on a construct of uni-dimensional low to high human-likeness. I introduce evidence for an alternative view according to which [...]
Safe Learning in Robotics
Abstract: A great deal of research in recent years has focused on robot learning. In many applications, guarantees that specifications are satisfied throughout the learning process are paramount. For the safety specification, we present a controller synthesis technique based on the computation of reachable sets, using optimal control and game theory. In the first part [...]
Bipolar Robotics – From the Arctic to the Antarctic with a stop for Fisheries in the middle latitudes.
Abstract: The Arctic, Antarctic and Greenland remain some of the least explored parts of the planet. This talk looks at efforts over the last decade to explore areas under-ice which have traditionally been difficult to access. The focus of the talk will be on the robots, the role of communications over low bandwidth acoustic links, [...]
Learning Robot Manipulation Skills through Experience and Generalization
Abstract: In the future, robots could be used to take care of the elderly, perform household chores, and assist in hazardous situations. However, such applications require robots to manipulate objects in unstructured and everyday environments. Hence, in order to perform a wide range of tasks, robots will need to learn manipulation skills that generalize between [...]
Signal to Symbol (via Skills)
Abstract: While recent years have seen dramatic progress in the development of affordable, general-purpose robot hardware, the capabilities of that hardware far exceed our ability to write software to adequately control. The key challenge here is one of abstraction: generally capable behavior requires high-level reasoning and planning, but perception and actuation must ultimately be performed [...]
Multi-Modal Geometric Learning for Grasping
Abstract: In this talk, we will describe methods to enable robots to grasp novel objects using multi-modal data and machine. The starting point is an architecture to enable robotic grasp planning via shape completion using a single occluded depth view of objects. Shape completion is accomplished through the use of a 3D CNN. The network [...]
Building a Force-Controlled Actuator (Company)
Abstract: In 2014, I was lucky enough to be one of 5 people to start HEBI Robotics, with the dream of eventually making the task of building custom robots as easy as building with Lego. A few years later we are now 10 people, and our first product, a series of modular force-controlled actuators, is [...]
Minimalist Visual Perception and Navigation for Consumer Drones
Abstract: Consumer drone developers often face the challenge of achieving safe autonomous navigation under very tight size, weight, power, and cost constraints. In this talk, I will present our recent results towards a minimalist, but complete perception and navigation solution utilizing only a low-cost monocular visual-inertial sensor suite. I will start with an introduction of [...]
Social Perception for Machines
Abstract: Despite decades of progress, machines remain intelligent tools rather than collaborative partners in individual human enterprise. A key reason is that machine perception of inter-personal communication is largely unsolved and a computationally accessible representation of such behavior remains elusive. In this talk, I will describe our research arc over the past decade at CMU [...]
Geometry Processing in The Wild
Abstract: Geometric data abounds, but our algorithms for geometry processing are failing. Whether from medical imagery, free-form architecture, self-driving cars, or 3D-printed parts, geometric data is often messy, riddled with "defects" that cause algorithms to crash or behave unpredictably. The traditional philosophy assumes geometry is given with 100% certainty and that algorithms can use whatever [...]
Self Driving Cars and AI: Transforming our cities and our lives
Abstract: Artificial intelligence and machine learning are critical to reaching full autonomy in self driving cars. I will present two autonomy systems along with the use of machine learning in each of them. I will summarize recent progress in commercializing these systems and make some observations about the potential impact of these systems in our [...]
Robotic Morphing Matter
Abstract: Morphing matter harnesses the programmability in material structures and compositions to achieve transformative behaviors and integrates sensing, actuation, and computation to create adaptive and responsive material systems. These material systems can be leveraged to design soft robots, self-assembling furniture, adaptive fabrics, and self-folding foods. In this talk, Lining presents the recent works in the [...]
The Mechanical Side of Artificial Intelligence
Abstract: Artificial Intelligence typically focuses on perception, learning, and control methods to enable autonomous robots to make and act on decisions in real environments. On the contrary, our research is focused on the design, mechanics, materials, and manufacturing of novel robot platforms that make the perception, control, or action easier or more robust for natural, unstructured, and [...]
Three surprises and a story of prison education
Abstract: I will talk about three results that surprised me. First, I will show that the free configuration space of an elastic wire is path-connected, a result that makes easy a manipulation planning problem that was thought to be hard. Second, I will show a linear relationship between stimulation parameters, skin impedance, and sensation intensity [...]
Robots Learning from Human Teachers
Abstract: In this talk I will cover some of the recent work out of the Socially Intelligent Machines Lab at UT Austin (http://sim.ece.utexas.edu/research.html). The vision of our research is to enable robots to function in dynamic human environments by allowing them to flexibly adapt their skill set via learning interactions with end-users. We explore the ways in which [...]
Creative Robots with Deep Reinforcement Learning
Recent advances in Deep Reinforcement Learning (DRL) algorithms provided us with the possibility of adding intelligence to robots. Recently, we have been applying a variety of DRL algorithms to the tasks that modern control theory may not be able to solve. We observed intriguing creativity from robots when they are constrained in reaching a certain [...]
Active Learning in Robot Motion Control
Abstract: Motion motivated by information needs can be found throughout natural systems, yet there is comparatively little work in robotics on analyzing and synthesizing motion for information. Instead, engineering analysis of robots and animal motion typically depends on defining objectives and rewards in terms of states and errors on states. This is how we formulate [...]
Formalizing Teamwork in Human-Robot Interaction
Abstract: Robots out in the world today work for people but not with people. Before robots can work closely with ordinary people as part of a human-robot team in a home or office setting, robots need the ability to acquire a new mix of functional and social skills. Working with people requires a shared understanding [...]
Microsystems-inspired robotics
Abstract: The ability to manufacture micro-scale sensors and actuators has inspired the robotics community for over 30 years. There have been huge success stories; MEMS inertial sensors have enabled an entire market of low-cost, small UAVs. However, the promise of ant-scale robots has largely failed. Ants can move high speeds on surfaces from picnic tables [...]
Robotic Grippers for Planetary Applications
Abstract: The previous generation of NASA missions to the outer solar system discovered salt water oceans on Europa and Enceladus, each with more liquid water than Earth – compelling targets to look for extraterrestrial life. Closer to home, JAXA and NASA have imaged sky-light entrances to lava tube caves on the Moon more than 100 [...]
Improving Multi-fingered Robot Manipulation by Unifying Learning and Planning
Abstract: Multi-fingered hands offer autonomous robots increased dexterity, versatility, and stability over simple two-fingered grippers. Naturally, this increased ability comes with increased complexity in planning and executing manipulation actions. As such, I propose combining model-based planning with learned components to improve over purely data-driven or purely-model based approaches to manipulation. This talk examines multi-fingered autonomous [...]
Design, Modeling and Control of a Robot Bat: From Bio-inspiration to Engineering Solutions
Abstract: In this talk, I will describe our recent work building a biologically-inspired bat robot. Bats have a complex skeletal morphology, with both ball-and-socket and revolute joints that interconnect the bones and muscles to create a musculoskeletal system with over 40 degrees of freedom, some of which are passive. Replicating this biological system in a [...]
Deep Learning for Robotics
Abstract: Programming robots remains notoriously difficult. Equipping robots with the ability to learn would by-pass the need for what otherwise often ends up being time-consuming task specific programming. This talk will describe recent progress in deep reinforcement learning (robots learning through their own trial and error), in apprenticeship learning (robots learning from observing people), and [...]
Modeling, Design, and Analysis for Intelligent Vehicles: Intersection Management, Security-Aware Design, and Automotive Design Automation
Abstract: Advanced Driver Assistance Systems (ADAS), autonomous functions, and connected applications bring a revolution to automotive systems and software. In this talk, several research topics in the domain of automotive systems and software will be introduced: (1) graph-based modeling, scheduling, and verification for intersection management, (2) security-aware design and analysis considering timing, game theory, and [...]
DNA and gammaPNA in programmable nanomaterials for sensing, robotics and manufacturing
Abstract: When programmable nanomaterials are used in conjunction with rapid microfabrication techniques like two photon polymerization, it becomes possible to rapidly prototype microstructures with nanoscale components. In this research presentation I introduce DNA nanotechnology using a commonly used simple nanotube motif, and I will illustrate how nucleic acid nanotubes can be used in sensing, robotics [...]
The Robots are Coming – to your Farm! AKA: Autonomous and Intelligent Robots in Unstructured Field Environments
Abstract: What if a team of collaborative autonomous robots grew your food for you? In this talk, I will discuss some key advances in robotics, machine learning, and autonomy that will one day enable teams of small robots to grow food for you in your backyard in a fundamentally more sustainable way than modern mega-farms! [...]
Improving Robot and Deep Reinforcement Learning via Quality Diversity and Open-Ended Algorithms
Abstract: Quality Diversity (QD) algorithms are those that seek to produce a diverse set of high-performing solutions to problems. I will describe them and a number of their positive attributes. I will then summarize our Nature paper on how they, when combined with Bayesian Optimization, produce a learning algorithm that enables robots, after being damaged, to adapt in 1-2 minutes [...]
Toward telelocomotion: human sensorimotor control of contact-rich robot dynamics
Abstract: Human interaction with the physical world is increasingly mediated by automation -- planes assist pilots, cars assist drivers, and robots assist surgeons. Such semi-autonomous machines will eventually pervade our world, doing dull and dirty work, assisting the elderly and disabled, and responding to disasters. Recent results (e.g. from the DARPA Robotics Challenge) demonstrate that, [...]
Formal Synthesis for Robots
Abstract: In this talk I will describe how formal methods such as synthesis – automatically creating a system from a formal specification – can be leveraged to design robots, explain and provide guarantees for their behavior, and even identify skills they might be missing. I will discuss the benefits and challenges of synthesis techniques and [...]
Extreme Motions in Biological and Engineered Systems
Abstract: Dr. Temel’s work mainly focuses on understanding the dynamics and energetics of extreme motions in small-scale natural and synthetic systems. Small-scale biological systems achieve extraordinary accelerations, speeds, and forces that can be repeated with minimal costs throughout the life of the organism. Zeynep uses analytical and computational models as well as physical prototypes to learn about these systems, test [...]
CANCELLED
Yes, That’s a Robot in Your Grocery Store. Now what?
Abstract: Retail stores are becoming ground zero for indoor robotics. Fleet of different robots have to coexist with each others and humans every day, navigating safely, coordinating missions, and interacting appropriately with people, all at large scale. For us roboticists, stores are giant labs where we're learning what doesn't work and iterating. If we get [...]
CANCELLED
Abstract: Before learning robots can be deployed in the real world, it is critical that probabilistic guarantees can be made about the safety and performance of such systems. In recent years, safe reinforcement learning algorithms have enjoyed success in application areas with high-quality models and plentiful data, but robotics remains a challenging domain for scaling [...]
Optimizing for coordination with people
https://youtu.be/AQ-w5o2oGI8 Abstract: From autonomous cars to quadrotors to mobile manipulators, robots need to co-exist and even collaborate with humans. In this talk, we will explore how our formalism for decision making needs to change to account for this interaction, and dig our heels into the subtleties of modeling human behavior -- sometimes strategic, often irrational, [...]
Scaling Probabilistically Safe Learning to Robotics
Abstract: Before learning robots can be deployed in the real world, it is critical that probabilistic guarantees can be made about the safety and performance of such systems. In recent years, safe reinforcement learning algorithms have enjoyed success in application areas with high-quality models and plentiful data, but robotics remains a challenging domain for [...]
From kinematic to energetic design and control of wearable robots for agile human locomotion
Abstract: Even with the help of modern prosthetic and orthotic (P&O) devices, lower-limb amputees and stroke survivors often struggle to walk in the home and community. Emerging powered P&O devices could actively assist patients to enable greater mobility, but these devices are currently designed to produce a small set of pre-defined motions. Finite state machines [...]
The World’s Tiniest Space Program
Abstract: The aerospace industry has experienced a dramatic shift over the last decade: Flying a spacecraft has gone from something only national governments and large defense contractors could afford to something a small startup can accomplish on a shoestring budget. A virtuous cycle has developed where lower costs have led to more launches and the [...]
A future with affordable Self-driving vehicles
(Video to appear once approved) Abstract: We are on the verge of a new era in which robotics and artificial intelligence will play an important role in our daily lives. Self-driving vehicles have the potential to redefine transportation as we understand it today. Our roads will become safer and less congested, while parking spots will be repurposed as leisure [...]
Robotics and Biosystems
Abstract: Research at the Center for Robotics and Biosystems at Northwestern University encompasses bio-inspiration, neuromechanics, human-machine systems, and swarm robotics, among other topics. In this talk I will give an overview of some of our recent work on in-hand manipulation, robot locomotion on yielding ground, and human-robot systems. Biography: Kevin Lynch received the B.S.E. degree [...]
Drones in Public: distancing and communication with all users
Abstract: This talk will focus on the role of human-robot interaction with drones in public spaces and be focused on two individual research areas: proximal interactions in shared spaces and improved communication with both end-users and bystanders. Prior work on human-interaction with aerial robots has focused on communication from the users or about the intended direction [...]
Data Scalability for Robot Learning
Abstract: Recent progress in robot learning has demonstrated how robots can acquire complex manipulation skills from perceptual inputs through trial and error, particularly with the use of deep neural networks. Despite these successes, the generalization and versatility of robots across environment conditions, tasks, and objects remains a major challenge. And, unfortunately, our existing algorithms and [...]
Carnegie Mellon University
Learning to Generalize beyond Training
Abstract: Generalization, i.e., the ability to adapt to novel scenarios, is the hallmark of human intelligence. While we have systems that excel at cleaning floors, playing complex games, and occasionally beating humans, they are incredibly specific in that they only perform the tasks they are trained for and are miserable at generalization. One of the [...]
Enabling Robots to Cooperate & Compete: Distributed Optimization & Game Theoretic Methods for Multiple Interacting Robots
Abstract: For robots to effectively operate in our world, they must master the skills of dynamic interaction. Autonomous cars must safely negotiate their trajectories with other vehicles and pedestrians as they drive to their destinations. UAVs must avoid collisions with other aircraft, as well as dynamic obstacles on the ground. Disaster response robots must coordinate [...]
The Role of Manipulation Primitives in Building Dexterous Robotic Systems
Abstract: I will start this talk by illustrating four different perspectives that we as a community have embraced to study robotic manipulation: 1) controlling a simplified model of the mechanics of interaction with an object; 2) using haptic feedback such as force or tactile to control the interaction with an environment; 3) planning sequences or [...]
Design and Analysis of Open-Source Educational Haptic Devices
Abstract: The sense of touch (haptics) is an active perceptual system used from our earliest days to discover the world around us. However, formal education is not designed to take advantage of this sensory modality. As a result, very little is known about the effects of using haptics in K-12 and higher education or the [...]
Move over, MSE! – New probabilistic models of motion
Abstract: Data-driven character animation holds great promise for games, film, virtual avatars and social robots. A "virtual AI actor" that moves in response to intuitive, high-level input could turn 3D animators into directors, instead of requiring them to laboriously pose the character for each frame of animation, as is the case today. However, the high [...]
Human-Robot Interactive Collaboration & Communication
Abstract: Autonomous and anthropomorphic robots are poised to play a critical role in manufacturing, healthcare and the services industry in the near future. However, for this vision to become a reality, robots need to efficiently communicate and interact with their human partners. Rather than traditional remote controls and programming languages, adaptive and transparent techniques for [...]
Carnegie Mellon University
Robots “R” Us: 25 years of Robotics Technology Development and Commercialization at NREC
Abstract: Since its founding in 1979, the Robotics Institute (RI) at Carnegie Mellon University has been leading the world in robotics research and education. In the mid 1990s, RI created NREC as the applied R&D center within the Institute with a specific mission to apply robotics technology in an impactful way on real-world applications. In this talk, I will go over [...]
Towards an Intelligence Architecture for Human-Robot Teaming
Abstract: Advances in autonomy are enabling intelligent robotic systems to enter human-centric environments like factories, homes and workplaces. To be effective as a teammate, we expect robots to accomplish more than performing simplistic repetitive tasks; they must perceive, reason, perform semantic tasks in a human-like way. A robot's ability to act intelligently is fundamentally tied [...]
GANs for Everyone
Abstract: The power and promise of deep generative models such as StyleGAN, CycleGAN, and GauGAN lie in their ability to synthesize endless realistic, diverse, and novel content with user controls. Unfortunately, the creation and deployment of these large-scale models demand high-performance computing platforms, large-scale annotated datasets, and sophisticated knowledge of deep learning methods. This makes [...]
Design and control of insect-scale bees and dog-scale quadrupeds
Abstract: Enhanced robot autonomy---whether it be in the context of extended tether-free flight of a 100mg insect-scale flapping-wing micro aerial vehicle (FWMAV), or long inspection routes for a quadrupedal robot---is hindered by fundamental constraints in power and computation. With this motivation, I will discuss a few projects I have worked on to circumvent these issues in [...]
Dynamical Robots via Origami-Inspired Design
Abstract: Origami-inspired engineering produces structures with high strength-to-weight ratios and simultaneously lower manufacturing complexity. This reliable, customizable, cheap fabrication and component assembly technology is ideal for robotics applications in remote, rapid deployment scenarios that require platforms to be quickly produced, reconfigured, and deployed. Unfortunately, most examples of folded robots are appropriate only for small-scale, low-load [...]
The Search for Ancient Life on Mars Began with a Safe Landing
Abstract: Prior mars rover missions have all landed in flat and smooth regions, but for the Mars 2020 mission, which is seeking signs of ancient life, this was no longer acceptable. To maximize the variety of rock samples that will eventually be returned to earth for analysis, the Perseverance rover needed to land in a [...]
Robotic Cave Exploration for Search, Science, and Survey
Abstract: Robotic cave exploration has the potential to create significant societal impact through facilitating search and rescue, in the fight against antibiotic resistance (science), and via mapping (survey). But many state-of-the-art approaches for active perception and autonomy in subterranean environments rely on disparate perceptual pipelines (e.g., pose estimation, occupancy modeling, hazard detection) that process the same underlying sensor data in [...]
Enabling Grounded Language Communication for Human-Robot Teaming
Abstract: The ability for robots to effectively understand natural language instructions and convey information about their observations and interactions with the physical world is highly dependent on the sophistication and fidelity of the robot’s representations of language, environment, and actions. As we progress towards more intelligent systems that perform a wider range of tasks in a [...]
Robots that Learn through Language
Abstract: Advances in perception have been integral to transitioning robots from machines restricted to factory automation to autonomous agents that operate robustly in unstructured environments. As our surrogates, robots enable people to explore the deepest depths of the ocean and distant regions of space, making discoveries that would otherwise be impossible. The age of robots [...]
Towards Reconstructing Any Object in 3D
Abstract: The world we live in is incredibly diverse, comprising of over 10k natural and man-made object categories. While the computer vision community has made impressive progress in classifying images from such diverse categories, the state-of-the-art 3D prediction systems are still limited to merely tens of object classes. A key reason for this stark difference [...]
The Unusual Effectiveness of Abstractions for Assistive AI
Abstract: Can we balance efficiency and reliability while designing assistive AI systems? What would such AI systems need to provide? In this talk I will present some of our recent work addressing these questions. In particular, I will show that a few fundamental principles of abstraction are surprisingly effective in designing efficient and reliable AI [...]
Robotics and Warehouse Automation at Berkshire Grey
Abstract: This talk tells the Berkshire Grey story, from its founding in 2013 to its IPO earlier this year — the first robotics IPO since iRobot over15 years ago. Berkshire Grey produces automated systems for e-commerce order fulfillment, parcel sortation, store replenishment, and related operations in warehouses, distribution centers, and in the back ends of [...]
Resilient Exploration in SubT Environments: Team Explorer’s Approach and Lessons Learned in the Final Event
Abstract: Subterranean robot exploration is difficult with many mobility, communications, and navigation challenges that require an approach with a diverse set of systems, and reliable autonomy. While prior work has demonstrated partial successes in addressing the problem, here we convey a comprehensive approach to address the problem of subterranean exploration in a wide range of [...]
Towards Robust Human-Robot Interaction: A Quality Diversity Approach
Abstract: The growth of scale and complexity of interactions between humans and robots highlights the need for new computational methods to automatically evaluate novel algorithms and applications. Exploring the diverse scenarios of interaction between humans and robots in simulation can improve understanding of complex human-robot interaction systems and avoid potentially costly failures in real-world settings. [...]
Lessons from the Field: Deep Learning and Machine Perception for field robots
Abstract: Mobile robots now deliver vast amounts of sensor data from large unstructured environments. In attempting to process and interpret this data there are many unique challenges in bridging the gap between prerecorded data sets and the field. This talk will present recent work addressing the application of machine learning techniques to mobile robotic perception. [...]
Distributed Dissipativity: Applying Foundational Stability Theory to Modern Networked Control
Abstract: Despite its diverse areas of application, the desire to optimize performance and guarantee acceptable behaviour in the face of inevitable uncertainty is pervasive throughout control theory. This creates a fundamental challenge since the necessity of robustly stable control schemes often favors conservative designs, while the desire to optimize performance typically demands the opposite. While [...]
Haptic Perspective-taking from Vision and Force
Abstract: Physically collaborative robots present an opportunity to positively impact society across many domains. However, robots currently lack the ability to infer how their actions physically affect people. This is especially true for robotic caregiving tasks that involve manipulating deformable cloth around the human body, such as dressing and bathing assistance. In this talk, I [...]
Perception-Action Synergy in Uncertain Environments
Abstract: Many robotic applications require a robot to operate in an environment with unknowns or uncertainty, at least initially, before it gathers enough information about the environment. In such a case, a robot must rely on sensing and perception to feel its way around. Moreover, it has to couple sensing/perception and motion synergistically in real [...]
Designing Robotic Systems with Collective Embodied Intelligence
Abstract: Natural swarms exhibit sophisticated colony-level behaviors with remarkable scalability and error tolerance. Their evolutionary success stems from more than just intelligent individuals, it hinges on their morphology, their physical interactions, and the way they shape and leverage their environment. Mound-building termites, for instance, are believed to use their own body as a template for [...]
Snakes & Spiders, Robots & Geometry
Abstract: Locomotion and perception are a common thread between robotics and biology. Understanding these phenomena at a mechanical level involves nonlinear dynamics and the coordination of many degrees of freedom. In this talk, I will discuss geometric approaches to organizing this information in two problem domains: Undulatory locomotion of snakes and swimmers, and vibration propagation [...]
Robotic Cave Exploration for Search, Science, and Survey
Abstract: Robotic cave exploration has the potential to create significant societal impact through facilitating search and rescue, in the fight against antibiotic resistance (science), and via mapping (survey). But many state-of-the-art approaches for active perception and autonomy in subterranean environments rely on disparate perceptual pipelines (e.g., pose estimation, occupancy modeling, hazard detection) that process the same underlying sensor data in different [...]
Safe and Stable Learning for Agile Robots without Reinforcement Learning
Abstract: My research group (https://aerospacerobotics.caltech.edu/) is working to systematically leverage AI and Machine Learning techniques towards achieving safe and stable autonomy of safety-critical robotic systems, such as robot swarms and autonomous flying cars. Another example is LEONARDO, the world's first bipedal robot that can walk, fly, slackline, and skateboard. Stability and safety are often research problems [...]
Towards $1 robots
Abstract: Robots are pretty great -- they can make some hard tasks easy, some dangerous tasks safe, or some unthinkable tasks possible. And they're just plain fun to boot. But how many robots have you interacted with recently? And where do you think that puts you compared to the rest of the world's people? In [...]
What (else) can you do with a robotics degree?
Abstract: In 2004, half-way through my robotics Ph.D., I had a panic-inducing thought: What if I don’t want to build robots for the rest of my life? What can I do with this degree?! Nearly twenty years later, I have some answers: tackle climate change in Latin America, educate Congress about autonomous vehicles, improve how [...]
Robots Should Reduce, Reuse, and Recycle
Abstract: Despite numerous successes in deep robotic learning over the past decade, the generalization and versatility of robots across environments and tasks has remained a major challenge. This is because much of reinforcement and imitation learning research trains agents from scratch in a single or a few environments, training special-purpose policies from special-purpose datasets. In [...]
Machine Learning and Model Predictive Control for Adaptive Robotic Systems
Abstract: In this talk I will discuss several different ways in which ideas from machine learning and model predictive control (MPC) can be combined to build intelligent, adaptive robotic systems. I’ll begin by showing how to learn models for MPC that perform well on a given control task. Next, I’ll introduce an online learning perspective on [...]
Learning Representations for Interactive Robotics
In this talk, I will be discussing the role of learning representations for robots that interact with humans and robots that interactively learn from humans through a few different vignettes. I will first discuss how bounded rationality of humans guided us towards developing learned latent action spaces for shared autonomy. It turns out this “bounded rationality” is not a [...]
Motion Planning Around Obstacles with Graphs of Convex Sets
Abstract: In this talk, I'll describe a new approach to planning that strongly leverages both continuous and discrete/combinatorial optimization. The framework is fairly general, but I will focus on a particular application of the framework to planning continuous curves around obstacles. Traditionally, these sort of motion planning problems have either been solved by trajectory optimization [...]
RE2 Robotics: from RI spinout to Acquisition
Abstract: It was July 2001. Jorgen Pedersen founded RE2 Robotics. It was supposed to be a temporary venture while he figured out his next career move. But the journey took an unexpected course. RE2 became a leading developer of mobile manipulation systems. Fast forward to 2022, RE2 Robotics exited via an acquisition to Sarcos Technology and [...]
Understanding the Physical World from Images
If I show you a photo of a place you have never been to, you can easily imagine what you could do in that picture. Your understanding goes from the surfaces you see to the ones you know are there but cannot see, and can even include reasoning about how interaction would change the scene. [...]
A Constructivist’s Guide to Robot Learning
Over the last decade, a variety of paradigms have sought to teach robots complex and dexterous behaviors in real-world environments. On one end of the spectrum we have nativist approaches that bake in fundamental human knowledge through physics models, simulators and knowledge graphs. While on the other end of the spectrum we have tabula-rasa approaches [...]
Next-Generation Robot Perception: Hierarchical Representations, Certifiable Algorithms, and Self-Supervised Learning
Spatial perception —the robot’s ability to sense and understand the surrounding environment— is a key enabler for robot navigation, manipulation, and human-robot interaction. Recent advances in perception algorithms and systems have enabled robots to create large-scale geometric maps of unknown environments and detect objects of interest. Despite these advances, a large gap still separates robot [...]
Structures and Environments for Generalist Agents
Abstract: We are entering an era of highly general AI, enabled by supervised models of the Internet. However, it remains an open question how intelligence emerged in the first place, before there was an Internet to imitate. Understanding the emergence of skillful behavior, without expert data to imitate, has been a longstanding goal of reinforcement [...]
Mars Robots and Robotics at NASA JPL
Abstract: In this seminar I’ll discuss Mars robots, the unprecedented results we’re seeing with the latest Mars mission, and how we got here. Perseverance’s manipulation and sampling systems have collected samples from unique locations at twice the rate of any prior mission. 88% of all driving has been autonomous. This has enabled the mission to [...]
Special RI Seminar
Title: Testing, Analysis, and Specification for Robust and Reliable Robot Software Abstract: Building robust and reliable robotic software is an inherently challenging feat that requires substantial expertise across a variety of disciplines. Despite that, writing robot software has never been easier thanks to software frameworks such as ROS: At its best, ROS allows newcomers to assemble simple, [...]
Transforming Hollywood Visual Effects with Graphics and Vision
Abstract: Paul will describe his path to developing visual effects technology used in hundreds of movies, including The Matrix, Spider-Man 2, Benjamin Button, Avatar, Maleficent, Furious 7, and Blade Runner: 2049. These techniques include image-based modeling and rendering, high dynamic range imaging, image-based lighting, and high-resolution facial scanning for photoreal digital actors. Paul will also [...]
Learning Meets Gravity: Robots that Learn to Embrace Dynamics from Data
Abstract: Despite the incredible capabilities (speed and repeatability) of our hardware today, many robot manipulators are deliberately programmed to avoid dynamics – moving slow enough so they can adhere to quasi-static assumptions of the world. In contrast, people frequently (and subconsciously) make use of dynamic phenomena to manipulate everyday objects – from unfurling blankets, to [...]
Learning and Control for Safety, Efficiency, and Resiliency of Embodied AI
Abstract: The rapid evolution of ubiquitous sensing, communication, and computation technologies has revolutionized of cyber-physical systems (CPS) across virous domains like robotics, smart grids, aerospace, and smart cities. Integrating learning into dynamic systems control presents significant Embodied AI opportunities. However, current decision-making frameworks lack comprehensive understanding of the tridirectional relationship among communication, learning and control, [...]
Data-Efficient Learning for Robotics and Reinforcement Learning
Abstract: Data efficiency, i.e., learning from small datasets, is of practical importance in many real-world applications and decision-making systems. Data efficiency can be achieved in multiple ways, such as probabilistic modeling, where models and predictions are equipped with meaningful uncertainty estimates, transfer learning, or the incorporation of valuable prior knowledge. In this talk, I will [...]
Robots at the Johnson Space Center and Future Plans
Abstract: The seminar will review a series of robotic systems built at the Johnson Space Center over the last 20 years. These will include wearable robots (exoskeletons, powered gloves and jetpacks), manipulation systems (ISS cranes down to human scale) and lunar mobility systems (human surface mobility and robotic rovers). As all robotics presentations should, this [...]
Becoming Teammates: Designing Assistive, Collaborative Machines
Abstract: The growing power in computing and AI promises a near-term future of human-machine teamwork. In this talk, I will present my research group’s efforts in understanding the complex dynamics of human-machine interaction and designing intelligent machines aimed to assist and collaborate with people. I will focus on 1) tools for onboarding machine teammates and [...]
Teaching a Robot to Perform Surgery: From 3D Image Understanding to Deformable Manipulation
Abstract: Robot manipulation of rigid household objects and environments has made massive strides in the past few years due to the achievements in computer vision and reinforcement learning communities. One area that has taken off at a slower pace is in manipulating deformable objects. For example, surgical robotics are used today via teleoperation from a [...]
Learning with Less
Abstract: The performance of an AI is nearly always associated with the amount of data you have at your disposal. Self-supervised machine learning can help – mitigating tedious human supervision – but the need for massive training datasets in modern AI seems unquenchable. Sometimes it is not the amount of data, but the mismatch of [...]
Why We Should Build Robot Apprentices And Why We Shouldn’t Do It Alone
Abstract: For robots to be able to truly integrate human-populated, dynamic, and unpredictable environments, they will have to have strong adaptive capabilities. In this talk, I argue that these adaptive capabilities should leverage interaction with end users, who know how (they want) a robot to act in that environment. I will present an overview of [...]
Toward an ImageNet Moment for Synthetic Data
Abstract: Data, especially large-scale labeled data, has been a critical driver of progress in computer vision. However, many important tasks remain starved of high-quality data. Synthetic data from computer graphics is a promising solution to this challenge, but still remains in limited use. This talk will present our work on Infinigen, a procedural synthetic data [...]
Reduced-Gravity Flights and Field Testing for Lunar and Planetary Rovers
Abstract: As humanity returns to the Moon and is developing outposts and related infrastructure, we need to understand how robots and work machines will behave in this harsh environment. It is challenging to find representative testing environments on Earth for Lunar and planetary rovers. To investigate the effects of reduced-gravity on interactions with granular terrains, [...]
Where’s RobotGPT?
Abstract: The last years have seen astonishing progress in the capabilities of generative AI techniques, particularly in the areas of language and visual understanding and generation. Key to the success of these models are the use of image and text data sets of unprecedented scale along with models that are able to digest such large [...]
Robot Learning by Understanding Egocentric Videos
Abstract: True gains of machine learning in AI sub-fields such as computer vision and natural language processing have come about from the use of large-scale diverse datasets for learning. In this talk, I will discuss how we can leverage large-scale diverse data in the form of egocentric videos (first-person videos of humans conducting different tasks) [...]
What Makes Learning to Control Easy or Hard?
Abstract: Designing autonomous systems that are simultaneously high-performing, adaptive, and provably safe remains an open problem. In this talk, we will argue that in order to meet this goal, new theoretical and algorithmic tools are needed that blend the stability, robustness, and safety guarantees of robust control with the flexibility, adaptability, and performance of machine [...]
Can Robots Based on Musculoskeletal Designs Better Interact With the World?
Abstract: Living robots represent a new frontier in engineering materials for robotic systems, incorporating biological living cells and synthetic materials into their design. These bio-hybrid robots are dynamic and intelligent, potentially harnessing living matter’s capabilities, such as growth, regeneration, morphing, biodegradation, and environmental adaptation. Such attributes position bio-hybrid devices as a transformative force in robotics [...]
Soft Wearable Haptic Devices for Ubiquitous Communication
Abstract: Haptic devices allow touch-based information transfer between humans and intelligent systems, enabling communication in a salient but private manner that frees other sensory channels. For such devices to become ubiquitous, their physical and computational aspects must be intuitive and unobtrusive. The amount of information that can be transmitted through touch is limited in large [...]
Building Generalist Robots with Agility via Learning and Control: Humanoids and Beyond
Abstract: Recent breathtaking advances in AI and robotics have brought us closer to building general-purpose robots in the real world, e.g., humanoids capable of performing a wide range of human tasks in complex environments. Two key challenges in realizing such general-purpose robots are: (1) achieving "breadth" in task/environment diversity, i.e., the generalist aspect, and (2) [...]