Robot Learning by Understanding Egocentric Videos
Abstract: True gains of machine learning in AI sub-fields such as computer vision and natural language processing have come about from the use of large-scale diverse datasets for learning. In this talk, I will discuss how we can leverage large-scale diverse data in the form of egocentric videos (first-person videos of humans conducting different tasks) [...]
Special Seminar
Speaker: Abhisesh Silwal Title: Robotics and AI for Sustainable Agriculture Abstract: Production agriculture plays a critical role in our lives, providing food security and enabling sustainability. Despite its immense importance, it currently faces many challenges including shortage of farmworkers, increasing production costs, excess use of herbicides just to name a few. Robotics and artificial intelligence-based [...]
Learning to Manipulate Using Diverse Datasets
Abstract: Autonomous agents can play games (like Chess, Go, and even Starcraft), they can help make complex scientific predictions (e.g., protein folding), and they can even write entire computer programs, with just a bit of prompting. However, even the most basic physical manipulation skills, like unlocking and opening a door, still remain literally out-of-reach. The [...]
Passive Ultra-Wideband Single-Photon Imaging
Abstract: High-speed light sources, fast cameras, and depth sensors have made it possible to image dynamic phenomena occurring in ever smaller time intervals with the help of actively-controlled light sources and synchronization. Unfortunately, while these techniques do capture ultrafast events, they cannot simultaneously capture slower ones too. I will discuss our recent work on passive ultra-wideband [...]
3D Perception In-The-Wild
Abstract: State estimation is a fundamental component of embodied perception. Contemporary autonomous vehicle (AV) benchmarks have advanced techniques for training 3D detectors, particularly on large-scale data. Notably, although prior work has nearly solved 3D object detection for a few common classes (e.g., pedestrian and car), detecting many rare classes in-the-tail (e.g., debris and stroller) remains [...]
Learning on the Move: Integrating Action and Perception for Mobile Manipulation
Abstract: While there has been remarkable progress recently in the fields of manipulation and locomotion, mobile manipulation remains a long-standing challenge. Compared to locomotion or static manipulation, a mobile system must make a diverse range of long-horizon tasks feasible in unstructured and dynamic environments. While the applications are broad and interesting, there are a plethora [...]
Differentiable Convex Modeling for Robotic Planning and Control
Abstract: Robotic simulation, planning, estimation, and control, have all been built on top of numerical optimization. In this same time, modern convex optimization has matured into a robust technology delivering globally optimal solutions in polynomial time. With advances in differentiable optimization and custom solvers capable of producing smooth derivatives, convex modeling has become fast, reliable, [...]
Simulation-Driven Soft Robotics
Abstract: Soft-bodied robots present a compelling solution for navigating tight spaces and interacting with unknown obstacles, with potential applications in inspection, medicine, and AR/VR. Yet, even after a decade, soft robots remain largely in the prototype phase without scaling to the tasks where they show the most promise. These systems are difficult to design and [...]
Plan to Learn: Active Robot Learning by Planning
Abstract: Robots need a diverse repertoire of capable motor skills to succeed in the open world. Such a skillset cannot be learned or designed purely on human initiative. In this thesis, we advocate for an active continual learning approach that enables robots to take charge of their own learning. The goal of an autonomously learning [...]