Vision-Language Models for Hand-Object Interaction Prediction
Abstract: How can we predict future interaction trajectories of human hands in a scene given high-level colloquial task specifications in the form of natural language? In this paper, we extend the classic hand trajectory prediction task to two tasks involving explicit or implicit language queries. Our proposed tasks require extensive understanding of human daily activities [...]
Robotics Institute Winter Party
All Robotics Institute Faculty. Staff, Students, and Visitors are invited to attend this event. Please join us for food, beverages, and casual conversation with colleagues. A calendar invite including details will be sent closer to the event.
In Search of Shmoo: The Quest to Build a Capable Low-cost Ballbot for Research and Education
RI Faculty Business Meeting
Meeting for RI Faculty. Agenda was sent via a calendar invite.
Universal Semantic-Geometric Priors for Zero-Shot Robotic Manipulation
Abstract: Visual imitation learning has shown promising results in robotic manipulation in recent years. However, its generalization to unseen objects is often limited by the size and diversity of training data. Although more large-scale robotic datasets are available, they remain significantly smaller than image and text datasets. Additionally, scaling these datasets is time-consuming and labor-intensive, [...]
RI Faculty Business Meeting
Meeting for RI Faculty. Agenda was sent via a calendar invite.
Personalized Context-aware Multimodal Robot Feedback
Abstract: In the field of human-robot interaction (HRI), integration of robots into social settings, such as healthcare and education, is gaining traction. Robots that provide individualized support to improve human performance and subjective experience will generally be more successful in these domains. Robots should personalize their interactions, be aware of the contextual nuances surrounding their [...]
Sensorized Soft Materials Systems with Integrated Electronics and Computing
Abstract: The integration of soft and multifunctional materials in emerging technologies is becoming more widespread due to their ability to enhance or improve functionality in ways not possible using typical rigid alternatives. This trend is evident in various fields. For example, wearable technologies are increasingly designed using soft materials to improve modulus compatibility with biological [...]
Enabling Reliable Model-Based Planning with Inaccurate Models
Abstract: This thesis aims to provide a framework for combining complementary tools that enable robots to manipulate objects in the world using diverse forms of knowledge. We consider heterogeneous types of knowledge, such as physics-based models, learned dynamics models, and model-free skills learned from human demonstrations. Each form of knowledge comes with its own assumptions [...]
Unlocking Generalization for Robotics via Scale and Modularity
Abstract: How can we build generalist robot systems? Looking at fields such as vision and language, the common theme has been large scale end-to-end learning with massive, curated datasets. In robotics, on the other hand, scale alone may not be enough due to the significant multimodality of robotics tasks, lack of easily accessible data and [...]