Student Talks
Strategy assessment for solving rich physical problems
Abstract: We present a framework that acts as an "intuitive physics reasoner" which takes in strategies expressed in natural language (whether from a human or LLM), and assesses their validity based on a physics knowledge library. We believe the ability to quickly determine whether a strategy is worth considering and allocating further resources to planning [...]
MSR Thesis Talk: Siva Kailas
Title: Multi-Robot Information Gathering for Spatiotemporal Environment Modelling Abstract: Learning to predict or forecast spatiotemporal (ST) environmental processes from a sparse set of samples collected autonomously is a difficult task from both a sampling perspective (collecting the best sparse samples) and from a learning perspective (predicting unseen locations or forecasting the next timestep). We investigate [...]
MRS Thesis Talk: Ruijie Fu
Title: Towards Mechanical Communication in Multi-Agent Locomotive Systems: Principally Kinematic Robots on a Shared Platform Abstract: Many biological multi-agent systems exhibit a mechanism for information exchange among individuals known as mechanical communication, which leads to the emergence of collective behavior within the group. One such example is the swarming behavior of bacteria, where they form rafts [...]
Personalized Context-aware Affective Nonverbal Robot Feedback
Abstract: We first consider the problem of estimating context, specifically key features of the human state. We predict engagement-related events in an educational activity before the end of that activity, which could allow the robot to provide feedback early enough to improve the human's experience. We then explore generating nonverbal affective robot behavior by correlating [...]
Redefining the Perception-Action Interface: Visual Action Representations for Contact-Centric Manipulation
Abstract: In robotics, understanding the link between perception and action is pivotal. Typically, perception systems process sensory data into state representations like segmentations and bounding boxes, which a planner uses to plan actions. However, this state estimation approach can fail in environments with partial observability, and in cases with challenging object properties like transparency and deformability. [...]
Architecture and Algorithms for Space-Based Global Wildlife Tracking
Abstract: Accurate satellite based positioning revolutionized several industries over the past two decades from agriculture to transportation. However, conventional GNSS receivers consume significant amounts of energy and are too large for many applications, including wildlife-tracking which is critical for conservation efforts and improving our understanding of the global climate. To address this capability gap, we [...]
Multi-Human 3D Reconstruction from Monocular Videos
Abstract: We study the problem of multi-human 3D reconstruction from videos captured in the wild. Human movements are dynamic, and accurately reconstructing them in various settings is crucial for developing immersive social telepresence, assistive humanoid robots, and augmented reality systems. However, creating such a system requires addressing fundamental issues with previous works regarding the data [...]
Language-Conditioned Object Detection and Manipulation
Abstract: Traditional object detection methods are often confined to predefined object vocabularies, limiting their versatility in real-world scenarios where robots need to understand and execute diverse household tasks. Additionally, the 2D and 3D perception communities have typically pursued separate approaches tailored to their respective domains. In this thesis, we present a language-conditioned object detector with [...]
How I Learned to Love Blobs: The Power of Gaussian Representations in Differentiable Rendering and Optimization
Abstract: In this thesis, we explore the use of Gaussian Representations in multiple application areas of computer vision and robotics. In particular, we design a ray-based differentiable renderer for 3D Gaussians that can be used to solve multiple classic computer vision problems in a unified manner. For example, we can reconstruct 3D shapes from color, [...]