Data Attribution for Text-to-Image Models
Abstract: Large text-to-image models learn from training data to synthesize "novel" images, but how the models use the training data remains a mystery. The problem of data attribution is to identify which training images are influential for generating a given output. Specifically, removing influential images and retraining the model would prevent it from reproducing that [...]
Differentiable Convex Modeling for Robotic Planning and Control
Abstract: Robotic simulation, planning, estimation, and control, have all been built on top of numerical optimization. In this same time, modern convex optimization has matured into a robust technology delivering globally optimal solutions in polynomial time. With advances in differentiable optimization and custom solvers capable of producing smooth derivatives, convex modeling has become fast, reliable, [...]
Knowledge and Data Dependence in Decision-Making
Abstract: This thesis explores diverse decision-making strategies for autonomous agents by examining knowledge-dependent and data-dependent approaches in stationary and dynamic data environments. We address five core research problems across three thematic areas: knowledge-dependent, stationary data-dependent, and evolving data-dependent decision-making. We first investigate knowledge-driven decision-making within robotic swarms, characterizing vulnerabilities in systems governed by consistent rule-following [...]
Communication Efficient and Differentially Private Optimization
Abstract: In recent years, the integration of communication efficiency and differential privacy in distributed optimization has gained significant attention, motivated by large-scale applications such as Federated Learning (FL), where both data privacy and efficient communication are critical. This thesis explores the development of novel techniques to address these challenges, with a focus on distributed mean [...]
Towards a Universal Data Engine for Robotics and Beyond
Abstract: Robotics researchers have been attempting to extend data-driven breakthroughs in fields like computer vision and language processing into robot learning. However, unlike vision or language domains where massive amounts of data is readily available on the internet, training robotic policies relies on physical and interactive data collected via interacting with the physical world -- [...]
Learning for Dynamic Robot Manipulation of Deformable and Transparent Objects
Abstract: Dynamics, softness, deformability, and difficult-to-detect objects will be critical for new domains in robotic manipulation. But there are complications--including unmodelled dynamic effects, infinite-dimensional state spaces of deformable objects, and missing features from perception. This talk explores learning methods based on multi-view sensing, acoustics, physics-based regularizations, and Koopman operators and proposes a novel multi-finger soft [...]
HaptiClay: An Interactive Haptic Interface for Gestured Concretization of Polynomial Functions
Abstract: In this work we present HaptiClay, a low-cost kinesthetic haptic interface that elevates the understanding of mathematics language by providing embodied non-verbal representations of math concepts. Our interface integrates four key components: a haptic device, a high-level simulation that communicates with a low-level controller for force and position updates, a low-level controller that executes [...]
High-resolution cloth simulation in milliseconds: Efficient GPU Cloth Simulation with Non-distance Barriers and Subspace Reuse Interactions
Abstract: We show how to push the performance of high-resolution cloth simulation, making the simulation interactive (in milliseconds) for models with one million degrees of freedom (DOFs) while keeping every triangle untangled. The guarantee of being penetration-free is inspired by the interior-point method, which converts the inequality constraints to barrier potentials. Nevertheless, we propose a [...]
Better Standards for Trajectory Forecasting: Data, Evaluation, and Methods
Abstract: Ensuring pedestrian safety in dynamic environments is a key challenge for autonomous systems, particularly in dynamic, multi-agent environments. Trajectory forecasting plays a central role in enabling these systems to anticipate pedestrian behaviors and respond appropriately. This thesis addresses three core limitations in trajectory forecasting systems which impede safe and robust trajectory forecasting: inadequate evaluation protocols [...]