Abstract:

Many everyday human actions—like adjusting the grip on a tool or folding clothes—require the ability to handle complex dynamics involving shifting contact, deformability, or high-speed motion. While easy for humans, object manipulation with complex dynamics or rapidly changing contact conditions presents significant challenges for robots. Enabling robots to perform complex dynamic manipulations would greatly enhance their capabilities and effectiveness. This talk explores two different approaches addressing these challenges from the action and perception aspects. In the first part, we introduce  SWIFT, a trial-and-error optimization method enabling a soft robotic hand to perform a challenging dynamic in-hand manipulation task— pen spinning—without explicitly modeling the dynamics or using simulation. We show that the dynamic non-prehensile manipulation task can be solved with surprisingly simple gradient-free optimization. We also highlight the potential of soft robotics systems in solving fast, contact-rich manipulation tasks. In the second part, we tackle the challenge of accurately capturing the dynamic behavior of deformable objects for its potential in downstream applications such as object-centric imitation learning or digital twin creation. We introduce DeformGS, a dense tracking method based on Gaussian splatting and a learned deformation model capable of accurately and densely tracking points on deformable objects from visual inputs. Combined, these two parts advance our understanding of manipulation in complex dynamic environments and suggest potential directions for developing more capable and adaptive robotic systems.

Committee:

Prof. Jeffrey Ichnowski (advisor)

Prof. Christopher G. Atkeson

Uksang Yoo