Automatically Tracking and Calibrating Robot Arms using SLAM Techniques

Abstract

Robots still struggle with everyday manipulation tasks. An overriding problem with robotic manipulation is uncertainty in the robot’s state and calibration param- eters. Small amounts of uncertainty can lead to complete task failure. This thesis explores ways of tracking and calibrating noisy robot arms using computer vision, with an aim toward making them more robust. We consider three systems with in- creasing complexity: a noisy robot arm tracked by an external depth camera (chap- ter 2), a noisy arm that localizes itself using a hand-mounted depth sensor looking at an unstructured word (chapter 3), and a noisy arm that only has a single hand- mounted monocular RGB camera estimating its state while simultaneously calibrat- ing its camera extrinsics (chapter 4). Using techniques taken from dense object tracking, fully dense SLAM and sparse general SLAM, we are able to automati- cally track the robot and extract its calibration parameters. We also provide analy- sis linking these problems together, while exploring the fundamental limitations of SLAM-based approaches for calibrating robot arms.