Manipulation Among Movable Objects for Pick-and-Place Tasks in Cluttered 3D Workspaces

Abstract

In cluttered real-world workspaces, simple pick-and-place tasks for robot manipulators can be quite challenging to solve. Often there is no collision-free trajectory that allows the robot to grasp and extract a desired object from the scene. This requires motion planning algorithms to reason about rearranging some of the "movable" clutter in the scene so as to make the task feasible. Our work focuses on solving these pick-and-place tasks in 3D workspaces where objects may tilt, lean on each other, topple, and slide. This problem setup can be used to describe common uses of robot manipulators tasked with packing boxes in warehouses, assembling structures in manufacturing industries, and assisting humans inside households. Robots operating in these environments will engage in contact-rich interactions with objects in the environment, and must be able to model and reason about the effect of these interactions on the environment. In this thesis we develop motion planning algorithms for robotic manipulation that make efficient use of a physics-based rigid-body simulator to solve pick-and-place manipulation tasks while accounting for the configuration of all objects in the environment and the effect that robot actions have on them. In particular, we require our planning algorithms to compute trajectories that satisfy all object-centric "interaction constraints". In the 3D workspaces we care about, these encode properties of objects such as whether the robot can make contact with an object, how far it can make objects tilt, whether objects are allowed to topple over, how fast they can be moved etc. The challenge in solving simple pick-and-place tasks in these environments lies in finding a solution trajectory that satisfies these constraints at all points in time.

In environments with relatively large object-free volumes, we show that it suffices to only model robot-object and any resultant object-object interactions near grasp poses given to the robot. These grasp poses are specified as intermediate goals for the robot from where the desired object may be grasped before being moved to its final goal (or "home") configuration. Our intuition in these workspaces suggests that we can exploit the large object-free volume to plan contact-free trajectories that get the robot near the grasp pose. Once such a trajectory has been found, the planner can forward simulate dynamically generated grasping primitives in the physics-based simulator to ensure that interaction constraints are not violated while grasping the target object.

With more clutter in the scene or in a more constrained workspace, we cannot assume the existence of a contact-free trajectory that gets close to the target object. In such cases, we have to reason about deliberately rearranging any movable clutter so that the object-of-interest (OoI) can be grasped and extracted. Such situations arise commonly when a robot must reach inside a cluttered shelf to retrieve a desired object. Manipulation planning algorithms for such scenes must be able to answer three fundamental questions -- which movable objects should the robot rearrange, where should they be moved, and how should the robot move them while still satisfying all interaction constraints? This thesis presents a family of algorithms that draw on a unique connection between Multi-Agent Pathfinding (MAPF), the problem of finding paths for multiple robots that need to get from their start states to some desired goal states, and Manipulation Among Movable Objects (MAMO), solving manipulation tasks in environments where the robot is allowed to rearrange movable clutter. Our algorithms solve an appropriately constructed MAPF abstraction of MAMO to answer the first two questions of which objects should be moved and where to. We then convert the MAPF solution into rearrangement actions for the robot to rearrange movable objects and make progress towards solving the MAMO problem of retrieving the OoI from a cluttered shelf.

We test all algorithms developed as part of this thesis in the real-world with the PR2 robot. We show that we can execute trajectories returned by our algorithms on the PR2 to solve complex manipulation tasks that require rearranging objects on a refrigerator shelf before reaching inside to grasp and extract a desired object. Our work takes a step towards solving MAMO problems in realistic real-world workspaces and we conclude this thesis by discussing some possible directions for future work to develop even more capable and versatile MAMO planning algorithms.