Controlling a motorized marionette with human motion capture data

Abstract

In this paper, we present a method for controlling a motorized, string-driven marionette using motion capture data from human actors and from a traditional marionette operated by a professional puppeteer. We are interested in using motion capture data of a human actor to control the motorized marionette as a way of easily creating new performances. We use data from the hand-operated marionette both as a way of assessing the performance of the motorized marionette and to explore whether this technology could be used to preserve marionette performances. The human motion data must be extensively adapted for the marionette because its kinematic and dynamic properties differ from those of the human actor in degrees of freedom, limb length, workspace, mass distribution, sensors, and actuators. The motion from the hand-operated marionette requires less adaptation because the controls and dynamics are a closer match. Both data sets are adapted using an inverse kinematics algorithm that takes into account marker positions, joint motion ranges, string constraints, and potential energy. We also apply a feedforward controller to prevent extraneous swings of the hands. Experimental results show that our approach enables the marionette to perform motions that are qualitatively similar to the original human motion capture data.