Fingersight: Fingertip Visual Haptic Sensing and Control

Abstract

We present a new method of sensing the 3D visual environment and controlling objects within it. The human hand is well suited to interrogate and manipulate objects by physical contact; however, the hand is limited to surfaces within its reach. Extending the hand? innate ability, we mount miniature cameras on individual fingertips, permitting rapid sweeping through the 3D visual environment at greater distances. The information gleaned from each fingertip camera is fed back to that finger by a small vibrator, so the sense of touch remains related to each finger? individual interaction with the environment. The muscles of the fingers, wrist, and arm can potentially provide motor control as fine as the eyes?(and we have ten fingers instead of only two eyes) although the sensory resolution and total bandwidth is likely to be less. Metaphorically speaking, we have given eyesight to the fingers, and thus we call the resulting capability, ?ingersightTM.?In addition to sensing the environment, we can remotely control certain identified targets with subsequent motion of the fingers. We can move or rotate objects on a computer screen or recognize inanimate objects such as a light switch, controlling environmental parameters with a flick of the finger.