Interaction of Visual and Haptic Cues in the Image-based Perception of Depth

Abstract

Many medical applications attempt to locate targets by using imaging techniques such as ultrasound. If the target is located in a compressible medium (e.g., human tissue), however, its position in the ultrasound image will shift as the medium is compressed. We investigated whether users can accommodate to such displacements by using visual and haptic cues and accurately judge target depth. Subjects were asked to locate targets underneath a soft rubber surface. Visual cues to the amount of compression were provided by a grid on the surface that deformed under pressure and by the visible displacement of the tip of the ultrasound probe. The first experiment tested whether these visual cues are sufficient for judging surface deformation and compensating so as to accurately locate the target. Subjects acquired ultrasound images of targets at different depths and localized them with a triangulation-by-pointing procedure. Using conventional ultrasound with a remote display, subjects consistently underestimated surface deformation and thus target depth. In a second experiment, haptic feedback was added so that resisting force increased with surface deformation. We found that the stiffer the surface, the less the underestimation of target depth due to compression. A third experiment used a different imaging display, the Sonic Flashlight, an augmented-reality tool that enables users to directly see the target in 3D space. The perception of target location with this device was accurate despite the surface compliance. An ongoing experiment is further examining the learning and transfer of skills to correct the compliance effect.