Incorporation of Audio in Virtualized Reality

Abstract

Virtualized Reality is a visual medium that allows the user to view a scene from any viewpoint independent of the physical limitations imposed on the cameras capturing the scene. However the 3D models of video do not provide the complete simulation of the real world. We present an approach for the addition of audio to the existing video capture and recreation mechanisms. Using a technique analogous to video, multiple audio streams are captured to provide the spatial distribution of sound. A 3D model of audio is then generated to find the approximate sound as a function of the world coordinates. Synchronized with the video, this enables the user to both see and hear from any dynamically defined view-coordinates.