Modeling from Reality: Representing and Integration

Abstract

Traditional virtual reality systems rely on manual construction of the virtual environment, which is labor-intensiveand usually not very realistic. This thesis describes a "modeling-from-reality " system which observes from multiple viewpoints, analyzes the geometrical shape of a scene or an object, and subsequently integrates the multiple views to build a complete scene or object model form the existing environment. Our system produces statistically optimal objects by adopting a new approach to modeling from reality -- an integral approach to object modeling. The integral approach consists of two parts: how to integrate and what to integrate. Using a polyhedral objecty as an example, this thesis shows how to integrate multiple views in a statistically optimal fashion. It is illustrated that multiple view integration can be formulated as a problem of principle component analysis with missing data (PCAMD). In spite of the noisy input and missing data at each view, the PCAMD algorithm makes use of input redundancy among different views and guarantees that the recovered object model is statistically optimal. It is shown that the problem of PCAMD can be generalized as a weighted least-squares problem, which is solved using an efficient bilinear iterative algorithm. Matching multiple views of a polyhedral object can be accomplished by tracking its planar surface patches. It is, however, difficult to match multiple views of a free-form (i.e., smooth) object. This mesh representation provides a one-to-one mapping between a resampled convex/concave mesh to its curvature distribution on a unit sphere. The correspondence between two meshes can then be established by minimizing the difference between two curvature distributions. The same mesh representation also enables one to compare 3D shapes and to synthesize new shapes. In summary, this thesis shows that PCAMD can be used to integrate multiple views to obtain a statistically optimal object model provided that those vies can be resampled and matched under appropriate representations. Based on the integral approach, our "modeling-from-reality" system has been successfully applied for modeling both polyhedral and free-form object.