We are investigating a new approach for representing 3-D curved objects for recognition and modeling. Our approach starts by fitting a discrete mesh of points to the raw data. The fitting is based on the concept of deformable surfaces: Starting with a spherical shape, a mesh is iteratively deformed, subject to attractive forces from the data points, until it reaches the stable shape which is the best fit to the input set of points.
Once a discrete set of points is fit to the surface, values such as discrete surface curvature can be computed at each of its nodes. Moreover, each node of the mesh can be mapped to a corresponding node of a reference spherical mesh with the same number of points and the same topology as the object mesh. By storing on the spherical mesh the values computed on the surface of the object, we have, in effect, created a spherical image of the object. We call this spherical image the Spherical Attribute Image (SAI).
The SAI representation has an important invariance property that makes it suitable for a number of applications in the area of 3-D object recognition and modeling: Assuming that the mesh fit to the object satisfies certain regularity constraints, the SAIs of two instances of an object which differ by a rigid transformation are identical up to a rotation of the sphere. Consequently, the problem of bringing two 3-D objects into registration is replaced by the much simpler problem of bringing spherical images into correspondence. Moreover, because of the way the mapping between object mesh and SAI is established, SAIs can be used to represent arbitrary non-convex objects and partial views of objects. We are taking advantage of these properties in three main areas which we describe below.
Object Recognition: Given a complete object model represented by its SAI and a partial SAI extracted from a view of a scene, we can compute the best transformation between model and observed object by registering the two spherical images. The registration of the SAIs yields a set of correspondences between nodes of the model mesh and nodes of the observed mesh and a measure of similarity measure between the two SAIs. The correspondences are used for computing the transformation between model and scene; the similarity measure is used for deciding whether the model corresponds to the observed scene. This object recognition algorithm can be applied to general curved objects.
Object Modeling: The general problem of 3-D object modeling is to build a complete object surface model given a number of partial views of the object. This problem is usually solved under the constraint that the transformations between viewing positions are at least approximately known. Using the SAI representation eliminates this constraint. Specifically, after a different SAI is created for every view, the transformation between views is computed using the matching algorithm described above. The data from all the views can then be transformed into a single reference frame and aggregated into a single surface model of the object. This approach has the advantage that no prior knowledge of the transformations is required.
Data Fusion: Although the previous discussion of the SAI representation was based on the idea of attaching curvature at every mesh node, any value computed at a node of the mesh could be stored at that node, for example, the color. In this case, matching two SAIs involves finding the rotation that yields the smallest distance between the spherical images of both color and curvature. This gives an opportunity to use geometric information and appearance information in the same framework.
