Object-Centered Surface Reconstruction: Combining Multi-Image Stereo and Shading

Our goal is to reconstruct both the shape and reflectance properties of surfaces from multiple images. To this end, we introduce an object-centered representation, specifically, a hexagonally-connected 3-D mesh of vertices with triangular facets. Such a representation accommodates the two classes of information mentioned above, as well as multiple images, self-occlusions and geometric constraints. We have chosen to model the surface material using the Lambertian reflectance model with variable albedo. Consequently, the natural choice for the monocular information source is shading, while intensity is the natural choice for the image feature used in multi-image correspondence. Not only are these the natural choices given a Lambertian reflectance model, they are complementary: intensity correlation is most accurate wherever the input images are highly textured, and shading is most accurate when the input images are not.

We argue that by combining shape from shading and stereo, we can do better than by using either of them alone while avoiding the limitations imposed by the constant depth assumption of traditional correlation-based stereo algorithms and the constant albedo assumption of conventional shape from shading methods. Furthermore, our technique can merge information from any number of very different viewpoints thereby eliminating blind spots while dealing effectively with self-occlusions.