When buildings and other objects are scanned, the information needed to construct a 3D model is almost never complete. The main cause of this is occlusion: some object is blocking the view of the sensor on another object. In the left hand side of the figure we can see a range image taken in the Bornholm Church in Denmark. It is a chair in front of a wall. When we convert this image into 3D and take another perspective (centre image) we can clearly see that part of the data is missing: in the range image, part of the wall is occluded by the chair. This is clearly a problem, because a 3D model (VRML) of this image would have a chair-shaped hole in the wall, making it visually unacceptable.
We have developed and implemented a method that addresses this problem. The main idea is to use information about the surroundings of the occlusions to locate and reconstruct the missing data. In the centre image for instance, we can clearly see that the two unconnected parts of the wall lie in the same plane. This is one of the cues we use. The result of reconstructing the missing data can be seen on the right.
F. Stulp, F. Dell'Acqua, R. B. Fisher, "Reconstruction of surfaces behind occlusions in range images", Proc. 3rd Int. Conf. on 3-D Digital Imaging and Modeling (3DIM01), Montreal, Canada, June 2001.