Discussion

The major problem with the surface reconstruction constraints is unsolvable - one cannot reconstruct the invisible when the image is taken from a single viewpoint. Some examples are seen in Figure 4.11. In the first case, the extended segments never intersect, and in the second, extension creates a larger, incorrect surface. Stereo or observer movement might help reconstruct the surface, however. Feature regularity (such as symmetry) might also be exploited to aid reconstruction. As the goal was to reconstruct enough to allow the rest of recognition to proceed, a few failures should be acceptable.

Figure 4.11: Unsuccessful Extensions

Another obvious criticism is over performance when applied to rich natural images. Realistic images are likely to have missing or erroneous data, such as for line labels or surface orientation, which is likely to degrade both the quality and rate of performance. In short, the processes described in this chapter seem fine for clear laboratory images, but it is hard to predict their performance on natural scenes.

Because segmentation criteria may break up or merge surfaces at different scales, surface hypotheses need to allow for alternative representations derived as a function of a locally relevant scale. These representations are related but are not interchangeable, nor are they equivalent.

Scale also affects surface reconstruction, as can be seen in Figure 4.12. The first figure shows the extension based on individual teeth of the gear, whereas at the larger scale, the extension is based on the smoothed convex envelope of the gears.

Figure 4.12: Scale Based Extension Problems

To avoid the problem of redundant hypothesis formation, only the reconstructed surface is kept. An alternative solution might isolate the description of the surface, which is common to all hypotheses, from that of the boundary of the surface.

Final Comments

The test image shows that we can expect substantial occlusion in three dimensional scenes. This chapter demonstrates that much of the lost information can be inferred from the physical and modeling constraints, without generating many errors or redundant hypotheses.