Making Complete Surface Hypotheses

The first step in the interpretation of surface information is the formation of surface hypotheses, which groups surface image regions to form complete surfaces of the unidentified objects. This is the first important transformation from image-based data to object-based data. Further, it reduces data complexity by representing one or more surface regions by a single symbolic entity.

If segmentation has been successful, then the surface image regions should closely correspond to the visible portions of the object's surface patches. Three problems may cause the patches to not correspond closely:

1. A data patch may correspond to several object surface patches, because of segmentation failure.
2. The object surface patch may be fragmented into several data patches, because of segmentation failure.
3. The object surface patch may be fragmented or smaller because of occlusion.

The first two problems are a concern for segmentation and are not considered closely here, though some corrective action is possible once a model has been selected (e.g. a predicted model SURFACE can be used to guide splitting or merging of patches). The third problem is the main concern of this chapter, which shows that the most common cases of occlusion can be overcome with the surfaces largely reconstructed.

Because of the pervasiveness of occlusion in natural scenes, this reconstruction is necessary. Reconstructed surfaces can only be hypothetical but, by the surface segmentation assumptions (Chapter 3), there are no extreme surface or boundary shape variations in a single segment. As many natural object boundaries exhibit surface smoothness over moderate distances (at appropriate scales), reconstruction should be possible. This is even more likely with most man-made objects.

Some research has tried to overcome occlusion directly by using visible cues. Guzman [81] used paired TEE junctions to signal the presence of occlusion and locate which two dimensional image regions should be connected. Adler [3] also used TEE information to infer depth ordering between scene features. The key problem is detection of occlusion, and their work relied on the use of TEE detections, which show where one surface boundary is abruptly terminated by the obscuring boundary of a closer surface. Because a fragmented obscured surface must have a pair of TEEs at the start and end of the obscuring boundary (under normal circumstances), the detection of a matched pair of TEEs suggests a likely occlusion boundary, and hence where the invisible portion of the surface lies. In the research described here, occlusion boundaries are directly labeled, so the occlusion cueing process is no longer necessary. The TEEs are still useful for signaling where along the occlusion boundary the obscured surfaces' boundaries terminate. They would also be useful for helping cope with missing, incorrect or ambiguous data (e.g. when a correct boundary label is not available).