The first step in the interpretation of surface information is the formation of surface hypotheses, which groups surface image regions to form complete surfaces of the unidentified objects. This is the first important transformation from image-based data to object-based data. Further, it reduces data complexity by representing one or more surface regions by a single symbolic entity.
If segmentation has been successful, then the surface image regions should closely correspond to the visible portions of the object's surface patches. Three problems may cause the patches to not correspond closely:
Because of the pervasiveness of occlusion in natural scenes, this reconstruction is necessary. Reconstructed surfaces can only be hypothetical but, by the surface segmentation assumptions (Chapter 3), there are no extreme surface or boundary shape variations in a single segment. As many natural object boundaries exhibit surface smoothness over moderate distances (at appropriate scales), reconstruction should be possible. This is even more likely with most man-made objects.
Some research has tried to overcome occlusion directly by using visible
Guzman  used paired
TEE junctions to signal the presence
of occlusion and locate which two dimensional image regions should be connected.
Adler  also used TEE information to infer depth ordering between scene
The key problem is detection of occlusion, and their work relied
on the use of TEE detections, which show where one surface
boundary is abruptly terminated by the obscuring boundary of a closer
Because a fragmented obscured surface must have a pair of TEEs at the start and
end of the obscuring boundary (under normal circumstances),
the detection of a matched
pair of TEEs suggests a likely occlusion boundary, and hence
where the invisible portion of the surface lies.
In the research described here,
occlusion boundaries are directly labeled, so the
occlusion cueing process is no longer necessary.
The TEEs are still useful for signaling where along the occlusion boundary
the obscured surfaces' boundaries terminate.
They would also be useful for helping cope with missing, incorrect
or ambiguous data (e.g. when a correct boundary label is not available).