Surface clusters need not be perfect, as the goal of the process is to produce a partitioning without a loss of information. They may be incomplete, as when an object is split up by a closer obscuring object, though the surface hypothesizing may bridge the occlusion. They may also be over-aggregated - from images where there is insufficient evidence to segregate two objects. These failures may reduce recognition performance (i.e. speed), but not its competence (i.e. success): incompletely merged surface clusters will be merged in a larger context and insufficiently split surface clusters will just cause more searching during hypothesis construction.
The process of forming primitive surface clusters is particularly interesting because it is a local process (i.e. connectivity is determined by classifications made in the neighborhoods of vertices and single boundary segments) that produces a global organization of the image. This also holds for the formation of the equivalent depth surface clusters, but not the depth merged clusters. Locality is important because (1) it is often required for efficient parallelization and (2) it is often the hallmark of a robust process.
A criticism concerns the combinatorial formation of the depth merged surface clusters. The general goal is to create hypotheses that correspond to the complete visible surface of an object and nothing more, necessitating merging surface clusters. Unfortunately, in the absence of context or object knowledge, there is no information yet to determine whether a surface cluster is related to the surfaces behind. As an object may be associated with any two consecutive surfaces, it is likely that the merging process needs to be based on either merging all surface clusters linked to the current one, or all possible combinations of consecutive depths. As each surface may be in front of more than one other surface, the latter alternative most likely leads to a combinatorial explosion, whereas the former leads to enlarged surface clusters. The combinatorial process, however, probably has better future potential, provided some further merging constraints can be elucidated and implemented. The use of equivalent depth clusters helped control the problem, and as seen in Table 5.1, most of the surface clusters correspond to object features, an object was never split between surface clusters, and usually only one new model was appropriate for each new level of surface cluster.
A general criticism about the surface cluster is that,
as formulated here, it is too literal.
A more suggestive process is needed for dealing with
natural scenes, where segmentation, adjacency and depth ordering
are more ambiguous.
The process should be supported by surface evidence, but should
be capable of inductive generalization - as is needed to see a
complete surface as covering the bough of a tree.
The complexity of natural scenes is also likely to lead to combinatorial
problems, because of the many objects overlapping each other in dense scenes.
Final Comments
Surface clusters provide a good intermediate visual representation between the segmented surface patches and the object hypotheses. In the chapters that follow, we will see how they are used to accumulate evidence to select models and reduce the complexity of model matching.