Discussion and Conclusions

This book has described an Artificial Intelligence approach to the problem of three dimensional object recognition, based on methods that lead to general purpose vision systems rather than to limited single application systems. While efficiency is ultimately important, competence must come first. Only a few researchers have used $2\frac{1}{2}{\rm D}$ sketch-like surface data, and the work described here has attempted to explore the whole path from surfaces to objects. The structure of the approach mirrors classical edge-based recognition processes, but the use of surface data required new definitions of the processes and their interconnections.

Some of the interesting aspects of the individual recognition processes were:

1. object modeling
   • a surface modeling method based on distinct curvature class patches.
   • criteria for how to group model SURFACEs into ASSEMBLYs.
2. surface data
   • proposed criteria for segmentation of surface image data into surface patches useful for object recognition.
3. surface hypothesizing
   • analysis of surface occlusion cases to show what cases occur, how to detect them and how to hypothetically reconstruct the missing data. Because the research used three dimensional surface image data, the reconstruction is more robust than that based on only two dimensional image data.
4. surface cluster formation
   • the use of the surface cluster as an intermediate representation between the surface image and the object hypotheses.
   • rules for aggregating the surface patches into the surface clusters corresponding to distinct objects.

5. description
   • a collection of data description modules that exploited the three dimensional character of the raw data.
6. model invocation
   • a network formulation that incorporated both image property evidence and relationship evidence from class and structural associations. The formulation was incremental, used operations that were based on general reasoning rather than strictly visual requirements and supported a low-level, object independent generic vocabulary.
7. hypothesis construction
   • new methods for estimating the three dimensional placement of objects from data associated with surface patches and the intersurface relationships specified by the object model.
   • methods for predicting and verifying the visibility of SURFACEs, including back-facing, tangential and partially or fully self-obscured front-facing structure.
   • rules for explaining missing structure as instances of occlusion by external, unrelated structure.
   • methods for joining non-rigidly connected structures and simultaneously estimating the connection degrees-of-freedom.
   • methods for completely instantiating hypotheses for both solid and laminar structures.
8. verification
   • criteria for verifying the physical existence of a hypothesis.
   • criteria for verifying the identity of an object based on surface evidence.

The research also emphasized the strong distinction between, but equally strong dependence on, the suggestive "seeing" of model invocation and the model-directed hypothesis construction and verification. Finally, the effect of occlusion was considered throughout the visual process, and methods were developed that helped overcome data loss at each stage.

When applying the recognition processes described in this book to the (hand) segmented range image shown in Figure 3.10, the IMAGINE I system correctly recognized all modeled objects in the test scene (the robot and trash can, and their subcomponents). No misidentifications of solids occurred, though several individual surfaces were misidentified. Since the model base included 16 ASSEMBLYs, all of about the same size, including several objects not in this scene (a chair and its subcomponents), this was a good result. The original geometric reasoning module gave decent, but not ideal position and joint angle estimation (seen in Figure 9.16) and a new network-based geometric reasoning method improved on this (see Figure 1.11).

Three test scenes were analyzed. The one used in this book was the most interesting, because it contained articulated objects with some curved surfaces, laminar surfaced objects, partially constrained degrees-of-freedom (joints) and considerable self-occlusion.

The success of the recognition process was largely because of the richness of the $2\frac{1}{2}{\rm D}$ sketch data, but the use of the surface-based hierarchical models, shape segmented surface patches and hierarchical surface cluster data representation made the results easier to obtain.

This recognition process was clearly successful on the test image. However, much research is still needed, and the following section tries to make explicit some of the problems that remain. I welcome others to add to the list and to solve them all.