Introduction

Despite long-term interest in shape-from-shading (SFS), and psychophysical evidence that it is a key process in 3D surface perception [1], there are few reports of its use in practice. One reason for this is the lack of robust algorithms capable of recovering fine surface detail. However, the original motivation for SFS research was not simply as a goal in itself, but as a useful source of input for what later became the Marr-ian model of a general vision system combining evidence from multiple Shape-from-X modules.

 

That SFS is a useful source of information in the human vision system may be intuitively demonstrated by considering the following. If a person is shown a photograph of a circle, and a photograph of a shaded sphere, the difference between the two objects is very clear, even in the absence of stereo or other cues. Koenderink and van Doorn demonstrate the use of SFS by humans in more elegant and rigorous fashion in a series of papers. Since there is clearly useful information present in shading patterns, it makes sense to attempt to use it.

With this motivation in mind, we attempt to use SFS for one of the major tasks which the human vision system undertakes, that of object recognition. Model-based recognition has long been the dominant paradigm for research, and in theory SFS fits best within this framework. Accurate recovery of surface shape using SFS, and similar Shape-from-X modules, holds out the prospect of automatic model generation from a set of representative images of an object. However, in practice, the poor performance of SFS algorithms has precluded its use in this fashion.

 

The emergence of the appearance-based object recognition paradigm from psychophysical experiments, such as those by Logothetis et al offers the opportunity to recast the role of SFS in recognition.

Much of the literature has focused on appearance-based object recognition using either iconic [3] or grey-scale manifolds [4]. This is a disappointing omission, since SFS can provide direct information concerning surface topography, for example natural characteristic, or typical, views [5,6] and aspect graphs [7,8].

View-based representations have recently been demonstrated to provide a powerful means of recognising 3D objects [9,10,11,12,13]. In essence the technique relies on constructing a distributed 3D representation which consists of a series of characteristic or typical 2D views. For instance, Seibert and Waxman [9] have a Hough-like method in which different views form distinct clusters in accumulator space. Gigus and Malik [10] present a method for computing the aspect graphs of polyhedra in line-drawings using visual events for faces, edges and vertices. Kriegman [11] uses the algebraic structure of occluding contours, whilst Petitjean [12] has developed these ideas to extract visual event surfaces for piecewise smooth objects. Several authors have considered the statistical distribution of characteristic views. For instance Malik and Whangbo [14] have shown that it is inappropriate to distribute the nodes of the aspect graph uniformly across the view-sphere. In a similar vein, Weinshall and Werman have characterised both the likelihood and stability of different characteristic views [13]. These ideas have been applied to the recognition of objects from large model-bases [15]. Meanwhile, Dorai and Jain have recently shown how histograms of surface curvature attributes can be used to recognise different views of curved objects in range images [16].

In practice, view-based object recognition is most easily realized if the different views are organised using either a geometric or relational structure. An example of the former is the view-sphere, while the latter is typified by the aspect graph. Although offering a convenient view-based object representations, both the view-sphere and the aspect graph have proved to be notoriously difficult to elicit from real-world imagery.

Our aim here is to consider how SFS can be used to generate a view-based representation of object appearance, and how this can in turn be used for 3-D object recognition using 2-D views. The starting point for our study is a recent series of papers [17,18] in which we have reported an improved shape-from-shading algorithm using robust-regularizers. The main advantage of this method is to limit the over-smoothing of fine curvature detail. The main contribution is to investigate whether needle-maps can be used for 3D object recognition. We develop two alternative, histogram-based recognition strategies, the first using the surface normals directly, and the second based upon the shape index of Koenderink and van Doorn [19].

The recognition strategies are evaluated on the Columbia University data-base of 20 arbitrarily-selected, real-world objects. Here we show that both representations provide useful recognition performance. However, the surface-normal histogram is found to be more effective than the shape-index histogram. A sensitivity study reveals that the method offers significant discrimination to the differential topology of object appearance on the view sphere. In other words, our needle-maps provide a viable computational basis for automatically extracting characteristic views from 2D images of 3D objects.