The earliest attempts in reconstruction of 3D models from photos used the silhouettes of objects as sources of shape information. A 2D silhouette is the set of close contours that outline the projection of the object onto the image plane. Segmentation of the silhouettes from the rest of the image and combination with silhouettes taken from different views provide a strong cue for image understanding.
Typically shape from silhouettes techniques start with an acquisition step where images of the object are taken from different locations around it. For each of these images the object silhouette is extracted using simple differencing or blue screen segmentation techniques. The computed silhouettes for every image along with the centre of the corresponding camera is then used to define a volume which if backprojected to 3D space can be assumed to bound the object. The intersection of these volumes associated with the set of acquired images yields a reasonable approximation of the real object. This intersection volume has been named the visual hull by Laurentini et. al. [30] and described as the maximal object that gives the same silhouette with the real object from any possible viewpoint.
Figure 2: Example failure of shape from silhouettes approach. For both (a),(b) the resulted reconstruction will be (c)
A property of the visual hull is that as the number of images used increases, its fit to the actual object volume becomes tighter. However, this number can be proved [31] to be unbound for reconstruction of general polyhedral objects. Even if the acquisition of an infinite number of images was possible, silhouettes can be insufficient clues for fully compute the shape of non convex objects. An example where the silhouette methodology will fail is illustrated in figure 2. The reason for this limitation is that concavities in the object geometry result in self occluded areas for the object that cannot be resolved from any viewpoint unless additional information is provided. Niem et al. [42] and Szeliski [61] have used turntables and calibrated cameras to produce 3D range information for pixels belonging to the object silhouette which subsequently integrated with the volume intersection model to correctly estimate the object's shape.
Unfortunately, the type of the object is not the only parameter affecting its corresponding visual hull form. The positioning of the cameras can significantly influence the computed model especially when the number of acquisition locations is small. An iterative method to specify the viewpoints that optimise the reconstruction taking into account some prior knowledge on the object shape has been presented in [54].
Shape from silhouettes is a particularly good approach if only a crude model of the real world is required. The methodology is intuitive and easy to implement and this is the main reason that systems generating and replaying 3D digital video [39] as well as commercial object modelling packages [42] are based on it. Nevertheless, reconstruction is restricted to small solid objects for which their whole geometry can be captured from photos around them and thus are not applicable to scene modelling.