Many applications in computational vision require that a large field of view is imaged. Examples include surveillance, teleconferencing, and model acquisition for virtual reality. A number of other applications, such as ego-motion estimation and tracking, would also benefit from enhanced fields of view. Unfortunately, conventional imaging systems are severely limited in their fields of view. Both researchers and practitioners have therefore had to resort to using either multiple or rotating cameras in order to image the entire scene.
One effective way to enhance the field of view is to use mirrors in conjunction with lenses. See, for example, [Rees, 1970], [Charles et al., 1987], [Nayar, 1988], [Yagi and Kawato, 1990], [Hong, 1991], [Goshtasby and Gruver, 1993], [Yamazawa et al., 1993], [Bogner, 1995], [Nalwa, 1996], [Nayar, 1997a], and [Baker and Nayar, 1998]. We refer to the general approach of using mirrors in combination with conventional imaging systems as catadioptric image formation. Dioptrics is the science of refracting elements (lenses) whereas catoptrics is the science of reflecting surfaces (mirrors). The combination of refracting and reflecting elements is therefore referred to as catadioptrics [Hecht and Zajac, 1974].
As noted in [Rees, 1970], [Yamazawa et al., 1995], [Nalwa, 1996], and [Nayar and Baker, 1997], it is highly desirable that a catadioptric system (or, in fact, any imaging system) have a single viewpoint (center of projection). The reason a single viewpoint is so desirable is that it permits the generation of geometrically correct perspective images from the image(s) captured by the catadioptric cameras. This is possible because, under the single viewpoint constraint, every pixel in the sensed image(s) measures the irradiance of the light passing through the viewpoint in one particular direction. Since we know the geometry of the catadioptric system, we can precompute this direction for each pixel. Therefore, we can map the irradiance value measured by each pixel onto a plane at any distance from the viewpoint to form a planar perspective image. These perspective images can subsequently be processed using the vast array of techniques developed in the field of computational vision which assume perspective projection. Moreover, if the image is to be presented to a human, as in [Peri and Nayar, 1997], it needs to be a perspective image in order to not appear distorted. Naturally, when the catadioptric imaging system is omnidirectional in its field of view, a single viewpoint permits the construction of panoramic images in addition to perspective ones.
We begin this tutorial in Section 2 by deriving the entire class of catadioptric systems with a single effective viewpoint and which are constructed just using a single conventional lens and a single mirror. As we will show, the 2-parameter family of mirrors which can be used is exactly the class of rotated (swept) conic sections. Within this class of solutions, several swept conics prove to be degenerate solutions and hence impractical, while others lead to practical sensors. During our analysis we will stop at many points to evaluate the merits of the solutions, as well as the merits of closely related catadioptric sensors proposed in the literature.
An important property of a sensor that images a large field of view is its resolution. The resolution of a catadioptric sensor is not, in general, the same as that of the sensors used to construct it. In Section 3 we study why this is the case, and derive an expression for the relationship between the resolution of a conventional imaging system and the resolution of a derived catadioptric sensor. This expression should be carefully considered when constructing a catadioptric imaging system in order to ensure that the final sensor has sufficient resolution. It could also be used to design conventional sensors with non-uniform resolution, which when used in an appropriate catadioptric system have a specified (e.g. uniform) resolution.
Another optical property which is modified by the use of a catadioptric system is focusing. It is well known that a curved mirror increases image blur [Hecht and Zajac, 1974]. In Section 4 we analyze this effect for catadioptric sensors. Two factors combine to cause blur in catadioptric systems: (1) the finite size of the lens aperture, and (2) the curvature of the mirror. We first analyze how the interaction of these two factors causes defocus blur and then present numerical results for one specific mirror shape: the hyperboloid. The results show that the focal setting of a catadioptric sensor using a curved mirror may be substantially different from that needed in a conventional sensor. Moreover, even for a scene of constant depth, substantially different focal settings may be needed for different points in the scene.