One of the central themes in the field of computer graphics is the generation of images of artificial environments. These environments may be purely fictional or may be models of actual places but in either case the goal is to convince the viewer that they are looking into a real scene.
The classical approach to this problem is to model the environment using geometric primitives with attributed surface properties and to specify the position and properties of light sources. Images of the scene are then generated by projecting the modelled environment onto an image plane using a camera model and determining the colour of image pixels using the surface properties and the specified lighting. Although this approach has been used to generate relatively convincing scenes it is generally agreed that further realism requires increasing the complexity of the environments.
Unfortunately, increasing the complexity of artificial environments presents two major problems. The first is that constructing geometric models of environments is already difficult and time-consuming and this increases as the amount of detail and complexity is increased. The second problem is that the conventional process of rendering a scene also depends upon the complexity of the environment so rendering progressively more detailed scenes requires progressively more computational power or time. This is particularly a problem for interactive applications where fast rendering times are essential.
In response to these problems researchers are investigating alternative approaches to scene modelling and rendering known as image-based modelling and image-based rendering respectively. In the image-based approach, the world is modelled by a collection of example images and these are used to generate novel images representing the scenes appearance at arbitrary points in the world. This greatly simplifies the modelling of real scenes as only a number of example images need to be acquired. Modelling of synthetic environments is still done in the conventional way initially but the time needed to render the scene is no longer an issue as this is done off-line. A second advantage of the image-based approach is that the complexity of rendering the scene is decoupled from the complexity of the scene.
Over the last few years, image-based rendering has caught the imagination of the computer graphics community and this has been accompanied by a range of different image-based rendering techniques. An overview of the field is presented in this short paper and most of important techniques are categorised in Figure 1.
Figure 1: A Categorisation of image-based rendering techniques