Automatic feature extraction and matching are probably the most well studied problems in computer vision research field. Once the 2D projection of a point in the real scene has been found, its position in 3D can be assumed somewhere along the ray connecting the camera optical centre and the corresponding spot in the image plane. Tracking its projections across multiple images and using triangulation [23] allows the relatively accurate localisation of the point in 3D. If extraction and correspondence can be performed for a sufficient number of points and lines and over images acquired from different directions then estimates for both the 3D locations of the features and the camera positions can be deduced. The obtained reconstruction however differs from the true structure by a projective transformation. Knowledge of camera intrinsic parameters reduces this ambiguity to metric. This reconstruction method is called structure from motion (SFM) and is based on the bundle adjustment process [62] of minimising the distances between estimated 3D structure projections and actual image measurements. An extensive literature exists for SFM but covering this is out of the scope of this study. Nevertheless a good review of this methodology can be found in [18].
Earlier attempts like the DROID 3D system and the Vanguard project aim to build a 3D representation of a site by using passive sensors and utilise SFM to obtain 3D estimates of the scene structure. Unfortunately, both focused on the approximation of the scene geometry and fail to address the problem of reconstructing the surfaces that connect the available cloud of 3D features. Solutions like off line fitting of surfaces [4, 20] resulted in simplified and unconnected models while attempts to build a continuous triangulated model by a single view planar triangulation method [22] are only valid in the absence of occlusions.