Inferences about the 3D structure can be made from a single uncalibrated camera if vanishing points and lines can be computed. Assuming a simple camera model with zero skew and aspect ratio of one and using three orthogonal pairs of parallel lines for recovering three of the remaining internal camera parameters a weak calibration can be performed [7]. Once calibration is achieved a single vanishing line allow complete rectification of the corresponding plane and determination of its orientation relative to the camera. Using this methodology however only 3D directions for lines and planes can be estimated. Nevertheless automatic identification of perpendicular sets of lines is very difficult as projective transformations do not preserve angles.
Several interactive methods that circumvent these limitation by using information provided by the system user have been presented. Liebowitz et. al. [34] initially choose a reference plane and compute its vanishing line and a vanishing point for its orthogonal direction using sets of manually supplied parallel lines. By specifying the relative distance of a single point from the reference plane the orthogonal distance of every point in the image to that plane can then be computed. Metric rectification of the reference plane allows absolute 3D positioning of points in space and feature grouping yields a simple planar scene approximation. Similar results has been presented in [60]. However, in that approach reconstruction has been formulated as a minimisation process over the sum of squared distances for pairs of planes and points or lines. Originally this technique has been proposed by Shum et al. [56] and applied on panoramic images. They estimated camera rotation and plane normals and recovered plane distances, 3D structure and camera translation by solving a constrained least squares problem. Their system also extended to fuse multiple panoramic views using bundle adjustment.