The reconstruction problem

Consider a set of three-dimensional points viewed by N cameras with matrices $\{ \tilde{\bf P}^i \} _{i=1 \ldots N}$. Let $\tilde{\bf m}_j^i \simeq\tilde{\bf P}^i \tilde{\bf w}_j$ be the (homogeneous) coordinates of the projection of the j-th point onto the i-th camera. The reconstruction problem can be cast in the following way: given the set of pixel coordinates $\{\tilde{\bf m}_j^i \}$, find the set of camera matrices $\{\tilde{\bf P}^i \}$ and the scene structure $\{\tilde {\bf w}_j \}$ such that

$$\tilde{\bf m}_j^i \simeq\tilde{\bf P}^i \tilde{\bf w}_j .$$ (19)

Without further restrictions we will, in general, obtain a projective reconstruction [34,43,5] defined up to an arbitrary projective transformation. Indeed, if $\{\tilde{\bf P}^i \}$ and $\{\tilde {\bf w}_j \}$ satisfy (19), also $\{\tilde{\bf P}^i \tilde{\bf T} \}$ and $\{ \tilde{\bf T}^{-1}\tilde{\bf w}_j \}$ satisfy (19) for any $4 \times 4$ nonsingular matrix $\tilde{\bf T}$.

A projective reconstruction can be computed starting from points correspondences only, without any a-priori knowledge [17,18,45,44,19,3,2,48]. Despite it conveys some useful in formations [40,39], we would like to obtain a Euclidean reconstruction, a very special one that differs from the true reconstruction by an unknown similarity transformation. This is composed by a rigid displacement (due to the arbitrary choice of the world reference frame) plus a a uniform change of scale (due to the well-known depth-speed ambiguity: it is impossible to determine whether a given image motion is caused by a nearby object with slow relative motion or a distant object with fast relative motion).

Maybank and Faugeras [31,10] proved that, if intrinsic parameters are constant, Euclidean reconstruction is achievable. The procedure is known as autocalibration.

In this approach, the internal unchanging parameters of the camera are computed from at least three views. Once the intrinsic parameters are known, the problem of computing the extrinsic parameters (motion) from point correspondences is the well-known relative orientation problem, for which a variety of methods have been developed [24,13,23]. In principle, from the set of correspondences $\{\tilde{\bf m}_i\}$ one can compute the fundamental matrix, from which the essential matrix is immediately obtained with (13). Motion parameters ${\bf R}$ and the direction of translation ${\bf t}$ are obtained directly from the factorization (12) of ${\bf E}$. In [28] direct and iterative methods are compared.

Recently, new approaches based on the idea of stratification [30,7] have been introduced. Starting from a projective reconstruction, which can be computed from the set of correspondences $\{\tilde{\bf m}_j^i \}$ only, the problem is computing the proper $\tilde{\bf T}$ that upgrades it to a Euclidean reconstruction, by exploiting all the available constraints. To this purpose the problem is stratified into different representations: depending on the amount of information and the constraints available, it can be analyzed at a projective, affine¹, or Euclidean level.