Metric, affine and projective point recovery summary

Bob Fisher

Assume a set of 3D points $\{ \vec{X}_i \}$ and the corresponding projected 2D image points $\{ \vec{x}_i \}$ and $\{ \vec{x}_i' \}$. Let the projection matrices be P and P', such that $\vec{x}_i = {\bf\rm P}\vec{X}_i$ and $\vec{x}_i' = {\bf\rm P}'\vec{X}_i$.

Given the observed set of corresponding 2D image points $\{ \vec{x}_i \}$ and $\{ \vec{x}_i' \}$, the goal of the stereo reconstruction process is to recover the set of 3D points $\{ \vec{X}_i \}$, and sometimes also the projection matrices P and P'.

If only the projected points $\{ \vec{x}_i \}$ and $\{ \vec{x}_i' \}$ are known, the fundamental matrix F can be computed [1], from which a projective reconstruction can be computed [2]. If the plane at infinity can be estimated, then an affine reconstruction is possible, whereby angles are correct and parallelism holds, but metric scale is unknown [3]. If the absolute conic can be computed [4], then a metric reconstruction is possible.

1. R. I. Hartley, "Estimation of relative camera positions for uncalibrated cameras", Proc. Eur. Conf. on Computer Vision, Vol 588, Springer-Verlag, pp 579-587, 1992.

2. R. I. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge Univ Press, 2004.

3. J. J. Koenderink, A. J. van Doorn, "Affine structure from motion", J. of Optical Society of America, 8(2), pp 377-385, 1991.

4. R. I. Hartley, "Euclidean Reconstruction from uncalibrated views", In Mundy, Zisserman, Forsyth (Eds), Applications of Invariance in Computer Vision, LNCS Vol 825, Springer-Verlag, pp 237-256, 1994.