Approximations of camera projection and related homographic relations

Diane Lingrand
Diane.Lingrand@sophia.inria.fr
INRIA - RobotVis project
B.P. 93 -- 06902 Sophia Antipolis Cédex -- FRANCE

Camera projection

The most commonly accepted hypothesis states that a 3D-point M is projected with a perspective projection onto an image plane on a 2D-point ${\bf m} = [u\ v\ 1]^T$ . Choosing a reference frame attached to the camera, the projection equation is:

$\begin{displaymath} Z\,{\bf m} = \begin{pmatrix}\alpha_u & \gamma & u_0 & 0 \\ 0 & \alpha_v & v_0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix}\, {\bf M} \end{displaymath}$

(1)

where $\alpha_u$ and $\alpha_v$ represent the horizontal and vertical lengths, u₀ and v₀ correspond to the image of the optical center and $\gamma$ is the skew factor.

This model can be refined, by taking optical distortions into account [18,4,7]. In this paper, we will consider that the needed corrections have been done as a preprocessing.

Two approximations have been proposed in the literature :

The para-perspective model :

The perspective projection model may be approximated [2,15,12,13,14] to its first order with respect to the 3D coordinates. This is equivalent to approximate the perspective projection in two steps: (i) a projection parallel to the gaze direction onto an auxiliary plane ${\cal P}_a$ which is parallel to the image plane and passes through the scene center ${\bf M_0} = [X_0\ Y_0\ Z_0]^T$ followed by (ii) a perspective projection onto the image plane. This so called para-perspective model yields linear equations (2).

$\begin{displaymath}{\bf m} = \begin{pmatrix}\alpha_u & \gamma & \beta_u & u_0 \\... ...pha_v & \beta_v & v_0 \\ 0 & 0 & 0 & 1 \end{pmatrix}\, {\bf M} \end{displaymath}$

(2)

$\beta_u$

$\beta_v$

$\begin{displaymath}\begin{array}{lcl} \beta_u &=& \alpha_u\,{X_0}/{Z_0} + \gamma\,{Y_0}/{Z_0}\\ \beta_v &=& \alpha_v\,{Y_0}/{Z_0} \end{array}\end{displaymath}$

(3)

Equation 2 corresponds to the most general case of para-perspective projection although more simple expressions have been proposed [16].

The orthographic model :

The zero-order development with respect to the 3D depth consists in a rougher approximation. It is also equivalent to another two steps approximation: (i) an orthogonal projection onto the auxiliary plane ${\cal P}_a$ followed by (ii) a perspective projection onto the image plane. This approximation, called the orthographic model (4), is well adapted to foveal attention and is characterized by linear equations without any new parameter. It is an approximation of the para-perspective model when the observed objects are in the fovea, i.e. close to the optical axis :

$\begin{displaymath}{\bf m} = \begin{pmatrix}\alpha_u & \gamma & 0 & u_0 \\ 0 & \alpha_v & 0 & v_0 \\ 0 & 0 & 0 & 1 \end{pmatrix}\, {\bf M} \end{displaymath}$

(4)

Those three projection models can be integrated in the following expression :

$\begin{displaymath} \kappa \; {\bf m} = \underbrace{\begin{pmatrix} \alpha_u & ... ...\ 0 & 0 & \mu & (1-\mu) \\ \end{pmatrix}}_{\bf A} \, {\bf M} \end{displaymath}$

(5)

with :

projection case	$\boldsymbol{\lambda}$	$\boldsymbol{\mu}$
perspective projection	1	1
orthographic projection	0	0
para-perspective projection	1	0

Relations between two frames

Let I₁ and I₂ denote two images. In the general case, there exists a fundamental relation between points ${\bf m_2}$ in I₂ and points ${\bf m_1}$ in I₁ : ${\bf m_2}^T\,{\bf F}\,{\bf m_1} = {\bf0}$ where ${\bf F}$ is called the fundamental matrix [9].

However, this relation is not defined in some singular cases. For example, it is well known that, in the perspective projection case, if the displacement is a pure rotation or, if the scene is planar, the relation between points is homographic : ${\bf m_2} = {\bf H}\,{\bf m_1}$ where ${\bf H}$ is called the homographic matrix.

Homographic relation in the para-perspective case

In the para-perspective case, we write the projection and displacement equations by extracting the third column from matrix ${\bf A}$ :

$\begin{displaymath}{\bf m} = \underbrace{\begin{pmatrix}\alpha_u & \gamma & u_0... ...{\bf A})_3} = ({\bf A})_{-3}\,\underline{\bf M}+ Z\,({\bf A})_3\end{displaymath}$

where $({\bf A})_{-3}$ is an invertible square matrix since $\det(({\bf A})_{-3})=\alpha_u\,\alpha_v \neq 0$ .

Thus ${\bf m_1}= ({\bf A_1})_{-3}{\bf\underline{M}_1} + Z_1({\bf A_1})_3 \Rightarrow ... ...1} = (({\bf A_1})_{-3})^{-1}{\bf m_1} - Z_1(({\bf A_1})_{-3})^{-1}({\bf A_1})_3$ .

Remember that ${\bf m_2} = {\bf A_2}\,{\bf M_2}$ and ${\bf M_2} = {\bf [R\vert t]\,M_1}$

Let ${\bf K} = ({\bf A_2\,[R\vert t]})_3 - ({\bf A_2\,[R\vert t]})_{-3}\,(({\bf A_1})_{-3})^{-1}\,({\bf A_1})_3$

and ${\bf H}_{\infty_{para}} = ({\bf A_2\,[R\vert t]})_{-3}\,(({\bf A_1})_{-3})^{-1}$ .

Previous equations lead to : ${\bf m_2} = {\bf H}_{\infty_{para}}\,{\bf m_1} + Z_1\,{\bf K}$

This relation is homographic if and only if ${\bf K}=0$ or if there exists a (3 $\times$ 3) matrix ${\bf H_Z}$ such as $Z_1\,{\bf K} = {\bf H_Z}\,{\bf m_1}$ . The first condition induces a displacement constraint. It leads to the simple equation ${\bf r} = \theta\,{\bf M_0}$ meaning that the rotation axis is parallel to the gaze direction. The second condition induces a geometric relation on the 3D point : Z₁ is an affine function of X₁ and Y₁, meaning that the 3D points must belong to a plane ${\cal P}$ , which cannot contain the optical axis and the gaze direction (see [13] for a demonstration).

Homographic relation in the orthographic case

The orthographic case is a particular case of para-perspective projection for which the gaze direction is the optical axis. Following a demonstration similar to the para-perspective case, we also obtain two constraints; the displacement constraint states that the rotation axis must be parallel to the optical axis, and the geometric constraint states that the 3D-points must belong to the same plane which does not contain the optical axis. All constraints on displacement and scene geometry for homographic relations are summarized in the following table :

projection	displacement constraint	geometric constraint
perspective	${\bf t } = {\bf0}$	plane
para-perspective	${\bf r} \parallel {\bf CM_0}$	plane Z=f(X,Y)
orthographic	${\bf r} \parallel {\bf0z}$	plane Z=f(X,Y)

Bibliography

2: J.Y. Aloimonos.
Perspective approximations.
Image and Vision Computing, 8(3):179-192, August 1990.
4: P. Brand, R. Mohr, and P. Bobet.
Distorsions optiques : correction dans un modèle projectif.
Technical Report 1933, LIFIA-INRIA Rhône-Alpes, 1993.
7: Frédéric Devernay.
Vision stéréoscopique et propriétés différentielles des surfaces.
PhD thesis, École Polytechnique, Palaiseau, France, February 97.
9: O. Faugeras.
Three-Dimensional Computer Vision: a Geometric Viewpoint.
MIT Press, 1993.
11: R. Horaud, S. Christy, and F. Dornaika.
Object pose: The link between weak perspective, para perspective, and full perpective.
Technical Report 2356, INRIA, September 1994.
12: Radu Horaud, Fadi Dornaika, Bart Lamiroy, and Stéphane Christy.
Object pose: The link between weak perspective, paraperspective, and full perspective.
IJCV, 22(2), 1997.
13: D. Lingrand.
Analyse Adaptative du Mouvement dans des Séquences Monoculaires non Calibrées.
PhD thesis, UNSA, INRIA, Sophia Antipolis, France, July 1999.
14: D. Lingrand.
Particular Forms of Homography Matrices.
BMVC, vol. 2, pp 596--605, Bristol (U.K.), September 2000.
15: Conrad J. Poelman and Takeo Kanade.
A paraperspective factorization method for shape and motion recovery.
Technical Report CMU-CS-93-219, Carnegie Mellon University, School of Computer Science, December 1993.
16: Long Quan.
Self-calibration of an affine camera from multiple views.
IJCV, 19(1):93-105, May 1996.
17: O. Rodrigues.
Des lois géométriques qui régissent les déplacements d'un système solide dans l'espace, et de la variation des coordonnées provenant de ces déplacements considérés indépendamment des causes qui peuvent les produire.
Journal de Mathématiques Pures et Appliquées, 5, 1840.
pp. 380-440.
18: C. C. Slama, editor.
Manual of Photogrammetry.
American Society of Photogrammetry, fourth edition, 1980.

Diane Lingrand