Estimation of the epipolar curves

Our goal is to recover the depth $\lambda$ and the curvature k_t at P. It has been shown that these values are related to the position and the curvature of the epipolar curves. Therefore, the problem is to estimate two of these curves given three tangents. The previous section showed that the epipolar curves are, up to order 2, parabolas of the epipolar planes. Moreover, although the epipolar curves are not in the same plane, by using proposition 2, linear estimations of both $\lambda$ and k_t can be derived.

Now consider the epipolar plane ${\cal E}_1$. We denote by $\beta_{{\cal E}_1}$the angle between the normal N at P and its projection $N_{{\cal E}_1}$ in ${\cal E}_1$:

$$\cos\beta_{{\cal E}_1} = N.N_{{\cal E}_1}.$$

By proposition 2, the epipolar curve with tangents T and T₁ is defined by (up to order two):

$$z_{{\cal E}_1} = \frac{1}{2}k_{{\cal E}_1} x^2, \hspace{3cm} k_{{\cal E}_1}= \frac{k_t}{\cos \beta_{{\cal E}_1}},$$

where $(x,z_{{\cal E}_1})$ form an orthogonal basis of ${\cal E}_1$. Let x_P1 be the abscissa of P₁ as shown in figure 8.

  

Figure 8: The epipolar plane ${\cal E}_1$.

Since P₁ belongs to the epipolar curve, the tangent to ${\cal S}$ at P₁ in the direction T₁ goes through the point $(\frac{x_{P_1}}{2},0)$ and it follows that:

$$\left\{ \begin{array} {ll} T_1.N_{{\cal E}_1} = \frac{k_{{\c... ...\ sign(x_{P_1}) = - sign(T_1.N_{{\cal E}_1}),\end{array}\right.$$

thus:

$$x_{P_1}= \frac{1}{k_{{\cal E}_1}}\;\frac{-T_1.N_{{\cal E}_1}}{\sqrt{1 - (T_1.N_{{\cal E}_1})^2}}.$$

Since $k_t= k_{{\cal E}_1}\; \cos\beta_{{\cal E}_1}$, the above expression can be rewritten as:  

$$x_{P_1}= \frac{\cos\beta_{{\cal E}_1}}{k_t}\;\frac{-T_1.N_{{\cal E}_1}}{\sqrt{1 - (T_1.N_{{\cal E}_1})^2}}.$$ (7)

T₁, $N_{{\cal E}_1}$ and $\cos\beta_{{\cal E}_1}$ can be computed from image measurements. Thus, we can determine the distance $\frac{x_{P_1}}{2}$ between the intersection of the tangents and P given the curvature k_t. In addition, it was shown (section 2.2) that the distance between the camera centre C and the intersection of the tangents can also be computed from image measurements. Hence, if this distance is denoted by d₁, we have the following relation:  

$$\lambda = d_1 - \frac{1}{2}x_{P_1},$$ (8)

which allows the depth at P to be computed given the normal curvature along the viewing direction.

Likewise, since T is also tangent to the second epipolar curve in the epipolar plane ${\cal E}_{-1}$, the depth at P can also be computed in the plane ${\cal E}_{-1}$. Hence, by (8) and (9):

$$\left\{ \begin{array} {ll} x_{P_{-1}}= \frac{\cos\beta_{{\ca... ...,\ \lambda = d_{-1} - \frac{1}{2}x_{P_{-1}}.\end{array}\right.$$

Consequently, we obtain the following system in two unknowns k_t and $\lambda$: 

$$\left\{ \begin{array} {ll} \lambda = d_{-1} +\frac{\cos\beta... ...al E}_1}}{\sqrt{1 - (T_1.N_{{\cal E}_1})^2}}.\end{array}\right.$$ (9)

Note a crucial property of the above system, i.e., its linearity in $(\frac{1}{k_t},\lambda)$.

Remark The connection with the depth formula (3) as written in 2.2 becomes clear if we write (see figure 8):

$$\begin{array} {ll} r_P - r_{P_1} = -\frac{1}{2}x_{P_1}T - \s... ...{{\cal E}_1}\;x_{P_1}^2/2)^2 + (x_{P_1}/2)^2}\; T_1,\end{array}$$

thus:

$$\Delta r.N_1= -\frac{1}{2}x_{P_1}\;T.N_1,$$

and substituting in (3) with P₁=P and P₂=P₁ gives:

$$\lambda = \frac{-\Delta C.N_1}{\Delta T.N_1} - \frac{1}{2}x_{P_1}.$$

This shows that the second term of (3) is a function of the curvature k_t of ${\cal S}$ at P.

The special case $\cos\beta_{{\cal E}_{-1}}= 0$ or equivalently $\cos\beta_{{\cal E}_1}= 0$ occurs when the normal N at P is orthogonal to the epipolar plane. This implies that P is a multiple point and thus $\Delta T.N_{\pm1}=0$. However, if the camera motion is not along the line of sight at P ($\Delta T \neq 0$), the depth at such points can still be computed since in this case two or more different image projections of P are available. This points out that d_-1 and d₁ should be computed using a robust formula instead of $d_{\pm1}=\frac{-\Delta C.N_{\pm1}}{\Delta T.N_{\pm1} }$ which is not defined at a multiple point (i.e. $\Delta T.N_{\pm1}=0$). See the appendix for details on how we compute $d_{\pm 1}$.

Finally, by solving (10) and assuming that camera motions are not along any line of sight, we obtain the following solutions:  

$$\left\{ \begin{array} {ll} \left. \begin{array} {l} \lambda... ..._{-1}=d_1, & \mbox{if $(a_{-1},a_1)= (0,0)$},\end{array}\right.$$ (10)

where:

$$\left\{ \begin{array} {ll} a_{-1}= \cos\beta_{{\cal E}_{-1}}... ...dge T_1)}{(T-T_1).((T\wedge T_1)\wedge T_1)}.\end{array}\right.$$

A geometrical interpretation of d₁ and d_-1 is that they represent the distances from the camera centre position C to the viewing line intersections (see 2.2). To give an interpretation of the terms a₁ and a_-1, consider the projection of P₁ and P_-1 onto the viewing line at P. Intuitively, a₁ and a_-1) can be seen as the positions of these projected points with respect to P. Hence, if P is a double point, then either a₁ or a_-1 is null and if P is a triple point, then both a₁ and a_-1 are null.

Remark The above solutions are not defined if a_-1 = a₁ and $(a_{-1},a_1)\neq (0,0)$. This corresponds to situations where the projections of P_-1 and P₁ onto the viewing line at P are the same. Thus, the contributions of the viewing directions T_-1 and T₁ in (10) are equal and the system has an infinity of solutions. However, unique solutions for depth and the normal curvature k_t at such points P can still be found. The idea is to first compute the depth at one of the epipolar correspondents P₁ or P_-1. This can be done by applying (10) at p₁ (or equivalently p_-1). To this aim, two epipolar correspondents to p₁ (or equivalently p_-1) must be found. We already have p and we can also use p_-1 (or equivalently p₁) since p_-1 and p₁ are epipolar correspondents in that particular case. Once the depth at P₁ has been computed, we can determine the depth at P by the fact that they both belong to the same parabola. Then, the equation given by applying (10) at P gives the normal curvature k_t.$\triangle$

We conclude that the position of a rim point and the normal curvature k_t in the viewing direction can be estimated at any regular point and double point, except if the camera motion is in the viewing direction. For points where more than two rims intersect (i.e., (a_-1,a₁) = (0,0)) and with the same exception, depth can be estimated but not the normal curvature. And if the camera motion is along one of the viewing directions T_-1 or T₁, neither curvatures nor the depth at P can be computed.