next up previous
Next: References Up: Computer Vision IT412 Previous: Structure from motion

Optical flow techniques

Optical flow techniques are based on the idea that for most points in the image, neighbouring points have approximately the same brightness. In other words, the world is made up of continuous objects over which brightness varies smoothly.

Let I(x,y,t) be the irradiance at time t at the image point (x,y). If a little patch of uniform brightness around (x,y) does not change with time, then $\frac{\partial I}{\partial t} = 0$, implying

Ix dx/dt + Iy dy/dt + It = 0.

Now let u = dx/dt and v = dy/dt. Using the values of u and v at a grid point (i,j) and its neighbours, we can measure a departure from smoothness by

\begin{displaymath}
S_{ij} = \frac{1}{4}\{(u_{i+1,j} - u_{i,j})^2 + (u_{i,j+1} -...
 ...,j})^2 + (v_{i+1,j} - v_{i,j})^2 + (v_{i,j+1} - v_{i,j})^2 \}, \end{displaymath}

while any error in the optical flow constraint equation is given by

Cij = (Ixui,j + Iyvi,j + It)2.

So we want a set of values ui,j and vi,j that minimize

\begin{displaymath}
E = \sum_i \sum_j S_{ij} + \lambda C_{ij}, \end{displaymath}

where $\lambda$ is a regularisation constant.

We differentiate E with respect to ukl and vkl to get

\begin{displaymath}
\frac{\partial E}{\partial u_{kl}} = 2(u_{kl} - \overline{u_{kl}}) + 2 \lambda (I_xu_{kl} + I_y v_{kl} + I_t)I_x, \end{displaymath}

and

\begin{displaymath}
\frac{\partial E}{\partial v_{kl}} = 2(v_{kl} - \overline{v_{kl}}) + 2 \lambda (I_xu_{kl} + I_y v_{kl} + I_t)I_y, \end{displaymath}

where $\overline{u}$ and $\overline{v}$ are local averages of u and v.

We equate these expressions to zero to get

\begin{displaymath}
(1 + \lambda I_x^2)u_{kl} + \lambda I_xI_yv_{kl} = \overline{u_{kl}} - \lambda I_xI_t \end{displaymath}

and

\begin{displaymath}
(1 + \lambda I_y^2)v_{kl} + \lambda I_xI_yu_{kl} = \overline{v_{kl}} - \lambda I_yI_t. \end{displaymath}

This gives us two equations in the two unknowns ukl and vkl. These can be solved directly and suggest the iterative scheme

\begin{displaymath}
u_{kl}^{n+1} = \overline{u}_{kl}^n - \frac{I_x \overline{u}_...
 ...I_y \overline{v}_{kl}^n + I_t}{1 + \lambda (I_x^2 + I_y^2)}I_x \end{displaymath}

and

\begin{displaymath}
v_{kl}^{n+1} = \overline{v}_{kl}^n - \frac{I_x \overline{u}_...
 ..._y \overline{v}_{kl}^n + I_t}{1 + \lambda (I_x^2 + I_y^2)}I_y. \end{displaymath}

In other words, the new value of (u,v) at a point is equal to the average of the surrounding values minus an adjustment in the direction of the brightness gradient.

Note that this scheme also requires estimates of the values Ix, Iy and It. These can be computed by taking local averages in neighbourhoods about the grid point (i,j,k).


next up previous
Next: References Up: Computer Vision IT412 Previous: Structure from motion
Robyn Owens
10/29/1997