Face Tracking

In this experiment we have verified, whether tracking can be carried out in the wavelet subspace. This subspace method is an enhancement of the approach in [Krüger and Sommer, 2000]. First, we have optimized a WN $({\bf\Psi}, {\bf v})$ on the face image $I$ that we want to track, where ${\bf\Psi}= (\psi_{{\bf n}_1},\ldots,\psi_{{\bf n}_N})^T$. This WN will serve as our face template. Using eq. (4) and the notation of eq. (1), we can deform the template WN affinely:

$$\hat{I}({\bf S}{\bf R}({\bf x}-{\bf t}))=\sum_{i=1}^N v_i \psi_{{\bf n}_i}({\bf S}{\bf R}({\bf x}-{\bf t}))$$ (10)

where ${\bf S}$, ${\bf R}$ and ${\bf t}$ define, as in (1), dilation, rotation and translation. In [Szu et al., 1992,Szu et al., 1996], $\hat{I}$ was called a superwavelet, as the linear combination of wavelets is again a wavelet.

Tracking is established by finding at each time step the appropriate deformation parameters of the superwavelet such that the sum-of-squared difference between the image at time $t$ and the deformed template is minimized. To do so, we project at each time step $t$ the image $J_t$ into the wavelet subspace. This was done by first setting ${\bf S}$, ${\bf R}$ and ${\bf t}$ of the superwavelet $\hat{I}$ in eq. (11) to roughly appropriate values (e.g. by using the computed deformation values from the previous time step) and by then using the deformed dual wavelets to compute the corresponding wavelet coefficients ${\bf w}$. The difference

$$\Vert{\bf v}-{\bf w}\Vert _{{\bf\Psi}}$$ (11)

measures how well the deformation parameters were chosen. Based on this difference, we can minimized the energy functional

$$E = \operatornamewithlimits{min}_{{\bf S}, {\bf R}, {\bf t}}\Vert{\bf v}-{\bf w}\Vert _{{\bf\Psi}}$$ (12)

to compute optimal deformation values.

Estimating the optimal deformation values can be done efficiently: Since the linear combination of the wavelets $\psi_{{\bf n}_i}$ is a wavelet, $\hat{I}$ from above is again a wavelet and the optimization scheme of Section 2 can be applied. The employed Levenberg-Marquardt algorithm needs a number of cycles in which the deformation parameters are refined until a certain optimum is reached. In each cycle ${\bf w}$ in eq. (13) has to be recomputed. For a WN with 16 wavelets, this, however, needs just 16 projections of the filters onto the image. Since it can be shown that the matrix $({\bf\Psi})_{i,j}$ in eq. (7) is invariant (except some factor) to the changes due to ${\bf S}$, ${\bf R}$ and ${\bf t}$, it needs to be computed only once at the beginning. With a WN with $N=16$ wavelets we have reached 30 fps on a 700 MHz Linux-Pentium. Because of the high frame rate, the differences between successive images were small and the Levenberg-Marquardt algorithm seldomly exceeded 7 cycles. An example can be seen in fig. 5. The white box indicates the tracked inner-face region, on which our template WN was optimized. We have also experimented with different number $N$ of wavelets and noticed a linear decrease in speed, but an increase in precision for larger $N$ ($N<116$). A more detailed description of our experimental results can be found in [Feris et al., 2001].

Figure 5: Sample frames of our wavelet subspace tracking experiment. Note that the tracking method is robust to facial expressions variations as well as affine deformations of the face image.