As a simple demonstration, we built a demonstration program called
``FingerPaint
''. FingerPaint runs on an Apple Quadra AV/840. It
uses a work-space projected with an overhead projector using a
liquid-crystal display ``data-show''. A CCD camera with an 18mm lens
observes this workspace and provides visual input. ``Finger down'' and
``finger up'' events are simulated using the space bar of the keyboard
but they could be sensed using a microphone attached to the surface of
the desk. As illustrated in Figure 1, any ``natural pointing
device'' such as a finger can be used to draw pictures and letters, or
to move a drawing.
The image at time 
to be searched will be noted as
. The search process can generally be accelerated by
restricting the search to a region of this image, denoted s(i,j), and
called a ``Region of Interest''. Our system uses a square search
region of size M by M, whose center is is denoted as (io, jo). The
center corresponds to the location where the reference template was
detected in the previous image.
The robustness of the tracking system is reasonable but, as discussed below, its performance is inadequate with respect to Fitt's law. Preliminary experiments with local users indicate however that the current performance is acceptable for investigation purposes. In addition, the widespread availability of image acquisition and processing hardware adequate for real time correlation should alleviate our current performance problem. Most importantly, this demonstration has permitted us to explore the problems involved in watching gesture.