In July 1989, the European Commission funded a consortium of six laboratories to investigate control of perception in a continuously operating vision system. The consortium partners set out to build a test-bed vision system for experiments in control and integration. A system was constructed which integrates a 12 axis robotic stereo camera head mounted on a mobile robot, dedicated computer boards for real-time image acquisition and processing, and a distributed system for image description. The distributed system includes independent modules for 2D tracking and description, 3-D reconstruction, object recognition, and control. On March 18 1992, a fully integrated test-bed system was demonstrated to the European Commission, operating at speeds of 1 hz to 10 hz depending on the task.
Real time response requires limiting the amount of information processed at any instant. Applying this idea to stereo vision leads to a system in which reconstruction is limited to the region of a scene around a fixation point. At a macroscopic level, perception is then controlled as a serial process in which fixation is momentarily posed on interesting points in the scene, and the local structure is described.
In addition to the problem of control of fixation, the construction of a "active" 3D vision system required solutions to a number of technical problems, some of which are new, and some of which are classic problems posed in a new form. This tutorial concerns these problems and their solutions. In particular, it addresses the techniques developed for:
1) The reflexive control of focus, aperture and convergence;
2) Estimation and command of a fixation point defined by the intersection of optical axes from the stereo cameras;
3) Reconstruction of the 3D structure of objects in a local coordinate system which is intrinsic to the object. 4
) Dynamically correcting camera calibration for changes in focus, aperture, convergence, and head movements.