CAVE On-Line Video Demonstrations

ON-LINE VIDEO DEMONSTRATIONS

Computation of Reflectance Ratios from Images
Reflectance Based Object Recognition
Microscopic Shape from Focus
Real-Time Active Defocus Range Sensor
Telecentric Optics
100 Object Recognition System
Real-Time Robot Positioning
Real-Time Visual Tracking
Real-Time Temporal Inspection

Computation of Reflectance Ratios from Images

A photometric invariant, called reflectance ratio, has been proposed for image segmentation and object recognition. For a large class of reflectance functions, the reflectance ratio is invariant to local surface geometry and illumination conditions. Reflectance ratios at all image points can be computed in a single raster scan of a black and white or color image. The reflectance ratios of regions in the scene can be computed with a second scan of the image. The first clip shows the illumination of a scene being varied [1]. In the second clip, the region reflectance ratios, computed for the image sequence in the previous clip, are shown in color (shades between blue to red) [2]. Despite drastic illumination changes, the region reflectance ratios (colors of regions) are seen to remain constant (invariant).

Related Publications

"Reflectance Based Object Recognition,": S. K. Nayar and R. M. Bolle,; International Journal of Computer Vision,; to appear in 1996.; [gzipped][uncompressed]
"Computing Reflectance Ratios from an Image,": S. K. Nayar and R. M. Bolle,; Pattern Recognition.; Vol. 7, August 1993.; [gzipped][uncompressed]

Reflectance Based Object Recognition

The reflectance ratio invariant has been used for three-dimensional object recognition. Each object is represented using the positions and reflectance ratios of regions on the object. During recognition, triplets of regions (ratios) are used as indices to generate hypotheses. These hypothesis are verified using the positions and ratios of neighboring regions in the image. The first clip shows an object being moved in a cluttered scene [1]. The second clip shows region reflectance ratios computed for the previous image sequence [2]. The last clip shows a region triplet (shown as a triangle) on the moving object that was successfully verified by the recognition algorithm. The object is thus recognized and its pose is computed using the positions of the regions that constitute the triangle [3].

Related Publications

"Reflectance Based Object Recognition,": S. K. Nayar and R. M. Bolle,; International Journal of Computer Vision,; to appear in 1996.; [gzipped][uncompressed]
"Reflectance Ratio: A Photometric Invariant for Object Recognition,": S. K. Nayar and R. M. Bolle, R; Proceedings of International Conference on Computer Vision (ICCV'93),; Berlin, May 1993.

Microscopic Shape from Focus

A shape from focus system has been developed for microscopic objects. The stage of an off-the-shelf optical microscope has been motorized to allow computer controlled sample positioning in the z-direction. An analytically derived illumination pattern is projected onto the sample to force a dominant single-frequency texture. A small set of images (10-20) is taken as the sample is displaced in steps towards the objective lens. The images correspond to different focus settings of the sample. The first clip [1] shows an industrial sample being automatically moved in increments by the system. The second clip [2] shows the computed three-dimensional texture-mapped surface of rectangular structures on a silicon wafer. The height of the structures is approximately 13 microns. The last clip [3] shows the computed structure of a green leaf. In the middle of the structure is a stomata (air vent) that is approximately 30 microns in height.

Related Publications

"Microscopic Shape from Focus Using Active Illumination,": M. Noguchi and S. K. Nayar,; Proceedings of International Conference on Pattern Recognition (ICPR 94),; October 1994.; [gzipped][uncompressed]
"Shape from Focus,": S. K. Nayar and Y. Nakagawa,; IEEE Transactions on Pattern Analysis and Machine Intelligence,; Vol. 16, No. 8, pp. 824-831,; August 1994.

Real-Time Active Defocus Range Sensor

A real-time range sensor has been developed that produces 512x480 depth maps of a scene at 30 Hz (video rate). The sensor is based on depth from defocus and uses a coaxially projected illumination pattern to ensure a single dominant frequency in the scene. A telecentric lens is used to ensure that the magnification of the scene is invariant to defocus. A prism splits the image formed by the lens into two. Two CCD cameras are used to detect two defocused images of the scene, simultaneously. The two cameras are positioned such that they have different effective focal lengths. The depth from defocus algorithm is implemented on Datacube's MV200 processing hardware. The first clip shows gray-coded depth maps of a moving hand displayed in real-time [1]. The second clip shows the depth map of milk being poured out of a cup [2]. The third clip shows a wireframe representation of the computed depth map of a dynamic scene [3]. The last clip shows depth maps (wireframes) of an object being computed as it is rotated by a motorized turntable [4]. The sequence of depth maps can be fused to obtain a CAD model of the rotating object.

Related Publications

"Real-Time Implementation of Depth from Defocus,": M. Watanabe, S. K. Nayar, and M. Noguchi,; Proceedings of SPIE Conference,; Philadelphia, October 1995.
"Real-Time Focus Range Sensor,": S. K. Nayar, M. Watanabe, and M. Noguchi,; Proceedings of International Conference on Computer Vision (ICCV 95),; pp. 995-1001, June 1995.; [gzipped][uncompressed]
"Real-Time Focus Range Sensor,": S. K. Nayar, M. Watanabe, and M. Noguchi,; Technical Report CUCS-028-95, November 1994.; IEEE Transactions on Pattern Analysis and Machine Intelligence,; to appear in 1996.; [gzipped][uncompressed]

Telecentric Optics

In most conventional lens, magnification varies with focus setting. This can cause severe problems for vision algorithms such as depth from defocus/focus. The magnification of an off-the-shelf lens can be made invariant to focus/defocus by placing an aperture at the front focal plane of the lens. The first clip shows a sequence of images obtained by varying the focus setting of an off-the-shelf lens (Nikon f=12.5mm) [1]. In the next clip, optical flow vectors are computed between frames to show that magnification varies significantly with focus setting [2]. The lens is converted to telecentric by attaching an external aperture at a calculated location [3]. The next clip shows the image sequence obtained by varying the focus setting of the telecentric lens. The optical flow vectors for this sequence are seen to be negligible, demonstrating the magnification invariance of telecentric lenses [4].

Related Publications

"Telecentric Optics for Computational Vision,": M. Watanabe and S. K. Nayar,; Proceedings of Image Understanding Workshop (IUW 96),; Palm Springs, February 1996.; [gzipped][uncompressed]

Real-Time 100 Object Recognition System

A comprehensive framework has been developed for efficient appearance matching, based on the parametric eigenspace representation. Automatic algorithms have been implemented for the efficient computation of appearance models. Fast algorithms have also been introduced for the recognition of novel image vectors [1]. Appearance matching has been used to develop a real-time 3D recognition system with 100 complex objects in its database. A complete recognition and pose estimation cycle is completed in less than 1 second using no more than standard workstation and a color CCD camera. A recognition cycle includes scene change detection, object segmentation, brightness and scale normalizations, and appearance matching. The result is displayed using a sample image of the recognized object and its pose in degrees [2].

Related Publications

"Real-Time 100 Object Recognition System,": S. K. Nayar, S. A. Nene, and H. Murase; Technical Report CUCS-019-95, September 1994.; Proceedings of ARPA Image Understanding Workshop,; San Fransisco, February 1996.; [gzipped][uncompressed]

Real-Time Robot Positioning

The appearance matching framework has been used for real-time positioning of a robot with respect to an object. The hand-eye system is displaced by an unknown (random) position with respect to the object. The novel image (within a fixed preselected window) is projected to the parametric eigenspace to determine the displacement of the robot with respect to the desired position. This computed distance is used to drive the robot end-effector back to the desired position [1]. This system has been used for precise insertion of chips on circuit boards.

Related Publications

"Subspace Methods for Robot Vision,": S. K. Nayar, S. A. Nene, and H. Murase,; Technical Report CUCS-06-95, March 1995.; IEEE Transactions on Robotics and Automation,; to appear in 1996.; [gzipped][uncompressed]
"Learning, Positioning, and Tracking Visual Appearance,": S. K. Nayar, H. Murase, and S. A. Nene,; Proceedings of IEEE International Conference on Robotics and Automation (ICRA 94),; San Diego, May 1994.; [gzipped][uncompressed]

Real-Time Visual Tracking

The efficiency of appearance matching makes it possible to incorporate the above positioning algorithm in a real-time robot control loop. Displacement vectors computed by appearance matching are used as errors in a feedback control loop that enables the robot to track a three-dimensional object as it travels through an unknown trajectory in space [1].

Related Publications

"Subspace Methods for Robot Vision,": S. K. Nayar, S. A. Nene, and H. Murase,; Technical Report CUCS-06-95, March 1995.; IEEE Transactions on Robotics and Automation,; to appear in 1996.; [gzipped][uncompressed]
"Learning, Positioning, and Tracking Visual Appearance,": S. K. Nayar, H. Murase, and S. A. Nene,; Proceedings of IEEE International Conference on Robotics and Automation (ICRA 94),; San Diego, May 1994.; [gzipped][uncompressed]

Real-Time Temporal Inspection

A fast algorithm has been demonstrated for the visual inspection of complex manufactured parts. A hand-eye robot system is driven through a preselected trajectory that allows visual scanning of all the pertinent areas of a manufactured part (e.g. a printed circuit board). The sequence of images taken along the trajectory is compactly represented as a curve in eigenspace that is parametrized by robot travel time (along the trajectory) [1]. In the next clip, the hand-eye system is driven through the same trajectory and the images are projected to eigenspace [2]. The novel projections (yellow) are seen to precisely overwrite the stored appearance model (blue). Defects are being introduced to the part to test the inspection algorithm [3]. The hand-eye system is now driven through the same trajectory and a red flashing of the inset image is used to communicate large visual deviations from the stored model [4].

Related Publications

"Subspace Methods for Robot Vision,": S. K. Nayar, S. A. Nene, and H. Murase,; Technical Report CUCS-06-95, March 1995.; IEEE Transactions on Robotics and Automation,; to appear in 1996.; [gzipped][uncompressed]

WebMaster