Don Braggins
http://dialspace.dial.pipex.com/vision/
October 1996
This is a very personal view of the 'state of the art' in industrial vision, trying to define the border area where the results of research in vision have either just been commercialised, or look just ready for commercialisation (and look likely to be of economic benefit). It is based on a trawl through my memory, and takes into account only things I have seen or heard about first-hand and have been interested in when I heard about them. It is not based on any formal database search.
I divide industrial vision applications sectors into Recognition, Guidance, and Inspection.
Recognition of two-dimensional image objects on the basis of showing an example seems more or less commercially developed, e.g Cambridge Neurodynamics Ltd. system for searching image databases by image content, based on work at Cambridge University Engineering Labs with adaptive non-linear filtering. Applications range from house sales (We like the look of this house but not its location - what do you have similar?) to searching of criminal records.
Recognition based on Synergetic Computing techniques, demonstrated by Fraunhofer AIS Erlangen, looks fairly close to industrial usability, potentially easier to teach than neural nets. The latter are very slowly becoming used in industry, usually in embedded form so that the user does not know about them. An example is their use in recognition of containers whose plastic content can be recycled most productively if the original product (hence container material and colour) can be determined, even if it is squashed, dirty and distorted. Vehicle number plate recognition, based on neural nets and other techniques, is now developed to the point that it probably equals human observers - both are foiled by exhaust plumes in condensing conditions, for instance. Applications include automatic ticketing of speeders, toll enforcement, origin and destination research, road tax enforcement, and security.
Recognition of 3-D shapes has attracted a great deal of research attention, for instance the work at Reading and Leeds Universities on wire-frame modelling of vehicles and people. This seems a little before, rather than on the point of, commercialisation, with enormous potential for intelligent surveillance.
Recognition of 3-D shapes from 2-D images, with attitude and/or size invariance, using frequency domain analysis, is probably well developed for military applications but various attempts to move the technology into the civilian area do not seem to have been successful so far. As spatial light modulators improve and become less expensive there must surely be a breakthrough soon. Early work pointed to recognition and classification of natural products as being the area of application with best potential.
2-D guidance as in PCB and chip alignment is very well established commercially. Most commercial robot guidance is based on 2-D scene analysis with the third dimension not fully sensed (an exception being weld-seam following). The robust stereo vision developed by Cipolla at Cambridge should have considerable potential in industry, but it may be some time before it gets adopted industrially.
There is considerable interest, in Germany, in comprehensive recycling of vehicles, and a semi-intelligent robot working with some human guidance (point and say guidance) may be the key to economical dismantling on a distributed basis (you cannot afford to move 'dead' cars far). Speech recognition technology is probably far enough advanced, and Cipolla's finger-following work could be the basis for, such a robot. One can imagine saying "This is a Vauxhall Cavalier Estate; I am pointing at the battery. Remove it intact." At which point the robot consults its database of car designs, 'looks at' the battery and surrounding area, works out how distorted the body is with respect to the original design, and formulates a plan of action to remove the battery without further human assistance.
True 3-D mapping using fast Moiré fringe techniques, Gray Code, or projected pattern-assisted stereo matching are all reasonably well developed technologies which could contribute to true 3-D guidance of robots.
Autonomous vehicles are already a commercial reality (Transitions Research service robots taking special meals, drugs, and medical records around in hospitals, Paris Metro automated sweepers) but there is still a lot of R&D needed for them to become widespread.
further divided into:-
Several commercial suppliers are pushing low-distortion lens design, and calibration of residual errors, to practical (and expensive) limits. An alternative approach at Otto von Guericke University, Magdeburg, uses 'deterministic neural nets' to calibrate out all such errors (from lenses and sensors) by 'showing' a known geometrically 'perfect' pattern to the system, which then works out the relationship between neighbourhood pixels needed to reconstruct the pattern accurately. If adopted commercially, this could make high performance optical technology less important, and transfer cost to computing power, the cost of which drops with time in contrast to the cost of optical manufacturing.
Several years ago the Italian company FASE (now out of vision) showed the potential for 'inspection against a CAD drawing' with its Iride 800 product, and SIRA Ltd. was working on a similar concept; it is disappointing that the concept does not seem to have become a commercial reality.
Very high resolution sensors are now available (at a price). Mapvision Oy, Finland, working on developments at VTT, have shown how a 'patchwork' of ordinary resolution cameras can be made to cover a large area at effectively high resolution, without needing careful positioning or aiming of each camera. In their case it is for stereo image capture but could be for any purpose, with potential for cost reduction.
Time-of-flight laser scanning for 3-D mapping of relatively large objects (ships, quarry faces, steel cupola linings) is commercially quite well developed now, though with scope for wider applications and more suppliers.
In the field of surface inspection, the German company ISRA, working (I think) with the local University of Darmstadt, has recently shown how flaws in patterned surfaces can be found (textiles, wallpaper etc.) with no knowledge of the pattern and no requirement for the pattern to be geometrically 'perfect'. Work by the University of Surrey on classifying natural textures (marble etc.) looks close to commercialisation.
In the field of image capture, some very high dynamic range cameras have been developed by IMS Stuttgart (now commercial) and PSI Zurich (not yet commercial), capable of yielding useful information in all parts of scenes even with bright sunlight and deep shade in the same image. Applications include automatic road vehicle guidance and arc welding guidance. These and other sources offer 'addressable pixel' capability, with potential for much faster analysis of only the 'interesting' parts of images.
Imaging Spectroscopy is being commercialised, based on developments by VTT Oulu. This produces multiple images of the same scene at different wavelengths, with immediate potential for forestry planning, crop condition monitoring, and mineral exploration (all using aircraft to sweep the scene) but there are probably industrial applications yet to be discovered, as multispectral imaging has not been used significantly in industry until now.