Reflectance Estimation

Estimation of Surface Colour

Summary:

The light which enters a colour camera is a combination of the colour of the surface and the colour of the light and and so camera measurements depend on the colour of the viewing illuminant. Colour constancy research in computer vision aims to remove the confounding effect of the illuminant colour so that the colour camera sees only surface colour (i.e. RGB measured for a surface viewed under canonical lighting conditions).

Current research:

The colour constancy problem has been the source of a large research effort[MW86,Lan77,GJT88,FD88]. However, it is only in the most recent research[For90,FDF94a,Fin95] that effective algorithms (i.e. algorithms which work on real images) have been delivered. The key ideas driving progress are that the range of colours that a camera sees depend on the illumination[For90] and that illumination itself varies only within certain bounds[Fin95]. Indeed the latter constraint has been shown to be especially useful in scenes where there is more than one viewing illuminant[FFB95,BFF96] in a scene (e.g. sun and sky). Good colour constancy is returned so long as there is sufficient colour diversity (i.e. range of colours) in the scene.

Future research:

The developed algorithms only recover surface colour up to an unknown scaling e.g. the direction of the RGB vector is returned but its magnitude is unknown. As such whites and greys cannot currently be distinguished. Recovering surface colour magnitude is the main open problem in surface colour estimation. However, a colour constancy algorithm which would work in scenes of low colour diversity would also be of great value.

Estimation of surface spectral reflectance

Summary:

For some applications it would be useful to recover the full spectral reflectance functions of surfaces (e.g. material classification).

Current research:

Funt, Ho and Drew[HFD90] have developed an algorithm which takes a colour signal as input (surface spectral reflectance multiplied by the spectral power distribution of illumination) and returns an estimate of the surface reflectance function. Their approach has the advantage that with a single measurement of the scene (albeit a full spectrum) they can recover an estimate of reflectance and so, their method provides colour constancy at a single pixel. However, the reflectance estimate is often quite inaccurate.

Recent research[CH95] has attempted to improve the accuracy of Ho et al's estimation procedure. Unfortunately little progress was made with only very minor improvements reported.

Future research:

It is unclear whether the separate problem is soluble. However, any progress made here would have direct implications for research into the colour constancy problem.

Reflectance texture Invariants

Summary:

While progress has been made into surface colour estimation, the recovered estimates are coarse approximations of the true colour. Moreover, obtaining these estimates takes considerable time. This circumstance has spawned research into reflectance/texture invariants: functions of surface colour which can be estimated quickly and with good accuracy.

Current research:

Glenn Healey's group at UC Irvine is the leader in research in this field. They have produced a comprehensive series of invariants which are independent of viewing illuminant and/or viewing geometry[KH94,HS94,HW95]. By adopting a more concise model of image formation[FDF94b], Finlayson et al have shown that some of these invariants can be simplified[FCF96b,GFF95,FCF96a]. Importantly the simpler invariants have been shown to capture more useful information. Nayar and Bolles[NB93] nad Van Gool et al's[VGMU96] work on photometric invariants is also significant.

Reflectance invariants have been applied to object recognition[FF95], Image indexing[FCF96b], scene annotation[HJ96] and even the modelling of aspects of human colour vision[FN94].

Future research:

The wider role of Finlayson et al's simpler image model needs to be explored for all reflectance/texture invariants. Moreover, there exists imaging conditions not accounted for by Healey's or Finlayson's image models---images of many surfaces viewed under spectrally non-uniform fields (e.g. outdoor scenes). Work needs to be done on constructing invariants for all likely viewing conditions.

Shape and Colour

Summary:

Research has shown that the colours recorded in a scene and surface shape (surface normals) are inexorably intertwined[Pet93]. The challenge for computer vision is to exploit this relationship in designing algorithms for shape recovery and reflectance estimation.

Current research:

Petrov has shown that the RGB measurements in a colour image are a linear transform from surface shape of surfaces (e.g. surface normals) in a scene. Moreover, assuming that the scene is illuminated by at least two spectrally distinct light sources (e.g. sun and sky) it is possible to extract surface shape from an image[DK94]. The relationship between shape and colour has been exploited in algorithms for segmentation[PK94], specular highlight detection[Dre94] and face recognition[FDFD96].

Future research:

Current research is quite theoretical in nature. The colour/shape relationship identified by Petrov has never been used for recovering shape or colour in real images and this is probably because to do so is a hard problem. Taking the theory into practice is the key research that needs to be carried out in this area.

Surface and Illuminant gamut constraints

Summary:

The surface gamut constraint states that the range of colours that a camera measures depends on the colour of the light and the illuminant gamut constraint states that the range in the colour of lights is limited.

Current research:

Forsyth[For90] and Finlayson et al[Fin95] have shown how these gamut constraints can exploited in solving for surface colour.

Future research:

The colour gamut constraints are quite general and should be readily applicable to a variety of computer vision problems. Gamut constraints, based on geometric as oppose to colour features have already found application in determining correspondences between model and test images[BJ95]