Experimental Results

Before giving details of the formal method we used to evaluate our proposal, we would like to emphasize a feature that we believe is an important contribution of our proposal: the use of a combination of colour and texture properties. To illustrate this, Figure 3 shows a simple experiment which consists of the segmentation of a mosaic composed by four regions, each of which has a common property, colour or texture, with their adjacent regions. Specifically, each region has the same colour as its horizontal neighbouring region, while the vertical neighbouring region has the same texture. As is stated in the first two examples of Figure 3, the method allows the colour texture properties to be modelled and the four regions to be correctly segmented. On the other hand, we included in Figure 3 a third experiment which consists on the segmentation by using only colour information. Third image shows the smoothed version of the first mosaic image, and clearly illustrates that the original image contains only two colours. Therefore, the segmentation of the image using colour information, although different techniques can be used (considering a Gaussian distribution on the smoothed image, modelling using a mixture of Gaussians on the original image, or other techniques such as the kernel density estimation), will only allow us to identify two colour regions and it is not possible to distinguish regions with the same colour but different texture. As is stated, in some cases colour alone does not provide enough information to perform colour texture analysis, and in order to correctly describe the colour texture of a region, we need to consider not just the colour of pixels, but the relationships between them.

Figure 3: Segmentation of 4 regions composed from two textures and two colours. First row shows the mosaic images. Second row shows the borders of segmented regions.

The described segmentation method can be performed over any set of textural features. The result of comparing the relative merits of the different types of features have been nonconclusive and an appropriated set of features has not emerged in all cases. For the experimental trials shown in this article we used the co-occurrence matrices proposed by Haralick et al. [24]. Two of the most typical features, contrast and homogeneity, are computed for distance $1$ and for $0^{\circ}$,$45^{\circ}$,$90^{\circ}$ and 135$^{\circ}$ orientations to constitute a 8-dimensional feature vector. Moreover, the ($L^*,u^*,v^*$) colour space has been chosen to model the colour.

In order to evaluate the proposed colour texture segmentation technique, we created $9$ mosaic images by assembling $4$ subimages of size $128\times128$ of textures from the VisTex natural scene collection by MIT ( http://www-white.media.mit.
edu/vismod/imagery/VisionTexture/vistex.html), which we have called from $M1$ to $M9$. Furthermore, we added $3$ mosaics $M10$, $M11$ and $M12$, provided by Dubuisson-Jolly and Gupta which were used to evaluate their proposal on colour texture segmentation described in [8]. A subset of colour texture mosaic images with obtained segmentation results is shown in Figure 4.

Figure 4: Subset of mosaic colour texture images. Borders of segmented regions are drawn over original images.

M2	M6	M7
M10	M11	M12

The evaluation of image segmentation is performed by comparing each result with its ground truth and recording the error. Specifically, we use both region-based and boundary-based performance evaluation schemes [25] to measure the quality of a segmentation. Region-based scheme evaluates the segmentation by measuring the percentage of not-correctly segmented pixels considering the segmentation as a multi-class classification problem. Meanwhile, boundary-based scheme evaluates the quality of the extracted region boundaries by measuring the distance from ground truth to the estimated boundary.

Images were processed by our segmentation algorithm using various set of parameter values for the weight of colour (parameter $\beta$) and texture information, as well as the relative relevance of region (parameter $\alpha$) and boundary information in the segmentation process, and best results have been obtained with $\beta =0.6$ and $\alpha =0.75$. Note that a predominant role is given to colour and region information. Table 1 shows the quantitative evaluation of results obtained using this parameters setting over the set of mosaic images. Summarising, a mean error of $2.218\%$ has been obtained in the region-based evaluation for the whole set of test images. While the mean error at the boundary has been of $0.841$ pixels. Furthermore, our proposal obtained errors of $0.095\%$, $3.550\%$ and $1.955\%$ in the segmentation of $M10$, $M11$ and $M12$, respectively (see segmentation results of these mosaic images in second row of Figure 4), which can be compared to the segmentation results shown in the work Dubuisson-Jolly and Gupta [8]. Their proposal is a supervised segmentation algorithm based on the fusion of colour and texture segmentations obtained independently. Both segmentations are fused based on the confidence of the classifier in reaching a particular decision. In other words, the final classification of each pixel is based on the decision (from colour or texture) which has obtained a higher confidence. Our results have to be considered as very positive since they significantly improve colour texture segmentation results presented in [8].

Table 1: Region-based and boundary-based evaluation for the best results of colour texture segmentation over mosaic images ($\beta =0.6$ and $\alpha =0.75$).

	Region-based	Boundary-based
	(% error)	(pixels distance)
M1	2.207	0.352
M2	0.280	0.145
M3	0.731	0.237
M4	2.375	0.588
M5	1.663	0.786
M6	2.352	0.341
M7	1.451	0.596
M8	6.344	1.774
M9	3.609	3.430
M10	0.095	0.028
M11	3.550	0.962
M12	1.955	0.852
Mean	2.218	0.841
Std	1.711	0.940

Furthermore, the performance of our proposal for colour texture segmentation has been finally tested over a set of real images. Natural scenes predominate among these images, since nature is the most complex and rich source of colour and textures. Some colour texture segmentation results are shown in Figure 5. Meaningful regions in images are successfully detected and the usefulness of our proposal for colour texture segmentation is demonstrated. Furthermore, we want to emphasize some aspects related to the obtained results. See the last example of Figure 5 which shows the segmentation of a monkey among some leaves. The monkey is correctly segmented and, moreover, although the animal is absolutely black several parts of its skin are identified due to their different textural properties. Similar situations occur with other images in which animals are present. In the image with a leopard, region at neck which is not composed by typical spots of the animal, is detected and the same occurs with the lizard image in which the body of the animal, neck and belly are segmented as different regions. It is true that in these cases many of human would group all these regions to compose a single region related to the whole animal body. Nevertheless, this process of assembling is more related to the knowledge that we have about animals that to the basic process of segmentation. Hence, we believe that the segmentation performed by our proposal is correct as it distinguishes regions with different colour texture. The task of region grouping, if necessary, should be carried out by a posterior process which uses higher-level knowledge.

The correctness of boundaries obtained in these segmentations is also shown by the sketch of detected borders over original images. As has been pointed out, texture segmentation is specially difficult at boundaries and great errors are often produced at them. Hence, we want to note the accuracy of segmentations considering not only the correct detection of regions, but also the precise localisation of boundaries between adjacent textures.

Figure 5: Colour texture segmentation results on real images ($\beta =0.6$ and $\alpha =0.75$). Borders of segmented regions are drawn over original images.