Morphological Scene Change Detection
Elena Stringa
European Commission – Joint Research Centre
Institute for Systems, Informatics, and Safety
TP 270, 21020 Ispra (VA), Italy
elena.stringa@jrc.it
Introduction
Background: Elementary Morphological Operations
Morphological Scene Change Detection
Results
References
Video surveillance systems with detection tasks usually include a module aiming at detecting relevant changes in the guarded environment [1-4]. If the output of the CD module is further processed, it is necessary to have a CD algorithm as accurate as possible.
Change detection (CD) techniques presented in the literature can be divided in two classes: pixel-based and region-based algorithms [5]. Pixel-based algorithms compute the output by analyzing the values assumed by correspondent pixels in the two analyzed images; region-based algorithms compare features extracted from correspondent regions in the two images. Pixel based methods, (e.g. CD based on binary difference) present the advantage of the simplicity that makes possible real-time applications, whereas region-based techniques (e.g. CD based on the illumination model [6]) provide results more robust to false alarms introduced by noise. A further class of CD algorithms detects changed regions by means of edge comparisons [5]. These methods are quite expensive from the computational point of view and therefore they are not indicated for applications having strict temporal requirements.
Each CD technique is characterized by the metric introduced in order to quantify the difference between pixels or regions, and by a threshold value allowing to decide if the analyzed pixel or region is changed with respect to the reference image. The threshold setting is a crucial step because different threshold values provide extremely different results.
It would be desirable that CD algorithms presented the advantages of both pixel and region based techniques. This goal has been reached by considering binary morphological filters, that compute the output value of a pixel according to the values of its neighbouring set, applied to the output of binary difference of CD systems [7][8].
In the literature, it is possible to find four kind of morphological operators, of which two are deterministic (mathematical morphology [7] and soft morphology [9]) and two are stochastic (statistical morphology [10] and statistical soft morphology [11-12]).
Mathematical morphology [7] is based on
the definition of a structuring element B, that can be interpreted as
filtering mask, and two elementary operators, erosion and dilation, whose
output is respectively the minimum and the maximum value found inside the
structuring element.
In case of binary data, it is possible to compute the output of dilation (erosion)
as follows:
where 
denotes the SE B centres at pixel i,
and 
represents the number of '1' elements
inside .
Erosion and dilation operators are combined in order to form more complex
filters, such as opening (dilation followed by erosion) and closing (dilation
followed by erosion).
In soft morphology [9] the structuring element B is
composed by two disjoint sets A (hard structuring element HSE) and B\A
(soft structuring element SSE) and it is characterized by an r parameter
related to the rank of an order filter. The elementary operations are soft
erosion and dilation that can be combined to form more complex filters. The
output of soft dilation (erosion) is the r-th greatest (smallest) value in the
set:
where denotes the repetition operation of the
intensity values I(a) falling inside the hard structuring element A. The
repetition of elements in the hard structuring elements means that a greater
importance is associated to them.
In case of binary data, the output of soft erosion (dilation) becomes:
Statistical morphology [10] and statistical
soft morphology [11-12] are a statistical extensions of mathematical
morphology and soft morphology, respectively. The SE centred at pixel i
is represented by a neighbourhood set 
such that:
The output of statistical dilation (erosion) is:
In statistical soft morphology, the neighbourhood set is
composed by the union of two disjoint sets 
and 
and the output of statistical soft dilation (erosion) at pixel
i is:
b is a parameter of
the Gibbs distribution introduced for computing the elementary operations.
In case of binary images, a fast implementation is possible taking into account
that only two values can be assumed by input data.
The output for binary statistical dilation (erosion) at pizel i is given by:
The output for binary statistical soft dilation (erosion) at pixel i is given by:
Morphological Scene Change
Detection
The fast implementation of binary morphological operators
allows their application to video surveillance systems.
Change detection based on binary morphology is performed according to the
following architecture:
Figure 1. Architecture of
morphological change detector.
A reference image representing the guarded environment is compared to
the image representing the current situation [1][2][3][4]. This
comparison is based on the pixel-to-pixel binarized difference between
the two images. Let 
the background image, composed by N
pixels, and 
the current analyzed frame (
is the instant in which the current image has been
acquired), for each pixel i the result of the binarized difference is obtained
by setting to the value ‘1’ the output if the difference is greater than a value
th. The threshold th indicates the minimal illumination difference to
consider two pixels different; in the ideal case, this distance is equal to one
gray level, but due to the noise introduced by the acquisition device, the
operation is more robust by setting a higher threshold.
When the processed images assumes vectorial values, for instance they presents
R, G, and B color components, then the operation is applied to each image
channel, and the binary result is obtained by performing a logical OR among the
results obtained from each channel.
After the binary difference, morphological processing is introduced in order to
eliminating isolated noisy pixels and to compact into regions pixels belonging
to the same changed area. Usually morphological, statistical or soft
statistical openings are applied because the erosion presents the capabilities
of removing isolated noisy pixels and the dilation recompose the regions split
by the preliminary erosion.
When there aren't time constraints and it is necessary to have high accuracy in
CD operations, it is possible to filter the difference image and to binarize
the filtered difference.
In this section we are analyzing the performances of different
morphological change detection. We are performing two kinds of evaluation: the
firmer analyzes CD results at pixel level, the latter considers the impact of
CD results to higher level operations.
:
where:
- Pma represents the probability that a pixel belonging to a changed area is
not recognized;
- Pfa represents the probability that a pixel is erroneously detected as belonging to a changed area.
The input images are reported in Figs.2a-b, and Fig.2c
reports the ideal CD result.
(a)
(b)
(c)
Figure 2. Images used for CD evaluation: a) reference image; b) image
with changes; c) ideal CD result.
The following experiments
have been performed:
-
Exp1. Binary differences followed by morphological opening with square
5x5 structuring element;
-
Exp2. Binary differences followed by 2 statistical openings (5x5 square
SE, beta scheduling such that 
and 
);
-
Exp3. Binary difference followed by 2 binary statistical soft openings
(5x5 square SE, 3x3 square hard SE, r=8, beta scheduling such that and );
-
Exp4. Pixel-to-pixel difference followed by 2 statistical soft openings
(5x5 square SE, 3x3 square hard SE, r=8, beta scheduling such that and ) and binarization as last operation.
For all the experiments, binarization threshold values ranging from 10 to 35 have been considered. Experience suggest that threshold values out of the considered range are not convenient for real applications because too much high false alarm or misdetection rates would be obtained.
Figure 3. CD errors obtained with
the four considered experiments
Fig.3 shows the obtained error curves and Table I shows the
mean error value obtaining by averaging the results of different threshold.
|
Table I - Average errors obtained with different CD |
|||
|
Exp1 |
Exp2 |
Exp3 |
Exp4 |
|
11.423 |
8.122 |
5.703 |
3.646 |
Statistical soft morphology-based algorithms present errors
with small variations for different threshold values. In particular, it is
interesting that in the range 
the performance
curve of SSM-based CD algorithm is approximately flat. This means that small variations
of environmental conditions (e.g. illumination variations) do not affect CD
performances meaningfully.
Region-based Evaluation
Further results are related to the influence of the low level change detection algorithm in object detection. Experiments have been carried out by considering different change detection algorithms in a simple system for object (blob) detection. The blob detection algorithm is used in [4] and it is based on the following operation:
- “split and merge” algorithm, in order to detect and label different regions;
-
region filtering: regions with area lower that a threshold
(100 pixels, for the considered experiments) are deleted;
-
blob definition by tracing the minimum bounding box
surrounding regions;
-
blob merging: blobs presenting border with distance lower that
a threshold are merged in a unique blob (in the experiments, bobs have been
merged if their distance was lower than two pixels).
Average results have been measured by processing three
laboratory sequences of 5 frames. These small sequences are interesting because
they present variations of the global luminance and high reflections in the
floor, which may cause easily false alarms.
In these experiments, performances are based on the blob detection probability.
The definition of detection probability is important: it has been computed by
counting the object correctly detected in each frame of the sequence. Two or
more separate objects detected and merged into one blob, or objects split in
more blobs, are not counted because in this case the detection is not good.
Tables II and III show the detection and misdetection probability,
respectively, obtained by considering the four change detection algorithms used
in experiments Exp1-4, applied with binarization threshold th=15. Values reported in Tables II
and III confirm the conclusions obtained by performing algorithm evaluation at
a lower level: statistical soft morphology-based CDs provide more stable
results.
|
|
|||
|
Exp1 |
Exp2 |
Exp3 |
Exp4 |
|
0.30 |
0.58 |
0.75 |
0.92 |
|
|
|||
|
Exp1 |
Exp2 |
Exp3 |
Exp4 |
|
0.41 |
0.25 |
0.08 |
0.08 |
In the computation of misdetection probability, we consider only object not
detected at all. Objects erroneously detected as merged into one blob or split
in several blobs, even not counted in the correct detection probability, do not
increase misdetection probability.
- C.S.Regazzoni, G.Fabri, and G.Vernazza, “Advanced Video-based Surveillance Systems”, (1999), Kluwer Academic Publisher [ISBN: 0-7923-839203].
- E.Stringa and C.S.Regazzoni, “Real Time Video-Shot Detection for Scene Surveillance Applications”, IEEE Trans. Image Processing, Vol.9, No1, January 2000, pp. 69-79.
- J.D.Courtney, “Automatic Video Indexing via Object Motion Analysis”, Pattern Recognition, Vol.30, No.4, 1997, pp. 607-625.
- M.Bogaert, N.Chleq, P.Cornez, C.S.Regazzoni, A.Teschioni, M.Thonnat, “The PASSWORD Project”, Proc. IEEE International Conference on Image Processing, Lausanne (Switzerland), September 1996, Vol.III, pp. 675-678.
- A. K. Jain, “Foundamentals of Digital Image Processing,” Prentice-Hall, 1989.
- K. Skifstadt, R. Jain, “Illumination Independent Change Detection for Real World Sequences”, Computer Vision, Graphics, and Image Processing, Vol. 46, 1989, pp. 395 – 418.
- J. Serra, “Image Analysis and Mathematical Morphology: Theoretical Advances,” Vol.2, Academic Press, London 1988.
- B. Chanda, "Application of Binary Mathematical Morphology to Separate Overlapped Objects," Pattern Recognition Letters, September 1992.
- L.Koskinen, J.Astola, and J.Neuvo, "Soft Morphological Filters: a Robust Morphological Filtering Method", Journal of Electronic Imaging, Jan. 1994, pp. 60-70.
- A.Yuille, L.Vincent, and D.Geiger, “Statistical Morphology and Bayesian Reconstruction,” J. Math. Imag. and Vision, 1992,1(3), pp 223-238.
- E. Stringa and P.C. Smits, "Soft morphology and Bayesian reconstruction for SAR image filtering," Proc. International Geoscience and Remote Sensing Symposium (IGARSS) 2001, Sidney, Australia.
- E.Stringa, "Nonlinear
Image Processing and Its applications to Video Surveillance Systems",
(2000), Neos Editions [ISBN: 88-87262-07-1].