shadows and clouds detection in high resolution images

SHADOWS AND CLOUDS DETECTION IN HIGH RESOLUTION IMAGES USING
MATHEMATICAL MORPHOLOGY
Thiago Statella, Professor
Centro Federal de Educação Tecnológica de Mato Grosso – CEFET MT
Zulmira Canavarros street nº 95, Centro
Cuiabá – MT, Brazil
[email protected]
Erivaldo Antônio da Silva, Professor
Universidade Estadual Paulista Júlio de Mesquita Filho
Faculdade de Ciências e Tecnologia
Roberto Simonsen street nº 305
Presidente Prudente – SP, Brazil
[email protected]
ABSTRACT
The spatial resolution improvement of orbital sensors has broadened considerably the applicability of their images in
solving urban areas problems. But as the spatial resolution improves, the shadows become even a more serious
problem especially when detailed information (under the shadows) is required. Besides those shadows caused by
buildings and houses, clouds projected shadows are likely to occur. In this case there is information occlusion by the
cloud in association with low illumination and contrast areas caused by the cloud shadow on the ground. Thus, it’s
important to use efficient methods to detect shadows and clouds areas in digital images taking in count that these
areas care for especial processing. This paper proposes the application of Mathematical Morphology (MM) in
shadow and clouds detection. Two parts of a panchromatic QuickBird image of Cuiabá-MT, Brazil urban area were
used. The proposed method takes advantage of the fact that shadows (low intensity – dark areas) and clouds (high
intensity – bright areas) represent the bottom and top, respectively, of the image as it is thought to be a topographic
surface. This characteristic allowed MM area opening and closing operations to be applied to reduce or eliminate the
bottom and top of the topographic surface.
INTRODUCTION
The spatial resolution improvement of orbital sensors has broadened considerably the applicability of their
images in solving urban areas problems. Sensors like QuickBird can generate images up to 60 cm of spatial
resolution but as this resolution improves, the shadows become even more a serious problem especially when
detailed information (under the shadows) is required. Although shadows play an important role for images
interpretation they confuse targets identification.
As for Rosin and Ellis (1994), shadows areas can be seen as image semitransparent regions which retrieve
texture and bright levels reducing contrast.
Also, shadows regions have particular characteristics: they have gain less than 1 (i.e. contrast reduction) if
compared to the “bottom” of the image; this gain keeps constant over regions with shadows except at the edges
where the effect of a little illumination coming from the neighborhood rises contrast and bright.
Besides those shadows caused by buildings and houses, clouds projected shadows are likely to occur. In this
case there is information occlusion by the cloud in association with low illumination and contrast areas caused by
the cloud shadow on the ground.
Thus, it’s important to use efficient methods to detect shadows and clouds areas in digital images taking in
count that these areas care for especial processing. This paper proposes the application of Mathematical Morphology
(MM) in shadow and clouds detection. Two parts of a panchromatic QuickBird image of Cuiabá-MT, Brazil urban
area were used. The proposed method takes advantage of the fact that shadows (low intensity – dark areas) and
clouds (high intensity – bright areas) represent the bottom and top, respectively, of the image as it is thought to be a
topographic surface. This characteristic allowed MM area opening and closing operations to be applied to reduce or
eliminate the bottom and top of the topographic surface.
Pecora 17 – The Future of Land Imaging…Going Operational
November 18 – 20, 2008 Š Denver, Colorado
ON SHADOWS AND CLOUDS DETECTION
For Polidoro et al. (2005), shadows and clouds cause serious interferences in aerial images radiometry and also
some target occlusion. Besides that, for satellite images, clouds can reduce the useful area of the image as much for
the occlusion as for their projected shadows on the ground. Moreover, they confuse images automatic correlation
processes, DTM generation and automatic extraction of buildings and highways (SAINTS et al, 2006).
Li at al (2004) developed a methodology for automatic shadows detection and removal in high resolution
images of urban areas. The considered system is based on the use of DTM and an initial hypothesis on the angle of
solar inclination so the places of shadows occurrence can be defined. Once shadows have been detected, the
algorithm chooses neighboring regions whose histograms are used as model for equalizing the histogram of regions
with shadows. Although the process is automatic and has been successful in some applications, there are at least two
problematic points: one is the needing of a high precision DTM whose resolution be compatible to the image;
another one is the automatic neighborhood shadows areas election, because according to the direction chosen the
elected area might not have the same targets as the shadow area.
Prati et al (2001) worked with shadow detection of objects in movement. Basically, the aiming of the studied
algorithms is to prevent that shadows of objects in movement be confused with themselves. Results show that the
applied algorithms are efficient in detecting vehicles passing through on a road. However, the used images are
relatively simple if compared to those taken over urban areas, where a greater variability of targets is found.
Bevilacqua and Roffilli (2001) developed a shadows detection method for applications in vehicles traffic flow.
First a regions growth algorithm is applied to attenuate possible noises, trying to reduce the number of false
detections. Then shadows are detected and removed from the image to identify objects in movement. In this method
spatial and radiometric information about the objects are used to separate shadows from the other objects.
Every commented works present some limitations in clouds and shadows detection. This work aims to present
an alternative methodology to do that.
MATHEMATICAL MORPHOLOGY
Mathematical Morphology (MM) is a tool for image components extraction that are useful in regions form
representation and description, as borders, skeletons and convex latch; and also a tool for pre-processing as filtering,
thining and pruning (GONZALEZ & WOODS, 2000).
Mathematical Morphology is base don two basic operations: erosion and dilation.
Definition 3.1 Let B be a subset of Z2. f eroded by B is the minimum from the translations of f through the vectors
−b in B. B is called Structuring Element.
εB ( f ) =
∧
f _b
(1)
b∈B
Definition 3.2 Dilation of f by B is de maximum from the translations of f through the vectors −b in B.
δB ( f ) =
∨
f _b
(2)
b∈B
More details about erosion and dilation properties and exemples can be found in Banon and Barrerra (1998),
Serra (1986), Barrera (1987), Haralick et al. (1987) and Facon (1996).
Based on erosion and dilation operations it is possible to define increasing and idempotent transformations:
opening and closing filters.
Definition 3.3 The opening of f by B is the erosion of f by B followed by a dilation using B transposed (Soille,
1998).
γ B ( f ) = δ ~ [ε B ( f )]
(3)
B
Definition 3.4 The closing of f by B is the dilation of f by B followed by an erosion using B transposed (Soille,
1998).
Pecora 17 – The Future of Land Imaging…Going Operational
November 18 – 20, 2008 Š Denver, Colorado
φB ( f ) = ε [δ B ( f )]
~
(4)
B
Properties on opening and closing can be found in Banon and Barrerra (1998).
According to Soille (1998) another transformation which has the same properties as the opening but cannot be
written just as erosion followed by dilation is called algebraic opening. The same way an algebraic closing can be
figured out. Powerful tools for filtering connected components are algebraic closing and opening g known as area
closing and area opening.
Definition 3.5 Area opening is equivalent to the union of every openings by B whose size in pixels equals λ (Soille,
1998).
γλ =
∨ {γ
Bi
| Bi conected , Area( Bi ) = λ}
(5)
i
Definition 3.6 Area closing is equivalent to the intersection of all closings by B whose size in pixels equals λ
(Soille, 1998).
φλ =
∧{φ
Bi
| Bi conected , Area ( Bi ) = λ}
(6)
i
OPENING AND CLOSING ON TOPOGRAPHIC SURFACES
Shadows regions present lower brightness values what is caused by the attenuation in illumination and
contrast, making such regions behave as minimum in the digital images. Otherwise, clouds tend to present high
brightness and behave as maximum due the high reflectance, caused mainly for non selective scattering of the direct
illumination solar. More details about illumination effects in Remote Sensing can be found in Richards and Jia
(1998), Novo (1992) and Moreira (2001).
Taking a gray image affected by the presence of shadows and clouds and representing it as a topographical
surface where the heights correspond to the pixels values, visually it will be easy to realize that shadows and clouds
define valleys and peaks. Figure 1 shows an image with peaks. Cells highlighted represent a cloud region.
5
5
5
5
5
5
5
5
5
5
20
20
5
5
5
5
5
5
20
20
5
5
20
5
5
5
5
5
5
5
20
5
5
30
30
30
5
5
5
5
5
30
30
30
10
10
5
5
5
30
30
30
10
10
5
5
5
5
5
5
5
5
5
5
Figure 1. Gray image with clouds presented as topographic surface.
Pecora 17 – The Future of Land Imaging…Going Operational
November 18 – 20, 2008 Š Denver, Colorado
Figure 2 shows an image with valleys. Cells highlighted represent a shadow region.
50 50 50 50 50 50 50 50
50 2
2 20 50 50 2 50
50 20 20 20 50 50 2 50
50 20 20 20 50 50 50 50
50 50 50 50 50 50 50 50
50 50 50 50 50 50 50 50
50 50 50 50 50 50 1 50
50 50 50 50 50 50 50 50
Figure 2. Gray image with shadows presented as topographic surface.
Taking advantage of these characteristics, area opening and closing operations can be applied to reduce or
eliminate, respectively, peaks and valleys. The difference between reduction and elimination depends on the choice
of the area to be removed.
Figure 3 shows examples of area openings with sizes 4 (Figure 3a) and 10 (Figure 3b) applied to the image
showed in Figure 1. In a) the isolated peak with area 2 (level 20) is eliminated; in b) the peak with area 4 (level 20)
is eliminated and peak with area 9 (level 30) is reduced to make a plate with the lower peak (area 4, level 10).
Peaks
eliminated
Area 9 peak reduced
b
a
Figure 3. Area opening with area 4 (a) and 10 (b).
Area closing with sizes 2 and 3 were applied to the image in Figure 2 and the results are shown in figure 4. In
a) the isolated valley with area 1 (level 1) is eliminates; in b) valley with area 2 (level 2) is eliminates and area 9
valley (levels 2 and 20) is smoothed to form a basin with area 9 and level 20.
Area 2 valley eliminated
Basin
a
b
Figure 4. Area closing with sizes 2 (a) and 3 (b).
Pecora 17 – The Future of Land Imaging…Going Operational
November 18 – 20, 2008 Š Denver, Colorado
Both cases used connectivity 4 given by the structuring element (SE) cross. The small connectivity makes it
possible that even the smaller valley and peak areas be detected.
ORBITAL IMAGES PROCESSING
For evaluation of MM applied to detection of clouds and shadows two QuickBird satellite panchromatic band
sections of Cuiabá-MT, Brazil urban area (dated of 24/10/2005) were chosen as study areas: the first one is affected
by a cloud and its shadow (study area 1); the second one (study area 2) shows Cuiabá downtown with many
shadows caused by constructions.
Study Area 1
For study area 1 shown in Figure 5 the method was used to detect the cloud and its projected shadow over the
ground.
Figure 5. Study area 1 presenting a cloud and its shadow.
Cloud and some buildings in the study area 1 form regions of maximum in the image. The brightness of these
targets is saturated and even surpasses the cloud response. This peculiarity demands that the peaks formed by the
buildings be preliminarily smoothed, so they won’t affect the cloud detection.
An area opening whose area equals 15,000 pixels, less than the cloud area, was applied and produced an image
whose histogram (Figure 6) presents 3 peaks: one in the dark region of the histogram, referring to the shadow,
another one in the clearest region, referring to the cloud, and one third in the intermediate region, referring to the
other targets (constructions, streets, vegetation, etc.).
Pecora 17 – The Future of Land Imaging…Going Operational
November 18 – 20, 2008 Š Denver, Colorado
Figure 6. Area opening histogram
Then it was possible to enhance and detect the cloud. Figure 7 shows the processing sequence.
Figure 7. Processing sequence to detect the cloud.
The original image (Figure 7a) was processed with an area opening (Figure 7b) to smooth some targets
brightness peaks. After that, aiming to produce a greater contrast between the cloud and other targets, the image
resulting from area opening had its gray values inverted (Figure 7c) and subtracted from the resulting image in
Figure 7b. The resultant image (Figure 7d) eventually was limiarized (Figure 7e) for cloud detection.
Shadow detection process was similar to the cloud detection except for the initial processing that was an area
closing to smooth basins as it is shown in Figure 8.
Pecora 17 – The Future of Land Imaging…Going Operational
November 18 – 20, 2008 Š Denver, Colorado
Figure 8. Shadow detection process
For the shadow detection an area closing whose area equals 200,000 pixels, less than the projected shadow
area, was applied to smooth other targets shadows (Figure 8b). The result had its gray levels inverted (Figure 8c) and
then the smoothed image was subtracted from the gray levels inverted image (Figure 8d). The resulting image was
limiarized. Figure 9 shows both shadow and cloud detected superimposed to the original image.
Shadow
Cloud
Figure 9. Detected shadow and cloud superimposed to the original image
Study Area 2
In study area 2 (Figure 10) the goal was to detect shadows caused by the targets.
Pecora 17 – The Future of Land Imaging…Going Operational
November 18 – 20, 2008 Š Denver, Colorado
Figure 10. Study area 2 with shadows cause by targets in the urban area.
Area opening is an extensive transformation so that γ λ ( f ) ≥ f . Thus the goal is reached by doing
γλ ( f ) − f
and detecting all dark objects with size λ (the choice of λ in this case was based on the shadows areas
size to be detected). The processing is shown in Figure 11.
The image in Figure 11a was processed with an area opening (Figure 11b). The original image was subtracted
from the one in Figure 11b and eventually limiarized to detect shadow as it is shown in Figures 11c and 11d,
respectively.
f
a)
b)
γλ
c)
d)
T
Figure 11. Processings for urban area shadows detection.
Figure 12 shows detection resulting image superimposed to study area 2. The image was zoomed to make it
easier to notice details visually.
Pecora 17 – The Future of Land Imaging…Going Operational
November 18 – 20, 2008 Š Denver, Colorado
Figure 12. Urban area shadows detected.
CONCLUSION
Morphological operations succeed to detect shadow and cloud from the panchromatic image QuickBird. The
method is highly sensitive to shadows presence and the control given by the parameter λ makes it possible to select
which targets are to be detected. Analogous considerations can be thought when it comes to clouds. Area
segmentation (given by the detection) can be used in automatic classification processes to generate thematic maps
(using detected information to avoid interferences caused by shadows and clouds in the classification results), edge
detection, image interpretation, etc.
REFERENCES
BANON, G. J. F.; BARRERA, J., 1998. Bases da Morfologia Matemática para a análise de imagens binárias. São
José dos Campos: Instituto Nacional de Pesquisas Espaciais (INPE).
BARRERA, J. Uma abordagem unificada para os problemas de Processamento Digital de Imagens: a Morfologia
Matemática. 1987. Dissertação de mestrado, São José dos Campos, INPE.
BEVILACQUA, A.; ROFFILLI, M., 2001. Robust denoising and moving shadows detection in traffic scenes.
Proceedings of the IEEE Computer Sosiety Conference on Computer Vision and Pattern Recognition.
USA, pp. 1-4.
FACON, J., 1996. Morfologia Matemática: Teoria e Exemplos. Curitiba: PUC.
GONZALEZ, R. C.; WOODS, R. E. , 2000. Processamento de Imagens Digitais. São Paulo: Edgard Blucher
LTDA.
HARALICK, R. M.; STERNBERG S. R.; ZHUANG X. Image analysis using mathematical morphology. IEEE
Pattern Anal. Machine Intell., vol. PAMI-9, no. 4, pp. 532-555, Jul., 1987.
LI, Y.; SASAGAWA, T.; GONG, P., 2004. A system of the shadow detection and shadow removal for high
resolution city aerial photo. Proceedings of XXth ISPRS Congress, Commission 3.
MATHERON, G. , 1975. Random sets and integral geometry. Wiley.
MOREIRA, M. A. , 2001. Fundamentos do sensoriamento remoto e metodologias de aplicação. São José dos
Campos: INPE.
NOVO, E. M. L. M. , 1992. Sensoriamento remoto: princípios e aplicações. São Paulo: Edgard Blucher.
POLIDORO, A. M. et al., 2005. Detecção automática de sombras e nuvens em imagens CBERS e Landsat 7 ETM.
Anais XII Simpósio Brasileiro de Sensoriamento Remoto, Goiânia, Brasil, INPE, p. 4233-4240.
PRATI, A. et al., 2001. Shadow detection algorithms for traffic flow analysis: a comparative study. Proceedings of
IEEE Int’l Conference on Intelligent Transportation Systems, pp. 340-345.
RICHARDS, A. J.; JIA, X., 1998. Remote sensing digital image analysis: an introduction. Canberra, Australia:
Springer-Verlag.
Pecora 17 – The Future of Land Imaging…Going Operational
November 18 – 20, 2008 Š Denver, Colorado
ROSIN, P. L.; ELLIS, T., 1994. Image difference threshold strategies and shadow detection. Proceedings of the
sixth British Machine Vision Conference.
SANTOS, D. R.; DALMOLIN, Q.; BASSO, M., A. Detecção automática de sombras em imagens de alta resolução.
Boletim de Ciências Geodésicas, sec. Artigos, Curitiba, v. 12, no 1, p.87-99, jan-jun, 2006.
SERRA, J. P. F., 1986. Introduction to mathematical morphology. Computer Vision. Graphics and Image
Processing.
Pecora 17 – The Future of Land Imaging…Going Operational
November 18 – 20, 2008 Š Denver, Colorado