Thesis Title

Transcription

Thesis Title

Detection of Pigment Network in Dermoscopy Images
Ana Catarina Fidalgo Barata
(Degree in Biomedical Engineering)
Master Thesis in
Biomedical Engineering
Jury
President: Patrı́cia Margarida Piedade Figueiredo
Supervisor: Jorge dos Santos Salvador Marques
Co-Supervisor: Armando Jorge de Mariz Rozeira
Member: Teresa Maria de Gouveia Torres Feio Mendonça
Member: Maria Margarida Campos da Silveira
Friday 2nd December, 2011
Research is to see what everybody else has seen, and to think what nobody else has thought.
Albert Szent-Györgi (1893-1986) U. S. biochemist
Acknowledgments
This work was supported by FCT in the scope of project PTDC/SAU-BEB/103471/2008.
I would like to thank to Prof. Jorge Salvador Marques for all the supportive guidance, ideas and
help during this thesis, regarding its engineering aspects. For all the time invested, I am also thankful
to him. I also would like to thank to Dr. Jorge Rozeira for all the support, help with the thesis’ medical
aspects, for the tutoring and for providing the access to the dermoscopy database of Hospital Pedro
Hispano. Without their collaboration and guidance, this thesis would not have been possible.
Ms. Bárbara Amorim, who selected and organized the database used in this work, thanks to her
for this. I would also like to express my thanks to Profs. Teresa Mendonça, André Marçal from the
University of Porto, Drs. Ofelia Morais, Marta Pereira, Marta Teixeira and nurse Lina Santos from
Hospital Pedro Hispano, for their support during this work.
To my parents, who always encouraged and supported me and that gave me the conditions to
study and to pursuit the future that I would like to have, and to my sister and my grandfather, for all
their support and comprehension, I thank to them.
Because life is more than just work and I needed to rest at some points during the conception of
this thesis, to all my friends thank you for the enjoyable moments! I would like to address a special
thanks to Jó, for keeping me sane and for all the patience that he showed during my most stressed
and grumpy moments.
i
Abstract
Melanoma is one of the deadliest forms of cancer. However, an early diagnosis significantly improves the probability of survival. Several medical procedures have been proposed to examine and
diagnose skin lesions. Dermoscopy is an inpection method used by dermatologists, which magnifies
the size of a lesion up to 100x, allowing a better visualization of several dermoscopic structures that
are invisible to the naked eye. One of the dermoscopic features that becomes visible is Pigment Network, which is used in several medical procedures (ABCD rule, 7-point checklist) not only to classify
skin lesions but also to diagnose a melanoma. The automatic detection of pigment network and the
extraction of its shape can help dermatologists performing a diagnosis. Despite its importance, pigment network is not used in automatic systems which try to classify skin lesions. A few works which
address the detection of this structure can be found in literature. However, most of them rely only
on detecting its presence, without extracting its skeleton, which is important for dermatologists. This
thesis proposes a new method for detecting and extracting pigment network in dermoscopy images
based on its color and geometry. The detection and extraction of pigment network will be accomplished by using a bank of directional filters and a connected component analysis. A methodology for
classifying lesions regarding the presence of pigment network was also developed. The algorithm for
network detection achieves a Sensitivity (SE) of 78% and a Specificity SP of 77% in a database of 57
clinical images acquired at Hospital Pedro Hispano.
Keywords
Skin lesion classification, Dermoscopy, Melanoma, Pigment Network detection, Directional Filters,
Connected Component Analysis, Features Extraction
iii
Resumo
O melanoma é uma das formas de cancro mais letais. Contudo, um diagnóstico precoce aumenta
consideravelmente as hipóteses de sobrevivência. Os dermatologistas utilizam diferentes procedimentos para observar e diagnosticar uma lesão cutânea. A dermoscopia é um desses métodos e
permite a ampliação de uma lesão até 100x, possibilitando a observação de estruturas que são indetectáveis a olho nu. Uma das estruturas que se torna visı́vel é a rede pigmentada, que é utilizada
em vários métodos médicos (regra ABCD e 7-point checklist) para classificar lesões e diagnosticar
melanomas. A detecção e extracção automáticas desta estrutura em imagens dermoscópicas pode
ajudar os dermatologistas. Todavia, os sistemas automáticos que classificam lesões cutâneas ignoram esta estrutura. Na literatura são poucos os trabalhos que dizem respeito à detecção automática
de rede pigmentada e a maior parte deles foca-se apenas na detecção de rede, sem proceder à
sua extracção, que pode ser importante para os dermatologistas. Esta tese propõe um novo método
para detecção e extracção de rede pigmentada, utilizando duas das suas propriedades: cor e geometria. O algoritmo automático utiliza filtros direccionais e uma análise de componentes conexas
para detecter a rede pigmentada. Um algoritmo que classifica lesões relativamente à presença de
rede pigmentada também foi desenvolvido. Este algoritmo apresenta uma SE = 78% e uma SP=77%
numa base de dados de 57 imagens, obtidas no serviço de dermoscopia do Hospital Pedro Hispano.
Palavras Chave
Classificação de lesões cutâneas, Dermoscopia, Melanoma, Detecção de rede pigmentada, Filtros direccionais, Análise de componentes conexas, Extracção de features.
v
Contents
1 Introduction
1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
1.2 Skin Lesions Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
1.3 Pigment Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
1.4 Thesis Objectives and Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
1.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 Dermoscopy Analysis
11
2.1 Dermoscopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 Dermoscopic Criteria and Pattern Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Medical procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.1 ABCD Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.2 7 Point-checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.4 Automatic Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.4.1 Automatic Systems for Melanoma detection . . . . . . . . . . . . . . . . . . . . . 15
2.4.2 Automatic Systems for Dermoscopic Features Extraction . . . . . . . . . . . . . 16
3 Directional Filters
19
3.1 Directional Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2 Directional Filter Bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4 Pre-Processing
25
4.1 Gray Scale Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.2 Reflection Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.3 Hair Detection
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.4 Inpainting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.4.1 Inpainting Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5 Network Detection
35
5.1 Previous works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.2 Network Enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.2.1 Directional Filters Bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.2.2 Thresholding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
vii
5.3 Network Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6 Region Validation and Lesion Classification
43
6.1 Region and Lesion Ground Truths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
6.2 Background Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.3 Region Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.4 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.4.1 Image Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.4.2 Topological Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.5 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
6.6 Lesion Classification algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
7 Results and Discussion
53
7.1 Database of dermoscopy images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
7.2 Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
7.2.1 Assessment of Region Detection Algorithm . . . . . . . . . . . . . . . . . . . . . 55
7.2.1.A Pixel based statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
7.2.1.B Region-based statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
7.2.2 Assessment of the validation step . . . . . . . . . . . . . . . . . . . . . . . . . . 57
7.2.3 Assessment of Lesion Classification . . . . . . . . . . . . . . . . . . . . . . . . . 57
7.3 Comparison between 4 and 8 pixels-neighborhood . . . . . . . . . . . . . . . . . . . . . 57
7.4 AdaBoost parameters selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
7.4.1 α weight parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
7.4.2 Features Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
7.4.3 Number of iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
7.5 Comparison between gray-level conversion methods . . . . . . . . . . . . . . . . . . . . 63
7.6 Region Detection System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
7.6.1 Detection examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
7.6.2 Statistical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
7.6.2.A Pixel-based statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
7.6.2.B Region-based statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
7.7 Lesion Classification Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
8 Conclusions and Future Work
71
8.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
8.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Bibliography
Appendix A Inpaiting Algorithm
viii
75
A-1
List of Figures
1.1 Melanocytic Nevi [2] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
1.2 Skin Lesions’ Classification Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
1.3 Melanoma that mimics Clark Nevus [3] . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
1.4 Melanoma that mimics Blue Nevus [3] . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.5 Melanoma that mimics Seborrheic Keratosis [3] . . . . . . . . . . . . . . . . . . . . . . .
5
1.6 Melanoma that mimics BCC [3] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.7 Examples of Pigment Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
1.8 Pigment Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
1.9 Thesis objectives (examples): Pigment network detection and lesion classification. . . .
8
1.10 Artifacts usually found in dermoscopy images . . . . . . . . . . . . . . . . . . . . . . . .
9
1.11 Block Diagram of the Detection System . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
3.1 Directional Filters Bank. The inpute I and all the outputs Ii are images . . . . . . . . . . 22
3.2 Impulse responses of 19 directional filters used in the pigment network’s enhancement
block. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3 Final output of the directional filters bank. . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.1 Block Diagram of the Pre-Processing step . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.2 Intensity studies in Reflection regions. Figures 4.2(a) and 4.2(b) are the original images
with a green line showing the direction along which the intensity profile study and the
deviation study were performed. Figures 4.2(c) and 4.2(d) are the intensity profiles
corresponding to the figures that are above them. Figures 4.2(e) and 4.2(f) are the
same as the previous two, but represent the deviation from average intensity. . . . . . . 28
4.3 Reflection detection in dermoscopy images . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.4 Presence of Hair artifacts in dermoscopy images . . . . . . . . . . . . . . . . . . . . . . 29
4.5 Block Diagram of the Hair detection system . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.6 Hair detection in dermoscopy images
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.7 Gray Scale images after artifacts removal . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.8 Propagation direction as the normal to the signed distance to the boundary of the region
to be inpainted [9] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.9 Gray Scale images after inpaiting algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 33
ix
5.1 Network Enhancement and Detection blocks . . . . . . . . . . . . . . . . . . . . . . . . 36
5.2 Output for the directional filters bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.3 Output of the thresholding step. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.4 Output of the network detection block. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.5 Detection of Pigment Network: original images(left) and pigmented network (right) . . . 41
5.6 Detection errors: original image(left) and pigmented network (right) . . . . . . . . . . . . 42
6.1 Ground truth segmentation of the pigmented network regions. . . . . . . . . . . . . . . . 44
6.2 Steps of the background extraction.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
6.3 Region labeling. Overlapping: A(blue) = 74% (label 1), B(green) = 0% (label 0), C(pink)
= 0% (label 0), D(yellow) = 0% (label 0), E(orange) = 76% (label 1), F(red) = 24% (label
0)
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.4 Lesion Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
7.1 Output of the network detection block for different connectivities. . . . . . . . . . . . . . 58
7.2 ROC curve obtained for the three different features’ sets by varying the α ∈ {1, 3, 5, 10, 15}
weight parameter and T = 2000. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
7.3 Histogram of the average rate of selection of each topological feature for α = 15 and
T = 2000. Each one of the ”Area Holes i” feature stands for the i − th bin of the
histogram of the area of holes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
7.4 Histogram of the average rate of selection of all the features for α = 15 and T = 500.
Each one of the ”H i” , ”HoG i” feature and ”H i” feature stands for the i − th bin of,
respectively, the histogram of intensities, HoG and histogram of the area of holes. . . . . 61
7.5 Correlation Graphics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
7.6 Output of the automatic detection system for lesions with Pigment Network: original
image (left), output of the network detection block (center) and final output after the
region validation block (right) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
7.7 Output of the automatic detection system for lesions without Pigment Network: original
image (left), output of the network detection block (center) and final output after the
region validation block (right) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
x
List of Tables
2.1 ABCD Rule of Dermoscopy [50] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 7 Point-Checklist: Criterion and Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Summary of recent CAD systems. Mel. stands for Melanoma. . . . . . . . . . . . . . . . 16
7.1 Percentage of melanocytic and non-melanocytic lesions in the dataset. . . . . . . . . . . 54
7.2 Percentage of . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
7.3 Statistical Results for the different α values and T = 2000. FP-rate stands for the
percentage of not pigment network regions that are wrongly classified ; TP-rate is the
percentage of correctly classified pigment network regions. . . . . . . . . . . . . . . . . 59
7.4 Statistical results for different numbers of iterations for α = 15 and topological features.
FP-rate stands for the percentage of non-network regions that are wrongly classified
as so while TP-rate is the percentage of correctly classified pigment network regions. . 62
7.5 Statistical results for AdaBoost performance using the highest entropy channel (see
(4.2)) as gray level conversion method for α = 15, T = 100 and topological features. . . 63
7.6 Statistical results for AdaBoost performance using the linear combination of RGB values (see (4.1)) a gray level conversion method for α = 15, T = 100 and topological
features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
7.7 Statistical results at a pixel level for the Network detection system before the validation
step, for α = 15, T = 100 and topological features. . . . . . . . . . . . . . . . . . . . . . 67
7.8 Statistical results at a pixel level for the Network detection system after the validation
step, for α = 15, T = 100 and topological features. . . . . . . . . . . . . . . . . . . . . . 67
7.9 Statistical results at region level for the network detection system before and after the
validation step. The validation is performed with α = 15, T = 100 and topological
features. CD - correct detection; FA - false alarm; DF - detection failure. . . . . . . . . . 68
7.10 Statistical results for lesion classification after validation, with α = 15, T = 100 and
topological features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
7.11 Statistical results for lesion classification before validation, with α = 15, T = 100 and
topological features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
xi
Acronyms
ANN
Artificial Neural Network
BCC
Basal Cell Carcinoma
CAD
Computer Aided Diagnosis
CD
Correct Detection
DF
Detection Failure
FA
False Alarme
FN
False Negatives
FP
False Positives
HoG
Histogram of Gradient
kNN
k-Nearest Neighbor
LoG
Laplacian of Gaussian
PDE
Partial Differential Equation
ROC
Receiver Operator Characteristics
SE
Sensitivity
SP
Specificity
SVM
Support Vector Machines
TDS
Total Dermoscopy Score
TN
True Negatives
TP
True Positives
xiii
1
Introduction
Contents
1.1
1.2
1.3
1.4
1.5
Motivation . . . . . . . . . . . . .
Skin Lesions Classification . . .
Pigment Network . . . . . . . . .
Thesis Objectives and Structure
Contributions . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2
2
6
8
10
1
1.1
Motivation
Melanoma is one of the deadliest forms of cancer, being responsible for most of the deaths related
to skin cancer. However, when detected in an early stage, it can be cured by recurring to a simple
excision of the lesion. Therefore, it is important do develop methodologies that allow a premature and
correct detection of melanomas.
Dermoscopy is a non-invasive medical technique for in vivo observation of pigmented skin lesions,
allowing their magnification and a better visualization of their surface and subsurface structures. This
medical tool has improved the accuracy of melanoma detection as well as an increasing of early detections. Nevertheless it requires a trained physician to make a competent use of this tool, otherwise
the medical diagnosis can be compromised. Based on this technique there are, nowadays, a set
of medical procedures that simplify the classification of skin lesions and, as a result, the detection of
melanomas. These procedures use the structures that were enhanced by dermoscopy to help dermatologists in their diagnosis. The most common are ABCD rule and 7 point checklist, just to cite some
of them. These rules use a set of dermoscopic criteria as the color of the lesion and the presence
or absence of certain kind of structures to classify the lesions (see Section 2.2). One of the major
structures analyzed by these rules is the pigment network. [3]
Pigment network has an essential role in skin lesion classification since it allows the distinction
between different classes of skin lesions as well as an identification of the lesions as melanoma or
not, since the presence of an atypical network is usually a sign of a malignant lesion. With that in
mind, the detection of this structure poses as an important task due to its great significance in medical
diagnosis. [3, 7]
Recently, several works have been developed in order to create automatic algorithms for melanoma
detection. Some of them extract a large set of features and use them to train a classifier so that it
will be able to distinguish between melanoma and other skin lesions. Other works choose a different approach and try to detect structures used by dermatologists to classify the lesions. The work
developed and described in this master thesis belongs to this second group.
Despite the importance of pigment network, only a few works try to detect this feature [5, 10, 13,
20,26,42], most of which do not focus on extracting its shape which is usually used by dermatologists
when performing a diagnosis. Therefore, new approaches for Pigment Network detection and extraction are required and this work presents a new, different and functional algorithm that can be used to
detect Pigment Network and extract its characteristic structure.
1.2
Skin Lesions Classification
Pigmented skin lesions classification is done in several steps. First, a distinction between melanocytic
and non-melanocytic lesions must be performed. This distinction is made based on a set of dermoscopic features, some of which are described in Section 2.2.
There are four main types of non-melanocytic lesions, each one of them with specific charecteristics. Three of the four are benign neoplasms that can appear in most parts of the human body.
2
Their names are: Seborrheic Keratosis, Vascular Lesions and Dermatofibroma. The malignant type
of non-melanocytic lesions is called Basal Cell Carcinoma (BCC). This is the most common primary
malignant neoplasm in humans. However, it grows very slowly, so usually a BCC is considered innocuous. Nevertheless it should be treated when detected, because it can still cause an extensive
tissue destruction. [3]
Melanocytic lesions originate from melanocytes, which are a type of cells responsible for the production of a protein pigment called melanin. This kind of lesions can be divided in two separate groups
(benign lesion, which include regular and atypical lesions, and malignant lesions or melanomas) considering their characteristics and if they are benign or malignant. Benign melanocytic lesions are
usually called moles or nevi and can be classified as acquired or congenital, when they are present
at the age of birth or appearing by the age of two. Another classification can be performed based on
the origin of these nevi within the skin. Melanocytic nevi are formed by organized clusters of nevus
cells (which have as precursors the melanocytes) arranged in various levels in the skin. The nest of
these clusters can be localized in the dermis (deepest skin layer), in the dermoepidermal junction or
even in both of them. Based on this a mole can be classified as intradermal, junctional or compound
since each one presents a set of specific features [1]. In Figure 1.1 examples of each type of nevi
can be seen. It is also possible to distinguish between nevi that have the same origin within skin,
considering their prevailing color (for example Blue Nevus, that can be seen in Figure 1.4(a) presents
a blue coloration) and other dermoscopic features such as Pigment Network, dots and streaks (see
Section 2.2) allowing a more specific classification of a skin lesion.
(a) Intradermal
(b) Junctional
(c) Compound
Figure 1.1: Melanocytic Nevi [2]
Some melanocytic nevi are irregular. Usually they are called Atypical Nevi and are histologically
distinctive from other melanocytic lesions. They can occur as solitary or multiple lesions and are
considered as an intermediate state between benign and malignant skin neoplasms. Atypical nevi
can be distinguished from regular melanocytic ones since they are usually larger, have an irregular
and indistinct border, exhibit a mixture of colors and an irregular surface, that usually shows a central
or eccentric papule, surrounded by a proeminent macular component. Atypical nevi can also be called
Clark Nevi. [1]
Melanoma is a common malignant form of melanocytic lesions that has the potential to metastasize. The term ”Melanoma in situ” refers to the early stage of the neoplasm, when it is situated within
the epidermis. Since it is not present in the dermis it has no contact with the vascular plexus and can
3
not metastasize. Therefore a simple excision of the neoplasm should be enough to cure the disease.
Nowadays, melanoma detection can be achieved by a combined use of the dermoscopy technique
described in Section 2.1 and of the medical procedures described in Section 2.3. Usually melanomas
in an early stage show an irregular shape and several colors as well as outlined macules or slightly
elevated plaques. Invasive melanomas can be papular or nodular, ulcerated and with a coloration that
ranges from brown to black, exhibiting also regions of red, white or blue. [3]
The several steps for lesion classification can be resumed in a classification tree as the one in Figure 1.2. This tree allows a better understanding of the lesion classification described in the previous
paragraphs.
Figure 1.2: Skin Lesions’ Classification Tree
One of the major pitfalls in lesion classification is the distinction between melanoma and other
types of skin lesions (melanocytic or not). Sometimes, a melanoma is underdiagnosed as being a
different kind of lesion, which results in a dangerous false negative, because melanoma mimics its
appearance. The existence of these kind of errors requires that in case of uncertainty in the diagnosis,
an histological exam must be performed. [3]
In the following figures some examples of the most common misleading melanocytic lesions are
presented. In Figures 1.3 and 1.4 it is shown how melanoma can mimic melanocytic lesions as
Atypical Nevi (Clark Nevi) and Blue Nevus.
(a) Clark Nevus
(b) Melanoma
Figure 1.3: Melanoma that mimics Clark Nevus [3]
4
(a) Blue Nevus
(b) Melanoma
Figure 1.4: Melanoma that mimics Blue Nevus [3]
Melanomas can also be mistakenly classified as non-melanocytic lesions. Figures 1.5 and 1.6 are
examples of that.
(a) Seborrheic Keratosis
(b) Melanoma
Figure 1.5: Melanoma that mimics Seborrheic Keratosis [3]
(a) BCC
(b) Melanoma
Figure 1.6: Melanoma that mimics BCC [3]
These images are quite elucidative of how difficult it is for dermatologists to correctly identify
melanomas sometimes. This is the main reason for the necessity of creating new automatic systems
for melanoma detection that can help dermatologists in lesion classification.
5
1.3
Pigment Network
Pigment network is one of the most important structures in dermoscopy [7]. It appears as a grid of
thin brown lines over a diffuse light brown background, having a shape very similar to a honeycomb [1].
In Figure 1.7 several examples of Pigment Network can be seen.
(a)
(b)
(c)
(d)
Figure 1.7: Examples of Pigment Network
Pigment network can cover most parts of a lesion (as illustrated in the two top images of Figure
1.7) or only small areas (see the two bottom images of Figure 1.7). However, in both cases, it forms
a pattern called reticular pattern, which is the most common feature in melanocytic lesions. It is
considered the dermoscopic hallmark of benign acquired melanocytic nevi and of thin melanomas,
although a pigment network can also be found in other kind of skin lesions as seborrheic keratosis
and dermatofibromas [3].
In order to have a better understanding of how the pigment network can help in the detection of
melanomas and other kinds of melanocytic lesions one must make use of histophatology. Skin has
two different layers: epidermis and dermis. Epidermis has a basal layer that possesses a type of cells
called melanocytes and the melanin produced by them is mostly distributed to keratinocytes localized
in the basal layer at the dermoepidermal junction.
If one does a vertical cut on the epidermis, it is possible to see that its structure is wave-like with
high and low points, this kind of structure is usually called rete ridge pattern. Usually there is a higher
concentration of melanin in the lower rete than in the upper rete near the skin surface.
If a horizontal slice of epidermis is taken, it is possible to see that the rete make a continuous
6
structure, having round walls with a hole in the middle. The walls of the rete are thick, project down
into the dermis and contain melanin. When viewed from above these round walls look like dark circular
lines and since they are connected with adjacent rete they are responsible for the dark grid pattern
characteristic of pigment network. The holes inside the walls contain a smaller amount of melanin and
because of this, when viewed from above, they look like a light circle [1]. Based on this explanation it
is possible to conclude that the grids in a pigment network are actually regions with a high amount of
melanin and that the holes are regions of a low concentration of this pigment.
The presence of pigment network represents an important feature for dermoscopy image analysis.
The assessment of its alterations is also very important, since it allows the distinction between benign
and malignant melanocytic lesions. Pigment network can be typical or atypical. As the classification
suggests, a network is considered typical when it is light to dark-brown pigmented, regularly meshed,
narrowly spaced and its distribution is more or less regular (see Figure 1.8(a)). An atypical pigment
network is characterized by a black, brown or gray, irregular network (with thickened lines), distributed
more or less irregularly throughout the lesion and usually ending abruptly at the periphery (see Figure
1.8(b)). [3]
(a) Typical Network
(b) Atypical Network
Figure 1.8: Pigment Network
A melanoma is caused by malignant melanocytes that vary in size and degree of pigmentation.
They can move through the epidermis in all directions, distorting its normal structure and causing variations in pigmentation patterns. Malignant cells are also capable of multiplying themselves, causing a
thickening of the rete, which is responsible for the thickening of the grid and being also responsible for
the darkening of the network [1].This analysis suggests that an atypical network is an unmistakable
sign of melanoma.
Achieving an automatic detection of pigment network and posterior assessment of its structure
can be an important tool in the future for the detection of melanomas.
7
1.4
Thesis Objectives and Structure
This master thesis has two main objectives. The first one is to develop an automatic algorithm
for the detection of pigment network regions in dermoscopy images and the second one is to classify
skin lesions regarding the presence or absence of pigment network. Figure 1.9 illustrates these two
problems applied to two different kinds of melanocytic skin lesions. The left image is a melanocytic
lesion labeled as with pigment network and the pigment network regions that must be detected are
highlighted. The right image is also a melanocytic lesion, but labeled as without pigment network. For
this lesion the algorithm should not detect pigment network.
(a) Lesion with pigment network - Network
regions are highlighted
(b) Lesion without pigment network
Figure 1.9: Thesis objectives (examples): Pigment network detection and lesion classification.
As was explained in Section 1.3, pigment network is part of a reticular pattern that can cover the
entire lesion or only some areas of it. Figure 1.7 shows that this pattern may present itself with different
visual characteristics such as color and width. However, the main properties of pigment network (dark
grid lines over a lighter diffuse background and the organization of these connected lines that lead to
the existence of holes) are always present and can be used for its detection. Chapters 3 and 5 explain
how these two properties are used is this work for network detection. Chapter 3 gives an introduction
to one of the main procedures used in network detection while Chapter 5 describes step by step how
the detection is performed. The second main objective is to perform a binary labeling of lesions, i.e.,
to be able to classify lesions as with or without pigment network. The work developed regarding this
second objective is be discussed in Chapter 6.
The work was developed using dermoscopy images from a database of Hospital Pedro Hispano.
As it is common with dermoscopy images, these images present a set of artifacts (hair and reflections)
that hamper the detection of pigment network. These artifacts can be seen in Figure 1.10 and needed
to be removed before a detection could be performed. The removal of these artifacts is performed in
a pre-processing step described in Chapter 4.
The results obtained with the network detection block are improved in a post-processing step.This
block checks if the regions detected have a pigmented network structure. This is done using a binary
classifier (AdaBoost [54]) trained from the data. Chapter 6 describes in detail the region validation
block.
8
(a) Hair and air bubbles
(b) Reflection
Figure 1.10: Artifacts usually found in dermoscopy images
Figure 1.11 summarizes the detection system briefly described above. It consists of five successive blocks: a pre-processing block where the hair and reflections are removed from the dermoscopy
image, a network enhancement block, a network detection block, a block of post-processing, where
the regions are validated and a final block where a binary classification of the whole lesion is performed. The chapters where each one of the blocks is discussed in detail are marked on the figure.
Figure 1.11: Block Diagram of the Detection System
Before the chapters which explain the work developed in this thesis, a review of the state of the art
is made in Chapter 2. In this chapter the dermoscopy technique as well as medical and computational
algorithms for melanoma diagnosis will be presented.
Chapter 7 presents an experimental evaluation of the proposed system using a database of 57
images. This chapter also addresses some of the decisions made in the design of the system, such as
using the best entropy channel for gray-scale conversion and AdaBoost parameters will be discussed.
The final Chapter 8 addresses the conclusions of the work performed and some proposals of work
to be done in the future.
9
1.5
Contributions
In this master thesis a new algorithm for automatic detection and classification of pigment network
in dermoscopy images (see Figure 1.11) is proposed. This algorithm includes the following original
contributions:
Pre-Processing: A method for pre-processing dermoscopy images is proposed in which the detection of reflection and hair artifacts is accomplished by using, respectively, thresholding methods
and a bank of directional filters.
Network Enhancement: This work proposes a new procedure for enhancing the lines of pigment
network. The enhancement is performed by recurring to a bank of directional filters followed by a
threshold step.
Network Detection: The new methodology proposed for network detection makes use of the geometrical properties of pigment network. The detection algorithm uses connected component analyses
to detect and extract the skeleton (the mesh) of pigment network.
Post-Processing/Region Validation: The proposed method for region validation can be seen as
a post-processing step. A binary classification of the pigment network regions detected is performed
by training a classifier (AdaBoost) with a set of features (which discriminate intensity, texture and
geometry) extracted from the detected regions.
Lesion Labeling: A method for a binary lesion classification based on a statistical approach is
also proposed.
Most of this work was published in the main conference on IEEE Biomedical Engineering Society:
C. Barata, J.S. Marques and J. Rozeira [8] - ”Detecting the pigment network in dermoscopy images: a directional approach” - in proceedings of the 33rd Annual International Conference of the
IEEE Engineering in Medicine and Biology Society (August 30th- September 3rd) at Boston (USA) .
10
2
Dermoscopy Analysis
Contents
2.1
2.2
2.3
2.4
Dermoscopy . . . . . . . . . . . . . . . . . .
Dermoscopic Criteria and Pattern Analysis
Medical procedures . . . . . . . . . . . . . .
Automatic Systems . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
12
12
13
14
11
2.1
Dermoscopy
Dermoscopy is a non-invasive diagnostic technique used to see a variety of patterns and structures in skin lesions that are not discernible to the naked eye and that can be very useful in the
characterization of skin lesions. It has several other names such as epiluminescence microscopy
(ELM), dermatoscopy or skin surface microscopy.
The technique consists in placing mineral oil, alcohol or even water on the surface of the lesion.
These fluids make epidermis more transparent to light and eliminate part of the surface reflection [1].
After placing the fluid, the lesion is inspected using a dermatoscope, a stereomicroscope, a camera
or a digital imaging system. Depending on the instrument used, the magnification of a given lesion
ranges from 6x to 40x and even up to 100x [3]. With the magnification, pigmented structures within the
epidermis, dermoepidermal junction and superficial dermis become visible. In Section 2.2 a reference
to the different structures detected is done.
Several studies have demonstrated that dermoscopy analysis improves the accuracy of melanoma
detection by dermatologists by 10-27%, when compared with a naked eye analysis [35]. However,
this technique’s performance is highly dependent on the expertise of the dermatologist as has been
shown by [11], and even trained users can be lured in some particular cases of melanomas that have
no characteristic dermoscopy findings [1].
2.2
Dermoscopic Criteria and Pattern Analysis
As was described in Section 2.1, dermoscopy magnifies a pigmented skin lesion, highlighting a set
of dermoscopic structures. The structures enhanced can be used as an input in several procedures
that are used by dermatologists to distinguish between different kinds of skins lesions (melanocytic
and non-melanocytic) as well as in the detection of melanomas. Some examples of these algorithms
are: ABCD rule [50], 7 poins-checklist [6] (both described in Section 2.3), Menzies method [36] and
pattern analysis [39].
The dermoscopic features detected can be divided in two separate groups: global and local features. Global features allow a quick, albeit preliminary categorization of a given pigmented skin lesion.
They are considered as patterns that may or may not be present in a given lesion. For example, the
Reticular Pattern (Section 1.3) is considered a global feature. Other cases of global features are:
Globular Pattern (presence of numerous variously sized, round to oval structures with different colors), Starburst Pattern (characterized by the existence of pigmented streaks in a radial arrangement
at an edge of a given pigmented lesion), and Cobblestone Pattern, which is very similar to globular but
is composed of closely aggregated, angulated globules. Local features are the ones that are usually
used when performing a diagnosis of a malignant lesion, since it is the analysis of these features
that will be considered in medical algorithms. Pigment network, dots and globules, pigmentation,
blue-whitish veil, streaks, regression and vascular structures are all considered local features. [3]
Pattern Analysis is the classic dermoscopic method for diagnosing skin lesions [1]. It evolves in
two steps. In the first one, the lesion is classified as being melanocytic or non-melanocytic. This
12
classification is performed based on the dermoscopic features described previously, namely based
on global features. A reticular pattern, a globular or cobblestone patterns and a starburst pattern are
usually identifiers of melanocytic lesions. The second step of the algorithm is to distinguish between
malignant and benign lesions, considering only the lesions that were classified as melanocytic in the
first step. To do so, an analysis of the local features and global features is performed. If a lesion
presents atypical features it is considerer as malignant. However, if the features are typical, the lesion
is benign. [39]
2.3
Medical procedures
The Pattern Analysis described previously in Section 2.2 proved itself a very useful tool for improving the clinical diagnosis of melanoma, increasing the accuracy of correct diagnosis by 10% to
30% [3]. However, it relies on subjective basis, since the assessment of dermoscopic criteria and
its classification as typical or atypical may change between dermatologists. In order to try to solve
these problems, additional diagnostic methods have been presented: ABCD rule and 7 point checklist
are two notable examples. Both of these algorithms require a preceding step, in which the lesion is
classified as melanocytic or not. This step is similar to the first step of Pattern Analysis, since it makes
use of the assessment of dermoscopic global and local features to perform the classification. In the
following subsections both these procedures will be explained.
2.3.1
ABCD Rule
For performing ABCD rule four parameters (Asymmetry, Border, Color and Differential structure)
must be assessed. Each of these characteristics is analyzed separately and each one is scored
based on how atypical they are. For instance, in asymmetry the score can be 0, 1 or 2 based in how
asymmetric the lesion is (2 is for asymmetry along two orthogonal axis and 0 is given when the lesion
is fully symmetric). After giving a score to each one of the 4 criterion (usually they are called Ascore ,
Bscore , Cscore and Dscore ), the Total Dermoscopy Score (TDS) is computed as a liner combination of
these parameters (see (2.1). Table 2.1 and expression (2.1) summarize the ABCD rule.
Table 2.1: ABCD Rule of Dermoscopy [50]
Criterion
Asymmetry
Border
Color
Differential
Structures
Description
In 0,1 or 2 axes; assess not only contour, but
also colors and structures.
Abrupt ending of pigment pattern at the periphery, in 0-8 segments.
Presence of up to 6 colors 1-6 (white,
red, light-brown, dark-brown, blue-gray and
black).
Presence of network, structureless or homogeneous areas,streaks, dots and globules.
Score
0-2
Weight
1.3
0-8
0.1
1-6
0.5
1-5
0.5
T DS = 1.3Ascore + 0.1Bscore + 0.5Cscore + 0.5Dscore
(2.1)
13
After computing the TDS score, the diagnosis is made based on its value. If TDS for a given lesion
is higher than 5.45, there is a high probability that it might be a melanoma. For TDS values below
4.75, the lesion is considered as being benign. The lesions that have TDS values between the two
referred previously, are considered suspicious and a close follow-up is advised.
2.3.2
7 Point-checklist
7 point-checklist is an alternative medical algorithm for detecting melanomas. It requires only the
identification of 7 atypical local features to perform a diagnosis. The 7 dermoscopic criteria assessed
are divided in two classes: major and minor criteria, based on the capacity of each criterion to increase
the probability of a melanoma diagnosis. Each feature present in a lesion receives a score. If it is a
major feature it receives a score of 2 and if it is a minor criteria a score of 1. In the end, a sum of the
individual scores is computed. A lesion is classified as being a melanoma if the final score is higher
than 3 [6]. Table 2.2 shows the 7 features that can be detected and their score.
Table 2.2: 7 Point-Checklist: Criterion and Scores
Local Features
Major Criteria
1.Atypical Pigment Network
2.Blue-Whitish Veil
3.Atypical Vascular Pattern
Minor Criteria
4.Irregular Streaks
5.Irregular pigmentation
6.Irregular dots/globules
7.Regression structures
2.4
Score
2
2
2
1
1
1
1
Automatic Systems
The diagnosis of a skin lesion by one of the previous medical methodologies is a subjective one,
since it depends on human vision and on the clinical experience of the dermatologist, lacking accuracy
and reproducibility. Moreover, its accuracy is highly dependent on the expertise of the dermatologist
[11].
Computerized dermoscopy image analyses systems do not have the limitation of subjectivity.
These systems consider that a computer can be used as a second independent diagnostic method,
which can be used by non-experienced operators to perform a preliminary evaluation of a clinical
case and that can also be used to help dermatologists. They can also be used to improve biopsy
decision-making and follow-up. [28]
Several systems for lesion classification, and ultimately for melanoma detection, have been proposed by different groups [12, 15, 23, 30, 38, 41, 43, 45]. They will be described briefly in the following
subsection. There are other systems which try to incorporate detailed medical knowledge and detect the structures which characterize a melanocytic lesion (see Section 2.2). These systems will be
14
described in Subsection 2.4.2.
2.4.1
Automatic Systems for Melanoma detection
CAD systems can be found in literature as early as 1987 [14]. These methods usually start by
performing an automatic segmentation of the lesion, which by itself is already a challenge, because
of the great variety of lesion shapes, sizes and colors, along with different skin types and textures. A
good lesion segmentation is required, since the accuracy of the subsequent steps of the CAD system
are affected by it [44].
The following step is feature extraction. The features extracted are selected based on the ABCD
rule. Therefore, the CAD systems usually uses features that describe shape (asymmetry and border),
color and texture.
The final step is to perform a binary lesion classification (malignant or not). This is done by
recurring to learning algorithms (artificial neural networks (ANN), support vector machines (SVM),
boosting). Some systems have an intermediary step between feature extraction and classification, in
which they reduce the size of the feature space by eliminating redundant, irrelevant or noisy features.
In recent years very promising results have been obtained by some of these systems. Ganster
et al. [23] achieved a sensitivity (SE) (melanoma detection accuracy) of 73% and a specificity (SP)
(benign detection accuracy) of 89%. They used a large dataset of more than 5000 images of which
only 2% were melanomas and their results were obtained by training a k nearest neighbor (kNN)
classifier. Rubegni et al. [41] proposed a classification system using an ANN. Their results were quite
good, achieving a SE of 94.3% and a SP of 93.8% on a dataset of 550 images, 200 of which were
cases of melanoma. The work developed by Blum et al. [12] reported a SE of 83.3% and a SP of
86.9% on 753 cases of nevi and 84 cases of melanoma, with a logistic regression model. By using
SVM and a dataset of 564 images, containing 88 cases of melanoma, Celebi et al. [15] achieved a
SE of 93.3% and a SP of 92.3%. In 2008 Iyatomi et al. [30] improved their Internet-based melanoma
screening system that was developed in 2004 [38], increasing their former SE and SP values, so there
is currently one automatic system available on the net. Table 2.3 summarizes the information related
with the different CAD systems described in this paragraph.
Despite the promising results, the approaches resumed above have some issues. First of all they
do not try to incorporate the medical knowledge. They are based on a machine learning paradigm:
if features are rich enough and if there is a (very) large database of lesions classified by experts,
inference methods will be able to learn the classification rule [18]. This can affect the results obtained
by the different CAD systems. Another problem is related with comparison between different methods,
since each group uses a different dataset it is not possible to determine which system actually gives
the best results.
15
Table 2.3: Summary of recent CAD systems. Mel. stands for Melanoma.
Authors
Ganster et al. [23]
Rubegni et al. [41]
Blum et al. [12]
Oka et al. [38]
Celebi et al. [15]
Iyatomii et al. [30]
2.4.2
Classifier
kNN
ANN
Logistic Regression
ANN
SVM
ANN
Images
5363
550
837
247
564
1258
Mel.
107
200
84
59
90
198
SE (%)
73.0
94.3
83.3
87.0
93.3
85.9
SP (%)
89.0
93.8
86.9
93.1
92.3
86.0
Automatic Systems for Dermoscopic Features Extraction
The dermoscopic or medical features detection and extraction is a relatively unexplored area. Nevertheless there are already some remarkable works in which the detection of dermoscopic features
used by dermatologists such as asymmetry and border, color and differential structures (dots, pigment network and blue-white veil) is accomplished [5, 10, 13, 16, 19, 20, 26, 34, 40, 42, 47–49, 55]. By
detecting these kind of features it is possible to incorporate detailed medical knowledge in the CAD
systems.
Blotches are irregular spots or discolorations in a skin lesion. Two interesting studies can be found
on literature that focus on the detection of this medical feature, one proposed by Pellacani et al. [40]
where they try to automatically extract dark areas in melanocytic lesions, and another one proposed
by Stoeker et al. [47], which is more focused on the detection of asymmetric blotches. Recently, the
detection of the presence of specific color regions such as blue-white veil [16, 19] or regression structures (white areas, where loss of pigmentation has occurred) [19] have been developed. Interesting
works have also been proposed for the detection of asymmetry [48] and extraction of the degree of
border irregularity [34].
Differential structures have also received some attention from different groups. By using a morphological closing operation Yoshino et al. [55] developed an automatic system for dots extraction.
Their automatic system had some problems since it could also detect the outline of pigment network.
Nevertheless their results are very promising. An algorithm for globules analysis had already been
proposed by Fleming et al. [20]. More recently, a work which aimed to detect granularity in dermoscopy images of malignant melanomas was proposed. They detected this pattern by using color
and texture features to train a back-propagation neural network [49]. The detection of streaks has
also received some attention and an algorithm for the automatic detection of irregular streaks has
been proposed by Fabbrocini et al. [19]. This algorithm achieved a SE of 86% and a SP of 88%.
The main aim of this thesis is to develop an automatic algorithm for pigment network detection.
This problem was ignored for a long time in the literature. However, several works have been recently
published [5, 10, 13, 20, 26, 42]. Each one of these detection systems proposes a different approach
for performing the detection of pigment network in dermoscopy images. Their final result is usually a
binary classification of a lesion as with or without pigment network. This is very useful for performing
a classification of a lesion as melanocytic or not. However, for dermatologists the structure of pigment
network (the mesh of dark lines) is also very important, since they use it to classify the pigment
16
network as typical or atypical and, consequently, to diagnose a lesion as benign or malignant. Most of
the works found in literature do not extract the skeleton of pigment network. Two notable exceptions
can be found in the works of Fleming et al. [20] and Grana et al. [26]. This thesis tries to overcome
this issue and proposes a new method for detecting the pigment network in dermoscopy images
based on color and geometry, which is also capable of extracting the dark lines of pigment network.
The system proposed can still perform a lesion classification regarding the presence or absence of
pigment network. The detection algorithm proposed is then a new approach to the problem of pigment
network detection.
17
18
3
Directional Filters
Contents
3.1 Directional Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Directional Filter Bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
22
19
One of the main objectives of this thesis is to detect and extract pigment network. Another objective
is the removal of hair artifacts which can occlude important regions of the lesions to be studied. Both
these structures contain linear strokes, with highly directional shapes. Therefore, filters which are able
to detect features in a specific orientation can be used to detect and enhance them. The directional
filters proposed in this thesis belong to this group of filters so they have the correct properties and are
capable of enhancing pigment network and hairs.
In this chapter the bank of directional filters used to sharp both pigment networks and hairs will be
presented. An introduction to the basis of directional filtering will also be made, in order to provide a
better understanding of the main reasons to choose these kind of filters.
3.1
Directional Filters
Directional filters have been proposed in different areas . One of them is a biology related area
and the other one is the signal processing area, namely processing signals with noise.
The biology behind directional filters is related with the human perception of vision. Visual perception can be defined as the psychological manifestation of visual information. This information comes
from visible light that is detected by the photoreceptors of retina and the processing of that information occurs in the visual cortex. The transmission of information between the eyes and the brain is
performed first by the optic nerve and then by the optic tract, which ends at the thalamus. It is the
thalamus that is responsible for relaying the visual image to the visual cortex for processing. The path
described previously forms the sensory system of the human visual system. Each one of the components of the sensory system possesses a receptive field, which can be defined as the region of space
to which a receptor organ and receptor cells respond. For instance, the receptive field of an eye is the
part of the world that it can see, i.e., the light that each photoreceptor receives. The receptive fields
in visual cortex of mammals have been studied by Hubel and Wiesel [29]. They found and classified
three distinct receptive fields of cells in the visual cortex: simple, complex and hypercomplex cells.
Their initial studies also concluded that the majority of neurons in the primary visual cortex respond
to a line or edge of a certain orientation in a given position of the visual field, they are directional. Between the three different kinds of neurons, two of them were defined as orientation-selective neurons,
one that was sensitive to the contrast polarity of lines and edges, the simple cells, and other that was
not, the complex cells. [29]
Several computational models have been proposed to simulate the functioning of these neurons.
One of the more interesting works was developed by Jones and Palmer [31]. They proposed that the
functioning of simple cells could be modeled by a family of 2D Gabor Filters. Their work proved that
simple cells in mammal visual cortex perform a linear and directional filtering. 2D Gabor filters have
been proposed by Daugman [17] as a 2D generalization of the family of linear filters described by
Gabor [22], and each filter has the following impulse response
gλ,φ,γ,θi (x, y) = C exp {
x02 + γ 2 y 02
x0
}
cos
(2π
+ φ)
2σ 2
λ
where C is a normalization constant, λ is the wavelength and
20
1
λ
(3.1)
the spatial frequency of the cosine
factor and γ is the spacial aspect ratio, that determines the ellipticity of the mask. The parameter
φ ∈ [−π, π] is a phase offset that determines the symmetry of gλ,φ,γ,θi (x, y) with respect to the origin
[17]. The values of (x0 , y 0 ) are related with (x, y) as follows
x0 = x cos θi + y sin θi
(3.2a)
y 0 = y cos θi − x sin θi
(3.2b)
where θi defines the direction of the filter.
These filters have been used with good results in texture discriminator [21, 52] and edge detection
algorithms [27]. Usually a bank of Gabor filters with different orientations θi is used and for each
direction the original image I is convoluted twice, with a pair of filters with a phase difference of
π
2
rλ,0,γ (x, y) = gλ,0,γ,θi (x, y) ∗ I
(3.3a)
rλ,− π2 ,γ (x, y) = gλ,− π2 ,γ,θi (x, y) ∗ I
(3.3b)
The final response for each direction is a combination of the information provided by the pair of filters
Eλ,γ,θi =
q
2
2
rλ,0,γ
(x, y) + rλ,−
π
,γ (x, y)
2
(3.4)
The use of Gabor filters as models for simple cells which perform a directional filtering suggested that
this family of filters could be used to enhance highly directional structures such as pigment network
lines and hairs. As will be shown in the next section the filters used have an impulse response
which has the same gaussian base as Gabor filters and the bank is also composed of filters with
different orientations. However, the cosine term is not used and the impulse response is a difference
of gaussians.
Detection theory [53] can also be used to explain the use of directional filters. Based on the
methods used to detect signals in noise it is possible to determine the ideal detector.
The general detection problem consists in checking if a pattern IN (x, y), that in this case can be
pigment network or hair artifacts, is present in a given signal I(x, y) (the dermoscopy image). Two
hypotheses are formulated regarding this problem
H0 : I(x, y) = w(x, y) x ∈ X, y ∈ Y
(3.5a)
H1 : I(x, y) = IN (x, y) + w(x, y) x ∈ X, y ∈ Y
(3.5b)
where w(x, y) is additive noise.
The optimal solution to this problem consists of filtering the image I(x, y)
J(x, y) = I(x, y) ∗ h(x, y)
(3.6)
with a linear filter for which the impulse response is
h(x, y) = IN (−x, −y)
(3.7)
where h(x, y) is designated by matched filter and has the shape of the signal that one wants to detect
(apart from a spatial inversion) [51]. This theory suggested that in order to detect the skeleton of
21
pigment network and hairs, directional filters should be used. Therefore, it was decided that the best
approach should be to design a filter by recurring to gaussians equations, as will be shown in the next
section.
3.2
Directional Filter Bank
In this section it will be shown how the properties of an ideal detector discussed previously were
used in order to develop a bank of directional filters capable of detecting both pigment network and
hairs. Figure 3.1 summarizes the functioning of the directional filters bank.
Figure 3.1: Directional Filters Bank. The inpute I and all the outputs Ii are images
The analysis bank is composed of several directional filters each of them tuned to a specific
orientation θi ∈ [0, π], i = 0...N . The filters were designed this way because the linear structures that
compose the artifacts and pigment network’s dark lines not only have an unknown direction but may
appear in a dermoscopy image with more than one orientation as well.
Each one of the filters from the analysis block has the impulse response
hθi (x, y) = G1 (x, y) − G2 (x, y)
(3.8)
where Gk is a Gaussian filter:
Gk (x, y) = Ck exp {−
y 02
x02
− 2 }, k = 1, 2
2
2σxk
2σyk
(3.9)
In (3.9) Ck is a normalization constant and the values of (x0 , y 0 ) are related with (x, y) by a rotation
of amplitude θi
22
x0 = x cos θi + y sin θi
(3.10a)
y 0 = y cos θi − x sin θi
(3.10b)
The values for the parameters σxk and σyk are chosen in such a way that the second filter is highly
directional and the first one is less directional or even isotropic. A difference of gaussians was chosen
since this design allows a better enhancement of directional structures while removing the effect of
the background. The image I is decomposed in N + 1 filtered images Ii by using all the directional
filters
Ii = hθi ∗ I
(3.11)
Figure 3.2 shows the filter bank designed for pigment network enhancement. The orientation of
the filters ranges between 0 and π and in this image the differences between each impulse response
are easily detected. It is possible to see that, as expected, all the impulse responses have a linear
shape. This makes them good matched filters for the detection of pigment network and hairs.
θ = 0º
θ = 10º
θ = 20º
θ = 30º
θ = 40º
θ = 50º
θ = 60º
θ = 70º
θ = 80º
θ = 90º
θ = 100º
θ = 110º
θ = 120º
θ = 130º
θ = 140º
θ = 150º
θ = 160º
θ = 170º
θ = 180º
Figure 3.2: Impulse responses of 19 directional filters used in the pigment network’s enhancement block.
The output of the filters bank is composed of N + 1 filtered images Ii . To synthesize the final
image J a simple maximization is performed at each pixel
J(x, y) = max Ii (x, y)
i
(3.12)
An example of the final output can be seen in Figure 3.3.
23
Figure 3.3: Final output of the directional filters bank.
When compared with the original image from Figure 1.7(a) it is possible to see that this bank of
directional filters performs a good pigment network and hair enhancement.
24
4
Pre-Processing
Contents
4.1
4.2
4.3
4.4
Gray Scale Conversion
Reflection Detection .
Hair Detection . . . . .
Inpainting . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
26
27
29
31
25
The dermoscopy images used in computer-aided diagnosis often display artifacts (e.g., hair, reflections) produced during the acquisition process. These artifacts can interfere with many analytic
procedures required for accurate diagnosis such as border detection and dermoscopic criterion extraction (see Figure 1.10). Therefore, a pre-processing step in which these artifacts are detected and
removed is required in order to ensure that the results obtained by the network detection algorithm
will not be compromised by them.
The scheme of Figure 4.1 presents in detail the operation of the pre-processing block of the detection system.
Figure 4.1: Block Diagram of the Pre-Processing step
In the following section each block of the previous figure will be analyzed and explained in detail.
4.1
Gray Scale Conversion
The dermoscopy images used in this work are RGB images stored in bitmap and jpeg formats. In
order to perform the network detection these images are converted into 8-bit gray scale images.
There are two ways for performing this conversion. The first is done by applying the linear combination of RGB values
Y = 0.299R + 0.578G + 0.114B
(4.1)
where Y is the 8-bit gray scale image and R, G and B are the red, green and blue color components,
respectively. [24]
Another way is by selecting the color channel with the highest value of entropy, since usually
this channel is the one which allows the best discrimination in most dermoscopy images [44]. The
selection of the color channel is an automatic process. First the entropy of each one of the color
components i is computed
S(i) = −
L−1
X
hi (k) log [hi (k)]
(4.2)
k=0
where hi (k) is the histogram of the color component i. This histogram is computed using L = 256
bins, since each color channel ranges from 0, ..., 255. After computing the entropy values, the color
plane with the highest one is selected:
i∗ = argmax Si
(4.3)
i
Section 7.5 shows the results for region classification after the post-processing step described in
Chapter 6 for the two methods for gray-scale conversion.
26
4.2
Reflection Detection
Reflections appear in dermoscopy images as a result of the presence of air bubbles in the immersion oil at the interface between the skin and the face plate of the dermatoscope or other acquisition
system. This kind of artifacts may appear around a lesion as in Figure 1.10(b) but they can also be
seen on top of the lesion, as is the case of image 1.10(a). The presence of reflections may influence
a correct detection of the border of the lesion as well as interfere with the main objective of this work,
since sometimes air bubbles can be on top of pigment network and so it will compromise its detection.
An algorithm for air bubbles removal has been proposed by Fleming et al.. Their algorithm was
based on two distinctive characteristics of air bubbles: the presence of speckles in their interiors and
the presence of a strong, bright edge in their boundaries. For performing the air bubbles detection they
used top hat filters for speckles segmentation and a radial search (having as starting point the centroid
of the speckle) combined with an outlier rejection procedure in order to improve the segmentation. In
the end the contours detected were filled and a air bubble mask was obtained [20].
In this thesis a simpler approach is proposed. After a brief study, it was concluded that pixels that
belong to regions classified as reflection have higher values of intensity than other pixels in the same
image, and that this value is usually above a certain threshold. Figures 4.2(c) and 4.2(d) show the
intensity profile along the green line draw in the two above pictures. It is possible to see that in the
positions that correspond to the edges of the reflection regions the pixels’ intensity values are very
high and that inside those regions the values are high as well. Another study showed that the pixels
that belong to reflection regions also have intensity values that diverge substantially from the average
intensity Iavg computed in the neighborhood of the pixel, as can be seen in Figures 4.2(e) and 4.2(f),
where it is shown the deviation from the average intensity along the two green lines. Both the intensity
profiles were determined for gray-scale images.
These examples suggest that reflection can be detected by comparing if the pixel intensity is high
and well above the average intensity in its neighborhood, i.e.,
I(x, y) > TR1 ∧ I(x, y) − Iavg (x, y) > TR2
(4.4)
where I is the gray scale image, Iavg is the average intensity in a local neighborhood of the pixel and
Tr1 , Tr2 are both threshold values.
Experimentally it was determined that the values of TR1 , TR2 that gave the best results were:
• TR1 = 180
• TR2 = 25
these values allow a good relationship between the correctly and wrongly detected regions.
27
(a)
(b)
250
220
200
180
200
Intensity
Intensity
160
150
140
120
100
100
80
60
50
0
100
200
300
400
500
600
700
40
800
0
100
200
300
90
90
80
80
70
70
60
50
40
30
300
400
(e)
500
600
700
800
30
10
200
800
40
20
100
700
50
10
0
600
60
20
0
500
(d)
Deviation from average intensity
Deviation from average intensity
(c)
400
500
600
700
800
0
0
100
200
300
400
(f)
Figure 4.2: Intensity studies in Reflection regions. Figures 4.2(a) and 4.2(b) are the original images with a green
line showing the direction along which the intensity profile study and the deviation study were performed. Figures
4.2(c) and 4.2(d) are the intensity profiles corresponding to the figures that are above them. Figures 4.2(e) and
4.2(f) are the same as the previous two, but represent the deviation from average intensity.
Figure 4.3 exemplifies the application of this algorithm to images 4.2(a) and 4.2(b).
28
(a)
(b)
(c)
Figure 4.3: Reflection detection in dermoscopy images
4.3
Hair Detection
Hair artifacts are long, dark and linear structures. Their presence in dermoscopy images usually
occludes some information of the lesion such as its border and texture (dermoscopic features), as
can be seen in Figures 1.10 and 4.4. Therefore, the removal of this kind of artifacts is an important
step before a lesion segmentation or a feature extraction process can take place.
(a)
(b)
Figure 4.4: Presence of Hair artifacts in dermoscopy images
A few methods for hair detection and removal have already been developed and are resumed in a
paper written by Abbas et al. [4]. They describe that most of the hair detection methods developed in
the past were based on the fact that hair pixels are much darker than skin or tumor area. However,
this principle is not always true, as can be seen in Figure 4.2(a) where the hair artifacts have a color
quite similar to the one of the lesion. The proposed methods used detection techniques such as
morphological closing operation, which is limited because the masks applied have only a discrete
number of orientations, thresholding and 2-D Derivative of Gaussian. This last technique was used
by Abbas [4] to propose a new hair detection method and brings some differences to the previous
methods since it also considers the thickness of the hair, their length and magnitude and increases
the number of orientations.
Hair artifacts are highly directional structures with a linear shape. The algorithm proposed in this
thesis for hair detection is different from the ones found in literature. First to enhance this structure
a bank of directional filters is used. The filter impulse responses are defined by (3.8) and (3.9) (see
29
Section 3.2). The algorithm uses not only the color information but also the characteristic shape of
hair (thin and linear structures). These informations can be used to define the best parameters for
(3.8) and (3.9), in order do provide good matched filters. Also, by using a difference of gaussians
as impulse response, hairs that have colors similar to lesions can be detected more easily, without
compromising the lesion.
The hair detection block runs in three different steps, as can be seen in Figure 4.5.
Figure 4.5: Block Diagram of the Hair detection system
First, the original image is filtered with a bank of directional filters. Hairs are long and thin structures, therefore it was decided that the impulse responses should be highly directional, as can be
seen by the parameters used for (3.8) and (3.9)
• σx1 = 20 and σy1 = 6
• σx2 = 20 and σy2 = 0.5
The filters’ mask size is 41x41. The step between θi and θi+1 should be small, since hair orientation
does not change sharply. Therefore, it was determined experimentally that a good value for N , the
number of rotations, should be N = 64. These filters enhance linear structures (hair artifacts) present
in the image.
The next step is to enforce a thresholding condition in order to perform a binary classification of
each one of the pixels from the filters response J
J(x, y) > TH
(4.5)
Each one of the pixels (x, y) that is above the threshold is classified as hair. The threshold value
is TH = 0.06. This parameter and the ones from directional filters were determined empirically, and
are the ones which allow a better global hair detection, without detecting pigment network as well.
The morphological operation applied to the output of the thresholding algorithm is a dilation. The
gray scale dilatation of A(x, y) by a structuring element B(x, y) is defined by the following expression:
(A ⊕ B)(x, y) = max{A(x − x0 , y − y 0 ) + B(x0 , y 0 ) | (x0 , y 0 ) ∈ DB }
(4.6)
where A(x, y) is the output of the thresholding and DB is the domain of the structuring element
B(x, y) [25], that in this case was the following matrix:

1

B = 1
1
1
1
1

1

1
1
Using a morphological operation such as dilation in the last step allowed a simple dilation of hair
lines. This dilation is performed in order to secure that hair lines are entirely removed. This morphological operation will increase the performance of the artifacts removal and inpaiting step that will be
described in the next section.
30
Figure4.6 shows the result of applying the hair detection algorithm to the 3 images of Figures
1.10 and 4.4. It is possible to see that this algorithm performs reasonably well, achieving a good hair
detection. Nevertheless it still has some wrongly detected regions as can be seen in image 4.6(c) and
also misses some hair artifacts (see image 4.6(b)).
(a)
(b)
(c)
Figure 4.6: Hair detection in dermoscopy images
4.4
Inpainting
After applying the detection algorithms for the two kinds of artifacts described previously, the
masks obtained are used to remove both of the artifacts from the original gray scale image, as can
be seen in the scheme of Figure 4.1, and replacing them by zeros (see Figure 4.7). This removal
resulted in the appearance of dark regions without information, as can be seen in Figure 4.7.
(a)
(b)
(c)
Figure 4.7: Gray Scale images after artifacts removal
These regions must be filled, since they are darker than their surroundings and sometimes have
a linear shape, which will interfere with the directional filters used in the network enhancement step.
Abbas et al. [4] report in their paper that there are three main alternatives to perform hair removal
after the detection. Since the main principle is the same, each one of these alternatives can also
be applied in this work to remove the artifacts detected. The first one is performed by replacing the
hair artifacts by bilinear interpolation, followed by a smoothing of the final result using a median filter.
However, this operation often leads to undesirable blurring and color bleeding, so it can disturb the
tumor’s texture and so should not be considered . An example of this kind of hair removal method is
the DullRazor software developed by Lee et al. [33].
31
The second case is performing a hair removal by non-linear PDE-based diffusion methods. These
methods are also called image inpainting [9] and have some advantages over the bilinear interpolation methods because they use the neighboring information in a natural manner through non-linear
diffusion, filling large gaps in an image while maintaining a sharp boundary [4]. However, image inpainting algorithms can introduce some blurring and have some problems when they try to reproduce
large textured regions because the inpainting algorithm does not synthesis textures. Therefore these
methods should not be used if large areas of a lesion need to be reconstructed, since it will disrupt
the lesion texture.
The third alternative is based in an inpaintg method as well, but in this case it combines the PDEdiffusion with texture synthesis techniques. This kind of inpainting is called exemplar-based and is
more difficult to use since the parameters required by this algorithm are hard to determine in general.
This algorithms also requires a larger number of iterations than the previous one, which makes it more
slow and with a higher computational complexity. Therefore, the exemplar-based inpainting method
does not provide, in practice, an effective solution for hair and other artifacts removal. [4]
The technique chosen to replace the hair and reflection artifacts was the PDE-based inpainting.
In this method a region Ω that has to be inpainted will be fulfilled progressively by the propagation
of lines, called isotopes, that arrive at its boundary ∂Ω. Each time the lines are propagated the
different regions inside Ω that were created by them are then filled with color, matching those of ∂Ω.
By performing this two steps the region Ω will progressively shrink in a smooth way. In Figure 4.8
it is possible to see how the isotopes are progressively prolongated (the dashed lines represent the
~ ). [9]
successive prolongations) and the direction of drawing (represented by the vectors N
Figure 4.8: Propagation direction as the normal to the signed distance to the boundary of the region to be
inpainted [9]
In appendix A a mathematical description of the inpainting algorithm developed by [9] and used in
this thesis will be made.
4.4.1
Inpainting Results
Figure 4.9 shows some examples of applying the inpainting algorithm to some images of the
dataset. These images are reconstructions of the ones from Figure 4.7 and if a comparison is performed it can be seen that the inpaiting algorithm performs reasonably well. Nevertheless this algo-
32
(a)
(b)
(c)
Figure 4.9: Gray Scale images after inpaiting algorithm
rithm does not perform a perfect removal of artifacts and does not propagate textures, only intensities
as can be seen in the three images, so in the future other algorithms should be tested and the results
compared with the ones obtained in this work.
33
34
5
Network Detection
Contents
5.1
5.2
5.3
5.4
Previous works . . . .
Network Enhancement
Network Detection . . .
Examples . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
36
38
39
41
35
The following step after the pre-processing block is to enhance and detect pigment network. This
Chapter will be divided in two main sections: one in which it will be described how the network dark
lines were sharpened and a second section where the actual network detection will be performed. In
the scheme of Figure 5.4 the main operations of the network enhancement and detection blocks are
presented in detail.
Figure 5.1: Network Enhancement and Detection blocks
5.1
Previous works
The detection of pigment network was ignored for a long time in literature. However, several works
have been recently published.
Fleming et al. [20] proposed a set of techniques for extracting and measuring pigment network’s
characteristics such as its thickness and the variability of thickness of network lines and the size and
variability of network holes. The method used for network detection was based on an algorithm for
detecting lines (curvilinear structures) proposed by Steger [46]. In this algorithm the lines are regarded
as ridges. The preliminary step of the algorithm is to find the ridge points, i.e., the line centers and
those are identified by recurring to the first and second derivatives values in the direction orthogonal
to the line. A vanishing first derivative and a high second derivative value are the identifiers. After
the detection of the ridge points, the second step is to link them and form lines. This is done by
recurring to information obtained in the previous step such as second derivative values, orientation
and proximity. Grana et al. [26] proposed a similar algorithm for network detection in their of network’s
texture characterization. However, their proposal for line linking is to use a set of morphological
masks to find the line ending points, followed by a extension that begins at each termination found.
An interesting aspect of their work is that they try to distinguish between different kinds of lesions
with pigment network. They classify a lesion as complete network and partial network regarding the
spatial distribution of pigment network within the lesion. Their algorithm achieves and overall network
detection performance of 88.3%.
Two algorithms for network detection were proposed by Anantha et al. [5]. The first one uses a
statistics over neighboring gray-level dependence matrices and the second method involves filtering
with Laws energy masks. The masks used have different dimensions and the output obtained by
applying each one of the masks is squared. In the end a color filtering block is applied to perform the
network detection. Their algorithm achieves an accuracy of 80%.
36
Betta et al. [10] work’s main objective was to the detect the presence of atypical pigment network
in a lesion. To achieve that aim they developed an algorithm which combines structural and spectral
information. The structural technique uses a median filter followed by a close-opening operation. The
combination of these two processes allows the detection of lines and/or points. The spectral method
makes use of a Fourier analysis (Fast Fourier Transform, high-pass filtering, Inverse Fast Fourier
Transform and thresholding) to determine the spacial period of the texture and, as a consequence,
the identification of regions where pigment network exists. In the end the two masks obtained are
combined with a segmentation of the lesion, resulting in one image where areas that belong to the
lesion and posses pigment network are highlighted. In the detection process they appear to achieve a
SE = 50% and a SP = 100%. Those areas are then screened for the presence or absence of atypical
pigment network.
Sadeghi et al. [42] algorithm for network detection is based on two main features of pigment
network: its color (dark lines over a diffuse background) and spatial characteristics. To detect the
sudden transitions in intensity they used a LoG filter. This allows the enhancement of the potential
meshes of the pigment network. After, they perform an analysis based on the spatial characteristics,
transforming the filtered image in a set of graphs, using a 8-connected neighboring. They consider
that the presence of round or cyclic structures in a lesion represent the presence of pigment network
so, after obtaining the set of graphs, they apply the Iterative Loop Counting Algorithm [32] which
allows the detection of all the cyclic subgraphs by transforming the network into a tree and performing
a dept first search on the tree for loops. In the end the subgraphs are filtered and noise, artifacts and
wrongly detected structures (such as dots and globules) are removed and a graph of pigment network
is presented. Their algorithm has an accuracy of 94.3%.
Pigment network is considered an hallmark of melanocytic lesions and is one of the features
used by dermatologists for performing a diagnosis (see Sections 2.2 and 2.3). When inspecting a
lesion which possesses pigment network, experts usually assess the structure of the network, i.e.,
the thickness of the lines and the size of the ”holes”, and classify it as typical or atypical. Therefore,
the development of an automatic algorithm which is capable of detecting and extracting the skeleton
of pigment network could be very useful for dermatologists. Most of the network detection systems
described previously do not try to extract the skeleton of pigment network (the dark lines), they only
intend to determine if pigment network is present or absent and the regions of the lesion where it
can be found. Nevertheless there are two remarkable exceptions [20, 26]. The detection system
proposed is capable not only of detecting the presence of pigment network but also of extracting its
skeleton. A different approach for detecting pigment network is also proposed in this thesis. The main
difference regarding previous works is that color and shape (dark lines over a diffuse background)
are not the only characteristics of pigment network that will be used to perform its detection. The
spacial organization of pigment network is also an important feature that has only been exploited by
Sadeghi et al. [42] and that will be used in this work to extract the skeleton of pigment network, by
recurring to a connected component analysis. Prior to the detection and extraction of pigment network
an enhancement step will be performed. This step uses the bank of directional filters described in
37
Chapter 3, which is also a new proposal for network enhancement.
5.2
Network Enhancement
The dark lines that compose the mesh of pigment network are ones of its most common characteristics. Therefore, these lines can be used to perform a preliminary separation between pigment
network and other structures of the skin.
The network enhancement step occurs after the pre-processing block where hair, reflections and
air bubbles have been removed. The main operations that occur in the network enhancement block
will be presented in the next two subsections.
5.2.1
Directional Filters Bank
The dark lines of pigment network are linear structures. To enhance these linear structures, the
bank of directional filters described in Chapter 3 is used (see (3.8) and (3.9)) with the following parameters
• σx1 = 40 and σy1 = 40
• σx2 = 3 and σy2 = 0.5
and each filter mask has the dimensions 11x11. The number of rotations for θi is N = 18, which
means that there are 19 directional filters in the analysis bank. These parameters were all determined
experimentally and are the ones which gave the best global results for the dataset. When comparing
these values with the ones used in the hair detection block (Section 4.3) it is possible to conclude that
they are very different. This happens because the length of each line stroke in the pigment network
is much smaller than the case of the hair artifacts.
Figure 5.2 show a gray scale dermoscopy image (left) and the output of the directional filters bank
(right)
(a) Original Gray-Scale Image
(b) Filter Response
Figure 5.2: Output for the directional filters bank
38
5.2.2
Thresholding
As can be seen in image 5.2(b) the output of the directional filters bank still presents some information without interest such as noise and artifacts. By applying a simple thresholding step it is
possible to reduce the amount of information, keeping only the importante one. Being If the filter
response, the thresholding step operates in the following way:
If (x, y) > TN
(5.1)
where TN is a threshold value. Only the pixels (x, y) above the threshold value are validated as
belonging to the pigment network.
It was experimentally determined that the best value for TN is 0.0185. By applying this threshold
to the image 5.2(b) was possible to obtain the binary Figure 5.3, where a great amount of noise was
removed.
Figure 5.3: Output of the thresholding step.
5.3
Network Detection
The previous step independently classifies each pixel, without taking neighbor pixels into account,
and does not consider the topological relationships among pixels of the pigmented network. In this
block the network geometry will be considered. Now, it will be assumed that the pigment network is
a set of connected regions, so by performing a connected component analysis over the binary image
obtained in the previous block, the connected dark lines can be extracted.
Connected component analysis works by scanning an image, pixel-by-pixel, in order to identify the
connected pixel regions. A region A (a group of pixels) is said to be connected if for any two pixels
p1 , p2 ∈ A, there is a path that links these two pixels and that belongs to A. The path is composed of
a sequence of neighbor pixels which starts in p1 and ends in p2 . The detection of connected regions
depends on the connectivity or neighborhood used. The two most common neighborhoods are 4connectivity (4 adjacent pixels) and 8-connectivity (8 adjacent pixels). In this work a 8-connectivity
was used because it considers not only vertical and horizontal links but also the ones along the
diagonals. This allows the detection of larger regions and the linking of some dark lines that were not
39
going to be linked otherwise. The qualitative comparison between these two ways of assessing the
connectivity of a pixel can be seen in Section 7.3. The 8-connectivity or 8-neighbors of a pixel p(x, y)
is defined as:
N8 (p) = {(x + 1, y), (x − 1, y), (x, y + 1), (x, y − 1), (x + 1, y + 1), (x + 1, y − 1), (x − 1, y + 1), (x − 1, y − 1)}
(5.2)
Of all the connected regions detected by the connected component analysis only the ones which
have areas bigger than a threshold are selected as being part of the pigment network. Let Ri be the
i − th connected region. The area criteria that the region must fulfill is
(5.3)
A(Ri ) > Amin
where A(Ri ) denotes the area of the region Ri and Amin is the threshold. This threshold was determined experimentally and it is Amin = 700 pixels. By enforcing this conditions it is possible to exclude
connected components with small areas.
The final pigment network region R is computed as the union of all the connected components
which met condition (5.3)
R=
∪
i:A(Ri )>Amin
Ri
(5.4)
The final result for the network detection block can be seen in Figure 5.4. This is the output for
image 5.2(a) and if this image is compared with Figures 5.3 and 5.2(b) it is possible to see that these
two final steps allow the removal of the incorrect data that remained after the thresholding step.
Figure 5.4: Output of the network detection block.
40
5.4
Examples
For a better visualization of the final output of the network enhancement and detection blocks
some examples will be presented this section. Figure 5.5 shows two examples for which the detection
algorithm performed well and Figure 5.6 shows two detection errors. In this last figure the pattern
present in the lesion is the so called cobblestone pattern, which is composed of dark circular structures (see Section 2.2). The algorithm detects the boundaries of the circular structures which form a
connected component.
(a)
(b)
(c)
(d)
Figure 5.5: Detection of Pigment Network: original images(left) and pigmented network (right)
The detection algorithm tends to detect regions in excess. This is a good characteristic since the
number of detection failures (pigment network regions which are not detected) is very small. However,
there are many false alarms as well.
41
(a)
(b)
(c)
(d)
Figure 5.6: Detection errors: original image(left) and pigmented network (right)
42
6
Region Validation and Lesion
Classification
Contents
6.1
6.2
6.3
6.4
6.5
6.6
Region and Lesion Ground Truths
Background Extraction . . . . . . .
Region Labeling . . . . . . . . . . .
Feature Extraction . . . . . . . . . .
Classification . . . . . . . . . . . .
Lesion Classification algorithm . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
44
45
45
47
49
50
43
Chapter 6 addresses two problems:
(i) How to improve the detection of pigmented network regions?
(ii) How to classify the whole lesion?
The algorithm proposed in Chapter 5 detects pigment network regions in excess. This means that
it detects regions in all the images of the database used in this thesis which contain pigment network,
but has many false alarms too (see Figure 5.6). The algorithm used in the network detection block
(see Section 5.3) is simple and does not explore all pigment network properties. Therefore, some
region post-processing can be done to improve the detection results. To do so, a pattern recognition
approach will be adopted. First, features will be extracted from each of regions detected. These
features characterize each one of the regions regarding their intensity, texture and topology and will be
then used to train a classifier (AdaBoost [54]) to separate pigment network regions from not pigment
network regions. The procedures used to perform the features extraction will be addressed in Section
6.4 and the classification algorithm used will be explained in Section 6.5.
To assign a binary label to the whole lesion, a classifier based in area ratio will be used. By
using this procedure it is possible to classify a lesion as with or without pigment network. The lesion
classification method will be discussed in Section 6.6.
6.1
Region and Lesion Ground Truths
In order to train and test the classifiers, each dermoscopy image used in this thesis was classified
by the author under the guidance of an expert.
The expert classified each lesion regarding the presence or absence of pigment network. This
classification will be denoted as lesion ground truth in this thesis. For the cases where the expert decided that pigment network was present, he manually segmented the regions where pigment network
existed (see Figure 6.1). This manual segmentation will be used as the binary region ground truth GT
in this thesis.
(a) Original image with pigment network regions highlighted
(b) Binary Ground Truth
Figure 6.1: Ground truth segmentation of the pigmented network regions.
44
6.2
Background Extraction
The output of the network detection block (see Section 5.3) contains only the dark lines which
form the mesh (the skeleton) of pigment network. The top left image of figure 6.2 shows the binary
output of the network detection block. To perform the region validation step several features which
characterize the intensity, texture and topology of the regions will be extracted. To extract the intensity
and texture features it is necessary to provide the whole region, i.e., the lines and the background.
Therefore, it is necessary to add the background, the ”holes”, to the output of the network detection
block in order to obtain the complete region from which the features will be extracted.
To obtain the background, the regions Ri obtained in the detection block are processed in the
following way:
1. A morphological dilation, such as the one described in (4.6), is applied using a structuring
element equal to the one used in section 4.3. This operations allows the linking of some of the
lines from pigment network which are not completely closed and that, therefore, decreased the
performance of the next morphological operation.
2. By using a morphological filling it is possible to fill the holes in the mesh and create a binary
mask N of the possible network regions Rfi in an image.
Figure 6.2 displays the steps of region processing described above. The results showed are binary
masks. The final regions Rfi are obtained by multiplying the gray scale image by the respective binary
mask N .
The regions Rfi are used for the region labeling process described in Section 6.3 and, the ones
which were not discarded, for the extraction of image features, which will be described in Section 6.4.
6.3
Region Labeling
A classification algorithm is used to assign a binary label to all the detected regions. This step is
called region validation.
In order to train the classifier, a database of regions and the desired label for each region (training
data): 1 for pigmented network and 0 otherwise is required. Therefore, it is necessary to label the
detected regions. This is done by overlapping the detected regions Rfi with the ground truth GT . The
algorithm used to perform region labeling will be explained in this section.
For each image I there is a set of detected regions Rfi such that {Rf1 , Rf2 , . . . , Rfn } ∈ I. Each
one of the regions is analyzed separately. First, the intersection between the region Rfi and the
Ground Truth mask GT of image I is computed
Rf∗i = Rfi ∩ GT
(6.1)
The amount of overlap λ is obtained as follows
λ=
A(Rfi ∩ GT )
A(Rfi )
(6.2)
45
(a) Original
(b) Output of the network detection block
(c) Morphological Dilation
(d) Morphological Reconstruction
Figure 6.2: Steps of the background extraction.
where A(Rfi ∩ GT ) is the area of the overlapping and A(Rfi ) is the area of the region Rfi .
A region Rfi is classified as pigment network (1), not pigment network (0), or discarded, according
to
Rfi


1
= discarded


0
if λ ≥ 0.7
if 0.3 < λ < 0.7
if λ ≤ 0.3
(6.3)
By establishing a labeling condition based on an area ratio, λ, it is possible to classify the regions
with a high level of confidence. The regions discarded are the ones that could induce a bad learning
in the training group so they are excluded and not used for training. The regions that have an area
equal or larger than 70% of pigment network receive the label 1 and are used for training. The same
happens with the regions that have less than 30% of pigment network, which receive the label 0. By
enforcing this last condition, it is possible to ensure that all the detection errors will receive the label 0
and will be used as negative examples.
Figure 6.3 exemplifies the labeling process for the regions detected in the original left image of
Figure 6.1. The left image shows the regions detected in the lesion (there are 6 detected regions:
A-blue, B-green, C- pink, D-yellow, E-orange and F-red) and the right image shows the overlapping
between the regions detected and the region ground truth (see Section 6.1).
46
(a) Detected regions
(b) Overlapping between the detected regions and
the ground truth
Figure 6.3: Region labeling. Overlapping: A(blue) = 74% (label 1), B(green) = 0% (label 0), C(pink) = 0% (label
0), D(yellow) = 0% (label 0), E(orange) = 76% (label 1), F(red) = 24% (label 0)
6.4
Feature Extraction
The vector of features extracted from each region is composed of features which describe texture,
intensity and geometry or shape (area of the network, number of holes, etc). The total number of
features is 51 and they can be divided in two different groups: image features and topological features.
In this section the procedures used to extract these two group of features will be presented.
6.4.1
Image Features
The following image features are used to characterize each region obtained with the processing
described in Section 6.2. They comprise information about intensity (histogram of intensities, variance) and texture (histogram of gradient, entropy), leading to a total of 38 features.
Next, each set of features is defined:
• Histogram of Intensities (26 features) - The histogram of pixel intensities inside the region is
computed by splitting the intensity range into L=26 bins. The histogram is normalized by the
number of pixels in the region Rfi .
• Variance (1 feature) - The variance σ 2 for each region is computed using the following equation
[24, 25]
n
σ2 =
1X
(Ii − I)2
n i=1
!2
(6.4)
where Ii is the intensity of the pixel i, I is the average intensity of the region and n is the number
of pixels in the region Rfi .
• HoG (10 features) - The Histogram of gradient is computed by recurring to the gradient vector
of an image. An image gradient is a directional change in the intensity in an image and is given
by the expression
∇I(x, y) =
∂I(x, y) ∂I(x, y)
,
∂x
∂y
(6.5)
47
where I(x, y) is the intensity of region Rfi and the two partial derivatives are computed by using
a Sobel Operator S [24]

−1

S = −2
−1
0
0
0

1

2
1
such that [24]
∂I(x, y)
' S ∗ I(x, y)
∂x
∂I(x, y)
' S T ∗ I(x, y)
∂y
(6.6a)
(6.6b)
The two partial derivatives are actually two images which at each point contain the horizontal
and vertical derivative. The gradient vector can also be characterized by the amplitude and
orientation (angle - θ) defined as follows [24]
s
2 2
∂I(x, y)
∂I(x, y)
G=
+
∂x
∂y
(6.7)
The direction of the gradient θ is also important for computing the HoG [24]
!
∂I(x,y)
∂y
∂I(x,y)
∂x
θ = tan−1
(6.8)
The histogram of gradient (HOG) is defined as follows
hogi =
X
G(x, y) · bi (θ(x, y))
(6.9)
if θ ∈ i-th bin
if otherwise
(6.10)
(x,y)∈I(x,y)
where
(
bi (θ(x, y)) =
1
0
10 bins, with an angular interval of 18◦ , are used so a total of 10 features is computed (10 bins
x 1 region). A normalization is performed by using the number of pixels in the region Rfi .
• Entropy (1 feature) - The entropy S characterizes the texture of the region and is computed in
the following way [24, 25]
S=−
L
X
h(k)log[h(k)]
(6.11)
k=1
where h(k) is the histogram of intensities extracted previously and L = 10 bins.
6.4.2
Topological Features
These features are used in an attempt to separately characterize the structure of the network and
holes. 13 topological features, that can be divided in 4 different sets, are extracted
• Network Area (1 feature) - This feature characterizes the dark lines of pigment network and is
computed by calculating the area value of the region Ri obtained after the network detection
block. The final value is normalized by the area of the region Rfi .
48
• Network Area after erosion (1 feature) - This feature characterizes the mesh after applying
a morphological erosion to region Ri . The erosion of region Ri by a structuring element B is
defined as
(Ri B)(x, y) = min{Ri (x + x0 , y + y 0 ) − B(x0 , y 0 ) | (x0 , y 0 ) ∈ DB }
(6.12)
where DB is the domain of the structuring element B, which is the same element as the one
used in Section 4.3 for the morphological dilation [25]. In the end the area value is computed for
the region obtained and this value is normalized by the area of the region Rfi .
• Number of Holes (1 feature) - The number of holes characterizes the background. The background RBi is extracted from region Rfi
RBi = Rfi \ Ri∗
(6.13)
where Ri∗ is the result of applying the morphological dilation described in Section 6.2 to the
output of the network detection block Ri . To determine the number of holes a connected component analysis is performed to extract all the connected regions from RBi . The number of
holes is also normalized by the area of the region Rfi .
• Histogram of the Area of Holes (10 features) - These 10 features also characterize the background. The connected component analyses used to compute previous feature can also be
used to determine the area of each connected region (hole) present in RBi . The area of the
holes is normalized in the same way as the previous features. The histogram is composed of
10 bins, which result in a total of 10 features.
Since these features describe the lines and the background separately, they can be used in the
future to discriminate between atypical and typical pigment network.
6.5
Classification
The algorithm used to learn the classification function was the AdaBoost [54] which classifies the
data using an ensemble of simple classifiers denoted as weak classifiers. This algorithm was chosen
not only because it gives good results in challenging object recognition problems,e.g. face detection,
but also because this algorithm is able to select a good subset of informative features for a given
problem (feature selection). This best subset is selected by assuming that each classification function
depends on a single feature.
The AdaBoost procedure can be seen as a greedy feature selection process. At the beginning
it receives as an input the feature vectors xk = {x1k , x2k , . . . , xNk } which characterize the regions
Rfk and the corresponding label yk = 0, 1. The weights wk are associated to each region in such way
that the positive examples receive weights with higher values, since a correct classification of these
examples is more important
(
wk =
α
m
1
l
if yk = 1
if yk = 0
(6.14)
49
where m is the number of positive examples, α is a parameter and l is the number of negative examples.
The final classification function is learned in T iterations. In each iteration the best weak classifier
is trained, being each classifier restricted to a single feature. They are called weak classifiers because
each of them is too simple to provide a good classification of the data. The combination of all weak
classifiers outputs may achieve a good classification performance (strong classifier). After the weak
learning, the best weak classifier, and as a result the best feature, is selected with respect to the
weighted error
t = minxik ,p,θ
X
wk | h(xik , p, θ) − yk |
(6.15)
k
where t is the iteration number and h(xik , p, θ) is a learned weak classifier, restricted to feature xik
(
1 if pxik < pθ
h(xik , p, θ) =
(6.16)
0 otherwise
where p ∈ {+1, −1} is a polarity and θ is the optimal threshold for which the minimal number of
examples are misclassified.
After selecting the best weak classifier hb (xbk , pb , θb ) the weights are updated
(1−ek )
(6.17)
wt+1,k = wt,k βt
where ek = 0 if Rfk is classified correctly and 1 otherwise and βt =
t
1−t .
This way of updating the
weights makes possible to emphasize the examples that were incorrectly classified by the best weak
classifier.
Finally, a strong classifier is defined by combining the decisions of the weak classifiers. This final
classifier is a combination of weighted best weak classifiers and a threshold
(
Pt
Pt
1
1 if
j=1 ηj
j=1 ηj hbj ≥ 2
Ct (Rfk ) =
0 otherwise
where ηj =
(6.18)
1
βj .
After T iterations a final classification CT (Rfk ) is obtained using the data belonging to the training
set. The parameters (α, T and the best feature vector) used in this work were selected empirically.
The reasons for choosing this parameters will be discussed in Section 7.4.
6.6
Lesion Classification algorithm
The second main objective of this work is to perform a binary a classification of skin lesions as
with or without pigment network (1 or 0, respectively, just to consider a simple labeling). Classifying a
lesion regarding the presence or absence of pigment network is not a trivial task. In a first approach
a simple classification based on the regions detected and validated as pigment network should be
enough, i.e., a lesion would be classified as with pigment network if it possessed at least one region
detected and validated as network. Despite being simple, this proposal is not a good one since it
would not consider the size of the lesion, the size of the region or regions detected and the distribution of pigment network throughout the lesion. Therefore it could induce the appearance of several
50
classification errors (lesions classified as with pigment network without being so), since even after the
post-processing block there are still some regions wrongly classified as pigment network (see Section
7.6.2.B).
It was decided that a different approach should be used to perform the lesion classification. Pigment network is usually a structure regularly meshed with narrow spaces, which is distributed more
or less regularly throughout the lesion [3]. This suggests that pigment network is a dense structure. Therefore a good method for performing a lesion classification could be based on the area ratio
between the regions detected and the area of the lesion. This was the approach used.
To perform the area ratio it was necessary to segment the lesion. Since the segmentation is not
the main focus of this thesis, this step was not automated. The dermoscopic images were manually
segmented and each segmentation is actually a binary mask SGT , as can be seen in Figure 6.4.
(a) Original image
(b) Binary Segmentation Mask
Figure 6.4: Lesion Segmentation
The statistical procedure used will be described in detail now. First the total area of the lesions
is computed by using the binary mask SGT . After that, the total network area is determined. This is
done by computing the area of the total pigment network region R obtained as follows
R=[
∪
i:A(Ri )>Amin ∧CT (Ri =1)
Ri ] ∩ SGT
(6.19)
R is then defined as the union of all the regions Ri detected by the network detection block and
validated as pigment network by the region validation block, that can be found inside the lesion. The
region R is only composed of the potential skeleton of pigment network, the ”holes” are not considered
to perform lesion classification.
After determining the area of the pigment network region the classification of a lesion L is performed in the following way
(
with pigment network
if λ ≥ TA
L=
without pigment network if λ < TA
(6.20)
where TA is a threshold and
λ=
A(R)
A(SGT )
(6.21)
A(R) is the area of R and A(SGT ) is the area of SGT . It was determined that the best value for
TA = 0.1.
51
52
7
Results and Discussion
Contents
7.1
7.2
7.3
7.4
7.5
7.6
7.7
Database of dermoscopy images . . . . . . . . . . . .
Performance Measures . . . . . . . . . . . . . . . . . .
Comparison between 4 and 8 pixels-neighborhood .
AdaBoost parameters selection . . . . . . . . . . . . .
Comparison between gray-level conversion methods
Region Detection System . . . . . . . . . . . . . . . .
Lesion Classification Results . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
54
54
57
58
63
64
69
53
This chapter presents and discusses the application of the proposed algorithm to a database
of dermoscopy images. Some parameters choices will also be discussed in this chapter, namely
decisions regarding the number of neighbors used in the connected component analysis (Section
7.3), the Adaboost parameters (Section 7.4) and the gray-level conversion method (Section 7.5).
7.1
Database of dermoscopy images
The algorithm described in the previous chapters was tested on a set of 57 dermoscopy images,
extracted from the database of Hospital Pedro Hispano, Matosinhos. This set contains RGB images
of size around 576x767, stored in bitmap and jpeg formats. Each color component is in the range 0,
. . . , 255. The images were acquired during clinical exams using a dermatoscope and a magnification
of 20x.
The images were selected in such a way that there were examples of melanocytic and nonmelanocytic lesions in the dataset used, but more melanocytic examples. The percentages can be
seen in Table 7.1.
Table 7.1: Percentage of melanocytic and non-melanocytic lesions in the dataset.
Melanocytic
91%
Non-melanocytic
9%
Pigment network is present in 13 of the melanocytic lesions, this means that around 23% of all
images in the dataset were classified by an expert as ”with pigment network”.
Table 7.2: Percentage of .
Pigment Network
23%
Non-pigment Network
77%
Both classifications were performed by an expert dermatologist with training in the analysis of
dermoscopy images.
7.2
Performance Measures
In this section, the procedures used to evaluate the performance of the detection algorithm are
addressed.
Since the database of dermoscopy images is small and it is not possible to use the same image for
training and testing the classifiers, a K fold cross validation procedure (K = 5) was adopted to perform
the evaluation. The advantage of this method is that all observations are used for training and for
test but in different experiments. Each one is used for test exactly once. The K fold cross validation
method requires the dataset to be sampled in K subsets with approximately the same size. Therefore,
54
the dataset of 57 dermoscopy images was divided into 5 subsets with approximately the same number
of positive and negative examples (regions with and without pigment network, respectively).
One of the subsets is used to test the data while the other four are used for training. In the test
phase, the output of the algorithm for the test data is compared with the respective ground truth and a
statistics is computed. The process is repeated five times, each one with a different subset as the test
data. The final statistics, which evaluates the performance of the algorithm, is computed by averaging
the statistical values obtained for the five different subsets. All the statistical values presented in this
chapter were obtained using the 5 fold cross validation method described.
This thesis addresses two main problems: the detection of regions containing pigment network
and the binary labeling of lesions with respect to the presence of pigment network. These two problems were evaluated with different statistics that will be presented in the following subsections. The
statistic used to evaluate the performance of the AdaBoost classifier will also be discussed, since
an evaluation process is required in order to be able to select the best parameters for training the
classifier.
7.2.1
Assessment of Region Detection Algorithm
The output of the network detection system consists of a set of regions that must be compared
with the binary region ground truth (see Section 6.1). This comparison can be done using two types
of statistics: pixel based and region based statistics. These two alternative will be presented next.
7.2.1.A
Pixel based statistics
This evaluation method is done by comparing the (binary) segmented image BN to the ground
truth segmentation GT ) (see Section 6.1). A pixel p(x, y) can be classified as T P, T N, F N or F P as
follows

TP



FP
p(x, y) =

TN



FN
if BN (x, y) = 1 ∧ GT (x, y) = 1
if BN (x, y) = 1 ∧ GT (x, y) = 0
if BN (x, y) = 0 ∧ GT (x, y) = 0
if BN (x, y) = 0 ∧ GT (x, y) = 1
(7.1)
where
TP: Correctly detected as pigment network pixel.
FP: Wrongly detected as pigment network pixel.
TN: Correctly undetected pixel (not pigment network).
FN: Wrongly undetected pixel (not pigment network).
The performance of the detection system is assessed by determining the number of T P, T N, F N
and F P in the test images and by computing the SE and SP values
55
#T P
#T P + #F N
#T N
SP =
#T N + #F P
SE =
7.2.1.B
(7.2a)
(7.2b)
Region-based statistics
To evaluate the performance of the network detection algorithm at the region level the binary
mask BN of each image in the test set is compared with the respective binary ground truth GT
(see Section 6.1) and classified in one of the the following classes: correct detection (CD), false
alarm (FA) and detection failure (DF). This procedure is called region matching [37] and establishes a
correspondence between the detected regions and the ground truth. The three cases considered are
the following
CD: The detected region matches one or more regions in the GT .
FA: The detected region has no correspondence.
DF: One GT region has no correspondence.
Cases regarding the splitting or merging of regions [37] are not differentiated from CD.
The region matching algorithm is performed by computing a correspondence matrix C which defines the correspondence between the active regions of a pair of images [37]. Assuming that GT is
composed of P regions and that BN contains M detected regions, C will be a P × M matrix, defined
as follows
C(i, j) =


1


0
if A(BNj ∩ GTi ) ≥ 0.3A(BNj )
∀ i ∈ {1, . . . , P}, j ∈ {1, . . . , M}
if otherwise
(7.3)
where A(BNj ∩ GTi ) is the area of the region BNj ∩ GTi and A(BNj ) is the area of BNj . It is also
useful to add the number of ones in each line or column, defining two auxiliary vectors [37]
L(i) =
P
X
C(i, j)
i ∈ 1, . . . , M
(7.4)
j ∈ 1, . . . , P
(7.5)
j=1
K(j) =
M
X
C(i, j)
i=1
A region BNj is classified as CD if K(j) ≥ 1. By imposing this condition it is possible to ensure that
detected regions which are actually the merge of two regions of the ground truth are only accounted
once as CD. Detected regions which result from a split of a ground truth region are independently
classified as CD. FA and DF are determined by computing the number of empty columns or lines in
C, respectively [37].
The final statistics is performed by computing the total values of CD, FA and DF for the test set.
56
7.2.2
Assessment of the validation step
The output of the AdaBoost classifier consists of a binary classification of a region as 1 (pigment
network) or 0 (not-network). The classifier can be evaluated by computing the SE and SP values. To
do so, the output of the classifier is compared the ground truth label (see Sections 6.1 and 6.3). By
performing this comparison is possible to assess the values of
#TP: Number of regions correctly classified as pigment network.
#FP: Number of regions wrongly classified as pigment network.
#TN: Number of regions correctly classified as not pigment network.
#FN: Number of regions wrongly classified as not pigment network.
After computing the previous values, it is possible to determine the SE and the SP of the classifier
for the test set.
7.2.3
Assessment of Lesion Classification
To evaluate the performance of the lesion classification algorithm (see Section 6.6) the binary
output of the classifier is compared with the binary ground truth (see Section 6.1) for each one of the
images of the test set. The output of the classifier can be regarded as
TP: The lesion is correctly classified as with pigment network.
FP: The lesion is incorrectly classified as with pigment network.
TN: The lesion is correctly classified as without pigment network.
FN: The lesion is incorrectly classified as without pigment network.
The numbers of T P, T N, F N and F P are determined for the test set and the sensitivity (SE) and
specificity (SP) are computed.
7.3
Comparison between 4 and 8 pixels-neighborhood
Connected component analysis (see Section 5.3) depends on the neighborhood system chosen
by the user. The two most usual neighborhood systems are 4 and 8 connectivity. The main difference
between the two connectivities is that by using the 8-connectivity one uses the information of the
8 adjacent pixels (vertical, horizontal and the two diagonals), while in the 4-connectivity only four
neighbors are used (vertical and horizontal). In theory, 8-connectivity allows the detection of larger
linked areas since it also uses the information from the upper diagonal neighbors. Both connectivities
were tested and a qualitative result can be seen in Figure 7.1. Both images are outputs of the network
detection block.
By examining these two images it is possible to conclude that, although the outputs are quite
similar, the one obtained by using a neighborhood of 8 pixels is slightly better, since more network
57
(a) Original
(b) 4-connectivity
(c) 8-connectivity
Figure 7.1: Output of the network detection block for different connectivities.
regions are detected. Therefore the final decision was to use 8-connectivity. However, it is important
to say that for most examples of lesions with pigment network the output of the detection block was
exactly the same for the two connectivities. This suggests that a 4-connectivity could also be used
and the detection block would perform well too.
7.4
AdaBoost parameters selection
The parameters used to train the classifier were selected in order to be the ones which give the
best final results, i.e., the best values for SE and SP (see Section 7.2.2). It was decided that wrongly
classifying positive network regions should be seen as a more problematic error, since pigment network is a very important structure in dermoscopy image analyses. To enforce the correct classification
of this kind of regions different weights were assigned to the positive and negative regions (see Section 6.5). Different values for the α parameter were tested and will be discussed next. Simultaneously,
a study was performed in order to determine which features’ combination could discriminate better
the positive from negative regions. Finally, the number of iterations of the classifier were also chosen in order to achieve the best classification. After computing the best α value different tests were
performed in order to determine the best number T of iterations.
7.4.1 α weight parameter
The classifier was trained using different α ∈ {1, 3, 5, 10, 15} values (see (6.14)) and 5-fold cross
validation.
A large number of iterations was considered (T = 2000). This value guarantees that there are no
classification errors in the training set.
The final results for the different α values can be seen in Table 7.3. This table shows the percentage of correctly classified network regions (SE or TP rate) and the percentage of wrongly classified
not-pigment network regions (100% − SP or FP rate) for three different sets of features composed
of: only Image Features, only Topological Features, and both. These statistics are displayed in the
receiver operating characteristic (ROC) shown in Figure 7.2.
By inspecting the previous table it is possible to conclude that when α increases the percentage
58
Table 7.3: Statistical Results for the different α values and T = 2000. FP-rate stands for the percentage of not
pigment network regions that are wrongly classified ; TP-rate is the percentage of correctly classified pigment
network regions.
α
1
3
5
10
15
Image Features
FP-rate
TP-rate
22%
30%
26%
35%
27%
36%
29%
37%
30%
39%
Topological Features
FP-rate
TP-rate
20%
44%
21%
45%
24%
45%
24%
46%
25%
47%
FP-rate
18%
22%
23%
24%
26%
Both
TP-rate
38%
42%
42%
45%
45%
of correctly classified regions increases for the three sets of features. However, the number of false
positives increases as well. Nevertheless, the correct identification of pigment network regions is considered more important than rejecting detection errors. Therefore, and based on the results obtained,
the value chosen for parameter α was 15. The low percentage of TP might be explained by the large
number of iterations used, since it could have caused an overfitting of the classifier to the training data
used.
Higher values for α could be tested and a better ROC curve could be obtained (see Figure 7.2.
However, an increase in the value of this parameter resulted in an increase of the number of iterations
of the learning algorithm, which is time consuming, and in a decrease of the correctly classified
negative examples.Therefore, the decision was made to use 15 as the highest value for parameter α.
This is not the better approach to select a parameter so, in the future, different solutions for computing
this weight index should be considered, e.g. using the bayesian probability.
7.4.2
Features Comparison
The previous study was performed using three different sets of features. The results from Table
7.3 were used to create a ROC curve for each one of the features’ sets. The various ROC curves can
be seen in Figure 7.2.
The receiver operator charecteristics (ROC) curve is a good performance measure. It is a plot of
the TP-rate (percentage of correctly classified positive samples) versus the FP-rate (percentage of
wrongly classified negative samples). The curve illustrates the behavior of a classifier without regard
to class distributions or error costs, and thus decouples the classification performance of these factors
[15]. The area under the curve for the peak value gives information about the classifier performance.
The higher the value the better the test is.
The ROC curves obtained show the difficulty of the problem. The TP rate is far from 100%. Nevertheless, they provide some information regarding the set of features that better discriminates between
positive and negative examples. By analyzing the results it is possible to conclude that the best set of
features is the one composed of only topological network features. This is the set that simultaneously
has the lowest percentage of wrongly classified negative examples (25%) the highest TP-rate of 47%,
when compared with the other two sets for α = 15 (see Table 7.3). Another comparison can be done
59
Figure 7.2: ROC curve obtained for the three different features’ sets by varying the α ∈ {1, 3, 5, 10, 15} weight
parameter and T = 2000.
by inspecting Figure 7.2, where it is clear that the ROC curve for the topological features set is the
one which have a higher area under the curve value, therefore, this is the best set of features.
An interesting study is to determine which features, among all the topological ones, play a major
role in the learning procedure. To perform this study a statistical analysis was made considering all
the features selected by the AdaBoost algorithm (see Section 6.5), i.e., the features used to define
the strong classifier. The results of this study can be seen in the histogram of Figure 7.3.
Figure 7.3: Histogram of the average rate of selection of each topological feature for α = 15 and T = 2000.
Each one of the ”Area Holes i” feature stands for the i − th bin of the histogram of the area of holes.
The previous histogram plots the number of times each topological feature is chosen during the
Adaboost training with five fold cross validation, α = 15 and T = 2000. Five different trainings groups
60
are used, each one using a different combination of the five existing subsets. Therefore the results
shown in Figure 7.3 are an average of the results of the five training groups. By inspecting the results
it is possible to conclude that there are four topological features that have a more preponderant role
in the final classification rule. Those are, by rate of selection, ”Number of Holes”, ”Network Area”,
”Network Area after erosion” and the 10 − th bin of the ”Histogram of the Area of Holes”. The most
important feature is the ”Number of Holes”. This may be due to the fact that ”holes” are characteristic
of a mesh structure such as the pigment network; other structures that are not meshed do not have
holes (or at least not as many). Therefore the ”Number of Holes” can be a feature that performs a
good discrimination between mesh-like structures and others.
Despite the fact that topological features were the ones selected to obtain the final results, it was
considered that a similar study should be performed using the set composed of both kinds of features,
since it could be interesting to verify if the same behavior was observed. Figure 7.4 presents the
results obtained.
Figure 7.4: Histogram of the average rate of selection of all the features for α = 15 and T = 500. Each one of
the ”H i” , ”HoG i” feature and ”H i” feature stands for the i − th bin of, respectively, the histogram of intensities,
HoG and histogram of the area of holes.
As previously, the ”Number of Holes” is the most used feature. The ”Network Area” is also used
several times. These findings and the previous ones suggest that ”Number of holes” and ”Network
Area” might be cue features in distinguishing between true and false pigment network regions.
The correlation between different features was also studied. By plotting one feature against the
other in scatter plots it is possible to extract information regarding how well the two features can
discriminate positive from negative samples. In Figure 7.5 four scatter plots between pairs of the three
most used topological features can be seen. By observing these graphics it is possible to conclude
that the positive and negative examples, labeled respectively as ”network” and ”no network”, are
not linearly separable and show a strong overlap. This makes the discrimination between pigment
and non-pigment network regions very difficult. Since some positive and negative regions have very
61
similar values for the same features this explains why the classification scores shown in 7.3 and 7.4
are far from perfect since it is not possible to simultaneously obtain high SP and SE scores.
(a) ”Number of Holes” vs ”Network Area”
(b) ”Number of Holes” vs ”Network Area after
Erosion”
(c) ”Network Area” vs ”Network Area after
Erosion
Figure 7.5: Correlation Graphics
7.4.3
Number of iterations
The number of iterations determines the number of best weak classifiers that will be selected
to compute the final strong classifier. By keeping constant α = 15 and the input of features used
(topological features) several tests were performed in order to determine which is the best number of
iterations. The cross-validation results for the several tests are resumed in Table 7.4.
Table 7.4: Statistical results for different numbers of iterations for α = 15 and topological features. FP-rate stands
for the percentage of non-network regions that are wrongly classified as so while TP-rate is the percentage of
correctly classified pigment network regions.
T
100
150
200
300
2000
FP-rate
49%
46%
45%
40%
25%
TP-rate
86%
77%
75%
74%
47%
These results show unequivocally that an increasing in the number of weak classifiers results in
a dramatically decrease of the percentage of true positives. Despite the decrease of the FP-rate,
62
those are not good results since pigment network is a very important dermoscopic structure which
should not be missed. This reinforces the idea that increasing the number of iterations will lead to
an overfitting of the classifier. Therefore, it was decided that to prevent a high percentage of false
negatives (pigment network regions wrongly classified) and an overfitting, the number of iterations
should be equal to T = 100. The hypothesis of using T = 150 was also tested. However, the results
for lesion classification (which will be presented in Section 7.7) regarding lesions with pigment network
were worse.
7.5
Comparison between gray-level conversion methods
In order to select the best gray-scale conversion method (see Section 4.1) the performance of
the AdaBoost was evaluated for both methods. Tables 7.5 and 7.6 show the results using (4.2) and
(4.1), respectively. The results were obtained by using the best parameters for AdaBoost determined
previously (α = 15, T = 100 and topological features).
Table 7.5: Statistical results for AdaBoost performance using the highest entropy channel (see (4.2)) as gray
level conversion method for α = 15, T = 100 and topological features.
Truth
Not pigment network
Pigment Network
Detected
Not pigment network
Pigment Network
51%
49%
14%
86%
Table 7.6: Statistical results for AdaBoost performance using the linear combination of RGB values (see (4.1)) a
gray level conversion method for α = 15, T = 100 and topological features.
Truth
Not Pigment Network
Pigment Network
Detected
Not Pigment Network
Pigment Network
33%
67%
27%
73%
By inspecting both tables it is possible to notice that the classifier achieves a SE = 86% and a
SP = 51% when using the highest entropy channel and a SE = 73% and a SP = 33% when using the
linear combination of RGB values. Since both SE and SP have higher values for (4.2) it is possible
to conclude that the conversion method which gives the best discrimination and, therefore, the best
results for the detection algorithm is the highest entropy channel. This was the method used to obtain
the results presented in the following sections.
63
7.6
Region Detection System
7.6.1
Detection examples
Figure 7.6 shows some examples of the output of the detection system for lesions which contain
pigment network. In these figures, the left column corresponds to the original image, the middle
column is the output of the detection block before the validation steps described, respectively, in
Sections 6.2 and 6.3 and the left column corresponds to the final output of the region detection
system.
By observing the examples of Figure 7.6 it is possible to conclude that the region validation block
does not affect significantly the extraction of pigment network areas. In most cases, some of which
are exemplified, the output of the two blocks is exactly the same. There are other cases for which the
output is slightly different and this is due to some misclassified pigment network region. Nevertheless,
most of the pigment network skeleton is still extracted, as can be seen in the last set of three images
from Figure 7.6.
The main objective of the post-processing block is to eliminate detection errors or false alarms.
Figure 7.7 shows some examples of applying the detection algorithm to lesions without pigment network. The results demonstrate that the post-processing block removes some of the wrongly detected
regions but not all of them. Actually, in the last two rows of Figure 7.7 it is possible to see that the
post-processing block is almost ineffective, since most part of the false alarms remain. These results
suggest that in the future new approaches for removing some misleading structures such as dots
(last row of the figure) or the cobblestone pattern (7.7(g)) should be tested. The post-processing
block must also be altered and different combinations of features should be tried, like combining topological features with only HoG or variance, in order to provide the classifier with some features that
discriminate intensity or texture.
64
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
(l)
(m)
(n)
(o)
Figure 7.6: Output of the automatic detection system for lesions with Pigment Network: original image (left),
output of the network detection block (center) and final output after the region validation block (right)
65
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
(l)
Figure 7.7: Output of the automatic detection system for lesions without Pigment Network: original image (left),
output of the network detection block (center) and final output after the region validation block (right)
66
7.6.2
Statistical Results
The examples presented in the previous section are very interesting. However, a quantitative
evaluation, in which the output of the detection system developed is compared with the region ground
truth (see Section 6.1), is also required. In this section the assessment of the performance of the
detection algorithm, with 5 fold cross validation, at the pixel and region levels, will be presented. The
methods used are described in Section 7.2.1.
7.6.2.A
Pixel-based statistics
By assessing the performance of the detection algorithm on a pixel level it is possible to establish a
comparison between the area of the detected regions and the area of the segmentation performed by
the expert. The results presented compare the output of detection block before and after the validation
block. The statistics can be seen in Tables 7.7 and 7.8, respectively.
Table 7.7: Statistical results at a pixel level for the Network detection system before the validation step, for
α = 15, T = 100 and topological features.
Truth
Not Pigment Network
Pigment Network
Detected
Not Pigment Network
Pigment Network
95%
5%
43%
57%
Table 7.8: Statistical results at a pixel level for the Network detection system after the validation step, for α = 15,
T = 100 and topological features.
Truth
Not Pigment Network
Pigment Network
Detected
Not Pigment Network
Pigment Network
96%
4%
48%
52%
The results show that the algorithm has a SE = 57% and a SP = 95% before the validation and
a SE = 52% and a SP = 96% after the validation step. It is possible to see that the validation step
has a minor effect on the number of correctly classified not-network pixels. However, it reduces the
number of correctly classified pigment-network pixels around 5%, which means that the validation
step wrongly rejects some pigment network regions. By inspecting the values obtained it is possible
to see that the regions detected by the detection algorithm are smaller (smaller number of pixels) than
the ones from the ground truth, since has can be seen in the next subsection the algorithm does not
have many DF and, therefore, the low SE value is not mainly related with undetected regions. This
is not an unexpected result, since it is not possible to segment perfectly a pigment network region
(see example in Section 6.1). The segmentation will always include more pixels than the ones which
actually belong to pigment network.
67
7.6.2.B
Region-based statistics
Another interesting quantitative evaluation is computing the number of regions which are correctly
detected, the ones which are false alarms and the regions which the algorithm misses. This is done
by comparing the output of the detection system with the lesion ground truth as described in Section
7.2.1 for region level statistics.
Tables 7.9 shows the results for the 5 fold cross validation before and after the validation step.
Table 7.9: Statistical results at region level for the network detection system before and after the validation step.
The validation is performed with α = 15, T = 100 and topological features. CD - correct detection; FA - false
alarm; DF - detection failure.
CD
FA
DF
Before Validation
15
53
1
After Validation
13
27
3
By inspecting the previous table it is possible to conclude that the algorithm presents a high number of CD and FA before the validation. This confirms the observations made in Chapter 5 (see
Section 5.4) in which was observed that the network detection block detects regions in excess, which
results in a small number of DF but also in a high number of FA. After the validation step it is possible
to observe that the number of FA decreases considerably. The number of CD decreases slightly and
the number of DF increases in the same proportion, which was expected based on the results of
both the AdaBoost (see table 7.5) and pixel validation(see table 7.8). By analyzing this results it is
possible to conclude that the validation step brings benefits to the final detection algorithm, in terms
of region statistics, since the number of wrongly rejected pigment network regions is very small and
it removes a considerable part of the FA (detection errors). In a larger scale it is possible to prove
that the detection algorithm proposed performs well in the dataset provided, achieving good network
detection scores.
By comparing this validation method with the pixel-based one it is possible to conclude that this
method is better to evaluate the performance of the region detection system, since it allows a more
viable comparison between the output of the detection system and the ground truth.
68
7.7
Lesion Classification Results
Table 7.10 shows the results of the lesion classification algorithm, using 5 fold cross validation
and the statistical method described in section 7.2.3. The algorithm obtains a final SP = 77% and
a SE = 78% in this dataset of clinical examples. These statistics prove that the detection system
developed achieves good detection scores and performs reasonably well in the dataset provided.
Table 7.10: Statistical results for lesion classification after validation, with α = 15, T = 100 and topological
features.
Truth
Without network
With Network
Without Network
77%
22%
Detected
With Network
23%
78%
In order to perform a final assessment of the benefits provided by the region validation (postprocessing) block a statistical study similar to the one described previously was applied to the output
of the network detection block. In this study, the classification is performed as described previously.
However, since there was no post-processing, all the regions detected inside a lesion that fulfilled
the area thresholding condition were considered. The results, obtained by the 5 fold cross validation
method, can be seen in Table 7.11.
Table 7.11: Statistical results for lesion classification before validation, with α = 15, T = 100 and topological
features.
Truth
Without network
With Network
Without Network
63%
15%
Detected
With Network
37%
85%
These results show that if no region validation block were applied, the algorithm would achieve a
SP = 63% and a SE = 85%. By comparing these results with the ones obtained previously (see table
7.10) it is possible to conclude that by applying the post-processing block it is possible to increase the
SP by 14%. The SE decreases around 7%. The decrease in the SE value maybe due to misclassified
pigment network regions that will be excluded from the final region R and, therefore, reduce the area
ratio. Another explanation can be the fact that this statistic was performed without performing a preclassification of regions as the one described in section 6.3 and, therefore, without discarding regions.
Since this pre-classification is performed before the post-processing block, this might as well explain
the reduction in the area ratio of a given lesion and consequent misclassification. Despite reducing the
number of correctly classified lesions with pigment network, it is considered that the post-processing
block brings some benefits to the final automatic system, since a relevant reduction of the SP value is
achieved and a comparison with a ground truth is performed, which suggests that the results obtained
for lesions with pigment network have a high degree of confidence.
69
70
8
Conclusions and Future Work
Contents
8.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
72
72
71
8.1
Conclusions
This thesis proposes an automatic algorithm for pigment network detection in dermoscopy images.
The algorithm involves several processing steps: pre-processing, network enhancement and detection, region validation and lesion classification. In each one of these blocks new approaches were
developed, trying to improve the performance of the whole system.
The pre-processing block works by detecting and removing reflection and hair artifacts from dermoscopy images. The results obtained for this block were quite promising and suggest that this block
can be used in other automatic algorithms which try to extract dermoscopic features, different from
pigment network, from dermoscopy images or even in automatic algorithms which perform lesion
segmentation or lesion classification.
The network enhancement and detection blocks explore line color and geometry using a bank
of directional filters and a connected component analysis. By combining these two blocks with the
region validation block it is possible to achieve good detection scores both for region detection (CD =
13, F A = 27 and DF = 3) and lesion classification (SE = 78% and SP = 77%). This algorithm
is also capable of extracting the mesh of pigment network, which can help dermatologists when
performing a diagnosis. These results prove that the automatic algorithm proposed is an useful tool
in a dermoscopy analysis system.
8.2
Future work
In this section future work suggestions will be discussed.
Improve the region validation step results - The results obtained in this thesis for the validation
demonstrate that selecting the best features and training a classifier to perform a binary distinction between pigment network and not pigment network regions is a challenging problem.
The results obtained were far from perfect, which suggests that future work should regard the
improvement of these results. This can be done in two different ways:
• New combination of features: Try to combine the topological features already used, with
only one or two image features like HoG or variance, in order to provide some information
regarding the intensity of the region. Another approach could be the use of features which
characterize the color of the region instead of only the intensity, i.e., extracting features
from the three color channels R,G and B. By working only with the gray scale image it is
possible that some information regarding pigment network regions is being lost. Therefore,
by using color features this problem could be overcome.
• Different classifier: AdaBoost is a good classifier, since it is capable of performing a
feature selecting process. However, different classifiers like ANN or SVM could be tested
as well, and their results compared with the ones from AdaBoost,
Detecting other structures - Directional filters are matched filters. Therefore, their impulse response can be designed to adapt to a specific shape and so, a bank of directional filters with
72
an impulse response which matches the shape of dots or globules can be used in the future
to try to detect these dermoscopic features in dermoscopy images. These filters could also be
integrated in the pre-processing step of the automatic system developed in this thesis to help
distinguishing the dots pattern from pigment network, since one of the detection errors is related
with the presence of this pattern.
Larger database - The algorithm proposed achieves good detection scores in the dataset of clinical
cases used. However, this dataset is small (only 57 images). In order to improve the results
obtained an to analyze their reproducibility this algorithm must be tested in a larger dataset,
with more lesions with pigment network, since a better learning rule could be achieved if more
positive examples are present.
Comparison with other works - This thesis proposes a new approach for pigment network detection, different from the ones found in literature. In the future, this algorithm should be compared
with other algorithms for pigment network detection, since a comparison between the several
methods will allow a better realization of the value of the algorithm proposed, its assets and
weakest points. The comparison should be performed with the same dataset that was used in
this thesis or with a larger dataset.
Distinguishing between typical and atypical pigment network - The detection of pigment network
is important for two reasons. First, since it is an hallmark of melanocytic lesions, its detection
can help in a preliminary classification of lesions as melanocytic or not. Secondly, an atypical
pigment network is an unmistakable sign of melanoma. The algorithm proposed does not distinguish between typical and atypical pigment network. Therefore, future work should also focus
on the distinction between typical and atypical since this will increase the medical value of the
algorithm.
73
74
Bibliography
[1] Dermnet. http://www.dermnet.com/.
[2] Dermoscopy atlas. http://www.dermoscopyatlas.com.
[3] Dermoscopy tutorial. http://www.dermoscopy.org/atlas/base.htm.
[4] Q. Abbas, M. Celebi, and I. Garcı́a. Hair removal methods: A comparative study for dermoscopy
images. Biomedical Signal Processing and Control, 2001.
[5] M. Anantha, RH. Moss, and WV.. Stoecker. Detection of pigment network in dermatoscopy
images using texture analysis. Computarized Medical Imaging and Graphics, 28(5):225–234,
2004.
[6] G. Argenziano, G. Fabbrocini, P. Carli, V. De Giorgi, E. Sammarco, and M. Delfino. Epiluminescence microscopy for the diagnosis of doubtful melanocytic skin lesions. comparison of the
ABCD rule of dermatoscopy and a new 7-point checklist based on pattern analysis. Archives of
Dermatology, (134):1563–1570, 1998.
[7] G. Argenziano, H.P. Soyer, and S. Chimenti. Dermoscopy of pigmented skin lesions: results of a
consensus meeting via the internet. Journal of American Academy of Dermatology, 148(5):375–
389, 2003.
[8] C. Barata, J.S. Marques, and J. Rozeira. Detecting the pigment network in dermoscopy images:
a directional approach. In IEEE, editor, 33rd Annual International Conference on the IEEE EMBS,
pages 5120–5123, September 2011.
[9] M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester. Image inpainting. SIGGRAPH, pages
417–424, 2000.
[10] G. Betta, G. Di Leo, G. Fabbrocini, A. Paolillo, and P. Sommella. Dermoscopic image-analysis
system: estimation of atypical pigment network and atypical vascular pattern. In International
Workshop on Medical Measurement and Applications, April 2006.
[11] M. Binder, M. Puespoeck-Schwartz, A. Steiner, H. Kittler, K. Wolff, M. Muellner, and H. Pehamberger. Epiluminescence microscopy of small pigmented skin lesions : short-term formal
training improves the diagnosis performance of dermatologists. Journal American Academy of
Dermatology, pages 197–202, 1997.
75
[12] A. Blum, H. Luedtke, U. Ellwanger, R. Schwabe, G. Rassner, and C. Garbe. Digital image analysis for diagnosis of cutaneous melanoma. development of a highly effective computer algorithm
based on analysis of 837 melanocytic lesions. British Journal of Dermatology, 151:1029–1038,
2004.
[13] B. Caputo, V. Panichelli, and G.E Gigante. Towards a quantitative analysis of skin lesions. Studies
in health technology and informatics, 90:509–513, 2002.
[14] N. Cascinelli, M. Ferrario, T. Tonelli, and E. Leo. A possible new tool for clinical diagnosis of
melanoma: The computer. Journal American Academy of Dermatology, 16(2):361–367, 1987.
[15] M.E. Celebi, Kingravim H., B. Uddin, H. Iyatomi, Y. Aslandogan, W. Stoecker, and R. Moss. A
methodological approach to the classification of dermoscopy images. Computerized Medical
Imaging and Graphics, 31:362–373, 2007.
[16] M.E. Celebi, H. Iyatomi, W. Stoecker, R.H. Moss, H. Rabinovitz, G. , and H.P. Soyer. Automatic
detection of blue-white veil and related structures in dermoscopy images. Computerized Medical
Imaging and Graphics, 32:670–677, 2008.
[17] G. Daugman. Uncertainty relation for resolution in space, spatial frequency, and orientation
optimized by two-dimensional visual cortical filters. Journal of the Optical Society of America,
(7), 1985.
[18] R. Duda, P. Hart, and D. Stork. Pattern Classification. Wiley, 2001.
[19] G. Fabbrocini, G. Betta, G. Di Leo, C. Liguori, and et al. Epilluminescence image processing for
melanocytic skin lesion diagnosis based on 7-point check list: a preliminary discussion on three
parameters. The open dermatology journal, 4:110–115, 2010.
[20] M. Fleming, C. Steger, J. Zhang, J. Gao, A. Cognetta, I. Pollak, and C. Dyer. Techniques for
a structural analysis of dermatoscopic imagery. Computarized Medical Imaging and Graphics,
(5):375–389, 1998.
[21] I. Fogel and D. Sagi. Gabor filters as texture discriminator. Biological Cybernetics, (61):103–113,
1989.
[22] D. Gabor. Theory of communication. J. IEE Lond., 93:429–457, 1946.
[23] H. Ganster, A. Pinz, E. Wildling, M. Binder, and H. Kittler. Automated melanoma recognition.
IEEE transaction on Medical Imaging, 20(3):233–239, 2001.
[24] R. Gonzalez and R. Woods. Digital Image Processing. Prentice Hall, 2 edition, 2002.
[25] R. Gonzalez, R. Woods, and S. Eddins. Digital Image Processing using MATLAB. Gatesmark
Publishing, 2009.
76
[26] C. Grana, R. Cucchiara, G. Pellacani, and S. Seidenari. Line detection and texture characterization of network patterns. In In ICPR’06: Proceedings of the 18th International Conference on
pattern Recognition, 2006.
[27] C. Grigorescu.
Contour detection based on nonclassical receptive field inhibition.
IEEE
transactions on image processing, 7, 2003.
[28] K. Hoffman, T. Gambichler, A. Rick, M. Kreutz, M. Anschuetz, and T. Grunendick. Diagnostic
and neural analysis of skin cancer (DANAOS, a multicentre study for collection and computeraided analysis of data from pigment skin lesions using digital dermoscopy. Britain Journal of
Dermatology, 149(4):801–809, 2003.
[29] D.H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction and functional architecture
in the cat’s visual cortex. Journal of Physiology, 160:106–154, 1962.
[30] H. Iyatomi, H. Oka, M.E. Celebi, M. Hashimoto, M. Hagiwara, M. Tanaka, and K. Ogawa. An improved internet-based melanoma screening system with dermatologist-like tumor area extraction
algorithm. Computerized Medical Imaging and Graphics, 32:566–579, 2008.
[31] J. P. Jone and L. A. Palmer. An evaluation of the two-dimensional gabor filter model of simple
receptive fields in cat striate cortex. Journal of Neurophysiology, 58(6):1233–1258, 1987.
[32] J. Kirk. Count loops in a nework. http://www.mathworks.com/matlabcentral/fxfiles/10722/
1/content/html/runloopshtml.html, 2006.
[33] T. Lee, V. Ng, R. Gallagher, A. Coldman, and D. McLean. Dullrazor:a software approach to hair
removal from images. Comp. Biol. Med., 27(6):533–543, 1997.
[34] TK. Lee, D. McLean, and MS. Atkins. Irregularity index: a new border irregularity measure for
cutaneous melanocytic lesions. Medical Image Analysis, 7(1):47–64, 2003.
[35] J. Mayer. Systematic review of the diagnostic accuracy of dermatoscopy in detecting malignant
melanoma. Medical Journal Australia, (167):206–210, 1997.
[36] S.W. Menzies, C. Ingvar, K.A. Crotty, and W.H. McCarthy. Frequency and morphologic characteristics of invasive melanomas lacking specific surface microscopic features. Archives of
Dermatology, (132):1178–1182, 1996.
[37] J. C. Nascimento and J. S. Marques. Novel metrics for performance evaluation of object detection
algorithms. In IEEE, editor, Sixth IEEE International Workshop on Performance Evaluation of
Tracking and Surveillance, May 2004.
[38] H. Oka, M. Hashimoto, and M. Tanaka. Internet based program for automatic discrimination of
dermoscopic images between melanoma and clark nevi. British Journal of Dermatology, 150(5),
2004.
77
[39] H. Pehamberger, A. Steiner, and W. Wolff. In vivo epiluminescence microscopy of pigmented
skin lesions. I. pattern analysis of pigmented skin lesions.
Journal American Academy of
Dermatology, (17):571–583, 1987.
[40] G. Pellacani, C. Grana, R. Cucchiara, and S. Seidenari. Automated extraction of dark areas in
surface microscopy melanocytic lesion images. Dermatology, 208(1):21–26, 2003.
[41] P. Rubegni, G. Cevenini, M. Burroni, R. Perotti, G. Dell’Eva, P. Sbano, and C. Miracco. Automated
diagnosis of pigment skin lesions. International Journal of Cancer, 101:576–580, 2002.
[42] M. Sadeghi, M. Razmara, T. Lee, and M. Stella Atkins. A novel method for detection of pigment
network in dermoscopic imagesusing graphs. Computerized Medical Imaging and Graphics,
(35):137–143, 2011.
[43] P. Schmid-Saugeon, J. Guillod, and J. Thiran. Towards a computer-aided diagnosis system for
pigmented skin lesions. Computerized Medical Imaging Processing and Graphics, 27:65–78,
2003.
[44] M. Silveira, J. Nascimento, J. Marques, A. Marçal, T. Mendonça, S. Yamauchi, J. Maeda, and
J. Rozeira. Comparison of segmentation methods for melanoma diagnosis in dermoscopy images. IEEE Journal of selected topics in signal processing, (1), 2009.
[45] N. Situ, X. Yuan, J. Chen, and G. Zouridakis. Malignant melanoma detection by bag-of-features
classification. In EMBC, 2008.
[46] C. Steger. An unbiased detector of curvilinear structures. IEEE Trans Pattern Analysis Machine
Intell, 20(2), 1998.
[47] W.V. Stoecker, K. Gupta, R.J. Stanley, and et al. Detection of asymmetric blotches in dermoscopy
images of malignant melanomas using relative color. Skin Research and Technology, 11(3):179–
184, 2005.
[48] W.V. Stoecker, W.V. Lee, and R.H. Moss. Automatic detection of asymmetry in in skin tumors.
Computerized Medical Imaging and Graphics, 16(3):191–197, 1992.
[49] W.V. Stoecker, M. Wronkiewicz, R. Chowdhury, and et al. Detection of granularity in dermoscopy
images of malignant melanoma using color and texture features. Computerized Medical Imaging
and Graphics, 35:144–147, 2011.
[50] W. Stolz, A. Riemann, and A. B. Cognetta. ABCD rule of dermatoscopy: a new practical method
for early recognition of malignant melanoma. European Journal Dermatology, (4):521–527,
1994.
[51] G. Turin. An introduction to matched filters. IRE TRANSACTIONS ON INFORMATION THEORY,
pages 311–329, 1960.
[52] R. Turner. Texture discrimination by gabor functions. Biological Cybernetics, (55):71–82, 1986.
78
[53] H.L. Van Trees. Detection, Estimation and Modulation Theory. John Wiley & Sons, Inc., 2001.
[54] P. Viola and M. Jones. Robust real-time face detection. International Journal of Computer Vision,
57(2):137–154, 2004.
[55] S. Yoshino, T. Tanaka, M. Tanaka, and H. Oka. Application of morphology for the detection of
dots in tumors. In SICE Annual conference in Saporo, 2004.
1
A
Inpaiting Algorithm
A-1
Consider I0 (x, y) as the original gray image that has a region Ω to be inpainted. The inpainting
algorithm will create a family of I n (x, y) images, such that I 0 (x, y) = I0 (x, y) and limn→∞ I n (x, y) =
IR (x, y), where IR is the final output of the algorithm. This algorithm can then be written as
I n+1 (x, y) = I n (x, y) + 4tItn (x, y), ∀(x, y) ∈ Ω
(A.1)
where n denotes the inpainting ”time”,(x, y) are the pixel coordinates, 4t is the rate of improvement
and Itn (x, y) stands for the update of the image I n . With this equation, the image I n+1 is an improved
version of I n , with the ”improvement” given by Itn (x, y)
Itn (x, y) = ∇Ln (x, y) · N~n (x, y)
(A.2)
where, considering that L~n (x, y) is the information that has to be propagated from outside Ω into Ω,
∇Ln (x, y) is a measure of the change in that information and N~n (x, y) is the propagation direction.
The algorithm will converge when ∇Ln (x, y)· N~n (x, y) = 0, i.e., when the information has been exactly
~.
propagated in the direction N
The propagation must be a smooth process, so L~n (x, y) must be an image smoothness estimator.
Based on that, L~n (x, y) will be described by a simple Laplacian
∂ 2 I n (x, y) ∂ 2 I n (x, y)
L~n (x, y) =
+
∂x2
∂y 2
(A.3)
The propagation direction N~n (x, y) changes with each iteration and it is given by
N~n (x, y) = ∇⊥ I n (x, y)
N~n (x, y) is then defined as a time varying estimation of the isotopes direction field.
A-2
(A.4)

Thesis Title

Transcription

Similar documents

criteria for melanocytic lesions

BIOLOGY DAY

Buschke-Ollendorff syndrome accidentally diagnosed after a left

EConversion_MRT brochure_E

Endoscopic Tattoo of the Colon might be Standardized to

experienced. focused. evolving. just like you.

Oral white and red lesions

Untitled - Melanie Grant

A Phlebolith in the Anterior Portion of the Masseter Muscle

MEF Receives Grants to Provide Facial Skin Analyzer Machines to