the Presentation

A General Introduction to
Lexical Databases
Emmanuel Keuleers
Department of Experimental Psychology
Ghent University
EMLAR 2015 - Utrecht, April 15-17, 2015
• What can you find in a lexical database?
• How can you find it?
Lexical Databases
• Like a dictionary
• Lexical properties of interest to
psycholinguists
• Frequency, orthography, phonology,
morphology, syntax, …
• Subjective ratings of those words
• Behavioural responses to those data
Lexical Databases
• No standard: each database has its own
format, peculiarities, ...
• Text files, web interfaces, e-mail services,
etc ...
• In essence, a lexical database is just a list
with a bunch of information about words.
Lexical Databases
• The truth: you'll have to find out where
to find something and be prepared to do
some processing work.
CELEX:
the big and complex
lexical database
History
• Centre for Lexical Information
• Founded in Nijmegen in 1986
• Max Planck Institute for Psycholinguistics &
Interfaculty Research Unit for Language and
Speech of the University of Nijmegen (now CLS)
• Project ended in 2000
• Three large databases with lexical information for
Dutch, English, and German
• Dutch Database
• 124,136 lemmata
• 381,292 wordforms
• 211,389 corpus types
• English Database
• 52,446 lemmata
• 160,594 wordforms
• 220,271 corpus types
• German database
• 51,728 lemmata
• 365,530 wordforms
• 290,712 corpus types
Wordforms, lemmas,
and corpus types
Corpus types
• Letter strings, regardless of part of speech
• a walk in the park = to walk slowly = i walk alone = you walk alone
Wordforms
• Letter strings disambiguated for part of
speech (and sometimes meaning)
• a walk in the park ≠
to walk slowly ≠ i walk alone ≠ you walk alone
• (walk, noun, singular), (walk, verb,
infinitive), (walk, verb, 1p), (walk, verb, 2p)
Lemmas
• Headwords
• (walk, noun): a walk in the park = the
•
long walks
(walk, verb): I'm walking slowly = i walk
alone = he walks too fast
Celex Build Up
• Information from dictionary sources
• Corpus counts or correlation with existing
frequency counts
• Almost completely biased towards written
language
Dutch Database Sources
• Van Dale's Comprehensive Dictionary of Contemporary
Dutch (1984)
• 80,000 lemmata
• Word List of the Dutch Language ('Het Groene Boekje')
•
(1954), plus later revisions, including the 1994 spelling
reform
65,000 lemmata
The most frequent lemmata from the text corpus of the
Institute for Dutch Lexicology (INL) 42,380,000 words
in all
•
• 15,000 lemmata
English Database Sources
• Oxford Advanced Learner's Dictionary (1974)
• 41,000 lemmata
• Longman Dictionary of Contemporary English (1978)
• 53,000 lemmata
German Database Sources
• Bonnlex, supplied by the Institute for
Communication Research and Phonetics in
Bonn
• Molex, supplied by the Institute for German
Language in Mannheim
• Noetic Circle Services (MIT) German
spelling lexicon
Dutch Frequency Sources
• INL Corpus (42 million tokens)
• 930 entire fiction and non-fiction books
(approx. 30% fiction, 70% non-fiction)
published between 1970 and 1988.
Newspapers, magazines, children's books,
textbooks and specialist literature do not
feature in the collection.
English Frequency Sources
• COBUILD/Birmingham corpus (17.9 million
tokens)
• 16.6 million tokens from written texts
• 1.3 million tokens from transcribed
dialogue
German Frequency Sources
• Mannheimer Korpus I, Mannheimer Korpus II
and Bonner Zeitungskorpus 1
• 5.4 million tokens
• written texts like newspapers, fiction and nonfiction
• Freiburger Korpus
• 600,000 tokens
• transcribed speech
• Corpus Types
• Frequency
• Orthography
• Lemma lexica
• Frequency
• Orthography
• Phonology
• Derivational Morphology
• Grammatical information
• Wordform Lexica
• Frequency
• Orthography
• Phonology
• Inflectional Morphology
Frequency
Verb
Frequency
Deviation
Freq/Million
accept
3712
0
207.37
accord
2010
12
112.29
achieve
2121
0
118.49
act
2212
430
123.58
add
4190
0
234.08
agree
3424
0
191.28
Lexicon
Form
Frequency
Deviation
Frequency/
million
lemma
act
2212
430
123.58
wordform
act
269
1233
15.03
wordform
acted
92
366
5.14
wordform
acting
489
103
27.32
wordform
acts
187
80
10.45
wordform
act
269
1233
15.03
wordform
acted
92
366
5.14
wordform
act
269
1233
15.03
wordform
acted
92
366
5.14
wordform
act
269
1233
15.03
wordform
acted
92
366
5.14
wordform
acted
92
366
5.14
• Lemma frequency
• Frequency over all wordforms of the lemma
• Wordform frequency
• Deviation == 0 : exact count
• Deviation > 0 : result of disambiguation
• Less than 100 tokens
• Manual disambiguation
• More than 100 tokens
• Disambiguation on a sample of 100
tokens
• Frequency ± deviation = 95 % confidence
interval
• No disambiguation for verbal flection
• Frequency divided between forms
• Frequency Deviation > Frequency
• No disambiguation for German
• Frequency divided between forms
• English and German databases have
separate fields for written and spoken
frequencies
• Spoken frequencies based on very small
corpora
• 1.3 million for English
• 0.4 million for German
• What does it mean when an entry in CELEX has a
frequency of zero
• Many entries in the database sources were not
found in the frequency sources
• A few entries do not come from database
sources but are left with a zero frequency after
disambiguation
• will have deviation > zero
• Many entries added to CeLex for morphological
decomposition of other lemmas have a
frequency of zero
Word frequency
distributions
Word frequency distributions
word
frequency
rank
the
of
and
to
a
in
that
it
i
1 093 546
540 085
514 946
483 428
422 334
337 995
217 376
199 920
198 139
1
2
3
4
5
6
7
8
9
• Let's plot the rank of each word in the
COBUILD corpus against its frequency.
• The word with the highest frequency gets
the highest rank (1), the word with the
lowest frequency gets the lowest rank
(220,270).
• In total there are 17.9 million word tokens
in the COBUILD corpus.
• Not very clear.
• Let's plot it again so that the difference
between a frequency of 1 and a frequency
of 10 is the same as the difference between
a frequency of 10 and a frequency of 100
106=1,000,000
the
of
and
to
a
in
103=1,000
all these words have frequency 1
100=1
• Word frequency lists are composed of very
few words with a very high frequency
• Most words (corpus types) occur only
once in the corpus!
• The relation between word frequency and
rank is log linear.
Comparing frequencies
• Word frequencies from different databases
cannot be easily compared because of
different corpus sizes
• Example: Celex Dutch ±42m vs Celex
English ±18million
• Solution: frequency per million words
Frequency per million
word
frequency
frequency per million
rank
the
of
and
to
a
in
that
it
i
1 093 546
540 085
514 946
483 428
422 334
337 995
217 376
199 920
198 139
60 955.74
30 105.07
28 703.79
26 946.93
23 541.47
18 840.30
12 116.83
11 143.81
11 044.54
1
2
3
4
5
6
7
8
9
Comparing frequencies
• Beware! Some frequency lists contain
words with a frequency of 0
• Log10(0) is not something that can be
computed
• Solution: always add 1 to the raw
frequencies when you are transforming to
frequencies per million
Formula
Frequency per million = Raw Frequency +1
(adjusted) Corpus size in million
FPM ('that') = 217 376 +1 =12116.89
17.94
log10(12116.89)=4.08
Zipf Values
Van Heuven, Mandera, Keuleers, & Brysbaert (2014)
Formula
Freq. per billion = Raw Frequency +1
Corpus size in billion
FPB ('that') = 217 376 +1 =12116889.63
.01794
log10(12116889.63)=7.08
word
frequency
Relative Frequency
log10(fpm)
zipf
the
of
and
to
a
in
that
it
i
1 093 546
540 085
514 946
483 428
422 334
337 995
217 376
199 920
198 139
0.0602191
0.0297413
0.0283569
0.0266213
0.0232570
0.0186127
0.0119704
0.0110092
0.0109111
4.78
4.47
4.45
4.43
4.37
4.27
4.08
4.04
4.04
7.78
7.47
7.45
7.43
7.37
7.27
7.08
7.04
7.04
Orthography
• Lemma and wordform lexica list orthographic variants
with separate frequencies
Status
• Dutch: preferred, non-preferred, informal
• preferred & non-preferred: in “Groene Boekje”
non-standard forms occurring at least once in
• informal:
INL corpus
• English: British, American
• British: acceptable for British
• American: occurs only in American
• German
• No orthographic variants
Lemma ID
Form
Status
Frequency
1070
aardappelcroquet
preferred
0
1070
aardappelkroket
non-preferred
0
1138
aardelektrode
preferred
0
1138
aardelectrode
non-preferred
0
1202
aardolieprodukt
preferred
6
1202
aardolieproduct
non-preferred
0
1357
abductie
preferred
0
1357
abduktie
non-preferred
0
Lemma ID
Form
Status
Frequency
1359
anaesthesia
British
12
1359
anesthesia
American
1
1360
anaesthetic
British
47
1360
anesthetic
American
4
1361
anaesthetic
British
8
1361
anesthetic
American
0
1362
anaesthetist
British
16
• Abstract stems for Dutch
• if a stem with final s or f changes to z or
v anywhere in its inflectional paradigm, an
abstract stem is given ending with z or v.
Type
Stem
Abstract Stem
Adjective
approximatief
approximatiev
Noun
arbeidershuis
arbeidershuiz
Noun
arbeidersparadijs arbeidersparadijz
Noun
arbeidsbeurs
arbeidsbeurz
Adjective
arbeidsextensief
arbeidsextensiev
Adjective
arbeidsintensief
arbeidsintensiev
Adjective
arbeidsloos
arbeidslooz
Phonology
• Canonical phonetic transcriptions for
written forms
• English: primary and secondary
pronunciation
• Dutch, German: no phonetic variants
• Syllabified
• Stress and CV patterns
• IPA-like character sets
• SAM-PA
• CeLex
• CPA
• DISC character set
• one character per phoneme
• no ambiguity
• unreadable
Idnum
Spelling
Status
DISC
Syllables
IPA
Stress
CV
42577
sleekness
primary
sliknIs
sliːk-nɪs
10
CCVVCCVC
42577
sleekness
secondary
sliknIs
sliːk-nəs
10
CCVVCCVC
42582
sleepily
primary
slipIlI
sliː-pɪ-lɪ
100
CCVVCV-CV
42582
sleepily
secondary
slipIlI
sliː-pə-lɪ
100
CCVVCV-CV
42584
sleepiness
primary
slipInIs
sliː-pɪ-nɪs
100
CCVVCV-CVC
42584
sleepiness
secondary
slipInIs
sliː-pɪ-nəs
100
CCVVCV-CVC
• Dutch and German
• Separate phonetic trancsriptions for headwords
and stems
• English
• First variant is always the primary one, as listed
in the English Pronouncing Dictionary
• Newer versions use BBC English and Network
English, transcriptions in CeLex are probably RP.
• Phonological transcriptions for
morphologically complex Dutch and
German stems with indication of
morpheme boundaries
• Only with CELEX and CPA character sets
Stem
Phonological
Transcription
Phonetic
Transcription
Arbeiter
arbait+@r
[ar][bai][t@r]
Arbeitsplatz
arbait+s#plats
[ar][baits][plats]
Arbeitgeber
arbait#ge:b+@r
[ar][bait][ge:]
[b@r]
arbeitsamkeit
arbait#za:m#kait
[ar][bait][za:m]
[kait]
Morphology
• Lemma Morphology
• Morphstatus: indicates if the lemma has a
relevant morphological decomposition
• Segmentation
• Immediate, Flat, Hierachical
Immediate Segmentation
aansprakelijkheidsverzekering
aansprakelijkheid
s
verzekering
Flat Segmentation
aansprakelijkheidsverzekering
aan
spreek
elijk
heid
s
ver
zeker
ing
Hierachical Segmentation
aansprakelijkheidsverzekering
aansprakelijkheid
aansprakelijk
verzekering
aanspreek
aan
spreek
verzeker
elijk
heid
s
ver
zeker
ing
• Wordform morphology
• Inflectional Features
Form
Flection
Frequency
adjusted
past,1p,singular
35
adjuster
singular
5
adjusters
plural
1
adjusting
present,participle
71
adjustment
singular
150
adjustments
plural
84
adjusts
present,3p,singular
14
Grammatical
Information
• Syntactic class for lemmas
• Dutch: Expression, Noun, Adjective,
Quantifier/Numeral, Verb, Article,
Pronoun, Adverb, Preposition,
Conjunction, Interjection
• English: Noun, Adjective, Numeral,Verb,
Article, Pronoun, Adverb, Preposition,
Conjunction, Interjection, Single,
Complex, Letter, Abbreviation, Infinitival
• German
• Noun, Adjective, Quantifier/Numeral,
Verb, Article, Pronoun, Adverb,
Preposition, Conjunction, Interjection
• Additional subclassification
Form
Class
Subclasses
videotape
Noun
uncountable
videotape
Verb
transitive
vide supra
Interjection
vie
Verb
linking
Vietnam
Noun
proper
Vietnamese
Adjective
ordinary
Vietnamese
Noun
countable
Form
Class
Subclasses
magnetisch
Adjective
magnetiseren
magnetiseerb
aar
magnifiek
Verb
Adjective
nonadverbial
lexical intransitive
transitive
nonadverbial
Adjective
adverbial
Magyaars
Adjective
adverbial
maharadja
Noun
maharishi
Noun
Mahdi
Noun
Mahler
Noun
propername
How to get information
from CeLex
• Option 1: CeLex CD with textfiles
• Typical ‘text processing’ languages
(AWK, Perl)
• Elegant language (Python)
• Import in spreadsheet application
• Option 2: Public web interface at MPI
(WebCeLex)
• Good tool for selection
• Process with scripting language
Examples
cued by the orthography. Thus both MOUTH and TENTH
end in TH but differ in voicing; although both form the plural by adding S in the orthography, the inflections are pronounced differently. The mappings between spelling and
pronunciation for these words are therefore rather complex.
In summary, the words we excluded differ in some respects
from the words in the training corpus and raise additional
challenges for our approach that need to be addressed.
tates performance o
age of the space of
Performance o
and on inflected w
comparison we als
feedforward netwo
feedforward model
these words, with
models’ capacities
To explore these questions, we conducted a replicaing them on nonwo
tion simulation using
a much larger corpus. Monosyllables
Here the attractor
132
A. Albright, B. Hayes / Cognition 90 (2003) 119–161
were extracted from the CELEX electronic corpus (Baayen,
than the feedforwa
Table 2
Piepenbrock, & van Rijn, 1993). All items fitting a CCwith the conclusion
Past tenses for gleed derived by the analogical model
CVVCCC
template
were used, yielding 7,839 words. Most
learning complex
Output
Score
Analogs
of the additional words are inflected items. The phonologlearning of the larg
gleeded
0.3063
plead, glide, bleat, pleat, bead, greet, glut, need, grade, gloat, and 955 others in our
ical network was
expanded from 66 to 88 units to accommotion. The larger m
learning set
gled
0.0833
bleed, lead, breed, read, feed, speed, meet, bottle-feed
gleed
0.0175
bid, beat, slit, let, shed, knit, quit, split, fit, hit, and 12 others
gleet M. W. & Seidenberg,
0.0028 M. S.
lend,
build,
bend, send,
spend
Harm,
(1999).
Phonology,
reading
acquisition, and dyslexia: insights from connectionist models.
glade Rev, 106(3),
0.0025
Psychol
491-528. eat
glode
0.0017
weave, freeze, steal, speak
glud
0.0005
sneak
[skraI dt] are phonologically filtered; the remaining candidates are submitted to the core
GCM algorithm for evaluation, as described above.
As an illustration of how the model works, Table 2 shows the outcomes it derives for
gleed, along with their scores and the analog forms used in deriving each outcome.
To conclude, we feel that a model of this sort satisfies a rigorous criterion for being
“analogical”, as it straightforwardly embodies the principle that similar forms influence
one another. The model moreover satisfies the criteria laid out in Section 2.1: it is fully
susceptible to the influence of variegated similarity, and (unless the data accidentally help
it to do so) it utterly ignores the structured-similarity relations that are crucial to our rulebased model.
2.5. Feeding the models
We sought to feed both our rule-based and analogical models a diet of stem/past
tense pairs that would resemble what had been encountered by our experimental
participants. We took our set of input forms from the English portion of the CELEX
database (Baayen, Piepenbrock, & Gulikers, 1995), selecting all the verbs that had a
lemma frequency of 10 or greater. In addition, for verbs that show more than one past
tense (like dived/dove), we included both as separate entries (e.g. both dive-dived and
dive-dove). The resulting corpus consisted of 4253 stem/past tense pairs, 4035 regular
and 218 irregular. Verb forms were listed in a phonemic transcription reflecting
American English pronunciation.
A current debate in the acquisition literature (Bybee, 1995; Clahsen & Rothweiler,
1992; Marcus, Brinkmann, Clahsen, Wiese, & Pinker, 1995) concerns whether prefixed
forms of the same stem (e.g. do/redo/outdo) should be counted separately for purposes
Albright, A. & Hayes, B. (2003). Rules vs. analogy in English past tenses: A computational/experimental
of learning.
We prepared
a version of our learning set from which all prefixed forms
study. Cognition,
90(2), 119-161.
were removed, thus cutting its size down to 3308 input pairs (3170 regular, 138
irregular), and ran both learning models on both sets. As it turned out, the rule-based
model did slightly better on the full set, and the analogical model did slightly better on
the edited set. The results below reflect the performance of each model on its own best
learning set.
Feeding
the models
B. New et al. / 2.5.
Journal
of Memory
and Language 51 (2004) 568–585
573
withboth
Baayen
al.!s findings
Dutch, but models
deviates afrom
s
We sought to feed
ouretrule-based
andin analogical
diet of stem/past
Serenoresemble
and Jongman!s
findings
rds were selected from Lexique
and 44that would
tense pairs
what had
beenin English.
encountered by our experimental
the two
experiments
reported
thusof the CELEX
rds were constructed. Selection
andWe took So,
participants.
ouron
setthe
ofbasis
inputofforms
from
the English
portion
far,
it
seems
that
Dutch
and
French
plurals
are
ria were the same as in Experiment
1,
database (Baayen, Piepenbrock, & Gulikers, 1995), selecting all theproverbs that had a
the same way, and both differ significantly
quencies of the word forms. One list
lemma frequency ofcessed
10 oringreater.
In addition, for verbs that show more than one past
from the findings in English. In addition, there is some
singular frequency of 16, and a plural
tense (like dived/dove), we included both as separate entries (e.g. both dive-dived and
suggestive evidence that the Dutch/French pattern could
he other list had a singular frequency
The resulting
4253 stem/past
tenseand
pairs,
also becorpus
presentconsisted
in Italianof(Baayen
et al., 1996)
in 4035 regular
frequency of 4. The twodive-dove).
lists of words
irregular. Spanish
Verb forms
were listed
in a phonemic
reflecting
(Dominguez
et al., 1999),
making thetranscription
English
the number of letters (6.4and
and 218
6.3) and
American
English
pronunciation.
finding even more isolated. Therefore, we decided to relables (1.7 and 1.7). A complete list of
A currentB debatepeat
in the
(Bybee, 1995; Clahsen & Rothweiler,
the acquisition
Sereno and literature
Jongman experiments.
in Appendix A (see also Appendix
1992; Marcus, Brinkmann, Clahsen, Wiese, & Pinker, 1995) concerns whether prefixed
).
forms of the same stem (e.g. do/redo/outdo) should be counted separately for purposes
Experiment
3
of learning. We prepared
a version
of our learning set from which all prefixed forms
e was identical to thatwere
described
in thus cutting its size down to 3308 input pairs (3170 regular, 138
removed,
A closer
look
at Sereno
and Jongman
revealed
cept that in this experiment
only the
irregular),
and ran both
learning
models
on both
sets. As (1997)
it turned
out, the rule-based
a
number
of
methodological
differences
between
rms were presented. Because
of
this,
model did slightly better on the full set, and the analogical model didthat
slightly better on
study and all of the other studies. For a start, Sereno
ed in -s.
the edited set. The results below reflect the performance of each model on its own best
and Jongman presented their singular and plural stimuli
learning set.
in two different experiments (their Experiments 2A and
2B). This blocked presentation may have encouraged
Albright, A. & Hayes, B. (2003). Rules vs. analogy in English past tenses: A computational/experimental
participants
to ignore the end -s in the experiment with
ys mean reaction time and percentage
study. Cognition,
90(2), 119-161.
the plural stimuli. Another problem is that Sereno and
ent 2. Extreme reaction times were reJongman!s word frequencies were based on the Brown
to the procedure described in Expericorpus which only includes one million words. This is
5.5% of the RT data in the subjects
a quite limited corpus if we compare it to the French
s 4.7% in the item analysis were discorpus used in Lexique (31 million tokens) and the Enwith one repeated measure revealed
glish corpus used in Celex (17.9 million tokens). For
he frequency of the plural form both
these reasons, we decided to repeat the Sereno and Jongover participants (F1(1,14) = 24.09,
man experiments, following the same procedure as in
< .001) and in the analysis over items
our French studies (and in the Dutch studies).
Mse = 1371.50, p < .001). Participants
to singular word forms with high-freMethod
n to singular word forms with low-fre-
ng of Experiment 2 was the presence of
effect when the singular forms were
ce frequency. When two singular forms
rface frequency but differ in the freural forms, the singular with the more
processed faster. This result agrees
(in ms), standard deviation and percentage
2
m
Presented forms: singular
M
SD
%ER
al
540
50
2.1
al
r]
596
63
3.6
Participants
Thirty-eight students from Royal Holloway, University of London, took part in the experiment in return for
course credits. They were all native English-speakers and
had normal or corrected-to-normal vision.
Stimulus materials
The word stimuli were two lists of 24 nouns drawn
from the Celex database (Baayen, Piepenbrock, & Gulikers, 1995), based on a corpus of 16.6 million written
words and 1.3 million spoken words. The first list consisted of singular dominant items, with an average frequency of 25 per million for the singular forms and
eight for the plural forms. The second list consisted of
plural dominant items with average frequencies of 9
and 26, respectively. The base frequencies (34 vs. 35)
did not differ between the lists. The stimuli were further
matched on the number of letters (6.3 and 6.3) and the
number of syllables (2 and 2). A complete list of the
stimuli is presented in Appendix A. As in Experiment
1, two versions of the word list were created, so that
each participant saw only one form of a word.
New, B., Brysbaert, M., Segui, J., Ferrand, L., & Rastle, K. (2004). The processing of singular
and plural nouns in French and English. Journal of Memory and Language, 51(4), 568-585.
Appendix A (continued)
Word [in English]
Mean reaction time
pavilion [bungalow]
perruque [wig]
prêtre [priest]
582 [river]
rivière
sculpteur [sculptor]
tige
[stem] A (continued)
Appendix
torse [chest]
Word [in English]
vallée [valley]
491
71
17
536
58
4
635
178
20
B. New et al. / Journal of Memory
584
107 and Language
34 51 (2004) 568–585
625
95
5
578
84
12
618
113
12
Mean reaction time
SD
Singular frequency
520
76
26
pavilion [bungalow]
perruque [wig]
prêtre [priest]
rivière
[river]
Materials
used in Experiment 3
sculpteur [sculptor]
Word
Singular
tige [stem]
Mean reaction
torse [chest]
vallée
[valley]
Singular dominant items
491
536
635
584
625
578
time
618
520
beast
468
belief
439
cathedral
476
clinic
480
Materials used in Experiment 3
dragon
482
famine
525
Word
Singular
hat
422
Mean reaction time
journal
495
Singular dominant603
items
lieutenant
monument
547
beast
468
moustache
519
belief
439
prophet
532
cathedral
476
regiment
586
clinic
480
salad
490
dragon
482
sister
463
famine
525
studio
450
hat
422
sum
480
journal
495
sword
467
lieutenant
603
talent
473
monument
547
task
473
moustache
519
texture
433
prophet
532
tribe
478
regiment
586
valley
483
salad
490
sister
Plural dominant items463
studio
450
acre
584
sum
480
ancestor
569
biscuit
434
sword
467
critic
546
talent
473
disciple
560
task
473
dollar
514
texture
433
glove
455
tribe
478
heel
490
valley
483
ingredient
lip
Plural dominant
molecule
acre
neighbour
ancestor
nostril
biscuit
sandal
critic
shoe
disciple
sock
dollar
soldier
glove
heel
ingredient
lip
molecule
neighbour
nostril
sandal
shoe
sock
soldier
SD
SD
68
43
68
65
43
52
33
SD
78
67
112 68
88 43
90 68
126 65
78 43
59 52
67 33
90
78
73
67
67
112
68
45 88
65 90
85 126
78
59
67
90
73
67
68
45
65
85
Singular frequency
71
58
178
107
95
84
Frequency113
76
17
4
20
34
5
Plural
12
Mean reaction
time
12
26
17
67
15
15
8
7
53
18
14
11
16
10
10
16
82
22
32
13
24
65
11
23
49
516
477
553
548
495
642
439
456
609
588
547
606
572
470
489
498
483
490
506
468
490
505
495
Frequency
17
67
15
15
8
7
53
18
14
11
16
10
10
16
82
22
32
13
24
65
11
23
49
535
444
items
498
584
466
569
562
434
536
546
438
560
434
514
466
101
77
43
92
114
65
59
94
75
50
57
101
67
71 77
81 43
64 92
56 114
35 65
15
6
5
12
4
15
5
11
4
17
5
15
19
2 6
1 5
14 12
3 4
26 15
455
490
535
444
498
466
562
536
438
434
466
59
94
75
50
57
67
71
81
64
56
35
5
11
4
17
5
19
2
1
14
3
26
559
568
458
534
533
503
441
479
553
482
515
475
551
516
448
441
448
Plural frequency
559
568
458
534
533
503
441
479
553
482
515
475
551
516
448
441
448
85
95
85
85
137
76
50
48
110
67
83
61
87
111
84
64
42
Plural frequency
4
2
13
8
3
8
Frequency
2
6
SD
96
58
83
77
80
145
Plural
51
Mean reaction time
52
132
110
516
106
477
108
553
133
548
63
495
52
642
81
439
98
456
78
609
107
588
74
547
65
606
78
572
91
470
489
498
483
490
506
468
490
505
495
4
2
13
8
3
8
2
6
SD
96
58
83
77
80
145
51
52
132
110
106
108
133
63
52
81
98
78
107
74
65
78
91
85
95
85
85
137
76
50
48
110
67
83
61
87
111
84
64
42
11
24
3
5
2
1
15
6
1
6
2
6
2
4
32
6
17
4
12
17
2
15
7
23
22
11
23
13
53
15
18
11
61
12
31
10
8
65
16
57
Frequency
11
24
3
5
2
1
15
6
1
6
2
6
2
4
32
6
17
4
12
17
2
15
7
23
22
11
23
13
53
15
18
11
61
12
31
10
8
65
16
57
Exercises and practical applications
http://crr.ugent.be/emlar2015