Underspecification in RC attachment: A speed

Comments

Transcription

Underspecification in RC attachment: A speed
Underspecification in RC attachment: A speed-accuracy
tradeoff analysis
Pavel Logačev and Shravan Vasishth
University of Potsdam
[email protected]
March 13, 2014
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
1 / 23
Overview
1
Two models of ambiguity resolution
Underspecification
Unrestricted Race Model (URM)
2
Experiment
Method
Predictions
Results
3
Asymptotic accuracy and attention
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
2 / 23
Ambiguity Advantage
Traxler et al. (1998): Eye-tracking experiment
(1)
a. high attachment
The driver of the car [that had the moustache] was pretty cool.
b. low attachment
The car of the driver [that had the moustache] was pretty cool.
c. globally ambiguous
The son of the driver [that had the moustache] was pretty cool.
moustache was read faster in condition (c) than in (a) or (b)
several replications (e.g., van Gompel et al., 2001, 2005)
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
3 / 23
Underspecification
Swets et al. (2008): Underspecification
When the parser detects an ambiguity, it disambiguates only if the
task demands require it.
One processing stage is omitted in ambiguous conditions.
→ Minimum processing time in ambiguous condition smaller than in
unambiguous conditions.
time
...
check
check potential
potential
attachment
attachment sites
sites attach high
...
High attachment
attach low
...
Low attachment
check potential attachment sites
...
check potential
NP1?
attachmentNP2?
sites ...
check potential
attachment sites Pavel Logačev and Shravan Vasishth
...
Underspecified RC attachment
Ambiguous
March 13, 2014
4 / 23
Underspecification
Swets et al. (2008): Underspecification
When the parser detects an ambiguity, it disambiguates only if the
task demands require it.
One processing stage is omitted in ambiguous conditions.
Pr(completion ≤ t)
→ Minimum processing time in ambiguous condition smaller than in
unambiguous conditions.
1.00
attachment
0.75
ambiguous
0.50
high
0.25
low
0.00
400
600
800
1000
time (ms)
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
5 / 23
Unrestricted Race Model (URM)
Van Gompel et al. (2001): Unrestricted Race Model
The parser tries to build two parses simultaneously.
Whichever parse is constructed first, wins the race.
Attachment completion times vary from trial to trial.
Probability of finishing early is higher if two processes are involved,
because their finishing times are independent.
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
6 / 23
Unrestricted Race Model (URM)
Van Gompel et al. (2001): Unrestricted Race Model
time
Trial 1
time
Trial 2
Low
Trial 1
time
High
High
attachment
High
High
Low
Trial 1
Pavel Logačev and Shravan Vasishth
Low
Low attachment
Trial 2
High
Low
Ambiguous
Trial 2
Underspecified RC attachment
March 13, 2014
7 / 23
Unrestricted Race Model (URM)
Van Gompel et al. (2001): Unrestricted Race Model
The parser tries to build two parses simultaneously.
Whichever parse is constructed first, wins the race.
Attachment completion times vary from trial to trial.
Pr(completion ≤ t)
Probability of finishing early is higher if two processes are involved,
because their finishing times are independent.
1.00
attachment
0.75
ambiguous
0.50
high
0.25
low
0.00
400
600
800
1000
time (ms)
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
8 / 23
Experiment: Stimuli
a.
[high]
Was dachte die Managerin
des
Sängers,
die
schwieg?
What thought the manager.fem of the singer.masc, who.fem was silent
b.
[low]
Was dachte der Manager
der
Sängerin, die
schwieg?
What thought the manager.masc of the singer.fem, who.fem was silent
c.
[ambiguous]
Was dachte die Managerin
der
Sängerin, die
schwieg?
What thought the manager.fem of the singer.fem, who.fem was silent
d.
*[ungrammatical]
Was dachte der Manager
des
Sängers,
die
schwieg?
What thought the manager.masc of the singer.masc, who.fem was silent
‘What did the manager of the singer who was silent think?’
Four further conditions, with masculine relative pronouns
Thirty-two sentence sets
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
9 / 23
Experiment: Method
Participants judged the
acceptability of sentences at
predetermined lags.
14 signal tones on every trial,
with 400 ms SOA. (MR-SAT)
We cannot measure the
probability of successful
completion, but we can
measure accuracy
+
Was
450ms
{
der Manager
der Sängerin
die schwieg?
→ Bias-free measure of accuracy
at different lags:
d 0 = z(hits) − z(false alarms)
Pavel Logačev and Shravan Vasishth
dachte
Underspecified RC attachment
400ms
{
March 13, 2014
10 / 23
Experiment: Method
d'
1.5
●
1.0
0.0
●
●
0
●
asymptote
●
●
●
●
●
●
intercept
0.5
●
●
1/rate
1
2
3
4
time (sec)
Intercept in d 0 reflects the earliest time at which processing can have
finished
Asymptote in d 0 reflects the level of accuracy achieved in a particular
condition
Rate in d 0 has no straightforward to the rate in %, because the
relationship between % correct and d 0 is nonlinear
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
11 / 23
Experiment: Stimuli
20 participants (2 excluded).
192 acceptable experimental
sentences (64 per attachment
condition).
Pavel Logačev and Shravan Vasishth
192 unacceptable sentences
(unacceptable sentences
repeated for counterbalancing).
128 filler sentences
Underspecified RC attachment
March 13, 2014
12 / 23
Experiment: Predictions
Underspecification
Lower intercept in the ambiguous condition.
URM
Intercept in the ambiguous condition equal to the unambiguous
condition(s).
Pr(completion ≤ t)
Underspecification I
URM I
1.00
attachment
0.75
ambiguous
0.50
high
0.25
low
0.00
400
600
800
1000
400
600
800
1000
time (ms)
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
13 / 23
Experiment: Results
3
●
●
●
●
●
●
●
sensitivity (d')
●
●
2
condition
●
●
ambiguous
high attachment
●
1
0
low attachment
●
●
0
1
2
3
4
time (sec)
intercept differences
low − ambiguous
●
high − ambiguous
●
● ●
−200
●●
●
●
●
● ●
● ● ●● ●
●
●
●
●●
●
●●
●
● ●●
0
200
●●
●
● ●
400
time (ms)
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
14 / 23
Experiment: Results
Accuracy at the latest lag
% acceptable
d0
ambiguous
0.91 (0.07)
2.9 (0.1)
high
0.75 (0.1)
2.1 (0.1)
low
0.8 (0.09)
2.3 (0.1)
none
0.11 (0.07)
asymptote differences
low − ambiguous
high − ambiguous
●
●
●
●●
−2
●
●●
●●
● ●
● ●
●
● ●
●
●● ●● ●
● ●
●●●
●
●
● ● ●
−1
●
●
0
sensitivity (d')
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
15 / 23
Discussion
No significant differences in intercepts: No evidence for
underspecification.
Significantly higher asymptote in the ambiguous condition.
Higher asymptotes in SAT tend to correspond to faster reading in other
paradigms.
→ Higher asymptote seems to correspond to the ambiguity advantage.
Model comparison on the basis of BIC: the same results
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
16 / 23
Asymptote differences in the URM
RC attachment may be fallible (e.g., due to retrieval from memory)
When retrieval of an NP fails, attachment fails and the respective
attachment is considered impermissible.
Assuming that each attachment process has a probability of failing:
Pr(completion ≤ t)
pfail
ambiguous
= pfail
high
· pfail
low
URM II
1.00
0.75
0.50
0.25
0.00
attachment
ambiguous
high
low
400
600
800
1000
time (ms)
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
17 / 23
Asymptote differences in Underspecification
Checking possibility of RC attachment may also be subject to failure.
When retrieval of an NP fails, the parser could consider the
corresponding attachment impermissible.
→ Predicts higher asymptotes in the ambiguous condition.
time
check potential attachment sites
...
check
check potential
potential
NP1?
attachment
attachmentNP2?
sites
sites attach high
...
High attachment
attach low
...
Low attachment
check potential attachment sites
...
check potential
NP1?
attachmentNP2?
sites check potential attachment sites
...
check potential
NP1?
attachmentNP2?
sites Pavel Logačev and Shravan Vasishth
...
Underspecified RC attachment
Ambiguous
March 13, 2014
18 / 23
Underspecification does not need Underspecification
Higher asymptotes tend to correspond to faster reading (e.g., Martin
& McElree, 2008; Van Dyke & McElree, 2011).
Predicted difference in asymptotes explains the ambiguity advantage.
Underspecification requies a processing step (checking attachment
sites), which can explain the ambiguity advantage.
→ Underspecification is not necessary to explain the speedup in reading.
time
check potential attachment sites
...
check
check potential
potential
NP1?
attachment
attachmentNP2?
sites
sites attach high
...
High attachment
attach low
...
Low attachment
attach
...
Ambiguous
check potential attachment sites
...
check potential
NP1?
attachmentNP2?
sites check potential attachment sites
...
check potential
NP1?
attachmentNP2?
sites Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
19 / 23
Asymptotic accuracy and attention
% acceptable
ambiguous
0.91 (0.07)
high
0.75 (0.1)
low
0.8 (0.09)
ungrammatical
0.11 (0.07)
Attention-loss I
Participants sometimes ‘zone out’ for an entire trial, and try to guess
the correct answer
→ Predicts equal numbers of attention-related errors in all conditions
Attention-loss II
Because of our mode of presentation, participants may sometimes fail
to process the gender of a noun-phrase (due to e.g., momentary lack
of attention, or lack of time)
→ Predicts less errors in the ambiguous condition.
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
20 / 23
Asymptotic accuracy and attention
Parameters for six models estimated by-participant.
Models assuming that asymptote differences are attention-related
have much worse fits.
asymptotes
different
different
different
same
same
same
attention
attention-loss
attention-loss
attention-loss
attention-loss
attention-loss
attention-loss
Pavel Logačev and Shravan Vasishth
I
II
I+II
I
II
I+II
∆BIC
0
3
72
2823
901
930
Underspecified RC attachment
logLik
-151
-152
-148
-1639
-678
-654
npar
3 · 18
3 · 18
4 · 18
1 · 18
1 · 18
2 · 18
nselected
8
8
1
1
March 13, 2014
21 / 23
Conclusion
The ambiguity advantage corresponds to asymptote differences.
Ambiguous RCs are processed more successfully than unambiguous
RCs.
No evidence for differences in intercepts predicted by
underspecification.
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
22 / 23
Thank you.
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
23 / 23
d 0 as a measure of accuracy
Core assumptions of the
signal-detection theory (SDT)
The internal representation of a
feature is affected by random
(normally-distributed) noise.
A ‘yes’-response is made if the
noisy representation is above a
threshold (criterion c).
Sensitivity d 0 is a bias-free
measure of discrimination.
(Macmillan, 2000)
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
24 / 23
Parameter Estimation
The change in sensitivity (d 0 ) and in decision criterion (c) was
approximated by negatively accelerated exponential functions.
Together, these two functions predict the proportions of hits and false
alarms at any point in time.
d 0 (t) = asymptoted 0 (1 − e −rated 0 (t−interceptd 0 ) )
for t ≥ interceptd 0 , otherwise d 0 (t) = 0
c(t) = asymptotec (1 − e −ratec (t−interceptc ) ) + initialc
for t ≥ interceptc , otherwise c(t) = initialc
64 models (with and without differences between conditions in
intercept, rate, and asymptote) were fit to each parsticipant’s data
Maximum-likelihood estimation with a correlation parameter between
successive data points datapoints within one trial
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
25 / 23
Model Comparison
BIC = −2 · LL + log (nobs ) · npar
BF12 =
(1)
P(D|H1 )
P(D|H2 )
(2)
1
BF12 ≈ exp( ∆BIC21 )
2
Y
GBF12 =
BF12(k)
(3)
(4)
k
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
26 / 23
Model Comparison
64 models (with and without differences between conditions in
intercept, rate, and asymptote) were fit to each parsticipant’s data
MLE with a correlation parameter between successive data points
datapoints within one trial
Bayesian information criterion (BIC) for model selection and the BIC
approximation to the Bayes factor for inference (Wagenmakers, 2007)
BIC = −2 · log (Lik) + npar · log (nobs )
(5)
1
BF12 ≈ exp( ∆BIC21 )
2
(6)
Log-group Bayes Factor to aggregate evidence over participants
(Stephan & Penny, 2006).
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
27 / 23
Model Comparison: Best-ranked models according BIC
asymptote
rank
high
low
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
42.
-0.8
-0.5
-0.7
1/rate
high
intercept
low
high
low
∆LL
∆BIC
LGBF1
npar
99
0
-104
-133
-218
-61
10
-80
-83
-166
-9
-67
0
48
107
119
121
138
160
166
173
177
453
0
50
79
111
61
43
80
83
112
63
175
5
4
4
3
5
6
5
5
4
6
7
1307
-0.5
-0.8
-0.6
-0.4
-112
-64
-0.7
613
47
-0.6
1337
336
682
576
80
64
∆BIC: the model’s BIC minus the smallest BIC.
∆LL: the model’s log-likelihood minus the log-likelihood of the model selected according
to BIC.
LGBF values above 3 and 5 can be considered strong and very strong evidence,
respectively.
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
28 / 23
Model Comparison: By-participant
Numbers of participants for which particular parameterizations were
selected on the basis of BIC.
Brackets: Numbers of participants with a preference supported by a
Bayes factor of 3 or more.
contrast
1
2-high
2-low
3
Pavel Logačev and Shravan Vasishth
asymptote
7 (6)
4 (3)
1 (1)
6 (5)
rate
15 (11)
1 (1)
2 (2)
0
Underspecified RC attachment
intercept
16 (12)
0
2 (2)
0
March 13, 2014
29 / 23
Results: 1/rate Differences
invrate differences
low − ambiguous
high − ambiguous
●
●
●●
−1000
●
● ●
●
● ● ●●●●● ● ● ●
●
●
●
●●●
●
●● ● ● ●
0
●
●
●
1000
2000
time (ms)
Pavel Logačev and Shravan Vasishth
Underspecified RC attachment
March 13, 2014
30 / 23

Similar documents