On Road – Simulator comparison

Transcription

Appendices for Technical Report 2013-17A
The Effects of a Production Level “Voice-Command”
Interface on Driver Behavior: Reported Workload,
Physiology, Visual Attention, and Driving Performance
Limited Release Version 2013-17 Issued: November 4, 2013
Minor Revisions for Public Version 2013-17A: November 18, 2012
Bryan Reimer, Bruce Mehler, Jonathan Dobres &
Joseph Coughlin
Massachusetts Institute of Technology > AgeLab > 77 Massachusetts Ave, E40-291, Cambridge, MA 02139
Phone: 617.452.2177 > fax: 617.258.7570 > [email protected] > agelab.mit.edu
AgeLab Technical Report 2013-17A Appendices (draft)
Table of Contents
APPENDIX A: GLANCE-TO-DEVICE ANALYSIS ......................................................... 7
Introduction .......................................................................................................................... 7
Methods ................................................................................................................................ 7
Results .................................................................................................................................. 7
Selected Glance Metrics Summary Table (Glance-to-Device Analysis) .............................. 8
Mean Glance-to-Device Duration ....................................................................................... 9
Percentage of Long Duration (> 2s) Glances .....................................................................11
Total To-Device Glance Time ............................................................................................14
Number of To-Device Glances ..........................................................................................17
Glance Duration Analysis ..................................................................................................19
Effect of Task Completion Time.........................................................................................20
APPENDIX B: ERROR-FREE TASK ANALYSIS ......................................................... 21
Introduction .........................................................................................................................21
Methods ...............................................................................................................................21
Results .................................................................................................................................22
Self-Reported Workload ....................................................................................................22
Total Completion Time ......................................................................................................25
Heart Rate .........................................................................................................................27
Skin Conductance (SCL) ...................................................................................................29
Mean Velocity....................................................................................................................31
Variability of Velocity .........................................................................................................33
Acceleration Events...........................................................................................................35
Steering Wheel Angle ........................................................................................................37
Minor Wheel Reversals .....................................................................................................39
Major Wheel Reversals .....................................................................................................41
Selected Glace Metrics Summary Table (Error Free Cases) .............................................43
Mean Off-Road Glance Duration .......................................................................................44
Total Off-Road Glance Time ..............................................................................................48
Number of Glances ...........................................................................................................50
Orienting Response...........................................................................................................52
APPENDIX C: TRIAL COMPARISON ANALYSIS ....................................................... 53
Introduction .........................................................................................................................53
Methods ...............................................................................................................................53
Results .................................................................................................................................54
© MIT AgeLab 2013
Page A2 of 293
Task Completion Time ......................................................................................................54
Heart Rate .........................................................................................................................57
Skin Conductance (SCL) ...................................................................................................60
Mean Velocity....................................................................................................................63
Variability of Velocity .........................................................................................................66
Acceleration Events...........................................................................................................69
Steering Wheel Angle ........................................................................................................72
Minor Wheel Reversals .....................................................................................................75
Major Wheel Reversals .....................................................................................................78
Selected Glance Metrics Summary Table (First & Second Trails) ......................................81
Mean Glance Duration ......................................................................................................82
Total Off-Road Glance Time ..............................................................................................88
Number of Glances ...........................................................................................................91
Orienting Response...........................................................................................................94
Task Completion Data .......................................................................................................96
Glance Distribution Analyses .............................................................................................98
APPENDIX D: ORIENTING RESPONSE CODING ...................................................... 99
Introduction .........................................................................................................................99
Methods ...............................................................................................................................99
Coding Guide ......................................................................................................................99
Results ...............................................................................................................................100
APPENDIX E: BASELINE PERIOD ANALYSIS ........................................................ 104
Introduction .......................................................................................................................104
Methods .............................................................................................................................104
Results ...............................................................................................................................105
Consistency Across Baseline Periods .............................................................................105
Off-Road Glance Behavior ..............................................................................................106
A Visualization of Glance Time by Object across the Seven Baseline Periods ................119
Individual Participant Data ...............................................................................................120
APPENDIX F: GLANCE CODING GUIDELINES & PROCEDURES.......................... 127
Introduction .......................................................................................................................127
Methods .............................................................................................................................127
Video Recording ..............................................................................................................127
Annotator Software..........................................................................................................127
Coding & Mediation .........................................................................................................128
Data Storage & Reduction ...............................................................................................129
© MIT AgeLab 2013
Page A3 of 293
Glance Coding Guide ......................................................................................................129
APPENDIX G: TASK PERFORMANCE CODING GUIDELINES ............................... 132
Introduction .......................................................................................................................132
Methods .............................................................................................................................132
Coding Guide ....................................................................................................................133
Data ....................................................................................................................................134
APPENDIX H: DETAILED EXPERIMENT PROTOCOL & TASK SCRIPTS .............. 138
Voice Based In-Vehicle Systems Study Experimental Checklist ...................................139
Section 1 - Intake...............................................................................................................139
Protocol Summary ...........................................................................................................140
Phase 1-1. Inside Set-Up Before the Participant Arrives (RA1) ......................................142
Phase 1-2. When the Participant Arrives ........................................................................143
Phase 1-3. Study Overview .............................................................................................144
Phase 1-4. Consent Forms..............................................................................................145
Phase 1-5. Driver’s License .............................................................................................146
Phase 1-6. Payment and Emergency Contact Forms ......................................................146
Phase 1-7. Review of Initial Screening Criteria ................................................................147
Phase 1-8. Mental Status Screening (Montreal Cognitive Assessment - MoCA) .............150
Phase 1-9. Verification of Eligibility Review .....................................................................154
Phase 1-10. Checklist for Participant Consent & Verification ...........................................154
Phase 1-11. Pre-Experimental Questionnaire.................................................................155
Phase 1-12. N-Back Training .........................................................................................156
Phase 1-13. Instructions on How to Fill-Out Workload Rating Scale ...............................159
Phase 1-14. Bathroom Break Opportunity ......................................................................159
Phase 1-15. Physiology Sensor Attachment ...................................................................160
Phase 1-16. Physiology Sensor Verification (NeuGraph)................................................162
Phase 1-17. Moving to Car .............................................................................................164
Section 2 Vehicle Setup ....................................................................................................165
Support Materials: ...........................................................................................................167
Protocol Summary: ..........................................................................................................167
Phase 2-1. Checklist for Setting-up Vehicle and Eye Tracking System...........................168
Phase 2-2. Setting Participant Up in Vehicle ..................................................................170
Phase 2-3. Back Out & Back into Parking Space............................................................172
Phase 2-4. Create Head Model for Eye Tracker .............................................................173
Phase 2-5. In-Vehicle N-back Practice ...........................................................................175
Phase 2-6. Final Eye Tracking / Camera Configuration ..................................................176
Phase 2-7. Training in MIT parking lot on first set of in-vehicle tasks ..............................177
© MIT AgeLab 2013
Page A4 of 293
Phase 2-8. Voice System Calibration .............................................................................178
Section 3 Leaving the Parking Lot ...................................................................................179
Phase 3-1. Introduction to Driving ..................................................................................180
Phase 3-2. Start Driving .................................................................................................182
Section 4 Approaching Rest Stop ....................................................................................185
Section 5 - Leaving Rest Stop ..........................................................................................187
Section 6 Return to MIT ....................................................................................................189
Phase 6-1. Return Prep ..................................................................................................189
Phase 6-2. Post drive .....................................................................................................191
Phase 6-3. In 290 or Other .............................................................................................192
Phase 6-4. Data Clean-Up to be Completed after Participant Leaves.............................193
Phase 6-5. Notes on Training and Evaluation Issues .....................................................196
Section A_t1 Radio Manual Training - MIT parking lot ...................................................197
Section A_t2 Radio Manual Training -495 Training.........................................................198
Section A_e Radio Manual Evaluation .............................................................................199
Section B_t Navigation Training ......................................................................................203
Section B_e - Navigation Evaluation ...............................................................................204
Section C_t1 Radio Voice Training - MIT parking lot ......................................................208
Section C_t2 Radio Voice Training – 495 Stop................................................................215
Section C_e Radio Voice Evaluation ...............................................................................221
Section D_t Song Selection Training ...............................................................................226
Section D_e – Song Selection Evaluation .......................................................................229
Section E_t Phone Task Training .....................................................................................232
Section E_e Phone Task Evaluation ................................................................................235
Phase E-1. Phone Task Part I .........................................................................................237
Phase E-2. Phone Task Part II: .......................................................................................238
Phase E-3. Phone Task II-B ............................................................................................239
Phase E-4. Phone Task Part III: ......................................................................................241
Section X_1 – N-back Evaluation Manual Radio / Nav ....................................................242
Section X_2 – N-back Evaluation Voice Radio / Song ....................................................244
Appendix I: Self-Reported Workload Materials ...................................................... 246
Instructions .......................................................................................................................247
Example Rating Sheet.......................................................................................................248
Appendix J: Questionnaires ..................................................................................... 250
Pre-Experiment Questionnaire .........................................................................................250
Supplemental Health Questionnaire ................................................................................261
Task Rating Questionnaire ...............................................................................................265
© MIT AgeLab 2013
Page A5 of 293
Post-Experimental Questionnaire ....................................................................................270
Appendix K: Questionnaire Data ............................................................................. 275
Pre-Experiment Questionnaire .........................................................................................275
Task Rating Questionnaire ...............................................................................................279
Post-Experiment Questionnaire I: Q1-Q21 ......................................................................281
Post-Experiment Questionnaire II: Q22-Q25 ...................................................................283
Supplemental Health Questionnaire ................................................................................290
Appendix L: Misc. Information ................................................................................. 293
Rank Ordering of Reasons for Excluding Participants ..................................................293
Note: These appendices were developed as part of an integrated technical report and should be
considered as part of a single document for citation purposes even if the main body of the
report and the appendices appear as separate files due to size considerations when supplied in
electronic form.
© MIT AgeLab 2013
Page A6 of 293
APPENDIX A: GLANCE-TO-DEVICE ANALYSIS
Introduction
NHTSA’s visual-manual distraction guidelines define an off-road glance as any glance “away
from the forward roadway” (Section VI.F.1, p. 273). This includes glances to the vehicle’s
mirrors. NHTSA’s rationale for classifying mirror glances as “off-road” comes from the
assumption that, “Due to the [simulated] driving scenario [described in the guidelines], eye
glances to the side of the roadway or to the vehicle’s mirrors are expected to be minimal”
(Section VI.F.1, p. 273). However, NHTSA’s recommended simulated driving scenario may not
always reflect a realistic driving scenario, where glances to the mirrors are often necessary and
more frequent.
Since this experimental design utilized an on-road driving protocol, and not a simulator,
glances to the mirrors are likely to be substantially more frequent than NHTSA’s guidelines
might suggest. Therefore, the following section re-analyzes the glance metrics from the Primary
Analysis, but considers only glances to the in-vehicle device, rather than the broader category of
all off-road glances. It is important to note that is approach is consistent with methods proposed
by The Alliance (2006).
Methods
The methodology is the same as described in Primary Analysis Methods. However, off-road
glance metrics in this section consider only glances to the in-vehicle device (glances to the
mirrors and other non-device areas are not considered). Measures that would not be affected by
the change in glance classification are not considered here. These include: self-reported task
difficulty, heart rate, skin conductance, driving behavior metrics, task completion time, task
completion ability, and orienting response data. Only glance-relevant data are presented.
Results
A summary table for the Glance-to-Device analysis appears on the next page.
© MIT AgeLab 2013
Page A7 of 293
Selected Glance Metrics Summary Table (Glance-to-Device Analysis)
If one were to apply NHTSA distraction cutpoints to participants in the younger, older, and overall
cohort, the table below ( Table A-1) shows the percentage who would meet each of the criteria if a glanceto-device analysis is used instead of glances-off-the-forward-roadway. Entries for cases where less than
85% of a group meet a threshold are bolded and shown in red.
Task
Age
Group
Long Duration
Glances
Mean Glance
Duration
Total ToDevice Glance
Time
Nav Cancel
Younger
100.00%
100.00%
100.00%
Older
100.00%
100.00%
91.30%
(all)
100.00%
100.00%
96.20%
Younger
100.00%
100.00%
26.70%
Older
100.00%
100.00%
26.10%
(all)
100.00%
100.00%
26.40%
Younger
90.00%
96.70%
100.00%
Older
82.60%
100.00%
91.30%
(all)
86.80%
98.10%
96.20%
Younger
96.70%
100.00%
73.30%
Older
87.00%
100.00%
8.70%
(all)
92.50%
100.00%
45.30%
Younger
100.00%
100.00%
100.00%
Older
100.00%
100.00%
82.60%
(all)
100.00%
100.00%
92.50%
Younger
100.00%
100.00%
96.70%
Older
100.00%
100.00%
78.30%
(all)
100.00%
100.00%
88.70%
Younger
96.70%
100.00%
86.70%
Older
95.70%
100.00%
60.90%
(all)
96.20%
100.00%
75.50%
Younger
93.30%
100.00%
30.00%
Older
100.00%
100.00%
60.90%
(all)
96.20%
100.00%
43.40%
Younger
100.00%
100.00%
96.70%
Older
100.00%
100.00%
72.70%
(all)
100.00%
100.00%
86.50%
Nav Entry
Radio Manual Easy
Radio Manual Hard
Radio Voice Easy
Radio Voice Hard
Song Select
Song Fail
Phone
© MIT AgeLab 2013
Page A8 of 293
Mean Glance-to-Device Duration
Figure A-1: Mean duration of to-device glances.
These plots present the data from the off-road analysis (dark bars, replicated from the data
shown in the previous section) alongside data from a to-device analysis. The mean duration of
to-device glance is almost always kept to less than 2.0 seconds. Note the nearly complete
absence of to-device glances during the N-Back tasks, as these tasks did not utilize the in-vehicle
device.
© MIT AgeLab 2013
Page A9 of 293
Figure A-2: Statistical summary plot for mean duration of to-device glances.
Mean glance duration was not significantly affected by gender (p = .099). However, age group
significantly affected mean glance duration (p = .001), as did comparisons between the radio
tasks, and a comparison between the radio manual task and navigation entry task (p < .001 for
both).
© MIT AgeLab 2013
Page A10 of 293
Percentage of Long Duration (> 2s) Glances
Figure A-3: Percentage of glances greater than 2 seconds towards the in-vehicle device.
Note that long glance rates for the Radio Manual Easy task (Radio Easy M) may be inflated due
to the task’s relatively brief duration. Although this task’s mean glance rate skews fairly large,
its 85th percentile is still below the NHTSA criterion line for our sample.
© MIT AgeLab 2013
Page A11 of 293
Figure A-4: Statistical summary plots for percentage of glances longer than 2 seconds to the invehicle device.
Male participants had a significantly higher proportion of long glances compared to females (p
= .049). Older participants had a significantly higher proportion of long glances compared to
younger participants (p = .020). Lastly, the percentage of glances longer than 2 seconds varied
significantly between radio tasks and between the radio manual and navigation entry tasks (p <
.001 for both). This may be because the task was relatively brief and was typically performed in
© MIT AgeLab 2013
Page A12 of 293
a single motion (turning a dial to the correct station), necessitating a low number of glances
overall.
© MIT AgeLab 2013
Page A13 of 293
Total To-Device Glance Time
Figure A-5: Total time spent glancing to the in-vehicle device. One outlier data point on the Nav
Entry task is excluded from view to aide plot readability. The blue horizontal line represents
The Alliance’s criterion for total glance time (criterion 2.1 A), which is 20 seconds.
The dark gray bars are reproduced from the plots in the Primary Analysis, and visualize the
total time spent glancing off-road. In contrast, the light gray bars represent the time spent
glancing only to the in-vehicle device. This gives a sense of what proportion of all glances are
directed toward the device interface. For example, almost no glances are directed to the device
during the N-Back tasks and baseline periods, whereas the Navigation Entry task requires a
substantial amount of glance time to the device. For the Radio Manual Hard task, nearly all offroad glances are glances to the device itself. Note also that the change of glance criteria affects
this samples’ ability to meet guidelines if they are applied to it. For example, the verbal Radio
© MIT AgeLab 2013
Page A14 of 293
Hard task meets the 12 second criterion if a glance-to-device metric is used, but does not if the
eyes-off-the-forward-roadway metric is applied. It can be observed that the manual Radio Hard
task studied here meets The Alliance 20 second criterion (2.1 A) level, but does not meet the
NHTSA 12 second criterion regardless of whether the eyes-off-road or glance-to-device metrics
are used. Note again that this sample does not fully conform to NHTSA’s recommend age
distribution.
Figure A-6: Statistical summary plots for to-device glance time.
© MIT AgeLab 2013
Page A15 of 293
Gender did not significantly impact glance time (p = .462). Age group significantly affected
glance time (p = .002), with older participants spending more time glancing at the device. Total
glance time differed significantly between the radio tasks and the radio manual and navigation
entry tasks (p < .001 for both). Almost no time is spent looking at the device during the 1-Back
task, particularly in contrast to the Navigation Entry task, which required a long total glance
time.
© MIT AgeLab 2013
Page A16 of 293
Number of To-Device Glances
Figure A-7: Number of glances to the device for each task. . One outlier data point on the Nav
Entry task is excluded from view to aide plot readability.
These data are similar to the total glance time metric discussed earlier.
© MIT AgeLab 2013
Page A17 of 293
Figure A-8: Statistical summary plots for number of glances to the in-vehicle device.
Gender did not significantly affect the number of glances to the device (p = .866). Age group
was a significant factor (p = .041), with older participants making slightly more glances to the
device overall. The number of glances required between the radio tasks and between the
navigation entry and radio manual task were significantly different (p < .001 for both).
© MIT AgeLab 2013
Page A18 of 293
Glance Duration Analysis
Distributions of glances to the in-vehicle device are shown in Figure A-9. As would be expected,
few glances were made to the device during the baseline driving periods (upper graphs) in
comparison to the secondary task periods (lower graphs).
© MIT AgeLab 2013
Page A19 of 293
Effect of Task Completion Time
Figure A-10 presents correlation plots between total task completion time and total device
glance time for the hard radio tuning task (manual and voice), as well as the Navigation Entry
task. As in the Primary Analysis, correlations are quite strong across tasks (Pearson R > 0.65 for
all three tasks, all p < .001). This suggests that during task periods, most glances away from the
forward roadway were made to the device.
Figure A-10: Device glance time vs. task completion time for three tasks.
© MIT AgeLab 2013
Page A20 of 293
APPENDIX B: ERROR-FREE TASK ANALYSIS
Introduction
NHTSA’s (2013) guidelines on visual-manual distraction state that only data from “error-free”
test trials should be used when determining whether a task meets the visual-manual safety
criteria. An error occurs whenever the participant backtracks through the steps of the task,
deletes previously entered information, or fails to complete the task (Section VI.H.1-2, p. 277278).
In many real-world driving scenarios (new car purchase, rental car, borrowing a friend’s car),
drivers are likely to be confronted by unfamiliar (and sometimes questionably implemented)
interfaces that may result in errors while performing a task. For this reason, we included all task
trials in the Primary Analysis, as long as relevant data were available. However, it is instructive
examine only trials that meet NHTSA’s definition of “error-free”, to investigate how the subset
of best-case task performance scenarios compare to the overall dataset.
Methods
The methodology is the same as described in Primary Data Analysis - Methods. In this section, a
subset of the primary dataset is considered: only task trials that were completed “error-free”,
per NHTSA’s guidelines. Each individual task performance was rated on an eight-point scale
(see Appendix G for rating data and guidelines). Only cases that achieved the highest rating on
both trials are considered in this analysis (in other words, the participant performed the task
perfectly on both replications). To achieve the highest rating, the participant must have
successfully complete the task without backtracking through the task steps or deleting
previously entered information, and without additional prompting or help from the research
assistant.
It should be noted that the definition of “error-free” used here could be considered “generous”
to the device interface under evaluation. Following NHTSA’s guidelines, the first on-road trial
could be considered a “practice” trial and the second as the “evaluation” trial. The analysis
presented here only includes cases where the participant was error-free on both the first
(practice) trail and the second (evaluation) trial.
This section omits the “Glance Distribution Analysis”. Since younger participants completed
many more tasks without error than older participants, a specific comparison of glance
frequencies between the age groups would be highly distorted.
© MIT AgeLab 2013
Page A21 of 293
Results
As in Appendix A, the plots below present two datasets side-by-side: the data from the Primary
Analysis in dark gray, and the error-free data in light gray. Since the light gray bars represent
only cases where error-free performance was obtained for both the first and second trials,
sample size varies between tasks, depending on how many participants were able to achieve
overall error-free performance (sample sizes shown for each column at the top of figures).
Self-Reported Workload
Figure B-1: Self-reported workload for each task regardless of performance (dark gray) and for
cases where participant performance was error free across both trials (light gray).
© MIT AgeLab 2013
Page A22 of 293
The mean workload rating changes little between the primary and error-free samples. Note that
few participants were able to complete the Navigation Entry, Song Fail, and Song Select tasks
without some level of difficulty or error.
© MIT AgeLab 2013
Page A23 of 293
Figure B-2: Statistical summary plots for self-reported workload among the error-free sample.
Neither gender nor age group significantly affected workload ratings (p = .733 and p = .359,
respectively). Workload ratings did differ between the two radio tasks (p = .002) and between
the radio manual and navigation entry tasks (p = .019).
© MIT AgeLab 2013
Page A24 of 293
Total Completion Time
Figure B-3: Task completion time for error-free cases (light gray).
As would be expected, restricting the analysis to error-free trials does result in a reduction in
task completion time values. However, it does not substantially change the relative pattern of
across tasks.
© MIT AgeLab 2013
Page A25 of 293
Figure B-4: Statistical summary plots for task completion time for error-free cases.
Completion time was not significantly affected by gender (p = .100). There was no effect of age
group (p = .582), with older participants taking slightly longer to complete tasks. Task
completion time differed significantly between the radio tasks (p < .001) and between the
navigation entry and radio manual tasks (p < .001).
© MIT AgeLab 2013
Page A26 of 293
Heart Rate
Figure B-5: Heart rate change data relative to an averaged baseline period of single-task driving
for error-free cases (light gray).
© MIT AgeLab 2013
Page A27 of 293
Figure B-6: Statistical summary plots for changes in heart rate relative to an averaged baseline
period of single-task driving for error-free cases.
There was no significant effect of gender or age group (p = .123 and p = .797, respectively).
There were no significant differences between the radio tasks (p = .710) or between the radio
manual and navigation entry tasks (p = .212).
© MIT AgeLab 2013
Page A28 of 293
Skin Conductance (SCL)
Figure B-7: Changes in skin conductance levels (SCL) relative to an averaged baseline period of
single-task driving for error free cases (light gray).
© MIT AgeLab 2013
Page A29 of 293
Figure B-8: Statistical summary plots for SCL changes relative to an averaged baseline period of
single-task driving during error-free cases.
SCL changes differed significantly by gender (p = .007), with women showing elevated changes
in SCL compared to men. SCL also differed significantly by age group, with older participants
showing elevated changes compared to younger (p = .002). SCL changes did not differ
significantly between the radio tasks (p = .861) or between the radio manual and navigation
entry tasks (p = .855).
© MIT AgeLab 2013
Page A30 of 293
Mean Velocity
Figure B-9: Mean vehicle velocity during error-free cases (light gray).
© MIT AgeLab 2013
Page A31 of 293
Figure B-10: Statistical summary plots of mean vehicle speed during error-free cases.
Mean speed was not affected by gender (p = .686), but was affected by age group (p = .003).
Mean speed did not differ between the radio tasks (p = .209) or between the navigation entry
and radio manual tasks (p = .640).
© MIT AgeLab 2013
Page A32 of 293
Variability of Velocity
Figure B-11: Variability (standard deviation) of velocity during error-free cases (light gray).
There are few differences between the error-free and the overall samples.
© MIT AgeLab 2013
Page A33 of 293
Figure B-12: Statistical summary plots for variability of velocity for error-free cases.
Variability was not affected by gender or age group (p = .982 and p = .279, respectively).
Variability differed significantly between the radio manual and navigation entry tasks (p <
.001), but did not differ between the two radio tasks (p = .138).
© MIT AgeLab 2013
Page A34 of 293
Acceleration Events
Figure B-13: Minor acceleration events (MinAE, defined as all accelerations greater than 0.1g)
for error-free cases (light gray).
© MIT AgeLab 2013
Page A35 of 293
Figure B-14: Statistical summary plots for acceleration events for error-free cases.
Event frequency was not affected by gender (p = .441) or task type (p = .198 between radio
tasks, p = .932 between the radio manual and navigation entry tasks), but accelerations differed
significantly between age groups (p = .004), with younger participants showing more
acceleration events than older ones.
© MIT AgeLab 2013
Page A36 of 293
Steering Wheel Angle
Figure B-15: Variability (standard deviation) of steering wheel angle for error-free cases (light
gray).
© MIT AgeLab 2013
Page A37 of 293
Figure B-16: Statistical summary plots for variability of wheel angle for error-free cases.
Wheel angle was not affected by gender (p = .096) or age group (p = .532). Steering wheel angle
variability differed significantly between the radio manual and navigation entry tasks (p = .016),
but not between the radio tasks (p = .249).
© MIT AgeLab 2013
Page A38 of 293
Minor Wheel Reversals
Figure B-17: Minor steering wheel reversals (see Methods) for error-free cases (light gray).
© MIT AgeLab 2013
Page A39 of 293
Figure B-18: Statistical summary plots for minor steering wheel reversals for error-free cases.
Wheel reversal rate was significantly affected by gender or age group (p = .432 and p = .523).
Wheel reversal rate did not differ between the radio tasks (p = .748) or between the radio
© MIT AgeLab 2013
Page A40 of 293
Major Wheel Reversals
Figure B-19: Major wheel reversals for error-free cases (light gray).
© MIT AgeLab 2013
Page A41 of 293
Figure B-20: Statistical summary plots for major wheel reversal rate for error-free cases.
Wheel reversals were not significantly affected by gender or age group (p = .415 and p = .809,
respectively). Wheel reversals differed significantly between the radio tasks (p < .001) and
between the radio manual and navigation entry tasks (p = .001).
© MIT AgeLab 2013
Page A42 of 293
Selected Glace Metrics Summary Table (Error Free Cases)
If one were to apply the NHTSA distraction cutpoints to the younger, older, and overall cohort, the table
below (Table B-1) shows the percentage who would meet each of the criteria when only error-free cases
are considered. Two entries are shown in italic and underlined where use of error-free cases changes a
value from the entire analysis sample from below to above an 85% threshold (i.e. Song Select).
Task
Age Group
Long Duration
Glances
Mean Glance
Duration
Total Off-road
Glance Time
Nav Cancel
Younger
96.60%
96.60%
100.00%
Older
100.00%
100.00%
100.00%
(all)
98.00%
98.00%
100.00%
Younger
100.00%
100.00%
18.20%
Older
100.00%
100.00%
0.00%
(all)
100.00%
100.00%
11.80%
Younger
93.10%
100.00%
100.00%
Older
80.00%
100.00%
95.00%
(all)
87.80%
100.00%
98.00%
Younger
96.70%
100.00%
73.30%
Older
88.20%
100.00%
11.80%
(all)
93.60%
100.00%
51.10%
Younger
100.00%
100.00%
100.00%
Older
100.00%
100.00%
93.80%
(all)
100.00%
100.00%
97.80%
Younger
100.00%
100.00%
87.50%
Older
100.00%
100.00%
88.90%
(all)
100.00%
100.00%
87.90%
Younger
95.50%
100.00%
90.90%
Older
100.00%
100.00%
75.00%
(all)
96.20%
100.00%
88.50%
Younger
100.00%
100.00%
20.00%
Older
100.00%
100.00%
50.00%
(all)
100.00%
100.00%
25.00%
Younger
100.00%
100.00%
93.30%
Older
100.00%
100.00%
68.20%
(all)
100.00%
100.00%
82.70%
Nav Entry
Radio Manual Easy
Radio Manual Hard
Radio Voice Easy
Radio Voice Hard
Song Select
Song Fail
Phone
© MIT AgeLab 2013
Page A43 of 293
Mean Off-Road Glance Duration
Figure B-21: Mean duration of glances during error-free cases (gray).
Mean glance duration is largely unchanged compared to the primary sample, despite a large
reduction in sample size for some tasks, such as Navigation Entry.
© MIT AgeLab 2013
Page A44 of 293
Figure B-22: Statistical summary plots for mean glance duration for error-free cases.
Mean glance duration was not affected by gender (p = .306) or age group (p = .135). Mean
glance rate differed significantly between the radio tasks and between the radio manual and
navigation entry tasks (p < .001).
© MIT AgeLab 2013
Page A45 of 293
Figure B-23: Long glance rate for error-free cases (light gray).
The manual Radio Easy task represents one of the few cases where restricting the analysis to
error-free cases appears to have an appreciable impact on the Long Glance Rate metric. In this
instance, the 85% point for the sample moves from being just under the criterion to being
markedly below.
© MIT AgeLab 2013
Page A46 of 293
Figure B-24: Statistical summary plots for long glance rate for error-free cases.
Long glance rate was not affected by gender (p = .381), or age group (p = .143). Long glance rate
differed significantly between the radio tasks (p = .001) and between the radio manual and
navigation entry tasks (p = .022).
© MIT AgeLab 2013
Page A47 of 293
Total Off-Road Glance Time
Figure B-25: Total off-road glance time during error-free cases (light gray).
Note the significant reduction in glance time for the error-free cases for the Navigation Entry
task compared to the overall sample. Nonetheless, total glance time for these cases was still
markedly above the 85% criteria for both The Alliance 20 second threshold (criterion 2.1 A), and
the NHTSA 12 second threshold for this task. The Phone Dialing, manual Radio Hard tuning
task, and the Song Fail task also produced the same result where or not the error-free restriction
on case inclusion was applied. Only in the cases of the voice command version of the Radio
Hard task and the Song Selection task, does use of the use of the error-free restriction make a
difference in whether the 85% criteria is met.
© MIT AgeLab 2013
Page A48 of 293
Figure B-26: Statistical summary plots for total glance time during error-free cases.
Glance time was not affected by gender (p = .756), or age group (p = .624). Glance time also
differed significantly between the radio tasks and between the radio manual and navigation
entry tasks (p < .001 for both).
© MIT AgeLab 2013
Page A49 of 293
Number of Glances
Figure B-27: Total number of off-road glances during error-free cases (light gray).
© MIT AgeLab 2013
Page A50 of 293
Figure B-28: Statistical summary plots for total number of off-road glances during error-free
cases.
Number of glances was not affected by gender or age group (p = .907 and p = .112,
respectively). Number of glances did not differ significantly between the radio tasks (p = .079),
but did differ between the radio manual and navigation entry tasks (p < .001).
© MIT AgeLab 2013
Page A51 of 293
Orienting Response
Figure B-29: An illustration of the Orienting Response for error-free cases.
Due to the varying number of participants in each subgroup, formal statistical analyses were
not undertaken. However, it is apparent from the data that the OR effect is influenced most
strongly by older women, even in error-free scenarios. As noted elsewhere, it should be noted
that this analysis does not explicitly distinguish glances for visual confirmation from glances
associated with OR behavior, and it is recognized that this is a partial confounding factor in
evaluating this behavior pattern. (See Appendix D for definitions and coding details.)
© MIT AgeLab 2013
Page A52 of 293
APPENDIX C: TRIAL COMPARISON ANALYSIS
Introduction
The study design exposed drivers to each of the secondary tasks a number of times. First, each
task was practiced while the vehicle remained parked. Tasks were practiced until participants
felt comfortable performing them. During the on-road portions of the protocol, all tasks were
performed twice (except the Song Fail task). In most instances, the two trials were sequential.
For example, the two radio easy tasks (selecting a preset station) were presented one after
another. In the case of the Navigation tasks, participants were presented with the first
Navigation Entry task, followed by the first Navigation Cancel task, and then the second
Navigation Entry task.
The Primary Analysis represents an aggregation of the two trial replications for each task, i.e.
metrics were averaged across the two trials. Sampling behavior across two trials helps reduce
extraneous variability in the data that might have arisen from factors such as momentary
changes in driving demands or fluctuating attention. However, this aggregation may mask
informative differences between the First Trial and Second Trial replications. As participants
become more comfortable balancing the demands of each task with the demands of driving,
their behaviors and physiological responses may change in a meaningful way. In other words,
there may be learning or other experiential effects. Therefore, it would be useful to examine
each set of replications separately. As noted earlier, it is also the case that the new NHTSA
visual-manual guidelines for measuring visual-manual distraction recommend a practice trial
under (simulated) driving conditions followed by one or more evaluation trials. Under this
model, trial one would be considered the driving practice trial and the second trial would be
considered the evaluation trial.
Methods
The methodology is the same as described in the Primary Analysis. However, all data
presentation and analyses now examine the First Trial and Second Trial replications of each task
separately. Task plots present trial data side-by-side, with dark gray bars representing the First
Trial, and light gray bars representing the Second Trial. Statistical summary plots are presented
separately for each trial type.
Note that the Song Fail task is excluded from this analysis, as this task was performed only
once.
© MIT AgeLab 2013
Page A53 of 293
Results
Task Completion Time
Figure C-1: Task completion for each task trial (dark gray for trial 1 and light gray for trial 2).
Completion time generally decreases during the second trial, except for the Navigation Entry
and Phone tasks, which both increased (both statistically significantly, p < .05).
© MIT AgeLab 2013
Page A54 of 293
Figure C-2: Statistical summary plots for task completion time (First Trial).
The effect of gender was not significant (p = .654). Age group was significant (p < .001), with
older participants taking longer to complete tasks than younger participants. Task completion
time differed significantly between the radio tasks and between the radio manual and
navigation entry tasks (p < .001 for both).
© MIT AgeLab 2013
Page A55 of 293
Figure C-3: Statistical summary plots for task completion time (Second Trial).
The pattern of findings for Trial 2 was consistent with the findings for Trial 1. The effect of
gender was not significant (p = .686). Age group was significant (p < .001), with older
participants taking longer to complete tasks than younger participants. Task completion time
was significantly different between the radio tasks and between the radio manual and
navigation entry task (p < .001 for both).
© MIT AgeLab 2013
Page A56 of 293
Heart Rate
Figure C-4: Percent change in heart rate relative to an averaged baseline period of single-task
driving during each task trial.
Heart rate tended to show a somewhat smaller response on the second trial of each task. The
sole overt exception was the second phone dialing task, which also had a somewhat longer task
completion time and higher mean SCL value for the second task.
© MIT AgeLab 2013
Page A57 of 293
Figure C-5: Statistical summary plots for heart rate relative to an averaged baseline period of
single-task driving (First Trial).
Heart rate differed significantly between genders (p = .016), with men showing greater heart
rate changes than women. Change in heart rate was not affected by age group (p = .260). Heart
rate differed significantly between the radio tasks (p = .01) and between the navigation entry
and radio manual tasks (p = .015).
© MIT AgeLab 2013
Page A58 of 293
Figure C-6: Statistical summary plots for heart rate relative to an averaged baseline period of
single-task driving (Second Trial).
The pattern of findings for Trial 2 was again consistent with the findings for Trial 1. Change in
heart rate was not affected by gender (p = .260) or age group (p = .279). Heart rate did not
change significantly between the radio tasks (p = .078), but did change significantly between the
radio manual and navigation entry tasks (p .028).
© MIT AgeLab 2013
Page A59 of 293
Skin Conductance (SCL)
Figure C-7: Change in skin conductance level (SCL) relative to an averaged baseline period of
single-task driving during each task trial.
Similar to what was seen in the heart rate data, there was a general trend for mean change in
SCL values relative to baseline to be somewhat smaller during the second task trial. Exceptions
appear in n-backs and the radio hard task where the mean arousal value is quite similar for both
trials. The second phone task shows a nominally higher SCL value for the second trial as was
the case for heart rate and task completion time.
© MIT AgeLab 2013
Page A60 of 293
Figure C-8: Statistical summary plots for change in SCL relative to an averaged baseline period
of single-task driving (First Trial).
SCL was significantly affected by gender (p = .035), with women showing elevated changes in
SCL compared to men. Changes in SCL were also affected by age group (p = .009), with older
participants showing greater percentage changes compared to younger. SCL did not differ
between task comparisons (radio tasks p = .412, radio manual and navigation entry tasks p =
.732).
© MIT AgeLab 2013
Page A61 of 293
Figure C-9: Statistical summary plots for change in SCL relative to an averaged baseline period
of single-task driving (Second Trial).
The age and gender findings were the same for the First and Second trials. SCL was
significantly affected by gender (p = .042), with women having slightly greater changes in SCL
than men. Age group also affected percentage based changes in SCL (.015), with older
participants showing greater changes than younger ones. SCL did not differ between task
comparisons (radio tasks p = .697, radio manual and navigation entry tasks p = .978).
© MIT AgeLab 2013
Page A62 of 293
Mean Velocity
Figure C-10: Mean vehicle velocity during each task trial.
© MIT AgeLab 2013
Page A63 of 293
Figure C-11: Statistical summary plots for mean vehicle speed (First Trial).
The effect of gender was not significant (p = 0.959). Vehicle speed was significantly affected by
age group (p = .001). Mean velocity did not differ between the radio tasks (p = .805) or between
© MIT AgeLab 2013
Page A64 of 293
Figure C-12: Statistical summary plots for mean vehicle speed (Second Trial).
The age and gender findings were the same for the First and Second trials. The effect of gender
was not significant (p = 0.854). Vehicle speed was significantly affected by age group (p = .003).
Mean velocity did not differ between the radio tasks (p = .174) but did differ between the radio
© MIT AgeLab 2013
Page A65 of 293
Figure C-13: Variability (standard deviation) of vehicle velocity for each task trial.
© MIT AgeLab 2013
Page A66 of 293
Figure C-14: Statistical summary plots for SD of vehicle speed (First Trial).
SD of vehicle speed was not affected by gender (p = .358) or age group (p = .440). SD of vehicle
speed did not differ between the radio tasks (p = .304), but did differ between the radio manual
and navigation entry tasks (p < .001).
© MIT AgeLab 2013
Page A67 of 293
Figure C-15: Statistical summary plots for SD of vehicle speed (Second Trial).
As with the First trial, SD of vehicle speed was not affected in the Second by gender (p = .458)
or age group (p = .513). SD of vehicle speed did not differ between the radio tasks (p = .594), but
did differ between the radio manual and navigation entry tasks (p < .001).
© MIT AgeLab 2013
Page A68 of 293
Acceleration Events
Figure C-16: Acceleration events during each task trial.
© MIT AgeLab 2013
Page A69 of 293
Figure C-17: Statistical summary plots for acceleration events (First Trial).
Accelerations were significantly affected by gender (p = .034), with women demonstrating
slightly more minor acceleration events than men. Age did not affect acceleration events (p =
.120). Acceleration events differed significantly between the radio tasks (p = .040), but not
between the radio manual and navigation entry tasks (p = .616).
© MIT AgeLab 2013
Page A70 of 293
Figure C-18: Statistical summary plots for acceleration events (Second Trial).
In contrast with the high consistency seen for other variables across trials, the gender effect
apparent in the First trial (women showing slightly more minor acceleration events than men),
this gender difference did not appear in the Second trial (p = .660). Consistent with the First
trial, age did not affect minor acceleration events in the Second (p = .070). Acceleration events
differed significantly between the radio tasks (p = .013), but not between the radio manual and
© MIT AgeLab 2013
Page A71 of 293
Figure C-19: Steering wheel angle variability (standard deviation) during each task trial.
© MIT AgeLab 2013
Page A72 of 293
Figure C-20: Statistical summary plots for variability of steering wheel angle (First Trial).
Wheel angle was not affected by gender or age group (p = .582 and p = .096, respectively).
Wheel angle differed significantly between the radio manual and navigation entry tasks (p =
.006), but not between the two radio tasks (p = .604).
© MIT AgeLab 2013
Page A73 of 293
Figure C-21: Statistical summary plots for variability of steering wheel angle (Second Trial).
The overall findings for gender and age were consistent across the First and Second Trials.
During the Second trial, steering wheel angle was again not affected by gender or age group (p
= .485 and p = .102, respectively). Wheel angle differed significantly between the radio manual
and navigation entry tasks (p < .001), and between the two radio tasks (p = .049).
© MIT AgeLab 2013
Page A74 of 293
Minor Wheel Reversals
Figure C-22: Minor steering wheel reversal (SWR) rate for each task trial.
© MIT AgeLab 2013
Page A75 of 293
Figure C-23: Statistical summary plots for minor SWR (First Trial).
SWR was not affected by gender or age group (p = .335 and p = .467). Minor SWR differed
significantly between the radio tasks (p = .019), but not between the radio manual and
© MIT AgeLab 2013
Page A76 of 293
Figure C-24: Statistical summary plots for minor SWR (Second Trial).
The nominal pattern seen in the First trial with females and older participants showing a higher
minor SWR again appeared for the Second trial. However, this patterning was again not
statistically significant (p = .643, and p = .485, respectively). Minor SWR did not significantly
between the radio tasks (p = .473), or between the radio manual and navigation entry tasks (p =
.255).
© MIT AgeLab 2013
Page A77 of 293
Major Wheel Reversals
Figure C-25: Major SWR during each task trial.
© MIT AgeLab 2013
Page A78 of 293
Figure C-26: Statistical summary plots for major SWR (First Trial).
Major SWR was not affected by gender or age group (p = .552 and p = .390, respectively). Major
SWR differed significantly between the radio tasks and between the radio manual and
navigation entry tasks (p < .001 for both).
© MIT AgeLab 2013
Page A79 of 293
Figure C-27: Statistical summary plots for major SWR (Second Trial).
For the Second trials, major SWR was again not affected by gender or age group (p = .440 and p
= .741, respectively). SWR differed significantly between the radio tasks and between the radio
manual and navigation entry tasks (p < .001 for both).
© MIT AgeLab 2013
Page A80 of 293
Selected Glance Metrics Summary Table (First & Second Trails)
If one were to apply the NHTSA distraction cutpoints to participants in the younger, older, and
overall cohort, Table C-1 below shows the percentage who would meet each of the criteria
considering the First and Second trials independently. Entries for situations where less than
85% of a group meet a threshold are bolded and shown in red.
Task
Nav Cancel
Nav Entry
Radio M Easy
Radio M Hard
Radio V Easy
Radio V Hard
Song Select
Song Fail
Phone
Age Group
Long Duration Glances
Mean Glance Duration
Total Off-Road Glance
Time
First Trial
Second Trial
First
Trial
Second Trial
First Trial
Second Trial
Younger
96.70%
96.70%
96.70%
96.70%
100.00%
96.67%
Older
100.00%
100.00%
100.00%
100.00%
81.82%
95.65%
(all)
98.10%
98.10%
98.10%
98.10%
92.31%
96.23%
Younger
100.00%
100.00%
100.00%
100.00%
20.00%
10.00%
Older
95.70%
95.70%
100.00%
100.00%
4.35%
0.00%
(all)
98.10%
98.10%
100.00%
100.00%
13.21%
5.66%
Younger
90.00%
96.70%
90.00%
100.00%
100.00%
100.00%
Older
82.60%
91.30%
95.70%
100.00%
86.96%
95.65%
(all)
86.80%
94.30%
92.50%
100.00%
94.34%
98.11%
Younger
93.30%
96.70%
100.00%
100.00%
73.33%
90.00%
Older
82.60%
91.30%
95.70%
100.00%
13.04%
34.78%
(all)
88.70%
94.30%
98.10%
100.00%
47.17%
66.04%
Younger
100.00%
100.00%
100.00%
100.00%
96.55%
100.00%
Older
100.00%
100.00%
100.00%
100.00%
69.57%
78.26%
(all)
100.00%
100.00%
100.00%
100.00%
84.62%
90.57%
Younger
100.00%
100.00%
100.00%
100.00%
83.33%
96.67%
Older
95.70%
95.70%
95.70%
100.00%
52.17%
73.91%
(all)
98.10%
98.10%
98.10%
100.00%
69.81%
86.79%
Younger
96.70%
100.00%
96.70%
100.00%
80.00%
90.00%
Older
95.70%
100.00%
100.00%
100.00%
43.48%
60.87%
(all)
96.20%
100.00%
98.10%
100.00%
64.15%
77.36%
Younger
93.30%
N/A
100.00%
N/A
26.67%
N/A
Older
100.00%
N/A
100.00%
N/A
52.17%
N/A
(all)
96.20%
N/A
100.00%
N/A
37.74%
N/A
Younger
100.00%
100.00%
100.00%
100.00%
96.67%
90.00%
Older
100.00%
100.00%
100.00%
100.00%
63.64%
68.18%
(all)
100.00%
100.00%
100.00%
100.00%
82.69%
80.77%
© MIT AgeLab 2013
Page A81 of 293
Figure C-28: Mean duration of off-road glances during each trial.
For the majority of participants, mean glance duration falls well below 2 seconds. The
patterning of mean values and the 85% point for the sample either remains relatively consistent
between the first and second trials, or shows a lower value for the second trial. An apparent
improvement with the second trial on this metric is most evident in the two manual radio tasks.
© MIT AgeLab 2013
Page A82 of 293
Figure C-29: Statistical summary plots for mean glance duration (First Trial).
There was a borderline difference in glance duration between genders (p = .061), with men
having slightly longer mean glance durations than women. Glance duration also differed
significantly by age group (p = .001). Mean glance duration differed significantly between the
radio tasks and between the radio manual and navigation entry tasks (p < .001).
© MIT AgeLab 2013
Page A83 of 293
Figure C-30: Statistical summary plots for mean glance duration (Second Trial).
Mean glance duration was not affected by gender (p = .825). Mean glance duration was,
however, affected by age group (p < .001). Mean glance duration differed significantly between
the radio tasks and between the radio manual and navigation entry tasks (p < .001).
© MIT AgeLab 2013
Page A84 of 293
Figure C-31: Long glance rates during each task trial.
Again, glance rates during the Radio Manual tasks are likely elevated due to the low off-road
glance frequency necessitated by these tasks.
© MIT AgeLab 2013
Page A85 of 293
Figure C-32: Statistical summary plots for long glance rate (First Trial).
Long glance rate did not differ significantly between genders or age groups (p = .128 and p =
.094, respectively). Long glance rate differed significantly between the radio tasks and between
the radio manual and navigation entry tasks (p < .001 for both).
© MIT AgeLab 2013
Page A86 of 293
Figure C-33: Statistical summary plots for long glance rate (Second Trial).
Long glance rate did not differ significantly between genders, (p = .963), but did differ by age
group (p = .001), with older subjects having a higher percentage of long glances than younger
subjects. Long glance rate differed significantly between the radio tasks (p = .003), and between
© MIT AgeLab 2013
Page A87 of 293
Total Off-Road Glance Time
Figure C-34: Total off-road glance time during each task trial.
Note that off-road glance time increases substantially on the second trial of the Navigation
Entry task. Thus, using the Second trial value in place of the mean values for both trials does
not improve the criterion performance for this task. On the other hand, total glance time does
decrease in the Second trial for several tasks. In the case of the manual Radio Easy and voice
Radio Hard tasks, using the Second trial value instead of the mean for both trials might impact
whether the formal NHTSA criterion is met in a sample with the specified age distribution.
© MIT AgeLab 2013
Page A88 of 293
Figure C-35: Statistical summary plots for total glance time (First Trial).
Total glance time was not affected by gender (p = .506). Total glance time was significantly
affected by age group (p < .001), with older participants spending more time glancing off-road
than younger participants. Glance time also differed significantly between the radio tasks and
between the radio manual and navigation entry tasks (p < .001 for both).
© MIT AgeLab 2013
Page A89 of 293
Figure C-36: Statistical summary plots for total glance time (Second Trial).
The same overall findings for age, gender and pattern of findings for the selected tasks is
consistent between the First and Second trials. For the Second trial, total glance time was not
affected by gender (p = .360). Total glance time was significantly affected by age group (p <
.001), with older participants spending more time glancing off-road than younger participants.
Glance time also differed significantly between the radio tasks and between the radio manual
and navigation entry tasks (p < .001 for both).
© MIT AgeLab 2013
Page A90 of 293
Number of Glances
Figure C-37: Number of off-road glances during each task trial.
© MIT AgeLab 2013
Page A91 of 293
Figure C-38: Statistical summary plots for number of off-road glances (First Trial).
Number of glances was not affected by gender (p = .915). There was a significant effect of age
group (p < .001), with older participants requiring more glances than younger participants.
Number of off-road glances did not differ between the radio tasks (p = .253), but did differ
between the radio manual and navigation entry tasks (p < .001).
© MIT AgeLab 2013
Page A92 of 293
Figure C-39: Statistical summary plots for number of off-road glances (Second Trial).
The same overall findings for age, gender and pattern of findings for the selected tasks is
consistent between the First and Second trials. For the Second trial, the number of glances was
not affected by gender (p = .138). There was a significant effect of age group (p = .036), with
older participants requiring more glances than younger participants. Number of off-road
glances differed significantly between the radio tasks (p = .002), and between the radio manual
and navigation entry tasks (p < .001).
© MIT AgeLab 2013
Page A93 of 293
Orienting Response
Orienting Response codes are shown for each set of tasks in Figures X and Y (see Appendix D
for coding details). Younger participants show a slight reduction in their OR on the Second
trials as compared to the First. In contrast, older participants show relatively little improvement
in AOR during the Second trial.
Figure C-40: Orienting Responses for the First trial of each task.
Note that one participant was unable to complete the Phone tasks, and one participant
accidentally canceled a navigation command before the formal start of the task period, resulting
in one missing data point for the Navigation Cancel task. As noted elsewhere, it should be
© MIT AgeLab 2013
Page A94 of 293
noted that this analysis does not explicitly distinguish glances for visual confirmation from
glances associated with OR behavior, and it is recognized that this is a partial confounding
factor in evaluating this behavior pattern.
Figure C-41: Orienting Responses for the Second trial of each task.
Note that one participant was unable to complete the Phone tasks.
© MIT AgeLab 2013
Page A95 of 293
Task Completion Data
Task Completion codes are shown for each task trial in Figures X and Y. Both younger and
older participants were able to improve their task completion ability on the Second trial.
Figure C-42: Task completion codes for the First trial of each task.
Note that although the Song Fail task was designed to be impossible to complete, the codes
shown here reflect the participant’s ability to correctly execute the song search commands.
© MIT AgeLab 2013
Page A96 of 293
Figure C-43: Task completion codes for the Second trial of each task.
© MIT AgeLab 2013
Page A97 of 293
Glance Distribution Analyses
Figure C-44: Glance distributions for the First trial (upper panels) and Second trial (lower
panels) of each task.
Considering all tasks as a group, there was a substantial and significant reduction in the
number of glances between trials (p < .001), with a per-participant mean of 141 glances on the
First trial and 100 on the Second trial.
© MIT AgeLab 2013
Page A98 of 293
APPENDIX D: ORIENTING RESPONSE CODING
Introduction
The Ford SYNC® voice interface, once activated with the push-to-talk button, is generally able
to recognize the driver’s voice regardless of where he/she may be looking. Nevertheless, we
observed many drivers orienting their head and eyes as if they believed that the voice
recognition system was housed somewhere near the center console display that was utilized by
the Ford SYNC® . Drivers would sometimes speak “to” the center console display screen, turn
their heads or entire bodies toward the device, or lean in toward where the microphone was
(incorrectly) perceived to be. We believe that it may be useful to examine this Orienting
response (OR) in greater detail as a number of participants appeared to orient their attention,
including some direct glances, to the display at a level above what was required to actually
acquire necessary display information.
Methods
A single coder watched muted video recorded from a camera mounted on the vehicle
dashboard, facing the driver. The camera recorded the participant’s face and upper torso,
providing a clear view of the participant’s posture and head orientation. This made it relatively
easy to determine when the participant began orienting to the display screen. Participant’s
behaviors were coded according to the guide below. It should be noted that this analysis does
not explicitly distinguish glances for visual confirmation from glances associated with OR
behavior, and it is recognized that this is a partial confounding factor in the coding.
Coding Guide
Category
Code
Description
Unknown
0
Participant did not perform the task or its corresponding data are missing.
None
1
Participant exhibits no OR towards the center console display. This means there is
no head tilting or leaning of the body towards the device.
Slight
2
Participant exhibits some mild OR towards the device. The participant leans
his/her head towards the device periodically throughout the task, or briefly leans
his/her body toward the device.
Moderate
2.5
Participant exhibits a fair amount of OR towards the device. This means the
participant leans his/her head or body towards the device or speaks directly at
the device for a sustained period of time.
Prioritizing
3
Participant exhibits a clear and sustained OR toward the device. This means that
the participant fully leans his/her head towards the device or repositions his/her
body toward the device. The participant may also appear to be speaking directly
at the device while also glancing for prolonged periods of time at the screen.
© MIT AgeLab 2013
Page A99 of 293
Results
OR data are presented in the following tables for each participant and task trial. Plots of the OR
are presented in the Primary Analysis section and the corresponding sections of the subsidiary
analyses.
Participant
ID
Nav entry
1
Nav
cancel 1
Nav entry
2
Nav
cancel 2
Song
select 1
Song
select 2
Song
select 3
6
2
1
1
1
1
1
1
7
2
1
2
1
1
1
2
8
2
0
2
1
1
2
2
9
1
1
1
1
2
1
1
15
2
2
2
2
2
2
2
17
1
1
1
1
1
1
1
18
1
1
1
1
1
1
1
20
1
1
2
1
1
1
1
21
2
2
2
2
2
1
1
24
1
1
2
1
1
1
2
25
1
1
1
1
1
1
1
26
1
2
2
1
2
2
2
27
1
1
1
1
1
1
1
29
2
1
2
2
1
1
1
30
2
1
1
1
1
1
1
31
2
2
2
2
2
2
2
34
2
1
2
1
1
1
2
37
2
2
2
2
2
2
2
38
2
1
2
1
1
1
2
39
1
1
1
1
1
1
2
41
2
2
2
2
2
2
2
42
1
1
1
1
1
1
1
43
1
1
1
1
1
1
1
44
1
1
1
1
1
1
1
46
1
1
1
1
1
1
1
47
1
1
1
1
1
1
1
49
2
1
2
1
1
1
2
© MIT AgeLab 2013
Page A100 of 293
51
1
1
1
1
2
1
2
53
1
1
1
1
1
1
1
54
2
2
2.5
2.5
2.5
2.5
2.5
58
1
1
1
1
1
1
2
60
1
1
1
1
1
1
2
61
1
1
1
1
1
1
1
62
2
2
2
2
3
3
3
67
2
2
2
1
3
2
2
68
2
2
2
1
2
1
2
69
1
1
2
1
2
1
2
70
2
2
2
2
2
2
2.5
74
2
1
2
2
2
2
2
76
2
2
2
2
2
2
2
77
2.5
2
2
2
3
3
3
79
2
1
1
1
1
2
2
80
3
3
3
3
3
3
3
82
2
2
2
1
1
1
2
83
1
1
2
1
2.5
2
2
84
2
1
1
1
1
1
2
86
1
2
2
2
1
2
2
87
1
1
2
1
1
1
1
88
2
1
1
1
2
2
2
89
2
2
2
1
2
2
2
90
2.5
2.5
2.5
2.5
2.5
2.5
2.5
91
2.5
1
2
2
1
1
2
94
2
1
1
1
1
2
2
95
3
3
3
3
3
3
3
98
3
2
3
3
3
2.5
3
99
3
3
3
2
2
2
2
100
1
1
1
1
1
1
1
101
3
3
3
3
3
3
3
103
1
1
1
1
1
1
1
104
3
3
3
3
3
3
3
© MIT AgeLab 2013
Page A101 of 293
Listing Continued for Remaining Tasks:
Participant
Radiov 1
radiov_2
radiov_4
radiov_6
phone_1
phone_2
6
2
1
2
1
1
1
7
1
1
1
1
2
1
8
1
1
2
1
2
2
9
1
1
1
1
1
1
15
2
1
2
2
2
2
17
1
1
1
1
1
1
18
1
1
1
2
1
2
20
1
1
2
1
1
1
21
1
1
1
1
2
2
24
1
1
1
1
1
1
25
1
1
1
1
1
1
26
1
2
2
1
2
2
27
1
1
1
1
1
1
29
1
1
1
1
2
2
30
1
1
1
2
1
1
31
3
2
2
2
2
2
34
1
1
2
1
1
1
37
2
1
2
2
2
2
38
1
1
2
2
1
2
39
1
1
1
1
1
1
41
2
1
2
1
2
2
42
1
1
1
1
1
1
43
1
1
1
1
1
1
44
1
1
1
1
1
1
46
1
1
1
1
1
1
47
1
1
1
1
1
1
49
2
1
1
1
2
1
51
1
1
1
1
1
1
53
1
1
1
1
1
1
54
3
3
3
2
2.5
2
ID
© MIT AgeLab 2013
Page A102 of 293
58
1
1
1
1
1
1
60
1
1
1
1
1
1
61
1
1
1
1
1
1
62
2
2
2
1
2
2
67
2
2
1
1
1
2
68
2
1
2
2
2
2
69
1
1
1
1
1
1
70
2
3
2
2.5
2
2
74
2
1
2
2
2
1
76
2
2
2
2
2
2
77
3
3
3
3
3
3
79
1
1
1
1
2
1
80
3
3
3
3
3
3
82
1
2
1
1
1
1
83
1
1
2
2
1
1
84
1
1
1
1
1
2
86
2
2
2
1
1
1
87
1
1
1
1
1
1
88
1
2
1
1
1
1
89
1
1
1
2
2
2
90
2
3
2
3
2.5
2.5
91
1
1
1
1
2
2
94
2
2
2
1
2
1
95
2
2
2.5
2
3
3
98
2
1
3
2
3
3
99
2
2
2
2
2.5
2.5
100
1
1
1
1
0
0
101
3
2
2
3
3
3
103
1
1
1
1
1
1
104
2
2
3
3
3
3
© MIT AgeLab 2013
Page A103 of 293
APPENDIX E: BASELINE PERIOD ANALYSIS
Introduction
The use of a single reference period immediately prior to a defined secondary task period is a
common method by which dynamic measures such as physiology, glance behavior, and driving
performance metrics can be considered relative to “single task” driving. One of the challenges
for real-world studies is the reality that external events, such as truck passing close by to the
experimental vehicle, or internal events, such as thoughts about personal conflicts, will
influence the arousal level and driving performance of some percentage of participants during
any given reference point. This can have a measurable impact on the estimated change in
behavior associated with single task vs. dual task conditions as we have discussed previously
(Reimer, Mehler, Wang, et al., 2012). In the design of the present study, we collected a series of
baseline periods, one prior to each task block, and that could be combined for purposes of
establishing a broad single task driving reference. The extent to which the use of this combined
baseline may have influenced our findings as opposed to using individual baselines is
considered in this Appendix.
There were a total of seven baseline periods in which participants simply drove, without having
to focus on any secondary tasks. Each baseline was followed by a task block. Throughout the
main analyses of this report (Primary Analysis and Appendices A-C), dual task driving periods
are generally presented alongside a combined baseline period that represents the mean of the
seven separate periods. The following section examines physiological data, driving behavior
measures, and visual behavior as defined by NHTSA’s guidelines for visual-manual distraction
for each of the seven baseline periods individually. The intent here is to examine the extent to
which this method of combining the individual baseline periods into a common single task
driving reference period may have impacted the pattern of results as well as to characterize
various aspects of drivers’ behavior during the single task driving intervals that were formally
analyzed.
Methods
Methods for this supplementary analysis are consistent with those of the Primary Analysis.
Data for each baseline period are visualized separately. For each measure of interest, a
Friedman test is employed to determine whether there were any significant differences between
the baseline periods.
© MIT AgeLab 2013
Page A104 of 293
Additionally, total off-road glance time is further broken down by glance target. An analysis of
the total driving time necessary to accumulate 12 seconds of off-road glance time is also
presented.
Results
Consistency Across Baseline Periods
Results of statistical testing are presented in Table X. There were no significant differences
across the baseline periods for heart rate, skin conductance level, or any of the NHTSA visualmanual distraction criteria. However, there were statistical differences in several measures of
driving behavior, as shown in the table below. Corresponding plots of the mean values for each
variable are shown in Figures X-Y at the end of this section.
Table E-1: Friedman tests on various dependent measures across the baseline periods.
Dependent Measure
P-Value
Significance
Heart Rate
0.082
NS
Skin Conductance Level
0.841
NS
0.289
NS
Long Glance Rate
0.702
NS
Total Off-road Glance Time
0.922
NS
Number of Glances
0.729
NS
Mean Velocity
0.413
NS
0.016
*
Acceleration Events
0.048
*
< 0.001
***
Minor Steering Wheel Reversals
0.428
NS
Major Steering Wheel Reversals
< 0.001
***
© MIT AgeLab 2013
Page A105 of 293
Off-Road Glance Behavior
We also evaluated the length of time necessary to accumulate 12 seconds of off-road glances
during baseline driving. Table X presents summary statistics for this measure. Six younger
participants drove for over 400 seconds before accumulating twelve seconds of off-road glance
time. Table Y presents summary statistics with these outliers removed. Regardless of outlier
removal, older participants accumulate off-road glance time significantly faster than younger
participants (p = .003 with outliers, p = .04 without, ANOVA by ranks). The effect of gender is
not significant (p = .647 and p = .309), nor is the interaction of age group and gender (p = .732
and p = .885).
Table E-2: Summary statistics for off-road glance accumulation time (in seconds). Tables present
the mean (standard deviation) [range].
Female
Male
Younger
Older
203.08 (163.0)
112.37 (65.2)
[87.7 - 691.9]
[30.7 - 204.6]
239.20 (216.8)
96.90 (53.1)
[53.0 - 758.8]
[41.0 - 202.9]
Table E-3: The same summary statistics as in Table X, but with six younger outlier participants
removed.
Female
Male
Younger
Older
148.21 (60.3)
112.37 (65.2)
[87.7 - 263.5]
[30.7 - 204.6]
122.06 (49.9)
96.90 (53.1)
[53.0 - 210.0]
[41.0 - 202.9]
© MIT AgeLab 2013
Page A106 of 293
Figure E-1: Heart rate across the seven baseline periods.
There is some fluctuation in mean values but a Friedman test shows no statistically significant
difference in heart rate across the baselines collected prior to each of the seven data collection
blocks (p= 0.082).
© MIT AgeLab 2013
Page A107 of 293
Figure E-2: Skin conductance level (SCL) across the seven baseline periods.
A Friedman test shows no significant difference in SCL across the baselines collected prior to
each of the seven data collection blocks (p= 0.841).
© MIT AgeLab 2013
Page A108 of 293
Figure E-3: Mean off-road glance duration across the seven baseline periods.
Note that the 85% of sample reference points (blue lines) for mean duration of glances are not in
and of themselves criterion thresholds during single task driving as this metric is intended for
measuring glance behavior during secondary tasks. Nonetheless, looking at this metric across
the seven baseline periods does provide a useful way of considering the consistency of behavior
across the periods. A Friedman test shows no significant difference in mean off-road glance
duration across the baselines collected prior to each of the seven data collection blocks (p=
0.289).
© MIT AgeLab 2013
Page A109 of 293
Figure E-4: Long glance rate across the seven baseline periods.
Note that the 85% of sample reference points (blue lines) for long glance rates are not in and of
themselves criterion thresholds during single task driving as this metric is intended for
measuring glance behavior during secondary tasks. Nonetheless, looking at this metric across
the seven baseline periods does provide a useful way of considering the consistency of behavior
across the periods. A Friedman test shows no significant difference in long glance rate across
the baselines collected prior to each of the seven data collection blocks (p= 0.702).
© MIT AgeLab 2013
Page A110 of 293
Figure E-5: Total off-road glance time across the seven baseline periods.
Note that the 85% of sample reference points for total glance time are not in and of themselves
criterion thresholds during single task driving as this metric is intended for measuring time
glancing away from the roadway during secondary tasks which should, by their intended
nature, be relatively brief. Nonetheless, glance time away from the forward roadway across the
seven baseline periods does provide a useful way of considering the consistency of behavior
across the periods. A Friedman test shows no significant difference in total off-road glance time
across the baselines collected prior to each of the seven data collection blocks (p= 0.922).
© MIT AgeLab 2013
Page A111 of 293
Figure E-6: Number of off-road glances across the seven baseline periods.
A Friedman test shows no significant difference in the number of off-road glances across the
baselines collected prior to each of the seven data collection blocks (p= 0.729).
© MIT AgeLab 2013
Page A112 of 293
Figure E-7: Mean vehicle velocity across the seven baseline periods.
A Friedman test shows no significant difference in velocity across the baselines collected prior
to each of the seven data collection blocks (p= 0.413).
© MIT AgeLab 2013
Page A113 of 293
Figure E-8: Variability (standard deviation) of vehicle velocity across the seven baseline periods.
In contrast with the physiology and glance metrics, a Friedman test shows a significant
difference in standard deviation in velocity across the baselines collected prior to each of the
seven data collection blocks (p= 0.016).
© MIT AgeLab 2013
Page A114 of 293
Figure E-9: Number of acceleration events per minute across the seven baseline periods.
A Friedman test shows a significant difference in minimal acceleration events across the
baselines collected prior to each of the seven data collection blocks (p= 0.048).
© MIT AgeLab 2013
Page A115 of 293
Figure E-10: Standard deviation of steering wheel angle across the seven baseline periods.
A Friedman test shows a significant difference in the standard deviation of steering wheel angle
across the baselines collected prior to each of the seven data collection blocks (p< 0.001).
© MIT AgeLab 2013
Page A116 of 293
Figure E-11: Minor steering wheel reversal rate across the seven baseline periods.
A Friedman test shows no significant difference in minor steering wheel reversal rates across
the baselines collected prior to each of the seven data collection blocks (p= 0.428).
© MIT AgeLab 2013
Page A117 of 293
Figure E-12: Major steering reversal rate across the seven baseline periods.
A Friedman test shows a significant difference in major steering wheel reversal rates across the
baselines collected prior to each of the seven data collection blocks (p< 0.001).
© MIT AgeLab 2013
Page A118 of 293
A Visualization of Glance Time by Object across the Seven Baseline Periods
Figure E-13: Total glance time per object. “Instrument Cluster” refers to the steering wheel and
dashboard instrumentation directly behind it. “Center Stack” refers to the area containing
standard radio controls and the center console display screen associated with the voice
command interface.
© MIT AgeLab 2013
Page A119 of 293
Individual Participant Data
Individual participant data across the seven baseline periods for glance metrics, heart rate, skin
conductance, and mean vehicle speed are presented in the following series of plots. Generally
speaking, measurements across the seven baseline periods were highly stable within
participants.
© MIT AgeLab 2013
Page A120 of 293
© MIT AgeLab 2013
Page A121 of 293
© MIT AgeLab 2013
Page A122 of 293
© MIT AgeLab 2013
Page A123 of 293
© MIT AgeLab 2013
Page A124 of 293
© MIT AgeLab 2013
Page A125 of 293
© MIT AgeLab 2013
Page A126 of 293
APPENDIX F: GLANCE CODING GUIDELINES & PROCEDURES
Introduction
As discussed in the Primary Analysis (Methods: Automated Eye-Tracking), it was determined that
there were a number of significant validity questions and issues with the data collected using
the FaceLAB® eye tracking system. To ensure that we could analyze glance data with a high
level of confidence, we developed a manual glance coding protocol designed to capture
relevant glance behavior.
Methods
The methodology outlined here is based on similar procedures developed by the Crash
Avoidance Metrics Partnership (CAMP) Driver Workload Metrics project (see also Smith,
Chang, Glassco, Foley, & Cohen, 2005).
Video Recording
Driving sessions were recorded from up to six cameras simultaneously:
1. A right-side view of the driver, covering the head and torso from a rightward angle.
2. A forward view of the driver covering his/her head and upper torso, recorded from a
camera mounted on the dashboard behind the steering wheel.
3. A view of the steering wheel and dashboard, recorded from a camera mounted over the
driver’s right shoulder.
4. A view of the forward roadway, recorded from a camera mounted on the dashboard.
5. A wide-angle view of the forward roadway, recorded from a camera mounted on the
back of the rearview mirror.
6. A view out of the back window.
Research assistants could use any or all of these videos at their discretion to assist in coding of
glance data. Cameras 2 and 3 were the most frequently used.
Annotator Software
The MIT AgeLab commissioned the creation of a customized software package, dubbed the
“Annotator”, to assist in the reduction of eye glance data (pictured in Figure X). The Annotator
has several features useful for this type of work. Multiple video files can be loaded and played
synchronously, playback can be controlled frame-by-frame or at arbitrary playback speeds, and
new annotation categories can be created with great flexibility. This greatly improved the speed
© MIT AgeLab 2013
Page A127 of 293
of data acquisition, as the research assistant need only press one of the predefined keys to add
an annotation at the video’s current time code. The Annotator outputs a flat file containing
times, key codes, and their associated annotations.
Figure F-1: The Annotator program used for manual glance coding. The program is organized
into several panes: top left contains the videos (video of the participant has been distorted to
protect his anonymity), bottom left contains playback controls, top right contains session log
data, middle right contains a live list of the current video’s annotations, and the bottom right
contains a list of available annotations and their associated keys.
Coding & Mediation
Two research assistants separately coded each task trial, labeling the driver’s glance target
according to one of seven categories (see Glance Coding Guide, below). Data were then
© MIT AgeLab 2013
Page A128 of 293
compared to check for discrepancies between the coders. Figure X shows a glance comparison
plot. A trial was considered discrepant if any of the following occurred: the coders started or
ended their coding at different times, the coders described differing numbers of glances, the
coders identified a different glance target for a glance, or the timing of a glance differed by more
than 200ms. A third coder resolved any discrepancies, making a “final determination” as to
which of the original two coders was correct.
Figure F-2: A portion of a coder comparison plot for one task trial. Time in seconds is shown on
the x-axis, and the glance target is shown on the y-axis. Line colors (red and gray) represent
different coders. Note that the gray coder was prone to coding “left” and “right” glances, and
that there is a timing discrepancy at around 96 seconds.
Data Storage & Reduction
Annotator output files were stored on the AgeLab’s secure network. These files were converted
and unified into a secure MySQL database format, thus allowing all eye glance data to be
queried quickly and flexibly for subsequent analyses.
Glance Coding Guide
Driver glance behavior was coded into seven categories according to the following guidelines.
Keyboard
Shortcut
Glance
Location
Operational Definition
F1
Road
To code a glance as “road”, the participant should be looking forward, or
in the forward direction. The exact position of the eyes can vary,
depending where on the road he or she is looking. A participant looking
straight ahead, directly over the camera, is the easiest and most common
way to code for road. There is, however, a somewhat wide range of where
the participant can be looking that would be considered a road glance.
For example, a participant following a passing car to their left that is
ahead of their own car would be coded as a road glace. Similarly, a
participant may appear to be looking to their right, but is actually looking
forward and to the right, perhaps at a passing car. A good way to
determine if a participant is looking at the road is if, no matter if the
© MIT AgeLab 2013
Page A129 of 293
glance is slightly to the left or right, the participant can see what is in
front of them. To code a glance as road can sometimes be difficult, given
the range of forward-looking abilities. One important thing to keep in
mind when coding in the Annotator is that for the most part, any glance
off the road (i.e. center stack, instrument cluster etc.) will be followed by a
glance to the road, even if it is a very brief glance.
F2
Center Stack
To code a glance as “center stack”, the participant should be looking to
their right, and slightly downward. This part of the car consists of the
Ford Sync device system, and some of the buttons that are used to control
it. The angle at which the participant is looking down will depend on
their height. The participant will often look at the center stack when
engaging in a task that requires the use of this system, such as a
navigation entry task, a radio task, or song select task.
F3
Rearview
Mirror
To code a glance as “rearview mirror”, the participant should be looking
up and to the right. There are instances where it can be difficult to
determine if the participant is actually looking at the rearview mirror, or
just slightly upward at the road. The angle at which a person will look up
at the rearview mirror depends on the person’s height; a short participant
will look up at a more obvious and steep angle, whereas a tall participant
will have to look up at less of an angle.
F4
Other
To code a glance as “other”, the participant should be making a glance
that does not meet the criteria for any of the other coding’s. Some
examples of where it would be appropriate to code a glance as other are if
the participant turns his or her head around to speak with the person in
the backseat, if the participant looks at their watch, if the participant looks
up at the ceiling, if the participant turns their head past 90 degrees either
to the left or right (looking at a blind spot), if the participant takes a drink
or eats something.
F5
Instrument
Cluster
To code a glance as “instrument cluster”, the participant should be
looking downward at the steering wheel, which contains the voice
command button to control the Ford Sync device system. A participant
will often make this glance when he or she must engage the voice
command system. In some cases it can be hard for the coder to
differentiate between a downward glance (at the instrument cluster) and
a long blink, because participant’s eyelids may cover the eyes. One
exception, where one should not code instrument cluster for a glance to
the wheel, is if the participant is looking down to turn on the windshield
wipers. This would be coded as other. The angle at which the participant
will look down depends on their height.
F6
Left
To code a glance as “left”, a participant should be looking to their left,
with their head turning no more than 90 degrees. Because it is extremely
difficult to tell if a participant is looking directly at the left mirror, the
code is simply “left”, for any glance in that direction. Simply put, a glance
to the left is a glance out of the left window of the car.
F7
Right
To code a glance as “right”, a participant should be looking to their right,
with their head turning no more than 90 degrees. Because it is extremely
difficult to tell if a participant is looking directly at the right mirror, the
© MIT AgeLab 2013
Page A130 of 293
code is simply “right”, for any glance in that direction. Simply put, a
glance to the right is a glance out of the right window of the car.
F8
End
© MIT AgeLab 2013
Use the “end” code at the completion of the task period. This should
correspond with the end time given in the participant data.
Page A131 of 293
APPENDIX G: TASK PERFORMANCE CODING GUIDELINES
Introduction
In its Visual-Manual Driver Distraction Guidelines (2013), NHTSA provides the following
guidance on defining errors during task performance:
“Error means that a test participant has made a significant incorrect input when performing
a testable task during a test trial. An error has occurred if the test participant has to
backtrack during performance of the task or delete already entered inputs. If the device can
accommodate an incorrect entry without requiring backtracking and extra inputs beyond
those necessary to reach the desired end state of the task, then no error is deemed to have
occurred. Error-Free Trial means a test trial in which no errors are made by the test
participant while completing the task” (Section IV.B.5 & IV.B.6, p. 249)
NHTSA further recommends that when assessing the distraction potential of an in-vehicle task,
only data from error-free trials should be considered. The results of an analysis that considers
data only from error-free trials are presented in Appendix B. Task trials were assessed
according to the following methods.
Methods
A participant’s ability to complete a given task with or without some level assistance was
assessed via audio recordings of the driving session. In the case of the Radio Manual tasks, task
completion was coded according to notes taken during the driving session. Two coders
classified each task into one of several task completion ratings (see Coding Guide). Discrepant
ratings were mediated by a third, independent coder. Task trials rated “5” (error-free) were
subsequently included in the error-free analysis (Appendix B).
© MIT AgeLab 2013
Page A132 of 293
Coding Guide
Rating
Definition
1
Participant failed to complete the task due to user error.
1a
Participant failed to complete task due to system error (voice recognition, etc).
2
Participant completed the task with substantial assistance or prompting from the experimenter
(more than 1 interaction), repeatedly backtracked through the task’s steps or restarted the task,
or took a very long time to complete the task.
3
Participant completed the task with minimal prompting or assistance (1 interaction with the
experimenter).
3a
Participant completed the task, but encountered a system error (voice recognition), and needed
slight assistance to correct it.
4
Participant completed the task without prompting or assistance, but had to backtrack through
the task’s steps or restart the task to correct a user error.
4a
Participant completed the task without prompting or assistance, but had to backtrack to correct
a system error (ie, voice recognition error).
5
Participant completed the task “error-free”, without prompting or assistance from the
experimenter and without backtracking through the task or restarting it.
© MIT AgeLab 2013
Page A133 of 293
Data
partici
pant
Nav
cancel 1
Nav
cancel 2
Nav
entry 1
Nav
entry 2
Phone 1
Phone 2
Radiom
1
Radiom
2
Radiom
4
6
2
5
2
4
4
5
5
5
5
7
5
5
3
3
5
5
5
5
5
8
5
5
3
5
5
5
5
5
5
9
5
5
2
5
5
5
5
5
5
15
5
5
5
4a
5
5
5
5
5
17
5
5
5
5
3
5
5
5
5
18
5
5
5
5
5
5
5
5
5
20
5
5
5
4
5
5
5
5
5
21
5
5
5
5
4
3
5
5
5
24
5
5
4
2
5
5
5
5
5
25
5
5
5
5
5
5
5
5
5
26
5
5
3
4
3
5
5
5
5
27
5
5
5
5
5
5
5
5
5
29
5
5
5
5
5
4
3
5
5
30
5
5
5
4a
5
5
5
5
5
31
5
5
2
2
5
5
5
5
5
34
5
5
2
2
5
5
5
5
5
37
5
5
5
5
5
5
5
5
5
38
5
5
5
3
5
5
5
5
5
39
5
5
5
3
5
5
5
5
5
41
5
5
3
5
5
5
5
5
5
42
5
5
5
4
5
5
5
5
5
43
5
5
5
4a
5
5
5
5
5
44
5
5
5
5
5
5
5
5
5
46
5
5
4
5
5
5
5
5
5
47
5
5
5
5
5
5
5
5
5
49
5
5
5
5
5
5
5
5
5
51
5
5
5
4
5
5
5
5
5
53
5
5
5
5
5
4a
5
5
5
54
5
5
2
2
5
5
5
5
5
58
5
5
5
3a
5
5
5
5
5
© MIT AgeLab 2013
Page A134 of 293
60
5
5
5
4a
5
5
5
5
5
61
5
5
5
4a
5
5
5
5
5
62
5
5
4
5
5
5
5
5
5
67
5
5
5
4a
5
5
5
5
5
68
5
5
5
5
2
2
5
5
2
69
5
5
5
4
4
4
5
5
5
70
5
5
5
5
5
4
5
5
5
74
5
5
5
4
5
5
5
5
5
76
5
5
5
4a
5
5
5
5
5
77
5
5
2
2
4
4
1
5
1
79
5
5
5
5
4a
4a
5
5
1
80
4
5
4
2
5
4a
5
3
3
82
4a
5
5
5
5
5
5
5
5
83
5
5
5
4
5
5
5
5
1
84
5
5
3a
5
4a
5
5
5
5
86
5
5
5
5
5
5
5
5
5
87
5
5
5
3a
5
5
5
5
5
88
5
5
3a
4
5
5
5
5
5
89
2
5
5
2
4
4
5
5
5
90
2
2
2
2
4a
2
5
5
5
91
5
5
3
3
2
5
2
5
2
94
5
5
5
5
5
5
5
5
5
95
5
5
2
3a
3
5
5
5
5
98
1
1
2
2
5
4a
5
5
2
99
5
5
3
5
5
5
2
5
3
100
5
5
5
5
0
NA
5
5
5
101
5
5
2
5
3a
5
5
5
5
103
5
5
5
5
5
5
5
5
5
104
5
5
5
5
5
5
5
5
5
© MIT AgeLab 2013
Page A135 of 293
partici
pant
Radom
6
Radiov
1
Radiov
2
Radiov
4
Radiov
6
Song 1
Song 2
Song 3
Story 2
6
5
5
5
5
5
5
5
5
5
7
5
5
5
5
5
5
5
4
5
8
5
5
5
5
5
2
3
5
5
9
5
3
5
3
5
2
5
1
5
15
5
5
5
5
5
5
5
5
5
17
5
5
5
5
5
5
5
5
5
18
5
5
5
5
5
3
5
1
5
20
5
5
5
5
5
5
5
5
5
21
5
5
5
5
5
2
5
1
5
24
5
5
5
5
5
5
5
5
5
25
5
5
5
3
5
5
5
1
5
26
5
5
5
2
2
2
5
1
5
27
5
5
5
5
5
5
5
3
5
29
5
5
5
2
5
5
5
1
5
30
5
5
5
5
5
5
5
5
5
31
5
2
5
3
5
3
3
1
3
34
5
5
5
5
5
5
5
5
5
37
5
5
5
5
5
4a
3
5
5
38
5
5
5
3
5
5
5
5
1
39
5
5
5
5
5
5
5
5
5
41
5
5
5
5
5
4
3
1
5
42
5
5
5
5
5
5
5
1
5
43
5
5
5
4a
4
5
5
4
5
44
5
5
5
4
5
5
5
5
5
46
5
5
5
5
5
4
5
5
5
47
5
5
5
5
5
5
5
5
5
49
5
3
5
5
5
5
5
5
5
51
5
5
5
5
5
4
5
5
5
53
5
5
5
5
5
3
5
1
5
54
5
5
5
3a
5
2
3
4
5
58
5
5
5
5
3
5
5
5
5
60
5
5
5
5
5
4
5
5
3a
© MIT AgeLab 2013
Page A136 of 293
61
5
5
5
5
5
4
5
5
5
62
5
5
5
5
5
4a
1a
1
3a
67
5
4
5
5
5
1a
4a
1
5
68
5
2
5
2
5
2
5
4
5
69
5
5
5
5
5
5
5
1
5
70
5
1
1
5
5
5
5
4
5
74
5
5
5
4
5
2
4a
1
5
76
5
5
5
4
5
4
5
1
5
77
5
2
5
4
4
5
5
1
1
79
1
5
5
5
5
4
4
4
5
80
5
5
5
2
5
4
4
1
5
82
5
5
5
5
5
5
5
5
5
83
5
4
5
5
4
2
5
5
5
84
5
5
5
5
5
5
5
1
3
86
5
5
5
5
5
5
5
1
5
87
5
5
5
5
5
4
5
5
5
88
5
5
5
5
5
3a
5
5
5
89
5
5
5
5
5
2
3a
1
5
90
5
2
5
3
1
3a
1
1
2
91
5
5
5
5
5
5
5
1
3
94
5
5
5
5
5
5
5
5
5
95
5
5
5
2
5
3a
3a
1
5
98
5
5
5
4
5
4
5
1
5
99
3
5
5
2
1
1
5
1
5
100
5
5
5
5
4
3
5
1
NA
101
5
5
5
5
4
2
2
1
1
103
5
5
5
5
4
5
4
1
5
104
5
5
5
2
5
5
5
1
5
© MIT AgeLab 2013
Page A137 of 293
APPENDIX H: DETAILED EXPERIMENT PROTOCOL & TASK SCRIPTS
© MIT AgeLab 2013
Page A138 of 293
Voice Based In-Vehicle Systems Study Experimental Checklist
Section 1 - Intake
Participant Code
___________________ Protocol Configuration ________________
Date
_______________________________________________________________
RA (s)
_______________________________________________________________
Notes for the research assistant - Please review before each participant run.
As a research associate involved with this experiment, you are responsible for actively
monitoring the participant, ensuring that all instructions detailed below are described clearly to
the participant and that any questions are answered to the best of your ability. Please make
sure to read instructions clearly. If, at any time, the participant has reservations about
participating or continuing to participate in this experiment, you should encourage them to
withdraw. DO NOT ENCOURAGE PARTICIPATION! If a question arises that you are
unable to answer, seek assistance from a senior member of the research staff. DO NOT GUESS.
During the course of the study you should minimize any unnecessary actions, e.g. moving
around the room, getting papers organized, extra key strokes, etc. while participants are
completing tasks, physiological recordings are being made, etc. Low level noise and slight
movements will impact participant arousal. In addition, due to the significant number of
functional actions and active monitoring required of RAs at various points during this
experimental protocol, RAs are not to use the lab computers or other computers or
communication devices to monitor e-mail, work on other projects, etc. so that full attention can
be devoted to active monitoring of the participant, equipment, and protocol. The sole exception
would be the use of text messaging to communicate with another staff member regarding
significant equipment or participant related issues. While the desire of RAs to multi-task for
purposes of carrying out other MIT related work is appreciated, the nature of this particular
protocol does not lend itself to such activities.
[Note: Emergency contacts and phone numbers appear here.]
© MIT AgeLab 2013
Page A139 of 293
Protocol Summary
The protocol will normally be carried out by two Research Associates (RA1 & RA2). RA1 will
greet the participant at the AgeLab and take them through steps 1-5 in the lab. The participant
will then be passed off to RA2 who will introduce the participant to the vehicle and handle the
remainder of the study interaction. A third RA may assist as noted at end of list.

RA1 - Inside Set-up before Participant Arrives (290)

RA2 - Check Weather & Drive Conditions for acceptability; move vehicle to AgeLab
parking area (as needed)
In-Lab Start Phase
1. When Participant Arrives - Consent Forms / Payment Form / Emergency Contact Form
2. Review of Eligibility (Interview & MoCA)
3. Pre-Experimental Questionnaire
4. N-Back Training
5. Workload Scale Rating Explanation
Bathroom Break
6. Physiological Sensor Attachment
Move to Vehicle
7. Set Participant Up in Vehicle / Eye Tracking Calibration / N-Back Practice
8. Training in MIT parking lot on first set of in-vehicle tasks
9. Start On-Road Run to I95 North & turn onto I495 South
10. Part I on-road tasks
Rest Stop & Bathroom Break
11. Recalibrate eye-tracking system (if needed)
12. Training at rest stop on second set of in-vehicle tasks
13. Return to highway and drive I495 North
14. Part II on-road tasks
15. Exit I495 onto I93 South
16. Phone task
17. Return to MIT
Back at MIT (RA1/RA3 may assist with inside tasks so RA2 may make second run/return car)
18. Post-Experimental Questionnaire
© MIT AgeLab 2013
Page A140 of 293
19. Supplementary Health Questionnaire
20. In Lobby - Payment / Documents Stored Safely
21. Transfer Computer Data from vehicle to lab at End of Day
22. Return Vehicle to Garage Parking (if last run of day)
© MIT AgeLab 2013
Page A141 of 293
Phase 1-1. Inside Set-Up Before the Participant Arrives (RA1)
Forms:
Consent Form
Payment Record Form
Emergency Contact Form
Montreal Cognitive Assessment (MoCA) Scoring Sheet
N-Back Training Sheet (2008)
Instructions on How to Fill-Out Workload Rating Scale
Pre-Experimental Questionnaire
Workload Rating Sheet
Detailed Task Rating Questionnaire
Post-Experimental Questionnaire(s)
Supplemental Health Questionnaire
Data Set Summary (2 copies – one each for in-lab (RA1) and on-road portions (RA2))
Protocol section 1 (this form)
Protocol sections 2-6, A-E, X1, X2 in folder
1.
Ensure that all required forms are in the participant packet (see above)
2.
Check that you have water available to offer to participant later.
3.
Check that the charger is plugged into the front of the physio unit (MEDAC System/3) and
the green light above the charge plug is on.
4.
Turn on PC in Room 290 if not already on and log-in to your normal MIT account.
5.
Start NeuGraph by selecting NeuGraph icon on the desktop.
6.
AgeLab MIT should come-up as the default user.
Select [Login] button.
(No password required.) (FYI [password is agelab123])
7.
Select [Select Participant] button.
8.
Double Click on name “2012b Baseline” and then select [Return to Main Menu]
Leave computer running.
These steps will save time when it comes to actually check signal quality later.
© MIT AgeLab 2013
Page A142 of 293
Phase 1-2. When the Participant Arrives
1.
Record time participant arrives.
Time __________
Narrative for When the Participant Arrives
“Welcome to the MIT AgeLab. My name is insert name here. “
“I will be working with you throughout the experiment today.”
(or) (normal circumstances with two RAs)
“I will be working with you during the first part of the study today and insert name here, who
will join us later, will take over and help you through the completion of the study.”
“Do you have a cell phone with you? (If yes) – Would you please turn it off for the duration of
your time here? Thanks. “
(If unwilling to turn-off cell phone, then they cannot participate and do not qualify for payment since
incoming phone call interruptions will impact study results.)
© MIT AgeLab 2013
Page A143 of 293
Phase 1-3. Study Overview
“Before we begin, I’d like to describe the basic outline of the study. After hearing about the
study, if you are still interested in participating, I will ask you to read and sign an informed
consent form. The informed consent provides you with a written description of the study,
eligibility requirements, compensation, risks of participation, and your rights as a research
participant. If you agree to participate, please remember that you are still free to end your
participation at any time by alerting me or another member of the research staff that you
wish to do so.”
“The study that you are considering participating in is part of a larger on-going research
project called “Detecting Driver Stress” and in this particular phase we are looking at the
amount of effort or workload that is associated with carrying out different tasks while
driving.”
“We will ask you do a series of tasks during today’s drive. These tasks will include operating
the radio using traditional manual controls as well as using a voice command interface to do
a number of things including operating the radio, entering addresses into the navigation
system, selecting songs stored on a portable storage device, and dialling a pre-saved phone
number. In addition, you will be asked to do a number task that we will train you on here in
the lab.”
“The vehicle being used in this study is instrumented with several cameras to record what is
happening outside as well as inside the vehicle. There are sensors that detect how you are
operating the vehicle. A microphone will record all audio, including conversations, during
the course of the study. During the drive, you will be asked to perform a number of tasks by
an automated system in the car. A research associate will sit in the back of the vehicle and
monitor the recording equipment. During the study, the research associate is not permitted
to have casual conversations. However, if you have any questions about the vehicle, research
procedure, tasks, directions or anything else related to the study please feel free to ask.”
“The drive will be divided into two segments with a break in the middle. At the break, you
will be getting off the highway and stopping the car at a designated location. Because we will
be taking a break part way through the drive, we will introduce you to approximately half
the tasks initially and train you on how to do the other half of the tasks during the break.
This will reduce the number of new things you need to learn at one time.”
2.
Provide participant with the overview above.
© MIT AgeLab 2013
Page A144 of 293
Phase 1-4. Consent Forms
“This informed consent form describes the different stages of the experiment, risks to you,
compensation, confidentiality and your rights as a participant. As you read it, please pay
particular attention to the procedures section (point to it), and the risks and discomforts section
(point to it). I’d like to emphasis a few points in this specific section. You will be solely
responsible for any traffic tickets you may receive. If you feel uncomfortable with driving
during any part of the experiment, you are free to stop the experiment. If this happens, inform
the research associate and pull off to the side of the road at the nearest safe location.”
“Please read through each section thoroughly and decide if you would still like to participate.
As you read over the form, I would be happy to answer any questions you have.”
(Sit with the participant and answer any questions as they read the consent form)
“Now that we have reviewed the different parts of the protocol, would you still like to
participate?”
(If no) - “Thank you for coming in.”
(If yes) - “Ok, the last page has a place for you to sign and date the form. Please be sure to also
print your full name.”
“I will also sign and date the forms and provide you with copies in a few minutes.”
3.
Consent form.
Sit with the participant and answer any questions as they read the
consent form.
© MIT AgeLab 2013
Page A145 of 293
Phase 1-5. Driver’s License
“I now need to verify that you have a valid driver’s license. May I have your license?”
“Perfect, I’m going to hold on to your license and make a copy of it while you fill out the
questionnaire in a few minutes. I will give it back to you along with a copy of the consent
form.”
(If the license is not valid) - “I am sorry but your driver’s license has expired and we won’t be
able to run the experiment today.”
(Participant does not get paid.)
4.
Valid driver’s license.
yes / no
Phase 1-6. Payment and Emergency Contact Forms
“Now I need to collect some information required by MIT to pay you for participation. It is kept
separate from our recruitment database for confidentiality. A research associate will only have a
record of this information until it is passed on to MIT’s Accounting office to process, at which
point no reference to your social security number will be kept by the MIT AgeLab. You will be
paid a minimum of $70 at the end of today’s visit and will have the opportunity to earn a $10
incentive based upon your performance on a number of tasks during the drive. MIT reports all
payments of over $100 to the IRS as per federal requirements.”
(Note to RA if participant needs more information direct them to the IRS website. www.irs.gov )
5.
Fill out MIT payment form.
This is filled out by the experimenter – if the address is correct on
the license you can copy name and address to the form while the
participant is filling out the survey
“Now we need to fill out an emergency contact form. Who would you like us to contact in the
case of an emergency?”
6.
Fill out MIT Emergency Contact form.
7.
Place consent, payment form, emergency contact form, emergency
contact form and driver’s license on a clipboard and verify that
you have completed the steps.
© MIT AgeLab 2013
Page A146 of 293
Phase 1-7. Review of Initial Screening Criteria
“I now need to ask you a series of questions to verify your eligibility. Many of these
questions can be answered with a simple yes or no. Please answer each question as best as
you can. Answering any question is voluntary. If you feel uncomfortable with any question,
feel free not to answer it. Remember that participants from different backgrounds are
participating in this study, so some of the questions may not apply directly to you. “
(If a participant does refuse to answer a question, thank them for coming in but explain that all questions
do need to be answered if they want to be considered for inclusion.)
C1.
(Is the participant: MALE
/
FEMALE )
YES
NO
1
2
C2.
Are you between the ages of 20 and 29 or 60 and 69 years of age?
C3.
What is your Date of Birth __ ___/__ __/___ ___ ___ ___ MMDDYYYY)
C4.
Have you had a driver’s license for over three years?
1
2
C5.
Do you drive three or more times a week?
1
2
C6.
Would you be comfortable driving a full size sedan such as a
Ford Lincoln as part of the study?
1
2
C7.
Are you in reasonably good health for your age?
1
2
C8.
(Does the participant appear to understand and speak English?)
1
2
D1.
Have you been the driver in a police reported accident in the past
year?
1
2
D2.
Do you wear glasses to drive?
1
2
D3.
Do you wear contacts to drive?
1
2
D4.
Have you ever participated in a driving study at MIT?
1
2
D5.
If yes, have you ever been trained in the n-back number task?
1
2
E1.
Have you had any major medical illnesses resulting in any
hospitalization within the past 6 months?
1
2
© MIT AgeLab 2013
Page A147 of 293
E2.
Do you have a diagnosis of Parkinson’s, Alzheimer’s disease,
dementia, mild cognitive impairment (MCI), or any other
neurological problems?
1
2
E3.
Are you currently been treated for a psychological or psychiatric
disorder?
1
2
Have you ever had any of the following:
E4a.
Heart failure?
1
2
E4b.
Angioplasty or coronary artery bypass grafting (CABG)?
1
2
E4c.
A pacemaker (to control heart rate)?
1
2
E4d.
A stroke or transient ischemic attack (TIA)?
1
2
E4e.
A diagnosis of diabetes?
1
2
There are a few medications that I need to ask you about. Have you used any of the following in the
past 12 months?
F1.
Anti-convulsant medication
1
2
F2.
Immunosuppressive drugs or cytotoxic drugs
1
2
F3.
Anti-depressant medication
1
2
F4.
Anti-psychotic medication
1
2
F5.
Anti-anxiety medication
1
2
F6.
Medications to treat a major medical condition such as cancer
1
2
1
2
In the past two days have you used any?
F7.
Medications that made you drowsy?
If the potential participant’s status falls in any of the shaded regions, then they may be ineligible.
Please seek supervisor approval before continuing. If no supervisor approval is available,
reschedule participant.
Notes:
D2-D5 are not exclusion criteria; this data is being collected for evaluation purposes.
© MIT AgeLab 2013
Page A148 of 293
F2 - Intent is to screen for serious medical conditions – if immunosuppressive medication is for treatment
of mild to moderate arthritis that does not markedly impact the individual’s ability to drive, they may be
included; check with supervisor before continuing.
F7 – If individual regularly uses medications that cause drowsiness, they are to be excluded. If used on a
limited basis such as a cold medicine, individual may be rescheduled for a time slot when they have been
off medication for 48 hours or more.
1.
Is the individual eligible based on data collected on this screening
form?
Eligible / Ineligible
(If any of the light grey boxes are checked, they are not eligible
unless status reviewed and approved by a supervisor.)
© MIT AgeLab 2013
Page A149 of 293
Phase 1-8. Mental Status Screening (Montreal Cognitive Assessment - MoCA)
“The next step is a cognitive assessment; I will be guiding you through a series of tasks
and questions on this worksheet”
This step requires the MoCA task scoring sheet (Version 7.1 Original Version).
(The instructions below are a copy of MoCA Version August 18, 2010 © Z. Nasreddine MD.)
1. Alternating Trail Making:
Administration: The examiner instructs the participant: "Please draw a line, going from a
number to a letter in ascending order. Begin here [point to (1)] and draw a line from 1 then to A then
to 2 and so on. End here [point to (E)]."
Scoring: Allocate one point if the participant successfully draws the following pattern: 1 −A2- B- 3- C- 4- D- 5- E, without drawing any lines that cross. Any error that is not immediately
self-corrected earns a score of 0.
2. Visuoconstructional Skills (Cube):
Administration: The examiner gives the following instructions, pointing to the cube: “Copy
this drawing as accurately as you can, in the space below”.
Scoring: One point is allocated for a correctly executed drawing.
• Drawing must be three-dimensional
• All lines are drawn
• No line is added
• Lines are relatively parallel and their length is similar (rectangular prisms are accepted)
A point is not assigned if any of the above-criteria are not met.
3. Visuoconstructional Skills (Clock):
Administration: Indicate the right third of the space and give the following instructions:
“Draw a clock. Put in all the numbers and set the time to 10 past 11”.
Scoring: One point is allocated for each of the following three criteria:
• Contour (1 pt.): the clock face must be a circle with only minor distortion acceptable (e.g.,
slight imperfection on closing the circle);
• Numbers (1 pt.): all clock numbers must be present with no additional numbers; numbers
must be in the correct order and placed in the approximate quadrants on the clock face;
Roman numerals are acceptable; numbers can be placed outside the circle contour;
• Hands (1 pt.): there must be two hands jointly indicating the correct time; the hour hand
must be clearly shorter than the minute hand; hands must be centred within the clock face
© MIT AgeLab 2013
Page A150 of 293
with their junction close to the clock center. A point is not assigned for a given element if any
of the above-criteria are not met.
4. Naming:
Administration: Beginning on the left, point to each figure and say: “Tell me the name of this
animal”.
Scoring: One point each is given for the following responses: (1) lion (2) rhinoceros or rhino
(3) camel or dromedary.
5. Memory:
Administration: The examiner reads a list of 5 words at a rate of one per second, giving the
following instructions: “This is a memory test. I am going to read a list of words that you will have
to remember now and later on. Listen carefully. When I am through, tell me as many words as you can
remember. It doesn’t matter in what order you say them”. Mark a check in the allocated space for
each word the participant produces on this first trial. When the participant indicates that
(s)he has finished (has recalled all words), or can recall no more words, read the list a second
time with the following instructions: “I am going to read the same list for a second time. Try to
remember and tell me as many words as you can, including words you said the first time.” Put a
check in the allocated space for each word the participant recalls after the second trial. At the
end of the second trial, inform the participant that (s)he will be asked to recall these words
again by saying, “I will ask you to recall those words again at the end of the test.”
Scoring: No points are given for Trials One and Two.
6. Attention:
Forward Digit Span: Administration: Give the following instruction: “I am going to say some
numbers and when I am through, repeat them to me exactly as I said them”. Read the five number
sequence at a rate of one digit per second. Backward Digit Span: Administration: Give the
following instruction: “Now I am going to say some more numbers, but when I am through you
must repeat them to me in the backwards order.” Read the three number sequence at a rate of one
digit per second.
Scoring: Allocate one point for each sequence correctly repeated, (N.B.: the correct response
for the backwards trial is 2-4-7).
Vigilance: Administration: The examiner reads the list of letters at a rate of one per second,
after giving the following instruction: “I am going to read a sequence of letters. Every time I say
the letter A, tap your hand once. If I say a different letter, do not tap your hand”.
Scoring: Give one point if there is zero to one errors (an error is a tap on a wrong letter or a
failure to tap on letter A).
© MIT AgeLab 2013
Page A151 of 293
Serial 7s: Administration: The examiner gives the following instruction: “Now, I will ask you to
count by subtracting seven from 100, and then, keep subtracting seven from your answer until I tell
you to stop.” Give this instruction twice if necessary.
Scoring: This item is scored out of 3 points. Give no (0) points for no correct subtractions, 1
point for one correction subtraction, 2 points for two-to-three correct subtractions, and 3
points if the participant successfully makes four or five correct subtractions. Count each
correct subtraction of 7 beginning at 100. Each subtraction is evaluated independently; that is,
if the participant responds with an incorrect number but continues to correctly subtract 7
from it, give a point for each correct subtraction. For example, a participant may respond “92
– 85 – 78 – 71 – 64” where the “92” is incorrect, but all subsequent numbers are subtracted
correctly. This is one error and the item would be given a score of 3.
7. Sentence repetition:
Administration: The examiner gives the following instructions: “I am going to read you a
sentence. Repeat it after me, exactly as I say it [pause]: I only know that John is the one to help
today.” Following the response, say: “Now I am going to read you another sentence. Repeat it after
me, exactly as I say it [pause]: The cat always hid under the couch when dogs were in the
room.”
Scoring: Allocate 1 point for each sentence correctly repeated. Repetition must be exact. Be
alert for errors that are omissions (e.g., omitting "only", "always") and substitutions/additions
(e.g., "John is the one who helped today;" substituting "hides" for "hid", altering plurals, etc.).
8. Verbal fluency:
Administration: The examiner gives the following instruction: “Tell me as many words as you
can think of that begin with a certain letter of the alphabet that I will tell you in a moment. You can
say any kind of word you want, except for proper nouns (like Bob or Boston), numbers, or words that
begin with the same sound but have a different suffix, for example, love, lover, loving. I will tell you to
stop after one minute. Are you ready? [Pause] Now, tell me as many words as you can think of that
begin with the letter F. [time for 60 sec]. Stop.”
Scoring: Allocate one point if the participant generates 11 words or more in 60 sec. Record
the participant’s response in the bottom or side margins.
9. Abstraction:
Administration: The examiner asks the participant to explain what each pair of words has in
common, starting with the example: “Tell me how an orange and a banana are alike”. If the
participant answers in a concrete manner, then say only one additional time: “Tell me another
way in which those items are alike”. If the participant does not give the appropriate response
(fruit), say, “Yes, and they are also both fruit.” Do not give any additional instructions or
clarification. After the practice trial, say: “Now, tell me how a train and a bicycle are alike”.
© MIT AgeLab 2013
Page A152 of 293
Following the response, administer the second trial, saying: “Now tell me how a ruler and a
watch are alike”. Do not give any additional instructions or prompts
Scoring: Only the last two item pairs are scored. Give 1 point to each item pair correctly
answered. The following responses are acceptable:
Train-bicycle = means of transportation, means of travelling, you take trips in both;
Ruler-watch = measuring instruments, used to measure.
The following responses are not acceptable: Train-bicycle = they have wheels; Rulerwatch =
they have numbers.
10. Delayed recall:
Administration: The examiner gives the following instruction: “I read some words to you earlier,
which I asked you to remember. Tell me as many of those words as you can remember.” Make a check
mark ( √ ) for each of the words correctly recalled spontaneously without any cues, in the
allocated space.
Scoring: Allocate 1 point for each word recalled freely without any cues.
11. Orientation:
Administration: The examiner gives the following instructions: “Tell me the date today”. If
the participant does not give a complete answer, then prompt accordingly by saying: “Tell me
the year, month, exact date, and day of the week].” Then say: “Now, tell me the name of this place,
and which city it is in.”
Scoring: Give one point for each item correctly answered. The participant must tell the exact
date and the exact place (name of hospital, clinic, office). No points are allocated if
participant makes an error of one day for the day and date.
TOTAL SCORE: Sum all subscores listed on the right-hand side of administration / scoring
sheet. Add one point for an individual who has 12 years or fewer of formal education, for a
possible maximum of 30 points. A final total score of 26 and above is considered normal.
2.
3.
Enter Total Score at right:
Based on MoCA score above, is participant:
(25<= is ineligible)
Score: ________
Note: This initial automatic cutpoint was revised after a review of MoCA scores from a study initiated
prior to the current study. Following the revised guidelines, if a participant obtained a score between 23
and 25 (i.e. less than 26), the case was individually reviewed by the RA with a senior staff member prior to
approval to allow the participant to move on to the parking lot training portion of the protocol.
Participants obtaining scores less than 23 were automatically withdrawn from the study at this point. See
additional discussion of this issue in the main body of the report.
© MIT AgeLab 2013
Page A153 of 293
Phase 1-9. Verification of Eligibility Review
4.
If participant was ineligible in any section, then they are
ineligible for the study. Indicate if they are:
(If they are ineligible, respond as outlined below:)
“It appears as though you do not meet all of the eligibility requirements for this study. “
(Unless it is very obvious that the mistake is on the participant’s side) - Since it appears as though
this is due to an error on our part, I am going fill out a payment form and give you $70.
(If the mistake is in some way deceitful) - Since it appears as though you answered these questions
differently during the scheduling process, I will be unable to pay you for this visit.
Phase 1-10. Checklist for Participant Consent & Verification
Experimenters are to complete the checklist below to confirm that they have gone covered each
of the critical steps specified:
5.
6.
7.
8.
9.
10.
11.
Explained the steps of the protocol and sections of the consent
form.
Highlighted features with a pen while you move through them
with the participant.
Driver’s license valid and name correct.
Payment form is complete.
You have the driver’s license to be copied.
You have the consents, payment form, and emergency contact
form
Consents are signed by both you and the participant
Verification of eligibility is complete
© MIT AgeLab 2013
(YES / NO)
(YES / NO)
(YES / NO)
(YES / NO)
(YES / NO)
(YES / NO)
(YES / NO)
Page A154 of 293
Phase 1-11. Pre-Experimental Questionnaire
Completing Questionnaire Narrative
“Now I need to have you fill out a questionnaire.”
(Hand participant the pre-experimental questionnaire.)
“While you are doing this, I am going to go make a copy of the consent form and your driver’s
license. I will be back in a few minutes.”
(Copy items)
(Look through blinds to see if participant appears to have finished. If unsure, enter and ask how is it
going? Do they have any questions? If not done, leave again so then can finish. Recheck as needed.)
(When participant has finished the questionnaire, return license and copies of forms.)
“Here is your license and a copy of the consent form for your records.”
27.
28.
Participant was given a copy of the consents
(YES / NO)
Participant was given their license
(YES / NO)
© MIT AgeLab 2013
Page A155 of 293
Phase 1-12. N-Back Training
“Part of the experiment will involve performing a set of number tasks. You are going to
learn how to perform a few versions of these tasks and practice each with a few trials.
This sheet provides an overview of the task.”
(Direct the participant’s attention to the N-back Instructions sheet.)
“Please follow along as I explain each version.”
“Please follow along as I explain each version. The first version is called the zeroback. During this task, I will read a list of ten single digit numbers. As I read each
number, you are to repeat out loud the last number that you’ve heard. For example, if
I were to say the number 3, you would say 3; then if I said 2, you would say 2; then if
I said 6, you would say 6, and so on. Try to be as accurate as you can be.”
(Point to the appropriate “I say” and “you say” squares on the sheet as you read)
I say
You say
3
3
2
2
6
6
7
7
1
1
“Let’s practice with an actual set of numbers:”
7
4
6
8
9
0
5
2
1
Score: /10
3
(If the participant misses more than 1 response, repeat up to four more trials. Write the
numbers in
the trial above on a separate sheet backwards and then in the same order as they appear
alternating up to twice. Present one trial at a time trying to improve the participants
understanding
to the point where they respond correctly to seven of ten stimuli.)
“The second version of the task is called the one-back, which simply means that as I
read each list of ten numbers, you are to repeat out loud the number before the last
number that you heard. For example, if I said 3, you would say nothing, then if I said
2, you would say 3, then if I said 6, you would say 2, and so on. Try to be as accurate
as you can be.”
© MIT AgeLab 2013
Page A156 of 293
(Point to the appropriate “I say” and “you say” squares on the sheet as you read the
above.)
I say
You say
3
2
3
6
2
7
6
1
7
5
2
0
7
1
4
6
3
9
Score: /9
8
“Let’s try that again. Just repeat out loud the number before the last number that
you’ve heard. For example, if I were to say the number 1, you would say nothing,
then if I said 2, you would say 1, then if I said 3, you would say 2, and so on. Try to
be as accurate as you can be.”
“Let’s practice:”
1
7
3
8
9
0
5
Score: /9
6
2
4
(If the participant misses more than 2 in the last practice trial repeat up to four more
trials. Write the numbers in the two trials above on a separate sheet backwards and then
in the same order as they appear. Present one trial at a time trying to improve the
participants understanding to the point where they respond correctly to seven of ten
stimuli.)
“The final version of the task is called the two-back, which simply means that as I
read each list of ten numbers, you are to repeat out loud the number that was read
two numbers ago. For example, if I were to say the number 3, you would say
nothing, then if I said the number 2, you would say nothing, then if I said 6, you
would say 3, then if I said 7, you would say 2, and so on. Try to be as accurate as you
can be.”
(Point to the appropriate “I say” and “you say” squares on the sheet as you read the
above.)
I say
You say
© MIT AgeLab 2013
3
2
6
3
7
2
1
6
Page A157 of 293
Score:
9
0
6
7
1
4
2
3
5
/8
8
“Let’s try another example. Just repeat out loud the number that was read two
numbers ago. For example, if I were to say the number 1, you would say nothing,
then if I said 2, you would say nothing, then if I said 3, you would say 1, then if I said
4, you would say 2, and so on. Try to be as accurate as you can be.”
Score:
6
5
3
4
7
2
1
8
0
/8
9
“Let’s try another one. Just repeat out loud the number that was read two numbers
ago. For example, if I were to say the number 0, you would say nothing, then if I said
9, you would say nothing, then if I said 1, you would say 0, then if I said 5, you
would say 9, and so on. Try to be as accurate as you can be. “
Score:
0
9
1
5
8
2
4
6
3
/8
7
(If participants misses more than 4 in the last practice trial repeat up to six more trials.)
EXTRA 2-BACKPRACTICE SETS (use as needed)
8
9
3
2
4
1
7
6
0
Score:
5
0
8
1
2
7
4
3
5
Score:
6
/ 8
9
/ 8
9
Score:
0
7
3
6
© MIT AgeLab 2013
4
2
8
5
1
/ 8
Page A158 of 293
5
0
6
7
1
4
2
3
9
Score:
8
5
3
4
7
2
1
8
0
Score:
9
/ 8
6
/ 8
3
Score:
7
0
9
1
5
8
2
4
6
1. Did the participant complete the 0-back training?
(YES / NO)
(YES / NO)
(YES / NO)
/ 8
Phase 1-13. Instructions on How to Fill-Out Workload Rating Scale
Hand participant “Instructions on How to Fill-Out Workload Rating Scale” and ask them to read.
Answer questions as needed.
“At the mid-point and at the end of the study today we are going to ask you to rate how much
effort was involved in caring out each of the tasks. Please read through the instructions on this
sheet which explain how we would like you to make this rating. Let me know if you have any
questions about how to make these ratings.”
Phase 1-14. Bathroom Break Opportunity
“We are now going to take a short break. This will be a good time to visit the rest room since you
will be in the car for approximately 2 hours before the next opportunity.”
“We also have some bottles of water here if you are thirsty. Would you like some water?”
© MIT AgeLab 2013
Page A159 of 293
Phase 1-15. Physiology Sensor Attachment
Sensor Attachment Narrative
SEE DOCUMENT EKG & EDA Sensor Attachment Protocol (Pictorial Summary) FOR DETAILS OF
STEPS BELOW.
The narrative below is meant to be a guide. RA’s may vary the language as appropriate.
Now we are ready to attach the physiological sensors. We’ll use an alcohol pad to clean a
spot on your left side (point to your left side) and an area below either side of your
collarbone (point to just below your collarbone on both left and right). This helps us get a
better reading from the EKG sensor, which measures your heart rate. These sensors will be
placed on the areas where the skin is cleaned. We will use some paper medical tape on a
couple of locations on your skin and clothes to organize the wires. It doesn’t damage your
clothes or hurt when removed.
If you would feel more comfortable having a female (male) research assistant attach these
sensors, this can be arranged. If you have any questions about the sensor attachment, please
feel free to ask me at any point. Are you ready to begin or would you like me to get a female
(male) research assistant?
If they want an RA of another gender, tell them to take wait for a few moments and you will
arrange this. Leave the room and arrange for assistance.
As I mentioned before, I’m going to wipe your skin at the locations I described earlier with
an alcohol pad before placing the EKG sensors. Would you raise your shirt/blouse on the left
side a little for me? Thanks.
Use an alcohol pad to clean the participant’s skin on the left side at approximately the level of
the bottom rib. Then clean the area below the left & right collar bones and, on the non-dominant
side. (In studies using the PPG to measure blood flow from the earlobe, one would clean the
earlobe and just behind the ear over the mastoid bone at this point.)
Now I am going to attach the EKG sensors. These monitor heart rate.
Attach snap leads to the EKG electrodes before removing the electrodes from the plastic sheet.
Attach the EKG snap with blue dot below the RIGHT collarbone, the lead with the black snap
below the LEFT collarbone, and plain red snap on left side near the bottom rib. If the participant
has hair on their chest, the sensors can be placed on or above the collar bone.
NOTE: for studies that require participants to interact with device interfaces on the right side of
the steering wheel, such as this study, the electrodermal sensors are always attached on left
hand. For studies that do not require use of hands except for steering, then it is preferable to use the non© MIT AgeLab 2013
Page A160 of 293
dominant hand since people tend to be slightly more reactive on their non-dominant hand. Even though
handedness will not change sensor placement in this study, record if they are right or left
handed as this may be a meaningful variable for some participants.
Are you right handed or left handed?
This next sensor monitors the perspiration on your fingertips. This kind of sweat tends to
increase with stress and decrease with relaxation. These sensors will be attached to the ring
and middle fingers of your left hand.
Attach skin conductance sensors on the ring and middle fingers of the left hand. See details
below and pictorial summary document.
Can you make your left hand into a fist for me? And now move it across your body so it just
touches your right shoulder? Good. I am going to place some more paper tape on you to
hold the wires in place.
Apply tape to back of hand to hold down skin conductance wires as shown in the pictorial
document.
(YES / NO)
Check
4. 1 with participant if they want a different gender (male/female) research
assistant to attach the sensors.
Record
5.
if participant writes with right or left hand.
Handedness:
Right / Left
(However, sensors will always be placed on the left hand for this study.)
Use
6. an alcohol pad to clean participant’s skin on the left side at approximately
the bottom rib, and the area just below the left & right collarbones.
(YES / NO)
[If PPG was being recorded, one would, on the non-dominant side, clean the
earlobe and just behind the ear (over the mastoid bone).]
Attach
EKG snap with blue dot below RIGHT collarbone, black snap below the
7.
LEFT collarbone, and plain red snap on left side near the bottom rib. If the
participant has hair on their chest, the sensors can be placed on or above the collar
bone.
Attach
the gold skin conductance sensors on ring and middle fingers of the left
8.
hand (see pictorial document for details). The lead wires should fold up and back
over the top side of the fingers and held in place using paper tape. To determine
where paper tape should be placed on the back of the hand, have the participant
make a fist and draw their arm up toward their right shoulder, then attach tape.
Bending the fingers and elbow in this way corresponds to maximum pull that will
occur on the lead wire; tape lead wire on the back of the hand, part way up the
lower and upper segments of the arm and to top of shoulder.
The
9. lead wires should be brought together at the top of the left shoulder and
taped in place
© MIT AgeLab 2013
(YES / NO)
(YES / NO)
(YES / NO)
Page A161 of 293
Phase 1-16. Physiology Sensor Verification (NeuGraph)
(The steps below assume that a participant file has already been created in NeuGraph.)
“I’m going to check that the physiological sensors are recording correctly. If there is any
need to adjust the sensor connections, we will do this as I check the signals.”
If NeuGraph is NOT yet running and participant name not yet selected:
9.
Turn on PC in Room 290 if not already on and log-in to your normal MIT account.
Start NeuGraph by selecting NeuGraph icon on the desktop.
10.
AgeLab MIT should come-up as the default user. Select [Login] button.
11.
(No password required.) (FYI [password is agelab123])
Click on [Select Participant].
12.
Double Click on name “2012b Baseline”.
13.
(All data saved under this name and referenced by date and time.)
10.Click on [Return to Main Menu]
If participant name has been selected:
11.Unplug black charger cable from the front of the unit.
12.Click on [Select Protocol].
13.Click once on the protocol “EKG Verification” so it is highlighted.
14.Click on [Run Protocol] and then on [OK].
15.Verify EKG - Look for periodic upward spikes at a rate of approximately 1 to 2 per second
(i.e. 1 per second equals 60 beats per minute). If the signal is inverted (see below) switch
the active leads to correct the direction.)
If you are getting a flat blue line running at the top of the screen, try disconnecting the skin conductance sensor
and see if signal appears. (If Bruce is available check with him.) If necessary run with skin conductance sensor
disconnected from unit to get good EKG signal. (Let Bruce know that this occurred.)
16.Click on red X in upper right corner of screen to exit and return to the main menu.
The “EKG Verification” protocol displays raw 250 sample per second signal in burst mode:
What we want to see:
Normal EKG with positive going r-spike.
© MIT AgeLab 2013
What we don’t want to see:
EKG with inverted r-spike (FIX).
Page A162 of 293
Checking remaining signals.
17.Click once on ‘2012b Protocol’ to highlight.
18.Click [Run Protocol] and then [OK]
19.SCL (orange) should show a reading of at least 2.0 and SCR (gray) of at least 30.
Ask participant to take deep breath, look for SCL to increase a minimum of 0.5 units; the
SCR increase should be more dramatic and normally go over at least 40.
Adjust sensors as needed if you do not seem to be getting a correct signal. If blue EKG
signal is running as flat line at the top of the screen, see note on previous page.
20.Once you are done adjusting signals, click on [Start Session] and record approximately 30
to 40 seconds of baseline / reference data.
(This is intended to allow us to track what signal quality looked like in the lab.)
21.Click [End Session] and select option to [Save Session] data.
Entry of Notes is optional – just use if there is something unusual to report.
22.If another participant will be run later, select option to return to Main Menu.
If done for the day, select options to Exit software.
23.Plug black charger cable back into the front of the unit to charge physio unit.
Example SCL & SCR Tracings
The unit produces two electrodermal signals. Skin Conductance Level (SCL) is displayed in orange on the
graph above. Skin Conductance Response (SCR) is shown in gray. You can test the electrodermal signal by
having the participant take a deep breath. A deep breath should produce a clear rise in the gray SCR signal
as seen on the right side of the screen above. Typical SLC values can generally be expected to be in the 2-25
micromho range with most participants falling in the 2-5 range under low stress conditions. The base SCR
reading should be around 30 to 35 and should rise over at least 40 when the participant takes a deep breath.
“Everything looks good.”
© MIT AgeLab 2013
Page A163 of 293
Phase 1-17. Moving to Car
“I (or insert name here) will be meeting you back there in a few minutes and will then walk with
you out to the test vehicle.”
24.
Leave in a folder in the bin in room 288:
 Consents, payment form, copy of license, emergency contact pre
questionnaire(s), and your data summary sheet
Make sure to pass on to driving RA:
 Protocol sections 1, 2 – 5, A-E, X1, X2
 ALL Mid and Post-Experimental Questionnaires including
Supplemental Health
 Data summary sheet for driving portion
RA1 fill out data summary sheet for intake portion.
25.
Scan and upload all documents
26.
© MIT AgeLab 2013
Page A164 of 293
Section 2 Vehicle Setup
Participant Code
___________________ Protocol Configuration __________________
Date
________________________________________________________________
RA (s)
________________________________________________________________
Notes for the research assistant - Please review before each participant run.
It is important to remember that when you are involved in data collection with the AgeLab on-road
vehicle, safety is a priority. Good judgment and attention to detail is key to ensuring the safe collection of
good experimental data. If a non-critical derivation in the protocol occurs that does not compromise
participant safety (i.e. procedural, technical) correct things as best as possible and finish running the
participant. As a matter of policy, any comments, notes, or observations about the experimental procedure
or equipment should be recorded on this form. No detail is too small to record. At the conclusion of the
experimental session you should transcribe the major points to a data set summary form. You are also
responsible for making a log entry noting success or failure as well as entering a short comment or
description as appropriate.
As a research associate involved with this experiment, you are responsible for actively monitoring the
participant, ensuring that all instructions detailed below are described clearly to the participant and that
any questions are answered to the best of your ability. Please make sure to read instructions clearly. If, at
any time, the participant has reservations about participating or continuing to participate in this
experiment, you should encourage them to withdraw. DO NOT ENCOURAGE PARTICIPATION! If a
question arises that you are unable to answer, seek assistance from a senior member of the research staff.
DO NOT GUESS.
If an undergraduate researcher consents the participant, the RA operating the vehicle should review
eligibility materials before departing MIT.
If, during the course of an experiment, a situation arises where you feel as though the participant’s or your
safety is compromised, you need to use your best judgment to address the situation. For example, if you
feel that the participant is not adequately in control of the vehicle, immediately advise them in a calm
manner to pull off to the side of the road at a safe point. Call a contact to report the adverse event. Drive
directly back to MIT. Unless there is an equipment failure or planned pause in data collection, the DAQ
should ALWAYS be recording data. If an accident of any type occurs, try to handle the situation calmly. If
it is safe to do so, retrieve the removable drive and DO NOT RELEASE THIS TO ANYONE until the
matter is discussed with officials here at MIT. For purposes of emergency support, you should have a
working cell phone with you in the vehicle at all times.
During the course of the study you should minimize any unnecessary actions, e.g. moving around the
room, getting papers organized, extra key strokes etc. Low level noise and slight movements will impact
participant arousal. In addition, due to the significant number of functional actions and active monitoring
required of the RA in this experimental protocol, RAs are not to use the lab computers or other computers
or communication devices to monitor e-mail, work on other projects, etc. so that full attention can be
devoted to active monitoring of the participant, equipment, and protocol. The sole exception would be the
© MIT AgeLab 2013
Page A165 of 293
use of text messaging to communicate with another staff member regarding significant equipment or
participant related issues. While the desire of RAs to multi-task for purposes of carrying out other MIT
related work is appreciated, the nature of this particular protocol does not lend itself to such activities.
Points of contact: [Note: Emergency contacts and phone numbers go here.]
NOTE: Whenever this procedure instructs you to press an F key (F1, F12, etc.) you must make
sure that the DAQ window on the DAQ computer is active, or nothing will happen.
Checking External Factors that May Impact the Experiment
Before beginning with a participant during a driving assessment visit, you need to determine if
external factors such as traffic or weather are likely to adversely impact the experiment.
Boston.com and weather.gov are two sources that provide real-time updates. If you feel that
heavy traffic on I-93 North or South is occurring in an area that will impact data collection, you
need to make a decision based upon the traffic reports if some alteration of the experimental
procedure is needed or if there is a good chance that traffic will clear in time for data collection.
In cases such as a reported accident an hour before you are scheduled to depart, there is a very
good chance that the accident will be cleared before you get to that area. In the case of heavy rain,
you need to decide if visibility will be impaired during the period when you will be collecting
data. If you decide that there will be traffic, weather or other conditions are very likely to impact
the experiment, you need to decide if a brief delay will help ensure a successful run or if you
should reschedule the participant. You may choose to wait up to 30 minutes between the
procedures in 290 and driving. This option is available even if it will delay subsequent
participants! All efforts should be made to run the participant if traffic or weather may improve
between the time you leave MIT and the start of data collection.
© MIT AgeLab 2013
Page A166 of 293
Support Materials:
The “on-road” RA is responsible for confirming that he/she has each of the items listed below.
This includes checking the package of forms / questionnaires that may be handed off to the onroad RA by another RA.
Replenished Items
Intake Protocol (section 1) (to follow participant)
Post-Intake protocol sections in 3-ring binder
Workload Rating Scale
Detailed Task Rating Scale
Post-Experimental Questionnaire
Water & granola bars
Data summary Sheet (for driving portion)
Fixed Items
Clip board and pen
Address card(s) for Navigation Task
USB memory stick containing music
Phone
Protocol Summary:
23. Setting-up Vehicle and Eye Tracking System
24. Setting-up Participant in Vehicle
25. Back Out & Back into Parking Space
26. Create Head Model for Eye Tracker
27. In-Vehicle N-back Practice
28. Final Eye Tracking Configuration
29. Training in MIT Parking Lot on First Set of In-vehicle Tasks
30. Voice System Calibration
© MIT AgeLab 2013
Page A167 of 293
Phase 2-1. Checklist for Setting-up Vehicle and Eye Tracking System
Task
1.
Notes / Checked
Position vehicle in the space behind the building with power
plugged in to the outlet in front of the vehicle
Turn on climate control and set to 68 degrees on auto fan
2.
3.
Start and log into the computer –
user: AwareCar2, password: awarecardaq
Get a folder for all materials
4.
5.
6.
7.
8.
Make sure a clipboard and pen is available so the
participant can do the questionnaires in the car
Check that stickers to cover BLISS light in each side mirror
are in place.
Press the “engine” button once to put the car in accessory
mode.
Make sure the audio in the car is on and set to Media – Line
In. Test audio volume by playing an .mp3 from the desktop.
Check and note the mic volume. It needs to be set to 95%
9.
10.
Mic _______ %
Open n-backs folder in 2012b_audio on the C Drive. Then
open the file shuffle log andmatch the n-back trials to the
variation log. Refer to the log and write the number order
into the protocol.
Start the DAQ
11.
Select the configuration file to: 2-3 2012b_Intro.set
12.
VERIFY that the correct configuration file is selected.
(YES / NO)
13.
14.
15.
16.
17.
Turn on the IR Pod Switch and turn off the Physio Charger
Switch.
Start faceLAB and select stereo head name: MKS_test.
(note: THE DAQ NEEDS TO BE RUNNING FIRST)
“Save As” the current world model “AwareCarII_easy” to
“AwareCarII_easy_modified”.
Use the calibration square to Recalibrate the stereo-head on
faceLAB. Follow faceLAB instructions until finished. If
necessary, follow the FaceLAB procedures for Verify
Calibration.
© MIT AgeLab 2013
Page A168 of 293
18.
Review Radio Pre-sets
Pre-set
1
2
3
4
5
6
FM 1
92.9
94.5
100.7
104.1
106.7
107.9
FM2
92.9
92.9
92.9
92.9
92.9
92.9
AM
1030
1030
1030
1030
1030
1030
19.
Are all radio pre-sets correct?
20.
Set up radio
Turn Radio on; set on AM preset-4 (AM 1030)
Change to FM2, set on FM2 preset-4. (FM 92.9)
Turn radio off.
21.
Has the radio been properly set-up?
22.
Put the driver seat in an all the way back position so that the
participant will be largely forced to adjust seat after entering
the car.
Unplug vehicle power cord from outlet and store in trunk.
23.
© MIT AgeLab 2013
(YES / NO)
(YES / NO)
Page A169 of 293
Phase 2-2. Setting Participant Up in Vehicle
24.
Meet RA with participant, introduce yourself
and make sure to get all the materials you need
from the RA:
 First half of the protocol (intake)
 ALL Post-and Mid Experimental
Questionnaires needed for this visit
 Data summary sheet for driving
portion
“Welcome to the AgeLab instrumented vehicle. My name is (insert name here). Before we get
settled in the car, would you please check that you have your cell phone off and your driver’s
license with you? (Wait as needed.) Please take a seat in the driver’s seat.”
“Please take a moment to adjust the seat position and the mirrors as you would if you were
entering a car that you have never driven. If you are unfamiliar with electronic seats, the
adjustments are on the left. Please take care to ensure that the seat is in a comfortable position.”
25.
Pause until participant has completed adjusting the seat; explain how
as needed. Have them put on seat belt if they have not already.
26.
Connect physio sensors and computer system set-up and system
checks; (note: lead wires should run behind the participant’s neck –
not draped in lap.)
27.
Explanation of basic functions including how to turn-on vehicle. Test
turn signals, running lights, adjust climate controls, steering wheel
position, etc.
28.
Make sure they have the lights set all the way to the right so they will
have daytime running lights.
© MIT AgeLab 2013
Page A170 of 293
VERIFY THE FOLLOWING:
29.
Participant has both cell phone off and driver’s license with them?
( YES / NO)
If NO, have them turn off the phone and/or return to the lab to get their
license
30.
Does the participant have the seat belt on?
( YES / NO)
31.
Are daytime running lights on?
( YES / NO)
32.
If a student did the participant intake, the RA operating the vehicle
needs verify all forms are completed correctly and participant is
eligible. Are all materials filled out correctly and the participant
eligible?
(YES / NO) Your
initials: ________
33.
Vehicle is unplugged from outlet?
( YES / NO)
34.
Verify that sensors are reading properly:
EKG – Look for spikes in the correct direction
SCL ~3-20
Ask participant to take deep breath, look for increase in SCL.
35.
Go to the Physio tab, click auto calculate threshold, show the EKG and
EKG Threshold, and click on auto calculate threshold.
36.
Does the EKG look normal?
© MIT AgeLab 2013
(YES / NO)
Page A171 of 293
Phase 2-3. Back Out & Back into Parking Space
Participant is asked to back vehicle out and back into the parking space to encourage a second check that
they actually do have seat, mirror and other adjustments reasonable for actual driving. This also turns
vehicle systems on so that participant can be trained in interface functions.
“I would just like to point out two safety features that are installed in this car. When the car is
put in reverse, a camera image of the view directly behind the car will appear on the display
screen. In addition to looking over your shoulder and using the mirrors when backing up the
vehicle, the camera display is available if you wish to look at it. The car also has a proximity
sensor built into the back bumper that will beep if an object is detected right behind the car.”
37.
Explain about back-up camera and proximity sensor beep alert.
38.
The RA needs to be particularly attentive at this time, i.e. looking
themselves to be sure that no one is exiting the building adjacent to
the parking space and in danger of walking behind the car as the
participant is backing-up.
Prompt participant along the lines of the dialog below and have them
back out and back into parking space.
“In a moment I will ask you to back out of the parking space about 10 feet and then have you
pull back into the parking space. The intent here is to give you an initial feel for whether the
positioning of the car seat, the mirrors, etc. seem to feel right or need additional adjustment.
When you back-up, make sure that you check behind you for people who might be walking
across the parking lot or exiting the building and walking behind the car.”
“Please put your foot on the brake and then shift the car into reverse while keeping your foot
on the brake. Notice that an image from the rear backup camera appears when you put the car
in reverse. You are welcome to use the backup camera if you wish, but please also remember
to look behind the vehicle yourself and to use your mirrors when manoeuvring the car.”
“When you have checked for pedestrians and you feel ready, please back-up about 10 feet.”
“Good. Now pull forward back into the parking spot and stop. Ok, please put the car in
park.”
39.
Now that they have actually moved the vehicle in and out of the
parking space, encourage them to again check to see if they want to
further adjust the seat, mirrors, etc. so that they can see appropriately
and are comfortable.
“Please make any final adjustments that are needed so you are comfortable in the seat and the
mirrors are in a good position for you.”
© MIT AgeLab 2013
Page A172 of 293
Phase 2-4. Create Head Model for Eye Tracker
“Now that you are settled in, I will take a few pictures for the eye-tracking camera
calibration.”
40.
41.
42.
43.
44.
Ask the participant to make any adjustments to seat or steering wheel
to ensure proper head placement in faceLAB cameras (see his/her eyes
and mouth clearly). Make sure they are comfortable with this set up.
Adjust sunroofs as needed to get more or less light on the participant’s
face.
“Can you look straight ahead with neutral expression for me?”
Click “Set Model” to automatically create participant’s head model.
Select “HeadModel -> Adjust Eye Tracking”. If the eye corner and face
corner points were placed in wrong positions, click “Back” to adjust
manually. Then click “Next” to do gaze calibration.
“Can you look at the center of the right camera (A)?”
Click “Next”.
45.
“Can you look at the center of the left camera (B)?”
Click “Next” and “Finish”.
46.
“Can you look straight ahead and scan everywhere on the road?”
Check if participant’s gaze vector intersects with the BLUE object (front
view) correctly.
47.
“Can you look at the speedometer?”
Check if participant’s gaze vector intersects with the YELLOW object
(speedometer) correctly.
48.
“Can you look at the touch screen on the central console?”
Check if participant’s gaze vector intersects with the GREEN object
(touch screen, radio and air conditioner control) correctly.
49.
“Can you look at the radio controls on the central console?”
Check if participant’s gaze vector intersects with the GREEN object
(touch screen, radio and air conditioner control) correctly.
50.
51.
“Can you look at the rear view mirror?”
Check if participant’s gaze vector intersects with the ORANGE object
(rear view mirror) correctly.
“Save” the modified world model.
© MIT AgeLab 2013
Page A173 of 293
“Thanks. I need a minute or two to finish setting-up the calibration. While I do that, I am
going to play an audio recording that will review the n-back number task that you learned in
the lab and take you through additional practice so that you are comfortable with this task.”
Ensure that configuration file name =
2-3 2012b_Intro.set
(YES / NO)
52.
Press Start test on the DAQ
53.
Ensure that we are recording all 90 channels.
54.
55.
Entering the participant number
2012b_xyz_Intro (where xyz is the participant
number)
© MIT AgeLab 2013
Page A174 of 293
Phase 2-5. In-Vehicle N-back Practice
Hit F1 to start the n-back task practice
56.
The following n-back training is all practice - you do not need to monitor performance.
(start recording n-back_instructions.wav)
Now we are going to practice the N-back number tasks again. The tasks are the 0-back, 1-back, and the 2-back.
We are going to practice them in the order of 0, 1, and 2 for now, but they may be presented in a different order
during the actual drive. Each trial will consist of a set of 10 single digit numbers.
The first version of the task is called the zero-back task. As I read each number, you are to repeat out loud the
last number that you’ve heard. For example, if I were to say the number 3, you would say 3; then if I said 2,
you would say 2; then if I said 6, you would say 6, and so on. Try to be as accurate as you can be. (Pause 2.25
sec)
Let’s practice:
3
2
(Pause 5 sec)
6
8
9
0
5
4
1
7
The next version of the task is called the one-back task, which simply means that as I read each list of ten
numbers, you are to repeat out loud the number before the last number that you heard. For example, if I were
to say 3, you would say nothing, then if I said 2, you would say 3, then if I said 6, you would say 2, and so on.
Try to be as accurate as you can be. (Pause 2.25 sec)
Let’s practice:
3
2
6
7
1
4
0
5
9
8
The final version of the task is called the two-back task, which simply means that as I read each list of ten
numbers you are to repeat out loud the number that was read two numbers ago. For example, if I were to say
the number 3, you would say nothing, then if I said the number 2, you would say nothing, then if I said 6, you
would say 3, if I said 7, you would say 2, and so on. Try to be as accurate as you can be. (Pause 2.25 sec)
Let’s practice:
3
2
6
7
1
4
0
5
9
8
(end recording n-back_instructions.wav)
© MIT AgeLab 2013
Page A175 of 293
Phase 2-6. Final Eye Tracking / Camera Configuration
“Great I am just about done configuring the eye tracking equipment. I need you help me with the
final steps…”
Note: this step is a verification
57.
“If you look straight ahead you’ll notice three round cameras.
Can you please look directly into the canter of the lens of the lower of
the two cameras on the right?“
58.
“And now into the canter of the lens on left camera?”
“Thanks. We’re all set.”
© MIT AgeLab 2013
Page A176 of 293
Phase 2-7. Training in MIT parking lot on first set of in-vehicle tasks
Hit F2 to start the voice introduction
59.
(start recording 2012b_voice_intro)
Hello and welcome to the MIT AgeLab’s study vehicle. We will use this recording to introduce you to
some of the user interface features of this vehicle including the voice command system. The voice
command interface you will be using is the standard system supplied with this vehicle; it is not a
research system. After this short introduction, the research associate will talk you through the details of
how to do each task we will ask you to do during the first half of the drive and will give you a chance to
practice with the controls and voice interface. During this practice period, please feel free to ask the
research associate any questions that you have. It is important to us that you have the opportunity to
become familiar with how to use the interface now, so that you can use it during the drive without
needing to ask questions at that time.
The research associate will now point out the location of the push-to-talk button.
(end recording 2012b_voice_intro)
60.
61.
Confirm participant understands where the
push-to-talk button is located; point out as
needed.
Hit F3 to start the next part of the voice
introduction
(start recording 2012b_voice2_intro)
Pressing down on the left side of the push-to-talk button side with the face image activates the voice
command interface. The system will then say ‘Please say a command’. After a brief pause, you will hear
a tone indicating the system is ready for you to speak a command.
A menu will also appear on the display screen that provides information on commands that you can
speak. Looking at this menu may be useful while you are learning to use the system while the car is
parked. Once you are actually driving, the intent of the system design is to only look at the display
when you need to, since one of the major purposes of a voice command interface is to allow you to
control various functions while keeping your eyes on the road.
The voice system is designed to support a logical sequence of commands to allow you to get to a specific
function. For purposes of this study, we will focus on short-cut approaches for requesting specific tasks
rather than training you on all the levels of commands that could be used.
(end recording 2012b_voice2_intro)
© MIT AgeLab 2013
Page A177 of 293
Phase 2-8. Voice System Calibration
62.
Introduce voice calibration using the text below or something
equivalent based on your experience.
“Voice recognition systems are getting better all the time but they are still not perfect, so, if the
system does not recognize what you said, it may sometimes say ‘Sorry, please say a
command’. One way to increase the likelihood that the system will recognize your voice is to
go through a calibration procedure that helps the system learn your speaking style.”
63.
Prompt participant through the Voice Calibration Procedure
summarized below
“We are going to go through the voice calibration procedure now. In a moment, I will ask you
to press the push-to-talk button. When the system says, “Please say a command.”, listen for
the tone that indicates the system is ready to listen and then say “USER PROFILE” from the
next menu say “CREATE USER PROFILE 1” (or 2). Then follow the prompts from the system.
Do you have any questions? (Answer as needed.) OK, please press the push-to-talk button
now and be ready to say “Create user profile 1”.
Follow voice prompts from system.
64.
Did the voice system appear able to work reasonably with the
participant’s voice?
(YES / NO)
If NO – record details below.
NOTES IF PARTICIPANT WITHDRAWS / IS WITHDRAWN AT THIS POINT:
Enter extensive notes below if participant is withdrawn / withdraws on own at this point. Things
such as the participant being overly frustrated, system incapable of recognizing voice (i.e. heavy
accent, voice too soft, pauses too long by participant in multi-utterance responses, etc.), comments
on the participants tone of voice, cultural background, the participant withdrawing vs. RA
withdrawing them are all critical. Transfer later to participant log.
_____________________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
© MIT AgeLab 2013
Page A178 of 293
2012b_HMI_OnRoad - Voice Based In-Vehicle Systems Study
Experimental Checklist
Section 3 Leaving the Parking Lot
Participant Code
____________________ Protocol Configuration __________________
Date
________________________________________________________________
RA (s)
________________________________________________________________
Notes for RA during the drive:

Ctrl+F9 will allow you to manually set the trial time

F23 will pause experiment

F24 will silence any audio

Continue to check light levels on DAQ cameras, adjust shutter speed as needed when car
changes directions (e.g. 93 north to 93 south)
If participant cannot complete evaluations before reaching Exit 19 on 495 South

Exit 18: Coming off exit, take a left at the light. At the next light take left into parking lot of
Home Depot/Uno’s. Turn around

Exit 17; 140 Bellingham: Coming off exit, take a left at the light and go over the bridge. At the
next light, take a right into the mall. Turn around

Exit 16; King St./Franklin/Woonsocket, RI: Coming off exit, take left off exit. At next light, take
a right and pull into the lot behind Joe’s American to turn around
If participant cannot complete evaluations before reaching I93 when returning on 495

Exit 41a, take right off exit, turn around in Dunkin Donuts on left, Not Ideal

Exit 42a, take a right off the exit, drive for a ¼ mile, take a left at light for Andover Mall, this is
the preferable turnaround point.

42b, take right off exit, take next right into Stadium Plaza, Not Ideal

43, take left at stop sign coming off exit, cross under bridge and get back on 495 South, Not
Ideal.
© MIT AgeLab 2013
Page A179 of 293
Phase 3-1. Introduction to Driving
Task
65.
Save the FaceLab Face Model (use participant
id as file name)
66.
Verify FaceLab is all set for data collection
67.
START LOGGING IN FACELAB
68.
70.
Select the configuration file to:
3-1 2012b_Start_Drive.set
3-1 2012b_Start_Drive.set
71.
72.
Entering the participant number:
2012b_xyz_sDrive (where xyz is the participant
number)
73.
Press F1 to start the general driving
instructions
69.
Notes / Checked
(YES / NO)
(start recording 2012b_drive_instructions.wav)
In a few moments, the research associate will begin giving you step-by-step instructions on how to drive
from here to I93 north. Once on I93, you will continue on to the I495 interchange. The drive north will give
you plenty of time to become familiar with driving this car before you are asked to do any of the tasks you
have practiced. When you reach I495 and have had a few minutes of driving on that highway, short
recorded prompts will tell you what task we would like you to do. When you hear these prompts, please do
not start a task until you hear the word ‘begin’.
You will have the opportunity to earn a small monetary bonus by engaging in each of the tasks. However,
you will also be responsible for paying for any citations that you might be issued for traffic regulation
violations. While we want you to do your best to complete each task to the best of your ability, you should
always give priority to safe driving. It is important for you to remember that some of these tasks are
designed to be difficult for everyone. That is to say, everyone will not be able to complete every task
correctly. If you feel for any reason that a task will interfere with your ability to drive safely, delay starting
© MIT AgeLab 2013
Page A180 of 293
the task until you feel it is safe to do so or skip the task entirely if you feel that is the best thing to do. Your
safety, and the safety of other people around you, is the highest priority.
If at any time you feel uncomfortable driving the vehicle or in your ability to drive safely, please let the
research associate know how you are feeling and they will confer with you about pulling off the roadway at
the nearest safe location.
During the course of the study, you may wish to talk with the research assistant. They will be happy to
answer any questions about directions, operation of the vehicle, or other aspects about the study. However,
they are unable to participate in casual conversation. Remember at any point you wish to engage the
research associate please do so. They will respond if it is appropriate.”
Do you have any questions?
(end recording 2012b_drive_instructions.wav)
74.
Ask if they have any questions and answer as appropriate. Make sure
to make the following statement to reinforce the recorded
instructions:
“I would just like to emphasise again that if you feel that any of the tasks affect your ability to
drive safely, please delay or skip the task completely and focus on your driving. If you should
become overly tired, sleepy, or feel that it is no longer safe for you to drive for any reason, just
let me know and we can stop the study and I can drive the vehicle back.”
© MIT AgeLab 2013
Page A181 of 293
Phase 3-2. Start Driving
75.
The RA needs to be particularly attentive at this time, i.e. looking
themselves to be sure that no one is exiting the building adjacent to
the parking space and in danger of walking behind the car as the
participant is backing-up.
“I would just like to point out again that in addition to using the car mirrors when backing up
the vehicle, that there is a back-up camera display available and that the car has a proximity
sensor that will beep if an object is detected right behind the car.”
“Ok. It’s time to head towards the highway, please check behind the car to see if it is safe to
back-up, and, when it is, back-out of the parking space and head towards the parting lot exit.”
“When you approach the exit, the barrier bar will raise automatically.”
Before you hit parking lot gate, ask them to stop and check their mirrors.
76.
Press F2 as you pull out of parking lot and turn left
77.
Record time
___________
78.
Take a right onto Amherst Street.
79.
Take a left onto Ames Street
80.
Cross Memorial Drive and take a left on Memorial Drive
81.
82.
When Memorial Drive splits up take the right lane onto Edwin H Land
Blvd
Take a right onto Monsignor Obrien Hwy/RT-28S – Sign to Boston
83.
84.
Get into left lane and bear left after intersection to get onto I-93 North
ramp
Take right lane to I-93 North in the direction of Concord NH
85.
Once participant merges onto I-93 North, press F3
86.
© MIT AgeLab 2013
Page A182 of 293
87.
ADJUST ALL VIDEO SHUTTER SPEEDS IF NEEDED TO IMPROVE
IMAGE BRIGHTNESS
(start
recording
2012b_enter_highway.wav)
We are going to be driving north on 93 for approximately 20 minutes and then exit onto 495 South. You
can continue driving in this lane or move into another lane so that you are comfortable with the traffic flow.
(end recording 2012b_enter_highway.wav)
88.
After 2012b_enter_highway plays (about a minute of comfortable driving on 93) Stop the
Daq and switch to first configuration file 2012b _RadioM_Nav_x, or 2012b
_RadioV_Song_x (where x is n-back config)
Stop and Start facelab to break files
89.
90.
2012b _RadioM_Nav_x, or 2012b _RadioV_Song_x, (double check on protocol)
91.
92.
93.
Entering the participant number: 2012b_xyz_RadioM_Nav_x, or
2012b_xyz_RadioV_Song_x (where xyz is the participant number)
(YES / NO)
Prompt to turn on Radio & Adjust Volume.
94.
At Exit 42, press F11
(start recording 2012b_habituation.wav)
In a few minutes we are going to exit and turn onto I495 south. Once we have driven for several minutes on
495, recorded instructions like this one will ask you to do a number of tasks that you have already practiced. As
a final practice task, when you hear the word ‘begin’, please turn on the radio by pressing the volume button,
listen for a moment and then adjust the volume if needed so that it is at a level where you can hear but not so
loud as would make it hard to hear recorded instructions like this one. Once you are comfortable with the
volume, please turn the radio off by pressing the volume button again. (pause) Begin.
(end recording 2012b_habituation.wav)
Just after Exit 43 give the driver the following instructions
95.
“You are approaching Exit 44B for route 495 south. Please prepare to take this exit”
If the driver does not move into the right lane prompt again.
© MIT AgeLab 2013
Page A183 of 293
As you approach Exit 44 give the driver the following instructions
96.
“Remember, please take Exit 44B for route 495 south.”
If the driver does not appear to be taking the exit prompt again.
Once participant merges onto I-495 South, press F1
97.
2012b_RadioM_Nav_x
or
2012b_RadioV_Song_x
(start recording 2012b_enter_495.wav)
We are going to be driving on 495 for approximately 30 minutes. You can continue driving in this lane or
move into another lane so that you are comfortable with the traffic flow. During portions of the drive you
will be asked to perform some additional tasks while you are driving.
(end recording 2012b_enter_495.wav)
98.
IMAGE BRIGHTNESS
99.
2012b_RadioM_Nav_x: move to next protocol (3 minute habituation to
495 South)
2012b_RadioV_Song_x: move to next protocol (3 minute habituation
to 495 South)
© MIT AgeLab 2013
Page A184 of 293
Section 4 Approaching Rest Stop
Participant Code
100.
Just after (Exit 20-Milford, Hopkinton, Uxbridge) give the driver the
following instructions
“You are approaching Exit 19 for route 109. Please prepare to take this exit”
*Refer to Section 2 of protocol if driver has not yet completed all evaluation sections prior to
reaching exit 19. Exits 18-16 with description provide optimal turn around points. See also if
driver does not complete all post rest stop evaluations before returning to I93.
As you approach Exit 19 give the driver the following instructions
101.
“Remember, please take Exit 19 for route 109.”
102.
As you enter the exit ramp Press STOP
FaceLab logging
103.
If recording press STOP on the DAQ
104.
Take the exit ramp to the right, and at the
light take a right. Be in the right lane and at
the next set of lights take a right into the
driveway. Pull into the lot behind the Burger
King.
105.
Once car is in park - tell participant that there
is a short rating scale and questionnaire that
you need to have them fill-out and then they
© MIT AgeLab 2013
Page A185 of 293
can take a break.
106.
Workload Rating Sheet - Hand the
participant the clipboard with the workload
rating sheet and ask them to read the
instructions before starting and then ask if
they have any questions.
Specifically point out which tasks they
should be rating at this time.
107.
Workload Rating
108.
Detailed Task Rating questionnaire – hand
to participant. Point out to them which of the
sections they should be filling-out.
Detailed Task Rating
109.
110.
Ask them to take a minute to stretch. If the
participant asks to use the bathroom please
make a note of it.
111.
Did the participant get out of the car?
112.
Set up radio
Turn Radio on; set on AM preset-4. (AM 1030)
Change to FM2, set on FM2 preset-4.(FM 92.9)
Turn radio off.
113.
Has the radio been properly set-up?
© MIT AgeLab 2013
___ Workload Rating Completed
___ Detailed Rating Completed
(YES / NO)
(YES / NO)
Page A186 of 293
Section 5 - Leaving Rest Stop
Participant Code
98.
Five
cams
99.
100.
2012b _RadioM_Nav_x, or 2012b
_RadioV_Song_x, (double check on protocol)
101.
102.
Ensure that we are recording all xyz
channels.
103.
Pull out of the rear exit of the Burger King lot,
after confirming that the left turn is protected
by the light. Pull into the left lane and take a
left at the light. Go under the bridge and at
the next set of lights, take a left to get onto 495
North
Once participant merges onto I-495 North, press F1
104.
Start logging in facelab
(YES / NO)
2012b _RadioM_Nav_x
Or
2012b _RadioV_Song_x
(start recording 2012b_enter_495.wav)
We are going to be driving on 495 for approximately 30 minutes. You can continue driving in this lane or move
into another lane so that you are comfortable with the traffic flow. During portions of the drive you will be
asked to perform some additional tasks while you are driving.
(end recording 2012b_enter_495.wav).
© MIT AgeLab 2013
Page A187 of 293
IMAGE BRIGHTNESS
105.
Move to next protocol (3 minute habituation to 495 North)
106.
NOTES:
If participant cannot complete evaluations before reaching I93 when returning on 495

Exit 41a, take right off exit, turn around in Dunkin Donuts on left, Not Ideal

Exit 42a, take a right off the exit, drive for a ¼ mile, take a left at light for Andover Mall, this is
the preferable turnaround point.

42b, take right off exit, take next right into Stadium Plaza, Not Ideal

43, take left at stop sign coming off exit, cross under bridge and get back on 495 South, Not
Ideal.
© MIT AgeLab 2013
Page A188 of 293
Section 6 Return to MIT
Participant Code
____________________ Protocol Configuration _________________
Phase 6-1. Return Prep
111.
Just before exit 28
Press STOP on the DAQ
Select the configuration file to
6-1 2012b_End_Drive.set
6-1 2012b_End_Drive.set
112.
113.
Entering the participant number
2012b_xyz_end (where xyz is the participant
number)
114.
Press STOP and START FaceLab logging to
split files
115.
Tell participant to take Exit 26, which will
lead to Storrow Drive exit on the left when
participant pass onto the on-ramp or begins to
wait in traffic press F1
116.
Take the RT-3 N ramp on the left to Kendall
Sq. / Gov’t Center
117.
Keep right at the fork, follow signs for
Cambridge / Memorial Drive
118.
Turn right at Cambridge St / RT-3 N
(Longfellow Bridge)
119.
Turn right onto the RT-3 N ramp at end of
107.
108.
109.
110.
© MIT AgeLab 2013
(YES / NO)
Page A189 of 293
bridge
120.
Turn right onto Memorial Drive
121.
Turn right onto Wadsworth Street
122.
Turn left onto Amherst Street
123.
Turn right onto Hayward Street
124.
Record time.
Time ________
125.
Turn right into parking lot and park in
AgeLab parking spot
126.
Stop logging Face lab and DAQ
© MIT AgeLab 2013
Page A190 of 293
Phase 6-2. Post drive
You can turn the car off. I have a short questionnaire for you to complete before we are finished.
NOTE TO RA: WHEN PRESENTING PARTICIPANT WITH WORKLOAD RATING
QUESTIONNAIRE ASK THEM TO REREAD THE INSTRUCTIONS, wait until they have read
them, and then ask them if they have any questions.
Specifically point out which sections they should be rating at this time.
127.
Once the participant has the car is park, tell
them that there is a short rating scale that you
need to have them fill-out and then they can
take a break. Read them the workload
instructions above and then hand them the
clipboard with the workload rating sheet.
128.
Workload Rating
129.
Check and note the mic volume. It needs to be
set to 95%
___ Workload Rating Completed
Mic _______ %
“I am going to help you get out of the car and we are going back inside for a few minutes so we
can take the sensors off, complete a final set of questionnaires, and pay you the compensation for
your assistance today.”
Disconnect physiology sensors from car
130.
131.
Make sure participant takes all belongings
from the car
(Note: car can remain un plugged for 20 – 30 minutes between participants. Please remember to
plug it in at some point)
© MIT AgeLab 2013
Page A191 of 293
Phase 6-3. In 290 or Other
NOTE: Participant can be passed off to another staff member as needed at this point to moveon to next participant or to close down vehicle for the day.
132.
133.
Remove sensors from participant. Coil neatly
and secure with Velcro ties. Return sensors to
appropriate box.
Hand participant the Post-Experimental
Questionnaire and Supplemental Health
Questionnaire.
Post-Experimental Questionnaire was
completed by participant.
(YES / NO)
134.
Supplemental Health Questionnaire was
completed by participant.
(YES / NO)
135.
Here is the $90 that you earned for coming in today. This includes the $80 base compensation as
well as the $10 incentive.
If you know anyone who may be interested in participating in one of our experiments please
encourage them to register as a volunteer. I have cards with all of the information. Would you
like a few?
(Hand participant as many cards as they want)
Thank you for coming in.
(Participant leaves.)
© MIT AgeLab 2013
Page A192 of 293
Phase 6-4. Data Clean-Up to be Completed after Participant Leaves
Get the rest of the participant’s documents from the bin in 288.
136.
137.
Make sure you have all documents listed above and that they have
participant ID on each of them.
Clean up car and prepare for next participant
138.
139.
Eye Data: Convert files into text by going to the Logging tab and selecting
convert to text. There should be 4 folders, open each one and select all the
files and press open. When you are done converting all five folders. Exit
out of Facelab. (note this may be batched at the end of the day)
In each experimental segment verify that the audio and video data were collected
140.
Go to the shortcut on the desktop labelled 'Shortcut to Trip Reports'
Open all data files for 2012b_xyz….. where xyz is the participant number
Audio: Play the last audio file in each data folder and click through it until you hear the
participant or computer speaking. If you don’t hear anything, turn up the volume on the
receiver. The sound should be coming through the car speakers.
Video: To check the video open the 'Video' folder and click on the 'view' icon (white box
with blue) and click on thumbnails. Verify that there are the same number of files for
both cameras. As long as pictures appear in the thumbnails its fine.
Error logs: Make sure there are no error files in directory
FileShuffler Log: Record the file shuffle log onto the network
(if you have any files that do not appear correct tell Alea)
141.
RA 2, fills out two Data Set Summary Sheet for the driving portion of the
experiment.
© MIT AgeLab 2013
Page A193 of 293
Data good for Segment A (Radio Manual)?
(YES / NO)
Data good for Segment B (Nav)?
(YES / NO)
Data good for Segment C (Radio Voice)?
(YES / NO)
Data good for Segment D (Song)?
(YES / NO)
Data good for Segment E (Phone)?
(YES / NO)
Data good for Segment X1 (N-back set 1)?
(YES / NO)
Data good for Segment X2 (N-back set 1)?
(YES / NO)
142.
143.
144.
145.
146.
147.
148.
Copy data to Studies Folder
149.
To access the studies folder type win\username.
Transferring Data: Copy data to agelabfs\Studies\2012b_HMI_OnRoad(Voice)\Participant_Data\2012b_xyz where xyz stands for the participant
number. Within 2012b_xyz create folder “training” for all training and non-driving assessment data. All
experimental driving data should remain in the main participant folder.
DAQ data is located on the F drive and eye data at D:\faceLAB Log Files
All data copied to the studies folder:
(YES / NO)
150.
Get the rest of the participant’s documents
151.
from the bin in 288.
Make sure you have all documents listed
152.
below and that they have participant ID on
each of them.
Enter all information into log (\\agelab153.
nas\studies\2012b_HMI_OnRoad_(Voice)
Review documents and order according to the following chart:
Consent Form
Payment Form
Driver’s License
Emergency Contact Form
Workload Rating Sheet
Pre-Experimental Questionnaire
Supplemental Health
Data Set Summary 1 (Intake)
Data Set Summary 2 (OnRoad)
© MIT AgeLab 2013
Page A194 of 293
Protocol sections 1, 2, 3, 4, 5, 6 A-E, X1, X2
154.
155.
RA1 gather all documentation scan and
upload. Shred Emergency contact.
Hand xxxx forms Consent, Payment, and
Copy of Driver’s License should be on top.
© MIT AgeLab 2013
Page A195 of 293
Phase 6-5. Notes on Training and Evaluation Issues
Use the space below for notes taken during the drive concerning any issues a participant had
learning a task or attempting to do a task during the evaluation period. These notes are intended
to support development of insight concerning any issues people have in learning the technology
or attempting to operate under actual driving conditions.
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
© MIT AgeLab 2013
Page A196 of 293
Section A_t1 Radio Manual Training - MIT parking lot
Participant Code ______________________ Protocol Configuration __________________
Phase A_t1-1. Setup
159.
Select the configuration file:
Train1_M_Nav_First
Train1_M_Nav_First
160.
161.
Entering the participant number:
2012b_xyz_Train1_M_Nav_First (where xyz
is the participant number)
162.
Press F1 to begin
156.
157.
158.
© MIT AgeLab 2013
(YES / NO)
Page A197 of 293
Section A_t2 Radio Manual Training -495 Training
Phase A_t2-1. Setup
4.
Train2_V_Song_First.set
Train2_V_Song_First.set
5.
6.
Entering the participant number: 2012b_xyz_
Train2_V_Song_First2(where xyz is the
participant number)
7.
Press F10 to begin
1.
2.
3.
© MIT AgeLab 2013
(YES / NO)
Page A198 of 293
Section A_e Radio Manual Evaluation
Note: all five cameras are used
9.
If this is the first or first evaluation or first
evaluation after the Rest, verify that the
selected configuration file is:
2012b_RadioM_Nav_x.set (where x is the nback config)
Press F2 to begin
10.
Ensure facelab is logging
8.
(YES / NO)
(wait 2 minutes 30 seconds)
© MIT AgeLab 2013
Page A199 of 293
RadioM0
(start recording 2012b_rtask_0) – 44 seconds
A task period is about to start. The tasks will involve operating the radio using the manual controls. You
will be asked to do a number of tasks that you have already practiced. These include turning the radio on
and off, changing stations using the preset buttons, switching between the AM and FM frequency bands,
and manually locating a station using the tuning knob. There will be 10 to 60 second pauses between tasks.
Do not begin a task until you hear the word ‘begin’. When you have successfully completed a task, please
immediately say the word ‘done’. (Pause 3 seconds.) Your first task is to turn the radio on by pressing
the volume knob. (Pause 2 seconds) Begin.
(end recording 2012b_rtask_0)
Radio On
11.
-
[Vol] (FM 104.1)
When participant has says DONE Press F3.
(wait 1 minute)
RadioM1
(Start recording 2012b_rtask_1) – 5 seconds
Your task is to change the radio to preset 1. (Pause 2 seconds.) BEGIN
[Preset-1]
(go to 92.9)
12.
13.
Did you need to prompt the participant?
(YES / NO)
14.
Did they select the correct preset?
(YES / NO / don’t know)
(wait 1 minute)
RadioM2
[Preset-5]
(go to 106.7)
15.
16.
(YES / NO)
17.
(wait 10 seconds)
© MIT AgeLab 2013
Page A200 of 293
RadioM3
Please press the Radio button, change to AM and then turn the radio off. (Pause 2 seconds.) BEGIN
[Radio] → [AM] → [vol] (radio off)
18.
19.
If radio not set on AM before being turned off
– prompt to turn radio back on, select AM and
turn off.
Press F6.
____ Radio set in AM mode and off
(wait 1 minute)
on-FM2-100.7
RadioM4
Your task is to turn on the radio, switch to FM2, and tune to 100.7. (Pause 2 seconds.) BEGIN
[Vol] → [RADIO] → [FM2] → (92.9 – 100.7)
20.
21.
(YES / NO)
22.
Did they tune to the correct station?
(wait 10 seconds)
RadioM5
Your task is to turn off the radio.(Pause 2 seconds.) BEGIN
[Vol] radio off
23.
(wait 1 minute)
© MIT AgeLab 2013
Page A201 of 293
RadioM6
(Start recording 2012b_rtask_6) – 9 Seconds
Your task is to turn on the radio, switch to FM1, and tune to 95.3. (Pause 2 seconds.) BEGIN
[Vol] → [RADIO] → [FM1] → (100.7 – 95.3)
24.
25.
(YES / NO)
26.
(wait 10 seconds)
RadioM7
Your task is to turn off the radio.(Pause 2 seconds.)BEGIN.
[Vol] (radio off)
27.
(Start recording 2012b_end_task)
The task is complete. Please continue driving.
(end recording 2012b_end_task)
28.
Move to next protocol (30 sec separation).
© MIT AgeLab 2013
Page A202 of 293
Section B_t Navigation Training
Phase B_t-1. Setup
Note: interior cameras and one exterior at 15fps
29.
Continued from Train1_M_Nav_First.set or
Train2_V_Song First.set
30.
Confirm that the file card with practice
addresses is in the card holder:
223 Broadway, Cambridge, Massachusetts
155 Charles Street, Boston, Massachusetts
31.
Press F6 or F15to begin
© MIT AgeLab 2013
Page A203 of 293
Section B_e - Navigation Evaluation
33.
2012b_RadioM_Nav_x.set (where x is the nback config)
Press F13 to begin
34.
32.
(YES / NO)
(start recording 2012b_navtask_0) (28 sec + 2 for noise in media player = 30 sec)
A task period is about to start. The tasks will involve operating the navigation system using the voice
interface. You will be asked to do two kinds of tasks that you have already practiced. These consist of using
the command, ‘destination street address’ to begin an address entry and then later cancelling the route.
There will be pauses of approximately one minute between tasks. Do not begin a task until you hear the
word ‘begin’. There will be a one minute pause before the first task is presented.”
(end recording 2012b_navtask_0)
(wait 1 minute from end of briefing songtask_0)
© MIT AgeLab 2013
Page A204 of 293
Nav Entry1
(start recording 2012b_navtask_1) – 10 seconds
Your task is to enter the destination address: 177 Massachusetts Avenue, Cambridge, Massachusetts.
(Pause 2 seconds)
Begin.
[voice] → “Destination Street Address” →”Cambridge” →. “yes” → “Massachusetts Avenue”
→ “One Seven Seven” → ”yes” → “__” → “yes”
[voice] → “Set as Destination” → “yes”
If the participant is unsure how to proceed, you can, if necessary, prompt them to press the pushto-talk button and to say ‘destination street address’. For any other help, just say ‘you can look at
the screen for other prompts’ and they should be left on their own to try to complete the tasks.
If during tasks participant:
(a) verbally indicates that have stopped (wait 20 sec),
(b) appears to have given up but not verbally acknowledged (wait 1min),
(c) appears overly frustrated, having difficulty but not making progress etc. (allow 1 min
of difficulty),
(d) is making continual mistakes that imped completion (allow maximum of 2.5 min)
(wait 1 minute)
35.
When participant has completed the task or
given up Press F14.
36.
(YES / NO)
37.
Did they complete the address entry
correctly?
(YES / NO)
© MIT AgeLab 2013
Page A205 of 293
Nav Cancel 1
(Start recording 2012b_navtask_2) – 7 Seconds
Your task is to cancel the route using the command ‘Navigation Cancel Route’.
(Pause 2 seconds.)
BEGIN
[voice] → “Navigation Cancel Route” →”yes”
38.
given up Press F15.
39.
(YES / NO)
40.
Were they able to cancel the route without
significant intervention?
(YES / NO)
(wait 1 minute)
Nav Entry2
Your task is to enter the destination address: 293 Beacon Street, Boston, Massachusetts.
(Pause 2 seconds.)
BEGIN
[voice] → “Destination Street Address” →”Boston” →. “yes” → “Beacon Street” → “Two Nine
Three” → ”yes” → “__” → “yes”
[voice] → “Set as Destination” → “yes”
41.
given up Press F16.
42.
(YES / NO)
43.
Did they complete the address entry
correctly?
(YES / NO)
(wait 1 minute)
© MIT AgeLab 2013
Page A206 of 293
Nav Cancel 2
Your task is to cancel the route using the command ‘Navigation Cancel Route’.
(Pause 2 seconds.)
BEGIN
[voice] → “Navigation Cancel Route” →”yes”
44.
given up Press F17
45.
(YES / NO)
46.
Were they able to cancel the route without
significant intervention?
(YES / NO)
47.
© MIT AgeLab 2013
Page A207 of 293
Section C_t1 Radio Voice Training - MIT parking lot
48.
49.
51.
Train2_V_Song_first.set
Train2_V_Song_first.set
52.
53.
Entering the participant number: 2012b_xyz
Train2_V_Song_first 1(where xyz is the
participant number)
54.
Press F1 to begin
50.
© MIT AgeLab 2013
(YES / NO)
Page A208 of 293
Introduction & Setting Volume
(start recording 2012b_C_t1_intro1)
We will ask you to do a number of tasks today using the radio. The radio in this vehicle can be controlled
both by traditional push button and knob style controls and through a voice command system. During the
first part of the drive, we will ask you to work with the voice interface.
The radio stations that we will use today were selected for their spacing on the radio dial and not for specific
content. If some of the music that happens to play is not particularly to your musical preference, please just
to along with it for the relatively short duration that each station is played.
As a general rule, please do not start a task until you hear the word, ‘begin’. Waiting for ‘begin’ lets you
know that you been have given all of the instructions for the task.
Before going into the details of the voice control option, we will use the manual interface to set a comfortable
volume. The radio can be turned on and off by pressing on the large knob on the left side of the radio labelled
“V-O-L” for volume. When I say ‘begin’, please turn the radio on by pressing the volume control, adjust the
volume to a comfortable level, and then turn the radio off by again pressing the volume knob. Begin.
(end recording)
[In addition to setting an initial volume level, part of the goal here is to make sure the participant
understands how to manually turn the radio on and off as this will be requested at various points
in the protocol.]
Let them adjust the volume to a level so that it is loud enough to hear so that they (and
you) know that the radio station is on, but lower than one would typically use to actually
listen to the radio so that it will not mask recorded commands. If the participant has the
radio too loud it will also be difficult to hear the audio later when scoring the task. Have
the participant adjust the volume down as need to achieve these goals.
(VOL)
(Radio should come-up on FM 100.7.)
56.
Prompt participant as needed to use VOL to
turn-on radio, adjust volume to an
appropriate level, and turn off by pressing
VOL.
Remind about waiting for ‘begin’ if needed.
57.
Press F2
55.
(Radio OFF)
© MIT AgeLab 2013
Page A209 of 293
Introducing Voice Activation of Radio
Let’s move to using the voice interface to turn the radio on. This is done by pressing the push-to-talk button
on the steering wheel that has the image of a person speaking. The system will then ask you to say a
command.
Let’s practice by pressing the push-to-talk button and then saying ‘radio on’ when the system asks for a
command. Begin.
(end recording)
[voice] → “Radio On”
58.
59.
60.
61.
62.
Prompt and assist participant as needed until
they can turn on radio using voice commands.
If participant needs more practice, have them
manually turn the radio off by pressing the
VOL button and then prompt them to turn the
radio on using the ‘Radio On’ command.
Prompt participant to adjust volume if it
appears to be too high or low for training
purposes.
Participant is able to turn radio on using voice
commands
Press F3
(YES / NO)
(Radio ON)
(The radio is being left ON since it will be playing when they are asked to select a preset during the
evaluation. We need to evaluate if trying to listen to the following instructions during training while
the radio is playing is a problem with older participants or others who may have hearing issues is a
problem.)
© MIT AgeLab 2013
Page A210 of 293
Introduction to Presets
(Radio is ON)
(Audio same as t2 for rest.)
(start recording 2012b_C_t2_2)
One of the tasks you will be asked to do is to change radio stations by requesting a preset station.
If the radio is already on, as it is now, to request preset-2, you would press the push-to-talk button,
wait for the command prompt, and then say ‘preset-2’.
Please try this by pressing the push-to-talk button, wait for the system to respond, and then say
‘preset-2’. Begin.
(end recording)
[voice] → “Preset-2” → “yes”
[voice] → “Radio” → “Preset-2” → “yes”
otherwise)
(Radio command needed if radio off; optional
63.
Provide guidance as needed.
64.
65.
If the participant appears to totally
understand the task, offer them the option to
skip the next practice..
Press F4 to advance to next practice.
66.
Skipped next practice: yes / no
(Radio is playing as it will be during evaluation period.)
© MIT AgeLab 2013
Page A211 of 293
Presets prompted Practice
Let’s practice with the way you will be given instructions during the drive.
Your task is to change the radio to preset-5.(Pause 2 seconds.) BEGIN
(end recording)
[voice] → “Preset-5”
otherwise)
67.
68.
69.
70.
71.
Provide any additional guidance needed until
they demonstrate that they understand how
to use the presets. You can ask them to try a
different pre-set for addition practice if this
is indicated.
Participant demonstrated the ability to
correctly select a preset.
Have participant turn off radio by pressing
the VOL button.
Press F5
(YES / NO)
___ Radio off
(Radio OFF)
© MIT AgeLab 2013
Page A212 of 293
Voice Tuning
(Radio is OFF)
The next task is to tune the radio to a particular station. For example, let’s say you want to tune to
FM station 92.9. If the radio is off, as it is now, you would press the push-to-talk button, wait for
the command prompt, say ‘radio’, wait for the prompt, and then say ’92.9’. You do not have to say
AM or FM when requesting a radio station.
Let’s practice. Your task is to turn the radio on using the push-to-talk button and request FM
92.9. Begin.
(end recording)
[voice] → “Radio” → “92.9” → “yes”
[voice] → “Radio” → “FM 92.9” → “yes”
72.
Prompt as needed to get them to 92.9.
73.
If the system does not understand ‘ninety
two’, suggest that they try ‘nine two point
nine’. It is ok if they choose to say AM and FM
but you can also point out that it is not needed
if they are confused.
Make sure that they wait for ‘begin’.
74.
75.
the VOL button.
Press F6 to begin
___ Radio off
(Radio OFF)
© MIT AgeLab 2013
Page A213 of 293
Second Practice
Your task is to turn the radio on using the push-to-talk button and request FM 104.1 Begin.
(end recording)
[voice] → “Radio” → “104.1” → “yes”
76.
77.
78.
79.
Repeat as needed by having the participant
practice again going to the same station, FM
104.1
correctly use voice commands to tune radio to
a specific station.
If no, please make brief notes on the training
notes sheet to give a best estimate of the
nature of their difficulty
(YES / NO)
Confirm that the radio is left in the correct
settings by verbally instructing the participant
to select in order the following bands and
then pressing preset-4:
FM1 > pre-set-4 (104.1)
FM2 > pre-set-4 (92.9)
AM > pre-set-4 (1030)
Steps above completed.
80.
Instruct participant to turn off the radio by
pressing the VOL button.
© MIT AgeLab 2013
___ Radio off
Page A214 of 293
Section C_t2 Radio Voice Training – 495 Stop
84.
Train1_M_Nav_First.set
Train1_M_Nav_First.set
85.
Ensure that we are recording all xyz channels.
86.
Entering the participant number:2012b_xyz
_M_Nav_First 2(where xyz is the participant
number)
87.
Press F10 to begin
81.
82.
83.
© MIT AgeLab 2013
(YES / NO)
Page A215 of 293
Introducing Voice Activation of Radio
During the second part of the drive, instead of using the manual radio controls, we will ask you to use the
voice command system to control the radio. Let’s start by using the voice interface to turn the radio on. This
is done by pressing the push-to-talk button on the steering wheel. The system will then ask you to say a
command. Let’s practice by pressing the push-to-talk button and then saying ‘radio on’. Begin.
(end recording)
88.
89.
Prompt and assist participant as needed until
they can turn on radio using voice commands.
If participant needs more practice, have them
manually turn the radio off by pressing the
VOL button and then prompt them to turn the
radio on using the ‘Radio On’ command.
90.
Note – advise participant not to say ‘done’ if
needed.
91.
Prompt participant to adjust volume if it
appears to be too high or low for training
purposes.
Participant is able to turn radio on using voice
commands
Press F11
92.
93.
(YES / NO)
(Radio ON)
(The radio is being left ON since it will be playing when they are asked to select a preset during the
evaluation. We need to evaluate if trying to listen to the following instructions during training while
the radio is playing is a problem with older participants or others who may have hearing issues is a
problem.)
© MIT AgeLab 2013
Page A216 of 293
Introduction to Presets
(Radio is ON)
One of the tasks you will be asked to do is to change radio stations by requesting a preset station. If the
radio is already on, as it is now, to request preset-2, you would press the push-to-talk button, wait for the
command prompt, and then say ‘preset-2’.
Please try this by pressing the push-to-talk button, wait for the system to respond, and then say ‘preset-2’.
Begin.
(end recording)
otherwise)
94.
Provide guidance as needed.
95.
If error occurs because participant pauses too
long between “preset” and “2”, explain about
saying as if one word.
96.
97.
98.
skip the next practice.
(Radio is playing as it will be during evaluation period.)
© MIT AgeLab 2013
Page A217 of 293
Presets prompted Practice
Let’s practice with the way you will be given instructions during the drive.
Your task is to change the radio to preset-5.(Pause 2 seconds.) BEGIN
(end recording)
[voice] → “Preset-5”
otherwise)
99.
100.
101.
102.
103.
Provide any additional guidance needed until
they demonstrate that they understand how
to use the presets. You can ask them to try a
different pre-set for addition practice if this
is indicated.
correctly select a preset.
the VOL button.
Press F13
(YES / NO)
___ Radio off
(Radio OFF)
© MIT AgeLab 2013
Page A218 of 293
Voice Tuning
(Radio is OFF)
The next task is to tune the radio to a particular station. For example, let’s say you want to tune to FM
station 92.9. If the radio is off, as it is now, you would press the push-to-talk button, wait for the command
prompt, say ‘radio’, wait for the prompt, and then say ’92.9’. You do not have to say AM or FM when
requesting a radio station.
Let’s practice. Your task is to turn the radio on using the push-to-talk button and request FM 92.9. Begin.
(end recording)
[voice] → “Radio” → “92.9”→ “yes”
[voice] → “Radio” → “FM 92.9”→ “yes”
104.
Prompt as needed to get them to 92.9.
105.
If the system does not understand ‘ninety
two’, suggest that they try ‘nine two point
nine’. It is ok if they choose to say AM and FM
but you can also point out that it is not needed
if they are confused.
106.
107.
the VOL button.
___ Radio off
(Radio OFF)
© MIT AgeLab 2013
Page A219 of 293
Second Practice
Your task is to turn the radio on using the push-to-talk button and request FM 104.1 Begin.
(end recording)
[voice] → “Radio” → “104.1”→ “yes”
108.
109.
110.
111.
Repeat as needed by having the participant
practice again going to the same station, FM
104.1.
correctly use voice commands to tune radio to
a specific station.
nature of their difficulty.
Confirm that the radio is left in the correct
settings by verbally instructing the participant
to select in order the following bands and
then pressing preset-4:
(YES / NO)
FM1 > pre-set-4 (104.1)
FM2 > pre-set-4 (92.9)
AM > pre-set-4 (1030)
Steps above completed.
112.
Instruct participant to turn off the radio by
pressing the VOL button.
© MIT AgeLab 2013
___ Radio off
Page A220 of 293
Section C_e Radio Voice Evaluation
114.
2012b_RadioV_Song_x.set (where x is the nback config)
Press F2 to begin
115.
113.
(YES / NO)
© MIT AgeLab 2013
Page A221 of 293
RadioV0
(start recording 2012b_rtask_10) – 45seconds
A task period is about to start. The tasks will involve operating the radio using the voice interface. You will
be asked to do a number of tasks that you have already practiced. These include turning the radio on,
changing stations using the preset commands, switching between the AM and FM frequency bands, and
requesting specific stations by frequency number. To turn the radio off, press the volume knob; for all other
interactions, use the voice interface. There will be 10 to 60 second pauses between tasks. Do not begin a task
until you hear the word ‘begin’.”(Pause 3 seconds.)Your first task is to turn the radio on by pressing the
push-to-talk button on the steering wheel and saying “radio-on”. (Pause 2 seconds) Begin.
116.
(FM 104.1)
When DONE Press F3.
(wait 1 minute)
RadioV1
(go to 92.9)
117.
When DONE Press F4.
118.
(YES / NO)
119.
(wait 1 minute)
© MIT AgeLab 2013
Page A222 of 293
RadioV2
[voice] →”Radio” →”Preset-5” → “yes”
(go to 106.7)
120.
When DONE Press F5.
121.
(YES / NO)
122.
(wait 10 seconds)
RadioV3
Please press the push-to-talk button, say Radio-AM and then turn the radio off.
(Pause 2 seconds.)
BEGIN
[voice] → “Radio AM” → [vol]
123.
124.
If radio not set on AM before being turned off
– prompt to turn radio back on, select AM and
turn off.
When DONE Press F6.
(off – AM 1030)
_____ Radio set in AM mode and off
(wait 1 minute)
© MIT AgeLab 2013
Page A223 of 293
RadioV4
Your task is to turn the radio on using the push-to-talk button and requesting FM 100.7. (Pause 2
seconds.) BEGIN
[voice] → “Radio” → “100.7” → “yes”
125.
When DONE Press F7.
126.
(YES / NO)
127.
(wait 10 seconds)
RadioV5
Your task is to turn off the radio. (Pause 2 seconds.) BEGIN.
[vol]
128.
(off)
When DONE Press F8.
(wait 1 minute)
© MIT AgeLab 2013
Page A224 of 293
RadioV6
Your task is to turn the radio on using the push-to-talk button and requesting FM 95.3. (Pause 2
seconds.) BEGIN.
[voice] → “Radio” → “95.3” → “yes”
129.
When DONE Press F9.
130.
(YES / NO)
131.
(wait 10 seconds)
RadioV7
Your task is to turn off the radio. (Pause 2 seconds.) BEGIN.
(loops back if participant taking too long)
[vol]
132.
(off)
When DONE Press F10.
133.
© MIT AgeLab 2013
Page A225 of 293
Section D_t Song Selection Training
134.
Continued from
Train1_M_Nav_first.set
or Train2_V_Song_first.set
135.
136.
Ensure that DAQ is running
Press F16 or F7 to begin
© MIT AgeLab 2013
(YES / NO)
Page A226 of 293
(start recording 2012b_D_t_1)
We are now going to introduce you to the operation of a feature that allows you to use the voice interface to
play music that has been stored on a portable music device. A USB memory stick with a set of recorded
music files has already been connected to the vehicle. All that you will be asked to do is to request specified
music selections using the voice interface.
To access this feature, you first press the push-to-talk button on the steering wheel. When this button is
pressed, the system will say ‘please say a command’. You then say ‘USB’. The system will then resume
playing a previous music request. Please try this now by pressing the push-to-talk button and then saying
‘USB’. Begin.
(end recording)
Remind participant of the location of the [voice] button on the steering wheel if needed, and then
prompt as needed through the steps below. It is ok to exit interface and start over as needed.
[voice] → “USB” → “yes”
137.
138.
Prompt as needed through steps above to play
a song. Repeat as needed.
Press F17 or F8 when USB mode is entered
correctly and music has played for a moment.
(start recording 2012b_D_t_1b)
To request a new music selection, you would press the push-to-talk button, and again say ‘USB’.
At this point, you can request a specific artist or album. For example, to request music by Elton
John, you could say ‘play artist Elton John’. If you wanted to listen to a specific album, you could
say ‘play album’ followed by the name of the album.
Let’s try a music request. Start by pressing the push-to-talk button, then say ‘USB’, and at the next
command prompt, say ‘play artist Elton John’. Please try this now. Begin.
(end recording)
[voice] → “USB” → “yes” → “Play Artist Elton John” → “yes”
139.
140.
141.
Prompt as needed through steps above to play
a song. Repeat as needed.
skip the next practice.
Press F18 or F9 to advance to next practice.
© MIT AgeLab 2013
Page A227 of 293
(start recording 2012b_D_t_2)
Let’s practice with another artist request with instructions the way they will be presented during
the drive. Remember, the sequence is ‘USB’, then “play artist’ and the name of the artist or “play
album” and the name of the album. Do not begin the task until you hear the word ‘begin’. (2
seconds) Your task is to play music by Bob Dylan.. (2 seconds) Begin.
(end recording)
[voice] → “USB” → “yes” → “Play Artist Bob Dylan” → “yes”
142.
143.
Answer questions and repeat as needed to
confirm participant understands task.
correctly use v commands to select a song.
© MIT AgeLab 2013
(YES / NO)
Page A228 of 293
Section D_e – Song Selection Evaluation
145.
2012b_RadioV_Song_x.set (where x is the nback configuration)
Press F13 to begin
146.
144.
(YES / NO)
(start recording 2012b_songtask_0) (33 sec + 2 for noise in media player = 35 sec)
A task period is about to start. The primary tasks will involve requesting music stored on a USB device
using the voice interface. As you practiced, this is done by pressing the push-to-talk button, saying ‘USB’,
then, to request music by a specific artist, say ‘play artist’ and the artist name. There will be a one minute
pause between tasks. If you have difficulty getting any of the music selections to play, you can stop at any
point and wait for the next selection task. Do not begin a task until you hear the word ‘begin’.
Before the first selection task, I would like you to activate the USB interface by pressing the push-to-talk
button, say ‘USB’, and allow the current selection to play. There will be a one minute pause before the first
music selection task is presented. Begin.
(end recording 2012b_songtask_0)
The USB interface needs to be turned on and allow existing music selection to play until
the first song selection task is presented so that two evaluation tasks are comparable and
because USB command defaults to playing a track before user can specify a selection.
[voice] → “USB” → “yes”
147.
148.
Prompt participant as needed to turn-on the
USB interface.
USB interface selected and music track is
playing.
(YES / NO)
(wait 1 minute from end of briefing sontask_0)
© MIT AgeLab 2013
Page A229 of 293
Song Select 1
(start recording 2012b_songtask_1) – 5 seconds
Your task is to play music by The Rolling Stones. (Pause 2 seconds) Begin.
If the participant is unsure how to proceed, you can, if necessary, prompt them to press
the push-to-talk button and to say ‘USB’. For any other help, just say ‘you can look at the
screen for other prompts’ and they should be left on their own to try to complete the
tasks.
[voice] → “USB” → “Play Artist The Rolling Stones”
149.
given up Press F14.
150.
(YES / NO)
151.
Were they able to play the correct artist?
(YES / NO)
(wait 1 minute)
Song Select 2
(Start recording 2012b_songtask_2) – 5 seconds
Your task is to play music by Johnny Cash. (Pause 2 seconds.) BEGIN
[voice] → “USB” → “Play Artist Johnny Cash”
152.
given up Press F15.
153.
(YES / NO)
154.
Were they able to play the correct artist?
(YES / NO)
(wait 1 minute)
© MIT AgeLab 2013
Page A230 of 293
Song Select 3
(Start recording 2012b_songtask_3) – 6 Seconds
Your task is to play ‘Let It Be’ by ‘The Beatles’ (Pause 2 seconds.) BEGIN
(NOTE: This step is to deliberately simulate a condition where the system does not appear to understand a request. The
introductory message gives the participant permission to discontinue a task if they experience difficulty. We are
looking in part at how long participants will persist if they system does not appear to understand them.)
[voice] → “USB” → “???”
155.
If participant asks for help, etc., tell them to
“Please just try whatever you think might
work.”
(wait 1 minute plus 10 seconds for instruction audio)
(Start recording 2012b_songtask_4)
The song task is complete. Please press the volume button now to turn off the music system. (Pause 2
seconds) The last task deliberately requested a song that did not exist on the storage device to simulate a
condition where the voice system did not appear to recognize your request. This is the only time that this
will be done intentionally during the study. Please continue driving
156.
© MIT AgeLab 2013
Page A231 of 293
Section E_t Phone Task Training
157.
158.
159.
Continued From Train1_M_Nav_First.set, or
Train2_V_Song_First.st
Verify that the car has synched with the
phone. (There will be a small bluetooth icon
on the top of the touch screen interface)
Press F19 to begin
© MIT AgeLab 2013
Page A232 of 293
(start recording 2012b_E_t_1)
We are now going to introduce you to a feature that allows you use the voice interface to place a phone call
to a number stored in a contact list. We will be using contacts that have already been created. After you
place a call, it will be routed to a voice-mailbox. A message will be played. Follow the instructions.
The volume control you use to adjust the radio volume may also be used to adjust the volume of the phone
interface.
To place a call, first press the push-to-talk button. The system will say ‘please say a command’. You then
say ‘phone’.
At the next prompt, you say, “call” followed by the contact name. For example, to call contact 1, you say
“call contact 1”.
Let’s practice by placing a call to contact 1. Remember, start by pressing the push-to-talk button, then say
‘phone, and then say ‘call contact 1’. Please try this now. Begin.
(end recording)
Remind participant of the location of the [voice] button on the steering wheel if needed, and then
prompt as needed through the steps below. It is ok to exit interface and start over as needed.
[voice] → “phone” → “yes” → “call contact 1” → “yes” →[End] (touch screen)
PLAYED OVER PHONE INTERFACE:
(recording 2012b_E_t_1A)
When this message is done playing, you will hear a beep. Please then say out loud your first name to leave
as a message. After saying your first name, please end this call by pressing the END button that appears
above the image of a phone on the touch screen.
(end)
[End] (touch screen)
160.
161.
162.
Prompt as needed through steps above to
place a call. Explain how to interact with the
voice mailbox if they are confused in anyway.
VOLUME – if the volume seems low and/or
the participant seems to have difficulty
hearting, remind them that they may adjust
the volume.
© MIT AgeLab 2013
Page A233 of 293
(start recording 2012b_E_t_2)
Let’s practice calling another contact number. Remember, to place a call, the sequence is ‘phone’,
then “call contact” followed by the contact name. Do not begin the task until you hear the word
‘begin’.
(2 seconds)
Your task is to place a call to contact 2. (2 seconds) Begin.
(end recording)
[voice] → “phone” → “yes” → “call contact 2” → “yes” →::listen to entire audio::
PLAYED OVER PHONE INTERFACE:
(recording 2012b_E_t_2A)
You have reached contact 2.
During the actual task, a short story will be read to you at this point. Your task will be to listen carefully
and to try to remember the story just the way it is read to you. When the story is finished, leave a message
by telling the story back out loud and include all that you can remember. What you say will be recorded so
that it can be scored later. Then end this call by pressing the END button on the touch screen.
(end)
[End] (touch screen)
163.
164.
165.
Answer questions and repeat as needed to
confirm participant understands task.
VOLUME – if the volume seems low and/or
the participant seems to have difficulty
hearting, remind them that they may adjust
the volume.
correctly use voice commands to place and
end a call.
© MIT AgeLab 2013
(YES / NO)
Page A234 of 293
Section E_e Phone Task Evaluation
NOTE: There are two versions of this protocol. Version A has the participant make the second
phone call to “contact-4” and version B makes the phone call to “contact-5” where “contact-5” is
the degraded audio quality version of the second story:
Phone Task II-A
(Continued from Manual-Radio-Nav)
63000
contact-4
Phone Task II-B
(Continued From Verbal-Radio-Song)
66000
contact-5
Note: this protocol assumes continuous recording on the DAQ and facelab systems (data recording should begin
after entering the highway at the RT495 rest stop)
© MIT AgeLab 2013
Page A235 of 293
Assuming that the tasks have been completed on 495N prior to reaching RT 93 steps 1 – 2 should
be used. Insert details on other potential exits on 495 north of RT 93 here.
Heading north on 495 - just after Exit 39 give the driver the following
114.
instructions:
“You are approaching Exit 40A for Route 93 south. Please prepare to take this exit”
As you approach Exit 40A give the driver the following instructions:
115.
“Remember; please take Exit 40A for Route 93 South.”
116.
Once past Exit 42 on I93 south Press F18 (note
this allows 3 min of habituation to RT 93
before task baseline starts)
(wait 33 seconds + 2 for noise in media player = 35 seconds)
(start recording 2012b_phonetask_0)
A task period is about to start. The tasks involve using the voice system to place a call to stored contact
number and listening to recorded information. You may use the volume control to adjust the phone volume
as needed. As you practiced, the sequence to place a call is ‘phone’, then ‘call contact’ followed by the
contact name. When you are done leaving the message, end the call by pressing the END button on the
touch screen.
There will be a one minute pause before the first phone call task is presented.
(end recording 2012b_phonetask_0)
(where XX is 60 plus length of 2012b_phonetask_0)
© MIT AgeLab 2013
Page A236 of 293
Phase E-1. Phone Task Part I
(start recording 2012b_phonetask_1) – 5 Seconds
Please place a phone call now to contact-3. Begin.
117.
If the participant is unable to make the call,
prompt as needed so that they are able to
connect with the recording.
118.
Press F19 when system begins to dial.
(start recording phonetask_rec1.wav) - 14 Seconds
You do not have to remember any of the content of this recording. All we are interested in for this task is
assessing how much effort is involved in placing a phone call. When this recording is finished, please leave a
message like you did in practice by just saying your first name.
(end recording phonetask_rec1.wav).
119.
120.
Participant was able to place call on their own
without prompting.
(YES / NO)
Do NOT prompt them to leave their name if
they forget. Allow up to about 30 seconds of
silence then, if needed, prompt them to end the
call by pressing the END button.
© MIT AgeLab 2013
Page A237 of 293
Phase E-2. Phone Task Part II:
Note: If running Version B skip to section Phone Task II_B
Phone Task II-A
(Continued from Manual-Radio-Nav)
121.
(617) 564-1549
Press F20 to advance to story
(wait 1 minute)
(start recording 2012b_phonetask_2A) - 23 seconds
The next recorded message will be a short story. Listen carefully and try to remember it just the way it is read,
in as close to the same words as you can remember. When the recording finishes, leave a message telling back
everything that you heard. You should tell all that you can remember, even if you are not sure. What you say
will be recorded and scored later.
Please place a phone call now to contact-4.
Begin.
(end recording 2012b_phonetask_2A)
122.
123.
connect with the recording. Prompt participant
to turn volume up to ensure they do not miss
the beginning of the story.
Press F21when system begins to dial
Contact 4
(start recording 2012b_story_A.wav) (original version) – 24 Seconds
“Anna Thompson of South Boston, employed as a cook in a school cafeteria, reported at the police station that
she had been held up on State Street the night before and robbed of fifty-six dollars. She had four small children,
the rent was due, and they had not eaten for two days. The police, touched by the woman’s story, took-up a
collection for her.”
(end recording story_part2A.wav)
Move to Step 11
© MIT AgeLab 2013
Page A238 of 293
Phase E-3. Phone Task II-B
(Continued from Verbal-Radio-Song)
8.
(617) 863-0834
Press F20 to advance to story
(wait 1 minute)
(start recording 2012b_phonetask_2B) – 22 Seconds
The next recorded message will be a short story. Listen carefully and try to remember it just the way it is read,
in as close to the same words as you can remember. When the recording finishes, leave a message telling back
everything that you heard. You should tell all that you can remember, even if you are not sure. What you say
will be recorded and scored later.
Please place a phone call now to contact-5.
Begin.
(end recording 2012b_phonetask_2B)
9.
10.
connect with the recording. Prompt participant
to turn volume up to ensure they do not miss
the beginning of the story.
Press F21when system begins to dial
Contact 5
(start recording 2012b_story_B.wav) – 24 Seconds
(DISTORTED version)
(end recording story_part2B.wav)
Move to Step 11
© MIT AgeLab 2013
Page A239 of 293
11.
12.
Participant was able to place call on their own
without prompting.
(YES / NO)
Leave message - If the participant does not say
anything, is hesitant or seems to freeze up after
about 10 seconds, verbally prompt once
“Please go ahead and say all you can
remember.”
13.
14.
15.
Participant was able to begin leaving a message
on their own without prompting.
When the participant is finished responding or
doesn’t respond to the second prompt within
30 seconds – prompt them to end the call by
pressing the END button if they do not do so
themselves.
(YES / NO)
Press F22 to advance to story recall
© MIT AgeLab 2013
Page A240 of 293
Phase E-4. Phone Task Part III:
(wait 1 minute)
(start recording 2012b_phonetask_3) – 19 Seconds
For your final task, please take some time to review silently in your mind the story about the woman. Try to
remember if there were any details that you forgot to mention when you left the phone message. Think back
carefully over all the details in the recording, but do not say anything out loud right now.
Begin.
(wait 1 minute)
16.
Record next exit number.
Exit __________
17.
Did the participant engage in the entire task
(Please answer no and provide details if they
appeared to stop responding for part of all of
the task)
18.
Move to checklist section 6 (Returning to MIT)
© MIT AgeLab 2013
(YES / NO)
Page A241 of 293
Section X_1 – N-back Evaluation Manual Radio / Nav
Set number __________
166.
Press F12 to begin
167.
168.
As each task plays record the n-back level and all participant
responses.
(YES / NO)
Intro recordings
intro0.wav: We are now going to complete a series of scored trials of the zero-back task. Remember that in
this task, you are to repeat out loud the number that you just heard. Try to be as accurate as you can be.
intro1.wav: We are now going to complete a series of scored trials of the one-back task. Remember that in
this task, you are to repeat out loud
the number before the number that you just heard. Try to be as accurate as you can be.
intro2.wav: We are now going to complete a series of scored trials of the two-back task. Remember that in
this task, you are to repeat out loud the number that you heard two numbers ago. Try to be as accurate as
you can be.
© MIT AgeLab 2013
Page A242 of 293
NOTE CROSS OUT UN-USED STIMULUS SET DURING PROTOCOL SETUP
If protocol A, B, C
(wait 2minutes 30 seconds)
___- back task
(wait 1 minute after task)
___- back task
____- back task
169.
Score the 0-back
______ /10
170.
Score the 1-back
______ /9
171.
Score the 2-back
______ /8
172.
Move to next protocol (2 minute recovery and
30 sec separation).
© MIT AgeLab 2013
Page A243 of 293
Section X_2 – N-back Evaluation Voice Radio / Song
Set number __________
173.
Press F12 to begin
174.
175.
As each task plays record the n-back level and all participant
responses.
(YES / NO)
Intro recordings
intro0.wav: We are now going to complete a series of scored trials of the zero-back task. Remember that in
this task, you are to repeat out loud the number that you just heard. Try to be as accurate as you can be.
intro1.wav: We are now going to complete a series of scored trials of the one-back task. Remember that in
this task, you are to repeat out loud
the number before the number that you just heard. Try to be as accurate as you can be.
intro2.wav: We are now going to complete a series of scored trials of the two-back task. Remember that in
this task, you are to repeat out loud the number that you heard two numbers ago. Try to be as accurate as
you can be.
© MIT AgeLab 2013
Page A244 of 293
NOTE CROSS OUT UN-USED STIMULUS SET DURING PROTOCOL SETUP
If protocol D, E, F
(wait 2minutes 30 seconds)
___- back task
___- back task
____- back task
176.
Score the 0-back
______ /10
177.
Score the 1-back
______ /9
178.
Score the 2-back
______ /8
179.
Move to next protocol (2 minute recovery and
30 sec separation).
© MIT AgeLab 2013
Page A245 of 293
Appendix I: Self-Reported Workload Materials
Asking someone to rate how relatively easy or difficult a task is, how much concentration it
requires, etc. represents an apparently straightforward method of assessing workload. Self-report
based measures are particularly attractive since simple scale questions can easy be written and
administered. Several established workload scales, such as the NASA Task Load Index (TLX),
have been used extensively in published research, particularly in the astronautic and aerospace
fields, and provide an aura of measurement precision. Subjective measures and scales are also
quite commonly used in transportation related research and, as noted by Brookhus and De Waard
(2002, p. 1026). “It is hard to imagine research in the field without subjective measurement.”
However, while obtaining subjective feedback and ratings can be a very useful and important
part of the overall design assessment process (i.e. determining if individuals like or dislike an
interface, find it confusing, etc.), it is important to recognize potential limitations related to when
and how they are administered, and the extent to which training and experience is needed to
effectively use certain self-report measures.
Based upon our experiences with older participants, we believe that there may be limitations in
the use of workload scales developed for use by highly trained astronautic and aeronautical
specialists (pilots & engineers) for purposes of assessing automotive interfaces as they are used by
the general population. It should be recognized that devices such as the NASA TLX and the
Bedford workload scales (Hart & Staveland, 1988; Roscoe, 1992) were developed for experienced
pilots and related specialists to rate tasks and interfaces that would be used by other similarly
trained specialists. The manner in which such specialists approach and interact with an HMI
cannot be assumed a priori to be a good model of how non-specialists will experience the same
HMI. Similarly, providing a “normal” driver with extensive training on how to use such scales is
likely to influence the way they begin to think about and evaluate an HMI, which may result in
these individuals no longer being an accurate representation of a “normal” driver. In light of this,
we have generally relied on more simple and direct questions asking participants to rate how
relatively difficult or easy a task was, to rate how stressed they felt during the task, etc. Therefore,
our efforts have focused on the use of a more simplistic, Likert style global workload scale. To
enhance the sensitivity of the scale over traditional 4, 5, 7, 10 or 11 demarcation points,
participates were provided with 21 points rating points referenced against a more common 0 to 10
rating nomenclature as illustrated below. In addition, all self-reported workload ratings were
collected on a single sheet of paper allowing the participant to easily “benchmark” a ratings for a
specific interaction relative to earlier ratings.
© MIT AgeLab 2013
Page A246 of 293
Instructions
The following instructions on how to fill out the workload rating scales were given to participants to read
by a research associate. Participants were given an opportunity to ask any questions that they had. The
page following this one reproduces the paper and pencil rating form that was given to participants at the
mid-point rest period to rate the tasks completed during Part I of the drive, and again at the completion of
the experiment to rate the remaining tasks. The form was given on a larger, legal size sheet (11x17 inches)
and clipboard; it is reduced in size here. The standard sizing supports a 10cm long visual scale using the
numbers 0 to 10 but with the option to circle midpoint values corresponding to a potential 21 scalar values.
How to Fill-Out the Workload Rating Scales
One of the goals of this study is to understand how much effort is involved doing different tasks while
driving. At the mid-point rest stop and at the end of the drive, we will ask you to fill out a short form to rate
how much workload was associated with each type of task using a 0 to 10 scale where 0 = very low
workload and 10 = very high workload.
By workload, we mean the demand or effort that was associated with trying to do a task while continuing to
drive safely. Workload is experienced in different ways by different people. Workload could mean the
amount of mental effort that was involved in the task, the amount of attention that was required, how
physically difficult it was to complete, how much time pressure was involved, or how frustrating the task
was. You can decide for yourself how to define workload as it relates to doing these tasks while also
continuing to drive safely.
The instructions shown below will appear on the form:
Please circle a point along each scale that best corresponds to how much workload you felt
was involved in trying to do each task. Workload is best defined by the person doing the
task and may involve mental effort, the amount of attention required, physical effort, time
pressure, distraction or frustration associated with trying to do the task while continuing to
drive safely.
Example of Format that will be used to rate the workload associated with a task:
Task XXX
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
Low
0
1
2
3
4
5
6
7
8
9
10
High
Please let the research associate know if you have any questions about how to fill-out this form.
Thank you.
© MIT AgeLab 2013
Page A247 of 293
Example Rating Sheet
Workload Rating – 2012b
Please circle a point along each scale that best corresponds to how much workload you felt was involved in
trying to do each task. Workload is best defined by the person doing the task and may involve mental effort,
the amount of attention required, physical effort, time pressure, distraction or frustration associated with
trying to do the task while continuing to drive safely.
Radio - Manual (button) Interface to Select a Preset Station
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
Radio - Manual Interface to Change from AM to FM and tune to a Specific Station
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
Radio - Verbal Interface to Select a Preset Station
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
Radio - Verbal Interface to Change from AM to FM and tune to a Specific Station
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
Search Stored Music for a Specific Artist (first 2 tasks)
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
Final Stored Music Search for music by ‘The Beatles’
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
0-Back Number Task
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
1-Back Number Task
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
© MIT AgeLab 2013
1
2
3
4
6
7
8
9
10
High
Page A248 of 293
2-Back Number Task
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
Navigation System - Entering a Destination Street Address
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
Navigation System – Canceling a Route Request
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
Voice Phone Dialing using a Stored Number
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
Listening to a story that you have to remember over the phone interface
Low
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
0
5
1
2
3
4
6
7
8
9
10
High
Note: This form was presented on a single 11x17” legal size sheet of paper attached to a legal size
clipboard so that all scales were presented to the participant on the same page.
© MIT AgeLab 2013
Page A249 of 293
Appendix J: Questionnaires
Participant Code #______________
Date and Time ______________
Pre-Experiment Questionnaire
Please fill in the blanks or circle the one best response unless otherwise noted. If more space is needed,
please note the question number and continue on the back of each sheet. Remember, filling out this
questionnaire is voluntary. Skipping any question that makes you feel uncomfortable will not exclude you
from the study.
1. What is your date of birth? _________ (month / year)
2. Are you:
Male
Female
3. Thinking about how you feel today, how would you describe your current physical well-being?
a.
b.
c.
d.
e.
Excellent
Good
Mean
Fair
Poor
4. Thinking about how you feel today, how would you describe your current mental well-being?
a.
b.
c.
d.
e.
Excellent
Good
Mean
Fair
Poor
5. Do you feel sick right now?
a. Yes
b. No
(If yes, please inform the research assistant.)
6. Has anything happened in your life or have any events occurred recently that might influence
how you are feeling, your comfort in taking part in the experiment or your ability to drive
today?
a. Yes
(If yes, please describe briefly below.)
b. No
________________________________________________________________
________________________________________________________________
________________________________________________________________
7. Do you normally wear corrective lenses when driving?
© MIT AgeLab 2013
Page A250 of 293
a. No
b. Yes - glasses
c. Yes - contacts
8. Compared with others your age, how would you rate your overall vision? (If you wear glasses
or contacts, rate your corrected vision when you are wearing them.)
a.
b.
c.
d.
e.
Excellent
Good
Mean
Fair
Poor
9. Compared with others your age, how would you rate your overall hearing?
a.
b.
c.
d.
e.
Excellent
Good
Mean
Fair
Poor
10. Do you wear a hearing aid?
a. Yes
b. No
Please continue on to the next page.
© MIT AgeLab 2013
Page A251 of 293
Technology
11. On a scale of 1 to 10, with 1 being very inexperienced and 10 being very experienced, how
would you rate your level of experience with technology (e.g. cell phones, automatic teller
machines, digital cameras, computers, etc.)?
1
2
Very
Inexperienced
3
4
5
6
7
8
9
10
Very
Experienced
12. Some people prefer to avoid new technologies as long as possible while others like to try them
out as soon as they become available. In general, how would you rate yourself as being an
avoider or an early adopter of new technology?
1
2
Avoid as
long as possible
3
4
5
6
7
8
9
8
9
10
Try as
soon as possible
13. How would you rate your overall level of trust in technology?
1
2
Very
Distrustful
3
4
5
6
7
10
Very
Trustful
14. How would you rate your level of trust in established car technologies (e.g. anti-lock brakes,
automatic transmissions, air bags, etc.)?
1
2
Very
Distrustful
3
4
5
6
7
8
9
10
Very
Trustful
15. How would you rate your level of trust in new technologies that are being introduced into cars?
1
2
Very
Distrustful
3
4
5
6
7
8
9
10
Very
Trustful
16. How would you rate your ability to learn how to operate new technologies?
1
Very
Poor
© MIT AgeLab 2013
2
3
4
5
6
7
8
9
10
Very
Good
Page A252 of 293
17. On mean, how often do you use an electronic navigation system in a car or truck (using a builtin navigation system, portable navigation unit or a smart phone)?
a.
b.
c.
d.
e.
f.
More than once a day
About once a day
A few times a week
A few times a month
A few times a year
Never
18. How often do you use a voice command interface in any environment (on a smart phone, in
your car, or some other voice enabled system such as speech to text translation software)?
a.
b.
c.
d.
e.
f.
About once a day
A few times a week
A few times a month
A few times a year
Never
19. How often do you use a car or truck based voice command interface system?
a.
b.
c.
d.
e.
f.
About once a day
A few times a week
A few times a month
A few times a year
Never
20. How often do you use a touch screen interface in any environment (on a computer, smart
phone, an iPad, tablet computer, e-book, etc.)?
a.
b.
c.
d.
e.
f.
About once a day
A few times a week
A few times a month
A few times a year
Never
© MIT AgeLab 2013
Page A253 of 293
21. How often do you make a phone call while driving a car?
a.
b.
c.
d.
e.
f.
About once a day
A few times a week
A few times a month
A few times a year
Never
22. How often do you make a phone call using a voice dialing system while driving a car?
a.
b.
c.
d.
e.
f.
About once a day
A few times a week
A few times a month
A few times a year
Never
23. How often do you answer a phone call while driving a car?
a.
b.
c.
d.
e.
f.
About once a day
A few times a week
A few times a month
A few times a year
Never
© MIT AgeLab 2013
Page A254 of 293
Driving History
24. Do you currently have a valid state issued driver’s license?
a. Yes
b. No (Please inform the research assistant)
25. How often do you drive a car or other motor vehicle?
a.
b.
c.
d.
e.
Almost every day
A few days a week
A few days a month
A few times a year
Never
26. Over the last year, how many miles did you drive?
a.
b.
c.
d.
e.
f.
g.
h.
Under 1000
Between 1,001 and 5,000
Over 20,001
None
(Please inform the research assistant)
I don’t know
27. On a scale of 1 to 10 with 1 being very unsafe and 10 being very safe, how safe a driver do you
think you are?
1
Very
Unsafe
2
3
4
5
6
7
8
9
10
Very
Safe
28. In the past five years, how many times have you been stopped by a police officer and received
a warning (but no citation or ticket) for a moving violation (i.e. speeding, running a red light,
running a stop sign, failing to yield, reckless driving, etc.)?
Enter a number: _________? (Enter 0 for none.)
29. In the past five years, how many times have you been stopped by a police officer and received
a citation or ticket for a moving violation?
30. In the past five years, how many times have you been in a vehicle crash where you were the
driver of one of the vehicles involved?
© MIT AgeLab 2013
Page A255 of 293
Driving Behavior
31. Nobody is perfect. Even the best drivers make mistakes, do foolish things, or bend the rules at some time
or another. For each item below you are asked to indicate HOW OFTEN, if at all, this kind of thing has
happened to you. Base your judgments on what you remember of your driving over, say, the last month.
Please indicate your judgments by circling ONE of the numbers next to each item. Remember we do not
expect exact answers, merely your best guess; so please do not spend too much time on any one item.
How often do you do each of the following (for
example, in the past month)?
Never
Hardly
ever
Occasionally
Quite
often
Frequently
Nearly
all the
time
a
Try to pass another car that is signaling a left
turn.
0
1
2
3
4
5
b
Select the wrong turn lane when approaching
an intersection.
0
1
2
3
4
5
c
Fail to ‘Stop’ or ‘Yield’ at a sign, almost
hitting a car that has the right of way.
0
1
2
3
4
5
d
Misread signs and miss your exit.
0
1
2
3
4
5
e
Fail to notice pedestrians crossing when
turning onto a side street.
0
1
2
3
4
5
f
Drive very close to a car in front of you as a
signal that they should go faster or get out of
the way.
0
1
2
3
4
5
g
Forget where you parked your car in a parking
lot.
0
1
2
3
4
5
h
When preparing to turn from a side road onto
a main road, you pay too much attention to
the traffic on the main road so that you nearly
hit the car in front of you.
0
1
2
3
4
5
i
When you back up, you hit something
that you did not observe before but was
there.
0
1
2
3
4
5
j
Pass through an intersection even though
you know that the traffic light has turned
yellow and may go red.
0
1
2
3
4
5
© MIT AgeLab 2013
Page A256 of 293
k
0
1
2
3
4
5
Never
Hardly
ever
Occasionally
Quite
often
Frequently
Nearly
all the
time
0
1
2
3
4
5
m Forget that your lights are on high beam
until another driver flashes his headlights
at you.
0
1
2
3
4
5
n
Fail to check your rear-view mirror before
pulling out and changing lanes.
0
1
2
3
4
5
o
Have a strong dislike of a particular type
of driver, and indicate your dislike by any
means that you can.
0
1
2
3
4
5
p
Become impatient with a slow driver in
the left lane and pass on the right.
0
1
2
3
4
5
q
Underestimate the speed of an oncoming
vehicle when passing.
0
1
2
3
4
5
r
Switch on one thing, for example, the
headlights, when you meant to switch on
something else, for example, the
windshield wipers.
0
1
2
3
4
5
s
Brake too quickly on a slippery road, or
turn your steering wheel in the wrong
direction while skidding.
0
1
2
3
4
5
t
You intend to drive to destination A, but
you ‘wake up’ to find yourself on the
road to destination B, perhaps because B
is your more usual destination.
0
1
2
3
4
5
u
Drive even though you realize that your
blood alcohol may be over the legal limit.
0
1
2
3
4
5
v
Get involved in spontaneous, or spur-ofthe moment, races with other drivers.
0
1
2
3
4
5
l
When making a turn, you almost hit a
cyclist or pedestrian who has come up on
your right side.
Ignore speed limits late at night or very
early in the morning.
© MIT AgeLab 2013
Page A257 of 293
w Realize that you cannot clearly remember
the road you were just driving on.
0
1
2
3
4
5
x
0
1
2
3
4
5
You get angry at the behavior of another
driver and you chase that driver so that
you can give him/her a piece of your
mind.
© MIT AgeLab 2013
Page A258 of 293
Demographics
The following are standard questions that allow researchers to determine how representative the
group of participants in a study is of the general population. Remember, filling out this
questionnaire is voluntary. Skipping any question that makes you feel uncomfortable will not
exclude you from the study.
32. Please describe the highest level of formal education you have completed:
a.
b.
c.
d.
e.
f.
Some high school
High school graduate
Some college
College graduate
Some graduate education
Completed graduate or professional degree (e.g. Masters, LCSW, JD, Ph.D., MD, etc.)
33. Are you: (Please circle all that apply.)
a.
b.
c.
d.
e.
f.
g.
h.
A full time student
A part time student
Unemployed
Retired
Employed full time
Employed part time
A full time caregiver (e.g. children or elder)
A part time caregiver (e.g. children or elder)
34. Are you:
a.
b.
c.
d.
e.
Married
Divorced
Widowed
Single living with partner
Single never married
35. What best describes your total household income?
c.
d.
e.
f.
g.
h.
i.
j.
Less than $25,000
$25,000 – $49,999
$50,000 – $74,999
$75,000 – $99,999
$100,000 – $124,999
$125,000 – $149,999
$150,000 or more
I don’t know
© MIT AgeLab 2013
Page A259 of 293
Current State
36. On a scale of 1 to 10 with one being very awake and 10 being extremely drowsy, how do you
feel right now?
1
Very
Awake
2
3
4
5
6
7
8
9
10
Extremely
Drowsy
37. On a scale of 1 to 10 where one is not at all stressed and 10 is very stressed how would you
rate your stress level right now?
1
Not at all
Stressed
© MIT AgeLab 2013
2
3
4
5
6
7
8
9
10
Very
Stressed
Page A260 of 293
Date and Time ______________
Please fill in blanks or circle the one best response unless otherwise noted. If more space is needed
please note the question number and continue on the back of each sheet. Remember, filling out this
questionnaire is voluntary. Skipping any question that makes you feel uncomfortable will not
exclude you from the study.
1. Compared with others your age, how would you rate your overall health?
a.
b.
c.
d.
e.
Excellent
Good
Mean
Fair
Poor
2. Approximately how many times have you seen a medical doctor in the past 12 months? ______
© MIT AgeLab 2013
Page A261 of 293
3. Please check if you have been diagnosed (told by a doctor) that you have had, or currently
have, any of the following medical conditions:
a.
b.
c.
d.
e.
f.
g.
h.
i.
_____
_____
_____
_____
_____
_____
_____
_____
_____
heart attack
angina
irregular heart rhythm
high blood pressure
high cholesterol
diabetes
thyroid condition
stroke or TIA
arthritis, rheumatism
j.
k.
l.
m.
n.
o.
p.
q.
_____
_____
_____
_____
_____
_____
_____
_____
fibromyalgia
chronic pain
Parkinson’s disease
gastrointestinal problems, GRD
spinal surgery
blood clot in leg or lung
hip fracture
hip or knee replacement
4. Do you currently have any medical conditions not listed above?
If yes, please list below. If no, please check here ______ and skip to the next question.
a.
b.
c.
d.
___________________________________________________
___________________________________________________
___________________________________________________
___________________________________________________
5. To the best of your knowledge, are you currently taking any medications for the treatment of
high blood pressure, a heart condition, or other cardiovascular condition? If yes, please list
the medication or medications below:
________________________________________________________
________________________________________________________
6. To the best of your knowledge, are you currently taking any medications associated with the
treatment of diabetes? If yes, please list the medication or medications below:
________________________________________________________
________________________________________________________
7. Please list any other medications, herbs, or vitamins that you currently take:
________________________________________________________
________________________________________________________
________________________________________________________
© MIT AgeLab 2013
Page A262 of 293
8. To what extent are you limited in the degree to which you can easily turn your head, neck or
shoulders due to stiffness, pain, arthritis, general aging or any other condition?
1
Not at all
Limited
2
3
4
5
6
7
8
9
10
Very
Limited
9. When you are driving, to what extent does stiffness, pain, arthritis, general aging or any other
condition affect how easy it is for you turn around and look behind you, look up to the
rearview mirror, or check the side mirrors ?
1
Not at all
Affected
2
3
4
5
6
7
8
9
10
Very
Affected
For the next two questions on chronic or frequent pain, please consider any chronic or frequent
pain you have experienced during the past 12 months.
10. Please check all areas in which you experience chronic or frequent pain:
a.
b.
c.
d.
e.
f.
____ face / jaw / head
____ neck
____ shoulders / upper back
____ low back
____ seat / buttock / hip region
____ legs
g.
h.
i.
j.
k.
____ feet / toes
____ chest
____ abdominal or stomach region
____ arms
____ hands / fingers
11. To the best of your knowledge, what is the source of your pain? Check all that apply.
a. ____ Does Not Apply (do not experience chronic or frequent pain)
b. ____ general stiffness and discomfort
c. ____ chronic headaches
d. ____ arthritis
e. ____ muscle pain
f. ____ nerve pain
g. ____ fibromyalgia
h. ____ other (list)
__________________________________________________________
i. ____ unknown
© MIT AgeLab 2012
last revision 2012-03-11
Page 263 of 293
If any of the following questions do not apply to you (i.e. you are not currently experiencing
any pain or did not experience any pain last week), then rate a scale at 0.
Pain Intensity
Please circle a point along the scales below to show how intense your pain is. The zero (0)
end of the scale means no pain and the ten (10) end means extreme pain.
12. How intense is your pain now?
no
pain
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0
1
2
3
4
5
6
7
8
9
extreme
pain
10
14. How intense was your pain on mean last week?
no ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● extreme
pain 0
1
2
3
4
5
6
7
8
9
10 pain
Distressing - Now use the same method to describe how distressing your pain is.
15. How distressing is your pain now?
not at all
distressing
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0
1
2
3
4
5
6
7
8
9
10
extremely
distressing
16. How distressing was your pain on mean last week?
not at all
distressing
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0
1
2
3
4
5
6
7
8
9
10
extremely
distressing
Daily Activities - Now use the same method to describe how much your pain interferes with
your everyday activities.
17. How much does your pain interfere with your daily activities in general?
does not
interfere
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0
1
2
3
4
5
6
7
8
9
10
interferes
completely
18. How much does your pain interfere with your ability to drive?
does not
interfere
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0
©MIT AgeLab 2013
1
2
3
4
5
6
7
8
9
10
interferes
completely
Page A264 of 293
Date and Time ______________
Task Rating Questionnaire
Please fill in blanks or circle the one best response unless otherwise noted. Remember, filling
out this questionnaire is voluntary. Skipping any question that makes you feel uncomfortable
will not exclude you from the study.
Current State
(Mid-Point Rest Stop)
1. On a scale of 1 to 10 with 1 being very awake and 10 being extremely drowsy, how do
you feel right now?
1
Very
Awake
2
3
4
5
6
7
8
9
10
Extremely
Drowsy
2. On a scale of 1 to 10 where 1 is not at all stressed and 10 is very stressed how would you
1
Not at all
Stressed
2
3
4
5
6
7
8
9
10
Very
Stressed
IF YOU ARE FILLING THIS OUT AT THE
-------- MID-POINT REST STOP -------Please fill-in the sections on the following pages just for the tasks you have completed so
far.
You will be asked to fill-in the remaining sections at the end of the drive.
Thank you.
IF YOU ARE FILLING THIS OUT AT
-------- MIT -------Please fill-in the remaining sections.
Thank you.
©MIT AgeLab 2013
Page A265 of 293
Manual Control of the Radio
1. Thinking now about using the manual interface to operate the radio, on a scale of 1 to 10,
with 1 being not at all and 10 very, please indicate on mean:
Not at All
Very
a. To what extent did you feel that the
manual radio interface was intuitive or
easy to learn how to use? ……......…..
1
2
3
4
5
6
7
8
9
10
b. How difficult was it to use the manual
interface to operate the radio? ..............
1
2
3
4
5
6
7
8
9
10
c. To what extent did you feel frustrated
using the manual radio interface? ..…..
1
2
3
4
5
6
7
8
9
10
d. How much did using the manual radio
interface distract from driving? ............
1
2
3
4
5
6
7
8
9
10
e. How difficult was using the manual
interface compared to things you usually
do while driving? ……………..…..…..
1
2
3
4
5
6
7
8
9
10
Voice Control of the Radio
2. Thinking now about using the voice interface to operate the radio, on a scale of 1 to 10,
Not at All
a. To what extent did you feel that the voice
radio interface was intuitive or easy to
learn how to use? ……......................….. 1 2
3
4
5
6
7
8
9
10
b. How difficult was it to use the voice
interface to operate the radio? ...............
1
2
3
4
5
6
7
8
9
10
using the manual radio interface? ……..
1
2
3
4
5
6
7
8
9
10
d. How much did using the voice radio
interface distract from driving? .............
1
2
3
4
5
6
7
8
9
10
e. How difficult was using the voice
do while driving? ……………..….…....
1
2
3
4
5
6
7
8
9
10
©MIT AgeLab 2013
Very
Page A266 of 293
Voice Selection of Stored Music
3. (Please make the following ratings just for the first two music selection tasks – not the
third music selection task.) Thinking now about using the voice interface to select stored
music, on a scale of 1 to 10, with 1 being not at all and 10 very, please indicate on mean:
Not at All
a. To what extent did you feel that the music
interface was intuitive or easy to learn
how to use? ……...............................….. 1 2
Very
3
4
5
6
7
8
9
10
interface to select stored music? .............
1
2
3
4
5
6
7
8
9
10
using the voice music interface? ..……...
1
2
3
4
5
6
7
8
9
10
d. How much did using the voice interface
to select music distract from driving? .....
1
2
3
4
5
6
7
8
9
10
do while driving? ……………..….…....
1
2
3
4
5
6
7
8
9
10
Voice Controlled Navigation System
4. Thinking now about using the voice interface to control the navigation system, on a scale
of 1 to 10, with 1 being not at all and 10 very, please indicate on mean:
Not at All
Very
navigation interface was intuitive or easy
to learn how to use? ……................…..
1
2
3
4
5
6
7
8
9
10
interface to operate navigation system?
1
2
3
4
5
6
7
8
9
10
using the navigation interface? …..…...
1
2
3
4
5
6
7
8
9
10
d. How much did using the voice navigation
interface distract from driving? ............
1
2
3
4
5
6
7
8
9
10
do while driving? ……………..…..…..
2
3
4
5
6
7
8
9
10
©MIT AgeLab 2013
1
Page A267 of 293
[ THE REMAINING ITEMS TO BE COMPLETED AT MIT AT THE END OF THE DRIVE. ]
Voice Control of Phone Dialing
5. Thinking now about using the voice interface to place a phone call, on a scale of 1 to 10,
Not at All
Very
phone interface was intuitive or easy to
learn how to use? ……....................…..
1
2
3
4
5
6
7
8
9
10
interface to place a call? ........................
1
2
3
4
5
6
7
8
9
10
using interface to place a call? ………..
1
2
3
4
5
6
7
8
9
10
d. How much did placing a phone call
distract from driving? .............................
1
2
3
4
5
6
7
8
9
10
interface to place a call compared to
things you usually do while driving? …..
1
2
3
4
5
6
7
8
9
10
Listening to Phone Message Content
6. (If you found there to be any difference in listening to the two messages, think more about
the message that was more difficult to follow.) Thinking now about listening to the
messages initiated by the phone calls, on a scale of 1 to 10, with 1 being not at all and 10
very, please indicate on mean:
Not at All
Very
a. How difficult was it to listen to the
messages? ...............................................
1
2
3
4
5
6
7
8
9
10
b. To what extent did you feel frustrated
listening to the messages?.........………..
1
2
3
4
5
6
7
8
9
10
c. How much did listening to the messages
distract from driving? .............................
1
2
3
4
5
6
7
8
9
10
d. How difficult was listening to the
messages compared to things you usually
do while driving? ……………………….
1
2
3
4
5
6
7
8
9
10
©MIT AgeLab 2013
Page A268 of 293
Listening to Phone Message Content (continued)
7. Thinking about listening to the two different phone messages, on a scale of 1 to 10, with 1
being not at all and 10 very, please indicate on mean:
Not at All
a. To what extent was one message more
difficult to hear than the other? ...............
©MIT AgeLab 2013
1
2
Very
3
4
5
6
7
8
9
10
Page A269 of 293
Date and Time ______________
Please fill in blanks or circle the one best response unless otherwise noted. Remember, filling
out this questionnaire is voluntary. Skipping any question that makes you feel uncomfortable
will not exclude you from the study.
Current State
1. On a scale of 1 to 10 with 1 being very awake and 10 being extremely drowsy, how do
you feel right now?
1
Very
Awake
2
3
4
5
6
7
8
9
10
Extremely
Drowsy
2. On a scale of 1 to 10 where 1 is not at all stressed and 10 is very stressed how would you
1
Not at all
Stressed
©MIT AgeLab 2013
2
3
4
5
6
7
8
9
10
Very
Stressed
Page A270 of 293
General Impressions:
3. What was your overall impression of the vehicle you drove today?
1
Not at All
Positive
2
3
4
5
6
7
8
9
10
Very
Positive
9
10
Very
Easy
Considering the Voice Control Interface you worked with today:
4. As an overall rating, how easy was the voice control system to use?
1
2
Not Very
Easy
3
4
5
6
7
8
5. How easy was it to understand the meaning of the voice prompts provided by the system?
1
2
Not Very
Easy
3
4
5
6
7
8
9
10
Very
Easy
6. Did the system seem to have any difficulty understanding your verbal commands?
1
2
Not at All
3
4
5
6
7
8
9
10
A Lot
7. Thinking back to how you felt earlier today when this technology was introduced, and
comparing that with how you feel now after using the technology, how are you feeling
about this technology now?
a.
b.
c.
d.
e.
Much less positive
Somewhat less positive
About the same
Somewhat more positive
Much more positive
8. How has your experience today with this technology influenced your level of trust in new
technologies that are being introduced into cars?
a.
b.
c.
d.
e.
Much less trusting
Somewhat less trusting
About the same
Somewhat more trusting
Much more trusting
©MIT AgeLab 2013
Page A271 of 293
9. How well do you feel the introduction to the voice command system prepared you for
working with the system?
1
Not
Very well
2
3
4
5
6
7
8
9
10
Very
well
10. If you had the opportunity to gain more experience with the system (by using for several
weeks, etc.), how do you think this would affect your rating of the system?
a.
b.
c.
d.
e.
Would likely feel much more positive
Would likely feel somewhat more positive
Not likely to make any difference
Would likely feel somewhat less positive
Would likely feel much less positive
11. Based on your experience today, has your sense of your ability to learn new technologies
increased, stayed about the same, or decreased?
1
2
Decreased
3
4
5
6
7
8
9
10
Increased
12. Do you feel that the voice control system made operating the vehicle systems:
a.
b.
c.
d.
e.
much safer
somewhat safer
no difference
somewhat less safe
much less safe
13. In addition to providing audio prompts in response to your voice commands, the system
often displayed information or hints about how to work the system on the in-vehicle
display screen. Did you find this additional display information helpful?
1
Not
Helpful
2
3
4
5
6
7
8
9
10
Very
Helpful
14. When you used the voice command interface, to what extent were you able to keep your
eyes on the road or did you tend to look frequently at the additional status information on
the in-vehicle display screen?
1
2
Eyes
On Road
©MIT AgeLab 2013
3
4
5
6
7
8
9
10
Eyes
On Screen
Page A272 of 293
15. When you used the voice interface, to what extent did you find yourself becoming
absorbed in the interaction such that your attention was diverted from driving?
1
2
Not at All
3
4
5
6
7
8
9
10
A Lot
16. Given time to develop enough familiarity with the voice control interface, do you feel that
this technology would in any way enhance or extend your ability to safely operating the
car?
1
2
Not at All
3
4
5
6
7
8
9
10
A Lot
17. Given time to develop enough familiarity with the voice control interface, do you feel that
this technology would in any way detract from or reduce your ability to safely operating
the car?
1
2
Not at All
3
4
5
6
7
8
9
10
A Lot
18. If you purchased a vehicle with the voice control system you tried today, how satisfied
would you be with the operation of this system?
1
Not
Satisfied
2
3
4
5
6
7
8
9
10
Very
Satisfied
19. If you had a vehicle with the voice control technology you used today, how likely is it that
you would use it?
1
Not at All
Likely
2
3
4
5
6
7
8
9
10
Very
Likely
20. How likely is it that you would recommend to a friend or family member that they
consider buying a car with the voice control technology you used today?
1
Not at All
Likely
2
3
4
5
6
7
8
9
10
Very
Likely
21. If the limitations of the voice control system you worked with today were solved, to what
extent would you like to have a voice command interface in your next car?
1
2
Not at All
©MIT AgeLab 2013
3
4
5
6
7
8
9
10
A Lot
Page A273 of 293
22. What did you like most about the voice command interface?
______________________________________________________
______________________________________________________
______________________________________________________
23. What did you like least about the voice command interface?
______________________________________________________
______________________________________________________
______________________________________________________
24. Do you have any suggestions for the designers who created the technologies you worked
with today on changes that would make the systems easier to use, understand, or otherwise
improve the overall experience?
a. No
b. Yes
(If yes, please explain below.)
____________________________________________________
____________________________________________________
____________________________________________________
____________________________________________________
____________________________________________________
____________________________________________________
____________________________________________________
____________________________________________________
____________________________________________________
____________________________________________________
25. Please use the space below or the back of this page to write any comments or suggestions
you have regarding your experiences today:
____________________________________________________
____________________________________________________
____________________________________________________
____________________________________________________
____________________________________________________
©MIT AgeLab 2013
Page A274 of 293
Appendix K: Questionnaire Data
The following section presents descriptive statistics for the questionnaire data collected from
study participants. In the following tables, a cell reading “0.7 (0.5) [0 – 1]” would correspond
to a group mean of 0.7, a standard deviation of 0.5, and a collected response range between 0
and 1. In addition, the final section includes the verbatim comments provided by each
individual participant to several open-ended questions. See Appendix J for the complete text
and scales presented in the questionnaires.
Q.
Young Female
Young
Male
Older
Female
Older
Male
3
Physical well-being
(0= excellent, 4= poor)
0.7 (0.5)
[0 - 1]
0.5 (0.5)
[0 - 1]
0.3 (0.5)
[0 - 1]
0.5 (0.5)
[0 - 1]
4
Mental well-being
0.4 (0.5)
[0 - 1]
0.5 (0.5)
[0 - 1]
0.4 (0.6)
[0 - 2]
0.3 (0.5)
[0 - 1]
5
Sick right now
(0= yes, 1= no)
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
6
Uncomfortable now
(0= yes, 1= no)
0.9 (0.4)
[0 - 1]
0.9 (0.3)
[0 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
7
Wear corrective lenses
(0= no, 1= glasses, 2= contacts)
0.8 (1.0)
[0 - 2]
0.3 (0.7)
[0 - 2]
1.0 (0.8)
[0 - 3]
0.3 (0.6)
[0 - 2]
8
Self-rate overall vision
0.8 (0.7)
[0 - 2]
0.4 (0.5)
[0 - 1]
0.9 (0.6)
[0 - 2]
0.6 (0.6)
[0 - 2]
9
Self-rate overall hearing
0.7 (0.6)
[0 - 2]
0.5 (0.5)
[0 - 1]
0.7 (0.7)
[0 - 2]
1.1 (0.6)
[0 - 2]
10
Wear hearing aid
(0= yes, 1= no)
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
Technology Experience
(1= inexp., 10= experienced)
8.1 (1.2)
[6 - 10]
9.1 (0.8)
[8 - 10]
7.9 (1.2)
[5 - 10]
7.9 (2.0)
[4 - 10]
12
Adopt Technology
(1= avoid, 10= try)
7.2 (1.6)
[5 - 10]
8.1 (1.8)
[4 - 10]
6.9 (1.4)
[5 - 10]
6.9 (2.0)
[3 - 9]
13
Trust Technology
(1= distrust, 10= trust)
7.7 (1.6)
[4 - 10]
8.3 (1.4)
[5 - 10]
8.5 (0.9)
[7 - 10]
8.5 (1.1)
[6 - 10]
14
Trust Established Tech
9.1 (1.1)
[7 - 10]
9.3 (1.0)
[7 - 10]
8.8 (0.9)
[7 - 10]
9.6 (0.8)
[7 - 10]
11
©MIT AgeLab 2013
Page A275 of 293
Q.
Young Female
Young
Male
Older
Female
Older
Male
15
Trust New Car Tech
7.7 (1.3)
[6 - 10]
8.1 (1.2)
[5 - 10]
7.7 (1.0)
[6 - 9]
8.1 (1.1)
[6 - 10]
16
Ability Learn New Tech
(1= poor, 10= good)
8.3 (1.1)
[6 - 10]
8.7 (1.5)
[6 - 10]
8.2 (1.1)
[7 - 10]
8.8 (1.1)
[7 - 10]
17
Use Electric. Navigation
(0= daily, 5= never)
2.1 (1.0)
[0 - 4]
2.3 (1.0)
[0 - 4]
3.9 (1.1)
[2 - 5]
3.7 (1.1)
[2 - 5]
18
Voice Command Interface
3.5 (1.7)
[0 - 5]
3.3 (1.3)
[0 - 5]
4.3 (1.3)
[1 - 5]
4.1 (1.1)
[2 - 5]
19
Car Voice Interface
4.8 (0.6)
[3 - 5]
4.4 (1.4)
[0 - 5]
5.0 (0.0)
[5 - 5]
4.7 (0.6)
[3 - 5]
20
Touch Screen Use
1.5 (1.8)
[0 - 5]
0.7 (1.2)
[0 - 4]
2.3 (2.0)
[0 - 5]
2.1 (1.8)
[0 - 5]
21
Phone Call Driving
1.5 (1.1)
[0 - 3]
2.3 (1.4)
[0 - 5]
2.2 (2.0)
[0 - 5]
3.3 (1.4)
[0 - 5]
22
Voice Dialing Driving
4.7 (1.0)
[1 - 5]
4.7 (0.6)
[3 - 5]
4.3 (1.5)
[0 - 5]
4.7 (0.7)
[3 - 5]
23
Answer Calls Driving
1.6 (1.1)
[0 - 3]
2.0 (1.6)
[0 - 5]
2.5 (1.7)
[0 - 5]
2.9 (1.1)
[1 - 5]
24
Valid License
0.0 (0.0)
[0 - 0]
0.0 (0.0)
[0 - 0]
0.0 (0.0)
[0 - 0]
0.0 (0.0)
[0 - 0]
25
How Often Drive
0.2 (0.4)
[0 - 1]
0.6 (0.5)
[0 - 1]
0.1 (0.4)
[0 - 1]
0.2 (0.4)
[0 - 1]
26
Miles Driven
3.0 (1.6)
[1 - 7]
2.8 (2.5)
[0 - 7]
3.3 (1.6)
[1 - 7]
2.5 (0.9)
[1 - 4]
8.5 (1.2)
[6 - 10]
8.7 (1.0)
[7 - 10]
8.9 (0.8)
[8 - 10]
9.1 (0.9)
[7 - 10]
27
Safe Driver
(1=very unsafe, 10=very safe)
28
# Police Warning Past 5 yrs
0.0 (0.0)
[0 - 2]
0.0 (0.0)
[0 - 2]
0.3 (0.6)
[0 - 2]
0.6 (0.7)
[0 - 2]
29
# Citation or Ticket Past 5
Yrs
0.0 (0.0)
[0 - 1]
0.0 (0.0)
[0 - 5]
0.2 (0.4)
[0 - 1]
0.5 (0.7)
[0 - 2]
30
# Vehicle Crash Past 5 Yrs.
0.0 (0.0)
[0 - 1]
0.0 (0.0)
[0 - 1]
0.2 (0.4)
[0 - 1]
0.3 (0.6)
[0 - 2]
31-a
Pass Signaling Car
2.7 (1.4)
[0 - 5]
1.4 (1.0)
[0 - 3]
1.1 (1.3)
[0 - 3]
0.9 (1.4)
[0 - 4]
31-b
Wrong Turn Lane
1.6 (0.7)
[0 - 3]
1.3 (0.9)
[0 - 3]
1.0 (0.8)
[0 - 2]
1.1 (0.7)
[0 - 2]
(1=Never, 10= Nearly all the time)
©MIT AgeLab 2013
Page A276 of 293
Q.
Young Female
Young
Male
Older
Female
Older
Male
0.5 (0.5)
[0 - 1]
0.3 (0.5)
[0 - 1]
0.3 (0.6)
[0 - 2]
0.3 (0.6)
[0 - 2]
1.5 (0.9)
[0 - 3]
1.2 (0.6)
[0 - 2]
1.1 (0.7)
[0 - 2]
1.3 (0.5)
[1 - 2]
0.9 (0.7)
[0 - 2]
0.4 (0.6)
[0 - 2]
0.9 (0.5)
[0 - 2]
0.7 (0.8)
[0 - 2]
0.9 (1.1)
[0 - 3]
0.7 (1.0)
[0 - 3]
0.3 (0.5)
[0 - 1]
0.8 (0.8)
[0 - 2]
31-c
Fail to Stop or Yield
31-d
Misread Signs/Exits
31-e
Fail to notice Pedestrians
31-f
Tailgate
31-g
Forget Parking Spot
1.5 (1.1)
[0 - 4]
1.1 (1.0)
[0 - 3]
0.9 (0.8)
[0 - 2]
0.9 (0.5)
[0 - 2]
31-h
Side onto Main Road
Distraction (1= Never, - 10)
0.6 (0.6)
[0 - 2]
0.3 (0.5)
[0 - 1]
0.5 (0.5)
[0 - 1]
0.9 (0.6)
[0 - 2]
31-i
Back up into Something
0.3 (0.6)
[0 - 2]
0.3 (0.5)
[0 - 1]
0.5 (0.6)
[0 - 2]
0.7 (0.5)
[0 - 1]
31-j
Go through Yellow
2.8 (1.3)
[1 - 5]
1.7 (1.2)
[0 - 5]
1.8 (1.0)
[1 - 5]
1.8 (0.8)
[1 - 3]
31-k
Almost hit Pedestrian on
right turn (1= Never, - 10)
0.7 (0.6)
[0 - 2]
0.2 (0.4)
[0 - 1]
0.4 (0.5)
[0 - 1]
0.7 (0.6)
[0 - 2]
31-l
Ignore Speed Limit Off
Hours (1= Never, - 10)
2.1 (1.2)
[0 - 4]
1.5 (1.5)
[0 - 4]
1.0 (0.9)
[0 - 2]
1.5 (1.0)
[0 - 3]
31-m
Forget High Beams are on
0.5 (0.9)
[0 - 3]
0.7 (0.6)
[0 - 2]
0.6 (0.5)
[0 - 1]
0.7 (0.6)
[0 - 2]
31-n
Fail to check rear view
changing lane (1= Never, - 10)
0.8 (0.8)
[0 - 2]
0.4 (0.7)
[0 - 2]
0.5 (0.5)
[0 - 1]
0.6 (0.6)
[0 - 2]
31-o
Indicate dislike of type of
driver (1= Never, - 10)
0.9 (1.1)
[0 - 4]
0.4 (0.6)
[0 - 2]
0.3 (0.6)
[0 - 2]
0.6 (0.8)
[0 - 2]
31-p
Pass on right
2.0 (1.4)
[0 - 4]
1.3 (1.0)
[0 - 3]
1.5 (0.8)
[0 - 3]
2.0 (1.1)
[0 - 5]
31-q
Underestimate speed when
passing (1= Never, - 10)
0.7 (0.7)
[0 - 2]
0.4 (0.7)
[0 - 2]
0.5 (0.5)
[0 - 1]
0.8 (0.6)
[0 - 2]
31-r
Switch on wrong thing
0.7 (0.8)
[0 - 2]
0.5 (0.6)
[0 - 2]
0.6 (0.8)
[0 - 2]
0.8 (0.6)
[0 - 2]
31-s
Turn away from skid
0.4 (0.5)
[0 - 1]
0.3 (0.5)
[0 - 1]
0.3 (0.5)
[0 - 1]
0.6 (0.6)
[0 - 2]
31-t
Zone out, go toward wrong
destination (1= Never, - 10)
0.6 (0.6)
[0 - 2]
0.5 (0.6)
[0 - 2]
0.8 (0.8)
[0 - 2]
0.7 (0.6)
[0 - 2]
©MIT AgeLab 2013
Page A277 of 293
Q.
Young Female
Young
Male
Older
Female
Older
Male
0.3 (0.6)
[0 - 2]
0.1 (0.3)
[0 - 1]
0.1 (0.3)
[0 - 1]
0.3 (0.5)
[0 - 1]
31-u
Drive above alcohol limit
31-v
Race other drivers
0.0 (0.0)
[0 - 0]
0.1 (0.3)
[0 - 1]
0.1 (0.3)
[0 - 1]
0.1 (0.4)
[0 - 1]
31-w
Can't remember previous
road (1= Never, - 10)
0.1 (0.4)
[0 - 1]
0.3 (0.6)
[0 - 2]
0.2 (0.4)
[0 - 1]
0.5 (0.7)
[0 - 2]
31-x
Chase Driver
0.1 (0.3)
[0 - 1]
0.8 (0.4)
0.0 (0.0)
[0 - 0]
0.4 (0.3)
[0.2 - 1.6]
0.9 (0.4)
[0.0 - 0.9]
0.7 (0.4)
[0.4 - 1.9]
1.1 (0.6)
[0.0 - 1.4]
0.7 (0.5)
[0.4 - 2.4]
1.0 (0.4)
[0.0 - 1.9]
0.6 (0.3)
[0.4 - 2.0]
3.5 (1.1)
[2 - 5]
[0.0 - 1.0]
2.8 (1.1)
[2 - 5]
0.1 (0.3)
[0 - 1]
0.5 (0.3)
[0.1 - 1.1]
0.7 (0.3)
[0.1 - 1.3]
0.6 (0.3)
[0.1 - 1.3]
0.6 (0.3)
[0.3 - 1.2]
3.8 (1.3)
[2 - 5]
0.1 (0.3)
[0 - 1]
0.6 (0.4)
[0.0 - 1.4]
0.8 (0.3)
[0.4 - 1.3]
0.9 (0.3)
[0.3 - 1.4]
0.8 (0.3)
[0.3 - 1.4]
3.7 (1.5)
[1 - 5]
31_Error
Q31_Error
31_Lapse
Q31_Lapse
31_Violation Q31_Violation
31_Total
Q31_Total
32
Education
33
Current Status
2.6 (2.1)
[0 - 5]
2.1 (2.2)
[0 - 5]
3.3 (1.2)
[1 - 5]
3.4 (0.7)
[3 - 5]
34
Marital Status
3.7 (0.5)
[3 - 4]
3.9 (0.4)
[3 - 4]
1.4 (1.7)
[0 - 4]
0.4 (1.1)
[0 - 4]
35
Income
3.5 (2.7)
[0 - 7]
2.1 (2.7)
[0 - 7]
2.5 (1.5)
[1 - 5]
2.9 (1.9)
[1 - 6]
36
Awake
2.7 (1.2)
[1 - 5]
2.3 (1.3)
[1 - 6]
2.0 (1.8)
[1 - 8]
2.2 (2.0)
[1 - 9]
37
Stress
3.1 (1.8)
[1 - 7]
2.1 (1.0)
[1 - 5]
2.1 (1.1)
[1 - 5]
2.0 (0.8)
[1 - 3]
(0 – 5)
(1=very awake, 10 Extremely drowsy)
(1=very awake, 10 Extremely drowsy)
Note: Q1-DoB; Q2-Gender.
©MIT AgeLab 2013
Page A278 of 293
Q.
1
Awake
(1=very awake, 10 =Extremely
drowsy)
2
Stress
(1=not stressed at all,
10=verystressed)
Young Female
Young Male
Older Female
Older Male
3.7 (1.8)
[1 - 7]
3.7 (2.1)
[1 - 7]
2.3 (1.3)
[1 - 6]
2.5 (1.6)
[1 - 7]
2.5 (1.2)
[1 - 5]
1.8 (0.4)
[1 - 2]
2.5 (1.5)
[1 - 6]
2.9 (1.6)
[1 - 6]
1-a
RadioM Intuitive
(1=not at all, 10=very)
9.0 (0.7)
[8 - 10]
9.0 (1.0)
[7 - 10]
7.4 (2.4)
[1 - 10]
7.5 (2.6)
[1 - 10]
1-b
RadioM Interface
2.3 (2.4)
[1 - 9]
2.7 (2.4)
[1 - 8]
2.9 (2.3)
[1 - 8]
3.9 (3.4)
[1 - 10]
1-c
RadioM Frustration
2.2 (1.4)
[1 - 5]
2.2 (1.6)
[1 - 7]
3.5 (2.5)
[1 - 9]
2.9 (2.7)
[1 - 10]
1-d
RadioM Distraction
3.9 (1.4)
[1 - 7]
3.7 (1.9)
[1 - 7]
3.0 (1.8)
[1 - 8]
3.9 (1.9)
[1 - 7]
1-e
RadioM Comparitive
2.6 (1.5)
[1 - 6]
3.2 (2.2)
[1 - 8]
3.4 (2.8)
[1 - 9]
3.5 (2.6)
[1 - 10]
2-a
RadioV Intuitive
8.5 (1.4)
[5 - 10]
8.1 (1.9)
[3 - 10]
6.6 (2.7)
[2 - 10]
6.7 (2.5)
[2 - 10]
2-b
RadioV Interface
2.9 (2.8)
[1 - 10]
3.9 (2.8)
[1 - 10]
4.4 (2.7)
[1 - 9]
4.6 (3.0)
[1 - 10]
2-c
RadioV Frustration
2.6 (1.4)
[1 - 5]
2.9 (2.1)
[1 - 8]
4.0 (2.7)
[1 - 10]
3.9 (3.1)
[1 - 10]
2-d
RadioV Distraction
2.8 (1.8)
[1 - 7]
2.6 (1.6)
[1 - 7]
3.9 (2.4)
[1 - 10]
3.1 (1.9)
[1 - 7]
2-e
RadioV Comparative
2.9 (1.6)
[1 - 6]
2.9 (2.1)
[1 - 7]
4.0 (2.7)
[1 - 10]
2.9 (1.8)
[1 - 6]
3-a
Song Intuitive
8.1 (1.9)
[4 - 10]
8.4 (1.5)
[5 - 10]
7.4 (2.5)
[3 - 10]
6.9 (2.6)
[1 - 10]
3-b
Song Interface
4.4 (3.0)
[1 - 10]
3.2 (2.2)
[1 - 8]
4.3 (3.1)
[1 - 10]
3.7 (2.1)
[1 - 8]
3-c
Song Frustration
3.3 (2.1)
[1 - 7]
3.1 (2.0)
[1 - 7]
3.9 (2.4)
[1 - 8]
5.1 (2.7)
[1 - 8]
3-d
Song Distraction
3.0 (1.5)
[1 - 6]
2.3 (1.3)
[1 - 5]
3.6 (2.5)
[1 - 10]
3.8 (2.4)
[1 - 8]
3-e
Song Comparative
2.8 (1.8)
[1 - 6]
2.9 (2.0)
[1 - 7]
3.5 (2.4)
[1 - 8]
3.3 (2.3)
[1 - 7]
4-a
Nav Intuitive
8.1 (1.4)
[5 - 10]
9.1 (0.9)
[8 - 10]
6.7 (2.1)
[3 - 10]
7.2 (2.2)
[3 - 10]
©MIT AgeLab 2013
Page A279 of 293
Q.
Young Female
Young Male
Older Female
Older Male
4-b
Nav Interface
2.9 (2.1)
[1 - 7]
2.5 (1.7)
[1 - 7]
3.7 (2.3)
[1 - 8]
3.9 (2.6)
[1 - 10]
4-c
Nav Frustration
3.7 (2.3)
[1 - 8]
2.7 (1.9)
[1 - 8]
3.1 (2.1)
[1 - 8]
4.2 (2.6)
[1 - 10]
4-d
Nav Distraction
3.1 (1.6)
[1 - 6]
3.1 (2.0)
[1 - 7]
3.5 (2.4)
[1 - 9]
3.5 (1.8)
[1 - 6]
4-e
Nav Comparative
2.9 (1.9)
[1 - 6]
3.4 (2.5)
[1 - 9]
3.5 (2.4)
[1 - 10]
3.9 (2.8)
[1 - 10]
5-a
Dialing Intuitive
8.1 (1.7)
[5 - 10]
8.9 (1.1)
[6 - 10]
7.5 (2.0)
[3 - 10]
6.9 (2.6)
[1 - 10]
5-b
Dialing Interface
3.1 (3.1)
[1 - 10]
2.5 (2.2)
[1 - 10]
3.6 (2.5)
[1 - 8]
2.6 (1.6)
[1 - 6]
5-c
Dialing Frustration
2.1 (1.4)
[1 - 5]
2.0 (0.8)
[1 - 3]
3.3 (2.1)
[1 - 8]
2.6 (1.7)
[1 - 6]
5-d
Dialing Distraction
3.1 (1.6)
[1 - 6]
2.1 (0.9)
[1 - 4]
3.1 (1.8)
[1 - 7]
2.8 (1.6)
[1 - 7]
5-e
Dialing Comparative
2.6 (1.3)
[1 - 5]
2.3 (1.3)
[1 - 5]
3.5 (2.2)
[1 - 8]
2.4 (1.3)
[1 - 5]
6-a
Message Difficulty
5.8 (3.2)
[1 - 10]
6.3 (3.5)
[1 - 10]
6.3 (2.9)
[2 - 10]
6.5 (3.4)
[2 - 10]
6-b
Message Frustration
4.7 (2.8)
[1 - 10]
5.7 (3.0)
[1 - 10]
6.1 (3.2)
[2 - 10]
5.7 (3.5)
[1 - 10]
6-c
Message Distraction
4.1 (2.4)
[1 - 10]
4.3 (2.4)
[1 - 8]
3.7 (2.3)
[1 - 9]
3.8 (2.4)
[1 - 8]
6-d
Message Comparative
4.3 (2.6)
[1 - 10]
4.7 (2.4)
[1 - 9]
4.7 (2.9)
[1 - 10]
5.3 (3.5)
[1 - 10]
7
One Message Harder
6.8 (2.9)
[1 - 10]
7.3 (3.3)
[1 - 10]
8.3 (2.0)
[4 - 10]
6.2 (3.4)
[1 - 10]
©MIT AgeLab 2013
Page A280 of 293
Post-Experiment Questionnaire I: Q1-Q21
Q.
Young Female Young Male Older Female Older Male
4.1 (2.0)
[1 - 7]
3.3 (2.2)
[1 - 7]
3.2 (2.2)
[1 - 8]
2.8 (1.5)
[1 - 6]
2.2 (1.0)
[1 - 4]
1.9 (0.8)
[1 - 3]
2.1 (1.5)
[1 - 7]
2.3 (1.6)
[1 - 6]
(1=not at all positive, 10=very positive)
8.3 (1.1)
[7 - 10]
7.9 (1.4)
[5 - 10]
7.3 (2.8)
[2 - 10]
7.3 (2.3)
[3 - 10]
4
Voice Control
(1=not very easy, 10= very easy)
7.5 (1.2)
[5 - 9]
7.4 (1.5)
[4 - 10]
6.7 (2.3)
[2 - 10]
6.2 (2.3)
[2 - 9]
5
Voice Prompts
(1=not very easy, 10= very easy)
8.1 (1.3)
[6 - 10]
9.0 (1.3)
[7 - 10]
7.1 (2.5)
[1 - 10]
6.8 (2.0)
[3 - 10]
6
Verbal Commands
(1=not at all, 10=a lot)
5.3 (2.4)
[1 - 9]
3.4 (1.8)
[1 - 7]
4.7 (2.6)
[1 - 9]
4.3 (2.3)
[1 - 8]
7
Feel about technology
2.7 (0.7)
[2 - 4]
2.5 (0.8)
[1 - 4]
2.9 (1.0)
[1 - 4]
2.2 (1.2)
[0 - 4]
8
Trust in technology
2.4 (0.5)
[2 - 3]
2.5 (0.7)
[2 - 4]
2.9 (1.0)
[1 - 4]
2.4 (1.0)
[0 - 4]
9
Preparation by intro
(1=not very well, 10=very well)
8.6 (1.3)
[5 - 10]
8.9 (1.2)
[7 - 10]
8.2 (1.9)
[3 - 10]
7.9 (1.3)
[5 - 10]
0.5 (0.7)
[0 - 2]
1.1 (0.8)
[0 - 3]
0.3 (0.5)
[0 - 1]
1.1 (1.1)
[0 - 4]
7.1 (1.6)
[5 - 10]
6.1 (1.5)
[5 - 10]
7.6 (1.9)
[5 - 10]
6.5 (1.6)
[5 - 10]
1.0 (0.5)
[0 - 2]
1.0 (0.4)
[0 - 2]
1.3 (1.0)
[0 - 3]
1.4 (0.9)
[0 - 3]
1
Awake/drowsy
2
Stress
3
Vehicle Impression
(1=very awake, 10 =Extremely drowsy)
(1=not stressed at all, 10=very stressed)
10 More Experience?
11
Ability to learn new technology
(1=deceased, 10=increased)
12 Voice Control safety
13
Additional Displays
(1=not helpful, 10=very helpful)
7.1 (2.8)
[1 - 10]
7.3 (2.2)
[2 - 10]
6.4 (3.1)
[1 - 10]
6.5 (2.4)
[2 - 10]
14
Eyes on road + Screen
(1=eyes on road, 10=eyes on screen)
4.5 (2.0)
[2 - 8]
3.7 (1.5)
[2 - 8]
4.1 (1.9)
[1 - 7]
4.0 (1.9)
[2 - 8]
15
Absorbed in interaction
4.3 (2.0)
[2 - 7]
3.8 (2.1)
[1 - 7]
3.1 (1.8)
[1 - 7]
3.7 (1.7)
[1 - 6]
16
Enhance of Extend Safety
7.9 (1.4)
[6 - 10]
7.4 (2.1)
[3 - 10]
7.3 (1.8)
[5 - 10]
7.1 (2.7)
[1 - 10]
17
Detract from or reduce Safety
2.7 (1.4)
[1 - 6]
2.3 (1.5)
[1 - 6]
2.4 (1.2)
[1 - 5]
2.8 (1.8)
[1 - 6]
©MIT AgeLab 2013
Page A281 of 293
Q.
Young Female Young Male Older Female Older Male
18
Satisfied with System
(1=not satisfied, 10=very satisfied)
7.7 (1.7)
[3 - 10]
7.6 (1.5)
[5 - 10]
7.0 (2.6)
[2 - 10]
5.7 (3.0)
[1 - 10]
19
Would you use voice technology
(1=not at all likely, 10=very likely)
8.3 (1.7)
[4 - 10]
7.6 (2.5)
[2 - 10]
7.1 (2.5)
[2 - 10]
6.9 (3.5)
[1 - 10]
20
Recommend to friend
(1=not at all likely, 10=very likely)
7.8 (1.6)
[5 - 10]
5.9 (2.6)
[1 - 10]
5.7 (3.0)
[1 - 10]
6.5 (3.1)
[1 - 10]
21
If limitations solved
(1=not at all, 10=a lot)
8.9 (1.1)
[7 - 10]
8.7 (1.2)
[6 - 10]
7.8 (2.4)
[2 - 10]
7.9 (2.5)
[1 - 10]
.
©MIT AgeLab 2013
Page A282 of 293
Post-Experiment Questionnaire II: Q22-Q25
The following are the verbatim written responses to four open-ended questions at the end of the
questionnaire that was completed back at the laboratory following the completion of the drive.
The specific questions were:
22. What did you like most about the voice command interface?
23. What did you like least about the voice command interface?
24. Do you have any suggestions for the designers who created the technologies you worked with today
on changes that would make the systems easier to use, understand, or otherwise improve the overall
experience?
25. Please use the space below or the back of this page to write any comments or suggestions you have
regarding your experiences today:
6
67
7
24
8
69
9
67
15
24
#25 Comments on
Experience
The trip was rather long to
use one command instead of obtain the info. A shorter trip
I could use the
needing multiple commands to with the same tasks could be
phone hands free
it did not understand phone, hear radio, hear USB accomplished in much less
M while driving
some words
device, etc.
time.
condense certain tasks into
the number of times I one (e.g. instead of saying
needed to confirm
"USB" and then "play artist
commands/how many Johnny Cash," I wish I could
steps certain tasks
just say "USB Play Johnny
M making phone calls required
Cash"
badly designed:
commands obtuse e.g.
destination address
wrong versus
Badly designed: commands
destination street
obtuse e.g. destination address
address. Too many
wrong versus destination street
layers (? Couldn't
address. Too many layers (? don't have to explain
more able to look read handwriting)
Couldn't read handwriting)
everything, e.g. the basics of
M at road
bad screen de
bad screen de
using the radio
neutral tone,
straightforward
having to verify each
F commands
command
Some commands (e.g.
"Destination street
address") did not feel Get rid of the extra step of
the navigation was intuitive to me.
"USB" when changing an
efficient and would Having to "enter" a
artist. When a user is in USB, Good! Thank you! Perhaps a
make entering an mode (e.g. phone,
he/she will likely make many reminder earlier the day
address into a GPS USB) seemed like a new commands to change
before would have helped
much safer while time-consuming extra songs while remaining in the when explaining the extra
F driving
step.
mode.
time it would take.
17
23
M the hands free
ID
Age M/F
#22 Like Most
©MIT AgeLab 2013
#23 Like Least
the follow up yes/no
#24 Suggestions for Designers
less follow up yes/no
thanks for the money
Page A283 of 293
ID
18
Age M/F
60
#22 Like Most
#23 Like Least
operation and
questions
ability to keep eyes
on the road
clean & simple M easy to use
22
it allowed me to
keep my eyes on
the road more than
M normal
the music selection
M options
I thought it was
really useful for
some things, like
picking artists on
the USB, picking
radio stations that
weren't preset or
F calling contacts
25
24
very clear
especially when it
came to correcting
any mistakes as far
M as navigations
26
69
M convenience
27
22
20
26
21
61
24
amount of time to
execute a command
from start to finish
it uses very specific
commands and
sometimes had
difficulty
understanding me
having to confirm
each command
I wish I didn't forget to say
that the girl from south who
got robbed at state street was
working as a cook.
switch radio & temp controls
no suggestions as far as I can
think of, this study has
proven to be efficient for
drivers who can handle
simple tasks while driving, in
which save time and energy
I supposed with more
familiarity I would enjoy it
everything was explained
well and I was not
intimidated to ask either
research assistant questions
during the process
there isn't anything I
could say in regards
to what was the least
about the voice
command
29
29
30
22
31
68
F
©MIT AgeLab 2013
#25 Comments on
Experience
I thought it was more
difficult/time
consuming for easier
commands
my inability
I did not like the
I liked the ease of amount of
pressing a button
memorization
with prompted
required for a specific
F steps
task
response time was a
bit slower than I
the voice was clear would prefer; insisted
and its in
on confirming every
conjunction with
command every time;
manual controls
did not always read
rather than
my commands
M replacing them
accurately
I could focus more having to confirm
F on the road
every command
it allowed me to
confirmations especially for
the destination street address
portion
to be able to adapt voice
commands so that as a user &
the interface became more
familiar with one and other
that the commands & steps
could be shortened to take less
time
the interface needs to be more
conversational, such as "give
me directions to---" rather than
"destination street address."
also get rid of everything but
the final beep before voice
directions
Repeating and
improve on the core voice
recognition capability, that
way it will be able to operate
smoother and faster, creating a
more seamless experience for
the user
Nothing. Excluding the small
imperfections of the voice
command system, the
experience was a very
positive one.
it was as good as it could be
Page A284 of 293
ID
Age M/F
34
23
F
37
28
M
38
22
M
39
20
F
41
23
M
42
23
M
43
20
M
44
24
M
#22 Like Most
keep my heads on
the road
I liked that I could
choose a song by
name & the phone
was easy to use
that it understood
me
after becoming
familiar with it, it
did let me keep my
eyes on the road
while still
performing other
functions
allowed for more
eyes on the road
than on the screen
(hands on wheel
rather than buttons)
picks up commands
quite well
I didn't have to
look anything up. I
could just speak
what I wanted and
the car would play
it
it was relatively
fast to use and
input
ability to navigate
through processes
that would, in the
past, require my
eyes on the dash,
without taking my
eyes off the road
can navigate by
smaller commands
than I was
instructed
("destination street
address" breaks
into "Destination" -> "street address"
via screen
46
20
M
47
21
M overall it was
©MIT AgeLab 2013
#23 Like Least
waiting for sound
when to push media
button was sometimes
confusing. Story was
totally unintelligible
#25 Comments on
Experience
I didn't like having to
confirm everything
having to confirm
everything
the many steps it took
to do basic things,
like play a specific
song, would help to
allow user to create
detailed shortcuts
the many steps it took to do
basic things, like play a
specific song, would help to
allow user to create detailed
shortcuts
still required visual
and manual
interaction at some
points
asks you to confirm
yes or no too often
Where the touch
button was placed. I
had to keep looking
down to find it.
you have to use very
specific commands
nice testers
change where the talk button
is placed
more natural speech
recognition, less "steps" in
command listings
the pauses
Make buttons on dashboard
bigger, differently sized, or
Must stick to
farther apart. Put more touch
recognized
cues on steering wheel
phrases/commands. I buttons. Make commands
wish it could interpret faster to execute (skip "please
phrases better.
say a command")
too many
Liked the car but still was too
high and lacked headroom.
Also, start time is pretty
early! Thanks
System should ask for fewer
Page A285 of 293
ID
49
Age M/F
27
51
25
53
27
54
63
58
28
60
29
61
28
62
67
67
64
F
#22 Like Most
sound
it was pretty
simple. Most of the
problems I had
were due to me
overthinking. I did
like the way it
confirmed with me
before proceeding
forward.
#23 Like Least
confirmations
confirmations of commands.
Voice processing time could
be slightly improved.
When it did not
understand me. I'd say
a street, and it would
tell me to pick a
number to confirm the
street.
Don’t always ask if I am sure
about a command. The
the ease of making
technology seems strong
M a phone call
setting up navigation enough to guess
allows you to be
less distracted and Voice recognition
operate the vehicle atomization option.
F more safely
Ease of use.
the yes or no
clear instructions, questions were a bit
M push to talk button time-consuming
Too many steps (ex.
Phone --> call so and
so instead of just
has the possibility starting with 'call').
to decrease
Also, it had trouble
F distraction
understanding me
streamline commands
it could only be easier if it
were more similar to siri
easily understood the order of
technology, meaning engaging
my commands
commands can be
in conversation versus basic
F without repeating confusing
commands
I would allow for specific
the inconsistencies for names to be used, rather than
navigation always use contact numbers. I would also
"navigation" or
allow for either destination or
it was simple, not always use
navigation to be acceptable
F too complex
"Destination"
GPS commands
With more use it
would be easier to
use. Kept eyes on Voice recognition
M the road more.
needs improvement.
voice commands it those features that Screen is too far away for a
does things without required pressing a taller person, buttons on
having to push
screen are too small - poor
"button" on the
M buttons or look
screen or looking at combination. Always provide
©MIT AgeLab 2013
#25 Comments on
Experience
the experience was fun. One
minor safety flaw is the
length of the cables. I could
only move my arm so far
with the length the sensor
cables were. Even if they
were just 5 in. longer, I feel I
could have in a more safe
way, turned the wheel
I had fun!
had a positive experience
everyone was professional
and helpful and explained
everything completely
Positive. I would say that the
car dealer must do the
training by offering a …or
group gathering to go over
clearly (could not read
handwriting)
label cameras so when you
are calibrating at the start of
the drive, it is clearer where
the participant should be
looking
I'm glad to have helped
science :)
Further practice with the
system would be helpful,
some commands, navigation
commands, are similar but
Page A286 of 293
ID
Age M/F
#22 Like Most
away after one is
familiar with the
system and it is
trained for
#23 Like Least
the screen to make a
choice defeated the
whole purpose
F
It only allows a few
seconds to 'hear' your
I can keep looking command and if you
at the road as I am talk too long you have
talking. Don't have to start over again.
to look at the
This takes time away
dashboard so
from concentrations
much.
on driving.
21
F
the navigation was
much easier than
typing it in while
driving
70
69
hands free phone,
voice GPS, stored
M music access
74
66
M nothing
68
69
64
How to undo a
mistake. The prompts
afterwards weren't
clear.
radio controls harder
than manual
not intuitive,
annoying
what happens when
you forget to wait for
clear voice - easy the beep and say
to understand
"yes" too soon
hands on the wheel, voice cleaner
eyes on the road
commands
at every command point a way
to either back up to the
previous step or cancel
out/quit "cleanly." if the
system must present info
The screen could be at a better
angle to the driver to be able
to use it better. Right now it
faces forward, but it could
maybe turn to the left a little
for better access and
readability. I had to keep
straining my eyes to the right a
lot
All of the mistakes I made
were understandable. It cannot
exactly recognize
EVERYTHING everyone
says.
Simplify comments to
conversational parlance. "end
call" should be voice
command. (if I have to
manually end call, what's the
point…)
67
F
77
66
F
69
Music system should accept
compound commands. "USB
and artist" as one command.
Phone display should be
Too much repetition simplified: too much info
easy one button
of commands
displayed. Display system for
initiation, relatively required. Should be
the radio is too busy. I did not
good recognition of capable of compound like the combination of … and
M voice commands
commands
display controls f
No hands use is
great. With time I
think I would be
at times it did not
fine and really
understand what I
F enjoy it.
said.
the ability to make
a call + play from the fuzzy message
F USB drive
that was hard to hear
80
66
82
28
©MIT AgeLab 2013
Also it's hard to touch the
screen with long fingernails
so I had to use my knuckle a
lot. But my knuckle worked
okay. Also the radio dial
knob was really small and the
preset buttons were very very
small.
was a very efficient
experiment
2nd call - message garbled very frustrating, unnecessary
stress! [Note: second call
was intentionally “garbled”
to assess impact of poor call
quality.]
Offer an option for the amount
of time lapse between
commands. Once familiar with
tech., I might want to speed it
up
76
79
#25 Comments on
Experience
not exactly like command
and responses from other
Nav/Gps equipment. This is a
source of confusion
it was fun!! Thank you for
the experience
today's test experience was in
accordance with my
expectations of a test
environment with controlled
variables; a bit tedious but
necessary
3 hours on the road is a long
time. Shorter would be better
- it was a little tiring at the
end
Interesting experience
because I've never used voice
command before. I can see
Page A287 of 293
ID
83
84
Age M/F
64
24
F
F
86
60
F
87
67
F
88
23
F
89
65
F
90
67
91
68
94
24
95
62
#22 Like Most
fairly simple and
easy to become
accustomed to
Stored music
selection. Phone call by name or
stored ID
The ease at which
you can use it
#23 Like Least
slow - repetitive
when it
misunderstood what I
was trying to say
Too many steps
especially navigation.
Too much profilevoice recognition.
Should be improved.
commands could be shorter,
more like cell phone voice
commands
#25 Comments on
Experience
there are benefits
Find a better route - too
much congestion, followed
by a boring road. Then again,
I guess that info is input for
your study
, it was simple and painless
improve voice recognition,
sound quality,
remember to get a snack
streamline/improve navigation during stop - improve
commands
alertness
very frustrating to
have to listen to the
entire
prompt/commands
not having to type once you got the hang
in the address with of it & very
voice/language recognition
a GPS & it's speed "computer" sounding could be improved
the story line was so
garbled that I didn't
speaking into mic hear any of it [note:
is much safer than this refers to an
fumbling around
intentional test of the
for stations &
impact of poor phone
phone
connection.]
no problem
listening doing it
M another thing
voice command was
good
Somewhat redundant
commands especially
after becoming
familiar with system.
ease of use, ability Perhaps this just
M to correct mistakes during instructions.
Didn’t have to
reach to radio for
controls. Voice was Took WAY too long
pleasant and
to do anything. Had to
understood my
confirm everything I
M voice
said - very annoying
Always had to
confirm - too many
steps. I could just put
F
address in my GPS
©MIT AgeLab 2013
make it very easy like 1, 2, 3
Listening to instruction was
ok but doing it at my age was
(very) hard. Traffic was
good, just doing what was
said was hard.
this is a simple system for a
simple problem (eyes on the
road) that does not need to be
made complex
The concept of voice
command is very positive.
Ability to keep eyes on the
road is important when
driving
Make it quicker. Don't use
"say a command" - just a beep.
the voice in the vm with the
story began too softly so I
couldn't even get the street
address then the story was
Page A288 of 293
ID
#25 Comments on
Experience
told choppily rather than
smoothly
Very enlightening. Enjoyable
not having to locate
ride, comfortable vehicle.
various knobs &
took a while to
more prompts - "please wait," Find selection of music on
F buttons
become accustomed "speak now"
pre-recorded artists
It was interesting even
though stressful (because of
the unknown & learning new
things). Participants should
I don't use my cell
know in advance that
phone while
I don't use the phone
sunglasses cannot be worn.
driving, I'd
and hardly change the
Some people have transition
probably use the
radio while driving lenses that turn dark - then,
voice command
so doing those tasks
perhaps eye movement
F phone
was distracting
would no
voice activation button should
could keep eyes on commands were not be better differentiated from
M road
obvious
others
I have been doing youth
suicide for the past few years
and frequently crave the quiet
That it readily
the touch-end, that it
of my car and might be
identified what I
didn't always
the changing from steering
paying considerably more
was led (? Could
understand/hear my wheel to dial to screen was
for something I didn't really
F not read last word) response
confusing sometimes
need
System needs expanded
capability to learn what works
for user, e.g. for me a setup
worked by someone like an outline: I. A. 1. a.
able to do what I
else's idea of logic;
would have made sense. For
want to do without pronunciation; tone; someone else? Who knows
it was a long time to the
M looking down
pacing
until you ask them
bathroom stop for me
easy to understand, sometimes the system maybe somehow eliminate the
F hands free
didn't hear my "yes" "yes" all of the time
Age M/F
98
60
99
60
100
68
101
68
103
66
104
61
#22 Like Most
©MIT AgeLab 2013
#23 Like Least
and be off
Page A289 of 293
Voice Control Interface CSRC Final Report
Q.
Young
Female
Young
Male
Older
Female
0.6 (0.8)
[0 - 2]
0.7 (0.5)
[0 - 2]
Older
Male
1
Overall Health
0.6 (0.6)
[0 - 2]
2
Doctor Visits
1.7 (0.8)
[0 - 3]
1.3 (1.3)
[0 - 4]
1.9 (1.8)
[0 - 6]
0.7 (0.5)
[0 good]
2.4 (1.5)
[1 - 5]
3-a
Heart Attack
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
3-b
Angina
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
3-c
Irregular Heart Rhythm
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
3-d
High Blood Pressure
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.8 (0.5)
[0 - 1]
0.8 (0.4)
[0 - 1]
3-e
High Cholesterol
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.5 (0.5)
[0 - 1]
0.7 (0.5)
[0 - 1]
3-f
Diabetes
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
3-g
Thyroid Condition
0.9 (0.3)
[0 - 1]
1.0 (0.0)
[1 - 1]
0.7 (0.5)
[0 - 1]
1.0 (0.0)
[1 - 1]
3-h
Stroke or TIA
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
1.0 (0.0)
[1 - 1]
3-i
Arthritis, rheumatism
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
0.7 (0.5)
[0 - 1]
0.9 (0.3)
[0 - 1]
3-j
Fibromyalgia
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
3-k
Chronic Pain
0.9 (0.3)
[0 - 1]
0.9 (0.3)
[0 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
3-l
Parkinson's
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
3-m
Gastrointestinal problems
0.9 (0.3)
[0 - 1]
0.9 (0.3)
[0 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
3-n
Spinal surgery
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
3-o
Blood clot in leg or lung
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
©MIT AgeLab 2013
Page A290 of 293
Q.
Young
Female
Young
Male
Older
Female
Older
Male
3-p
Hip fracture
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
3-q
Hip or knee replacement
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.4)
[0 - 1]
8
Limited in turning head, neck or shoulders
1.2 (0.6)
[1 - 3]
1.1 (0.3)
[1 - 2]
1.8 (0.6)
[1 - 3]
2.9 (2.1)
[1 - 7]
9
Stiffness, pain, etc. affects using rearview or
side mirrors
1.1 (0.3)
[1 - 2]
1.2 (0.6)
[1 - 3]
1.5 (0.5)
[1 - 2]
3.4 (2.5)
[1 - 8]
face/jaw/head
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
10-b neck
0.9 (0.3)
[0 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.8 (0.4)
[0 - 1]
10-c
0.8 (0.4)
[0 - 1]
0.9 (0.3)
[0 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.4)
[0 - 1]
10-d low back
0.9 (0.4)
[0 - 1]
0.8 (0.4)
[0 - 1]
0.8 (0.4)
[0 - 1]
0.9 (0.4)
[0 - 1]
10-e
seat/buttock/hip region
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.8 (0.4)
[0 - 1]
0.9 (0.3)
[0 - 1]
10-f
legs
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
0.9 (0.3)
[0 - 1]
1.0 (0.0)
[1 - 1]
10-g feet/toes
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
0.9 (0.3)
[0 - 1]
10-h chest
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
10-i
abdominal or stomach region
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
10-j
arms
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
10-k hands/fingers
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
0.9 (0.4)
[0 - 1]
11-a
0.3 (0.5)
[0 - 1]
0.2 (0.4)
[0 - 1]
0.8 (0.5)
[0 - 1]
0.6 (0.5)
[0 - 1]
11-b General stiffness and discomfort
0.9 (0.4)
[0 - 1]
0.9 (0.3)
[0 - 1]
0.9 (0.3)
[0 - 1]
0.8 (0.4)
[0 - 1]
11-c
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
10-a
shoulders/upper back
Source of pain - does not apply
Chronic headaches
©MIT AgeLab 2013
Page A291 of 293
Q.
Young
Female
Young
Male
Older
Female
Older
Male
11-d Arthritis
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
0.8 (0.5)
[0 - 1]
0.7 (0.5)
[0 - 1]
11-e
Muscle pain
0.8 (0.4)
[0 - 1]
0.9 (0.3)
[0 - 1]
0.9 (0.3)
[0 - 1]
0.9 (0.4)
[0 - 1]
11-f
Nerve pain
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
0.9 (0.3)
[0 - 1]
11-g Fibromyalgia
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
11-h other
0.0 (0.0)
[1 - 1]
0.0 (0.0)
[ - 1]
0.0 (0.0)
[ - 1]
0.0 (0.0)
[0 - 1]
11-i
unknown
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
1.0 (0.0)
[1 - 1]
12
How intense is pain now?
0.7 (1.9)
[0 - 7]
0.8 (1.3)
[0 - 4]
0.3 (0.7)
[0 - 2]
0.7 (0.8)
[0 - 2]
14
How intense was pain on avg. last week?
0.5 (1.3)
[0 - 4]
1.0 (2.0)
[0 - 6]
0.5 (0.8)
[0 - 2]
0.9 (1.0)
[0 - 3]
15
How distressing is pain now?
0.2 (0.7)
[0 - 3]
0.3 (0.6)
[0 - 2]
0.3 (0.6)
[0 - 2]
0.4 (0.9)
[0 - 3]
16
How distressing was pain on avg. last
week?
0.1 (0.3)
[0 - 1]
0.5 (1.4)
[0 - 5]
0.5 (0.7)
[0 - 2]
0.9 (1.7)
[0 - 6]
17
How much does pain interfere w/ daily
activates?
0.1 (0.4)
[0 - 1]
0.3 (1.2)
[0 - 4]
0.3 (0.6)
[0 - 2]
1.1 (1.6)
[0 - 6]
18
How much does pain interfere w/ driving?
0.1 (0.4)
[0 - 1]
0.1 (0.3)
[0 - 1]
0.1 (0.3)
[0 - 1]
0.7 (1.8)
[0 - 7]
Notes:
1. Five participants did not fill in the Supplemental Health Questionnaire. They are participant 020
(young male, age 26), 029 (young male, age 29), 031 (older female, age 68), 076 (older female,
age 66), 099 (older female, age 60).
2. Only one participant (participant number 041, young male, age 23) reported for Q4A and Q7, and
no participant report on questions Q4 B-Q4D, Q5 – Q6; the results were therefore not shown in
the table. Participant 041 reported IBS and acid reflux for Q4A and Omeprazole and Benefiber
for Q7.
©MIT AgeLab 2013
Page A292 of 293
Appendix L: Misc. Information
Rank Ordering of Reasons for Excluding Participants
Table 1: Number of participants excluded from sample, and reasons for exclusion.
N
Reason for Exclusion
13
Unusable ECG (heart rate) recording
6
Participant did not pass cognitive screening (see additional details later in text)
6
Equipment failure (data acquisition equipment system crash (3) and hard drive
failure before data transfer to server (2); vehicle needed maintenance(2))
4
Formal pilot participants
3
Participant demonstrated questionable behavior in parking lot and withdrawn by
experimenter
2
System did not recognize participant’s voice (determined in parking lot before
drive)
2
Participants’ schedule prevented completion of study
2
Experimenter errors (1 protocol execution, 1 data copy error)
1
Did not meet screening criteria review during personal interview
1
Incontinence event during drive
1
Participant demonstrated unsafe behavior on road and withdrawn by experimenter
1
Unsafe conditions on road
42
Total Excluded
©MIT AgeLab 2013
Page A293 of 293

On Road – Simulator comparison

Transcription

Similar documents

Columbus KKC Flyer 2013 - Columbus Cocker Rescue!

Columbus KKC Flyer 2016_3

Michael Pachelo

Math Tier 2

flexible spending account enrollment form

post report

of Summer Swim - the City of Trenton, MI Web Site

Dear sports friends, We have the pleasure to invite you to our

PD16 David Congram - Institut canadien des actuaires

TROJAN