Measuring motor actions and psychophysiology for task

Measuring motor actions and
psychophysiology for task difficulty
estimation in human-robot interaction
Domen Novak, Matjaž Mihelj, Jaka Ziherl,
Andrej Olenšek, Marko Munih
Human-robot interaction
Measuring human psychology
ƒ most robots equipped with sensitive force
and movement sensors
ƒ much more difficult to measure the user’s
subjective feelings: stress, engagement…
ƒ possible approach:
combine motor actions and
psychophysiology
Psychophysiological measurements
ƒ measure how physiological processes
such as heart rate and skin conductance
are affected by psychological states
respiration
skin temperature
heart rate
skin conductance
Study goal
ƒ Our goal: combine measurements of
motor actions with psychophysiology to
obtain a more accurate estimate of the
user’s preferences
The scenario in action
7 difficulty levels: ball becomes progressively smaller and faster
Measurement protocol
ƒ user performs task for 2 minutes
ƒ forces, movements and psychophysiology
are measured
ƒ user asked if he/she would prefer easier
or harder task
ƒ difficulty changed, task continues
(total 12 minutes)
Classification
Skin conductance analysis
Feature extraction
Motor actions
task
performance
percentage of balls caught
mean HR
percentage of balls in basket
SDNN
difficulty level (1-7)
RMSSD
time since start
movement
sensors
heart rate
pNN50
mean absolute velocity
LF/HF ratio
mean absolute acceleration
HF power
mean frequency
of position signal
LF power
mean frequency
of velocity signal
skin conductance
response frequency
mean absolute force
force sensors
Psychophysiology
total work
mean frequency
of force signal
skin
conductance
mean skin conductance
response amplitude
final skin conductance level
skin
temperature
respiration
final temperature
mean respiratory rate
respiratory rate variability
Linear discriminant analysis
ƒ a statistical method that finds an linear
boundary between two classes in a
multidimensional space
Adaptive discriminant analysis
ƒ major inter-individual differences,
especially in psychophysiology
ƒ requirement: adaptation to the user
ƒ solution: adaptive discriminant analysis,
where the discriminant function is
recursively updated online using Kalman
filtering
The classification process
Results
ƒ measure of success: percentage of times
the system can estimate the user’s
preferences (prefer easier/ prefer harder)
ƒ participants:
24 healthy subjects
11 hemiparetic patients
ƒ leave-one-out crossvalidation
Results – healthy subjects
100
Accuracy rate (%)
90
80
nonadaptive
70
adaptive
60
50
motor actions
psychophysiology
both
Results – patients
100
Accuracy rate (%)
90
80
nonadaptive
70
adaptive
60
50
motor actions
psychophysiology
both
Conclusions
ƒ adaptive methods improve classification
of psychophysiological measurements
ƒ combination of motor actions and
psychophysiology offers small
improvement over only motor actions
ƒ potentially more applicable to physically
undemanding environments
Acknowledgements
Thank you for your attention
ƒ
Research work was supported by
ƒ
Research of the MIMICS project is funded through the
European Union under the 7th Framework programme
grant 215756
ƒ
MOOG FCS kindly loaned one of two HapticMaster devices
for MIMICS