Event Related Potentials in the Electroencephalogram for Human

Transcription

Event Related Potentials in the
Electroencephalogram for
Human-Robot Interaction
Svetlin Penkov
s1329991
NI VER
S
E
R
G
O F
H
Y
TH
IT
E
U
D I
U
N B
Master of Science by Research
Doctoral Training Centre in Neuroinformatics
and Computational Neuroscience
School of Informatics
University of Edinburgh
2014
Abstract
There is a great gap between the current state of robotics and the general population
expectations of what tasks an advanced robot should be able to perform. Even though
robots are irreplaceable factory workers, they are not able to fully autonomously perform even simple tasks in an uncontrolled human environment. One way to relax the
problem of autonomous behaviour is to rely on human-robot interaction, establishing
a semi-supervised scheme, where the human occasionally provides supervisory information to the robot. Such approach, however, requires an intuitive, robust and efficient
human-robot interface. The human brain constantly registers various events in the environment which causes certain patterns in the electromagnetic activity to emerge. This
thesis explores the detection and classification of event related potentials (ERPs) in
the electroencephalogram for human-robot interaction. Previous ERP research utilises
medical class EEG equipment which severely limits the mobility of the person wearing
it. We show that the Emotiv EPOC headset, which is a lightweight low-cost wireless
EEG capturing device, can detect ERPs evoked when an error is committed or observed
by the human. Furthermore, three types of classifiers (a discriminative, a generative
and a temporal one) were trained to recognise correct and incorrect events. It was
found out that the dimensionality of the EEG data could be significantly reduced and
that the discriminative classifier, which uses logistic regression, achieved the best performance compared to the other two. The evaluation of the Emotiv EPOC headset and
the reported classification results provide the basis for further research in the field of
human-robot interaction which will be conducted during the subsequent PhD.
iii
Acknowledgements
I would like to thank my supervisor Dr Subramanian Ramamoorthy who was shaping
my ideas when I did not know how to proceed. I would also like to thank Nantas
Nardelli and Alejandro Bordallo Micó for the numerous discussions. Last, but not
least, I would like to express my gratitude to all volunteers who participated in the
experiments.
iv
Declaration
I declare that this thesis was composed by myself, that the work contained herein is
my own except where explicitly stated otherwise in the text, and that this work has not
been submitted for any other degree or professional qualification except as specified.
(Svetlin Penkov
s1329991)
v
Table of Contents
1
2
Introduction
1
1.1
Motivation and Aim . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1.2
Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
Background
5
2.1
Neuroergonomics . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
2.1.1
Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
2.1.2
Motivation and research . . . . . . . . . . . . . . . . . . . .
6
2.1.3
Measuring brain activity . . . . . . . . . . . . . . . . . . . .
7
Electroencephalography (EEG) . . . . . . . . . . . . . . . . . . . . .
7
2.2.1
Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
2.2.2
Principles of operation . . . . . . . . . . . . . . . . . . . . .
8
2.2.3
Noise and artefacts . . . . . . . . . . . . . . . . . . . . . . .
9
2.2.4
Rhythmic activity . . . . . . . . . . . . . . . . . . . . . . . .
10
Event-related potentials (ERP) . . . . . . . . . . . . . . . . . . . . .
10
2.3.1
Readiness potential . . . . . . . . . . . . . . . . . . . . . . .
10
2.3.2
Contingent negative variation . . . . . . . . . . . . . . . . .
11
2.3.3
P300 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
2.3.4
N400 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
2.3.5
P600 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
2.3.6
Error related negativity (ERN) . . . . . . . . . . . . . . . . .
12
Brain-computer interfaces . . . . . . . . . . . . . . . . . . . . . . .
13
2.4.1
Spontaneous activity BCIs . . . . . . . . . . . . . . . . . . .
14
2.4.2
Event-related activity BCIs . . . . . . . . . . . . . . . . . . .
14
2.4.3
Human-robot interaction . . . . . . . . . . . . . . . . . . . .
14
2.2
2.3
2.4
vii
3
Methods
17
3.1
Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
3.1.1
Emotiv EEG EPOC headset . . . . . . . . . . . . . . . . . .
17
3.1.2
Experiment 1 - flanker task (FT) . . . . . . . . . . . . . . . .
18
3.1.3
Experiment 2 - robotic arm task (RAT) . . . . . . . . . . . .
20
ErrP Classification . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
3.2.1
Robust discriminant power (RDP) . . . . . . . . . . . . . . .
22
3.2.2
Dimensionality reduction - principal components analysis (PCA) 23
3.2.3
Discriminative classifier - logistic regression (LR) . . . . . .
24
3.2.4
Generative classifier - mixture of Gaussians (MOG) . . . . . .
24
3.2.5
Temporal classifier - hidden Markov model (HMM) . . . . . .
25
3.2
4
Observing error related negativity
27
4.1
EEG data preprocessing . . . . . . . . . . . . . . . . . . . . . . . . .
27
4.1.1
Extracting event centred EEG epochs . . . . . . . . . . . . .
27
4.1.2
Frequency filtering . . . . . . . . . . . . . . . . . . . . . . .
28
4.1.3
Analysis for eye movement artefacts . . . . . . . . . . . . . .
28
Mean ERPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
4.2.1
Mean response ErrP . . . . . . . . . . . . . . . . . . . . . .
31
4.2.2
Mean observation ErrP . . . . . . . . . . . . . . . . . . . . .
31
4.3
Significance of the data . . . . . . . . . . . . . . . . . . . . . . . . .
34
4.4
Evaluation of the Emotiv EPOC headset . . . . . . . . . . . . . . . .
34
4.2
5
6
ErrPs classification results
39
5.1
Channel selection . . . . . . . . . . . . . . . . . . . . . . . . . . . .
39
5.2
Dimensionality reduction . . . . . . . . . . . . . . . . . . . . . . . .
42
5.3
Logistic regression classifier . . . . . . . . . . . . . . . . . . . . . .
43
5.4
Mixture of Gaussians classifier . . . . . . . . . . . . . . . . . . . . .
46
5.5
Hidden Markov model classifier . . . . . . . . . . . . . . . . . . . .
46
5.6
Classifiers comparison . . . . . . . . . . . . . . . . . . . . . . . . .
51
Conclusion
53
6.1
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
53
6.2
Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
54
Bibliography
55
viii
Chapter 1
Introduction
1.1
Motivation and Aim
Robots have long been imagined as workers that work alongside humans, extending the
time an elderly person can live at home, providing physical assistance to a worker on
an assembly line, or helping with household chores [1]. Despite the rapid development
of technology, there is a significant mismatch between the current state of robotics and
the futuristic general public expectations.
Nowadays, robots are working relentlessly, performing tasks with unprecedented
strength, speed and accuracy in various factories where the environment is carefully
controlled. However, there are many issues related to the autonomous operation in
human environments where human-robot interaction should occur. First of all, human
environments vary a lot and tend to be complex. For example, imagine a typical kitchen
with all the furniture, equipment and utensils in it. There are millions of kitchens, but
most probably no two kitchens are absolutely the same. Secondly, other autonomous
actors are present who usually introduce dynamic variations in the environment, often
imposing real time constraints. Co-working with humans would require the abilities to
manipulate various objects including non-rigid ones.
The Fukushima Daiichi nuclear disaster clearly illustrated the need for advanced
rescue robots capable of acting in uncontrolled human environments. The Darpa
Robotics Challenge is an attempt to focus robotics research on those topics by challenging the robots to perform tasks such as climbing a ladder, opening a door and
removing debris [2].
Most of the current state of the art methods in robotics rely on supervised machine
learning techniques [1] which extract meaningful statistics from the incoming data
1
2
Chapter 1. Introduction
during the training stage. A key component is the supervisory signal which has to
be provided to the robot during the training. One possibility is to use labelled data,
but the amount of data required to learn a problem increases exponentially with its
complexity. Labelling large amounts of data is error prone, tedious and consumes
time. Another possibility is to have a robot and a human working together, establishing
a semi-supervised learning scheme, where the human actively interacts with the robot
providing necessary guidance.
Currently, human-robot collaboration occurs at the slave-master level, where the
safety of the human has the highest priority and the robot simply follows commands.
The robot could also react to the behaviour of the human by monitoring various behavioural characteristics including arm movements [3] and walking patterns [4]. However, more intuitive and efficient human-robot interaction methods are still required
[1].
Brain computer interfaces (BCI) have been successfully used to control simple
mobile robots moving in a plane [5, 6] and also quadcopters flying in 3D space [7].
BCIs rely on electroencephalographic (EEG) recordings of the electromagnetic brain
activity which constitutes of two components: spontaneous activity and event related
potentials (ERP) [8]. The ERPs are spatiotemporal patterns of brain activity which
correspond to various events such as anticipation of an action or a stimulus, violation
of the expected sensory input or perception of error [9]. Detection of such ERPs could
allow the robot to infer the intention of the human and so enabling a more intuitive and
efficient human-robot interaction.
There is a large amount of research on ERPs as they are of great interest to neuroscientists and cognitive neuroscientists. However, most of the experiments are conducted in controlled lab environments. The field of neuroergonomics attempts to bring
brain imaging methods to real-world environments in order to improve the interaction between a system and an operator and their overall performance. We believe that
such neuroergonomic approach is essential for the development of the next generation
intuitive and efficient human-robot interfaces.
The work presented here is a pilot study for a subsequent PhD in human-robot
interaction, which aims at exploring ERP detection and classification for the future
development of a human-robot interface such as the one shown in figure 1.1. The
human simply observes the robot and the working environment which causes various
ERPs to be generated by the human brain. Those ERPs are first recorded using an
EEG device and then detected and classified by performing suitable analysis of the
1.1. Motivation and Aim
3
Figure 1.1: Human-robot interface based on electromagnetic brain activity and the proposed cognitive control loop utilising event related potentials.
EEG data. The robot utilises this information in order to learn and take decisions
which hopefully lead to a more optimal behaviour. The altered behaviour evokes new
ERPs and the whole sequence is repeated forming a cognitive control loop based on
the electromagnetic brain activity of the operator which is modulated according to their
plan, action and preferences.
There are several important aspects of the work presented here. Firstly, a low-cost
and commercially available EEG recording device is used, namely the Emotiv EPOC
headset. It is a lightweight EEG device which allows wireless data acquisition from 14
channels making it suitable for real world environments and applications. One of the
aims of the project is to evaluate the Emotiv EPOC headset and determine whether it
really can be used for such scenarios. Secondly, we focus on the detection and classification of event related potentials which are evoked when an error is perceived, thus
forming a reward signal that can easily be fed into a reinforcement learning algorithm
later. Two types of error related event potentials are studied - when an erroneous action
is executed and when an error is simply observed. Last but not least, three types of classifiers (a discriminative, a generative and a temporal one) are developed to recognise
error related event potentials and their performance is analysed and compared.
4
1.2
Chapter 1. Introduction
Outline
Chapter 2
This chapter provides an introduction to the field of neuroergonomics describing
its scope and research methods. Electroencephalography, which is on of the
most often used brain imaging technique in the field, is described and issues
related to it are discussed. A concise overview of the most well known event
related potentials is also provided. The chapter finishes with a discussion on
brain-computer interfaces and human-robot interaction.
Chapter 3
This chapter describes the Emotiv EPOC headset used to collect EEG data. The
two experiments which were conducted are also explained. A short overview of
the mathematical tools utilised for the classification of the event related potentials is also provided.
Chapter 4
The chapter shows the analysis of the recorded EEG data, acquired during the
experiments. The steps for data preprocessing are described and the presence of
ocular movement artefacts is investigated. The obtained event related potentials
are reported and their statistical significance is examined. The chapter concludes
with an evaluation of the Emotiv EPOC headset.
Chapter 5
This chapter presents the channel selection procedure which was conducted and
the dimensionality analysis applied to the data. The results of training three types
of classifiers (a discriminative, a generative and a temporal one) to detect errors
from event related potentials are reported and their performance is compared.
Chapter 6
This chapter provides a a summary of the key points from this thesis. It concludes with a discussion on the presented work and points out potential future
directions for development during a PhD.
Chapter 2
Background
This chapter provides a short introduction to the field of neuroergonomics - describing
what it is and the current state of research within the field. Furthermore, the application
of neuroergonomics to brain-machine interfaces (BMI) and human-robot interaction is
discussed. One of the most widely used brain imaging methods in neuroergonomics,
which is the electroencephalography (EEG), is described and various issues related to
it are reviewed. The chapter also contains a non-exhaustive overview of the research
focused on event related potentials (ERPs) in EEG signals. The presented work is
based on ERPs related to errors (ErrPs), therefore the various types of ErrPs are also
explained. Finally, applications of ERPs to brain-computer interfaces are overviewed
and several key characteristics for intuitive, intelligent and efficient human-robot interfaces are outlined.
2.1
2.1.1
Neuroergonomics
Definition
Neuroergonomics is the study of the brain and behaviour in relation to performance
in natural everyday settings [10]. It is an interdisciplinary field which combines ergonomics and human neuroscience. Ergonomics is a discipline concentrated not only
on improving productivity in physical work environments, but it also studies the interaction between humans and various system components in order to ensure human
well-being and greater overall system performance [11]. Human neuroscience studies
the human nervous system and how it functions. Most of the studies rely on experiments conducted in controlled lab conditions which give very little information about
5
6
Chapter 2. Background
the tasks performed by the brain in natural environments. Often those experiments rely
on the fact that participants will be stationary, entirely focused on the task without any
distractions. However, the theory of embodied cognition clearly states that the cognitive capabilities of an agent depend on its ability to actively move and explore its
complex environment where many sources of distraction exist [12]. Neuroergonomics
addresses that issue by utilising brain imaging techniques and applying them to natural
environments which are extensively studied by ergonomists.
2.1.2
Motivation and research
The benefit of correlating brain activity with behaviour and performance during the
execution of real tasks is twofold. Firstly, more realistic models of the human brain
and cognition could be developed and validated. Secondly, new technologies emerge
which have critical impact and ensure the good overall system performance and safety.
Neuroadaptive systems are an excellent example, where measures inferred from brain
activity, such as fatigue in drivers [13] or the cognitive load of aircraft pilots [11], are
used to determine how much control the operator should have and how much autonomy
is granted to the system. If, for example, a pilot has to execute a cognitively demanding
task, the aircraft should autonomously resolve non-critical issues. On the other hand,
when the cognitive load of the pilot is low, those non-critical issues should be reported
to the pilot. Limiting the autonomy of the aircraft, when it is possible, would keep the
pilot focused and ensure optimal overall performance of the operator and the system.
Research in neuroergonomics also aids the development of brain-machine interfaces (BMI) which currently lack robustness and are mainly limited to operate in experimental conditions. As discussed in the previous chapter, a neuroergonomic approach to BMI is required in order to create intuitive, robust and efficient human-robot
interfaces. Having access to environment variables such as the mental state of the human would allow the robot to take more intelligent decisions and execute appropriate
actions. There is a large amount of research in the field of neuroergonomics focused on
decoding human intention [9]. The robot could use the intention of the human in order
to predict what it should do, rather than just simply react to the current situation. That
idea has resulted in an adaptive shared control paradigm for human-robot interaction
based on intention prediction from neural activity [5, 6].
2.2. Electroencephalography (EEG)
7
Figure 2.1: Brain imaging methods used in neuroergonomics compared in terms of
spatial resolution (x-axis), temporal resolution (y-axis) and degree of immobility (z-axis).
2.1.3
Measuring brain activity
Measuring brain activity is vital tool commonly utilised by neuroscientists. There is
a large number of brain imaging techniques which offer different levels of spatial and
temporal resolution. Methods such as the electroencephalography (EEG) have very
high temporal resolution (in the range of 1ms), but poor spatial resolution. In the
other end of the spectrum are methods such as functional magnetic resonance imaging
(fMRI) which could provide spatial resolution of less than 1mm, but could record only
slow events (in the range of 2s). Additionally, neuroergonomics imposes one more
constraint to the brain imaging techniques. The subject should be able to move freely
within their environment, unobstructed by the equipment which monitors the brain
activity. Figure 2.1 illustrates how the available brain imaging techniques compare
to each other based on 3 criteria - spatial resolution, temporal resolution, immobility
[11]. The compactness and mobility of EEG equipment and its fine temporal resolution
make EEG one of the most used brain imaging techniques in neuroergonomics.
2.2
Electroencephalography (EEG)
EEG is one of the oldest brain imaging techniques with the first recordings from animals made in 1875 by the English physiologist Richard Caton.
8
Figure 2.2: A diagram of showing the operational principles of EEG [14].
2.2.1
Types
There are three broad classes of EEG depending on the location of the electrode used to
record the brain electromagnetic activity. During intracortical EEG a single electrode
or an array of electrodes is inserted in the brain tissue providing high quality signal.
Because of the intrusiveness of the method, it is applicable only to animals and in
very rare cases to humans. The second type is cortical EEG where usually an array of
electrodes is placed on the surface of the brain. It is mainly applied to treating patients
with epilepsy. The last type is surface EEG where one or many electrodes are placed
on the scalp. This type of EEG is non-intrusive which makes it suitable for recording
EEG activity of human subjects while they perform different tasks.
2.2.2
Principles of operation
Figure 2.2 shows a diagram which explains the principles of operation of surface EEG.
The postsynaptic activity of the cerebral neurons, which form the grey matter, generates an electromagnetic field. An electrode which is placed on the scalp picks up that
field and measures its potential with respect to another reference electrode placed elsewhere on the scalp. Modern EEG equipment can record simultaneously from as many
as 256 electrodes that are placed according to the international 10-20 system which
2.2. Electroencephalography (EEG)
9
Figure 2.3: The EEG electrode locations according to the international 10-20 system.
standardises electrode locations as shown in figure 2.3. The maximum amplitude of
the detected voltages across the scalp does not exceed more than 200µV for healthy
subjects. Subsequently, the analog signal is transformed into a digital one by first amplifying it and then sampling it. Once the EEG signal is digitised it could be filtered,
processed and analysed by a regular computer using the appropriate software.
2.2.3
Noise and artefacts
Surface EEG provides exceptionally fine temporal resolution, but there are several
factors which cause the recorded signal to be noisy. Firstly, the electromagnetic field
generated by the dendritic activity is very week. The signals which can be detected at
the scalp are a mixture of the activity of thousands of pyramidal neurons in the cortex.
Secondly, the electromagnetic field has to propagate through several layers of tissue as
illustrated in figure 2.2 until it reaches the electrode which significantly attenuates it
and severely reduces the signal to noise ratio.
Furthermore, the signal is often contaminated with artefacts generated by muscle
activity e.g. tongue movement, facial muscles contraction, ocular movement. In order to avoid such artefacts usually the participants in the experiment are instructed
to move as little as possible. A less frequent approach is to simultaneously record
the activity of skeletal muscles using electromyography (EMG) and reject data during
movements above a certain threshold. Ocular movement is hard to control and it is
10
often required in order to perform the experimental task. Therefore, various methods
for ocular movement artefacts removal have been proposed [15].
2.2.4
Rhythmic activity
EEG signals have complex oscillatory waveforms which have been of a great interest
to researchers for a long time. Several frequency bands have been identified, namely
infra-slow (< 0.2Hz), δ (0.2 to 3.5Hz), θ (4 to 7.5Hz), α and µ (8 to 13Hz), β (14
to 30Hz), γ (30 to 90Hz) and high frequency oscillations (HFO) (> 90Hz). Various
cognitive processes have been mapped to changes in the activity at different frequency
bands. For example, the power of the θ band is increased during memory recognition task [16]; phase locked oscillations in the γ band are associated with conscious
perception [17]; γ oscillations occur during voluntary isometric muscle contractions
[18]. Furthermore, synchronisation between the oscillations of separate brain regions
is commonly interpreted as either structural or functional connection between the regions [19].
2.3
Event-related potentials (ERP)
There is a large amount of research related not only to EEG oscillations, but also to
EEG activity which is time locked to a particular event. The potentials, which the event
evokes, are called event-related potentials (ERPs) and provide valuable information
regarding the operation of the brain - namely how it processes various events such
as sensory stimuli, movement preparation and execution, error detection. ERPs are
usually observed by averaging EEG signals from multiple trials, however real time
detection of ERPs requires single trial classifiers, which are the focus of this project.
The following subsections list some of the most prominent ERPs in the literature and
provide short descriptions.
2.3.1
Readiness potential
Readiness potential is one of the first reported ERPs. It is expressed as a slow decrease
in the voltage over the central scalp areas during movement initiation [9]. Lateralized
readiness potential (LRP) could be measured by analysing the difference between the
EEG signals at the C3 and C4 electrodes. If the subject is to move one hand then the
electrode on the contralateral hemisphere would detect more negative voltage than the
2.3. Event-related potentials (ERP)
11
ipsilateral electrode [20]. Readiness potential provides a direct measure of movement
intention and which arm is to be moved could also be predicted.
2.3.2
Contingent negative variation
Contingent negative variation (CNV) is an ERP which occurs when the subject expects
a particular stimulus associated with an already perceived clue [21]. The CNV is expressed as a gradual and continuous decrease of the EEG potential in the frontal lobe
until the expected stimulus is presented. Hence, CNV could be used as a measure of
anticipation of a particular stimulus.
2.3.3
P300
The P300 is characterised with a slow positive peak of the EEG potential in the centroparietal scalp region at approximately 300ms after the event onset [22]. It has large
amplitude and could even be measured on single trials, which makes it probably the
most studied and used ERP. It is evoked by the oddball paradigm. Two stimuli either
conceptual or perceptual are selected and randomly presented to the subject with lower
probability assigned to one of them. The amplitude of the evoked P300 ERP is larger
when the less probable stimulus is perceived by the subject. The amplitude of the P300
is also thought to reflect the amount of attention which the subject is paying to the task.
Furthermore, P300 is regarded as a reliable index of cognitive workload [10].
2.3.4
N400
The N400 ERP is related to the difficulty of integrating new incoming information with
prior beliefs. It is characterised as a widespread drop in the potential which occurs
approximately 400ms after an unexpected event [9].
2.3.5
P600
P600 is a language related ERP which is characterised by a positive peak in the potential of the posterior temporal lobe which occurs 600ms after a language error is
perceived [9].
12
Figure 2.4: Average response ErrP [24].
2.3.6
Error related negativity (ERN)
Error related negativity is an ERP which is observed between 100ms and 150ms after
an error is perceived. ERN is characterised with a negative peak, which is generated
by the anterior cingulate cortex (ACC) in the frontal lobe. The peak is much smaller
or absent when no error is perceived [23]. It is important to note that the amplitude
depends on the perceived error meaning that even if the person commits an error, but
does not perceive it as such then ERN will not be observed. Several sub-types of ERN
have been reported and are termed as error related potentials (ErrP).
2.3.6.1
Response ErrP
Response ErrP is invoked when a person executes an erroneous action and is characterised with a negative potential peak at approximately 100ms and a second positive
peak at approximately 300ms after the error has been perceived [24, 25]. Figure 2.4
shows the response ErrP which was formed by averaging multiple trials.
2.3.6.2
Feedback ErrP
Feedback ErrPs (or reinforcement ErrPs) are evoked when feedback indicating wrong
action or decision is delivered. Feedback ErrPs have similar waveform as the one
shown in figure 2.4, but they are usually slightly delayed as additional time is required
to process the feedback stimulus [26].
2.4. Brain-computer interfaces
13
Figure 2.5: Average observation ErrP [27]. Note that the y-axis is flipped.
2.3.6.3
Observation ErrP
Observation ErrPs are evoked when the subject observes an error committed by another
person or agent [27]. It is characterised with a negativity peak in the frontal lobe at
approximately 250ms after an error is observed as shown in figure 2.5. The strong
positive peak observed in a response ErrP is not present, or it has very small amplitude.
2.3.6.4
Interaction ErrP
Interaction ErrPs are invoked when a command is delivered and the machine executes
another one. Interaction ErrPs have similar shape to the response ErrP with one negative and one positive peak, however they are delayed with 100ms more [28]. Even
though classified as a new type of ErrP, one could argue that interaction ErrP is simply
a reaction ErrP, which is delayed due to the time required to realise the error.
2.4
Brain-computer interfaces
There are two main classes of brain-computer interfaces: the first one exploits spontaneous EEG activity which subjects could learn to control; the second one relies on
event related activity, which subjects have no conscious control of, and it is dependent
on the interaction with the environment [8].
14
2.4.1
Spontaneous activity BCIs
BCIs based on spontaneous activity have been applied to tasks such as control of a
wheel chair [6, 5] or flying quadcopter [7] (for a detailed list of application see [29]).
Spontaneous activity BCIs require a learning period during which the EEG activity of
the operator is recorded while they think about the command. Certain features are then
selected from those signals and passed through a classifier which outputs the detected
command. This approach enables intuitive and quick interaction, but there are two
main problems with it. Firstly, only a small number of commands could be learnt, thus
severely limiting the capabilities of the interface. Secondly, EEG signals are chaotic
and non-stationary [30], meaning that they will inevitably change over time resulting
in performance decrease over time of the brain-computer interface.
2.4.2
Event-related activity BCIs
Event-related activity BCIs are generally slower as they require constant interaction
with the environment in order to register events. They have been applied to virtual
spellers using the P300 ERP [31], virtual robotic arm control [32], BCI classifier adjustment [33]. An attempt has been made to remedy the issue with low interface speed
and throughput by using the readiness potential which predicts movements [34]. BCIs
based on event-related activity allow for higher throughput of information as the operator should not consciously think about the commands they are sending, because the
brain automatically generates the ERPs.
2.4.3
Human-robot interaction
Imagine a scenario where a human and a robot have to cooperate, for example assembling a computer. If the human had to constantly think about the actions which
the robot should take, then the human will not be able to perform their own tasks.
However, if an event-related BCI is used, then the human will subconsciously generate
instructions for the robot. For example if the robot approaches the wrong detail, an
error-related potential will be detected, informing the robot that it needs to adjust its
behaviour. If a readiness potential is detected the robot will know that the human is
about to move and it should assist the human if necessary. Intention prediction is a crucial feature for an intuitive human-robot interface. EEG activity is an excellent source
of information for that [9]. Other equipment could be used as well (e.g. eye trackers,
2.4. Brain-computer interfaces
15
RGBD cameras, EMG) in order to successfully infer the intention of the operator.
Error related potentials are particularly suitable for human-robot interfaces as their
meaning could be directly translated to the reward signal in an reinforcement learning
algorithm [32, 35]. ErrPs have been used to control a simulated 2-dimensional robotic
arm [32], however, it is to the best of the author’s knowledge, that ErrPs have not
been applied to an actual human-robot interface with a physical robot. This thesis is a
pilot study in exploring single trial ERP detection and classification using affordable,
compact and mobile EEG equipment with the purpose of developing actual HRI based
on event related potentials.
Chapter 3
Methods
This chapter provides a thorough description of the experiments which were conducted
during the project and the equipment which was used. A brief overview of the classifiers which are trained to detect single trial ERPs is given. Additionally, the dimensionality reduction method used to preprocess the EEG data is described as well as a
feature selection method which is used for channel selection.
3.1
Experimental setup
Two experiments were conducted during which the participants had to perform a behavioural task and their EEG activity was recorded. There were 10 participants in total.
Each one of them participated in both experiments and signed a consent form prior to
the experiments. Furthermore, ethical self-assessment was conducted and submitted
to the School of Informatics as required by the University of Edinburgh regulations.
The experiments were conducted in a sound insulated room in order to reduce the
distraction of the participants.
3.1.1
Emotiv EEG EPOC headset
The Emotiv EPOC headset was used to record the EEG activity of the participants during both of the experiments. One of the main aims of the current project is to evaluate
the Emotiv EPOC headset which is a lightweight wireless EEG capturing device. Providing great mobility, 14 EEG channels and a built in 2-dimensional gyroscope at a
comparatively low cost, the Emotiv EPOC headset is an extremely suitable device for
the purposes of the development of an intuitive and efficient neural based human-robot
17
18
Chapter 3. Methods
(a) Picture of the Emotiv EPOC headset.
(b) The channels available on the Emotiv EPOC
headset according to the 10-20 system are
highlighted in orange.
Figure 3.1: The Emotiv EPOC headset.
interface. A picture of the Emotiv EPOC headset is shown in figure 3.1a and the location of the 14 electrodes according to the international 10-20 system is shown in figure
3.1b. The Emotiv EPOC headset uses dry electrodes which rely on pressure against
the scalp to form good contact. Nevertheless, the electrodes need to be moistened with
saline before use, but skin preparation gel is not necessary unlike to the conventional
wet Ag/AgCl electrodes, which are typically used in research. Figure 3.2 lists some of
the key characteristics of the Emotiv EPOC headset.
3.1.2
Experiment 1 - flanker task (FT)
The aim of the first experiment is to confirm that response ErrPs can be recorded with
the Emotiv EPOC headset as reported by [36]. The detection of P300 with the Emotiv
EPOC headset has already been demonstrated [37].
During the experiment the participants had to perform a simple flanker task. They
had to fixate a cross in the middle of the screen for 500ms. Once that time elapsed, a
stimulus consisting of 5 arrows would be shown to the participant for 100ms and they
had to indicate the direction of the middle arrow (left or right) as fast as possible using
the keyboard arrow buttons. There were 4 possible stimuli - two congruent (<<<<<
and >>>>>) and two incongruent (<<><< and >><>>) stimuli. In each trial,
one of the stimuli was randomly chosen with equal probabilities for each stimulus.
3.1. Experimental setup
Number of channels
Channel names
19
14 (+2 reference channels)
AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4,
F8, AF4
Sampling method
Sampling rate
Sequential sampling. Single ADC.
128 SPS (2048Hz internal)
Resolution
14 bits, 1 LSB = 0.51 µV
Bandwidth
0.2 - 45Hz, digital notch filters at 50Hz and 60Hz
Filtering
Built in digital 5th order Sinc filter
Dynamic range
8400 µV (pp)
Coupling mode
AC coupled
Connectivity
Power
Battery Life (average)
Proprietary wireless, 2.4GHz band
LiPoly
12 hours
Impedance Measurement Real-time contact quality using patented system
Figure 3.2: Characteristics of the Emotiv EPOC headset.
Once the stimulus was presented a blinking dash would appear on the centre of the
screen with blinking frequency of 2Hz until the subject reacts. This puts additional
time pressure on the participant. Once the subject pressed a keyboard button indicating
left or right, the screen would become blank for 500ms and then the next trial would
begin with the fixation cross. During the experiment the screen background was black,
while all stimuli were white. 6 sessions with 40 trials each were conducted per subject
with 1 minute time for rest between the sessions. Thus the experiment duration was
approximately 15 minutes per subject and 240 flanker task trials were recorded per
subject.
In order to keep the EEG data as noise free as possible, the participants were asked
at the beginning of the experiment to sit comfortably and minimize their movements
including tongue movement and blinking.
Given the time pressure, the short exposure time to the stimulus and the incongruence of some of the stimuli, the subjects were prone to make errors, hence reaction
ErrPs to be invoked. Even though subjects were instructed to be as fast as possible
some of them were too careful not to make a mistake. Therefore, an improvement of
the experiment protocol could involve the gamification of the task. It is possible to
20
Chapter 3. Methods
give the participant a feedback score which puts larger weight on speed rather than
accuracy.
The synchronisation between the recorded EEG data and the behavioural response
is essential for the results of the experiment. The Emotiv EPOC headset works only
with a proprietary API which is implemented in C++. In order to avoid additional
timestamping and minimise the margin for error in synchronisation the experimental
task programs were implemented in C++ as well with the aid of the open computer
vision library (OpenCV).
3.1.3
Experiment 2 - robotic arm task (RAT)
The aim of the second experiment is to investigate whether observation ErrPs could be
detected with the Emotiv EPOC head set. It is to the best of the author’s knowledge
that observation ErrPs have been recorded only with medical grade EEG equipment
[27, 32]. ErrPs have much smaller amplitude and so the signals provided by the Emotiv EPOC headset may be too noisy. The second experiment recreates the task used in
[32] to record observation ErrPs. If observation ErrPs are not detected in our experiment then the conclusion that the Emotiv EPOC headset does not provide signals with
sufficient quality could be safely drawn.
During the second experiment the subjects are seated comfortably in front of a
computer and they are asked to simply observe a planar robotic manipulator with 2
degrees of freedom (one rotational and one translational) which performs a reaching
task. There are five baskets and the subject is told that the robot should always reach
for the middle basket. Once a trail begins, the robot waits in idle position for 1 second
as shown in figure 3.3a. In the experiment reported in [32] the basket towards which
the robotic arm will reach is selected randomly from a uniform distribution over the
baskets. Initial trails revealed that this enables the participants to quickly realise that
the arm moves absolutely randomly and they lose concentration. Therefore, the selection of the basket for our experiment was altered. Firstly, the robot is not allowed to
reach to a wrong basket twice in a row. Secondly, if the previous action was correct,
the type of action is randomly selected - correct or wrong with 50% chance for each. If
a wrong action is chosen, then any of the wrong baskets is randomly picked following
a uniform distribution. When the basket is selected the robot instantaneously moves to
it as shown in figure 3.3c and figure 3.3b. Moving instantaneously provides a distinctive event which the EEG data to be synchronised to. The manipulator remains in the
3.1. Experimental setup
21
reaching position for 1 second and goes back to the idle position, ready to execute a
new movement. The experiment consisted of 6 session with 1 minute break between
sessions. During each session the robotic arm performed 5 sequences of 20 movements
with 5 seconds rest between each sequence, allowing the participant to re-adjust their
position should they want. Hence, the total length of the experiment is approximately
25 minutes per subject. The total number of reaching trials during which the EEG
activity was recorded is 6000.
(a) The robotic arm in idle position.
(b) The robotic arm reaching to a wrong basket.
(c) The robotic arm reaching to the correct basket.
Figure 3.3: The robotic arm experiment.
22
3.2
Chapter 3. Methods
ErrP Classification
Another aim of the presented thesis is to investigate to what extent single trial ERP
detection is possible, how reliable it is and what type of classifiers cope best with
the task of recognising ERPs provided with the data acquired with the Emotiv EPOC
headset. It is not a trivial problem since EEG signals are known to be chaotic [30],
noisy and last, but not least, ERPs have short duration of several hundred milliseconds.
3.2.1
Robust discriminant power (RDP)
EEG equipment records the electromagnetic equipment of the brain with as many as
256 electrodes; even sparse sensors such as the Emotiv EPOC headset provide 14 channels. The EEG data is noisy and highly correlated, therefore it will be naive to feed all
this data into a classifier and hope that it will make sense out of it. A common starting
point when working with EEG data is to select one or several channels of interest. The
process of channel selection requires a set of features which are measurable in order to
enable objective data-based comparison of the separate channels. The concept of feature selection based on RDP is introduced in this section and the application to channel
selection is described in section 5.1.
3.2.1.1
Feature selection
The robust discriminant power (RDP) measure is introduced in [38]. It estimates the
extent to which a given feature could be used to discriminate between several classes,
assuming normal feature distribution. Let us consider a two class example for the purpose of explanation. If the distribution of a feature f for class 1 is pd f 1 ( f ) and for
class 2 is pd f 2 ( f ) then feature f has high discriminant power if pd f 1 ( f ) is well separated from pd f 2 ( f ). If pd f 1 ( f ) and pd f 2 ( f ) are overlapping then feature f cannot
be used to discriminate between the two classes. The discriminant power algorithm
is computationally efficient because it is not necessary to estimate the actual distributions. It is sufficient to know the minimum and the maximum values of the feature
f for which there is non-zero probability according to pd f 1 ( f ) and pd f 2 ( f ). Given
those limits it is easy to estimate the overlap between the distributions. However, this
method is sensitive to outliers. Therefore, [38] recommends calculating the means µ1
and µ2 as well as the standard deviations σ1 and σ2 of pd f 1 ( f ) and pd f 2 ( f ). Then, in
order to estimate the overlap between the two distributions use the following limits for
3.2. ErrP Classification
23
each class k:
lkmin = µk − |σk |
lkmax = µk + |σk |
(3.1)
Thus outliers in the data will have very little effect on the estimate of the discriminant
power. The robust discriminant power of a feature f for a two class problem is given
by:
RDP( f ) =
ND1 + ND2
N1 + N2
(3.2)
where N1 and N2 is the number of samples for each class. ND1 and ND2 is the number
of discriminant samples which belong to each class. Discriminant samples are those
which belong to a non-overlapping region of the class distribution and so:
N1
ND1 = ∑ l(s1 (i) < l2min ) + l(s1 (i) > l2max )
(3.3)
ND2 = ∑ l(s2 (i) < l1min ) + l(s2 (i) > l1max )
(3.4)
i=1
N2 i=1
where sk (i) is the ith sample of class k and l(c) which is equal to 1 when the condition
c is true and 0 otherwise.
3.2.2
Dimensionality reduction - principal components analysis (PCA)
As discussed in section 3.2 EEG data has a large number of dimensions which are often
correlated and dimensionality reduction is often desirable before any further analysis.
The proposed method for ERP classification utilises the principal components analysis
(PCA) and so it is briefly described here.
If X is an n × d matrix, where n is the number of observed d dimensional points,
then the covariance matrix of X is given by
C = XT X
If the observed points in X do not have zero mean, they should be recentred before the
covariance matrix is calculated. C is a d × d matrix, the eigenvectors of which define
d orthogonal directions along which the variance of the data in X is maximised. The
magnitude of the corresponding eigenvalues provides information about the variance
along each eigenvector. Usually, the first few eigenvectors are picked to form a lower
dimensional space in which the data could be projected preserving as much variance
24
Chapter 3. Methods
as possible. The preserved accumulative variance of the data projected onto an m
dimensional subspace for (m < d) is given by
σ2acc =
∑m
i=1 λi
∑di=1 λi
(3.5)
where λi is the ith eigenvalue.
3.2.3
Discriminative classifier - logistic regression (LR)
A logistic regression classifier is considered for the detection of ErrPs in EEG signals
as it has already been reported to achieve satisfactory accuracy at classifying ERN
potentials [36]. It is a discriminative classifier meaning that it directly models the
class label probability distribution p(y|x, w) conditioned on the input data x and the
model weights w. Logistic regression corresponds to the following binary classification model [39]:
p(y|x, w) = Ber(y|sigm(wT x))
(3.6)
where sigm(x) is the sigmoidal function defined as:
sigm(x) =
1
1 + e−x
and Ber(x|θ) is the Bernoulli distribution given by:

θ
if x = 1
Ber(x|θ) =
1 − θ if x = 0
The sigmoidal function introduces a nonlinearity in the negative log-lokelihood (NLL)
of the model and so an analytical solution for the weights of the model does not exist.
However, the Hessian matrix of the NLL is positive definite meaning that there is a
single global optimum which could be found with an iterative optimisation algorithm
such as gradient descent. The model fitting to the data was performed using MATLAB
and the generalised linear model regression function (glmfit) part of the Statistical
Toolbox.
3.2.4
Generative classifier - mixture of Gaussians (MOG)
A generative classifier was also considered for the detection of ErrPs. A generative
classifier models the distribution of features x given the class label c and the parameters
3.2. ErrP Classification
25
of the model θ i.e. p(x|y = c, θ). The probability of a new data point xnew to belong to
class c could be found by using Bayes rule
p(y = c|xnew , θ) ∝ p(xnew |y = c, θ)p(y = c)
(3.7)
The new point is assigned to the class with highest probability.
It is common to assume that the class conditional distribution p(x|y = c, θ) could
be represented as a mixture of Gaussians when classifying ErrPs. [8, 33, 38]. The
mixture of Gaussians (MOG) model is given by
K
p(x|θ) =
∑ πk N(x|µk , Σk )
(3.8)
k=1
where N(x|µ, Σ) is a normal distribution over x with mean µ and covariance Σ. πk
are called mixing weights and they have to satisfy the 0 ≤ πk ≤ 1 and ∑K
k=1 πk = 1.
Fitting this model to the data is a non-convex problem and most often the expectation
maximisation (EM) algorithm is used. It is a two-step iterative algorithm which firstly
calculates the expected value of the NLL over the latent variables (the responsibility
of each mixture component) and then maximises it [39]. A MOG model was fitted for
each class using MATLAB and the (gmdistribution.fit) function part of Statistical
Toolbox.
3.2.5
Temporal classifier - hidden Markov model (HMM)
Markov models are widely applied to problems which involve time series such as EEG
signals. A Markov chain is the joint probability distribution over all observed variables
xt for a period T
T
p(x1:T ) = p(x1 ) ∏ p(xt |xt−1 )
t=2
where p(xt |xt−1 ) is the transition probability distribution. A hidden Markov model
consists of a discrete-time, discrete-state Markov chain, with hidden states zt and observation model p(xt |zt ) often called emission probability distribution [39]. The HMM
joint distribution is given by
"
T
#"
p(z1:T , x1:T ) = p(z1 ) ∏ p(zt |zt−1 )
t=2
T
#
∏ p(xt|zt )
t=1
Since the observed EEG data, even though discretised, has a continuous nature, mixture of Gaussians (see equation 3.8) is often used as an observation model [8, 30, 40].
26
Chapter 3. Methods
An HMM is fitted to each class and new time series are assigned to the class which
explains the new data with higher likelihood. Essentially, this is a generative classifier
which uses a temporal model for the class conditional probability distributions. The
HMMs were fitted to the data using MATLAB and the Hidden Markov Model (HMM)
Toolbox 1 written by Kevin Murphy.
1 http://www.cs.ubc.ca/
murphyk/Software/HMM/hmm.html
Chapter 4
Observing error related negativity
The first step in the analysis of the results is to examine the recorded EEG data from
the two conducted experiments and determine whether ERN components are present or
not. The presence of ERN has already been reported for signals recorded by the Emotiv
headset during the completion of the Flanker task [36]. However, the observation ErrP,
evoked by the robotic arm task, is as much as 10 times weaker than the response ErrP,
evoked by the Flanker task [27]. Observation ErrPs have been recorded only with
research class EEG equipment using wet Ag/AgCl electrodes [27, 32]. Therefore, it
is uncertain if observation ErrPs will be detected with dry electrodes and the Emotiv
headset. This chapter describes how the recorded data is pre-processed and the ocular
artefact analysis conducted. Furthermore, the mean recorded ERP signal is investigated
and statistical analysis for the significance of the findings is performed. The chapter
finishes with the evaluation of the Emotiv EPOC headset.
4.1
4.1.1
EEG data preprocessing
Extracting event centred EEG epochs
Since only ERPs are to be analysed, only the EEG activity during the occurrence of
an event is of interest. Therefore, 1 second long epochs were extracted from the EEG
data - starting half a second before and finishing half a second after an event. Table
4.1 lists the total count of extracted epochs for all participants in both experiments.
Due to trimming artefacts caused by the extraction of the 1 second segments some
epochs were dismissed. There are 2106 epochs in total for the Flanker task and 343 of
them capture the EEG activity when an error was made by the subject. There are 5400
27
28
Chapter 4. Observing error related negativity
Error event
No error event
Total epochs
Flanker task
343
1763
2106
Robotic arm task
1759
3641
5400
Figure 4.1: Number of extracted epochs for all participants in both experiments during
the occurrence of events with and without error.
epochs for the robotic arm task and 1759 of them correspond to an observed error.
4.1.2
Frequency filtering
One of the main disadvantages of electroencephalography is the low signal to noise
ratio caused by the attenuation of the signals from the cortical neurons. Careful data
pre-processing is essential for further analysis. Event related electromagnetic activity
is relatively slow change of the field potentials and so the EEG data was bandpass
filtered between 1Hz and 5Hz using Fast Fourier Transform. Figure 4.2 shows the raw
and the filtered signal recorded from channel F7 both during correct and erroneous
subject response. The amplitude of the raw signal is in µV measured with respect to
the reference electrodes of the Emotive headset. Since we are interested in the shape
of the EEG signal, not its absolute amplitude, the filtered signal is shifted such that
it has zero mean value. As expected the raw EEG data is quite noisy, but selecting
only slow frequencies improves the data quality significantly. At the bottom left plot
in Fig. 4.2 a typical error related potential [24] could be observed - a drop in potential
at approximately 100ms and then a positive peak at approximately 200 − 300ms after
the error event. It must be noted that that the signal in Figure 4.2 was particularly
picked such that it contains as little noise as possible for illustrative purpose only.
Later sections in the section focus on more informative statistics such as the mean of
the event related potentials, the variance of the ERPs and the statistical significance of
the acquired data.
4.1.3
Analysis for eye movement artefacts
Eye movements result in strong artefacts which deteriorate the quality of the EEG
data. Various techniques have been suggested to either remove those artefacts or reject
data which contains them [15]. Ultimately, one would use an eye tracking device and
4.1. EEG data preprocessing
29
F7 - Raw with error
F7 - Raw without error
F7 - Filtered with error
F7 - Filtered without error
Amplitude, µV
2300
2200
2100
2000
1900
Amplitude, µV
10
5
0
−5
−10
−400−200 0
200 400
Time, ms
−400−200 0
200 400
Time, ms
Figure 4.2: Top left is the raw signal at channel F7 recorded during the Flanker task
associated with a committed error. Bottom left is the filtered version of the raw signal
including only frequencies between 1 and 5 Hz. Top right is the raw signal recorded at
channel F7 during the Flanker task when the subject correctly responded to the stimulus. Bottom right is the filtered out version of the raw signal including only frequencies
between 1 and 5 Hz.
30
HEOG - Flanker task
HEOG - Roboarm task
Amplitude, µV
300
200
100
−400−200 0
200 400
Time, ms
−400−200 0
200 400
Time, ms
Figure 4.3: The estimated mean HEOG for both experiments. The blue line is the
mean HEOG during the extracted ERP epochs. The dashed red lines are ± 1 standard
deviation away from the mean, forming a 68.2% confidence interval, assuming normal
distribution. The horizontal red line is at 330µV which is the reported data rejection
threshold by [41].
develop a model which predicts the EEG artefacts based on the detected movement.
However, this requires additional equipment for the experiment and is not a trivial
task. A more common approach is to infer eye movement from EEG data (electroocolarography (EOG)) and just reject short segments which occur during well-pronounced
movements [35, 6, 13, 41]. The horizontal EOG (HEOG) can be extracted by calculating the absolute difference between the F7 and F8 channels. Figure 4.3 shows the
mean HEOG estimated from the extracted and filtered ERP epochs. The amplitude of
the observed HEOG is much smaller than the threshold of 330µV reported by [41] and
so the ocular movement artefacts should not affect any further analysis. There are two
potential reasons for the low HEOG amplitude. Firstly, the subjects were instructed to
move their eyes as little as possible which decreases the probability of a movement to
occur during an event. Secondly, eye movements produce narrow spikes in the EEG
signal which should be removed by the applied bandpass filter. Given the reported
threshold by [41] it is concluded that no further ocular artefact removal is necessary.
4.2
Mean ERPs
Usually, ERPs are observed by calculating the mean signal from many trials. Assuming
that the noise present in the EEG signal is white, then the mean ERP should reflect the
4.2. Mean ERPs
31
processes that take place in the brain during the processing of the event.
4.2.1
Mean response ErrP
The mean response ErrP at channel F7 obtained during a correct and incorrect action
while performing the FT is shown in figure 4.4a. When an error is present there is
a strong negative peak at approximately 100ms after the event and a positive peak
at about 250ms after the event. This waveform is identical to the one reported in
[24] (see figure 2.4). The peak-to-peak amplitude is approximately 10µV . When no
error is present there is also a very small negative deflection which is expected [24].
The mean ERP recorded at channel T8 is shown in 4.4b where response ErrP is not
expected due to the frontal distribution over the scalp of ErrPs. It can be seen in
4.4b that there are some peaks present, but the peak-to-peak amplitude of the signal is
approximately 1.5µV which is small enough to be safely ignored. Hence, the analysis
of the mean response ErrP shows that response ErrPs can be recorded with the Emotiv
EPOC headset from channels on the frontal lobe and are barely distinguishable in other
regions of the scalp such as the temporal lobe.
4.2.2
Mean observation ErrP
The mean observation ErrP at channel F7 obtained during a correct and incorrect action
while performing the RAT is shown in figure 4.5a. When an error is present there is a
strong negative peak at approximately 400ms after the event. This waveform is slightly
different from the one reported in [27] (see figure 2.5). First of all, the peak is delayed
with approximately 100ms and it is much steeper. This suggests that the detected
ERP is not exactly an observation ErrP, but probably the N400 potential (see section
2.3.4). The experiment, however, was very repetitive and the subjects quickly learnt
all possible states meaning that it is highly unlikely to be the N400. Moreover, the
same signal shape (even more delayed) is reported in [32], where the RAT experiment
is described. The delay could be potentially attributed to the time required to visually
process the event of error. The peak amplitude of the recorded observation ErrP is
approximately 4µV which is less than half of the response ErrP amplitude. Similar to
the response ErrP, when no error is present there is a very small negative deflection.
The mean ERP recorded at channel T8 is shown in 4.5b where observation ErrP is not
expected. It can be seen in 4.5b that there are some peaks present, but their amplitude
is negligible. Hence, the analysis of the mean observation ErrP shows that observation
32
Mean response ErrP at F7 (FT)
Amplitude, µV
10
5
0
−5
No error
Error
−400 −200
0
200
Time, ms
400
(a) Mean response ErrP recorded from channel F7.
Mean response ErrP at T8 (FT)
Amplitude, µV
0.5
0
−0.5
−1
No error
Error
−400 −200
0
200
Time, ms
400
(b) Mean observation ErrP recorded from channel T8.
Figure 4.4: The mean response ErrP recorded at channel F7 (top) and T8 (bottom)
during error and no error conditions .
4.2. Mean ERPs
33
Mean observation ERP at F7 (RAT)
1
Amplitude, µV
0
−1
−2
−3
−4
No error
Error
−400 −200
0
200
Time, ms
400
(a) Mean observation ErrP recorded from channel F7.
Mean observation ERP at T8 (RAT)
Amplitude, µV
0.2
0
−0.2
−0.4
−0.6
No error
Error
−400 −200
0
200
Time, ms
400
(b) Mean observation ErrP recorded from channel T8.
Figure 4.5: The mean observation ErrP recorded at channel F7 (top) and T8 (bottom)
during error and no error conditions.
34
ErrPs can be recorded with the Emotiv EPOC headset from channels on the frontal lobe
and are barely distinguishable in other regions of the scalp such as above the temporal
lobe.
4.3
Significance of the data
One way ANOVA analysis was carried out in order to determine how significant the
shape of the ErrPs recorded during correct and incorrect events is. The null hypothesis
is that ErrPs recorded during correct and incorrect events have no difference in the
mean. The p-value was calculated for each time sample and the results for channel F7
are shown in figure 4.6. The probability of the recorded data to be explained by the zero
hypothesis at the intervals of interest (between 0ms and 200ms after the event for the
response ErrP and between 400ms and 500ms after the event for the observation ErrP)
is virtually zero. Therefore, the results obtained from both experiments are statistically
significant. Figure 4.7 shows the same analysis, but for the channel T8, where the
difference between error and no error ERPs is statistically insignificant.
4.4
Evaluation of the Emotiv EPOC headset
Even though some of the mainly used electrodes in the study of ErrPs, which are Cz
and Fz, are not available on the Emotiv EPOC headset it was demonstrated that both
response ErrPs and observation ErrPs can be recorded. The reported results confirm
the fact the Emotiv EPOC headset actually records EEG activity rather than muscle
activity only. However, there were some issues with the device. Given the design of the
headset it was difficult to ensure correct placement of the electrodes. Not only that, but
after 45 minutes of use subjects complained of discomfort caused by the mechanical
pressure applied to the head by the headset. The dry electrodes manage to pick up
EEG signals, but the quality of the contact with skin deteriorates relatively quickly.
Figure 4.8 shows the quality of all contacts at the beginning of an experiment and at
the end. This information is provided by the Emotiv Control Panel software. The
quality of the contact is encoded with 5 colours - green, orange, yellow, red, black.
Green means that the contact is excellent, while black indicates lack of contact. It can
be seen that for the 45 minutes of use the state most of the contacts has worsened.
Therefore, for any prolonged use the electrodes of the Emotiv EPOC headset should
be periodically moistened with saline and repositioned. Additionally, there were some
4.4. Evaluation of the Emotiv EPOC headset
35
Significance of the mean response ErrP at F7 (FT)
1
10
5
0.6
0.4
p-value
Amplitude, µV
0.8
0
0.2
−5
0
−400 −200
No error
0
200
Time, ms
Error
400
0.05
p-value
(a) The statistical significance of the shape of the mean response ErrP (FT) recorded
at channel F7 for both error and no error conditions.
1
1
0
0.8
−1
0.6
−2
0.4
p-value
Amplitude, µV
Significance of the mean observation ErrP at F7 (RAT)
0.2
−3
0
−4
−400 −200
No error
0
200
Time, ms
Error
400
p-value
0.05
(b) The statistical significance of the shape of the mean observation ErrP (RAT)
recorded at channel F7 for both error and no error conditions.
Figure 4.6: ANOVA analysis for the signals recorded at F7 - Error vs. No error conditions
36
Significance of the mean response ErrP at T8 (FT)
10
1
5
0.6
0.4
0
p-value
Amplitude, µV
0.8
0.2
−5
0
−400 −200
No error
0
200
Time, ms
Error
400
0.05
p-value
(a) The statistical significance of the shape of the mean response ErrP (FT) recorded
at channel T8 for both error and no error conditions.
1
1
0
0.8
−1
0.6
−2
0.4
−3
0.2
−4
0
−400 −200
No error
0
200
Time, ms
Error
p-value
Amplitude, µV
Significance of the mean observation ErrP at T8 (RAT)
400
p-value
0.05
(b) The statistical significance of the shape of the mean observation ErrP (RAT)
recorded at channel T8 for both error and no error conditions.
Figure 4.7: ANOVA analysis for the signals recorded at T8 - Error vs. No error conditions
4.4. Evaluation of the Emotiv EPOC headset
37
(a) Contact quality at the beginning of
(b) Contact quality at the end of the
the experiment.
experiment.
Figure 4.8: Quality of contact before and after the experiment. The information is provided by the Emotiv Control Panel software and the quality of the contact is encoded
with 5 colours - green, orange, yellow, red, black. Green means excellent contact, while
black indicates lack of contact at all.
connectivity issues occasionally causing the device to spontaneously disconnect. Since
the system is closed source there was now way to track where the problem comes from.
Nevertheless, the Emotiv EPOC headset was found to be an amazing low-cost research
tool providing great opportunities for various experiments.
Chapter 5
ErrPs classification results
The previous chapter demonstrated that error related potentials can be detected in the
EEG signals recorded by the Emotiv EPOC headset. This chapter investigates the classification of single trial ErrPs. It presents the results on the performance of 3 types of
classifiers trained to discriminate between error or no error ErrPs. Various statistics
are reported for each classifier type and advantages and disadvantages discussed. Additionally, a channel selection method based on the robust discriminative power of a
feature is also presented.
5.1
Channel selection
The Emotiv EPOC has 14 channels which provide data related to the electromagnetic
activity of the subject’s brain. However, it is highly unlikely that the signals in all of the
14 channels are uncorrelated. Furthermore, it is known that the source of ErrPs is the
anterior cingulate cortex in the frontal lobe and so it is expected that channels located
on the frontal scalp regions would provide better signal than channels in the occipital
lobe for example. This section describes a measure based on the robust discriminative
power (RDP) which evaluates the extent to which the signal from a channel could be
used to discriminate between error and no error events.
The RDP of a feature was introduced in section 3.2.1, but how it is applied to
channel selection was not discussed. In [38], where RDP was described, it is applied
to a small set of features extracted from the power spectrum density of the EEG signal.
However, the analysis of ErrPs, which focuses on transient waveforms, is in the time
domain. Therefore, the following procedure to calculate ErrP score for each channel
is proposed.
39
40
Chapter 5. ErrPs classification results
The event-locked segments of 1 second EEG activity recorded by channel k could
be represent as k serr (t) and k snoerr (t) where t ∈ [1; 128] is the time sample index and
the subscript encodes the type of the event. There are two classes and the value of the
signal at each sample point could be considered as a separate feature resulting in 128
features in total. Hence, the discriminant power of channel k, DPch (k), could be the
sum of the RDPs of each sample point:
128
DPch (k) =
∑ RDP(k s(t))
(5.1)
t=1
However, nothing prevents the measure in equation 5.1 from rating two channels with
the same score even though the amplitude of the observed ERPs is much smaller in
one of the channels. Therefore, the following measure is proposed where the RDP of
each sample point is weighed with feature range which is estimated using the limits in
equation 3.1
DPch (k) =
128 n
∑
o
max
min
[lerr
(t) − lerr
(t)] RDP(k s(t))
(5.2)
t=1
Thus, if the peak-to-peak amplitude of the detected signal is larger the channel will be
assigned a higher ErrP score.
The ErrP score of each channel during both of the conducted experiments was
calculated according to equation 5.2 and the results are shown in figure 5.1. First of
all, it could be immediately noticed that electrodes in the frontal lobe have significantly
higher ErrP scores. This is expected as the ErrPs are reported to mainly occur in
the frontal lobe (see section 2.3.6). A threshold value of 250 (shown as a horizontal
red line in figure 5.1) was determined by inspection and so only channels with ErrP
score greater than 250 are considered to provide signal which could be used to reliably
discriminate between the presence and absence of error. This means that there are 6
channels which could be used for both tasks. Those channels for the FT are AF3, F7,
F3, F4, F8 and AF4, while for the RAT they are the same with only F4 replaced by
FC6. However, F4 in the FT has much higher score than FC6 in the RAT therefore F4
is considered for both tasks. Finally, a closer inspection of figure 5.1 reveals that the
ErrP score of the selected channels is higher for the FT which indicates that observation
ErrPs, invoked by the RAT, are more difficult to detect. This is expected, since the
observation ErrP has smaller amplitude and is more likely to be masked by noise.
Further data analysis, classifiers training and performance evaluation will be performed on the signals recorded from channels AF3, F7, F3, F4, F8 and AF4.
5.1. Channel selection
41
Flanker task
ErrP Score
350
300
250
200
AF3 F7 F3 FC5 T7 P7 O1 O2 P8 T8 FC6 F4 F8 AF4
Channel
Robotic arm task
ErrP Score
350
300
250
200
150
AF3 F7 F3 FC5 T7 P7 O1 O2 P8 T8 FC6 F4 F8 AF4
Channel
Figure 5.1: The ErrP score estimated for each channel during both the task according
to equation 5.2. The horizontal red line at 250 represents a threshold which was chosen
by inspection.
42
Explained data variance - FT
Explained data variance - RAT
1
Preserved accumulative variance
Preserved accumulative variance
1
0.9
0.8
0.7
0.6
0.5
0.4
0.9
0.8
0.7
0.6
0.5
0.4
0
2
4
6
Number of dimensions
AF3
F4
F7
F8
8
F3
AF4
0
2
4
6
Number of dimensions
AF3
F4
F7
F8
8
F3
AF4
Figure 5.2: The preserved accumulative variance of the data when it is projected in a
lower dimensional space for both the FT (left) and the RAT (right).
5.2
Dimensionality reduction
Initial attempts to train a logistic classifier used the 1 second filtered segments as 128
dimensional data points. However, computational issues were encountered due to the
fact that there was high correlation between the separate dimensions. This was not
surprising as each dimension is a consecutive sample. Therefore, PCA was used to
find a subspace with lower dimensionality for each of the channels in which the 128
dimensional data to be projected and preserve as much information as possible. Figure
5.2 shows the obtained results for each of the six channels of interest. Interestingly, if
the data dimensionality is reduced from 128 down to 6 it is still possible to reconstruct
it almost perfectly as indicated by the preserved accumulative variance value which is
extremely close to 1. Therefore, for the training of the LR and the MOG classifiers
the data was projected in a 6 dimensional space, which was separately determined for
each of the two tasks using PCA.
5.3. Logistic regression classifier
5.3
43
Logistic regression classifier
A logistic regression classifier was trained on the preprocessed singe trial ERPs data
with reduced dimensionality for each channel. The data was split into two sets - training (80%) and testing set (20%). The reported accuracy is estimated as the number of
correctly classified ERPs divided by the total number of ERPs in the testing set. The
achieved accuracy levels for each channel for both tasks are shown at the top row of
figure 5.3. The accuracy for the FT is similar to the reported results in [36], while the
accuracy for the RAT task is comparable to the results in [35]. It is important to note
that observation ErrPs are classified with worse accuracy than reaction ErrPs which
is expected because they have smaller amplitude as already discussed and are more
susceptible to noise.
The number of positive samples (i.e. events with error) in the recorded dataset
is less than a third from all samples (see figure 5.9). Therefore, a more informative
measure of the classifier performance is the F1 score which is defined as
F1 =
2Nt p
2Nt p + N f n + N f p
(5.3)
where Nt p is the number of true positive samples, N f n - number of false negative samples and N f p - number of false positive samples. The F1 scores of the logistic regression classifier for each channel of interest are shown at the bottom row of figure
5.3. The LR classifiers are much better at recognising reaction ErrPs corresponding
to the FT rather than observation ErrPs. For the observation ErrPs each of the trained
classifier performs slightly better than chance.
Since one of the main disadvantage of surface EEG is noise, it is interesting to
examine how the LR classifier copes with noise. An extreme case of noisy data, but
quite likely, is to attempt to classify an observation ErrP with a classifier trained on
response ErrP and vice versa. Therefore the classifiers for each channel trained on the
FT were tested on the RAT and vice versa. Both the accuracy and the F1 score results
for such cross-classification are shown in figure 5.4. The performance for either of the
tasks does not change significantly meaning that the LR results in robust classifiers.
Taking into account the fact that the F1 score is relatively low and the resistance to
noise it could be concluded that LR model has most probably underfitted the training
data.
44
Logistic Regression Classifier
ErrP classification - RAT
90
85
85
80
80
Accuracy, %
Accuracy, %
ErrP classification - FT
90
75
70
65
70
65
60
60
55
55
50
50
AF3 F7
F3
F4
F8 AF4
0.7
0.7
0.65
0.65
0.6
0.6
F1 Score
F1 Score
75
0.55
F3
F4
F8 AF4
AF3 F7
F3
F4
F8 AF4
0.55
0.5
0.5
0.45
0.45
0.4
AF3 F7
0.4
AF3 F7
F3
F4
Channel
F8 AF4
Channel
Figure 5.3: Top row: The classification accuracy achieved on the test set for each task
FT (left) and RAT (right) using logistic regression. Bottom row: The F1 score achieved
by the logistic regression classifiers trained for each channel and each task.
5.3. Logistic regression classifier
45
Logistic Regression Classifier
ErrP cross-classification - RAT
85
85
80
80
Accuracy, %
Accuracy, %
ErrP cross-classification - FT
75
70
65
70
65
60
60
AF3 F7
F3
F4
F8 AF4
0.65
0.65
0.6
0.6
F1 Score
F1 Score
75
0.55
0.5
0.45
0.45
F3
F4
Channel
F8 AF4
F3
F4
F8 AF4
AF3 F7
F3
F4
F8 AF4
0.55
0.5
AF3 F7
AF3 F7
Channel
Figure 5.4: Top row: The classification accuracy achieved on the test set for the FT
using a LR classifier trained on the RAT (left) and for the RAT using a classifier trained
on the FT (right). Bottom row: The F1 score achieved during cross-classification.
46
5.4
Mixture of Gaussians classifier
The second classifier which was trained on the data was a generative mixture of Gaussians classifier. It was trained on the filtered single trial ERPs data with reduced dimensionality. The number of mixtures Nk was kept the same for both error and no error
classes and it was determined using a model selection procedure. Values between 1
and 6 were tried, as suggested by [38], and it was found out that Nk = 4 gives the
highest likelihood results. Since, fitting a MOG distribution to the data is not a convex
problem, the EM algorithm was run 100 times and the best solution was selected. The
results obtained for both tasks by training individual classifiers for each channel are
shown in figure 5.5. It could be seen that the MOG classifiers perform slightly better
than random guess only for some of the channels. As noticed before, the RAT is more
difficult to learn than the FT. The obtained results are worse than the ones reported
in [38], which could be attributed to the lower quality signal provided by the Emotiv
EPOC headset. The results for cross-classification as described in the previous section
are shown in figure 5.6. Unlike to the LR, MOG classifiers seem to be more sensitive
to noise, which is expected given the fact that they model class properties instead of
boundary characteristics. None of the MOG classifiers trained on the RAT could generate even 1 true positive prediction for the FT as seen on the bottom left plot in figure
5.6.
5.5
Hidden Markov model classifier
The final classifier type considered is a generative classifier which models the class
conditional distribution using a continuous output hidden Markov model. Each HMM
was trained on the extracted 1 second long ERP segments, meaning that the sequences
to be learnt are 128 samples long. First, model selection was performed in order to
determine the number of hidden states and the number of Gaussian mixtures in the
emission probability distributions. Both the number of states Q and the number of
mixtures M were varied between 2 and 20, thus generating 400 different models. Based
on how well each model explained the data it was found out that the most optimal
choice is Q = 4 and M = 8. An HMM was fitted for each class and channel and for
every fit the EM algorithm was run 20 times and the best solution was considered as
the final one. The accuracy and the F1 score achieved using the HMM classifier are
shown in figure 5.7. It can be seen that the performance is quite poor - none of the
5.5. Hidden Markov model classifier
47
Mixture of Gaussians Classifier
90
85
85
80
80
Accuracy, %
Accuracy, %
90
75
70
65
70
65
60
60
55
55
50
50
AF3 F7
F3
F4
F8 AF4
0.7
0.7
0.65
0.65
0.6
0.6
F1 Score
F1 Score
75
0.55
F3
F4
F8 AF4
AF3 F7
F3
F4
F8 AF4
0.55
0.5
0.5
0.45
0.45
0.4
AF3 F7
0.4
AF3 F7
F3
F4
Channel
F8 AF4
Channel
FT (left) and RAT (right) using mixture of Gaussians classifier. Bottom row: The F1
score achieved by the MOG classifiers trained for each channel and each task.
48
Mixture of Gaussians Classifier
50
48
48
46
46
44
44
Accuracy, %
Accuracy, %
50
42
40
38
40
38
36
36
34
34
32
32
30
30
AF3 F7
F3
F4
F8 AF4
0.6
0.6
0.5
0.5
0.4
0.4
F1 Score
F1 Score
42
0.3
0.2
0.1
0.1
0
0
F3
F4
Channel
F8 AF4
F3
F4
F8 AF4
AF3 F7
F3
F4
F8 AF4
0.3
0.2
AF3 F7
AF3 F7
Channel
using a MOG classifier trained on the RAT (left) and for the RAT using a classifier trained
5.5. Hidden Markov model classifier
49
80
75
75
70
70
Accuracy, %
Accuracy, %
80
65
60
55
60
55
50
50
45
45
40
40
AF3 F7
F3
F4
F8 AF4
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
F1 Score
F1 Score
65
0.4
0.3
F3
F4
F8 AF4
AF3 F7
F3
F4
F8 AF4
0.4
0.3
0.2
0.2
0.1
0.1
0
AF3 F7
0
AF3 F7
F3
F4
Channel
F8 AF4
Channel
FT (left) and RAT (right) using a hidden Markov model classifier. Bottom row: The F1
score achieved by the HMM classifiers trained for each channel and each task.
50
80
70
70
Accuracy, %
Accuracy, %
80
60
50
40
50
40
30
30
AF3 F7
F3
F4
F8 AF4
0.6
0.6
0.5
0.5
0.4
0.4
F1 Score
F1 Score
60
0.3
0.2
0.1
0.1
0
0
F3
F4
Channel
F8 AF4
F3
F4
F8 AF4
AF3 F7
F3
F4
F8 AF4
0.3
0.2
AF3 F7
AF3 F7
Channel
using a HMM classifier trained on the RAT (left) and for the RAT using a classifier trained
5.6. Classifiers comparison
Classifier
51
Reaction ErrP
Observation ErrP
Channel
Accuracy
F1 score
Channel
Accuracy
F1 score
LR
F8
86.97
0.6061
AF4
69.66
0.5172
MOG
F8
60.00
0.5942
AF4
57.44
0.5297
HMM
F8
73.35
0.2710
F3
51.20
0.3973
Figure 5.9: Comparison between the best performing classifiers of each type
trained classifiers reaches a better performance than random guess according to the
F1 score. Potential reasons for that are discussed in the next section. The results for
cross-classification are shown in 5.8.
5.6
Classifiers comparison
The results achieved by the best performing classifiers for each of the 3 types are
summarised in figure 5.9. Logistic regression seems to be the most suitable approach
for the detection of both reaction ErrPs and observation ErrPs. It should be noted
that logistic regression is the only discriminative classifier amongst the 3 considered.
Since the EEG data is noisy, it is difficult to capture the key characteristics of each
class which a generative classifier attempts to do. A discriminative classifier models
the difference between the classes which seems to be beneficial in the given problem
of ErrP classification. However, as mentioned before, it is very likely that the logistic
regression model underfits the data. In subsequent analysis a multilayer perceptron
will be also trained on the problem.
There are several factors which could cause the poor performance of the HMM
classifier. First of all, the length of the analysed 1 second segments (128 samples) is
really small, compared to other reported applications of HMMs to EEG signals which
use sequences of approximately 10 seconds [30]. Secondly, EEG signals are know to
be chaotic and non-stationary, therefore a more complex model could be needed to
capture the dynamics of ERP signals.
The overall classification accuracy is not satisfactory and future work should address that issue by using other models for classification. However, it is naive to expect
close to perfect accuracy and so in order to close the HRI control loop learning from
noisy reward signals would be required.
Chapter 6
Conclusion
6.1
Summary
This thesis reported results on the usage of the Emotiv EPOC headset to detect error
related event potentials and explored several simple methods for classification. It was
shown that both observation ErrPs and response ErrPs can be detected with the Emotiv
EPOC headset. A channel selection method was proposed which measures the extent to
which the signals generated from each channel could be used for a later classification.
As expected from previous research, channels located on the frontal lobe (AF3, F7, F3,
F4, F8, AF4) detect the strongest ErrPs.
Three types of classifiers were trained to classify single trial ErrP as being evoked
from an event with or without error. The first classifier was a discriminative logistic
regression classifier and it achieved the best classification performance (accuracy of
86.97% and F1 score of 0.6). A generative classifier modelling the class conditional
densities with a mixture of Gaussians was also trained on the data, but its performance was not satisfactory. As the last classifier, a continuous hidden Markov model
was trained on the ERPs for each condition (error and no error), trying to exploit the
dynamics of the signal. However, the HMM classifier performed even worse than a
random guess. Therefore, from those three relatively simple classifiers the logistic
regression is the most suitable method for the given problem.
The Emotiv EPOC headset was used throughout all of the experiments. Despite
the minor problems which were encountered as discussed in section 4.4, the headset
is a remarkable device which provides numerous opportunities for the research and
development of neural based human-robot interfaces.
53
54
Chapter 6. Conclusion
6.2
Future work
Since this is only a pilot study for a subsequent PhD in the field of human-robot interaction there are many directions in which the presented work could be developed and
improved.
First of all, other types of classifiers should be investigated in order to improve both
the sensitivity and the precision rate of classification. Suitable candidates are a support vector machine and a multilayer perceptron. The poor performance of the discrete
state HMMs was surprising, and so a discriminative classifier using HMMs such as
the IOHMM should be tested. Furthermore, only principal components analysis was
used for dimensionality reduction, but maybe a method such as independent components analysis will be more suitable. Another idea is to attempt feature extraction and
detection in the frequency domain instead of the time domain.
Second, despite the fact that the importance of the neuroergonomic approach was
highlighted, the conducted experiments were in artificially controlled environment.
This was imposed mainly by the time constraints of the for the project. The next step
would be to perform an experiment similar to the robotic arm task, with an actual robot
in a more realistic environment. There are many issues which need to be resolved in order to that. The most difficult one is that the actual robot will not move instantaneously
as the simulated arm and there will be no distinguishable event which to synchronise
the EEG data to. Therefore, a method for asynchronous ERP detection will have to be
developed.
Last, but not least, other types of ERPs should be extracted from the EEG signals
which would enable a more robust intention inference. Other sensing modalities could
be used as well - for example eye-tracking, surface EMG, RGBD data from the scene.
It is not realistic to expect perfect classification rate for either of those modalities, but
fusing them together should significantly improve the robustness of the human-robot
interface.
Bibliography
[1] C. C. Kemp, A. Edsinger, and E. Torres-Jara, “Challenges for Robot Manipulation in Human,” no. March, pp. 20–29, 2007.
[2] E. Guizzo, “Rescue-robot show-down,” Spectrum, IEEE, vol. 51, pp. 52–55, January 2014.
[3] C. Lenz, G. Panin, T. Roder, and M. Wojtczyk, “Hardware-assisted Multiple Object Tracking for Human-Robot-Interaction,” Human-Robot Interaction (HRI),
2009 4th ACM/IEEE International Conference on, pp. 283–284, 2009.
[4] J. A. Rivera-Bautista, A. C. Ramirez-Hernandez, V. a. Garcia-Vega, and
A. Marin-Hernandez, “Modular control for human motion analysis and classification in Human-Robot interaction,” 2010 5th ACM/IEEE International Conference
on Human-Robot Interaction (HRI), pp. 169–170, Mar. 2010.
[5] X. Perrin, R. Chavarriaga, F. Colas, R. Siegwart, and J. D. R. Millán, “Braincoupled interaction for semi-autonomous navigation of an assistive robot,”
Robotics and Autonomous Systems, vol. 58, pp. 1246–1255, Dec. 2010.
[6] F. Galán, M. Nuttin, E. Lew, P. W. Ferrez, G. Vanacker, J. Philips, and J. D. R.
Millán, “A brain-actuated wheelchair: asynchronous and non-invasive Braincomputer interfaces for continuous control of robots.,” Clinical neurophysiology
: official journal of the International Federation of Clinical Neurophysiology,
vol. 119, pp. 2159–69, Sept. 2008.
[7] K. LaFleur, K. Cassady, A. Doud, K. Shades, E. Rogin, and B. He, “Quadcopter control in three-dimensional space using a noninvasive motor imagerybased brain–computer interface,” Journal of neural engineering, vol. 10, no. 4,
p. 046003, 2013.
55
56
Bibliography
[8] S. Chiappa, Analysis and Classification of EEG Signals using Probabilistic Models for Brain Computer Interfaces. PhD thesis, Ecole Polytechnique Federale de
Lausanne, 2006.
[9] S. T. Grafton and C. M. Tipper, “Decoding intention: a neuroergonomic perspective.,” NeuroImage, vol. 59, pp. 14–24, Jan. 2012.
[10] R. Parasuraman and M. Rizzo, Neuroergonomics: The brain at work. 2006.
[11] R. K. Mehta and R. Parasuraman, “Neuroergonomics: a review of applications to
physical and cognitive work.,” Frontiers in human neuroscience, vol. 7, p. 889,
2013.
[12] M. Anderson, “Embodied cognition: A field guide,” Artificial Intelligence,
vol. 149, pp. 91–130, 2003.
[13] J. Touryan, G. Apker, B. J. Lance, S. E. Kerick, A. J. Ries, and K. McDowell,
“Estimating endogenous changes in task performance from EEG,” Frontiers in
Neuroscience, vol. 8, June 2014.
[14] M. F. Bear, B. W. Connors, and M. A. Paradiso, Neuroscience: Exploring the
brain (3rd ed.). 2007.
[15] R. Croft and R. Barry, “Removal of ocular artifact from the EEG: a review,”
Neurophysiologie Clinique/Clinical Neurophysiology, 2000.
[16] A. Burgess and J. Gruzelier, “Short duration synchronization of human theta
rhythm during recognition memory,” Neuroreport, 1997.
[17] L. T. Trujillo, M. A. Peterson, A. W. Kaszniak, and J. J. Allen, “Eeg phase synchrony differences across visual perception conditions may depend on recording
and analysis methods,” Clinical Neurophysiology, vol. 116, no. 1, pp. 172 – 189,
2005.
[18] P. Brown, S. Salenius, J. C. Rothwell, and R. Hari, “Cortical correlate of the piper
rhythm in humans,” Journal of Neurophysiology, vol. 80, no. 6, pp. 2911–2917,
1998.
[19] M. Rubinov and O. Sporns, “Complex network measures of brain connectivity:
Uses and interpretations,” NeuroImage, vol. 52, no. 3, pp. 1059–1069, 2010.
Bibliography
57
[20] H. G. Vaughan, L. D. Costa, and W. Ritter, “Topography of the human motor
potential.,” Electroencephalography and clinical neurophysiology, vol. 25, pp. 1–
10, 1968.
[21] W. G. Walter, R. Cooper, V. J. Aldridge, W. C. Mccallum, and a. L. Winter,
“Contingent Negative Variation: an Electric Sign of Sensorimotor Association
and Expectancy in the Human Brain.,” Nature, vol. 203, pp. 380–384, 1964.
[22] J. Polich and A. Kok, “Cognitive and biological determinants of P300: An integrative review,” 1995.
[23] W. J. Gehring, B. Goss, and M. G. H. Coles, “A neural system for error detection
and compensation,” 1993.
[24] M. Falkenstein, J. Hoormann, S. Christ, and J. Hohnsbein, “Erp components on
reaction errors and their functional significance: a tutorial,” Biological Psychology, vol. 51, no. 23, pp. 87 – 107, 2000.
[25] S. Hoffmann and M. Falkenstein, “Predictive information processing in the brain:
errors and response monitoring.,” International journal of psychophysiology :
official journal of the International Organization of Psychophysiology, vol. 83,
pp. 208–12, Feb. 2012.
[26] S. Nieuwenhuis, C. B. Holroyd, N. Mol, and M. G. Coles, “Reinforcementrelated brain potentials from medial frontal cortex: origins and functional significance,” Neuroscience & Biobehavioral Reviews, vol. 28, no. 4, pp. 441 – 448,
2004.
[27] H. T. van Schie, R. B. Mars, M. G. H. Coles, and H. Bekkering, “Modulation of
activity in medial frontal and motor cortices during error observation.,” Nature
neuroscience, vol. 7, pp. 549–54, May 2004.
[28] P. Ferrez and J. d. R. Millan, “Error-related EEG potentials generated during simulated braincomputer interaction,” Biomedical Engineering, IEEE . . . , vol. 55,
pp. 923–9, Mar. 2008.
[29] L. F. Nicolas-Alonso and J. Gomez-Gil, “Brain Computer Interfaces, a Review,”
Sensors, vol. 12, pp. 1211–1279, 2012.
58
Bibliography
[30] S. Solhjoo, “Classification of chaotic signals using HMM classifiers: EEG-based
mental task classification,” Proc. European Signal . . . , pp. 18–21, 2005.
[31] V. Bostanov, “BCI competition 2003 - Data sets Ib and IIb: Feature extraction
from event-related brain potentials with the continuous wavelet transform and
the t-value scalogram,” IEEE Transactions on Biomedical Engineering, vol. 51,
pp. 1057–1061, 2004.
[32] I. Iturrate, L. Montesano, and J. Minguez, “Robot reinforcement learning using
EEG-based reward signals,” in 2010 IEEE International Conference on Robotics
and Automation, pp. 4822–4829, IEEE, May 2010.
[33] P. Ferrez and J. Millán, “You are wrong!Automatic detection of interaction errors
from brain waves,” 2005.
[34] M. Krauledat, G. Dornhege, B. Blankertz, F. Losch, G. Curio, and K.-R. Müller,
“Improving speed and accuracy of brain-computer interfaces using readiness potential features.,” Conference proceedings : ... Annual International Conference
of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in
Medicine and Biology Society. Conference, vol. 6, pp. 4511–4515, 2004.
[35] R. Chavarriaga and J. del Millán, “Learning from EEG error-related potentials in
noninvasive brain-computer interfaces,” Neural Systems and Rehabilitation Engineering, vol. 18, no. 4, pp. 381–388, 2010.
[36] C. Vi and S. Subramanian, “Detecting Error-Related Negativity for Interaction
Design,” in Proceedings of the 2012 ACM annual conference on Human Factors
in Computing Systems - CHI ’12, (New York, New York, USA), p. 493, ACM
Press, May 2012.
[37] M. Duvinage, T. Castermans, T. Dutoit, M. Petieau, T. Hoellinger, C. D.
Saedeleer, K. Seetharaman, and G. Cheron, “A P300-based Quantitative Comparison between the Emotiv Epoc Headset and a Medical EEG Device,” Biomedical
Engineering / 765: Telehealth / 766: Assistive Technologies, 2012.
[38] P. Ferrez, Error-Related EEG Potentials in Brain-Computer Interfaces. PhD thesis, 2007.
[39] K. Murphy, Machine learning: a probabilistic perspective. 2012.
Bibliography
59
[40] H. L. H. Lee and S. C. S. Choi, “PCA+HMM+SVM for EEG pattern classification,” Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings., vol. 1, no. 2, pp. 1–4, 2003.
[41] M. E. Maier, G. Di Pellegrino, and M. Steinhauser, “Enhanced error-related negativity on flanker errors: Error expectancy or error significance?,” Psychophysiology, vol. 49, pp. 899–908, 2012.

Event Related Potentials in the Electroencephalogram for Human

Transcription

Similar documents

Epilepsy - Identifying Epileptical Sources in the Human Brain

Electroencephalography (EEG) Sensor Data Sheet

Electroneurodiagnostic - Institute of Health Sciences

Neural Basis of the Ventriloquist

HM1700 BLUETOOTH® HEADSET

Fundamentals of Electroencephalography, Magneto

Luchtige Casusbespreking Een beetje Schudderig na Reanimatie

Avaya 10U100 Cordless Headset Telephone