LUCRARE DE LICENŢĂ

Universitatea Politehnica din Bucureşti
Facultatea de Automatică şi Calculatoare
Departamentul de Automatică şi Ingineria Sistemelor
LUCRARE DE LICENŢĂ
Interfaţă Creier-Calculator
(Brain-Computer Interface)
Absolvent
Roventa Madalin
Coordonator
Prof.dr.ing. Dumitru Popescu
Mdc. Claudine Lecoq
Bucuresti, 2013
Contents:
Foreword........................................................................................3
1. Introduction
1.1. Brain-Computer Interface...................................................1
1.2. Paper objectives..................................................................2
1.3. BCI over time......................................................................3
1.4. Introductory notions............................................................4
1.5. Summary.............................................................................1
2. EEG signal processing and classification.......................................1
2.1. Types of BCI.......................................................................1
2.2. Electroencephalography......................................................1
2.3. Neurophysiological signals used to drive a BCI.................1
2.3.1. Evoked signals...........................................................1
2.3.2. Spontaneous signals...................................................1
2.4. Conclusions EEG signals....................................................1
2.5. Preprocessing......................................................................1
2.6. Feature extraction................................................................1
3. Fuzzy classification........................................................................1
3.1. Fuzzy classification. Manual rules..........................................1
3.1.1. Inputs/Outputs............................................................1
3.1.2. Fuzzy Inference System.............................................1
3.2. Fuzzy classification. Automatic rules.....................................1
3.2.1. Clustering of training data.........................................1
3.2.2. Generation of the fuzzy rules based
on the clustered data..................................................1
3.2.3. Rules Optimization....................................................1
4. Studying the Use of Fuzzy Inference Systems for
Motor Imagery based BCI..............................................................1
4.1. Studying the use of FIS for a motor
imagery based BCI, using artificial data sets.................................1
4.2. Studying the use of FIS for a motor
imagery based BCI, using real data sets.........................................1
5. Conclusions. Improvements...........................................................1
6. Appendix A.
7. References
2|P ag e
Foreword
This project has been assigned to me by the BCI research team from
Polytech Lille. The initial project was to develop, only the fuzzy inference system
for the classification part, with both manual and automatic rules, because they
wanted to see if you can add apriori knowledge to the fuzzy system with automatic
rules.
Because I couldn‟t test my system without the other two components, I
decided to build the pre-classification and feature extraction parts, but those are
more rudimentary than the core of the BCI, the fuzzy inference system, because of
the lack of time.
So my paper will focus on the classification part, but I will present also the
pre-classification and feature extraction part.
3|P ag e
Chapter I. Introduction
1.1.
Brain-Computer Interface
A BCI is a communication system which enables a person to send
commands to an electronic device, only by means of voluntary variations of his
brain activity [1][2][3][4][5]. This term refers to a interface that takes signals from
the brain and it delivers them to an external piece of hardware.
Various types of brain-computer interfaces were developed along the time
for different purposes. Some of the earliest BCI were used just for recording signals
from the brain. First real BCI applications were neuroprostheses, developed for
restoring damaged hearing, sight and movement. Thanks to the remarkable cortical
plasticity of the brain, signals from implanted prostheses can, after adaptation, be
handled by the brain like natural sensor or effector channels [6].
The ultimate purpose of a direct brain-computer interface is to allow an
individual, who suffers from a disease that affects a motor function of his body to
have effective control over a hardware component that replaces that motor function,
simply by brain control. Such an interface would increase an individual‟s
independence, leading to an improved quality of life.
4|P ag e
1.2.
Paper Objectives
Despite the rapid grown from the 90‟s till now of the BCI field, it still
remains an uncharted research field. The brain is the most complex organism known
to humans, so it‟s not very easy to transpose functions of the brain in a computerbased machine, but we are on the right track, thanks to numerous achievements
already obtained.
In this paper I will present the following topics:
1. Designing a fuzzy inference system (FIS) for the development of a braincomputer interface, asynchronous, non-invasive, based on the detection of
real or imaginary movements, making distinction between the active and rest
state. I chose the fuzzy classification method because fuzzy sets are known to
be the most appropriate mathematical method to tackle an uncertain (fuzzy)
problem, as the BCI designing is.
The effectiveness of fuzzy classifiers have been proved on other pattern
recognition problems: hand-writing recognition [7], ElectroMyoGraphy
(EMG) classification [8] or even brain research, for EEG monitoring [9][10].
Also, another positive aspect of the fuzzy classification is that all the output
data can be easily understood and we can gather knowledge from such a
system. In the last period of BCI developing, all the classification methods
were based on algorithms that use a training set, so that after, it can
recognize a mental state. Of course, from those algorithms, we can only state
the mental state, but we cannot see the way the algorithm got to the
5|P ag e
conclusion, so that‟s why we cannot gather knowledge. Those types of
algorithms comport like a black box, you can only see the entries and the
outputs. A fuzzy system allows you to see the connections that were made to
get to a conclusion, so, the data is interpretable, and we can gain insights on
the brain dynamics. This represents the feature of data interpretability of
fuzzy systems.
2. Designing a whole BCI architecture for the recognition of imagined left and
right hand movements from two sets of experimental data:
 Artificial created data, based on common features responses from
individuals.
 Real data, taken from an Open Vibe scenario.
1.3.
BCI over time
Since the first experiments of ElectroEncephaloGraphy (EEG) on humans
by Hans Berger in 1929 [11], the idea that brain activity could be used as a
communication channel has rapidly emerged.
Berger‟s first recording device was rudimentary. It was composed of silver
foils attached to the patients head. The results of the tests were very disappointing,
but it represented the beginning of the brain signals recording. More sophisticated
measuring devices, such as the Siemens double-coil recording galvanometer led to
success, so EEG permitted studying the brain activity. However, only in 1973
appeared the first prototype of BCI, created by Dr. Vidal [12].
The boom of the BCI field was very late, in the 90‟s. Then, more and more
laboratories started to develop studies in this research area. Also, the first BCI
competitions arrived, were the researchers involved in BCI field could test their
systems and have a perspective what techniques of designing an interface are most
efficient throughout the world. Since then, more and more interfaces were proposed
especially in the medical field, but also in virtual reality.
Nowadays, researchers started the development of another Brain-Computer
Interface, which is still at a theoretical level, that aims at the development of
artificial intelligence. They state that all the brain functions could be uploaded in a
BCI, leading to an existence of a brain without a body. Of course, this represents the
climax of such a technology, and humans are far away from this objective.
6|P ag e
1
4
0
Y
e
a
r
r
s
Fig.1 A Lippmann electrometer is a device for
detecting small rushes of electric current and was
invented by Gabriel Lippmann in 1873. This device
was used in the first practical ECG machine.
Fig.2 “Pocket BCI”- the first commercially available
interface
1.4.
brain-computer
Introductory notions
Designing a BCI is a very difficult task. It requires knowledge in computer
science, neurology and signal processing. Usually, two phases are required to use a
brain-computer interface:


An offline phase- represents a training phase which allows the
researcher to gather information about the system, allowing him to
calibrate it. This phase is required because every individual has
different brain activity as a response to a stimulus.
An online phase- in this phase, the training is over and the interface is
used to recognize different mental states and translates them into
commands for an external hardware.
An online phase is composed usually from 6 steps, in this order:
1. Brain activity measurement: this step consists of measuring the brain activity
(in this case EEG measuring technology).
2. Preprocessing: the EEG signal has a lot of perturbations incorporated in it, so
in this step we have to denoise and clean the data to
enhance the power of the signals that interests us.
3. Feature extraction: is the step where we
extract from the signal only the features that interests
us.
7|P ag e
4. Classification: the classification step assigns a class to a set of features
extracted from the signals [13]. This class corresponds to an identified
mental state.
5. Translation into a command: after the class identification, a command
representing the class is given to an external hardware,
6. Feedback: this step provides the user a feedback about the mental state that
was identified. This aims at helping the user controlling his brain activity [1].
The whole architecture is summarized in Fig.3
Fig.3 Architecture
online BCI
1.5.
of
an
Summary %%%%mai am de lucru
The first chapter allowed us to make an opinion about what a brain-computer
interface may stand for. In the second chapter I will introduce the scenario that is
needed to understand what BCI means, more exactly, I will present all the
components of a brain-computer interface, starting from acquisition, preprocessing,
feature extraction, classification and finishing with command translation. Also, I will
present the different types of BCI that can be developed nowadays . This represents
the introductory part. After having established the grounds for the topic, the paper
will continue by presenting the fuzzy classification part, which represents the third
chapter. The fuzzy classifiers are particularly attractive for BCI design, because they
are "perfectly suitable to deal with the natural fuzziness of real-life classification
problems"[14], as Bezdek has highlighted.
The fuzzy classifying part it‟s also divided in two: the construction of a fuzzy
system with hand-made rules and one with automatic rules, based on Chiu‟s
algorithm. This part is divided in two, because fuzzy classifiers are known to be
suitable for adding a priori knowledge, under the form of hand-made fuzzy rules, so
the best results obtained in the hand-made fuzzy system can be added to the
automatic fuzzy inference system.
Chapter four of the project consists in designing a whole BCI architecture. In
this purpose, first, I specified the methods that I used, and after that their part in the
interface. This chapter is also divided in three parts: preprocessing, feature
extraction and the classification part. For classification, I used the fuzzy classifiers, ,
explained more thorough in the second chapter.
8|P ag e
Chapter V will summarize the results, conclusions and improvements for our
interface.
Chapter II. EEG signal processing and
classification
The first part of the chapter represents an introduction in the BCI world. It
details the main types of brain-computer interfaces and gives definitions for
phenomens associated to this type of brain research.
The second part consists of detailing the fuzzy inference systems and
offering a summary about Chiu‟s classification algorithm.
2.1.
Types of BCI
1. Regarding the type of method used to extract brain signals:

9|P ag e
Invasive BCI: research, mostly, has targeted repairing damaged sight
and providing new functionalities to paralyzed people. An invasive
BCI is implanted directly in the grey matter during a neurosurgical
intervention. The quality of the signal is very good because, it is
focused only on the critical areas, where we gather the information,
so the signal isn‟t very noisy. The down part of this method is that
every foreign object inside our body is treated like an enemy, so this
types of BCI are prone to scar-tissue build-up. This will make the
signal weaker over time, and it‟s possible to disappear completely. Of
course, the main disadvantage is that this procedure carries a very
high degree of risk for the patient.
 Partially invasive BCI: are implanted inside the skull, but not in the
grey matter. This will prevent scar-tissue build-up. The signal is
worse than in case of an invasive BCI, but better than a non-invasive
one,
because the skull doesn‟t
deflect
the signal.
Electrocorticography (ECoG) measures brain activity in a similar
way as Electroencephalography (EEG), except that the electrodes are
embedded in plastic and are placed beneath the dura mater and above
the cortex.
 Non-invasive BCI: as the name suggested, the electrodes are placed
above the skull, not beeing needed of any type of surgery. This type
of interface is the most common in our days, because it doesn‟t carry
any risk for the patient, but the signal received is very noisy and
deflected from the skull. The extraction method carries the name of
Electroencephalography (EEG). This project is based on a noninvasive interface.
2. Regarding the independence degree of the interface from the motor functions
of the user:


Dependent BCI: this type requires that the user controls some of his
motor functions during tests, but for severe paralyzed peoples, who
can‟t control their motor functions, this is not a solution.
Independent BCI: this type doesn‟t require any type of motor
control from the patient, so it‟s suitable for cases where motor control
doesn‟t exists.
3. Regarding the independence degree of the interface from stimulus


10 | P a g e
Synchronous Interfaces: with a synchronous BCI, the user can
interact with the targeted application only during specific time
periods, imposed by the system [15][16][1]. In this case the user has
to execute a movement, when a stimulus appears (visual or acoustic).
If the movement is executed in the specified interval, then the user
will have a feedback, if not, nothing will happen. So, the main
advantage of synchronous interfaces is that we always know when a
movement is happening and we can have a good identification of the
signal which represents a movement, but the disadvantage is that the
user is constraint to perform a motor function only in a specific
interval, so it‟s not a method for long term use.
Asynchronous Interfaces: in this case, the user can perform a motor
task anytime he wants, and the interface must response. This type is
also called a “self-paced” BCI. The principle that is behind this
interface is that the brain signal is permanently analyzed to
determine if the user is executing a motor task or is resting. Of
course, if he is performing a task, then the interface must decide what
kind of motor task the user is executing, hence the degree of
complexity of an asynchronous interface. Designing this type of
BCI‟s represents one of the main goals of this field‟s research, but it
is also one of the most challenging problems. In this paper an
asynchronous interface is presented.
In this paper, we focus our attention on a non-invasive, independent,
asynchronous brain-interface.
2.2. EEG (ElectroEncephalography)
The classified signal, in this project is an EEG signal, so it is important to
thoroughly understand the principles that are laying at the foundation of
Electroencephalography.
Electroencephalography measures the electrical activity generated by the
brain using electrodes placed on the scalp [17]. The man who invented this
technique is Hans Berger in 1929. He decided to name this technique
“electroencephalogram” [11].
Signals recorded from EEG have a very weak amplitude, in the order of
some micro volts, so the signal must be amplified before processing it. EEG
measurements are performed with electrodes attached to the head, in number, from 1
to 256, attached in different areas of the scalp.
Electroencephalography (EEG) is the most studied potential non-invasive
interface, mainly due to its fine temporal resolution, ease of use, portability and low
set-up cost [6], but how every good thing comes with a bad part also, the EEG signal
is very susceptible to noise and the use of electroencephalography in a braincomputer interface always comes with an extensive training period for the subject,
before users can exploit real results from it.
Fig.4 Recordings of
an electroencephalogram
11 | P a g e
brainwaves
produced
by
For example, experiments conducted by Niels Birbaumer at the University of
Tübingen in Germany, in the mid-90‟s, where he trained paralysed people to selfregulate their slow cortical potentials to such an extent that these signals could be
used as a binary signal to control a computer cursor [6]. The process was slow,
requiring more than an hour for patients to write 100 characters with the cursor,
while training often took many months.
A type of parameter that is specially interesting for these paper is oscillatory
activity, where research was concentrated on producing technologies that can allow
the user to choose the brain signals they found easiest to operate a BCI, including
mu and beta rhythm. These oscillations are different in terms of spatial and
spectral localization, and are called rhythms.
There are 6 different rhytms:
1. Delta rhythm: (1-4 Hz) slow rhythm.
2. Theta rhythm: (4-7 Hz) a more faster rhythm than delta.
3. Alpha rhythm: (8-12 Hz).
4. Mu rhythm: (8-13 Hz) is located in the motor and sensorimotor cortex. This
rhythm is activated when a person performs movements. It‟s one of the two
rhythms that are investigated in this paper.
5. Beta rhythm: (13-30 Hz) it‟s a fast rhythm that detects movements, so this is
the second type of rhythm that I used in this paper.
6. Gamma rhythm: (above 30 Hz) associated with cognitive functions.
Another parameter in brain-computer interfaces is the method of feedback
used, as shown in the P300 signals. The biofeedback method requires learning to
control brainwaves so the resulting brain activity can be detected.
In this paper, we use the mu and beta rhythm to operate the BCI.
2.3. Neurophysiological signals used to drive a BCI
The principle that lies at the basis of any interface is identifying several
neurophysiological signals that are specific to a motor activity, in order to associate
a command to each of these signals (brain patterns). These signals can be divided
into two main categories [18][1]:
1. Evoked signals: are generated by the user as a response at a stimulus,
unconsciously. Are called Evoked Potentials(EP)
2. Spontaneous signals: are generated voluntarily by the subject, without
receiving any stimulus. In this paper, we will deal with spontaneous signals.
2.3.1. Evoked signals:
12 | P a g e
An evoked potential or evoked response is an electrical potential recorded
from the nervous system of a human or other animal following presentation of
a stimulus, as distinct from spontaneous potentials as detected by EEG, EMG, or
other electrophysiological recording method [6]. Among the current BCIs, the
systems based on EPs have been studied for a long period since the 1970s [19]. This
is because it‟s easy to configure such a system, it has a high information transfer rate
and the training period for the user is reduced. The BCI research has focused on
visual evoked potentials (VEP), because it‟s easier for the user to execute a motor
task at the sight of a visual stimulus.
VEP
VEPs reflect the visual information-processing mechanism in the brain. In
1934, Adrian and Matthew noticed potential changes of the occipital EEG can be
observed under stimulation of light. Ciganek developed the first nomenclature for
occipital EEG components in 1961. During that same year, Hirsch and colleagues
recorded a visual evoked potential (VEP) on the occipital lobe (externally and
internally), and they discovered amplitudes recorded along the calcarine fissure were
the largest. In 1965, Spehlmann used a checkerboard stimulation to describe human
VEPs. An attempt to localize structures in the primary visual pathway was
completed by Szikla and colleagues. Halliday and colleagues completed the first
clinical investigations using VEP by recording delayed VEPs in a patient with
retrobulbar neuritis in 1972. A wide variety of extensive research to improve
procedures and theories has been conducted from the 1970s to today [6].
VEP Stimuli:
On a large scale, there are 3 types of stimuli used: the diffuse flashing light
and the checkerboard and grating patterns. In BCI research, the most popular is the
checkerboard, because of it‟s usage in P300 experiments.
VEP Electrode Placement
The placement of the electrodes is very important for getting free artifact
results. The International Society for Clinical Electrophysiology of Vision (ISCEV)
standards for VEP testing recommend that electrode placement follow the 10-20
system (Harding, G.F., Odom,1996). The active recording electrodes are placed over
the active source, which for visual evoked potentials is the visual (occipital) cortex.
A reference electrode is placed over an area unresponsive to visual stimuli and a
ground electrode connects a second inactive area to the ground terminal of the
equipment.
VEP extraction
The electrical signal is recorded from the surface of the scalp. The smallamplitude evoked potentials (1-20 µV) are embedded in the larger amplitude
potentials. To extract information from the experiment, repeated averaged responses
13 | P a g e
must be used. The VEP are time-locked to a stimulus, so when to the subject is
presented the stimulus, you will know after what period of time the electrical
activity in his brain should change.
The VEP nomenclature is determined by using capital letters stating whether
the peak is positive (P) or negative (N) followed by a number which indicates the
average peak latency for that particular wave. For example, P50 is a wave with a
positive peak at approximately 50 ms following stimulus onset.[wikipedia]
Some specific VEPs are:












Sweep visual evoked potential
Binocular visual evoked potential
Chromatic visual evoked potential
Hemi-field visual evoked potential
Flash visual evoked potential
LED Goggle visual evoked potential
Motion visual evoked potential
Multifocal visual evoked potential
Multi-channel visual evoked potential
Multi-frequency visual evoked potential
Stereo-elicited visual evoked potential
Steady state visually evoked potential
According to the knowledge of brain electrophysiology, VEPs corresponding
to low stimulus rates are categorized as transient VEP (TVEP), and those
corresponding to rapidly repetitive stimulations are categorized as steady-state VEP
(SSVEP) [19]. The SSVEP and the P300 potential will be the topic of the following
paragraphs.
1.SSVEP:
VEP introduce transient responses of the visual system, but using long
stimulus trains, a steady response will be produced, which can be displayed after
averaging.
Steady-state potentials are to be distinguished from transient potentials,
because their constituent discrete frequency components remain closely constant in
amplitude and phase over a long time period [20].
The SSVEPs have the same fundamental frequency (first harmonic) as the
stimulating frequency, but usually they also include higher (Regan 1989) and/or subharmonics (Herrmann 2001).
2.P300
14 | P a g e
It‟s an extensively studied stimulus in the BCI field, because it appears as a
response to meaningful, rare stimuli („oddball‟ stimuli- ; Donchin & Coles, 1988)
For example, if to the subject is presented a series of names, and every 3
seconds the subject names appears, a P3 wave is evoked, due to a meaningful
stimulus.
P3 is a positive-going wave with a scalp amplitude distribution in which it is
largest parietally (at Pz) and smallest frontally (Fz), taking intermediate values
centrally (Cz). (Fz, Cz, and Pz are scalp sites along the midline of the head.)[21].
The latency of the P3 wave is 300-1000 msec, so the name P300. The latency
depends of the stimulus complexity. The amplitude of P3 is inversely proportional to
the rareness of presentation.
2.3.2.
Spontaneous signals:
Spontaneous signals (e.g. EEG rhythms over sensorimotor cortex) do not
depend for their generation on a stimulation.
A BCI system which uses spontaneous signals as data entries generates a
control signal at a given interval of time based on the classification of EEG patterns
taken from a specific mental activity.
The most used spontaneous signals are sensorimotor rhythms
Sensorimotor rhythms:
Represents brain activity(rhythms) related to motor actions, such as foot,
hand movement. They are mainly located in the μ (≃ 8-13 Hz) and β (≃ 13-30 Hz)
frequency band and these types of rhythms can be controlled in amplitude by the
user, so that‟s why 2 strategies of controlling this phenomenon have been proposed:

Operant conditioning
A subject can learn to modify voluntarily the amplitude of his sensorimotor
rhythms through a (very) long training known as “operant conditioning”[22] [23]
[24][25]. The user is free to choose the control mental strategy. The feedback
represents the most significant part of an operant conditioning based system,
because from the feedback, the user can understand what‟s to change in his mental
activity to increase the control accuracy. Generally, a linear combined power of the
signals in the μ and β band is used to design the control system. The main drawback
of this method consists of a long training period, but after the calibration of the
system, very good results have been observed.
15 | P a g e

Motor imagery
For a user, performing motor imagery consists in imagining movements of
his own limbs [26][27][16]. The imagining of limbs movements have very good
determined spatial, frequential and temporal features. For example, the imagining of
a left or right hand movement is associated with an ERD (event-related
desynchronization) in the contra-lateral side during the movement and an
ERS(event-related synchronization) after the movement over in the μ and β rhythms.
Using this types of features, it can be determined the type of mental task the user is
trying to imagine. The advantage of such a system is that the user doesn‟t need an
extensive period of training, in some cases it works from the first try, but the
complexity of such a system is the main drawback, because it uses techniques of
advanced signal processing and machine learning algorithms.
Slow cortical potentials:
Slow Cortical Potentials (SCP) are very slow variations of the cortical
activity, which can last from hundreds of milliseconds to several seconds [28][29].
The training period for the user can last more than for an “operant conditioning”
system, but after that t can provide a more stable signal.
Non-motor cognitive tasks:
These tasks can produce specific EEG signals variations, and can be used in
operating a BCI. These tasks can represent: mental mathematical computations,
visual counting, music imagination, etc.
2.4. Conclusions EEG signals:
All the signals presented in this paper, so far, have been successfully used in
designing brain-computer interfaces, but the problem is that you cannot say that a
signal is “better” than another one in this field, because all of them have their pros
and cons. The evoked signals are unnatural, because it requires the use of a stimulus.
The spontaneous signals are natural, the user can execute the mental task
whenever he wants, but are tiring, because of the extensive training period.
However, it has been shown that using high performance signal processing
algorithms with machine learning algorithms, the training period can be reduced,
and sometimes be removed. So, that‟s why, the purpose of this paper is to build a
16 | P a g e
BCI based on spontaneous signals, more specifically motor imagery signals, which
are described thoroughly in today‟s literature.
This type of signal represents the entry in our brain-computer interface. The
next step consists of signal processing and machine learning algorithms. In other
studies conducted, this part represented a black box for the users, but now, with the
help of the fuzzy classification system, the user can understand the pathways of his
brain and adjust it to the needs of the interface, so he can learn, as well as the
machine learning algorithm.
The three following sections, preprocessing, feature extraction and
classification are dedicated to EEG signal processing. This part represent the core of
any brain-computer interface. To improve the results, you must improve this part.
However, in a BCI isn‟t compulsory to have all these three sections, they may
emerge or may be missing. For example, the preprocessing and feature extraction
part may be considered as one, or you can focus on those two parts and the
classification part be considered only as a minimal or maximal threshold.
Of course, the optimal solution is considered to be the use of all three
components, but this takes the interface to a higher degree of complexity.
In this paper, I focused my attention on the sensorimotor rhythms located
in μ and β band and the strategy for controlling this phenomena is motor
imagery.
2.5. Preprocessing
After the acquisition of the signal, this must be de-noised and must enhance
the significant information embedded in these signals. The acquired signal is very
noisy because of the “background” brain activity, which has no connection with the
experiment, and, also it is very affected by numerous artifacts, like ocular or facial
muscular activity. This signal retrieved from the artifacts has a larger amplitude than
the useful signal, so it‟s difficult to retrieve the “good” information without
damaging it.
This first step has the purpose to increase the signal-to-noise ratio. The
output will be a total different signal than the entry one.
The next paragraphs will present the main preprocessing techniques used in
BCI field.
Simple spatial and temporal filters
Temporal filters:
-low pass filter
-band pass filter
17 | P a g e
Are used to restrict the frequency to a specific domain, where we know that
the neurophysiological signals are located. For example, the BCI‟s based on
sensorimotor rhythms use a band pass filter between 8-30 Hz. This is where the µ
and β band is located, according to neurophysiological experts [30]. This filtering
method is also useful to eliminate slow variations of the EEG signal and power line
interference.
The purpose of a temporal filter in a BCI is to reduce the influence of
frequencies that are lying outside the area of interest. Such a filtering is generally
achieved using Discrete Fourier Transform (DFT) or using Finite Impulse Response
(FIR) or Infinite Impulse Response (IIR) filters [31].
Spatial filters
Similar to the temporal ones, spatial filters are used to isolate the relevant
spatial information, as we concentrate on different regions of the brain, depending
on the mental task we would like to execute.
This filtering method consists in selecting or weighting the contribution of
different electrodes.
As it is known, when we focus on hand movements, the regions of interest
are the motor and sensorimotor cortex areas, hence the use of electrodes C 3 and C4
or a weighted contribution of the electrodes surrounding for designing a spatial
filter.
Also, the electrodes used in SSVEP experiments are O1 and O2, that are
located over the visual areas [32].
Some common types of spatial filters used are: Surface Laplacian(SL) filter
and Common Average Reference(CAR).
Common spatial patterns
This method is based on the decomposition of the EEG signals into spatial
patterns [33][34][35]. These patterns are selected in order to maximize the
differences between the classes involved, once the data has been projected onto
these patterns. Determining these patterns is performed using a joint diagonalization
of the covariance matrices of the EEG signals from each class [31].
Inverse solutions
Relevant but much less used preprocessing methods for BCI are inverse
solutions. Inverse solutions are methods that attempt to reconstruct the activity in the
brain volume by using only scalp measurements and a head model [31][36][37].
Conclusions:
18 | P a g e
As showed in this section, the preprocessing part is very diversified, but
choosing the best method or combination of methods hasn‟t been identified so far. It
depends on your system‟s needs.
The preprocessing part has as entry the EEG signal and as output another
signal, that is de-noised, and has it‟s significant features enhanced, thus a better
signal-to noise ratio. This is theoretical, but practical, it‟s a very difficult job to
separate the artifacts and the background noise from the useful signal.
Nevertheless, the most appreciated and most common used method, is the
spatial filtering method. It is shown that it reduces noise drastically, so it improves
the performances of the system, if you are working with a sufficient number of
electrodes.
2.6. Feature Extraction
In this section we will discuss some of the feature extraction methods that
have received more attention in BCI systems.
Dealing with EEG signals will produce a high quantity of data. The
researchers work with a number of electrodes varying from 1 to 256 and with a
sampling frequency from 100 to 1000 Hz. This states the need of a feature extraction
method, that selects only the valuable information from data.
It is crucial to select only data that is meaningful to the system, otherwise the
classification algorithm will have entry data that has no relevance for it, making the
decision problem harder, or even impossible.
In BCI research, feature extractions methods have been divided into four
main categories:
 Time representation methods
 Frequency representation methods
 Hybrid representation methods, that includes also, time and frequency
methods
 Parametric modeling
Also, it exists other methods that aren‟t included in this categories.
Time representation methods
19 | P a g e
This extraction method use as features, temporal variations of the signal. Are
perfectly adapted to time specific processes, such as P300 or ERD, notably those
triggered by motor imagery [31].
Frequency representation methods
Frequency-based features have been widely used in signal processing
because of their ease of application, computational speed and direct interpretation of
the results. Specifically, about one-third of BCI designs have used power-spectral
features. Due to the non-stationary nature of the EEG signals, these features do not
provide any time domain information.
Hybrid representation methods
Mixed time-frequency feature extraction methods have shown that can
improve the system‟s performance. They map the one-dimensional signal into a twodimensional function of time and frequency, and are used to analyze the timevarying spectral content of the signals.
Parametric modeling
Parametric approaches assume the time series under analysis to be the output
of a given linear mathematical model. They require an a priori choice of the
structure and order of the signal generation mechanism model [38].
20 | P a g e
Fig.5 Feature extraction methods in BCI designs based on sensorimotor activity,
VEP, P300, SCP, response to mental tasks, activity of neural cells, and multiple
neuromechanisms.
Conclusions:
Choosing the appropriate method for feature extraction is a crucial step in
designing a BCI, but as the BCI field had shown us, there isn‟t an appropriate
method for all the cases, there are only subject-specific methods. Of course, this
means that feature extraction methods that can be tuned (e.g., band power features,
adapted to frequency bands of the user) are likely to have better results.
2.7. Classification
A third step in creating a BCI is the classification part. This part is crucial,
because it transforms features into commands.
Formally, classification consists of finding the class of a feature vector x,
using a mapping f, where f is learnt from a training set T. Class represents the
21 | P a g e
mental state of the user. The purpose of the learning stage is to provide the algorithm
pre classified labeled data (here, vectors of 320 features), from which the algorithm
builds the mapping in order to predict the labels of new data [38].
Classifiers are divided into 5 main categories:
1. Linear classifiers
-they use linear functions to distinguish classes
-the most common used: Linear Discriminant Analysis(LDA) and Support
Vector Machines(SVM).
-the most popular algorithms for BCI research
2. Neural networks
-are an assembly of artificial neurons that can produce nonlinear decision
boundaries [Bis96].
-are, with linear classifiers the most used in BCI
-the most used neural network technique is MultiLayerPerceptron(MLP).
3. Non linear Bayesian classifiers
-are more accurate than linear ones, but aren‟t that simple and effective.
-the most used are Bayesian quadratic and Hidden Markov Model
4. Nearest neighbor classifiers
-the principle is to assign a class to a feature vector according to it‟s nearest
neighbor.
-the most used techniques are k Nearest Neighbors(KNN) and Mahalanobis
distance
5. Combinations of classifiers
-commonly, only a single classifier technique is used in an interface, but,
recently a new trend occurred: to use multiple classifiers aggregated in some
way.
-there are a couple of strategies used to combine these types of classifiers:
Voting, Boosting, Stacking etc.
The fuzzy classifiers, relatively unknown to the BCI world, till a
couple of years are starting to be often used in designing brain-computer
interfaces, thanks to their interpretability and because the user can see what
to modify in his brain activity for a better recognition of his mental states.
In the next chapter, the fuzzy classifiers will be presented.
Chapter III. Fuzzy classification
22 | P a g e
In this chapter we discuss the classification part, more precisely, the fuzzy
classifiers and there use in a brain-computer interface.
Fuzzy systems are known to deal with uncertain (fuzzy) information, as such
our brain is, there the choice of using fuzzy sets for motor imagery classification.
This chapter will be divided into two sections:
1. Fuzzy classification. Manual rules
2. Fuzzy classification. Automatic rules
3.1.
Fuzzy classification. Manual rules
I have build a fuzzy classification system using Matlab, more specifically the
fuzzy graphical user interface (GUI).
This section is based on studies made by neurophysiologists, regarding the
event related desynchronization and event related synchronization before, during
and after a movement.
Fuzzy:



A fuzzy system consists of:
Inputs
Fuzzy Inference System(FIS)
Outputs
3.1.1. Inputs/Outputs:
The inputs for the system are:
23 | P a g e
Inputs
Nr.crt.
1
2
3
Variable
Range
C3µ
[-1 1]
before
C4µ
[-1 1]
before
C3β before [-1 1]
[-1 1]
[-1 1]
9
C4β before
C3µ
during
C4µ
during
C3β
during
C4β
during
C3µ after
10
C4µ after
[-1 1]
11
C3β after
[-1 1]
12
C4β after
[-1 1]
4
5
6
7
8
Definitions:
1. Electrodes
24 | P a g e
[-1 1]
[-1 1]
[-1 1]
[-1 1]
Membership functions
Tresneg
Nul
Trimf:
Triangular membership function
trespos
-C3= electrode or weighted sum of electrodes from the left side (depends on
the input data of the entire BCI)
-C4= electrode or weighted sum of electrodes from the right side (depends on
the input data of the entire BCI)
2. Power stamps
- β= beta power band
- µ= mu power band
3. Time stamps
I‟ve considered that a significant change appears in the two power bands 2
seconds before the movement and it ends one second after the movement has
finished, so a movement can be represented in the interval [-2 1] second.
-before= before the movement
- µ: between second -2 and -1
- β: between second -1.5 and -0.5
-during= during the movement
- µ: between second -1 and 0
- β: between second -0.5 and 0.5
-after= after the movement
- µ: between second 0 and 1
- β: between second 0.5 and 1
25 | P a g e
Fig.6 Input example
An input can take values between -1 and 1 so the range is [-1 1]. The range
for the input is given by the output of the precedent step in building a BCI: feature
extraction. The outputof feature extraction will be a feature vector named pDLE. In
this case the pDLE can only take values between -1 and 1 so, the range of our inputs.
In the next chapter, the pDLE will be presented in detail.
I have chosen 3 triangular membership functions for each entry:
-tresneg: if the entry is close to -1
-nul: if the entry isn‟t close to -1 or 1
-trespos: if the entry is close to 1
I have chosen triangular membership functions because they have thorough
boundaries, and, in my opinion I think there are better suitable for this type of
experiment, but, of course, it can be possible that a membership function with more
soften boundaries, such as a gaussian function can be more suitable. In the future, I
will try the rest of the membership functions available to see which are giving the
best results.
According to neurophysiologists, the pDLE has negative values before the
movement and positive values after the movement, and in case of a rest period the
values for the pDLE are around 0, so these 3 membership functions are covering
these areas.
26 | P a g e
Example:
If the input C3µbefore has the value -0.8129 it means that the entry value for
the electrode C3 in the µ band between second -2 and -1 (before the movement) is
tresneg.
The outputs for the system are:
Outputs
Nr.crt.
Variable
Range
1
Left
[0 1]
Membership functions
No
Possible
Gaussmf:
Gaussian membership function
2
Repos
[0 1]
3
Right
[0 1]
Definitions:
Mental state:
-left= the user is trying to imagine a left movement
-repos= the user is in the rest state
-right= the user is trying to imagine a right movement
27 | P a g e
Yes
Fig.7 Output example
I have chosen the range between 0 and 1 because a user can be in that mental
state (the output will be close to 1), the user isn‟t in that mental state (the output will
be close to 0) or it cannot be determined (the output will be 0.5 or close to 0.5),
hence the 3 gaussian membership functions: No, Possible, Yes
3.1.2. Fuzzy Inference System (FIS)
Represents a way of mapping an input space to an output space using fuzzy
logic. A FIS tries to formalize the reasoning process of human language by means of
fuzzy logic (that is by building fuzzy IF-THEN rules).
For instance:
“If the service is good, even if the food is not excellent, the tip will be
generous”
The process of fuzzy inference involves: Membership Functions, Logical
Operations, and If-Then Rules.
Matlab offers 2 possibilities of building a FIS through his GUI:


A Mamdani type FIS
A Sugeno type FIS
28 | P a g e
Mamdani vs. Sugeno
Mamdani's method is the most commonly used in applications, due to its
simple structure of 'min-max' operations.
The most fundamental difference between Mamdani-type FIS and Sugenotype FIS is the way the crisp output is generated from the fuzzy inputs. While
Mamdani-type FIS uses the technique of defuzzification of a fuzzy output, Sugenotype FIS uses weighted average to compute the crisp output.
The expressive power and interpretability of Mamdani output is lost in the
Sugeno FIS since the consequents of the rules are not fuzzy. But Sugeno has better
processing time since the weighted average replace the time consuming
defuzzification process. Due to the interpretable and intuitive nature of the rule base,
Mamdani-type FIS is widely used in particular for decision support application [39].
In this paper I have chosen a Mamdani type FIS because of the restrictions
that a Sugeno-type FIS is imposing.
The characteristics of Mamdani FIS used:
-Membership functions: already defined at the inputs/outputs section
-Logical Operations: Min/Max
-Defuzzification method: Centroid
-If-Then rules
If-Then rules
The rules have been based on the artificial signal. This signal represents the common
features of brain activity in power bands µ and β, during a real or imaginary
movement for all individuals, according to neurophysiologists experts.
29 | P a g e
Fig.8 Power band changes determined by movement realized by the dominant hand
(ipsilateral=situated on the same part of brain as the hand involved in the movement;
contralateral=situated on the opposite side of brain)
Differences between left and right movement:
ERD= event related desynchronization
ERS= event related synchronization
Taking in account that the movement starts at sampling time 400, and a
second lasts 100 sampling periods, you can see in the picture above that the
desynchronization(ERD) in the beta and mu band in contralateral appears
approximately 2 seconds before the movement, and in the ipsilateral side around 0.5
seconds before the movement.
The other main difference between left and right hand movement is that the
rebound (ERS) is much better defined in the contralateral region, especially in the
beta band where the rebound exceeds the reference value, and it only takes less than
a second.
30 | P a g e
Rule explanation:
This rule describes the sampling period from the beginning of the movement:
If (C3µ_before is nul) and (C4µ_before is nul) and (C3β_before is nul) and
(C4β_before is nul) and (C3µ_during is nul) and (C4µ_during is tresneg) and
(C3β_during is nul) and (C4β_during is tresneg) and (C3µ_after is not nul) and
(C4µ_after is tresneg) and (C3β_after is tresneg) and (C4β_after is tresneg) then
(mvt.g is Yes)(repos is No)(mvt.d is No) (1)
Movement
-2 sec.
-2 sec.
1s
+1 sec.
A (First rule)
When our fuzzy system verifies that the conditions for movement are
accomplished, it verifies that:
 “C4µ_during is tresneg”= on the contralateral side between second -1 and 0
the µ band is low (it‟s a ERD).
 “C4β_during is tresneg:= on the contralateral side between second -0.5 and
+0.5 the β band is low (it‟s a ERD).
 “C3µ_after is tresneg”= on the contralateral side between second 0 and +1
the µ band is low (it‟s a ERD).
 “C4µ_after is tresneg”= on the ipsilateral side between second 0 and +1 the µ
band is low (it‟s a ERD).
 “C3β_after is tresneg”= on the ipsilateral side between second 0.5 and +1 the
β band is low (it‟s a ERD).
 “C4β_after is tresneg”= on the contralateral side between second 0.5 and +1
the β band is low (it‟s a ERD).
The rest of the conditions may be null, or may not take them into account. On
artificial moves, the results were worse when the other inputs weren‟t taken
into account, so I choose to take them into account.
31 | P a g e
3.2.
Fuzzy classification. Automatic rules
The BCI community had raised a new issue regarding the knowledge
extracted from the classification algorithms. Most of the algorithms are black-boxes,
so you can‟t gather any new information from them, but with the help of a fuzzy
system, it‟s possible to view the extracted patterns from data and understand bits of
how our brain works.
In this section I will present a FIS based on Chiu‟s classification algorithm
(CFIS) [40].
 CFIS is robust to noise, an important quality when dealing with EEG signals,
that are known to be very noisy.
 According to Chiu, the CFIS is generally more efficient than neural
networks, which have been successfully used in BCI research [13].
 Another important feature of CFIS is that it‟s a clustering-based algorithm, a
thing that is important when dealing with small training sets.
CFIS:
Chiu‟s classification algorithm has 3 steps:
I.
II.
III.
Clustering of training data
Generation of the fuzzy rules based on the clustered data
Rules optimization
3.2.1. Clustering of training data
Clustering of numerical data forms the basis of many modeling and pattern
classification algorithms [40].
The purpose of a clustering algorithm is to find natural groupings of data for
pattern recognition.
Chiu proposed a substractive algorithm, where initially, each data point is
considered to be a cluster center.
We consider each data point as a potential cluster center and
define a measure of the potential of data point Xi as:
(1)
32 | P a g e
, where
 α=4/ra2;
 ||.||= Euclidean distance
 Ra= constant, defining the radius of the neighborhood
After each potential has been calculated, we select the point with the highest
potential as the first cluster center, and then we revise the potential of each data
point, according to the formula:
(2)
, where
β= 4/rb2;
The constant rb is effectively the radius defining the neighborhood which will
have measurable reductions in potential. We define r b=1.25 ra, X1* be the location of
the first cluster center and P1* be it‟s potential value
So, the point with the highest potential is chosen as the second cluster center.
This process continues until all remaining data potentials fall under some fraction of
the first potential chosen P1. These is the criteria for ending the search of the
clustering process. Beside this, other criteria are used for avoiding marginal clusters,
as an accept and reject ratio:
33 | P a g e
[40]
This substractive algorithm is provided by Matlab within the fuzzy logic
toolbox.
34 | P a g e
The algorithm is embedded in a function, called subclust, which returns the
centers of the clusters.
The syntax for the subclust function is [41]:
[C, S] = SUBCLUST(X, RADII, XBOUNDS, OPTIONS)
Outputs:
- returns the cluster centers in the matrix C; each row of C contains the position of a
cluster center.
- S vector contains the sigma values that specify the range of influence of a cluster
center in each of the data dimensions.
Inputs:
-X= the data that needs to be clustered
-RADII= has a value between 0 and 1 and specifies the size of the cluster in each of
the data dimensions.
-XBOUNDS= a matrix used to normalize X into a unit hyperbox. If XBOUNDS is
an empty matrix or not provided, the minimum and maximum data values found in
X, are used as defaults.
-OPTIONS: specifies a vector for changing the default algorithm parameters
1. OPTIONS (1): The squash factor, is used to multiply the RADII values to
determine the neighborhood of a cluster center within which the existence of
other cluster centers are discouraged.
2. OPTIONS(2): The accept ratio, sets the potential, as a fraction of the
potential of the first cluster center, above which another data point will be
accepted as a cluster center.
3. OPTIONS(3): The reject ratio sets the potential, as a fraction of the potential
of the first cluster center, below which a data point will be rejected as a
cluster center.
4. OPTIONS(4): Displays progress information unless it is set to zero.
The default values for the OPTIONS vector are [1.25 0.5 0.15 0].
35 | P a g e
After multiple trials on the artificial data and using the GUI for substractive
clustering, I‟ve considered that the next parameters for the subclust function are
suitable to this experiment:
[Cluster, S] = subclust ( pDLE, 0.11, [ ] , [11 0.5 0.06 0] );
Fig.8 Clustered pDLE
Considering that this is the training phase, we know which data is
representative for left, right movement and rest, so the data sets that have been
clustered are the classes for each movement and for rest. After this process is over,
we can say that this are the most significant values for each mental state considered.
For each of these found clusters centers, one rule will be automatically
created, based on Chiu‟s algorithm. This makes the topic of our next section.
36 | P a g e
3.2.2. Generation of the fuzzy rules based on the clustered data
A fuzzy “if-then” rule is generated for each center of cluster found
previously. For a given center j, belonging to class Cli, the generated fuzzy rule is
[31]:
if X1 is Aj1 and . . . and XN is AjN then class is Cli
, where N=dimension of the training data
Xk= Kth element of a feature vector X
Ajk= a gaussian membership function, which is defined as:
, where xjk= the kth element of the vector representing the center of the cluster
=standard deviation for the gaussian membership function.
More precisely, for each value of a cluster center we build a membership
function that has the peak equal to the center‟s value. This are the membership
functions for the entries. For the outputs we keep the same membership functions as
in the first case: standard gaussian membership functions.
37 | P a g e
Membership functions
for movement
Membership
functions for
transition from
movement to
rest
Membership
functions for
rest
Fig.9 Membership functions
As we stated in the last paragraph of the previous section, the outputs for the
substractive algorithm, respectively the inputs for this training phase are the cluster‟s
centers for each of the three classes: left, right movement and rest.
So for each of these centers, a rule will be created. Next, I will present the
Matlab functions from the fuzzy toolbox, that were useful in this step:




addvar(a,'input','C3m_av',[-1 1]): creating an input variable C3m_av for FIS
“a” in the range [-1 1]
addmf(a,'input',i,'good','gaussmf',[sigma_patrat,cluster(j,i)]):
creating
a
membership function for input “i”, named “good”, gaussian type, with equal
distribution “sigma_patrat” and with the center in “cluster(j,i)”
ruleList=[j j j j j j j j j j j j 3 1 1 1 1]: creating a rule where each input has to
belong to membership function “j” and the first output belongs to
membership function “3”, the other two, type “1” and the weight for this rule
is”1”.
a = addrule(a,ruleList): add the rule to our “a” FIS
These are the most important functions used in building this training FIS.
38 | P a g e
3.2.3. Rules optimization
After the FIS is build, the optimization part follows. This consists in tuning
each membership functions according to gradient descent formulas [40]:
, where
λ= a positive learning rate defined by the user,
c= the class of feature vector X
µc,max= the highest degree of fulfillment among all the rules that assign X to
class c
= the highest degree of fulfillment among all the rules that do not
assign X to class c
Only the fuzzy rules corresponding to µc,max and
are optimized.
The “+” sign is used for the rule corresponding to µ c,max and the “-“ sign for
.
The degree of fulfillment is defined as:
,where x= input vector
α= 4/ra2
39 | P a g e
This gradient descent algorithm is a type of competitive learning algorithm: a
“winner” in the “good rule” category is reinforced and a “winner” in the “bad rule”
category” is punished. Because only 2 rules are updated each time the algorithm is
highly efficient. [40].
The algorithm can be improved by changing the membership functions from
a gaussian function with equal standard deviation to a “two-sided” gaussian
function. This may have a flat plateau region and different standard deviations on
the left and right sides.
Fig.10 Two-sided Gaussian membership function
40 | P a g e
Chapter IV. Studying the Use of Fuzzy
Inference Systems for Motor Imagery
based BCI
This chapter is divided in two sections:
I.
II.
Studying the use of FIS for a motor imagery based BCI, using artificial
data sets
 FIS with manual rules
 FIS with automatic rules
Studying the use of FIS for a motor imagery based BCI, using real data
sets
 FIS with manual rules
 FIS with automatic rules
4.1. Studying the use of FIS for a motor imagery based BCI,
using artificial data sets
This section will present the methods used for the 3 essential steps in BCI:
preprocessing, feature extraction and classification
The EEG data sets that were used in this experiment are build upon experts
opinions about changes in mu and beta power band during specific time intervals
and it represents motor imagery experiments.
The artificial EEG data is a matrix N X 10, where N is the number of samples
and the number of columns represents 10 different signals, from 10 electrodes.
The sampling time is 128. A movement or a rest period lasts for 5 seconds so it
has 640 samples. The artificial signal has as inputs the number of right and left
movements. Between every movement there is a rest period. In Fig. 8 is shown a
diagram of the signal with 3 left movements and 2 right.
41 | P a g e
Rest
Left
mvt.
Rest
Left
mvt.
Rest
Left
mvt.
Rest
Right
mvt.
Rest
Right
mvt.
640 samples
To comport like a real one, you can add noise to the artificial signal. The
experiments were made with a percentage of 0.25 additional noise. The maximum
percentage is 1.
The order of the electrodes is:
Left side
1.
C3
2.
FC3
3.
CP3
4.
C1
5.
C5
Right side
6.
C4
7.
FC4
8.
CP4
9.
C2
10. C6
Preprocessing part
Because the data is composed from 10 different signals, it‟s almost necessary the
use of a Laplacian filter, so for the mu band we used a Laplacian filter and for the
beta band an averaging method:
Laplacian:


42 | P a g e
s(:,1)=signal(:,1) - (signal(:,2)+signal(:,3) + signal(:,4) +signal(:,5))./4;
Laplacian for mu left
s(:,2)=signal(:,6)- (signal(:,7)+signal(:,8) + signal(:,9) +signal(:,10))./4;
Laplacian for mu right
Rest
The coefficients used:
Laplacian for mu left
0
-1
0
-1
4
-1
0
-1
0
Represents
Laplacian for mu right
0
-1
0
-1
4
-1
0
-1
0
0
C5
0
FC3
C3
CP3
0
C1
0
0
C2
0
FC4
C4
CP4
0
C6
0
Represents
Averaging:


s(:,3)=(signal(:,2)+signal(:,3) + signal(:,4) +signal(:,5))./4; averaging for beta
left
s(:,4)=(signal(:,7)+signal(:,8) + signal(:,9) +signal(:,10))./4; averaging for
beta right
After the signal passes from 10 characteristics (the number of electrodes) to 4
characteristics(mu and beta band in left and right hemisphere) , the signal undergoes
a frequential preprocessing. This is achieved by using a bandpass Butterworth filter,
of the order of 2, between 8-25 Hz for both of the bands.
ordre_filtre = 2;
frequences_filtre = [8/une_seconde 25/une_seconde];
[bmu, amu] = butter ( ordre_filtre, frequences_filtre,'bandpass');
ordre_filtre = 2;
frequences_filtre = [8/une_seconde 25/une_seconde];
[bbet, abet] = butter (ordre_filtre, frequences_filtre,'bandpass');
43 | P a g e
At the end of the preprocessing part the signal is squared, to eliminate small
variations.
Feature extraction part
This section allows us to extract the features of the signal and transform them
into the inputs that the fuzzy classification system has as entries.
This section has 3 major parts:
1. Averaging the signal every 10 samples, to reduce the unwanted
characteristics
2. Setting a mean reference power for the beta and mu band: this is when the
user is in the rest mental state. I‟ve considered that the first 2 seconds are
suitable for this, because the signal starts with a 5 second rest period.
puissance_signal_reference_mu=signal(1:ceil(2*une_seconde/10),1:2);
moyenne_puissance_reference_mu = mean(puissance_signal_reference_mu);
puissance_signal_reference_beta=signal(1:ceil(2*une_seconde/10),3:4);
moyenne_puissance_reference_beta = mean(puissance_signal_reference_beta);
3. Extracting the suitable periods of time from the signal and making them an
average.
The time periods are:
Mu band
Beta band
Before
-2÷-1
-1.5÷-0.5
During
-1÷0
-0.5÷0.5
After
0÷1
0.5÷1
For mu band:
puissance_signal_avant_mu=signal(i-floor(2*une_seconde/10):ifloor(1*une_seconde/10),1:2);
puissance_signal_pendant_mu=signal(ifloor(1*une_seconde/10):i+floor(0.*une_seconde/10),1:2);
puissance_signal_apres_mu=signal(i+floor(0.*une_seconde/10):i+floor(1*une_seco
nde/10),1:2);
44 | P a g e
moyenne_puissance_avant
=
mean(puissance_signal_avant_mu);
moyenne_puissance_apres
=
mean(puissance_signal_apres_mu);
moyenne_puissance_pendant = mean(puissance_signal_pendant_mu);
For beta band:
puissance_signal_avant_beta=signal(i-floor(1.5*une_seconde/10):ifloor(0.5*une_seconde/10),3:4);
puissance_signal_pendant_beta=signal(ifloor(0.5*une_seconde/10):i+floor(0.5*une_seconde/10),3:4);
puissance_signal_apres_beta=signal(i+floor(0.5*une_seconde/10):i+floor(1*une_se
conde/10),3:4);
moyenne_puissance_avant = mean(puissance_signal_avant_beta);
moyenne_puissance_apres = mean(puissance_signal_apres_beta);
moyenne_puissance_pendant = mean(puissance_signal_pendant_beta);
To obtain percentage values for ERD/ERS, the power within the frequency
band of interest in the period after the event is given by A whereas that of the
preceding base line or reference period is given by R. ERD or ERS is defined as the
percentage of power decrease or increase, respectively, according to the expression
[DA2]:
ERD%=(A-R)/R*100
For the display of the time course of ERD/ERS, a scale displaying either
power changes with 0% in the reference period or relative power with 100% in the
reference period is recommended.
After this step, a feature vector, most commonly known in publications as
pDLE is formed.
For mu band
pDLE(a,1:2)
=
(moyenne_puissance_avant
moyenne_puissance_reference_mu)./moyenne_puissance_reference_mu;
-
pDLE(a,5:6)
=
(moyenne_puissance_pendant
moyenne_puissance_reference_mu)./moyenne_puissance_reference_mu;
-
45 | P a g e
pDLE(a,9:10)
=(moyenne_puissance_apres
moyenne_puissance_reference_mu)./moyenne_puissance_reference_mu;
-
For beta band
pDLE(a,3:4)
=
(moyenne_puissance_avant
moyenne_puissance_reference_beta)./moyenne_puissance_reference_beta;
-
pDLE(a,7:8)
=
(moyenne_puissance_pendant
moyenne_puissance_reference_beta)./moyenne_puissance_reference_beta;
-
pDLE(a,11:12)
=(moyenne_puissance_apres
moyenne_puissance_reference_beta)./moyenne_puissance_reference_beta;
-
pDLE represents the output of the feature extraction method. It also is the input
for our fuzzy classification system, so the correspondence between the columns of
the pDLE and the inputs of the FIS is:
Columns pDLE
Inputs FIS
1
2
3
4
5
6
7
8
9
10
11
12
C3µ before
C4µ before
C3β before
C4β before
C3µ during
C4µ during
C3β during
C4β during
C3µ after
C4µ after
C3β after
C4β after
46 | P a g e
Classification part
As we explained in the second chapter, in this paper we are focusing on
fuzzy classifications systems, more specifically:


FIS with manual rules
FIS with automatic rules, based on Chiu‟s algorithm
These 2 types were detailed earlier, so there is no need for further
information.
Results

FIS with manual rules
We tested the entire BCI presented so far with the artificial signal. It has to
detect 64 samples for movement and rest. Between movements there is a rest period.
The results are presented from a “fuzzy power” point of view. The outputs of
the FIS are the 3 mental states involved, in this order: left movement, rest, right
movement. It assigns for each sample a degree of membership to the mental states
already specified. The degree may vary from 0 to 1, so the highest degree for a
sample to be detected as a movement or rest state is 1. If the FIS doesn‟t have
sufficient information to determine the type of mental state, the three coefficients are
simple 0.5. I will present the results as a chart, as they are easy to understand. The
degree is represented on the Y axes and it‟s called “Fuzzy coefficients” and the
sampling time is on the X axes
Here are the charts representing the results:
47 | P a g e
1. 5 left movements and 6 right movements:


From the chart, the following conclusions can be drawn:
The system detects better the rest mental state than the movement
It detects movement in 50-51 samples from 64
2. 2 left movements and 1 right movement
48 | P a g e
3. 31 left movements and 22 right movements

FIS with automatic rules
We tested the BCI with the artificial signal. The parameters for the experiment are
the same as in the case of FIS with manual rules.
The results are:
49 | P a g e
1. 6 right movements and 6 left movements
2. 15 right movements and 12 left movements
50 | P a g e
Conclusions:
As it‟s shown in the figures above, the FIS based on Chiu‟s algorithm has a
better accuracy than the FIS with manual rules. It determines movement in 50-51
samples from 64, but the fuzzy coefficients for the movements have bigger values
than in the first case, with manual rules, so we can find an improvement.
4.2. Studying the use of FIS for a motor imagery based
BCI, using artificial data sets
This section will present the methods used to classify real motor imagery
data.
The data is taken from an Open Vibe scenario, “Handball” which is used to
identify left and right movements. Open Vibe is a specialized platform used to
perform BCI experiments.
“Handball” experiment uses as entry data an EEG signal from 10 electrodes,
taken from left and right side, exactly as the artificial signal. The classifier algorithm
used in this scenario is linear discriminant analysis (LDA), which characterizes or
separates two or more classes of objects or events. The resulting combination may
be used as a linear classifier. The classes are: left and right movement. The rest class
doesn‟t exists in the “Handball” scenario because the problem would get very
complicated.
The format of the EEG data used in scenario is Open Vibe (.ov). The format
used in BCI experiments conducted with the use of Matlab is General Data Format
for biosignals (.gdf), so the first step is to transform the OV data into GDF data. For
this I‟ve built an Open Vibe software which accomplishes this task.
The GDF format provides a common coding scheme for events, and supports
many useful features (different sampling rates and calibration values for different
channels, an automated overflow detection, support of different data types, encoding
of filter settings etc.), that are only partly implemented in other formats[42].
Briefly, it addresses the need for



Subject specific information (gender, age, impairment, etc)
Recording location, identification of recording software, etc.
Possibilities for storing the electrode positions in spatial coordinates,
electrode impedances, etc.
51 | P a g e


More efficient encoding of date and time, physical dimensions, filter
information
Non-equidistant (sparse) sampling[gdf.2.0].
This type of format stores an event table, which consists of all movements
performed during the tests, their position and length. This helps us to know exactly
when a movement has happened.
To use the GDF format with Matlab I‟ve installed a gdf library which allows
the user to load an EEG signal. The syntax is:
[signal,header]=sload(„name of gdf file‟.gdf);
The signal variable stores the EEG data, which in our case is a 10 columns
matrix. The header variable stores the support data for the signal, like the event table.
The results are shown below:

FIS with manual rules
The coefficients obtained demonstrates that the manual FIS doesn‟t
recognize the movements anymore.
52 | P a g e

FIS with automatic rules
This chart represents the fuzzy classification for both the training and the
experimental data. The training data is on the left side of the chart and the real
experiment data is on the right side. As we can see, the training data is well
classified, except for the rest period.
The experiment data is not so well classified, but we can see that it exists a
movement, but in some cases, when it should be a rest period, the classification
algorithm shows that it is a movement.
Taken into account all these, the FIS based on Chiu‟s algorithm is starting to
work. It recognizes almost all the movements.
53 | P a g e
Conclusions:
The FIS with manual rules showed good results on artificial data, but when was
tested on real data, it didn‟t worked, but we can get information from it to upgrade
the FIS with automatic rules.
The FIS with automatic rules showed better results on artificial data than the one
with manual rules, it recognizes the movements, except the rest period. The problem
is that the movements aren‟t well represented in all the samples where should be a
movements, but a first step has been achieved.
Improvements:
I think that an on-line session with a subject to better train the system would
improve it‟s performances.
It would help the system to better calibrate the rules and to find the correct
frequency bands for mu and beta rhythm, because there are user specific.
Another problem is the reference pDLE for mu and beta band. I think that choosing
another time interval for the reference, which is suited for the rest state, would bring
benefits.
And last, I think that modifications brought to the preprocessing and feature
extraction parts would help the FIS to better recognize the mental states.
54 | P a g e
References
1. [1]- J.R. Wolpaw, N. Birbaumer, D.J. McFarland, G. Pfurtscheller, and T.M.
Vaughan. Brain-computer interfaces for communication and control. Clinical
Neurophysiology, 113(6):767–791, 2002.
2. [2]- N. Birbaumer. Breaking the silence: Brain-computer interfaces (BCI) for
communication and motor control. Psychophysiology, 43(6):517–532, 2006.
3. [3]- G. Pfurtscheller, C. Neuper, and N. Birbaumer. Motor cortex in voluntary
movements, chapter Human brain-computer interface, pages 367–401. CRC Press,
riehle a, vaadia e. edition, 2005.
4. [4]- F. Cabestaing and A. Rakotomamonjy. Introduction aux interfaces
cerveaumachine
(BCI). In 21ème Colloque sur le Traitement du Signal et des Images,GRETSI„07,
pages 617–620, 2007.
5. [5]- U. Hoffmann, J. Vesin, and T. Ebrahimi. Recent advances in brain-computer
interfaces. In IEEE International Workshop on Multimedia Signal Processing, 2007.
6. [6]- www.wikipedia.com
7. [7]- E. Anquetil and G. Lorette. On-line handwriting character recognition system
based on hierarchical qualitative fuzzy modeling. In Proceedings of the 5th
International Workshop on Frontiers in Handwriting Recognition (IWFHR5), pages
47–52, 1996.
8. [8]- F. H. Y. Chan, Y. S. Yang, F. K. Lam, Y. T. Zhang, and P. A. Parker. Fuzzy
EMG classification for prosthesis control. IEEE transactions on rehabilitation
engineering, 8(3):305–311, 2000.
9. [9]- O. F. Bay and A. B. Usakli. Survey of fuzzy logic applications in brain related
researches. Journal of Medical Systems, 2003.
10. [10]- E. Huupponen, M. Lehtokangas, J. Saarinen, A. Varri, A. Saastamoinen, S. L.
Himanen, and J. Hasan. EEG alpha activity detection by fuzzy reasoning. In IFSA
World Congress and 20th NAFIPS International Conference, 2001. Joint 9th, pages
411–416, 2001.
11. [11]- H. Berger. Ueber das elektroenkephalogramm des menschen. Archiv für
Psychiatrie und Nervenkrankheiten, 87:527–570, 1929.
12. [12]- J. J. Vidal. Toward direct brain-computer communication. Annual Review of
Biophysics and Bioengineering, pages 157–180, 1973.
13. [13]- F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, and B. Arnaldi. A review of
classification algorithms for EEG-based brain-computer interfaces. Journal of
Neural Engineering, 4:R1–R13, 2007.
14. [14]- J. C. Bezdec and S. K. Pal. Fuzzy Models For Pattern Recognition. IEEE
PRESS, 1992.
55 | P a g e
15. [15]- J. Kalcher, D. Flotzinger, C. Neuper, S. Golly, and G. Pfurtscheller. Graz
brain-computer interface II: towards communication between humans and
computers based on online classification of three different EEG patterns. Medical
and Biological Engineering and Computing, 34:383–388, 1996.
16. [16]- G. Pfurtscheller, C. Neuper, G.R. Muller, B. Obermaier, G. Krausz, A.
Schlogl, R. Scherer, B. Graimann, C. Keinrath, D. Skliris, M. Wortz, G. Supp, and
C. Schrank. Graz-BCI: state of the art and clinical applications. IEEE Transactions
on Neural Systems and Rehabilitation Engineering, 11(2):1–4, 2003.
17. [17]- E. Niedermeyer and F. Lopes da Silva. Electroencephalography: basic
principles, clinical applications, and related fields. Lippincott Williams & Wilkins,
ISBN 0781751268, 5th edition, 2005.
18. [18]- E. A. Curran andM. J. B. Stokes. Learning to control brain activity: a review
of the production and control of EEG components for driving brain-computer
interface (BCI) systems. Brain and Cognition, pages 326–336, 2003.
19. [19]- YIJUN WANG,XIAORONG GAO,BO HONG, CHUAN JIA,AND
SHANGKAI GAO. Brain-computer interfaces based on VEP, 2008
20. [20]- Francois-Benoit Vialatte, Monique Maurice, Justin Dauwels, and
Andrzej Cichocki. Steady State Visual Evoked Potentials in the Delta range,
21. [21]- J. Peter Rosenfeld. Event-Related Potentials in detection of
Deception,1999
22. [22]- J.R. Wolpaw and D.J. McFarland. Control of a two-dimensional movement
signal by a noninvasive brain-computer interface in humans. Proc Natl Acad Sci U
S A, 101(51):49–54, 2004.
23. [23]- J. R. Wolpaw, D. J. McFarland, G. W. Neat, and C. A. Forneris. An
EEGbased brain-computer interface for cursor control. Electroencephalography and
clinical neurophysiology, 78:252–259, 1991.
24. [24]- T.M. Vaughan, D.J. McFarland, G. Schalk, W.A. Sarnacki, D.J. Krusienski,
E.W. Sellers, and J.R. Wolpaw. The wadsworth BCI research and development
program: at home with BCI. IEEE Transactions on Neural Systems and
Rehabilitation Engineering, 14(2):229–233, 2006.
25. [25]- J.R. Wolpaw. Brain-computer interfaces as new brain output pathways. J
Physiol, 579:613–619, 2007.
26. [26]- G. Pfurtscheller and C. Neuper. Motor imagery and direct brain-computer
communication. proceedings of the IEEE, 89(7):1123–1134, 2001.
27. [27]- G. Pfurtscheller, C. Brunner, A. Schlogl, and F.H. Lopes da Silva. Mu rhythm
(de)synchronization and EEG single-trial classification of different motor imagery
tasks. NeuroImage, 31(1):153–159, 2006.
56 | P a g e
28. [28]- N. Birbaumer, A. Kübler, N. Ghanayim, T. Hinterberger, J. Perelmouter, J.
Kaiser, I. Iversen, B. Kotchoubey, N. Neumann, and H. Flor. The thought
translation device (TTD) for completely paralyzed patients. IEEE Transactions on
Rehabilitation Engineering, 8:190–193, 2000.
29. [29]- B. Kleber and N. Birbaumer. Direct brain communication: neuroelectric and
metabolic approaches at Tübingen. Cognitive Processing, 6(1):65–74, 2005.
30. [30]- G. Pfurtscheller, F.H. Lopes da Silva. Event-related EEG/MEG
synchronization and desynchronization: basic principles, 1999
31. [31]- Fabien Lotte, Phd. Thesis, 2008
32. [32]- E. Lalor, S. P. Kelly, C. Finucane, R. Burke, R. Smith, R. Reilly, and G. McDarby. Steady-state VEP-based brain-computer interface control in an immersive 3D gaming environment. EURASIP journal on applied signal processing, 2005.
33. [33]- H. Ramoser, J. Muller-Gerking, and G. Pfurtscheller. Optimal spatial filtering
of single trial EEG during imagined hand movement. IEEE Transactions on
Rehabilitation Engineering, 8(4):441–446, 2000.
34. [34]- G. Dornhege, B. Blankertz, G. Curio, and K.-R.Müller. Increase information
transfer rates in BCI by CSP extension to multi-class. In Advances in Neural
Information Processing Systems, pages 733–740, 2004.
35. [35]- W. Wu, X. Gao, and S. Gao. One-versus-the-rest (OVR) algorithm: An
extension of common spatial patterns(CSP) algorithm to multi-class case. In 27th
Annual International Conference of the Engineering in Medicine and Biology
Society, 2005. IEEE-EMBS 2005, pages 2387– 2390, 2005.
36. [36]- C.M.Michel,M.M.Murray, G. Lantz, S. Gonzalez, L. Spinelli, and R. Grave
de Peralta. EEG source imaging. Clin Neurophysiol., 115(10):2195–2222, 2004.
37. [37]- S. Baillet, J.C. Mosher, and R.M. Leahy. Electromagnetic brain mapping.
IEEE Signal Processing Magazine, 18(6):14–30, 2001.
38. [38]- Ali Bashashati, Mehrdad Fatourechi, Rabab K Ward and Gary E Birch.
A survey of signal processing algorithms in brain-computer interfaces based
on electrical brain signals, Journ. Of Neural Engineering, 2007
39. [39]- Arshdeep Kaur, Amrit Kaur, Comparison of Mamdani-Type and
Sugeno-Type Fuzzy Inference Systems for Air Conditioning System,
International Journal of Soft Computing and Engineering (IJSCE), 2012
40. [40]- Stephen L. Chiu Rockwell Science Center, Extracting Fuzzy Rules from
Data for Function Approximation and Pattern Classification. Chapter 9 in
Fuzzy Information Engineering: A Guided Tour of Applications, ed. D.
Dubois, H. Prade, and R. Yager, John Wiley & Sons, 1997.
41. [41]- Matlab R2010 Help
57 | P a g e
42. [42]- Alois Schlögl. GDF - A GENERAL DATAFORMAT FOR
BIOSIGNALS VERSION 2.00. Institute for Human-Computer Interfaces,
University of Technology Graz (2004-2006)
58 | P a g e