Study on Virtual Control of a Robotic Arm via a Myo Armband for the

Transcription

International Journal of Applied Engineering Research ISSN 0973-4562 Volume 11, Number 2 (2016) pp 775-782
© Research India Publications. http://www.ripublication.com
Study on Virtual Control of a Robotic Arm via a Myo Armband for the SelfManipulation of a Hand Amputee
Asilbek Ganiev,
Associate Professor, Department of Digital Media,
Soongsil University, Seoul, Republic of Korea.
E-mail: [email protected]
Ho-Sun Shin,
Department of Cultural Contents,
Kang-Hee Lee,
Department of Digital Media,
fabricated and describes the specifications of hand amputees
and forearm muscles. Section 'Architecture design' shows the
architectural design and Section 'Experimental results'
discusses the experiment results obtained from virtual robotic
arms. From these procedures, the design for a virtual robotic
arm is proposed and experiments are carried out to
demonstrate its features. In Section 'Conclusion,' we compare
two different virtual robotic arms and determine which one is
suitable for the hand amputee.
Abstract
This paper proposes a Myo device that has electromyography
(EMG) sensors for detecting electrical activities from different
parts of the forearm muscles; it also has a gyroscope and an
accelerometer. EMG sensors detect and provide very clear and
important data from muscles compared with other types of
sensors. The Myo armband sends data from EMG, gyroscope,
and accelerometer sensors to a computer via Bluetooth and
uses these data to control a virtual robotic arm which was built
in Unity 3D. Virtual robotic arms based on EMG, gyroscope,
and accelerometer sensors have different features. A robotic
arm based on EMG is controlled by using the tension and
relaxation of muscles. Consequently, a virtual robotic arm
based on EMG is preferred for a hand amputee to a virtual
robotic arm based on a gyroscope and an accelerometer
Motivation and Related Research
Data was obtained from EMG, the gyroscope, and the
accelerometer and then analyzed to understand which parts of
the forearm muscles electro activated in each wrist gestures.
The data obtained from the Myo armband sensors are used to
control the robotic arm, which was built in Unity 3D. The
robotic arm can move by using five distinct motions,
including making a fist, spreading fingers, waving left,
waving right, and double tapping fingers (to burst balloons
the). The gyroscope data and accelerometer data are also used
for moving the robotic arm in the X, Y, and Z directions.
Keywords: Myo armband, Electromyography, Controlling
robotic arm, Hand amputee, Unity 3D
Introduction
The Myo armband detects electrical activity in forearm
muscles. The human forearm has different types of muscles,
each of which has a different arrangement, and these muscles
control the movements of the wrist, such as moving fingers,
making a fist, turning left or right, etc. The Myo armband has
eight parts, each of which can detect these electrical activities
in each part of the forearm.
The Myo armband has many advantages compared with other
types of sensors, because it can be worn on the hand without
cables, it connects with other devices or a computer via
Bluetooth, and is very comfortable.
In addition, the electromyography (EMG) sensor of the Myo
armband has a gyroscope sensor and an accelerometer [1].
Another advantage of this device is that, although the human
hand size differs and human hand poses vary, we can adapt
the Myo armband to anyone’s hand.
The remaining structure of this paper is outlined as follows.
Section 'Motivation and Related Research' briefly introduces
how virtual robotic arms (Myo armband and Unity 3D) are
EMG and MYO sensor
Most gesture-control systems still use cameras to detect
movements, which can be incorrect due to poor lighting
conditions, distance, and simple obstructions. By obtaining
gesture information directly from the arm muscles rather than
from a camera, the Myo circumvents these problems and
works with devices that do not have a camera [2].
EMG is an electro-diagnostic technique for evaluating and
recording the electrical activity produced by human skeletal
muscles. EMG detects the electrical potential generated by
muscle cells when these cells are electrically or neurologically
activated. The signals can be analyzed to detect medical
abnormalities, activation level, or recruitment order or to
analyze the biomechanics of human movement [3]. EMG
testing has a variety of clinical and biomedical applications.
EMG is used as a diagnostics tool for identifying
neuromuscular diseases, or as a research tool for studying
775
Internationnal Journal off Applied Enggineering Reseearch ISSN 09
973-4562 Voluume 11, Numbber 2 (2016) pp
p 775-782
© Researcch India Publiccations. http:///www.ripubliication.com
relaax their muscles wearing EM
MG sensors su
uch as those on
o the
MY
YO armband. It is possible to control an
n object using their
EM
MG in virtual space through the virtual rob
botic arm baseed on
EM
MG.
kinesiology and disorders
d
of motor
m
control. EMG signalss are
someetimes used too guide botuliinum toxin or phenol injecttions
into muscles. EMG
G signals are also used as control
c
signals for
prostthetic devicess such as prossthetic hands,, arms, and loower
limbbs [4].
Fiigure 1: Usingg Myo armbannd
Figure 4: H
Hand amputee
Figuure 1 shows thhe Myo armbaand and controolling applicattions
suchh as a shot gunn game, an ocuulus game, controlling a rem
mote
contrrol (RC) car, and
a drones.
ty 3D
Unity
Unitty 3D provides rendering off 3D objects, animations ettc. in
real time and haas relatively easy-scriptinng, including the
comppatibility of various
v
platfoorms. It is verry popular foor its
intuiitive interface application as
a shown in figures 2 and 3 [5].
It iss also a gam
me engine, similar
s
to Unreal
U
or Juppiter.
How
wever, Myo is the preferrred device foor interaction and
visuaalizing using the virtual robbotic arm. Thherefore, Unityy 3D
is ussed to createe virtual spacce and visualize experimeental
resullts.
5 Device for aamputee in practical life
Figure 5:
Figure 2: Inteerface of
F
Unity 3D
D
Forrearm musclees
Thee forearm referrs to the regioon of the lower limb betweeen the
elbo
ow and the wrrist. The term ‘forearm’ is used
u
in anatom
my to
disttinguish it from the arm, a word which is most often used
to describe
d
the enntire appendagge of the uppeer and lower limbs,
but which in anaatomy, techniccally, means only the region of
the upper arm, whereas
w
the low
wer "arm" is called
c
the foreearm.
It iss homologouss with the reggion of the leg
g that lies bettween
the knee and thee ankle joint, the crus. Thee forearm conntains
two
o long boness, the radiuss and the ulna,
u
formingg the
radiioulnar joint. The interosseous membraane connects these
bon
nes. Ultimatelyy, the forearm
m is covered by
y skin, the annterior
surfface usually being less hairyy than the possterior surface [9].
Figure 3: Unity 3D
m
platforrms
providing multiple
Han
nd amputee
A haand amputee is a person whose hand or arm has been
b
severed, specificcally the uppper parts. Today,
T
numeerous
assisstance devicess are availablee to help the movements
m
off the
handd amputee in practical lifee as shown inn figures 4 annd 5
[6][77], (e.g. synchhronizing the amputated paart with a robbotic
arm to insert a smart
s
chip inn the body [88]). However,, the
ware or virtuaal devices avvailable for haand amputeess are
softw
insuffficient. The virtual
v
roboticc arm is intennded for the hand
h
ampuutee. If their arm
a muscles are
a alive, the virtual
v
robotic arm
helps them. This means they can use EMG
G to contract and
776
p 775-782
F
Figure
6: Forrearm muscless
The forearm iss divided into two compartments
c
(a
ventrromedial or flexor compaartment and a dorsolateraal or
extennsor comparttment), as caan be seen in
i figure 6. The
musccles of thee forearm are segregaated into these
t
comppartments, coonsisting of ann anterior grouup (the flexorrs of
the wrist
w
and finggers and the prronators) and a posterior grroup
(the extensors of the
t wrist and fingers
f
and thee supinator).
Fiigure 8: Archhitecture of thee design for viirtual robotic arm
a
Architecture Deesign
A virrtual robotic arm
a is compossed of an EMG
G and robotic arm
in viirtual space. EMG
E
is used for
f measurem
ments made forr the
Myoo armband, while
w
the robbotic arm in virtual spacce is
prodduced by Unityy 3D. They arre connected to Bluetooth. The
Myoo armband hass eight differennt parts, each of which conttains
a meedical-grade EMG
E
sensor. The armbandd also has a thhreeaxis gyroscope and three-axxis accelerom
meter. The Myo
M
armbband and the numbers of thhe sensors aree shown in fiigure
7. These
T
numbers are used inn experimentaal results andd are
show
wn in figure 244.
Figure 9: Five hand gestures and theeir functions
Thee mapping ressult is shown in figure 9. ‘Up’ is assignned to
‘Fisst’, which haas distinctly different activating rates than
‘Fin
ngers spread’.. Obviously, ‘Down’ is assigned to ‘Fingers
spreead’. These arre the results ffrom the activ
vating rates off each
EM
MG sensor partt. Also, the m
mapped result can
c be changeed by
userrs. After thesse steps, the user can acctually controol the
virtu
ual robotic arm
m.
Exp
perimental results
r
Figure 7: Myyo armband and
a numbers of
o the sensors
This section doccuments the eexperimental results. Figurre 10
pressents the resullts from eight EMG sensorss in each gestuure. It
can be seen that in each gestuure, different EMG
E
sensors have
diffferent results. Subsection A shows the results from eight
EM
MG sensors inn five differeent gestures. In subsectioon B,
grap
phs show datta from the “W
Wave Right” gesture and show
thatt part, of the eight
e
parts off the forearm, that is electriically
actiivated; the eleectrode voltagge is shown in
n microvolts (μ V),
whiich was obtained using the Myo data
d
capture tool.
Sub
bsection E shoows the EMG
G sensor conttrolling the virtual
v
robo
otic arm.
Desiign for virtuall robotic arm
This section introoduces the dessign for the virtual robotic arm
[10].. Figure 8 shhows the enttire process of
o controllingg the
virtuual robotic arrm in Unity 3D. The firrst step invoolves
creatting space with
w
targets annd the virtuaal robotic arm
m in
Unitty 3D. The neext step involvves mapping the five functtions
of thhe robotic arm
a
in virtuaal space to the
t
five gesttures
meassured by the Myo
M armband..
777
p 775-782
Figure 15: Double tap
Fiigure 10: Datta from eight EMG
E
sensors in each gesturre
Datta from EMG
G sensors
Sub
bsection A ouutlines the EM
MG for each motion.
m
The EMG
E
senssor is compossed of eight parts with diffe
ferent data forr each
senssor as can bee seen in figuures 16-23. Su
ubsection B shows
s
grap
phs to check the
t activationn rates for each
h part of the EMG
E
senssor when maaking gesturess such as ‘W
Wave Right’. As a
resu
ult, parts 3, 4,, and 5 are eleectrically activ
vating and paarts 1,
2, 6,
6 7, and 8 are inactive.
In thhis case, five gestures
g
were used: wave leeft for turningg left
robootic arm, wave right for tuurning right, fingers
f
spreadd for
raisinng up, fist placed
p
down, and double tap with finngers
autom
matically find balloons and
a
burst theem. Subsectioon F
show
ws the gyroscoope and acceleerometer conttrolling the virrtual
robootic arm. In thhis case, the roobotic arm moves
m
in the X,
X Y,
and Z directions.
Dataa from EMG sensors
s
in eacch motion
In thhis section, thhe results from
m each EMG sensor in the five
gestuures are show
wn. Figures 111-15 show daata from the EMG
E
sensoors and it caan be seen that
t
in differrent motions, the
sensoors are either active or nott active [11].T
The Pod0 signnal is
matcched with the Myo armbandd part number 1 in figure 7.
Fiigure 11: Wave in (left)
Figure 12: Wave
W
out (rigght)
Figure 133: Fist
Figure 14:: Fingers spreaad
Figu
ure 16: Data ffrom EMG seensor1
Figu
Figu
778
p 775-782
Figu
ure 19: Data frrom EMG sennsor4
Figu
Figu
Acttive sensors in
n each gesturee
Thee eight sensorr parts are coomposed of EMG sensorss and
each
h part has a different pposition, wheereby each sensor
meaasures the EM
MG in each pposition. The EMG sensor can
thuss check whicch parts are aactive and in
nactive. The EMG
E
senssor can perceive each gestture and transmit informatiion to
the virtual robootic arm. T
The virtual robotic
r
arm then
imp
plements the previously assigned functtion, as show
wn in
figu
ure 24, indicating the ‘Wavve left’ and ‘W
Wave right’ of
o the
fivee gestures. When
W
performiing these gesttures, each paart of
the EMG sensor assumes a diffferent aspectt. Parts 1, 6, 7,
7 and
8 arre active in ‘W
Wave left’, whhile parts 3, 4, and 5 are actiive in
‘Waave right’.
Figu
Figu
Figure
F
24: Doominance tablle showing wh
hich of the eigght
sennsors are activve in each mo
otion
In [12], results were
w shown off the distinction of using muuscles
wheen making geestures, such as the brach
hioradialis muscle
m
whiich affects thee bending of tthe elbow, thee pronator which is
used
d to turn the wrist
w
left and down, and th
he flexor digittorum
superficialis whicch turns the w
wrist right and up.
In the
t case of ‘W
Wave left’, thhe pronator teeres is more active
a
than
n the flexor digitorum
d
supperficialis. The pronator terres is
locaated in a relaatively inner pposition. Therrefore, the seensors
locaated inside succh as 1, 6, 7, aand 8 show seensitive reactioon. In
the case of ‘Waave right’, thee reaction off sensors distiinctly
chan
nges as show
wn in figure 224. As shown
n in figure 255, the
high
hlighted musccles are activaated and the co
orresponding EMG
E
senssors show osccillations.
779
p 775-782
ntrolling virtu
ual robotic arm
m based on EMG
E
Con
This section show
ws the results of controlling
g the virtual roobotic
arm
m using the EM
MG sensor, aas shown in figures
f
28-32. Five
gesttures are usedd as follows: w
wave left for turning
t
roboticc arm
left,, wave right for
fo turning righht, fingers sprread for raisinng up,
fist facing down,, and double tap of fingerss for automatiically
find
ding balloons and bursting tthem.
Figuree 25: Muscle activities
a
in geestures
Dataa from accelerrometer and gyroscope
g
In ouur experimentts, we used ann accelerometter and gyrosccope
to coontrol the robootic arm, and figures
f
26 andd 27 illustrate data
from
m the acceleroometer and gyroscope
g
in the X, Y, annd Z
direcctions.
Figure 288: Wave Righht-turns robotiic arm right
Figgure 26: Dataa from acceleroometer in randdom movemennts
Figure 29: Wave Lefft-turns robotiic arm left
F
Figure
27: Daata from gyrosscope in randoom movementts
The results from the accelerom
meter and gyrroscope show
wn in
figurres 26 and 277 were obtaineed from randoom movementts of
the hand
h
in the X, Y, and Z direections.
a rises up
Figure 300: Fingers Spread –robotic arm
780
p 775-782
Fig
gure 33: Putting arm down
Figure 34
4: Raising arm
m up
Figure 31:
3 Fist –robottic arm is placced down
Figure
F
35: Turrning Right
F
Figure
32: Douuble tap – robbotic arm automatically findds
balloons andd bursts them
Figure 36:
3 Turning Leeft
Witthout the neeed for the sstep of mapp
ping gesturess and
funcctions, the virrtual robotic aarm controlled
d by the gyrosscope
and
d accelerometeer makes it ppossible to co
oincide movem
ments
with
h the real arm
m. This is a sstrong advanttage of the roobotic
arm
m based on thee gyroscope annd acceleromeeter. Howeverr, it is
con
nsidered inapppropriate for tthe case of haand amputee users
because hand amputees
a
findd it difficultt to perform
m the
namic actions shown in figuures 33-36. Ho
owever, the virtual
v
dyn
robo
otic arm baseed on EMG ccan be simply
y controlled by
b the
tenssion and relaxxation of musccles. This is th
he most signifficant
adv
vantage of the virtual robotic arm based on EMG andd why
the virtual robottic arm basedd on EMG is recommendeed for
han
nd amputees rather than an arm based on
n a gyroscopee and
acceelerometer.
Conttrolling virttual robotic arm by gyroscope and
acceelerometers daata in X, Y an
nd Z directionss
This virtual robottic arm is conntrolled by a gyroscope
g
annd an
accelerometer. A gyroscope iss inserted intoo the Myo deevice
and can detect chhanges in orieentation; this axis
a
is unaffeected
by tilting
t
or rottation of the mounting, according to the
consservation of angular moomentum. Because
B
of this,
gyrooscopes are useful for measuring or maintaiining
orienntation. Also, an accelerom
meter device is inserted inn the
Myoo device and it measures proper
p
accelerration ("g-forcce").
Propper acceleratioon is not the same
s
as coorddinate acceleraation
(ratee of change off velocity) [13]].
Figuures 33-36 shoow the virtuaal robotic arm
m being controolled
by using
u
the gyrooscope and accelerometer. In this case,, the
robootic arm moves in the X, Y, and Z directioons.
Con
nclusion
In this
t
paper, we
w showed hhow a virtuall robotic arm
m, the
prottotype of which was builtt in Unity 3D
D, is controlleed by
usin
ng EMG, gyrooscope, and aaccelerometerr sensors. Thee best
way
y to detect eleectro activities in muscles is to use the EMG
E
senssor. The EMG
G sensor allow
wed us to obttain very cleaar and
imp
portant data frrom forearm m
muscles and we
w used the daata to
con
ntrol a virtual robotic
r
arm. T
To verify the virtual
v
roboticc arm
baseed on EMG, we comparedd it with anotther virtual roobotic
arm
m we made, which
w
was ccontrolled by a gyroscopee and
acceelerometer.
Usin
ng the gyrosccope and acceelerometer to control the roobotic
arm
m showed thatt the dynamic movement of
o the entire arm
a is
781
International Journal of Applied Engineering Research ISSN 0973-4562 Volume 11, Number 2 (2016) pp 775-782
© Research India Publications. http://www.ripublication.com
[10]
needed. On the other hand, the virtual robotic arm based on
EMG needs specific muscle activities.
We view the work described in this paper as only the
beginning of a large project. We intend to carry out the
complete implementation of an EMG sensor and our aim is to
use our knowledge and experimental results to make a bionic
wrist for disabled people. The lives of hand amputees can be
improved by using an artificial wrist for different purposes. In
addition, it will be easy to remove the artificial wrist if the
user does not need it at any time.
[11]
[12]
Acknowledgment
This work was supported by the National Research
Foundation of Korea Grant funded by the Korean
Government(NRF-2013S1A5A8020988).
[13]
References
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
MYO, http://www.myo.com/.
Advantages of Myo armband, http://www.
digitaltrends
.com
/pc-accessory-reviews/myogesture-control-armband-review/.
Electromyography at the US National Library of
Medicine Medical Subject Headings (MeSH),
http://en.wikipedia.org/wiki/Electromyography.
Stella Y.Botelho, “Comparison of simultaneously
recorded electrical and mechanical activity in
myasthenia gravis patients and in partially curarized
normal humans,” The American Journal of Medicine,
vol. 19, no. 5, pp. 693-696, Nov. 1995.
Yongzhe Xu, Eunju Kim, Kyunjoo Lee, Jaesug Ki
and Byungsoo Lee, “Using PhysX Simulation Fire
Model of Backdraft in Unity 3D Game Engine.
International Journal of Multimedia and Ubiquitous
Engineering,” International Journal of Multimedia
and Ubiquitous Engineering, vol. 9, no. 6, pp. 243252, June. 2014.
Kim Myung-Hoe, Jang Dae-Jin, “A study of the
Myoelectronic Hand for a Hand Amputee,” The
journal of Korean academy of physical therapist, vol.
9, no. 2, pp. 133-141, Sep. 2002.
Haeng-Kon Kim, Hyun Yeo, “Design for U-health
Care Hybrid Control Systems,” International Journal
of Software Engineering and Its Applications, vol. 8,
no. 2, pp. 375-384, Feb. 2014.
Paolo M. Rossinia, Silvestro Micerac, Antonella
Benvenutoe, Jacopo Carpanetoc, Giuseppe Cavalloe,
Luca Citic, Christian Ciprianic, Luca Denarof,
Vincenzo Denarog, Giovanni Di Pinoe, Florinda
Ferreria, Eugenio Guglielmellie, Klaus-Peter
Hoffmannh, Stanisa Raspopovicc, Jacopo Rigosac,
Luca Rossinie, Mario Tombinia and Paolo Darioc,
“Double nerve intraneural interface implant on a
human amputee for robotic hand control,” Clinical
Neurophysiology, vol. 121, no. 5, pp. 777-783, May
2010
Houghton Mifflin Harcourt, “forearm”. Webster’s
New World Medical Dictionary, (WebMD) vol. 3,
pp. 166. ISBN 978-0-544-18897-6, 2009.
782
Ho-Sun Shin, Asilbek Ganiev, and Kang-Hee Lee,
“Design of a Virtual Robotic Arm based on the EMG
variation,” Proc. ASTL, 2015.
Nae-Joung Kwak, Teuk-Seob Song, “Human Action
Recognition
Using
Accumulated
Moving
Information,” International Journal of Multimedia
and Ubiquitous Engineering, vol. 10, no. 10, pp.211222, Oct. 2015.
C. H. Lee, S. I. Kang, Bae. J. W, Kwon. D. H. LEE,
“A study of a Module of Wrist Direction Recognition
using EMG signals,” Journal of Rehabilitation
Welfare Engineering & Assisitive Technology, vol.
7, no, 1, pp. 51-58, June. 2013.
Radek Votrubec, “Stabilizatioin of Platform Using
Gyroscope,” Proc. 24 th DAAAM International
Symposium on Intelligent Manufacturing and
Automation, vol. 69, pp. 410-414, 2014.

Study on Virtual Control of a Robotic Arm via a Myo Armband for the

Transcription

Similar documents

SaeboFlex / SaeboReach Brochure

MyoTrac Flyer - Thought Technology Ltd.

New York Arm Wrestling® and Mulcahy`s Pub Presents the

MegaWin Brochure - Mega Electronics Ltd

Name: RAYMOND CRUZ

LGOBO SALE SHEET

proswIng

Quick-Lift Arm - Binar Quick

How I survived teaching my first online course

damien elite 7 - Bax-shop