universidad técnica de ambato facultad de ingeniería en sistemas

UNIVERSIDAD TÉCNICA DE AMBATO
FACULTAD DE INGENIERÍA EN SISTEMAS, ELECTRÓNICA E INDUSTRIAL
CARRERA DE INGENIERÍA EN SISTEMAS COMPUTACIONALES E
INFORMÁTICOS
Tema:
“SERVICIO DE CLOUD COMPUTING PARA LA COMUNICACIÓN
BILATERAL EN TIEMPO REAL, APLICADA A BRAZOS ROBÓTICOS”
Proyecto de Trabajo de Graduación. Modalidad: Artículos Académicos, presentado previo
la obtención del título de Ingeniero en Sistemas Computacionales e Informáticos.
SUBLÍNEA DE INVESTIGACIÓN:
Tecnologías de Comunicación.
AUTOR: Cristian Mauricio Gallardo Paredes.
PROFESOR REVISOR: Ing. Victor Hugo Andaluz Ortiz, Dr.
Ambato - Ecuador
Agosto - 2016
APROBACIÓN DEL TUTOR
En mi calidad de Tutor del Trabajo de Investigación sobre el Tema: “SERVICIO DE
CLOUD COMPUTING PARA LA COMUNICACIÓN BILATERAL EN TIEMPO
REAL, APLICADA A BRAZOS ROBÓTICOS” del señor Cristian Mauricio Gallardo
Paredes, estudiante de la Carrera de Ingeniería en Sistemas Computacionales e Informáticos,
de la Facultad de Ingeniería en Sistemas, Electrónica e Industrial, de la Universidad Técnica
de Ambato, considero que el informe investigativo reúne los requisitos suficientes para que
continúe con los trámites y consiguiente aprobación de conformidad con el Art. 16 del
Capítulo II, del Reglamento de Graduación para Obtener el Título Terminal de Tercer Nivel
de la Universidad Técnica de Ambato.
Ambato, agosto del 2016
TUTOR
______________________________
Ing. Victor H. Andaluz, Dr.
ii
AUTORÍA
El presente Proyecto de Investigación titulado: “SERVICIO DE CLOUD COMPUTING
PARA LA COMUNICACIÓN BILATERAL EN TIEMPO REAL, APLICADA A
BRAZOS ROBÓTICOS”, es absolutamente original, auténtico y personal, en tal virtud, el
contenido, efectos legales y académicos que se desprenden del mismo son de exclusiva
responsabilidad del autor.
Ambato, agosto del 2016
______________________________
Cristian Mauricio Gallardo Paredes
CC: 2000093084
iii
DERECHOS DE AUTOR
Autorizo a la Universidad Técnica de Ambato, para que haga uso de este Trabajo de
Titulación como documento disponible para la lectura, consulta y procesos de investigación.
Cedo los derechos de mi Trabajo, con fines de difusión pública, además autorizo su
reproducción dentro de las regulaciones de la Editorial Springer y de la Universidad.
Ambato, agosto del 2016
______________________________
Cristian Mauricio Gallardo Paredes
CC: 2000093084
iv
APROBACIÓN COMISIÓN CALIFICADORA
La Comisión Calificadora del presente trabajo conformada por los señores docentes Ing. Pilar
Urrutia y Ing. David Guevara, revisaron y aprobaron el Informe Final del trabajo de
Investigación
titulado
“SERVICIO
DE
CLOUD
COMPUTING
PARA
LA
COMUNICACIÓN BILATERAL EN TIEMPO REAL, APLICADA A BRAZOS
ROBÓTICOS.”, presentado por el señor Cristian Mauricio Gallardo Paredes, de acuerdo al
Art. 17 del Reglamento de Graduación para obtener el título Terminal de tercer nivel de la
Universidad Técnica de Ambato.
__________________________
Ing.Vicente Morales. Mg.
PRESIDENTE DEL TRIBUNAL
_______________________
Ing. Pilar Urrutia. Mg.
_______________________
Ing. David Guevara. Mg.
DOCENTE CALIFICADOR
DOCENTE CALIFICADOR
v
DEDICATORIA
El presente proyecto se lo dedico a mis
padres, por su apoyo en mi continuo
crecimiento personal y profesional, a los
maestros que tuve a lo largo de mi vida
ya que siempre llevaré sus enseñanzas
en cada uno de mis pasos.
“-¿Por qué lo deseas tanto?
-Porque me dijeron que no lo lograría”.
Cristian Mauricio Gallardo Paredes
vi
AGRADECIMIENTO
Quiero expresar mis más sinceros
agradecimientos a mis padres, gracias a
ellos soy una persona de bien por
inculcarme siempre buenos valores y
por demostrarme que lo imposible no
existe.
A la Facultad de Ingeniería en Sistemas,
Electrónica
e
Industrial
de
la
Universidad Técnica de Ambato, por ser
el lugar en donde compartí con grandes
maestros y amigos que ayudaron en mí
desarrollo profesional, ético y moral.
Muchos buscan plenitud en tiempo y
espacio determinado y los llaman
“momentos especiales”, puedo decir
que me siento afortunado ya que en
medio
de
tristezas,
alegrías,
incertidumbre, festejos y dificultades, en
todos, he tenido personas invaluables a
mi lado y eso hizo que todos los
momentos sean especiales para mí.
A todos quienes supieron brindarme su
apoyo, mis más sincero y eterno gratitud
a cada uno de ellos.
Cristian Mauricio Gallrado Paredes
vii
ÍNDICE
APROBACIÓN DEL TUTOR
ii
AUTORÍA
iii
APROBACIÓN COMISIÓN CALIFICADORA
v
DERECHOS DE AUTOR
iv
DEDICATORIA
vi
AGRADECIMIENTO
vii
RESUMEN EJECUTIVO
x
ABSTRACT
x
SECCIÓN 1 Introduction
1 (500)
SECCIÓN 2 Bilateral Teleoperation System
3 (502)
SECCIÓN 3 Communication Channel: Cloud Computing Services
5 (504)
SECCIÓN 4
5 (504)
Remote Site: Robotic Arm
4.1 Kinematic and Dynamic Model
6 (505)
4.2 Control Design
7 (506)
4.2.1 Kinematic Controller
8 (507)
4.2.2 Controller with Dynamic Compensation
8 (507)
9 (508)
SECCIÓN 5 Local Site
SECCIÓN 6 Experimental Results
10 (509)
SECCIÓN 7 Conclusions
13 (512)
References
14 (513)
viii
INDICE DE FIGURAS
Fig. 1.
Block Diagram of the bilateral tele-operation system
3
Fig. 2.
Block diagram of the bilateral tele-operation system
4
Fig. 3.
FALCON Novint Technologies Incorporated
4
Fig. 4.
OpenStack: Infraestructure
6
Fig. 5.
The orthogonal projection of the point of interest over the path
7
Fig. 6.
Block diagram of the virtual environment
9
Fig. 7.
Robotic arm 6DOF developed at the University of the Armed Forces
10
ESPE
Fig. 8.
Local site of the tele-operation scheme
11
Fig. 9.
Robotic Arm 6DOF developed in SolidWorks
11
Fig. 10.
Stroboscopic movement of the robotic arm in Unity 3D
12
Fig. 11.
Comparison between the references generated by the human operator
13
and the actual velocities of the end effector (Color Figure online)
Fig. 12.
Evolution of the control actions of the robotic arm (color figure online)
ix
13
RESUMEN EJECUTIVO
Este trabajo presenta el diseño de un sistema de teleoperación bilateral de un brazo robótico. Se
propone un nuevo protocolo de comunicación prototipo con WebSockets para la comunicación
y JSON para la estructuración de datos en un entorno de computación en la nube con OpenStack
y OpenShift Origin. El operador humano recibe retroalimentación visual y fuerza desde el sitio
remoto, y envía los comandos de posición para el esclavo. Además, en el sistema de teleoperación se propone que el operador humano está inmerso en un entorno de realidad aumentada
para tener una mayor transparencia del sitio remoto. La transparencia de un sistema de teleoperación indica una medida de cómo el humano siente el sistema remoto. Finalmente, se
presentan los resultados experimentales para verificar el rendimiento del sistema propuesto.
ABSTRACT
This work presents the design of a bilateral teleoperation system for a robotic arms. It proposes
a new prototype communication protocol with Websockets for the communication and Json for
data structuration on a cloud computing environment with OpenStack and Openshift Origin. The
human operator receives visual and force feedback from the remote site, and it sends position
commands to the slave. Additionally, in the tele-operation system it is proposed that the human
operator is immersed in an augmented reality environment to have greater transparency of the
remote site. The transparency of a tele-operation system indicates a measure of how the human
feels the remote system. Finally, the experimental results are reported to verify the performance
of the proposed system.
x
Cloud Computing Services for Real Time
Bilateral Communication,
Applied to Robotic Arms
Cristian Gallardo1 and Víctor Hugo Andaluz1,2(&)
1
2
Universidad Técnica de Ambato, Ambato, Ecuador
[email protected]
Universidad de Las Fuerzas Armadas ESPE, Sangolquí, Ecuador
[email protected]
Abstract. This work presents the design of a bilateral teleoperation system for
a robotic arm. It proposes a new prototype communication protocol with
Websockets for the communication and Json for data structuration on a cloud
computing environment with OpenStack and Openshift Origin. The human
operator receives visual and force feedback from the remote site, and it sends
position commands to the slave. Additionally, in the tele-operation system it is
proposed that the human operator is immersed in an augmented reality environment to have greater transparency of the remote site. The transparency of a
tele-operation system indicates a measure of how the human feels the remote
system. Finally, the experimental results are reported to verify the performance
of the proposed system.
Keywords: Cloud computing
following Virtual reality
Bilateral teleoperation
Robotic arm
Path
1 Introduction
During the last decades, various models of field robots have been developed to take the
place of humans in dangerous tasks, such as rescue missions, Mars exploration, airport
patrols, and missions in war [1–3]. In order to operate in highly variable, unstructured,
unknown, or dynamic environments, an advanced tele-robotic system with increased
adaptability and human like manipulation capabilities is required. In general, a
tele-operation system is composed of a local site, where a human operator drives a
hand-controller named master; a remote site, where a robot named slave follows the
motion of the master to execute a given task in interaction with the environment; and a
communication channel that links both sites. The master is used to generate velocity or
position commands which are sent to the remote site, while the force due to the
interaction between the slave and the environment is back-fed to the human operator
through the actuators of the master [4–7].
Cloud computing is a paradigm in the way that the computing resources and
applications are used and delivered as a services. These main resources as a service are
computing, storage and the network infrastructure. Cloud computing refers to providing these resources as a service over the Internet to the public or in an organization
© Springer International Publishing Switzerland 2016
L.T. De Paolis and A. Mongelli (Eds.): AVR 2016, Part I, LNCS 9768, pp. 500–514, 2016.
DOI: 10.1007/978-3-319-40621-3_38
Cloud Computing Services for Real Time Bilateral Communication
501
that is of private use [8]. There are three types of cloud services and they differ in the
approach on how the resources are made available. The first approach is to make the
hardware infrastructure available as a service and is called Infrastructure as a Service
(IaaS). The second approach is to provide a platform (the OS along with the necessary
software, frameworks and tools) over the hardware infrastructure. This is called Platform as a Service (PaaS). The third approach is to provide the application as a service
and is called Software as a Service (SaaS) [9–12].
Recently, many researches about cloud communication are taking an increasing
interest in the field of robot services for manipulation and getting information from
them, e.g., Cloud-Based Robot services [13] that examining previous implementations
RT-Middleware, ROS and make a new proposal (RSNP) that is based on web service
technologies, similarly many researches have been made about cloud computing and
how we can interact with it, but almost all of this interactions are by standard protocol
http like http polling: that make a request to the server and the server answer this
request, other protocols that give more interaction client-server are: push comet, long
polling, flash, XMR, htmlfiles and others technologies like webservice. Cloud computing is a new wave in the field of technology information. Some see it as an emerging
field in computing. They it consists of a set of resources and services offered by
In-Internet. Therefore, “cloud computing” is also called “computer on the Internet.
“The word” cloud “is a metaphor to describe the Web as a space where the computer
was preinstalled and exists as a service. (Cloud Computing: Opportunities and Challenges) the main characteristic of cloud computing is the virtualization of network,
storage and compute [14–17].
In such context, this work proposes a bilateral teleoperation system in order to
allow the handling of objects. It comprises a robotic arm (slave) so that it can move and
manipulate objects. The human operator receives visual and force signals and sends
position commands generated by a haptic device (master) to the remote site; furthermore, the local site has proposed a virtual environment and augmented reality implemented in Unity3D. The augmented reality provides feedback of different signals from
the remote site in real time, so that through different haptic devices stimulate the senses
of sight, touch and hearing of the human operator so that it can “transmit” their skill,
experience and expertise to the robot to perform a task. Whereas for the channel
communication is proposed a new implementation of teleoperation-communication
(Robot + Cloud) but in the communication using WebSockets and Json data Structure
unlike in the works found in the literature. On the other hand, the design of the
teleoperation system structure is mainly composed by two parts, each one being a
controller itself. The first one is a minimum norm controller to solve the path following
problem which considered the desired position of the person to move of the arm. The
second one is a dynamic compensation controller, which receives as inputs the velocity
references calculated by the kinematic controller. To validate the proposed teleoperation system, results are included and discussed.
The work is organized as follows: in Sect. 2 describes the bilateral teleoperation
system proposed. Section 3 presents the implementation of communication channel
through the cloud computing services; while that the kinematic and dynamic modeling
of the robotic arm are shown in Sect. 4, also of the controllers design and the analysis
502
C. Gallardo and V.H. Andaluz
of the system’s stability. Next in Sect. 5 the design of the local site of the human
operator developed, and finally the results are presented and discussed in Sects. 6
and 7, respectively.
2 Bilateral Teleoperation System
A bilateral tele-operation system for a robotic arm is proposed, in which forward and
backward delay between local and remote sites are assumed to be irrelevant ðf1 ðtÞ ¼ 0
and f2 ðtÞ ¼ 0Þ. The proposed tele-operation system is shown in Fig. 1. In this system
both the force back-fed to the human operator are considered.
Fig. 1. Block diagram of the bilateral tele-operation system
For communication channel of the bilateral tele-operation system this work proposed a new implementation of teleoperation-communication (Robot + Cloud) with a
communication using websockets and Json data Structure as shown in Fig. 2. Websockets is a standardized communication protocol that give a full-duplex, bidirectional
channel communication and help to make scalable applications giving a high performance communication through Internet in this case on a cloud computing infrastructure; while that Json Data Structure (JavaScript Object Notation) is a light weight
format for data interchange.
On other hand, a given mission is generally composed of several operation processes, in this context the human operator controls the robotic arm by sending position
commands to the end-effector of the robot: hl , hm , and hn , one for each axis in respect to
the inertial frame RðX ; Y; ZÞ, using a haptic device hd ¼ ½ hl hm hn T . The human
operator commands are generated with the use of the FALCONTM from Novint
Technologies Incorporated as indicated in Fig. 3. Its positions Px , Py , and Pz are
translated into position commands hl , hm and hn for the manipulation mode, through the
following rotation matrix,
Cloud Computing Services for Real Time Bilateral Communication
2
3 2
hl
cosðq1 Þ
4 hm 5 ¼ 4 sinðq1 Þ
hn
0
32 3
sinðq1 Þ 0
Px
cosðq1 Þ 0 54 Py 5
0
1
Pz
503
ð1Þ
where q1 represents the first joint of the robotic arm that rotates about the axis Z.
Fig. 2. Block diagram of the bilateral tele-operation system
In the case of robotic manipulators, the force feedback is the result of a physical
interaction with the environment.
Fig. 3. FALCONTM Novint Technologies Incorporated
504
C. Gallardo and V.H. Andaluz
On the other hand, the human operator has the possibility of changing the desired
internal configuration of the robotic arm at any time during the execution of a mission,
through the visual display (HMI), i.e., g ¼ ½ q1d q2d qnd T . This desired
configuration vector may or may not have the values that maximize manipulability of
the robotic arm.
3 Communication Channel: Cloud Computing Services
The classification of communication infrastructure of the cloud computing is [13]
(i) Peer-based model where each virtual machine in the cloud is considered as a
computing unit and the robots and Vms form a fully distributed computing mesh;
(ii) Clone-based model where each robot corresponds to a system level clone in the
cloud. A task can be executed in the robot or in its clone. This model allows for
sporadic outage in the physical (Machine to machine) M2 M network; and (iii) ProxyBased Model in the group of networked robots, one unit functions as a group leader, the
communication with a proxy VM in the cloud infrastructure. For this work the
roxy-Based Model is implemented because this model has a best performance in the
communication speed. In our proposed model considers an internal cloud infrastructure
composed of three nodes, as shown Fig. 4., where, (a) Controller & Block Storage
node this node mainly contain: Controller administrate all the openstack resources,
administration of services, api end points, projects, users and roles. Block storage
service provides block storage devices to guest instances (Cinder service). Image
Storage service enables users to discover, register and retrieve virtual machine images
(Glance). Identity service perform tracking users and their permissions and providing a
catalog of available services with their API endpoints (Keystone). Dashboard service is
a web interface that enables cloud administrator and users to manage OpenStack
resources and services (Horizon); (b) Network & Object storage node this node mainly
contain: Network service allows to create and attach interface devices managed by
other OpenStack services to network (Neutron). Object Storage Service work together
to provide object storage and retrieval through a REST API (Swift); and finally
(c) Compute Node this node contain Compute Service allow to host and manage cloud
computing systems this host contain the Hypervisor (Nova).
4 Remote Site: Robotic Arm
The manipulator configuration is defined by a vector qðtÞ of n independent coordinates,
called generalized coordinates of the manipulator, where q ¼ ½ q1 q2 qn T
represents the generalized coordinates of the robotic arm. The configuration q is an
element of the manipulator configuration space; denoted by N . The location of the
end-effector of the mobile manipulator is given by the m–dimensional vector h ¼
½ h1 h2 hm T which defines the position and the orientation of the end-effector
of the manipulator in R. Its m coordinates are the operational coordinates of the
manipulator. The set of all locations constitutes the manipulator operational space,
denoted by M.
Cloud Computing Services for Real Time Bilateral Communication
505
Fig. 4. OpenStack: Infrastructure
4.1
Kinematic and Dynamic Model
The kinematic model of a robotic arm gives the derivative of its end-effector location as
a function of the derivatives of both the robotic arm configuration and the location of
the mobile platform,
h_ ðtÞ ¼ JðqÞq_ ðtÞ
ð2Þ
where JðqÞ is the Jacobian matrix that defines a linear mapping between the vector of
the robotic arm velocities q_ ðtÞ and the vector of the end-effector velocity h_ ðtÞ.
On the other hand, the mathematic model that represents the dynamics of a robotic
arm can be obtained from Lagrange’s dynamic equations, which are based on the
difference between the kinetic and potential energy of each of the joints of the robot
(energy balance) [18]. The dynamic model of the robotic arm, having as control signals
the reference velocities of the system, can be represented as follows [19],
506
C. Gallardo and V.H. Andaluz
MðqÞ€
q þ Cðq; q_ Þq_ þ gðqÞ ¼ q_ ref
ð3Þ
þ PÞ, gðqÞ¼ H1 gðqÞ. Thus, MðqÞ 2
þ DÞ, Cðq; q_ Þ¼ H1 C
where, MðqÞ ¼ H1 ðM
dn x dn
ðq; vÞv 2 <dn , G
ðqÞ 2 <dn and q_ ref 2 <dn is the
<
is a positive definite matrix, C
vector of velocity control signals, H 2 <dn x dn , D 2 <dn x dn and P 2 <dn x dn are constant
symmetrical diagonal matrices, positive definite, that contain the physical parameters of
the robotic arm, e.g., motors, velocity controllers. More details about the dynamic
model (5) can be found in [19].
4.2
Control Design
From the viewpoint of control theory, the end-effector of the robotic arm must follow the
path desired by a person and generated through brain signals, as shown in Fig. 5. As
represented in Fig. 5, the path to be followed is denoted as
, where
; the actual desired location Pd ðsD Þ ¼ ðxP ðsD Þ;
yP ðsD ÞzP ðsD ÞÞ is defined as desired point of the person, i.e., the closest point on
,
with sD being the curvilinear abscissa defining the point Pd ; ~x ¼ xP ðsD Þ x is the position
error in the
direction; ~y ¼ yP ðsD Þ y is the position error in the direction; ~z ¼
zP ðsD Þ z is the position error in the direction; q represents the distance between the
end-effector position of the robotic arm hðx; y; zÞ and the desired point for the person Pd ,
~ ¼ 0 q ¼ q, i.e., the desired distance
where the position error in the q direction is q
between the end-effector position hðx; y; zÞ and the desired point Pd must be zero.
Fig. 5. The orthogonal projection of the point of interest over the path
Hence, the path following problem is to find the control law for the robotic arm as a
function of the control errors (position and orientation of the end-effector) and the
desired velocities of the end-effector
Cloud Computing Services for Real Time Bilateral Communication
q_ ref ðsD ; hÞ ¼ f ðqðt; sÞ; tP ðsD ; hÞÞ
507
ð4Þ
such that ~x ¼ 0, ~y ¼ 0 and ~z ¼ 0. Therefore, if lim ~xðtÞ ¼ 0, lim ~yðtÞ ¼ 0 and
lim ~zðtÞ ¼ 0 then lim qðtÞ ¼ 0.
t!1
t!1
t!1
t!1
4.2.1 Kinematic Controller
The design of the kinematic controller of the robotic arm is based on the kinematic
model of the arm (4). The following control law is proposed
~ þ I J# J LB tanh L1 B K
q_ c ¼ J# td þ LK tanh L1
K Kh
B
ð5Þ
1
where J# ¼ W1 JT JW1 JT , being W a definite positive matrix that weighs the
~
control actions of the system, td is the desired velocities vector of the end-effector h, h
~ ¼ hd h, B and LB are definite positive
is the vector of control errors, defined as h
diagonal matrices that weigh the vector K. In order to include an analytical saturation
of velocities in the robotic arm the use of the tanh (.) function is proposed, which limits
~ and the magnitude of the vector K. The second term of (5) represents the
the error in h
projection on the null space of J, where K is an arbitrary vector which contains the
velocities associated to the robotic arm. Therefore, any value given to K will have
effects only on the internal structure of the arm, and will not affect the final control of
the end-effector at all. By using this term, different secondary control objectives can be
achieved effectively [20].
4.2.2 Controller with Dynamic Compensation
The proposed kinematic controllers presented in Subsect. 4.2.1 assume perfect velocity
tracking; nevertheless this is not true in real contexts, i.e., q_ ðtÞ 6¼ q_ c ðtÞ, mainly when
high-speed movements or heavy load transportation are required. Therefore, it becomes
essential to consider the robot’s dynamics, in addition to its kinematics. Then, the
objective of the dynamic compensation controller is to compensate the dynamics of the
system, thus reducing the velocity tracking error. This controller receives as inputs the
desired velocities calculated by the kinematic controllers, and generates velocity references for the mobile platform and robotic arm. Hence, relaxing the perfect velocity
~_ ðtÞ¼ q_ c ðtÞq_ ðtÞ. This
tracking assumption, there will be a velocity error defined as, q
velocity error motivates the dynamic compensation process, which will be performed
based on the inverse dynamics of the system. With this aim, the exact model of the
system without disturbances is considered, thus the following control is proposed,
q_ ref ¼ MðqÞr þ Cðq; q_ Þq_ þ gðqÞ
ð6Þ
_ . Now, Replacing (6) in (5) it results
€
~
~c þ Lv tanh L1
where r ¼ q
q
K
v
v
1
€
_
~ þ Lv tanh Lv Kv q
~ ¼ 0. For the stability analysis the following Lyapunov’s
q
508
C. Gallardo and V.H. Andaluz
T
~_ ¼ 12q
~_ ¼
~_ q
~_ , its time derivative is V_ q
candidate function is considered V q
T
~_ Lv tanh L1
~_ \0. Thus it can be immediately concluded that the error vector
q
v Kv q
~_ ðtÞ ¼ 0 asymptotically, provided that Kv and Lv are symmetrical positive definite
lim q
t!1
matrices.
5 Local Site
The local site consists of a virtual reality and augmented to allow the human operator to
have greater transparency in the remote site reality, as illustrated by the Fig. 6. The
Fig. 6 can be described in two stages, the first stage is (A) Inputs, These let read all the
variables considered in the bilateral tele-operation of both the robot and the environment in which develops the task, entries that are processed in the script are: (i) Output
states of the robot, represents the variables that describe the robot kinematics
q ¼ ½ q1 q2 qn T . In the script, the data acquired from the platform q are
Fig. 6. Block diagram of the virtual environment
Cloud Computing Services for Real Time Bilateral Communication
509
assigned to the characteristic of Rotation for each joint of the simulated arm Game
Object; (ii) Desired positions through the FALCONMT haptic device emit the desired
location hd ¼ ½ hl hm hn T for manipulation mode. The script obtains the variation
of position in world coordinates by a Game Object that simulates the movement of the
end-effector of FALCONMT haptic device in space. (iii) Video and audio transmits the
image and audio captured by the camera located at the end-effector of the robot; (iv)
Fictitious force is produced by a sensor that maps the area in which the robot can be
moved freely; y finally (v) Oculus Positional Tracking modifies the rotation of the
camera in the virtual and / or augmented environment, corresponding to the movement
performed by the human operator when using the Head-mounted Display, HMD, in
order to have a greater immersion in the experience of using the application.
Furthermore, the second stage of Fig. 6 is (B) Outputs Script that recreates the
sensations that the human operator should feel if he were at the remote site, this paper
considers three ways that help the human operator to “transmit” their ability and
experience to the robot to perform a task.
6 Experimental Results
In order to illustrate the performance of the proposed tele-operation structure, several
simulation experiments are carried out for control of a robotic arm, most representative
results are presented in this section. The experiments were carried with the kinematic
and dynamic models of a robotic arm 6 DOF which was developed at the University of
the Armed Forces ESPE (only 3 DOF of the 6 available DOFs are used in the
experiments, see Fig. 7). The links of the robotic arm are controlled by a network of
servomotors Dinamixel communicated via a RS-485 interface. Therefore to give
greater scalability for future functions of the cloud computing environment are used
three computers with the same characteristics, intel core i7-3770 Processor, 6 GB ram
memory, two Hard disk sata connection of 750 GB for each computer, CentOS Linux 7
X64 with minimal install for the base system, the network connection are using Cat6a
devices, for The IaaS infrastructure of the Cloud-Computing environment was used
Fig. 7. Rootic arm 6DOF developed at the University of the Armed Forces ESPE
510
C. Gallardo and V.H. Andaluz
OpenStack Kilo, then created a OpenShift project with php5.4 support on the cloud
infrastructure for the communication service and presentation of the data.
The local site has an Oculus Rift HMD and a haptic device FALCONTM Novint.
To transmit the comands generated by humans with was used a FALCONTM haptic
device from Novint Technologies Incorporated as indicated in Fig. 8; while the Fig. 9
shown a robotic arm 6DOF developed in SolidWorks.
Fig. 8. Local site of the tele-operation scheme
Fig. 9. Rootic Arm 6DOF developed in SolidWorks
The FALCONTM Novint communication functions were developed in C ++ language
to create dynamic link libraries to use on LabView environment to get FALCONTM signals,
for testing of FALCONTM hosts were used windows 8.1 X64 and windows 7 X86, for test
Cloud Computing Services for Real Time Bilateral Communication
511
of C ++ dll compatibility on each Operative System. On labView was created some
operations and functions, the data that will be send are set in Json data structure, to
encapsulate and send to the cloud was used websocket protocol, for keeping the connection
on the cloud with all the clients, each client must set a id and store the connection in a
structure list in memory where the data will be decoded and will show information of the
origin, the destination and the message, on the server for manage all the connections thread,
the service on the host were developed with php + javascript + html languages with
phpwebsockets, the fancywebsocket libraries and codeigniter framework. The connection
to the cloud of the arm and the functions are developed with C ++ language on GNU/Linux
Operative System.
Hence, Figs. 10–12 show the results of the experiment of bilateral tele-operation.
Figure 10 shows the stroboscopic movement on the X-Y-Z space of Unity3D. Figure 11
shows a comparison between the reference generated by the human operator and the
Fig. 10. Stroboscopic movement of the robotic arm in Unity 3D.
512
C. Gallardo and V.H. Andaluz
Eje x
[m/s]
1
0
-1
0
5
10
15
20
25
Tiempo [s]
Eje y
30
35
40
45
50
5
10
15
20
25
Tiempo [s]
Eje z
30
35
40
45
50
5
10
15
20
25
Tiempo [s]
30
35
40
45
50
[m/s]
1
0
-1
0
[m/s]
0.2
0
-0.2
0
Comandos del operador
Velocidades del Robot
Fig. 11. Comparison between the references generated by the human operator and the actual
velocities of the end effector (Color figure online)
0.2
0.1
V. Angular [rad/s]
0
-0.1
-0.2
-0.3
-0.4
-0.5
-0.6
-0.7
0
Art. 1
5
Art. 2
10
Art. 3
15
20
25
Tiempo [s]
30
35
40
45
50
Fig. 12. Evolution of the control actions of the robotic arm (Color figure online)
actual velocities of the end effector; while the Fig. 12 depicts the time evolution of the
control actions for the joints of the robotic arm.
7 Conclusions
In this paper a bilateral tele-operation system is proposed, in this system is considers
that human operator has greater transparency of the remote. This system allows a
human operator to perform complex tasks in a remote environment with a robotic arm.
The teleoperation system use a communication channel that implements a new prototype of communication protocol with WebSockets and Json data structures on a cloud
computing environment with OpenStack. Therefore, experimental results were also
presented showing the feasibility and the good performance of the proposed teleoperation structure.
Cloud Computing Services for Real Time Bilateral Communication
513
Acknowledgment. The authors would like to thanks to the Consorcio Ecuatoriano para el
Desarrollo de Internet Avanzado -CEDIA-, Universidad de las Fuerzas Armadas ESPE and the
Universidad Técnica de Ambato for financing the project Tele-operación bilateral cooperativo
de múltiples manipuladores móviles – CEPRAIX-2015-05, for the support to develop this paper.
References
1. Andaluz, V.H., Canseco, P., Varela, J., Ortiz, J.S., Pérez, M.G., Morales, V., Robertí, F.,
Carelli, R.: Modeling and control of a wheelchair considering center of mass lateral
displacements. In: Liu, H., Kubota, N., Zhu, X., Dillmann, R. (eds.) ICIRA 2015. LNCS,
vol. 9246, pp. 254–270. Springer, Heidelberg (2015)
2. Slawinski, E., Mut, V.: PD-like controllers for delayed bilateral teleoperation of
manipulators robots. Int. J. Robust Nonlinear Control 25(12), 1801–1815 (2015)
3. Voth, D.: A new generation of military robots. Intell. Syst. 19(4), 2–3 (2004)
4. Sheridan, T.B.: Telerobotics, Automation, and Human Supervisory Control. The MIT Press,
Cambrige (1992)
5. Stramigioli, S., Mahony, R., Corke, P.: A novel approach to haptic tele-operation of aerial
robot vehicles. In: Proceedings of the IEEE International Conference on Robotics and
Automation (ICRA 2010), pp. 5302–5308, May 2010
6. Niculescu, S.-I., Taoutaou, D., Lozano, R.: Bilateral teleoperation with communication
delays. Int. J. Robust Nonlinear Control 13(9), 873–883 (2003)
7. Yin, S., Ding, S.X., Xie, X., Luo, H.: A review on basic datadriven approaches for industrial
process monitoring. IEEE Trans. Ind. Electron. 61(11), 6414–6428 (2014)
8. Griffith, R., Joseph, A.D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A.,
Stoica, I., Armbrust, M., Fox, A., Zaharia, M.: Above the clouds: A berkeley view of cloud
computing [white paper] (2009). http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS2009-28.pdf
9. Amazon ec2. amazon elastic compute cloud (2009). http://aws.amazon.com/ec2/
10. Google app engine (2009). http://code.google.com/appengine/
11. Wurm, K.M., Stachniss, C., Burgard, W.: Coordinated multi-robot exploration using a
segmentation of the environment. In: International Conference on Intelligent Robots and
Systems (2008)
12. Fox, D.: Distributed multi-robot exploration and mapping. In CRV 2005: Proceedings of the
2nd Canadian Conference on Computer and Robot Vision, pp. 15–xv. IEEE Computer
Society, Washington, DC (2005)
13. Dhiyanesh, B.: Dynamic Resource Allocation for Machine to Cloud Communications
Robotics Cloud. In: IEEE International Conference on Emerging Trends in Electrical
Engineering and Energy Management, pp. 451–454 (2012)
14. Guizzo, E.: Robots with their head in the clouds. IEEE Spectrum, Robotics, March 2011
15. Asaro, P.: Remote-control crimes: roboethics and legal jurisdictions of tele-agency, Special
issue on Roboethics, Veruggio, G., Van der Loos, M., Solis, J. (eds.) IEEE Robot. Autom.
Mag. 18(1), 68–71 (2011)
16. Shiraz, M., Sookhak, M., Gani, A., Shah, S.A.A.: A Study on the critical analysis of
computational offloading frameworks for mobile cloud computing. J. Netw. Comput. Appl.
47, 47–60 (2015)
17. Riazuelo, L., Civera, J., Montiel, J.M.M.: C2TAM: a cloud framework for cooperative
tracking and mapping. Robot. Auton. Syst. 62(4), 401–413 (2014)
514
C. Gallardo and V.H. Andaluz
18. Siciliano, B., Sciavicco, L., Villani, L., Oriolo, G.: Robotics: Modeling, Planning, and
Control. Springer, London (2009). ISBN 978-1-84628-641-4
19. Andaluz, V., Roberti, F., Carelli, R.: Robust control with redundancy resolution and
dynamic compensation for mobilemanipulators. In Proc. of the IEEE-ICIT International
Conference on Industrial Technology, pp. 1449–1454 (2010)
20. Bayle, B., Fourquet, J.-Y.: Manipulability analysis for mobile manipulators. In: Proceedings
of the IEEE International Conference on Robots & Automation, pp. 1251–1256 (2001)