D3.1 - ClouT

Transcription

D3.1 - ClouT

FP7-ICT-2013-EU-Japan
ClouT: Cloud of Things for empowering the citizen clout in smart cities
FP7 contract number: 608641
NICT management number: 167 ア
Project deliverable
D3.1 - Reusable components, techniques and standards for
City Platform as a Service (CPaaS)
A B S TR A C T
The objective of this document is to describe the reusable components in the City Platform-as-aService (CPaaS) layer.
The services, and the API’s exposed by CPaaS infrastructure, depend on the requirements and on
the architecture defined in WP1. WP3 also describes the non-functional requirements, such as
the security requirements. This WP is focused on city information infrastructure, cloud storage
components, with the dependents interfaces, security layer and all the infrastructure necessary
to access and manage data. WP3 is focused also on the access control and fault-tolerance
methods for access and manage data.
D3.1 - Reusable components and techniques for CPaaS
Disclaimer
This document has been produced in the context of the ClouT Project which is jointly funded by the
European Commission (grant agreement n° 608641) and NICT from Japan (management number
167ア). All information provided in this document is provided "as is" and no guarantee or warranty is
given that the information is fit for any particular purpose. The user thereof uses the information at its
sole risk and liability. This document contains material, which is the copyright of certain ClouT
partners, and may not be reproduced or copied without permission. All ClouT consortium partners
have agreed to the full publication of this document. The commercial use of any information contained
in this document may require a license from the owner of that information.
For the avoidance of all doubts, the European Commission and NICT have no liability in respect of
this document, which is merely representing the view of the project consortium. This document is
subject to change without notice.
The ClouT consortium is composed of the following institutions
No.
Participant organisation name
Short name
Country
1
Commissariat à l’énergie atomique et aux énergies alternatives
CEA
(coordinator)
France
2
Engineering Ingegneria Informatica SpA
ENG
Italy
3
University of Cantabria
UC
Spain
4
STMicroelectronics S.r.l.
ST
Italy
5
Santander City Municipality
SAN
Spain
6
Genova Municipality
GEN
Italy
7
Nippon telegraph and telephone East Corporation
(NTT East)
NTTE
(coordinator)
Japan
8
Nippon Telegraph and telephone corporation (NTT
R&D)
NTTRD
Japan
9
Keio University
KEIO
Japan
10
Panasonic System Solution
PANA
Japan
11
National Institute of Informatics
NII
Japan
ClouT – 31.07.2013
Page 2
EU Editor
Cosimo Greco, ENG
JP Editor
Kenji Tei, NII
Authors
[Cosimo Greco, ENG], [Levent Gurgen, CEA], [Yazid Benazzouz, CEA],
[Stefania Manca, GEN], [Takuro Yonezawa, Keio], [Kenji Tei, NII], [Fuyuki
Ishikawa, NII], [Jose Antonio Galache, UC], [Fernando Mons Nunez, SAN]
Internal reviewer
Marco GRELLA, ST
Deliverable type
R
Dissemination level
(Confidentiality)
PU
Contractual Delivery Date
31/07/2013
Actual Delivery Date
31/07/2013
Keywords
ClouT, Cloud Computing, IoT, Smart Cities, CPaaS, Reusable Components
Revision history
Revision
Date
Description
Author (Organisation)
V0.1
10.06.2013
Table of Contents
created
ENG
V0.2
08.07.2013
First draft with
contributions from all
partners
ENG
V0.3
12.07.2013
Second draft with
contributions from all
partners
ENG
V0.4
19.07.2013
Added Santander
contributions.
ENG
V0.5
19.07.2013
First consolidated
version sent for internal
review
ENG
V0.6
26.07.2013
Reviewed version
ST
V1.0
31.07.2013
Final version
ENG
ClouT – 31.07.2013
Page 3
TABLE OF CONTENTS
TABLE OF CONTENTS ........................................................................................................................................ 4
LIST OF FIGURES ............................................................................................................................................... 5
LIST OF TABLES ................................................................................................................................................. 7
LIST OF ABBREVIATIONS AND DEFINITIONS ..................................................................................................... 8
EXECUTIVE SUMMARY ..................................................................................................................................... 9
1.
INTRODUCTION .................................................................................................................................... 10
1.1.
1.2.
1.3.
2.
SCOPE OF THE DOCUMENT ....................................................................................................................... 10
TARGET AUDIENCE .................................................................................................................................. 10
STRUCTURE OF THE DOCUMENT ................................................................................................................ 10
SERVICE COMPOSITION PLATFORMS FOR CITIZEN’S APPLICATIONS ................................................. 12
2.1.
INTRODUCTION ....................................................................................................................................... 12
2.2.
SERVICE COMPOSITION AND MASH-UP TOOLS REUSABLE COMPONENTS .......................................................... 13
WIRECLOUD ................................................................................................................................................ 13
Mycocktail .................................................................................................................................................... 14
Enterprise Mashup Markup Language ....................................................................................................... 15
OpenSocial ................................................................................................................................................... 16
Yahoo! Pipes ................................................................................................................................................. 17
Apache Shindig ............................................................................................................................................ 18
iPojo ............................................................................................................................................................. 20
BPMN ........................................................................................................................................................... 22
BPEL ............................................................................................................................................................. 24
2.3.
DEPENDABLE SERVICE COMPOSITIONS REUSABLE COMPONENTS ................................................................... 25
Robust Service Composition METHOD ........................................................................................................ 25
QoS-based Service/CLOUD Selection Method ............................................................................................. 26
Metadata-based Behavior Insertion FRAMEWORK ................................................................................... 27
Verification Framework of Time and Resource Constraints on Business Process ..................................... 28
Verification Framework of ECA Specification on Physical Interactions .................................................... 29
3.
BIG DATA PROCESSING ........................................................................................................................ 30
3.1.
INTRODUCTION ....................................................................................................................................... 30
3.2.
DATA/EVENT PROCESSING AND DECISION MAKING REUSABLE COMPONENTS ................................................... 31
Esper ............................................................................................................................................................ 31
FI-WARE Gateway Data handling ............................................................................................................... 33
Jboss Drools Expert ...................................................................................................................................... 35
MongoDB ..................................................................................................................................................... 36
3.3.
SELF-HEALING FOR DATA /EVENT STREAMING REUSABLE COMPONENTS .......................................................... 37
Self-healing Framework for Sensory Data .................................................................................................. 37
Fault Classification Model ........................................................................................................................... 39
4.
SECURE AND DEPENDABLE ACCESS TO CITY DATA ............................................................................. 40
4.1.
INTRODUCTION ....................................................................................................................................... 40
4.2.
OPEN CITY DATA HOSTING AND ACCESS REUSABLE COMPONENTS .................................................................. 41
HYPERTABLE ............................................................................................................................................... 42
ClouT – 31.07.2013
Page 4
Hbase............................................................................................................................................................ 43
HDFS ............................................................................................................................................................ 45
OPEN STACK SWIFT .................................................................................................................................... 47
CEPH ............................................................................................................................................................ 48
object Storage GE - FI-WARE Implementation ........................................................................................... 50
CDMI Proxy .................................................................................................................................................. 50
Rest/soap apis for idas access ..................................................................................................................... 51
TESTBED RUNTIME .................................................................................................................................... 51
SOA3 ............................................................................................................................................................. 52
4.3.
DEPENDABILITY TOOLS FOR ACCESSING CITY DATA REUSABLE COMPONENTS .................................................... 53
D-Case Monitoring tool ............................................................................................................................... 53
5.
OTHER REUSABLE CPAAS ASSETS FROM CITIES .................................................................................. 57
5.1.
INTRODUCTION ....................................................................................................................................... 57
5.2.
GENOA REUSABLE CPAAS ASSETS ............................................................................................................. 57
RoadVisior - eMixer Platform ...................................................................................................................... 57
WEATHER StatioN ....................................................................................................................................... 59
5.3.
SANTANDER REUSABLE CPAAS ASSETS ....................................................................................................... 60
GIS PlaTFORM .............................................................................................................................................. 60
OPEN DATA PLATFORM .............................................................................................................................. 65
Oracle Database Platform ........................................................................................................................... 67
SAE Platform ................................................................................................................................................ 69
Incisis Platform ............................................................................................................................................ 70
5.4.
FUJUSAWA REUSABLE CPAAS ASSETS ......................................................................................................... 72
Disaster prevention GIS system .................................................................................................................... 72
digital signage systeM .................................................................................................................................. 73
5.5.
MITAKA REUSABLE CPAAS ASSETS ............................................................................................................ 74
GIS system .................................................................................................................................................... 74
6.
CONCLUSIONS....................................................................................................................................... 75
6.1.
6.2.
6.3.
6.4.
7.
SERVICE COMPOSITION PLATFORMS FOR CITIZEN ’S APPLICATIONS .................................................................. 75
BIG DATA PROCESSING ............................................................................................................................. 76
SECURE AND DEPENDABLE ACCESS TO CITY DATA ......................................................................................... 76
CITY RESOURCES ..................................................................................................................................... 76
APPENDIX ............................................................................................................................................. 77
7.1.
7.2.
7.3.
TG3.1 REUSABLE COMPONENTS ............................................................................................................... 77
REFERENCES ................................................................................................................................................ 115
LIST OF FIGURES
ClouT – 31.07.2013
Page 5
Figure 1. ClouT City Architecture for Smart City Ecosystem ........................................................................ 10
Figure 2. WIRECLOUD Architecture ........................................................................................................................ 13
Figure 3. MYCocktail web tool ................................................................................................................................... 14
Figure 4 Open MyCocktail ........................................................................................................................................... 15
Figure 5. Yahoo pipes mashup tool.......................................................................................................................... 17
Figure 6. apache shindig overall architecture ..................................................................................................... 18
Figure 7. apache shindig server architecture ...................................................................................................... 19
Figure 2-8iPojo Component example ..................................................................................................................... 21
Figure 9Example of BPD (Stephen A. White) ...................................................................................................... 22
Figure 10. Robust Service Composition Method ................................................................................................ 25
Figure 11. QoS-based Service ..................................................................................................................................... 26
Figure 12. Metadata-based Behaviour Insertion Framework ...................................................................... 27
Figure 13. Verification Framework of Time and Resource Constrains on Business Process .......... 28
Figure 14. Verification Framework of ECA Specification on Physical Interactions ............................. 29
Figure 15CEP principle (Angelo Corsaro) ............................................................................................................ 31
Figure 16Esper integration to OSGi......................................................................................................................... 32
Figure 17Data Handling GE Architecture.............................................................................................................. 33
Figure 18High-Level production rule system ..................................................................................................... 35
Figure 19 Self-Healing Framework for Sensory Data ...................................................................................... 37
Figure 20 - Fault Classification Model .................................................................................................................... 39
Figure 21 - Hypertable Architecture Overview .................................................................................................. 42
Figure 22- Hbase architecture overview ............................................................................................................... 44
Figure 23 - Hbase cyclic replication overview architecture .......................................................................... 44
Figure 24. Apache Hadoop HDFS architecture overview ............................................................................... 46
Figure 25 - Apache Hadoop Cluster Server Roles .............................................................................................. 46
Figure 26 - CEPH UNIFIED STORAGE ..................................................................................................................... 48
Figure 27 Screenshot of D-Case Monitoring Tool .............................................................................................. 53
Figure 28 System Overview of D-Case Monitoring Tool ................................................................................. 54
Figure 29 Web Page for Detailed Monitoring Data ........................................................................................... 55
Figure 30 Santander cluster database architecture ......................................................................................... 67
Figure 31 SANTANDER INCISIS implementation architecture .................................................................... 70
Figure 32 Santander INCISIS FUNCTIONAL SCHEMA ..................................................................................... 70
Figure 33 INCISIS SERVICE EXAMPLE SCHEMA ................................................................................................ 71
Figure 30: Fujisawa disaster prevention gis system ........................................................................................ 72
Figure 31. Overview of Fujisawa Signage ............................................................................................................. 73
Figure 32:Mitaka GIS system...................................................................................................................................... 74
ClouT – 31.07.2013
Page 6
LIST OF TABLES
Table 1. List of ABBREVIATIONS .................................................................................................................................8
Table 2. List of KEY TERMS ............................................................................................................................................8
Table 3 List of Available Monitoring Programs .................................................................................................. 54
Table 4 – OPEN STACK SWIFT ................................................................................................................................... 77
Table 5- Wirecloud ......................................................................................................................................................... 78
Table 6- MyCoctails ........................................................................................................................................................ 79
Table 7- Enterprise mashup markup language .................................................................................................. 81
Table 8- opensocial ........................................................................................................................................................ 82
Table 9- Yahoo! pipes .................................................................................................................................................... 83
Table 10- apache shindig ............................................................................................................................................. 84
Table 11- iPojo ................................................................................................................................................................. 85
Table 12- BPMN ............................................................................................................................................................... 86
Table 13- BPEL ................................................................................................................................................................. 87
Table 14 – ROBUST SERVICE COMPOSITION Method ..................................................................................... 88
Table 15 – QOS-BASED service/cloud SELECTION method .......................................................................... 89
Table 16 – metadata-based BEHAVIOR INSERTION Frmaework ............................................................... 90
Table 17 – BPMN VerifiCation ................................................................................................................................... 91
Table 18 – ECA VERIFICATION ................................................................................................................................. 92
Table 19- ESPER .............................................................................................................................................................. 93
Table 20- FI-Ware Gateway Data Handling.......................................................................................................... 94
Table 21- JBoss DRools Expert .................................................................................................................................. 95
Table 22- MongoDB........................................................................................................................................................ 96
Table 23- REST/SOAP API’s for IDAS access ....................................................................................................... 97
Table 24- Testbed Manager ........................................................................................................................................ 98
Table 25 D-case monitoring tool .............................................................................................................................. 99
Table 26 – Self-Healing Framework for Sensory Data.................................................................................. 100
Table 27 – Fault Classification Model .................................................................................................................. 101
Table 28– HYPERTABLE ........................................................................................................................................... 102
Table 29 – HBASE......................................................................................................................................................... 103
Table 30 – HADOOP HDFS ........................................................................................................................................ 104
Table 31 – OPEN STACK SWIFT ............................................................................................................................. 106
Table 32 – CEPH ........................................................................................................................................................... 108
Table 33 – Object Storage GE - FI-WARE Implementation ......................................................................... 110
Table 34 – CDMI Proxy .............................................................................................................................................. 112
Table 35 – SOA3 ............................................................................................................................................................ 114
ClouT – 31.07.2013
Page 7
LIST OF ABBREVIATIONS AND DEFINITIONS
TABLE 1. LIST OF ABBREVIATIONS
CIaaS
CPaaS
CSaaS
IoT
IoT-A
TG
Wsan
RDF
OSGi
API
REST
JSON
NGSI
iPojo
OSGi
BPMN
BPD
UML
BPEL
GIS
RSS Feed
City Infrastructure as a Service
City Platform as a Service
City Service as a Service
Internet of Things
Internet of Things - Architecture
Task Group
Wireless sensor and actuator network
Resource Description Framework
Open Services Gateway Initiative
Application Programming Interface
REpresentational State Transfer
JavaScript Object Notation [RFC 4627].
Next Generation Services Interface
Plain Old Java Object
Open Services Gateway initiative
Business Process Modeling Notation
Business Process Diagram
Unified Modeling Language
Business Process Execution Language
Geographic information system
Really Simple Syndication Feed
TABLE 2. LIST OF KEY TERMS
Cloud Storage
Sensors
Actuators
Cloud Storage is a mode to abstract the physical infrastructure, based on silos
(federated servers, cluster servers, etc.) where the data are stored.
Equipment used to measure and convert physical quantity (e.g.
temperature) or human information (e.g. social data).
Mechanical devices that perform action on command, but also social networks
can be used as actuator.
ClouT – 31.07.2013
Page 8
EXECUTIVE SUMMARY
This document aims at collecting some existing and reusable components, techniques and
standards, focusing on the City Platform as a Service (CPaaS) layer. They belong to various
categories, such as the cloud storages, databases and standards used to access and manage data
stored in the Cloud.
Each partner has described all possible reusable components to be used in this phase of project:
in the document are described all the details of the components, with the assessment, for each
one, the Known Limitations of the components and the license of use. Some components are
used by the cities involved in the project: Santander and Genova in Europe, Mitaka and Fujisawa
in Japan.
The content of this deliverable will be used to define the Reference Architecture for ClouT - and
the subsequent implementation of the Smart City Ecosystem. Therefore, not all presented
objects will be used in the final version of the architecture, but only those ones that better
respond to requirements.
ClouT – 31.07.2013
Page 9
1. Introduction
1.1.
Scope of the Document
This document describes initiatives, tools, solutions and available assets from cities on which
leverage on to build the CPaaS layer of the ClouT architecture.
As this deliverable, along with D2.1, will serve as starting point for the initial definition of Cloud
Architecture (D1.2 deliverable), for each reusable asset it is proposed a short description of its
main features along with several useful details are provided. Furthermore a short assessment is
given, in terms of opportunities, capabilities and known limits and threats in adopting the
reusable component as part of the ClouT architecture.
Figure 1. ClouT City Architecture for Smart City Ecosystem
1.2.
Target Audience
The document targets the ClouT platform developers.
1.3.
Structure of the Document
Chapter 2 to 4 introduce all candidate components that are part of the TG3.1, TG3.2 and TG3.3
with reference to Description of Work. Chapter 5 lists a set of other existing assets from ClouT
involved cities that may be also included as part of CPaaS. The Appendix reports, for each
reusable component, a table containing additional details of the component itself. Finally,
conclusions are drawn in Chapter 7.
ClouT – 31.07.2013
Page 10
Each chapter is dedicated to a single topic, as shown in the following picture:

Chapter 2 is related to the Service composition and mash-up components.

Chapter 3 describes all the components dedicated to the Big data processing.

Chapter 4 is referred to the secure access and all the components involved.

Chapter 5 is related to the city reusable components.
In details, the Chapter 5 lists a set of further existing assets, extracted from ClouT involved cities
– Genoa (Italy), Santander (Spain), Mitaka (Japan) and Fujisawa (Japan). Finally, all conclusions
are exposed in the Chapter 6, while the Appendix reports a table for each reusable object,
containing general and additional details about them.
ClouT – 31.07.2013
Page 11
2. Service Composition Platforms for citizen’s applications
2.1.
Introduction
This section provides a list of tools that are potentially reusable for the service composition
and data mashup objective of the ClouT project. The main goal is to provide a tool to the
users (experienced developers, non-technical users, etc.) in order that they can build their
own services, therefore making them actively part of the city ecosystem. The list varies from
service component models to frameworks that provide verification and validation of the
composition.
The main idea behind the platform is to enable a user that comes with its own IoT device, to
create its service and publish it in the cloud and share (or un-share) it within a given
community. Other services can be created by composing high level services from existing
base services and platform services.
This section focuses on the service composition platform (at the CPaaS layer) that will give
to the users the opportunity of building their own IoT+Cloud services. Variety of tools exists
for different types of users, from most experienced ones (professionals) to inexperienced
users (end-users). With a service oriented approach, ClouT will define modular IoT and
Cloud service models and identify way of interactions between these services. The
interaction will be based on an event-based approach to better reflect the IoT device
interaction model. The tool will give the opportunity to the users to define actions to take
when specific events occur under given conditions (e.g., Event-Condition-Action rules) that
will determine the behavior of the created services.
Some of the tools presented here provide validation and optimization techniques for
dependability assurance in the service composition platform. The foundational model of
service composition is provided for analysis and reasoning, which supports event-driven,
context-aware behavior in the complex and dynamic environment. Functional validation is
conducted to detect and prevent undesirable situations, possibly caused by feature
interactions. Quality of service is also optimized and assured for user-oriented, end-to-end
criteria, through efficient exploration of different possibilities of service composition. Thus
these frameworks also provide the underlying components that help users, who develop
service compositions, refine and configure the composition to achieve dependability
properties.
ClouT – 31.07.2013
Page 12
2.2.
Service composition and mash-up tools reusable components
In this paragraph are listed all the mash-up reusable components: for each of them is reported a
short description, with figures if needed, and a link to the correspondent component table in the
Appendix.
WIRECLOUD
The Wirecloud Mashup Platform is a framework that allows end-users to create specific
mashups composing widgets that are the building blocks to create new applications. The widgets
(also called gadgets) can be downloaded from a catalogue and can be provided by 3rd parties. In
particular the Wirecloud Mashup Platform is composed by three different components: the
Application Mashup Editor, the Mashup Execution Engine and the Catalogue.
The Application Mashup Editor is a web-based composition editor used by the end users to
compose their own applications. The editor is a workspace where the users can graphically
connect the widgets, downloaded from the catalogue, performing an input-output mapping. The
widgets can be connected to back-end services or data sources through an extendable set of
operators, including filters, aggregators and adapters.
The Mashup Execution Engine is the component that provides the mashup functionalities
coordinating the gadgets execution and communication, mashup, state persistence, and crossdomain proxy. The engine functionalities are exposed to the editor through a set of API: new
external modules can be added in order to extend the functionalities.
The gadgets used in the editor can be found and downloaded from the Catalogue, that is an
independent component with respect to the previous two: the gadgets published in the
catalogue are described in a standard description language. In particular the gadgets can be
described through an XML Widget description language and with RDF widget description
language. The mashups created by the users can be published in the same catalogue:
additionally, the catalogue allows to tag and rate widgets to foster discoverability and share
ability.
FIGURE 2. WIRECLOUD ARCHITECTURE
ClouT – 31.07.2013
Page 13
The Wirecloud Mashup Platform has been developed in the FI-WARE project as Generic Enabler
reference implementation: it supports linked-USDL and it is integrated with FI-WARE's
Marketplace, Stores and Repositories.
For further information please refer to the table in the appendix.
MYCOCKTAIL
MyCocktail is a web application that provides a graphical user interface to easily build mashup
easily. MyCocktail provides two different web tools: the mashup builder and the page editor. The
mashup builder is a graphical environment that allows combining and modifying data, using
specific filters, information coming from REST services, and connecting these data with web
gadgets creating mashups. The gadgets can be exported as Google or Netvibes gadgets being
compliant with the open social specification. In particular there are three different types of
elements that can be combined in the builder:
FIGURE 3. MYCOCKTAIL WEB TOOL



Services: it is possible to use in the mashup information obtained by specific services.
MyCocktail provides a set of predefined services to get specific information from several
popular services or social networks such as Amazon, Delicious, Flickr, Google, Twitter
etc. For example it is possible to perform search on Google or get information about the
Twitter followers. Also it is possible to import in the editor new custom REST services
using the WADL descriptions (Web Application Description Language).
Operators: the data objects obtained from the services invocation can be manipulated
using a set of operators. These operators can work on objects, arrays or strings
performing a series of functions such as sorting, parsing, splitting etc.
Renderers: are graphical elements and widgets that allow rendering the information
obtained and manipulated using services and operators. The final graphical output can
be exported in different format such as Google or Netvibes gadgets.
ClouT – 31.07.2013
Page 14
The Page Editor allows designing a web page through a GUI allowing integrating mashups
created with MyCocktail with other HTML elements. It is based on FCKEditor, a web text editor.
The page editor has a complete toolbar in which the elements and the properties to be included
in the design area that shows the result page can be selected.
FIGURE 4 OPEN MYCOCKTAIL
ENTERPRISE MASHUP MARKUP LANGUAGE
EMML (Enterprise Mashup Markup Language) is an XMLmarkup language for creating
enterprise mashups promoted by the Open Mashup Alliance. EMML is a declarative mashup
domain-specific language (DSL) that eliminates the need for complex, time-consuming and
repeatable procedural programming logic to create enterprise mashups.
EMML provides a mashup-domain vocabulary to consume different data-sources. Additionally,
the EMML provides a uniform syntax to invoke heterogeneous service technologies (e.g. Web
services, rest services, RSS etc.) and to combine different data formats (JSON, XML, JDBC etc).
EMML files can be considered scripts that describe the process flow for a mashup: these scripts
must be processed by an EMML engine that interprets EMML statements to perform the mashup.
The EMML language provides several functionalities such as:






Data manipulation using filters
Connection with heterogeneous services
Semantic annotation of services
Merge and split datasets
Support for scripting languages (e.g. JavaScript, JRuby, Groovy, XQuery)
Conditional statements
ClouT – 31.07.2013
Page 15
 Data scraping from HTML page
The Open Mashup Alliance has made the EMML schema available for download as well as an
EMML reference runtime implementation that processes mashup scripts written in EMML.
OPENSOCIAL
OpenSocial is a set of common APIs (Application Programming Interface), initially developed by
Google, in order to create a single programming model for social applications such as gadgets
that can operate on any social network that uses this standard.
Based on HTML and JavaScript, as well as the Google Gadgets framework, OpenSocial includes
multiple APIs for social software applications to access data and core functions on participating
social networks. Each API addresses a different aspect. It also includes APIs for contacting
arbitrary third party services on the web using a proxy system and OAuth for security.
In particular, OpenSocial provides four different categories of APIs that can be used to:



develop Social Application, such as gadgets, that run in specified OpenSocial containers
implement Web Container: host social networking environments in which the social
applications can be executed
allow interaction between web sites and social networks that support the Open Social
standard. In particular, existing sites can have access to several information, for instance:
o the user profile (user data);
o information about user’s friends (social graph);
o profile activities (posts, photos, video, news feeds etc.)
The goal of the project is, therefore, to:





increase power and pervasiveness of the social Web capabilities, providing to the
developers the tools needed to build applications that can be accessed by an ever
increasing number of users, regardless of social networks used;
increase interoperability between applications;
promoting the portability of web-based components;
stimulate the creativity of the user, in order to have new ideas "from below";
create a "framework" open and free that is compatible with the highest number of social
networks.

OpenSocial is not, a product or a service distributed by Google, but it is an open standard
supported by a large community of developers. Furthermore, it is important to highlight how the
project is significantly different from what has been achieved in recent years by Facebook which,
despite the huge community of developers, has focused mainly on the use of proprietary
protocols and languages, maintaining a more conservative approach.
ClouT – 31.07.2013
Page 16
YAHOO! PIPES
Yahoo! Pipes is a visual programming environment that gives people the ability to create web
mashups and web-based applications that combine data from multiple sources. The name comes
from the Unix Pipes, which allowed connecting to data source filters and utility of different
types. Yahoo! Pipes does not require any type of installation because it is an online service,
available at http://pipes.yahoo.com, only a Yahoo! account is required.
It is a free service and allows the management of RSS Feeds and creating interesting mashups.
FIGURE 5. YAHOO PIPES MASHUP TOOL
It is also composed of a graphical editor intuitive and very user friendly.
The core component of Yahoo ! Pipes is the Pipe editor that is composed of three panes which
are the canvas, the library and the debugger. The user creates a pipe using these panes using
Drag & Drop through which the various modules are composed from existing services. Through
Yahoo! Pipes it is possible for example to combine many feeds into one, sort, filter and to
geocode the favorite feeds browsing the items on an interactive map. After creation the user can
decide to publish the pipes on the website http://pipes.yahoo.com to make them visible to other
people than can reuse or modify the pipes.
The output can be in different formats widgets, RSS, JSON, PHP.
ClouT – 31.07.2013
Page 17
APACHE SHINDIG
Shindig is an open source project of the Apache Software Foundation that has the aim to
implement an open container in compliance with the Open Social gadgets specifications.
According to the Apache Foundation, the project's goal "is to allow new sites to start hosting the
social apps in less than an hour's work", creating an infrastructure able to render gadgets,
process proxy requests, and handle REST and RPC requests.
In the market there are many examples of containers that are based on Shindig, such as
LinkedIn, hi5, Partuza, WSO2, Jartuza.
From the architectural point of view Shindig consists of four basic components:




Gadget Container JavaScript: component written in JavaScript that manages many
gadgets functionalities such as safety, communication, graphical user interface and the
access to the OpenSocial API.
Gadget Rendering Server: server used to render the gadget XML file in JavaScript and
HTML for the container in order to expose the gadget through the Container JavaScript.
OpenSocial Container JavaScript: the JavaScript environment that lies on the top of the
Gadget Container JavaScript and provides OpenSocial specific functionality, such as
profiles, list of friends, activities, datastore.
OpenSocial Date Server: is the implementation of a server interface for the access to
specific information of the container, and it includes the OpenSocial REST API.
Shindig, from architectural point of view, has a client and a server component. In particular the
client part of the system is composed by three elements:
 The container for gadgets, fully compliant with the OpenSocial gadgets specifications
(GadgetsContainer).
 The OpenSocial container itself (opensocial. Container).
 The container that supports the exchange of data in the JSON and Caja format to the
REST standard (RestfulContainer).
FIGURE 6. APACHE SHINDIG OVERALL ARCHITECTURE
In particular, the latter is an extension of the component OpenSocial container and deals to pass
all calls to the API OpenSocial REST endpoint that will take care of both the direct HTTP calls and
those from the OpenSocial API.
ClouT – 31.07.2013
Page 18
All calls to the server will be executed by the RestfulContainer and GadgetsContainer through
the XmlHttpRequest Gadgets.io that will instantiate an object in order to send requests to HTTP
Server.
The Shindig components of the server side are:



Persistent Data Access Layer: loading mechanism of the persistent data.
Gadget Rendering Server Components: infrastructure for rendering gadgets.
Open Social Components Server: server-side implementations of the OpenSocial API.
FIGURE 7. APACHE SHINDIG SERVER ARCHITECTURE
There are currently two versions of Apache Shindig written in Java and PHP, and a third, written
in .NET that is external to the project.
ClouT – 31.07.2013
Page 19
IPOJO
iPOJO (iPojo) (Plain Old Java Object) is a service component runtime aiming to simplify OSGi
application development and to build service-oriented component model.
Early efforts, such as the Service Binder and Service Tracker, attempted to give more flexibility
to OSGi developers and more recent efforts, such as Declarative Services and Spring Dynamic
Modules alleviate some of these issues. iPOJO is another such effort in this area. It has two goals:


providing services, using services, and dealing with dynamism should require as little
effort as possible from the developer;
component configuration, synchronization, and composition should be incorporated to
OSGi mechanisms.
For example, trying to create an OSGi-based application with services is challenging. The OSGi
API is complex and a lot of knowledge about internal mechanisms has to be known to avoid
synchronization issues. iPOJO provides a very simple development model. The component
(Figure 2-8) is a central concept in iPOJO. In the core iPOJO model, a component describes
service dependencies, provided services, and call backs; this information is recorded in the
component's metadata. In fact, a service is an object that implements a given Java interface. In
addition, iPOJO introduces a call back concept to notify a component about various state
changes.
To provide a service:
@Component
@Provides
public class MyServiceImplementation implements MyService {
//....
}
In this way, the component is declared as the MyService provider and corresponding OSGi
service is created automatically.
To require a service:
@Component
public class MyServiceConsumer{
@Requires
private MyService myservice;
// Just use your required service as any regular field !
}
After components, the next most important concept in iPOJO is the component instance. A
component instance is a special version of a component. By merging component metadata and
instance configuration, the iPOJO runtime is able to discover and inject required services,
publish provided services, and manage the component's life cycle.
ClouT – 31.07.2013
Page 20
FIGURE 2-8IPOJO COMPONENT EXAMPLE
iPOJO works on any R4.1 OSGi implementation. It also works on many Java virtual machines
such as Oracle JRockit, JamVM, Dalvik (Android), and Mika. iPOJO only requires a J2ME
Foundation 1.1 virtual machine. So, iPOJO can be embedded inside mobile phone applications or
inside your washing machine.
iPOJO is small and was designed to stay small. The core size of iPOJO is approximately 205k
(compared to 816k for Guice-Peaberry and 2112k for the minimal Spring-DM configuration). In
addition to the core, you only deploy the features you require. For example, if you need proxy
injection, just deploy the temporal dependency bundle (less than 70Kb). The run-time overhead
of iPOJO is also small. On the Peaberry benchmark, iPOJO has the following performance results:
Guice-Peaberry:
276.00 ns/call
iPOJO Service Dependency:
118.00 ns/call
Spring-DM:
2384.00 ns/call
iPOJO Temporal Dependency:
159.00 ns/call
iPOJO Temporal Dependency w/ proxy: 173.00 ns/call
ClouT – 31.07.2013
Page 21
BPMN
The Business Process Modeling Notation (BPMN) (BPMN) is a graphical notation that depicts the
steps in a business process. A business process spans multiple participants and coordination can
be complex. Moreover, well-supported standard modeling notation will reduce confusion among
business and IT end-users. BPMN depicts the end to end flow of a business process. The notation
has been specifically designed to coordinate the sequence of processes and the messages that
flow between different process participants in a related set of activities. The following figure
shows a Business Process Diagram.
FIGURE 9EXAMPLE OF BPD (STEPHEN A. WHITE)
A BPD is made up of a set of graphical elements. These elements enable the easy development of
simple diagrams that will look familiar to most business analysts (e.g., a flowchart diagram). The
elements were chosen to be distinguishable from each other and to utilize shapes that are
familiar to most modelers. For example, activities are rectangles and decisions are diamonds. It
should be emphasized that one of the drivers for the development of BPMN is to create a simple
mechanism for creating business process models, while at the same time being able to handle
the complexity inherent to business processes. The approach taken to handle these two
conflicting requirements was to organize the graphical aspects of the notation into specific
categories. This provides a small set of notation categories so that the reader of a BPD can easily
recognize the basic types of elements and understand the diagram. Within the basic categories of
elements, additional variation and information can be added to support the requirements for
complexity without dramatically changing the basic look-and-feel of the diagram. The four basic
categories of elements are: Flow Objects, Connecting Objects, Swimlanes and Artifacts.
A key goal in the effort to develop BPMN to help alleviate the modeling technical gap was to
create a bridge from the business-oriented process modeling notation to IT-oriented execution
languages that will implement the processes within a business process management system. The
graphical objects of BPMN, supported by a rich set of object attributes, have been mapped to the
Business Process Execution Language for Web Services (BPEL4WS v1.1), the standard for
process execution. For example, the core of jBPM (jBPM, 2013) is a light-weight, extensible
ClouT – 31.07.2013
Page 22
workflow engine written in pure Java that allows you to execute business processes using the
latest BPMN 2.0 specification. It can run in any Java environment, embedded in your application
or as a service.
Other solutions to BPMN exist for example, UML Activity Diagram, UML EDOC Business
Processes, IDEF, ebXML BPSS, Activity-Decision Flow (ADF) Diagram, RosettaNet, LOVeM, and
Event-Process Chains (EPCs). The following section illustrates the difference between UML and
BPMN and thus it helps to understand more the relationships between these solutions.
BPMN & UML (BPMN)
The unified modeling language (UML) takes an object-oriented approach for modeling
applications, while BPMN takes a process-oriented approach for modeling systems. Where
BPMN has a focus on business processes, the UML has a focus on software design and therefore
the two are not competing notations but are different views on systems. The BPMN and the UML
are compatible with each other. A business process model does not necessarily have to be
implemented as an automated business process in a process execution language. Where this is
the case, business processes and participants can be mapped to constructs such as use cases and
behavioral models in the UML.
B P M N TO X M L ( B P M N 2 . 0 b y E x a m p l e , 2 0 1 0 )
The XML serialization for BPMN is provided in machine-readable form, which has the OMG
Document Number dtc/2010-06-03. It is an international standard formally approved by OMG, it
creates XML code that is generated when a person creates a process model.
ClouT – 31.07.2013
Page 23
BPEL
BPEL (BPEL) is the standard for assembling a set of discrete services into an end-to-end process
flow, radically reducing the cost and complexity of process integration initiatives.
BPEL (BPEL Definition) is an orchestration language, and not a choreography language. The
primary difference between orchestration and choreography relies on the executablility and
control. An orchestration specifies an executable process that involves message exchanges with
other systems, such that the message exchange sequences are controlled by the orchestration
designer. Choreography specifies a protocol for peer-to-peer interactions, defining, e.g., the legal
sequences of messages exchanged with the purpose of guaranteeing interoperability. Such a
protocol is not directly executable, as it allows many different realizations (processes that
comply with it). A choreography can be realized by writing an orchestration (e.g., in the form of a
BPEL process) for each peer involved in it.
BPEL has no graphical notation witch conducted to a variety of different notations causing
ambiguities and misunderstanding. BPMN is then used to fill this gap.
The following figure (Leymann, 2010) explains more the relationship between BPEL and BPMN
using a concrete example.
ORACLE BPEL
Oracle BPEL (ORACLE BPEL Process Manger, 2009) process Manager is a tool for designing and
running business processes. This product provides a comprehensive, standards-based and easy
to use solution for creating, deploying and managing cross-application business processes with
both automated and human workflow steps – all in a service-oriented architecture. Oracle BPEL
Process Manager could be used in isolation to implement business processes, but the real power
comes when it is used in conjunction with other SOA components. For example, you can
instrument your BPEL processes using the Oracle BPEL Process Manager’s sensor framework. Sensors can fire events under conditions specified by the administrators and send events to any
endpoint that administrators choose. This makes it easy to monitor processes when there are
hundreds or thousands running in parallel.
ClouT – 31.07.2013
Page 24
2.3.
Dependable Service Compositions reusable components
In this paragraph are listed all the Dependable Service Composition reusable components. For
each component it is reported a short description and a link to the correspondent component
table in the Appendix.
ROBUST SERVICE COMPOSITION METHOD
This method provides foundations for service composition that deal with differences in service
functionality and quality as well as their faults or changes. It can be instantiated as an analysis
and design method by human developers, or an algorithm for automated service composition.
The core of the method is a modeling structure to efficiently represent and analyze slightly
different service functions. It can be used to:
- understand the differences
- efficiently find matching between service functions
- efficiently find alternatives for a certain function
- and assess feasibility or sustainability about realization of a certain function by external
providers. For the automated usage, an efficient algorithm is provided: it uses the model and it
constructs a robust service composition plan by analyzing alternatives or risks of lock-in. The
algorithm has some customizations using the alternative information on a genetic algorithm so
that semi-optimal solutions are obtained efficiently, even though the problem is much more
complex than standard ones by considering alternatives.
FIGURE 10. ROBUST SERVICE COMPOSITION METHOD
ClouT – 31.07.2013
Page 25
QOS-BASED SERVICE/CLOUD SELECTION METHOD
This method provides an algorithm for QoS-based selection of services or clouds to use. This
method basically follows and handles the standard problem of QoS-based service selection,
which selects services to use in a service composition according to a variety of QoS (Quality of
Service). The methods considers both of preferences over multiple criteria (e.g., price is more
important than execution time) and end-to-end constraints (e.g., total execution time must be
less than $1 per invocation). The method extends this standard setting by also considering
network QoS between interacting services. It thus includes model that defines aggregated
quality of services and networks involved in a service composition. It also provides an efficient
algorithm for QoS optimization by customizing a genetic algorithm to reflect location of services
in a network model.
FIGURE 11. QOS-BASED SERVICE
ClouT – 31.07.2013
Page 26
METADATA-BASED BEHAVIOR INSERTION FRAMEWORK
This framework provides a way to insert additional behaviors into service compositions to
achieve value-added functions often for non-functional requirements. It defines an algorithm to
properly insert additional behaviors according to constraints or rules. These constraints or rules
work on metadata attached on service compositions. Thus they are simple and one specification
item can trigger insertions in multiple execution points in multiple compositions (e.g., insert
logging behavior before "PRIVACY" data is sent to a "THIRD PARTY").
FIGURE 12. METADATA-BASED BEHAVIOUR INSERTION FRAMEWORK
ClouT – 31.07.2013
Page 27
VERIFICATION FRAMEWORK OF TIME AND RESOURCE CONSTRAINTS ON BUSINESS
PROCESS
This framework provides a capability to verify time and resource constraints on service
compositions or business processes through model checking. It provides a small annotation
syntax on BPMN (Business Process Modeling Language) for time and resource constraints. It has
conversion rules from BPMN with the annotations into UPPAAL, a timed model checker, which
exhaustively explores possible state changes to verify the constraints.
FIGURE 13. VERIFICATION FRAMEWORK OF TIME AND RESOURCE CONSTRAINS ON BUSINESS PROCESS
ClouT – 31.07.2013
Page 28
VERIFICATION FRAMEWORK OF ECA SPECIFICATION ON PHYSICAL INTERACTIONS
This framework provides a capability to verify complex combinations of ECA (Event-ConditionAction) rules that have effects on shared spaces often by multiple users. It provides a language
for high-level specification of physical interactions (e.g., visual, audio). It also defines conversion
rules of the description into SPIN, a model checker. Several templates are provided for proper
formula to check, which are difficult to properly define by temporal logic.
x
FIGURE 14. VERIFICATION FRAMEWORK OF ECA SPECIFICATION ON PHYSICAL INTERACTIONS
ClouT – 31.07.2013
Page 29
3. Big data processing
3.1.
Introduction
This section gives the currents state of the art in terms of reusable data processing components
that can be considered when developing ClouT’s data processing subsystem. Given the ubiquity and increasing number of devices in the future Internet of Things, scalable data processing
infrastructures are necessary to collect and process a ”terabyte torrent” coming from trillions of devices. The ClouT’s data processing subsystem will provide an event processing engine that will
collect events from the city, process them to obtain higher level information that can be accessed
by other services and applications (e.g., subscribers). Traditional SQL-based database languages
are not any more adequate for managing the high stream of data from IoT. The querying and
processing techniques should be revised in order to take into account the real-time nature of the
applications. One part of the processing should be when possible done at the device level, thus
exploiting devices processing capabilities.
Data need to be processed by data stream management systems in real-time and not by using
the typical store-and-process paradigm. This section thus analyses noSQL solutions for data
processing. In addition, since ClouT will provide an event processing mechanism with a
publish/subscribe facility, existing solutions on that are also listed in this section. ClouT project
also deals with the non-functional issues of the data processing system, such as the fault
detection. In fact, the data should be “clean” in order to be properly used by real-time
applications. Faulty data should be detected and isolated, and necessary measures should be
taken if the faults are frequent. One of the big challenges is that this should be done in real-time
on a big quantity of data. Some solutions of online fault detection are presented in this section.
ClouT – 31.07.2013
Page 30
3.2.
Data/event processing and decision making reusable components
In this paragraph are described all the Data/Event processing and decision making reusable
component. For each of them it is reported a short description with a link to the correspondent
component table in the Appendix.
ESPER
The Esper engine (Inc, 2012) has been developed to address the requirements of applications
that analyze and react to events. The Esper engine works a bit like a database turned upsidedown. Instead of storing the data and running queries against stored data, the Esper engine
allows applications to store queries and run the data through. Response from the Esper engine is
real-time when conditions occur that match queries. The execution model is thus continuous
rather than only when a query is submitted.
Esper provides two principal methods or mechanisms to process events: event patterns and
event stream queries. Event pattern are expression-based event that matches expected
sequences of presence or absence of events or combinations of events. It includes time-based
correlation of events. Esper event stream addressed event stream analysis requirements of CEP
applications. Event stream queries provide the windows, aggregation, joining and analysis
functions for use with streams of events. These queries are following the Event Processing
Language (EPL) syntax. EPL has been designed for similarity with the SQL query language but
differs from SQL in its use of views rather than tables. Views represent the different operations
needed to structure data in an event stream and to derive data from an event stream.
Figure 15 shows the event processing principle that Esper follows. First the data we would like
to analyze need to be structured in the form of objects before it can be thrown down the pipe to
our CEP engine. Esper provides multiple choices for representing an event for example Java
object and DOM node. There is no absolute need for you to create new Java classes to represent
an event. Then, we need to inform the engine about the kind of objects it will have to handle.
Events can then be thrower of the pipe of the CEP engine. The CEP engine will then filter the
data it receives, and trigger events whenever that data meets the selection rule, or fulfils the
pattern defined in the statement in the form of EPL.
FIGURE 15CEP PRINCIPLE (ANGELO CORSARO)
ClouT – 31.07.2013
Page 31
ESPER&OSGI
Esper is a server side component and it can be deployed on platforms that perform data
gathering and processing. It is possible to deploy Esper in terms of a bundle into an OSGi
gateway. The Figure 2-8 shows a possible integration of Esper to OSGi.
FIGURE 16ESPER INTEGRATION TO OSGI
An integration of Esper into OSGi has been developed by the FI-WARE project. Next section
briefly presents that component.
ClouT – 31.07.2013
Page 32
FI-WARE GATEWAY DATA HANDLING
FI-Ware Gateway data handling (Fiware Data Handeling) is a complex event processing
component proposed by Orange Labs France. It is based on Esper4FastData CEP engine. It is able
to collect vast amounts of asynchronous events of different types and correlate them into single
events, called Complex Events. It can read from and write to numerous different channels using
various different protocols. It is driven using a domain specific language called “Dolce”.
The Gateway Data Handling GE of the Figure 17 is also the first stage of intelligence transforming
data into events using smart rules. Applications are now able to collect in real-time large
amounts of data, but only relevant data avoiding boring and asynchronous data analysis. You
will not manage only raw data but also define some local rules to add value on raw data and
send only relevant events when a typical situation happens. You will also be able to add and
change the rules.
FIGURE 17DATA HANDLING GE ARCHITECTURE
The Data Handling API is NGSI compliant. It accepts events from any NGSI compliant Event
Producer. In the IoT Service Enablement architecture, the Gateway Device Management GE
publishes device events and data towards the Data Handling GE. In this setup, the Gateway
Device Management GE registers itself as an NGSI context provider, by calling the NGSI-9
register Context method exposed by the Data Handling GE. After context registration, the
Gateway Device Management GE sends events by calling the NGSI-10 update Context method of
the Data Handling GE.
Data Handling GE implementation allows rules building by graphically wiring blocks together.
The resulting diagram is then converted into EPL syntax, which is the CEP rule language. The
ClouT – 31.07.2013
Page 33
Graphical rules editor is optional, but useful to manage CEP rules, in a user-friendly
environment.
ClouT – 31.07.2013
Page 34
JBOSS DROOLS EXPERT
Drools Expert is a declarative, rule based, coding environment. This allows you to focus on "what
it is you want to do", and not on the "how to do this".
Drools life is a specific type of rule engine called Production Rule System (PRS) and was based
around the Rete algorithm (usually pronounced as two syllables, e.g., REH-te or RAY-tay). The
Rete algorithm, developed by Charles Forgy in 1974, forms the brain of a Production Rule
System and is able to scale to a large number of rules and facts. A Production Rule is a two-part
structure: the engine matches facts and data against Production Rules - also called Productions
or just Rules - to infer conclusions which result in actions. The process of matching the new or
existing facts against Production Rules is called pattern matching. The figure below shows the
high-level architecture of production system.
when
<conditionx>
then
<actions>;
FIGURE 18HIGH-LEVEL PRODUCTION RULE SYSTEM
The following is a simple "reactive" monitoring example that sends a message every four hours
when the alarm is raised. The calendar attribute ensures the rule only executes on weekdays.
Monitoring examples like this would be a long running application.
rule "Weekday Alarm Response" timer(int 4h) calendar "weekday" when
a : Alarm( )
then
sendMessage( "There is an alert" + a);
end
An interesting fact is that Drools can be integrated to jbpm. The drools camel server (droolscamel-server) (Drools-Camel Server) module is a war (Web application Archive) file which you
can deploy to execute Knowledge Bases remotely for any sort of client application. This is not
limited to JVM application clients, but any technology that can use HTTP, through a REST
interface. This version of the execution server supports stateless and state full sessions in a
native way.
ClouT – 31.07.2013
Page 35
MONGODB
MongoDB (MongoDB) (from "humongous") is an open-source document database, leading of
NoSQL databases. Data in MongoDB has a flexible schema. Collections do not enforce document
structure, although you may be able to use different structures for a single data set. In MongoDB,
different data models may have significant impacts on application performance.
MongoDB (MongoDB Overview) is open-source. It features: document data model with dynamic
schemas, full and flexible index support, auto-Sharding for horizontal scalability, built-in
replication for high availability, text search, rich queries and advanced security.
Instead of storing data in rows and columns as one would with a relational database, MongoDB
stores a binary form of JSON documents (BSON). Relational databases impose flat, rigid schemas
across many tables. By contrast, MongoDB is an agile NoSQL database that allows schemas to
vary across documents and to change quickly as applications evolve, while still providing the
functionality developers expect from relational databases, such as secondary indexes, a full
query language and strong consistency.
In addition, MongoDB is built for scalability, performance and high availability. Auto-sharding
allows MongoDB to scale from single server deployments to large, complex multi-data center
architectures. Leveraging native caching and RAM, MongoDB provides high performance for
both reads and writes. Built-in replication with automated failover enables enterprise-grade
reliability and operational flexibility. MongoDB also provides native, idiomatic drivers for all
popular programming languages and frameworks to make the development natural.
ClouT – 31.07.2013
Page 36
3.3.
Self-healing for data/event streaming reusable components
In this paragraph are listed the Self-healing for data/event streaming reusable components. For
each of them it is reported a shot description and a link to the correspondent component table in
the Appendix.
SELF-HEALING FRAMEWORK FOR SENSORY DATA
This framework provides runtime detection, classification, and correction capabilities for
sensory data faults that appear in sensory data. It includes a fault classification model and a
model-learning component based on statistical pattern matching.
FIGURE 19 SELF-HEALING FRAMEWORK FOR SENSORY DATA
Figure 19 illustrates overview of the framework. This framework addresses a full cycle of selfhealing mechanism, which includes detection, diagnosis, resiliency mechanisms and fault
models.
Detection phase is self-explanatory, and neighborhood vote with the statistical analysis of data is
sufficient for successful detection. Classification is a part of the diagnosis. It relies on the
duration and the impact faults have on the data behavior. To complete the classification, we have
a phase of fault model learning. Models are later applied to correct readings from the respective
faulty nodes. Each of these phases can be implemented with a different set of applicable
algorithms.
ClouT – 31.07.2013
Page 37
In the absence of ground truth, data modeling is vital. A good set of models enables grouping
data into good or faulty, including the type of fault. The challenge here is to define the distinction
between faulty behavior and an outlier of a new event. If the expected range of readings is
known, a reading that falls out of that range is not necessarily faulty, but it might indicate the
occurrence of a new event. We focus on the case where this knowledge is not available and
propose model learning from the past behavior of the network. Expectations of correct behavior
are established based on collected readings over the initial phase of operation. In this way, the
network is not bound by a-priori assumptions and has the freedom to establish what ground
truth is. Network learns a model of fault for each faulty node and can apply that model for the
fault correction.
Both modeling and classification rely on data features, which are usually statistical in nature.
Since the fault can belong only to a limited number of proposed classes, statistical pattern
recognition to learn an appropriate model of the fault is suitable. This model is then applied to
correct the readings of a respective node.
ClouT – 31.07.2013
Page 38
FAULT CLASSIFICATION MODEL
This model provides a decision tree to classify data faults into four types: bias fault, drift fault,
malfunction fault and random fault. Applied to sensory readings, this model proposes a complete
and consistent classification based on frequency and the continuity of occurrence and
observable and learnable patterns.
FIGURE 20 - FAULT CLASSIFICATION MODEL
Figure 20 illustrates this classification. From here we can define types of faults. If a sensor
reading is represented with ri + εi , where ri is a true value of a measured phenomena and εi is a
fault, we can define fault classification as follows:
• Discontinuous - Fault occurs from time to time, occurrence of εi is discrete.
– Malfunction – Frequent occurrence of faulty readings, εi> τ, where τ is threshold frequency. Also, there is no observable pattern in the fault occurrences.
– Random – Infrequent occurrence of faulty readings, εi≤τ. •Continuous - After the certain point in time, a sensor returns constantly inaccurate readings,
and it is possible to observe a pattern in the form of a function: εi = f(t,[α1,α2....])
where αi are coefficients of the function and t is time.
– Bias - The function of the error is a constant, εi = const. This can be a positive or a
negative offset.
– Drift - The deviation of data follows a learnable function, such as polynomial change
ε =α ∗ε
+α ∗ε
+ ⋯+ α
The classification is independent of the underlying cause of a fault.
ClouT – 31.07.2013
Page 39
4. Secure and dependable access to city data
4.1.
Introduction
The goal of this task is twofold. Firstly, it aims at providing a City Data Storage and Access
framework and integrating APIs for hosting and accessing in a secure way the big amount of
data produced by the City applications and the available sensors. Secondly, it will provide
dependability assurance tools that can be used for monitoring the dependability of running
systems.
With regard to Open City data hosting, several Cloud Storage solution among Object Storage,
Distributed File System and Data Base have been identified as candidates for building up the
Data Storage and Access Services of ClouT platform. These different technologies of distributed
data management, for their different features and limits, can be combined as the best in order to
enable a fast storage and retrieval of the variety of data produced by virtualized sensors and city
applications connected to the Cloud. These data may differ in terms of size (for example from a
single measurement of sensor to a large image file), amount, format, expected frequency of
access, type of usage (processing, long archiving).
Cloud databases are web-based services conceived for executing queries on structured data
stored on cloud data services. With respect to traditional approaches, cloud databases provide
the main capabilities of a database, that is simple querying of structured data and real-time
lookup, while being easy to use. Furthermore, most of the cloud databases present a distributed
deployment model: information are distributed among differently located hardware, but this is
transparent to the client, that see data as it resided in a unique place.
Storing and processing sensor measurements should support advanced queries and correlations
of information (for analytic purposes) and for these reasons, after this preliminary study, the
cloud database services turned out as the best technology for storing sensor data. Nevertheless,
historical data should be kept for a reasonable long period to enable statistical analysis over
time, even years or decades. And databases are not suitable for archiving large set of data for a
long period of time.
In this sense, different platforms, allowing not only to gather measurements and store historical
values from the measurements retrieved by the network, but also to send commands to the
deployed nodes and to subscribe to different events in the network.
Regarding to the network management, it is important to be able to remotely access and control
the deployed platform, in order to send/receive commands from the nodes as well as to change
their behavior through remote flashing procedures.
Cloud Object storage model was introduced quite recently in the world of distributed storage as
an alternative to file system storage, to face the explosion of mostly unstructured data being
stored on storage systems. With respect to file system, cloud object storage is much more
scalable, simple and provides important resiliency and reliability features. It is based on a flat
organization of containers and objects and use unique identifier to retrieve them. This data
model is flat in the sense that objects are organized in container with no relationship between
ClouT – 31.07.2013
Page 40
each other. Storage objects are replicated across multiple and commodity-hardware servers in
different locations in order to ensure reliability, The Cloud object storage provided limited
functionalities such as CRUD (create, read, update, delete) on objects through REST APIs, which
allows accessing files for users from anywhere.
Among the disadvantages of Cloud Object storage with respect to file systems, it is worth
mentioning the fact that usually the throughput is slower and also that in case of update of an
object, the change has to be propagated to all of the replicas in order for requests to return the
latest version (consistency lack). For this reason, while object storage it is the best solution for
those data that are updated un-frequently, like multimedia files or backups, it is not suitable for
data that change frequently.
Interoperability and standardized access to structured and unstructured data in the cloud are
considered urgent requirements for Cloud storage. Furthermore, it should support complex
queries for applications that store data or objects in the cloud.
At this purpose, in 2010 SNIA (Storage Network Agency) introduced the Cloud Data
Management Interface standard [CDMI], which is now designated by ISO/IEC as an international
standard. Adding a CDMI layer on heterogeneous and distributed storage services, we enable
interoperability and we provide support for querying the storage cloud based upon a regular
expression. For this reason, CDMI has been identified as key asset on which to leverage to build
the Data Access layer of the ClouT platform.
As the second point, this task will provide dependability assurance tools that can be used for
monitoring the dependability of running systems. Providing city data through Internet of Things
and cloud platform promises to penetrate into social infrastructure systems. Some of them are
used for critical systems, such as structure monitoring and medication management, where
failure of getting data and resources may damage human life, human health, and property.
Systems support for better monitoring, fault detection, and fault recovery is thus required to
keep infrastructure healthy. The infrastructure is, however, a complicated system consisting of
wireless sensor nodes, database servers, network appliances, etc. There is no existing system
that can manage such infrastructure in a comprehensive manner.
To tackle the issue in defining effective monitoring, technique for dependability case (D-Case)
are examined. It allows managers to describe management strategy, especially with a graphical
tool, which is linked to analysis of dependability requirements as well as action programs.
The best Cloud Data Management approach for ClouT will be delineated in the next project
phase in light of requirements collected in D1.1 and of other specific needs that may surface
when the first ClouT reference architecture will be identified.
4.2.
Open city data hosting and access reusable components
In this paragraph are listed all the cloud storages, distributed file systems, security platforms
and an implementation of cloud data management interface. There is also, for each component,
a link to the correspondent component table in the Appendix.
ClouT – 31.07.2013
Page 41
HYPERTABLE
Hypertable is a “NoSQL” database based on Google File System technology. It permits to attach,
as distributed file systems, Hadoop HDFS (Google File System open source implementation).
The files are distributed in all the data nodes and replicated in backup nodes. The availability of
the service is 100% also in case of failure of one data node. The software computation is based to
Hadoop MapReduce framework. Hypertable permits to process a big amount of data in parallel
by pushing the piece of code out to the machines where the data resides.
The final aggregation provides a way to re-order the data based on any arbitrary field.
Hypertable uses the Google Bigtable technology, which is implemented to create big tables with
an index key. Applications that use this technology include Google Earth, Google Analytics,
Google Maps, Gmail, Orkut, YouTube, and many more.
The security and data encryption are demanded to the back-end file system (physical disks,
Hadoop HDFS or Ceph). No security is implemented to access the data with Hypertable API or
Hypertable built in clients.
The access to the data is performed using SQL like language. It is also possible, and supported,
the JDBC connector. Hypertable provides the API for the following languages: Java, PHP, Python,
Perl, Ruby, C++.
The high performance of access to data is possible thanks to the indexing mechanism, based on
hash tables in the database. The best performance of Hypertable is in the random access. As that,
this database implementation is well suited for real time scenarios.
FIGURE 21 - HYPERTABLE ARCHITECTURE OVERVIEW
ClouT – 31.07.2013
Page 42
HBASE
HBase is a "NoSQL" and distributed database. HBase is designed to be a "datastore", but, with the
developed functionalities, it can be considered also a "database". It offers all the features of a
RDBMS (typed columns, triggers , indexes, etc). It is possible also to implement complex
queries.
HBase has been developed to scale well in horizontal (adding new nodes) and in vertical (adding
more hardware to each node). HBase clusters can be expanded adding new RegionServers. Each
server can be hosted on low cost hardware. Adding new hardware (new clusters) it is possible to
increase performance in terms of data storage space RDBMS can scale well, but only from the
point of view of the size of a single database server. To increase performance, RDBMS requires
specialized, expensive hardware and storage devices.
HBase, built on top of HDFS, provides high performance in record query (searches, updates and
inserts) for large big tables. HBase, internally, stores the data in indexed files that are
memorized on HDFS for high-speed access. As well, HDFS distributes, and replicates, the data in
the data nodes.
With the default Hbase configuration, everyone connected to the system can perform read and
write to tables in the system itself. This, of course, is not acceptable. HBase can be configured to
provide User Authentication: only authorized users can perform actions with Hbase. The
authorization system is implemented at the RPC level. This is based on the Simple
Authentication and Security Layer (SASL). This authentication supports Kerberos. SASL
implements, for each connection/transaction, the following services: authentication, encryption
negotiation and message integrity verification.
Because HBase is on top to HDFS and ZooKeeper, secure HBase uses secure HDFS and secure
ZooKeeper. For this, HBase servers create a secure service session to communicate with HDFS
and ZooKeeper.
HDFS access control is based (as implemented in UNIX physical file systems) on users, groups
and permissions. The files created by HBase are putted in HDFS with “hbase” user (default user
created by installation). The control access is based on the username of the system. HDFS
guarantees that the user used in the installation is secure and trusted with some authentication
steps.
In order to guarantee the access control HDFS uses Apache ZooKeeper, an open source
framework that coordinates distributed applications. ZooKeeper implements an Access Control
List (ACL) to control, (read and write actions), to control the users operations. This mechanism
is similar to the HDFS authentication mode.
ClouT – 31.07.2013
Page 43
FIGURE 22- HBASE ARCHITECTURE OVERVIEW
FIGURE 23 - HBASE CYCLIC REPLICATION OVERVIEW ARCHITECTURE
ClouT – 31.07.2013
Page 44
HDFS
The Apache Hadoop HDFS is a distributed file system designed to run on low cost hardware. It
has many similarities with existing distributed file systems (like Ceph, GlusterFS, etc.). HDFS is
designed, natively, to be fault tolerant system. HDFS permits fast access, more than the other
distributed file systems, to application data. It is designed for applications that need large data
sets access. HDFS permits to access data also in streaming mode.
HDFS instance is implemented in a high number of machines; each node stores a part of the file
system’s data. The probability of failure is too low thanks to the fact that each installation is
based on a large number of hardware and the data are replicated in more than one node. The
detection of a faults is demanded directly to a core, native, component of HDFS.
HDFS has a master to slave architecture. A typical HDFS cluster consists of a single NameNode,
server managing the file system namespace and controls the access to files by clients. Other
components of HDFS architecture are the DataNodes (in general one per node in the cluster).
The Data Node manages storage attached to the node that they run on. HDFS implements a file
system namespace that permits to a user to store data in files. In HDFS a file is split into more
than one block of data. These blocks are stored in more than one DataNode. In the NameNode is
executed the process that controls the file system namespace’s operations (open, close and rename files and directories). It calculates blocks map into DataNodes. The DataNodes servers
are responsible for serving read and write requests from the file system’s clients. The DataNodes execute the creation, or deletion or replication, of data blocks.
HDFS is designed to manage very large scale files stored in servers in a big cluster. HDFS
memorizes the file storing it as a sequence of data blocks; all blocks in a file, with the exception
of the last block, have the same dimension size. The data blocks are replicated and stored in
more than one server node (fault tolerant management). The data block dimension and
replication of a file are configurable in HDFS system. It is possible, for each application using
HDFS, to specify the number of replicas, and then the level of fault tolerance, of a file. The
number of replications could be specified when a file is created and, of course, can be changed
later. As is in all data storage systems, the files are created, or updated; it is not possible, for
example, append data to a file previously created. The NameNode instances are responsible for
all actions in the replication of blocks.
The Hadoop HDFS implements a file permissions control for files and directories that is much
more of the POSIX implementation. The file and the directory are associated with “owner” and
“group” (like UNIX physical file system). The object file, or the object directory, has separate
permissions for the owner user, for users are members of the group, and for all the other users.
For files, the read permission is required to read the file, and the write permission is required to
write or append to the file. For directories, the read permission is required to list the contents of
the directory, the write permission is required to create or delete files or directories, and
the execute permission is required to access a child of the directory. Obviously, there aren’t the executable files.
HDFS is Amazon S3[S3] compliant. The security access is based on Kerberos v5
ClouT – 31.07.2013
Page 45
FIGURE 24. APACHE HADOOP HDFS ARCHITECTURE OVERVIEW
FIGURE 25 - APACHE HADOOP CLUSTER SERVER ROLES
ClouT – 31.07.2013
Page 46
OPEN STACK SWIFT
Open Stack Swift is a cloud distributed object storage. It is conceptually similar to Amazon
S3[S3] and implements a subset of its API in WSGI middleware. Like S3, Swift neither can be
mounted as file system nor can be used as a raw block device. Rather, it allows storing,
retrieving and deleting objects and related metadata in virtual containers (which in Amazon S3
are “buckets”) via a RESTful HTTP interface.
The distributed and multi-tenant architecture can potentially scale up to thousands of storage
nodes with dozens of Petabytes of storage and has no single point-of-failure. Distributed means
that each blob data is replicated across a cluster of storage nodes, where the number of replicas
is configurable (at least three replicas are recommended for production infrastructures).
Objects in Swift are accessed via the REST interface, and can be stored, retrieved, and updated
on demand. The object store can be easily scaled across a large number of servers. It is
composed of the following main elements (the following description has been extracted from the
project’s documentation website1):
 Proxy Server - it orchestrates the storage access requests by searching for the location of
the account, container, or object in the ring and by routing the request accordingly;
 Ring - it maps the names of elements stored on disk with their physical location in the
clusters. There are separate rings for accounts, containers, and objects;
 Object Server- it is a blob storage server that can store, retrieve and delete objects
stored on local devices, where objects are stored as binary files on the filesystem and
metadata in the file’s extended attributes (xattrs);
 Container Server and Account server- they allow listing respectively the objects in the
container (the listings are stored as sqlite database files), and the container (per cluster,
per tenant)
 Replication - it manages the consistency of replicated objects in the cluster in case of
system failure;
 Updater - it processes the failed updates of container and data (for example in failure
scenarios or periods of high load).
With regard to the back-end file system, the only requirement is that it supports xattrs
(extended attributes file system), which is a file system feature that allows to relate computer
files with metadata not interpreted by the file system itself.
The security of the data is based on Swauth and Keystone implementations.
1http://docs.openstack.org/developer/swift/overview_architecture.html
ClouT – 31.07.2013
Page 47
CEPH
Ceph is a distributed object store and file system developed to provide high performance in
accessing the data objects and lies on a reliable, scalable and fault tolerant architecture.
The following description and figures have been extracted from the project’s website2.
FIGURE 26 - CEPH UNIFIED STORAGE
Ceph allows accessing exabytes of data from thousands of clients. A Ceph node leverages low
cost hardware and daemons, and a Ceph cluster manages a large numbers of nodes, which
communicate with each other to replicate and redistribute data in dynamic mode. Ceph
monitor can also run into a cluster of Ceph monitors to oversee the Ceph nodes in the Ceph
Storage Cluster (a monitor cluster guaranties the high availability of the service).
Ceph provides the high scalability of the clusters based on RADOS, which is a distributed object
store providing scalable storage service for a large variety of sized objects. Ceph components use
CRUSH algorithm to compute information, in efficient mode, instead of having to depend on a
centralized table. Ceph’s high-level stack includes a native interface to the Ceph Storage Cluster.
It also exposes service interfaces made on top of the system to permit a good accessible using all
the API technologies (Java, PERL, Python, etc.).
The Ceph architecture has been developed into two layers. The first one is RADOS based,
previously introduced. The second layer is composed of the Ceph file system, which is made on
top of that abstraction: file data is divided over objects, and the metadata server cluster provides
the distributed access to a POSIX file system namespace (directory hierarchy) that’s ultimately backed by more objects.
The security access to Ceph objects can be configured in various modalities: no security (all
users can access all objects) or “authentication user” (like Kerberos authentication). Cephx, the authentication sub system, uses shared “secret keys” for user authentication. The copy of the
“secret key” is stored in the client and in the monitor. The authentication protocol is such that both parties are able to prove the identity of the secret key in independent mode. This provides
2http://www.ceph.com/
ClouT – 31.07.2013
Page 48
reciprocal authentication, which means the cluster is sure the user possesses the secret key, and
the user is sure that the cluster has a copy of the secret key.
ClouT – 31.07.2013
Page 49
OBJECT STORAGE GE - FI-WARE IMPLEMENTATION
This Generic Enabler provides robust, scalable object storage functionality through an open,
standardized interface: it exposes a CDMI interface on top of OpenStack Swift. These RESTful
APIs can be accessed from any client technology that can communicate over HTTP.
By building on top of OpenStack Swift, all the benefits of this rapidly maturing open-source cloud
storage solution can be realized. The highly-available, distributed, and scalable features of Open
Stack SWIFT can be exposed using commodity hardware.
CDMI PROXY
CDMI Proxy has been developed within the FP7 European Research Project VENUS-C.
CDMI Proxy is a free implementation of SNIA CDMI standard3.
It is a storage server exposing CDMI-compliant interfaces for (currently) the following storage
back-ends:



Local disk: storing contents on a POSIX file system4
AWS S3: storing contents in Amazon public cloud
Azure Blob: storing content in Azure public cloud
The main features of CDMI Proxy are:



Python based
It supports CDMI Blobs and Multi-level containers (also for single-level backends, e.g.
AWS or Azure)
It stores metadata of the stored objects in CouchDB:
The CDMI Clients that work with CDMI Proxy include:




libcdmi5: java SDK for running CDMI calls
libcdmi6 : python SDK for running CDMI calls
cdmifs7: FUSE[FUSE] based file system using the CDMI standard (v1.0)
r2ad8: demo clients for OCCI[OCCI] and CDMI
3
http://www.snia.org/tech_activities/standards/curr_standards/cdmi
5https://github.com/livenson/libcdmi-java
6https://github.com/livenson/libcdmi-python
7https://github.com/koenbollen/cdmifs
8http://r2ad.net/
ClouT – 31.07.2013
Page 50
REST/SOAP APIS FOR IDAS ACCESS
IDAS platform offers two APIs based on two different Web Services technologies, SOAP and
REST and providing different functionalities mainly based on five different issues:





Data Model: IDAS organizes data into a hierarchy of different components. This block
offers functionality that allows to create, update, query and delete information of such
components (services, assets, devices and groups).
Sensor Data: for operations related to retrieve the last measure and historical
measures published by a device.
Command: for the operations related to sending commands and receiving results.
Subscription: for operations related to subscription to events.
Provision: for operations related to configuring internal components.
Every block has several entities, every entity has a specific and unique URI, several operations
(GET read, POST create, PUT update, DELETE delete), not all entities have all the
functions. All API requests/responses use JSON format, except notifications which are received
in XML format by the subscribed applications.
IDAS states as main asset in the FI-WARE IoT Back-end Device management Generic Enabler
(GE) within the Open Innovation Lab initiative.
TESTBED RUNTIME
Dense deployments need efficient modules and mechanism for managing deployed nodes in a
remote way. Testbed Runtime module offers these set of functionalities through several entities
defined next:


Sensor Network Authentication and Authorization (SNAA): the SNAA component offers
the basic functions for access control, through Shibboleth-based authentication and
authorization. This allows users to access to different resources of the platform
according to the corresponding rights associated to each user.
Reservation System (RS): this module allows authenticated users to make, query and
edit reservations of IoT nodes and supports the persistence of these reservations. In this
sense, a set of nodes can be selected for being sent a determined command or, indeed, for
being remotely flashed using MOTAP (Multihop Over The Air Programming) procedure.
Wireless Sensor Network API (WSN API): the WSN API represents the back-end implementation
of the set of functionalities required for interacting with the IoT nodes with commands such as
reset, reprogram, check if a node is alive, add/remove virtual links and many others. The iWSN
API provides also the implementation of a channel for exchanging debug and control message
with IoT nodes.
ClouT – 31.07.2013
Page 51
SOA3
Service Oriented Authorization, Authentication and Accounting (SOA3) is set of RESTful web
services providing User Management, Authentication, Authorization, Accounting and Identity
Federation.
The User Management Service is based on an LDAP and provides REST interfaces for performing
CRUD operations on users, groups and roles : on these identities the Authentication Service
provides REST and Java Methods to easily perform username/password authentication.
The Authorization Service is based on ABAC (Attribute Based Access Control) model. It takes
authorization decisions on the basis of stored policies composed by four sets of attributes:
 Subject, defining who requires doing something (e.g. userid, roles, groups...)
 Action, defining what he/she wants to do (e.g. read, write...)
 Resource, defining on what the action should be applied (e.g. a certain image, document...)
 Environment, defining the external context on which the action should be performed (e.g.
Hour, day ...)
The Accounting Service, also available as Java API, provides several options for record retrieving
and report generation in a simple and flexible data model. It also provides a billing module.
The Identity Federation Service can be considered as a part of the Authentication Service,
providing the possibility to compose a federation of domains in which a user of one domain can
access the others by using the same credentials. The technology used is SAML, which in SOA3 is
used with the extension SAML Condition to Delegate for providing Access Delegation.
ClouT – 31.07.2013
Page 52
4.3.
Dependability tools for accessing city data reusable components
D-CASE MONITORING TOOL
D-Case Monitoring Tool enables developers of ClouT applications to show their dependability
with statically captured evidences, such as results of testing, fault injection, verification, etc. It
also enables managers of ClouT applications to monitor their dependability in run time by
collecting dynamic evidences from the running systems. It is provided as an Eclipse plug-in, on
which users can author and execute D-Case documents, an extended form of assurance cases
written using Goal Structuring Notation. In this tool, static evidence is represented with an
evidence node to which one or more reasonable materials are associated. Dynamic evidence is
represented with a monitor node to which a piece of program is associated.
FIGURE 27 SCREENSHOT OF D-CASE MONITORING TOOL
Figure 27 shows a screenshot of D-Case Monitoring Tool. It is a plug-in for the well-known
development tool Eclipse providing the following two functionalities: editing and running a DCase. The whole configuration is depicted in Figure 28.
ClouT – 31.07.2013
Page 53
FIGURE 28 SYSTEM OVERVIEW OF D-CASE MONITORING TOOL
D-Case is an extended form of Assurance Cases, which is widely used to show safety of
infrastructural construction in European countries. Its idea is to (1) set the top goal, which is the
condition that the developers need to meet, (2) divide the goal reiteratively based on strategies
into a tree of small pieces of sub goals, and (3) show static evidence to the leaf sub goals in the
tree. If all the leaf sub goals get their evidence, the top goal is considered to be met. The evidence
can be shown in a D-Case using the evidence node, to which any form of material can be
associated.
The new scheme we propose is a dynamic way of getting evidence from running systems. Since
ubiquitous sensor networks are highly dynamic system, static evidence, such as results from test
cases, etc., is not always useful. Rather, values in running ubiquitous sensor networks are more
important to know whether the networks are running in a healthy state.
TABLE 3 LIST OF AVAILABLE MONITORING PROGRAMS
Name
Lines
Monitoring Target
SQLQuery
111
Faults in stored sensor data.
SNMPSwitch
328
Traffic of network switches
SNMPSwitchPort 132
Ports of a network switch
MuninClient
82
Wrapper of Munin
DNS
43
DNS entry
ping.sh
35
Reachability of an IP node
df.sh
28
Disk capacity of a host
lsusb.sh
51
USB device connectivity of a
host
uptime.sh
30
Load of a host
ClouT – 31.07.2013
Page 54
ifconfig.sh
43
Network interface of a host
To this end, we enable programmers to associate a piece of program to the monitor node in DCase. This piece of program is what we term as a Monitoring Program. We then enable
developers to execute a D-Case, resulting that all the monitoring programs associated to monitor
nodes are executed. We have successfully implemented monitoring programs, such as those
getting values from SNMP-enabled devices, failure detection mechanism from a series of sensor
data, and so on.
Figure 27, shows a D-Case being executed, in which the colored oval nodes are monitor nodes.
The red ones mean that the monitoring program associated to the node detected some faults,
while the blue ones mean the targeted system is running in a healthy state in the monitoring
programs' scope. Developers, by watching at a whole D-Case can easily find faults in running
ubiquitous sensor networks. They can also know detailed information gathered by the
monitoring programs on web pages, whose sample can be found in Figure 29, that are exported
by D-Case runtime.
FIGURE 29 WEB PAGE FOR DETAILED MONITORING DATA
ClouT – 31.07.2013
Page 55
D-Case, not only enabling developers to visually find faults, allows them to add an Action node to
monitor nodes. The role of an action node is to automatically resolve or mitigate the effect of the
faults. The idea of action nodes is to associate a piece of recovery program to the action nodes. In
Figure 27, the node at the bottom connected to all the monitor nodes is the action node. Simple
example of an action node include notifying developers/managers via e-mail, remotely
rebooting a sensor node, modify routing paths in a sensor network, etc…
Since D-case leverages natural languages and structuring figures, developers can assemble a DCase as detail, consistent, and exhaustive as they can. Oppositely, a D-Case may be too abstract,
inconsistent, or defective if the developers fail to use good strategies to divide goals into subgoals. To ease the development of a D-Case we recommend developers to first conduct risk
analysis, and then put its result into a D-Case. This will increase exhaustiveness of a D-Case with
help of the established risk analysis methods like FTA and FMEA.
ClouT – 31.07.2013
Page 56
5. Other Reusable CPaaS Assets from Cities
5.1.
Introduction
In this paragraph are described all the components used by the cities platforms involved in the
ClouT project.
5.2.
Genoa Reusable CPaaS assets
In this paragraph are listed all the reusable components proposed by Genoa municipality.
ROADVISIOR - EMIXER PLATFORM
e-miXerprovides a service-oriented middleware infrastructure enabling the integration of
data/services supplied by different operators in our domain of Traffic and Travel Information..
Key interoperability feature and element in e-m iXeris a multi-standard interface, which is able
to interconnect and combine data from different systems and map them to a common data and
service model. Diversified content/services offerings are then made accessible and suitable for
use by Traffic Information Service Providers (TISPs) via a standardized access layer: a B2B
interface implementing several European ITS reference standards in the various addressed
mobility domains (traffic, public transport, parking, etc.).
The e-miXer core drives the process of data acquisition based on parameters hosted by the
registry. The storage connector enables a temporary caching or permanent storage of the
information.
e-miXer engines process data for specific purposes (e.g. routing calculation, traffic data
processing etc.).
e-miXer provides both B2C and B2B services:
ClouT – 31.07.2013
Page 57
 The e-miXer client interface enables the provision of services for the end users (both
public and restricted access)
 B2B interfaces are available for external service providers whenever necessary (e.g.
providers of IVR, SMS, email services)
< G E O SE R V E R P L AT FO R M >
GeoServer is an open source software server written in Java that allows users to share and edit
geospatial data. Designed for interoperability, it publishes data from any major spatial data
source using open standards.
Being a community-driven project, GeoServer is developed, tested, and supported by a diverse
group of individuals and organizations from around the world.
GeoServer is the reference implementation of the Open Geospatial Consortium (OGC) Web
Feature Service (WFS) and Web Coverage Service (WCS) standards, as well as a high
performance certified compliant Web Map Service (WMS). GeoServer forms a core component of
the Geospatial Web.
GeoServer can create maps in a variety of output formats. OpenLayers, a free mapping library, is
integrated into GeoServer, making map generation quick and easy. GeoServer is built on
Geotools, an open source Java GIS toolkit.
Geoserver is a free software.
ClouT – 31.07.2013
Page 58
The municipality of Genoa has implemented a Geoserver solution to handle some geo-data
related to safety (Civil Protection, Industrial Areas, Public Works, Safety).
WEATHER STATION
The Weather Station System developed in collaboration with Civil protection can export a series
of detailed information about the temperature, the humidity level, and weather in general, for
more than twenty areas of the Municipality.
The Weather Station is based on a server side scripting language (PHP) in order to produce data.
The platform is composed by a Web server with a PHP processor module which generates the
resulting Web page. The platform’s scripts act as filters taking inputs from a series of parameters
and outputting another stream of data. The output will be HTML or WAP. The data are updated
every five minutes.
Data returned from Weather Station
Measure Unit Description
Temperatura:
°C
Temperature
Umidità:
%
Humidity
Vento:
km/h da
Wind
Vento medio:
km/h
Averagewind
ClouT – 31.07.2013
Page 59
mm
Rainday
Pioggia giorno:
Rain Rate:
mm/h
Rain Rate
Dewpoint :
°C
Dew point
Pressione:
hPa
Pressure
Temp minima:
°C
Minimum Temp
Temp massima:
°C
Max Temp
Raffica max:
km/h
Max Wind Speed
Rain rate max:
mm/h
Max Rain rate
Pioggia mese:
mm
MonthRain
Pioggia anno:
mm
YearRain
5.3.
Santander reusable CPaaS assets
In this paragraph are listed all the reusable components proposed by Santander municipality.
GIS PLATFORM
Santander City Council has a GIS Platform for storing and processing Geo referenced Data,
accordingly to the City needs. This platform uses ESRI Technology and offers a big amount of
Interoperability.
Some of the geo referenced data reused in this project are running in this platform so it
considered also a reused asset in the Project. Here is a simplified vision of the Platform
Architecture:
ClouT – 31.07.2013
Page 60
ArcGIS Server is a powerful and flexible platform for providing a variety of spatial data and
services to GIS users and applications requirements. It provides the ability for our organization
to implement server-based spatial functionality for focused applications utilizing the rich
functionality of ArcObjects. Building robust, scalable applications is not a simple task, and
proper application design is required to support systems having good performance.
ArcGIS Server is a distributed system consisting of several components that can be distributed
across multiple machines. Each component in the ArcGIS Server system plays a specific role in
the process of managing, activating, deactivating, and load balancing the resources that are
allocated to a given server object or set of server objects. The components of ArcGIS Server can
be summarized as follows:

GIS server: responsible for hosting and managing server objects, is the set of objects,
applications, and services that makes it possible to run ArcObjects components on a
server. The Gis Server is divided in the next set of components:
o Server Objects: A server object is a software object that manages and serves a GIS
resource such as a map or a locator. For example, a server object named
SantanderMap may serve a map document of data for the city of Santander, while
the server object SantanderGeocode may serve an address locator for geocoding
addresses. ArcGIS Server objects are themselves ArcObjects components.
Server objects are managed and run within the GIS server. A server object may be
preconfigured and preloaded in the server and can be shared between
applications. Server applications make use of server objects and may also use
other ArcObjects components that are installed on the GIS server.
ClouT – 31.07.2013
Page 61
o Server Object Manager (SOM): The GIS server is composed of a SOM, which is a
service running on a single machine, and SOCs (later described), which run on
one or more machines (container machines). The SOM manages the set of server
objects that are distributed across one or more container machines. When an
application makes a direct connection to a GIS server over a LAN or WAN, it is
making a connection to the SOM, so the parameter that is provided for the
connection to be made is the name or Internet Protocol address of the SOM
machine (URL).
o Server Object Container (SOC): The container machine or machines actually host
the server objects that are managed by the SOM. Each container machine is
capable of hosting multiple container processes. (A container process is a process
in which one or more server objects are running.) Container processes are
started and shut down by the SOM. The objects hosted within the container
processes are ArcObjects components that are installed on the container machine
as part of the installation of ArcGIS Server. All server objects have the potential to
run on all container machines and are balanced equally across all container
machines. Therefore, it is important that all container machines have access to
the resources and data necessary to run each server object. It is also important to
note that the GIS server assumes that all container machines are configured
equally, such that they are all capable of hosting the same number of server
objects. Server object resources and data are discussed in more detail in the next
section
o Server Directory: The server directory is a location on a file system. The GIS
server is configured to clean up any files it writes to a server directory. By
definition, a server directory can be written to by all container machines. The GIS
server hosts and manages server objects and other ArcObjects components for
use in applications. In many cases, the use of those objects requires writing
output to files. For example, when a map server object draws a map, it writes
images to disk on the Web server machine. Other applications may write their
own data; for example, an application that checks out data from a geodatabase
may write the checkout personal geo database to disk on the server.
Typically, these files are transient and need only be available to the application for a
short time, for example, the time for the application to draw the map or the time
required to download the checkout database. As applications do their work and write
out data, these files can accumulate quickly. The GIS server spawns a special SOC
process that will automatically clean up its output if that output is written to a server
directory.
A server directory can be configured such that files created by the GIS server in it are
cleaned based on either file age or time since they were last accessed. The maximum
file age is a property of a server directory. All files created by the GIS server that are
older than the maximum age or have not been accessed during the time defined by the
maximum age are automatically cleaned up by the GIS server.
In addition to supporting output of image files to the file system, ArcGIS Server also
supports streaming of images from the SOC (via the Web server) to the user. This is
useful since the streaming occurs over port 80, which avoids having to open additional
TCP ports for drive mapping from the SOC to the Web server.
ClouT – 31.07.2013
Page 62

Web browser: The Web server hosts server applications and Web services written using
the ArcGIS Server application program interface (API). These server applications use the
ArcGIS Server API to connect to a SOM, make use of server objects, and create other
ArcObjects for use in their applications. These Web services and Web applications can be
written using the ArcGIS Server Application Development Framework (ADF), which is
available for both .NET and Java developers. Examples of Web applications include
mapping applications, disconnected editing applications, and any other application that
makes use of ArcObjects is appropriate for Web browsers. Examples of Web services
include those for exposing map and geocode server objects that desktop GIS users can
connect to and consume over the Internet. It is possible for users to create their own
native .NET or Java Web services whose parameters are not ArcObjects types but do
perform a specific GIS function. For example, it is possible to write a Web service called
FindNearestHospital that accepts (x,y) coordinates as input and returns an applicationdefined Hospital object that has properties such as the address, name, and number of
beds.
Web applications connect to GIS servers within their organization over the LAN. In this sense,
the Web application or Web service is a client of the GIS server. Users connect to Web
applications and Web services over the Internet or Intranet, but all the Web applications'
logic runs in the Web server and sends HyperText Markup Language (HTML) output to the
browser client. The Web application itself makes use of objects and functionality running
within the GIS server. This allows the development of Web applications to make use of
ArcObjects in the server as would a desktop application connecting to the GIS server in
client/server mode over the LAN or WAN.
As users interact with their browsers, they make requests to the Web application, which in
turn makes requests on the SOM. The SOM returns a proxy to a server object or server objects
that are running within the GIS server. The Web application uses this proxy to work with the
object as if it existed in the Web applications process, but all execution happens on the GIS
server.

Desktop applications: Users can connect to ArcGIS Server using ArcGIS Desktop
applications to make use of map and geocode server objects running in the server. Users
use ArcCatalog to connect to a GIS server directly on the LAN. They can also specify the
URL of a Web service catalog to indirectly connect to a GIS server over the Internet to
make use of map and geocode server objects exposed by that Web service catalog. In
addition, the set of server objects and their properties are managed by the GIS server
administrator using ArcCatalog. Our administrators can connect to the GIS server over
the LAN and use ArcCatalog to add and remove map and geocode server objects. They
can also configure how server objects should be run including the set of container
machines that are available for the server and the directories on the server that they can
use to write any output.

ArcSDE: ArcSDE is ESRI's technology for accessing and managing geospatial data within
relational databases. ArcSDE technology supports reading and writing of multiple
standards including (among other data storage options) Open Geospatial Consortium,
Inc. (OGC), standards for simple features; the International Organization for
Standardization (ISO) standard for spatial types; and the Oracle Spatial format. ArcSDE
capabilities are next:
ClouT – 31.07.2013
Page 63
o It is open and interoperable across multiple database management systems
(DBMS). In Santander City Council it is running over Oracle Platform.
o It is standards based, using as its native data structure the OGC binary simple
features standard and the ISO spatial type.
o It supports full, open SQL access to geo databases stored in Oracle, IBM DB2, IBM
Informix, and PostgreSQL.
o It provides high performance and scales to a large number of users. ArcSDE geo
databases outperform all other solutions for storage and retrieval of spatial data.
ClouT – 31.07.2013
Page 64
OPEN DATA PLATFORM
Santander City Council has recently deployed an Open Data platform for opening to the Citizen,
all public data that resides in his internal databases. Main focus of this platform is directed
towards Companies and Entrepreneurs, in order to take advantage of this data and create
products and services based on these, thereby promoting the business and the creation of jobs.
The focus has also been placed on providing citizens proactively data, looking for a better
understand of how an administration works internally, and eliminating, in some cases, slow and
costly administrative procedures to access data that, although public, it is not available to
citizens.
The architectural definition of the platform is described in the next picture:
For a better understand of the platform, here is a description of main systems:
Front-End:
Front-End is the visible part of the Platform, it is the graphical interface for the final user. From
the technological point of view, it is composed by three popular components, developed by the
open source community, that had been reused and adapted to specific needs of the platform.
These components are
1. Wordpress: Prestigious CMS, aimed, from his birth, to build Blogs, but over time has
incorporated features to be considered the content management system most used in the
network. This component has the responsibility to implement the final graphical user
interface that provides access to the data and the Open Data Portal.
2. CKan: It is a CMS focused in Open Government Projects, used and managed mainly by
United Kingdom Government, and reused by main Open Data Portals World Wide. This
component is in charge to outfit with all infrastructure for defining and meta-dating
ClouT – 31.07.2013
Page 65
datasets and resources, and also supply APIs to allow developers, to automate data
consumption.
3. Virtuoso: Virtuoso is a Web tool that allows the terminology definition, for the creation
and promotion of Semantic Web. Within the Platform, his role is to define those specific
words of a Municipality or local Authority, that had not been defined so far by any
standardization corporation or other Open Data Platforms.
Back-End
Open data platform, has also a Back-End subsystem in charge of doing data gathering tasks and
supporting Sparql engine. This subsystem is composed by two components, also developed by
Open Source Community, and adapted for the particular platforms needs. These components
are:
1. iCMS: A data gathering system developed by Government of Andalucía, which main
function is to constantly maintain Front-End feeded with updated data. It should be
mentioned that the data contained on the website are not mere snapshots of available
data at a given time, but the system is being updated constantly by Municipal databases
to always provide updated data. Therefore, this tool or technology component plays a
transcendental role in the Platform to enable a real time open data platform.
As an example, in the case of census dataset, if we query the number of people from one
day to another, we see that the total amount will be varied depending on the
registrations and cancellations occurred in that time interval.
This real-time updating is made through ad-hoc drivers developed for every Municipality
data producer.
2. Marmota/Neogolsim: This tool is responsible for providing the SPARQL engine to the
platform. The purpose of this engine is the one hand, provide data in RDF format (format
of choice for reuse) and on the other, allow creating cross-data queries with other Open
Data Platforms world-wide, thus providing platform with advance features that allows it
to follow the Semantic Web path
3. Finally this platform is a reusable component within the Project because actually it is
serving several information that are valuable for the Multimodal City Transport use
cases. Also this platform can be a good channel to make any additional municipality data
available need by this project.
ClouT – 31.07.2013
Page 66
ORACLE DATABASE PLATFORM
Santander City Council has actually a Municipality database Cluster based on Oracle Technology
that stores and processes most of City data, including valuable data for the Clout Project as
traffic measures, and Participatory Sensoring events. Also other secondary database platforms
are running in this platform, like MySql or Microsft Sql Server, but we will focus on the Oracle
one, that has the best reusable capabilities for this project. Here is a global vision of the
architecture:
FIGURE 30 SANTANDER CLUSTER DATABASE ARCHITECTURE
The Oracle cluster comprises multiple interconnected servers that appear as if they are one
server to end users and applications. The cluster is mounted on Oracle RAC technology that
makes the hard work and enables us to cluster Oracle databases, and to bind multiple servers so
they operate as a single system.
Oracle RAC also provides high availability and scalability for all application types, and enables us
to implement grid computing architecture. Having multiple instances accessing a single
database, prevents the server from being a single point of failure. Applications deployed on
Oracle RAC don’t need to adapt code to support it.
Our Oracle Cluster has two database instances, each one contains memory structures and
background processes. The Oracle RAC unify this instances and create a single-instance with
ClouT – 31.07.2013
Page 67
same processes and memory structures as well as additional process and memory structures
that are specific to Oracle RAC. Any one instance's database view is nearly identical to any other
instance's view in the same Oracle RAC database; the view is a single system image of the
environment.
Each instance has a buffer cache in its System Global Area (SGA). But with Cache Fusion, Oracle
RAC environments logically combine each instance's buffer cache to enable the instances to
process data as if the data resided on a logically combined, single cache.
To ensure that each Oracle RAC database instance obtains the block that it needs to satisfy a
query or transaction as fast as possible, and to ensure data integrity, Oracle RAC instances use
two processes, the Global Cache Service (GCS) and the Global En-queue Service (GES). The GCS
and GES maintain records of the statuses of each data file and each cached block using a Global
Resource Directory (GRD). The GRD contents are distributed across all of the active instances,
which increases the size of the SGA for an Oracle RAC instance.
After one instance caches data, any other instance within the same cluster database can acquire
a block image from another instance in the same database faster than by reading the block from
disk. Therefore, Cache Fusion moves current blocks between instances rather than re-reading
the blocks from disk. When a consistent block is needed or a changed block is required on
another instance, Cache Fusion transfers the block image directly between the affected
instances. Oracle RAC uses the private interconnect for inter instance communication and block
transfers. The GES Monitor and the Instance En-queue Process manages access to Cache Fusion
resources and en-queue recovery processing.
This platform that actually gives database support to reusable components of this project like
Traffic data or Participatory sensoring events, must be also re-used to support some kind of
relational data storage needs.
ClouT – 31.07.2013
Page 68
SAE PLATFORM
Transportation service of Santander City Council is equipped with an Exploitation Supporting
System. This system, also known as SAE Platform, is a set of solutions that merges several
technologies to improve service and management of transportation.
The main component of this Platform is the onboard equipment composed by a GPS locator, a
central process unit and a communication system, capable to communicate in real-time (or a
configurable frequency) the vehicle's position to the control center.
Another major component is the control center, in charge of receiving all the data generated by
the on-board Equipment and give our staff the possibility to make decisions, through an
operation console, and take actions over the related vehicle. The applications of the SAE
platform are divers, however, in Santander Municipality are focused on managing bus fleet. This
system allows Municipality Transportation Service:
 To track frequency of the bus stopping in regular bus lines.
 To report passengers about estimated time to arrive of the next line bus.
 Optimize number of vehicles in line accordingly to each moment needs.
 Locate all vehicles through a GUI projected over Municipality Cartography
SAE platform provide tools that allow carrying out a comprehensive tracking of the
transportation network to enhance the social service offered to the citizen and to optimize the
planning of metropolitan transportation lines.
All the information generated by this system is a valuable asset for the Clout Project. For
Instance, the bus real stop frequency log, the arrival bus estimations, the planned lines
timetable, the bus stop locations, are clear examples of candidate data to be reused in the
project.
ClouT – 31.07.2013
Page 69
INCISIS PLATFORM
Within SmartSantander Project, Santander City Council implemented a Participatory Sensoring
App, named Pace of the City, to enable extension of IoT deployment to the Citizen mobile
devices. This initiative allows Citizen to send measures and Events to the SmartCity Platform.
Measures contribute to feed system with more valuable data, and Events contribute to create a
perspective view of what is happening on the city on each moment, and give the possibility to
study human behavior within an Urban Area.
FIGURE 31 SANTANDER INCISIS IMPLEMENTATION ARCHITECTURE
Some of this reported events, implies the Santander City Council to take actions, in order to give
response to matters of his responsibility. For example, if a hole in the street is reported, City
Council must take actions to repair it as soon as possible to avoid major damage or Citizen
injuries.
For this Purposes, Santander City Council deployed an internal Platform, named INCISIS, which
allows Municipality workers, to make the management of the event. The functional schema is
next:
Participatory
Service
Participatory
Service
Services
Allocation
Task
Reception
Filtering
Normalization
FIGURE 32 SANTANDER INCISIS FUNCTIONAL SCHEMA
Citizen Participatory Municipality service receives events in the platform and acts as a first level
helpdesk. This service makes a filtering of the events and normalization of the information
ClouT – 31.07.2013
Page 70
received. In those cases where a fast action or response can be given, this service resolves and
closes the event. If Event requires a further action, this service allocates it, to a more specialized
Municipality Service and the Event is treated as a second level of Helpdesk.
After reception of the task the service can perform several actions identified in the next figure:
Service
Resolve & Close
Resolve & Reallocation
Reallocation
Cancel
FIGURE 33 INCISIS SERVICE EXAMPLE SCHEMA




Resolve and Close: When a solution is attached to only the service allocated, this does the
job, and closes the event describing the actions taken.
Resolve an Reallocation: For events affecting multiple services, the service performs the
work necessary to solve its part of the task and reassign it to a new service reporting the
actions taken.
Reallocation: A service has been allocated with an event by mistake or simply considered
it is not on his scope, so it reallocates the event to a new service reporting the reason for
the reallocation
Cancel: The service cancels the incidence and reports a quite justifiably reason.
To Conclude, INCISIS platform is a value platform for the ClouT project, because it is closely
related with the Participatory sensoring uses cases planned for this Project. Also, this platform
offers SOAP APIs to query or register events, so any exchange of data with other platform of the
project could be accomplished.
ClouT – 31.07.2013
Page 71
5.4.
Fujusawa reusable CPaaS assets
In this paragraph are listed all the reusable components proposed by Fujusawa municipality.
DISASTER PREVENTION GIS SYSTEM
As possible CPaaS assets in Fujisawa, Fujisawa has disaster prevention GIS system. The system
has following services – quick providing list of disaster information, equipment map for disaster
prevention, evacuee search engine and communication BBS, etc. However, in 3.11 earthquake
disaster in Japan, the GIS system did not work perfectly. Thus, it is important to create this kind
system with dependability. In addition, it is necessary to combine this kind of GIS system with
more various sensors.
Figure 34: Fujisawa disaster prevention gis system
ClouT – 31.07.2013
Page 72
DIGITAL SIGNAGE SYSTEM
Fujisawa, Keio University and several organizations has been experimented digital signage
system in Fujisawa, called Fujisawa Signage. 15 digital signage are set to different 15 places such
as city office, citizen center, cultural center in Fujisawa. Each signage can show each different
contents which is useful for citizens who visits the place where the signage is placed. In addition,
citizens themselves can post their original information to digital signage through several
authentication steps.
Figure 35. Overview of Fujisawa Signage
ClouT – 31.07.2013
Page 73
5.5.
Mitaka Reusable CPaaS assets
In this paragraph are listed all the reusable components proposed by Mitaka municipality.
GIS SYSTEM
Mitaka city provides publically available web based GIS systems. It provides 24 kinds of maps in
6 categories; city status, living, welfare and health, security and untroubled life, sight-seeing
/culture/nature, and child-raising/education. For example, traffic information category, many
information about traffic such as “there are many bicycle in this road” and “it is hard to recognize surround situation in this sloping road” is provided to map. In terms of sensor-related
information, this map provides amount of radiation in many points in Mitaka because citizen’s concern to radiation has been increased due to 3.11 disaster. However, the information is very
static, not real-timely updated. Moreover, other kinds of sensors such as temperature, humidity
or noise sound level are not currently provided. Thus, it is important to provide real-time IoT
information to citizens by using this kind of GIS visualizing system.
Figure 36:Mitaka GIS system
ClouT – 31.07.2013
Page 74
6. Conclusions
This document presents different solutions for different problems using heterogeneous
technologies. Synthetic tables are provided in order to facilitate the analysis that we will do
on these components in order to choose the most adequate ones in the light of the project
use cases and objectives. Further analysis (e.g., compatibilities between the components
(used languages, data models, etc.), maturity analysis, API compatibility, etc.) and selection
will be done in the project and reflected in the next deliverable of the Work package 1, D1.2:
First version of Reference Architecture and reusable components.
As described in the introduction of this document, each chapter has been dedicated to a
specific topic (regarding to the Infrastructure layer); for each topic one or more reusable
objects have been identified, as shown in the following schema:
6.1.
Service composition platforms for citizen’s a pplications
The components presented in the Service composition platforms section include solutions
from 2 main domains: data mashup tools (SWIFT, WireCloud, Mycocktail, EMML, OpenSocial,
Yahoo Pipes and Apache Shinding) and service/business process composition solutions
(iPojo. BPMN and BPEL). Besides, the section also presented five methods to make the
composition robust, respecting QoS properties, verifying time and resource constraints and
detecting conflicting business rules. While the data mash up tools target non-technical endusers in order to create rapidly applications from existing data sources, service composition
tools target more experienced developers that require a minimum dependability and Quality
of Service requirements. While mash up technologies are mostly web based platforms,
service composition tools may require more sophisticated tools in order to provide required
ClouT – 31.07.2013
Page 75
properties to the services. Other criteria for selection of the components will be used
programming languages, licenses (free, open) and compatibility with other neighbor
components in the architecture.
6.2.
Big data processing
The components presented in the big data processing section can similarly be grouped into 3
classes: processing of data (Esper, FI-WARE Gateway Data Handling, JBoss Drools Expert),
storage of data (MongoDB) and solutions for self-healing of faulty data coming from IoT
devices. These components represent the data processing features that are necessary on the
platforms in order to process the IoT device data. Note that, all about data storage and
secure access to city data in the cloud are the issues belonging to the Section “Secure and dependable access to city data “.
6.3.
Secure and dependable access to city data
In the Secure and dependable access to city data paragraph are presented in two groups,
Open city data hosting and tools for accessing city data. The hosting of data is performed by
the noSQL database software (Hypertable and HBASE) and the cloud storage and distributed
file systems (Open Stack Swift, Ceph and Hadoop HDFS). The access to the resources (data
stored in database or file stored in the storages) is centralized thanks to the CDMI Proxy
framework. The security access to the resources is performed by the SOA3 framework.
The component presented in dependability tools for accessing city data reusable
components is D-Case Monitoring Tool. This tool is used to monitoring the resources.
6.4.
City resources
In the city resources are presented all the components used by the municipalities. There
tools used to store data (Oracle database for example), tools used to monitor of the risk
catastrophic event (Disaster prevention for example). More of the tools presented in this
section are GIS systems.
ClouT – 31.07.2013
Page 76
7. Appendix
7.1.
TG3.1 Reusable Components
In this paragraph are reported the tables with all reusable components you introduced in
chapter 2 “Service Composition Platform for Citizen’s applications”, chapter 3 “Big data Processing”, chapter 4 “Secure and dependable access to city data” and chapter 5 “Other Reusable CPaaS Assets from Cities”.
TABLE 4 – OPEN STACK SWIFT
Proposing
Partner
ENG
Anatomy
ID Card
Info
Name
Open Stack SWIFT
Owner(s)
OpenStack Foundation
Link to code
https://github.com/openstack/swift
Open Stack Swift: http://www.openstack.org/software/openstackstorage/
Project, plus link
Patents or other IPR
exploitation
(licenses)
The OpenStack project is provided under the Apache 2.0 license
Description
Open Stack Swift is cloud distributed object storage….
Type
Programming
language/based
technologies
Type
Exposed APIs
RESTful APIs provided by OpenStack services
Target User(s)
All end user applications and ClouT architectural components that need
to use ClouT storage and are able to interact with RESTful APIs
Rackspace: http://www.rackspace.com/cloud/
References
Assessment
Adopted
Standard(s)
Python
Companies Supporting The OpenStack Foundation:
http://www.openstack.org/foundation/companies/
 RESTful;
 HTTP;
The goal of Open Stack Swift is the capability to store petabyte of data
in a distributed, standard, infrastructure of cluster servers.
Open Stack Swift is not a file based storage system, but a very
distributed, with redundancy capabilities, cloud storage infrastructure.
Capabilities
/Opportunities
ClouT – 31.07.2013
Thanks to the replication of data (stored and replicated in more than
one server), Open Stack Swift has an high level of secure in case of
failure of one server.
Page 77
Known
Limitations/Threats The object stored must be less than 5 GB………
TABLE 5- WIRECLOUD
Proposing
Partner
ENG
Info
Name
Owner(s)
ID Card
Link to code
https://github.com/Wirecloud/
FI-WARE project Wirecloud page, http://catalogue.fiware.eu/enablers/documentation-6
Wirecloud homepage : http://conwet.fi.upm.es/wirecloud/
Project, plus link
Patents or other IPR Affero General Public License version 3 (AGPL v.3) :
exploitation
http://www.gnu.org/licenses/agpl-3.0.html
(licenses)
The WirecloudMashup Platform is a framework that allows end-users
to create specific mashups composing widgets that are the building
blocks to create new applications. The WirecloudMashup Platform, is
composed by three different components: the Application Mashup
Editor, the Mashup Execution Engine and the catalogue.
Description
Type
Anatomy
Wirecloud
UPM CoNWeT Lab (Universidad Politécnica de Madrid's School of
Computer Science)
Programming
language/based
technologies
Web application
Server side:
 Python
Client side:
 Javascript
 HTML
 CSS
The platform provides two different type of API:
 The Widget API is a JavaScript API that allows deployed
widgets in a Mashup Execution Engine to gain access to its
functionalities
 The Application Mashup API is a RESTful API that provides the
functionality to create and modify workspaces and to manage
the resources available for building these workspaces.
Exposed APIs
Target User(s)
References
ClouT – 31.07.2013
End users, software developers
The Wirecloud platform has been developed and used in the FI-WARE
project (www.fi-ware.eu): moreover Wirecloud is currently used in
different EU research projects such as Outsmart (http://www.fi-pppoutsmart.eu/) and Envirofi (http://www.envirofi.eu)
Page 78
Assessment
Adopted
Standard(s)
Capabilities
/Opportunities
RESTFul, HTTP
WirecloudMashup Platform provides several functionalities that could
be useful for the mashup tool to be developed in ClouT. In particular the
graphical editor, that allows to connect the gadgets data, is simple to
use for non-technical users and also the catalogue allows to share
mashups with other stakeholder of the ecosystem fostering reuse of
services in different contexts (example the different pilots of ClouT
project). Also the AGPL license allows a possible reuse of this platform.
Some problems could be related in technical incompatibilities with
other components of ClouT architecture: a deep study of the API will be
conducted in order to understand the real reusability of the platform
functionalities from external components.
The platform doesn’t allow creating or modifying the widgets using the
graphical editor: the single widget has to be created through software
programming and then imported in the catalogue.
Also it seems that the widgets cannot be exported or shared in different
Known
Limitations/Threats containers (e.g. open social gadget containers) limiting the use of the
mashup to the Wirecloud platform.
TABLE 6- MYCOCTAILS
Proposing
Partner
ENG
ID Card
Info
Name
MyCocktail - Romulus Mashup Builder
Owner(s)
InformáticaGesfor, Romulus Project
Binary:
http://sourceforge.net/projects/mycocktail/files/releases/v1/MyCocktail_1.5.0.z
ip/download
Source:https://mycocktail.svn.sourceforge.net/svnroot/mycocktail/trunk/MyCoc
ktail/
Online instance: http://www.ict-romulus.eu/MyCocktail/#
Romulus project: http://www.ict-romulus.eu
Link to code
Anatomy
Project, plus link
Patents or other
IPR exploitation
(licenses)
Description
Type
MyCocktail website: http://www.ict-romulus.eu/web/mycocktail/home
Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0 )
MyCocktail provides two different web tools: the mashup builder and the page
editor. The mashup builder is a graphical environment that allows combining and
modifying data, using specific filters, information coming from REST services, and
connecting these data with web gadgets creating mashups. The Page Editor allows
designing a web page through a GUI allowing to integrate mashups created with
MyCocktail with other HTML elements.
Programming
language/based
technologies
Web Application
Server side:
 Java
Client side:
 Javascript
 HTML
 CSS
Exposed APIs
None
ClouT – 31.07.2013
Page 79
Target User(s)
References
Assessment
Adopted
Standard(s)
Capabilities
/Opportunities
End users, software developers
MyCocktail platform has been developed in the European ICT Romulus Project. It
has been used by other European research projects such as ICT Omelette project
(http://www.ict-omelette.eu)
Open Social Specifications, WADL, Rest
The MyCocktail web application has different features that could be interesting in
the ClouT contest: in particular the possibility to import and use REST services to
create mashup could be useful in ClouT considering that many of the services
provided in the CPaaS layer can be provided as REST. Also the functionality to
export gadgets in other platforms that support open social specifications enlarge
the possibility to reuse the mashups in external environments.
The MyCocktail GUI doesn’t provide a clear way for the representation of the mashups, such as arrows that connect the different elements: these could confuse
Known
the final user in particular if he is not a technician. Also the platform doesn’t Limitations/Thre expose API for external systems that would use the server side engine to perform
ats
mashup.
ClouT – 31.07.2013
Page 80
TABLE 7- ENTERPRISE MASHUP MARKUP LANGUAGE
Proposing
Partner
ENG
Anatomy
ID Card
Info
Name
Enterprise MashupMarkup Language (EMML)
Owner(s)
Open Mashup Alliance (OMA) (http://www.openmashup.org/)
EMML reference runtime implementation:
http://www.openmashup.org/download/
Link to code
Project, plus link
Open Mashup Alliance: http://www.openmashup.org
exploitation
Creative Commons License of Attribution No Derivatives:
(licenses)
http://creativecommons.org/licenses/by-nd/3.0/
EMML (Enterprise MashupMarkup Language) is an XML markup
language for creating enterprise mashups promoted by the Open
Mashup Alliance. EMML is a declarative mashup domain-specific
language (DSL) that eliminates the need for complex, time-consuming,
and repeatable procedural programming logic to create enterprise
Description
mashups.
Type
Programming
language/based
technologies
Domain Specific Language based on XML
Exposed APIs
N/A
Target User(s)
Software developers
EMML is an open language specification that is promoted by the Open
Mashup Alliance that includes several international IT market leaders
such as Adobe, HP and Intel.
The EMML itself is not standard but the objective of the OMA is to
submit the EMML specification to a recognized industry standards
body.
EMML represents one of the few attempts of standardization in the field
of mashup: additionally, the language is sponsored by a strong
consortium that includes big IT international actors. The language
seems to cover most of the possible mashup needs in the ClouT contest
considering that support heterogeneous services and data formats. The
possible adoption in ClouT will be investigated in the next project
phases.
References
Assessment
Adopted
Standard(s)
Capabilities
/Opportunities
XML
The main limitation that can be identified at this stage is related to the
licence of EMML: the Creative Commons License of Attribution No
Derivatives implies that the language can be freely reused but it cannot
Known
Limitations/Threats be modified to create for example an extended version that could be
more suitable for the ClouT requirements.
ClouT – 31.07.2013
Page 81
TABLE 8- OPENSOCIAL
Proposing
Partner
ENG
Anatomy
ID Card
Info
Name
OpenSocial
Owner(s)
OpenSocial Foundation
Link to code
http://docs.opensocial.org/display/OSD/Specs
Project, plus link
OpenSocial Foundation: http://opensocial.org
exploitation
(licenses)
Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)
OpenSocial is a set of common APIs (Application Programming
Interface), based on HTML and JavaScript, for social software
applications that allow to access data and core functions on
Description
participating social networks.
Type
Programming
language/based
technologies
Exposed APIs
Target User(s)
References
Assessment
Adopted
Standard(s)
Capabilities
/Opportunities
Known
Limitations/Threats
ClouT – 31.07.2013
Web application framework
Java, PHP, C#, Javascript, HTML
OpenSocial defines a set of common application programming
interfaces for web-based applications.
Software Developers
Initially developed by Google it has been supported and implemented
by several Platforms and social networks. In particular different
enterprise social platforms are compliant with OpenSocial
specifications: some example are Liferay, Jive, and SugarCRM
OpenSocial is itself an open standard
OpenSocial is a mature standard that has been adopted by several
enterprise platforms in the last years. The gadget approach that can be
implemented through its specification is in line with the presentation
layer of the Mashup editor that is planned to develop in ClouT.
No particular limitation has been identified at this stage
Page 82
TABLE 9- YAHOO! PIPES
Proposing
Partner
ENG
Assessment
Anatomy
ID Card
Info
Name
Yahoo! Pipes
Owner(s)
Yahoo!
Link to code
N/A
Project, plus link
http://pipes.yahoo.com/pipes/
exploitation
Proprietary License:
(licenses)
http://info.yahoo.com/legal/us/yahoo/pipes/pipes-4396.html
Yahoo! Pipes is a visual programming environment that gives people
the ability to create web mashups and web-based applications that
combine data from multiple sources.
The core component of Yahoo! Pipes is the Pipe editor that is composed
of three panes which are the canvas, the library and the debugger.
Description
The output pipes can be in different formats: widgets, RSS, JSON, PHP.
Type
Programming
language/based
technologies
Web Application
Exposed APIs
N/A
Target User(s)
End Users
References
Yahoo! Pipes can be considered a mature service being supported since
2007 by a big IT player as Yahoo.
Adopted
Standard(s)
Capabilities
/Opportunities
RSS, JSON
Yahoo pipes is stable and widely used platform. The user interaction
approach, particularly easy, used in its editor could be replicated in the
ClouT mashup editor.
Yahoo pipes is not an open source platform so will be not possible to
reuse components in the ClouT project. Also the Yahoo pipes cannot be
considered a complete Mashup editor but a limited data mashup tool
Known
Limitations/Threats that is not designed to perform service mashup to create complex
applications.
ClouT – 31.07.2013
Page 83
TABLE 10- APACHE SHINDIG
Proposing
Partner
ENG
Anatomy
ID Card
Info
Name
Apache Shindig
Owner(s)
Apache Software Foundation
Link to code
https://svn.apache.org/repos/asf/shindig/trunk/
Project, plus link
Apache Shindig, http://shindig.apache.org
exploitation
(licenses)
Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)
Shindig is an open source project of the Apache Software Foundation
that has the aim to implement an open container in compliance with the
Open Social gadgets specifications.
There are currently two versions of Apache Shindig written in Java and
Description
PHP, and a third, written in. NET that is external to the project.
Type
Programming
language/based
technologies
Application framework
Exposed APIs
OpenSocial REST API
Target User(s)
Software Developers
Apache Shindig is used in several OpenSocial sites and projects such as:
Apache Rave (http://rave.apache.org/), LinkedIn
(http://www.linkedin.com/), hi5 (http://www.hi5.com/), Yahoo!
(http://apps.yahoo.com/ ), Orkut (http://www.orkut.com/), Zing
(http://code.google.com/p/zing/)
References
Assessment
Adopted
Standard(s)
Capabilities
/Opportunities
Known
Limitations/Threats
ClouT – 31.07.2013
JavaScript, PHP and Java
OpenSocial specifications
Apache Shindig is one of the most stable and used reference
implementation of the OpenSocial specification. Its implementation is
available in both java and php language giving the possibility to choose
the more suitable language for the ClouT needs: the reuse of this
component will be highly taken in consideration.
No particular limitation has been identified at this stage.
Page 84
TABLE 11- IPOJO
Proposing
Partner
CEA
Info
iPojo
ID Card
Name
Owner(s)
Apache Software Foundation
Link to code
http://felix.apache.org/site/apache-felix-ipojo.html
Project, plus link
exploitation (licenses)
http://felix.apache.org/documentation/subprojects/apache-felix-ipojo.html
http://www.apache.org/licenses/
iPOJO(Plain Old Java Object) is a service component runtime aiming to simplify
OSGi application development and to build service-oriented component model.
Anatomy
Description
Type
Programming
language/based
technologies
Service component model
Java
Exposed APIs
Service/application developers
Target User(s)
References
http://felix.apache.org/documentation/subprojects/apachefelix-ipojo/apache-felix-ipojo-why-choose-ipojo.html
OSGi
Assessment
Adopted Standard(s)
Capabilities
/Opportunities
iPOJO simplifies development of PSGi services and adds features, including
component configuration, synchronization, and composition. It designed to be
small: the core size of iPOJO is approximately 205k.
It requires learning of its component approach
Known
Limitations/Threats
ClouT – 31.07.2013
Page 85
TABLE 12- BPMN
Proposing
Partner
Name
ID Card
Owner(s)
Link to code
Project, plus link
Anatomy
Description
Type
Programming
language/based
technologies
Exposed APIs
Target User(s)
References
CEA
Info
BPMN
OMG (Microsoft, IBM, etc.)
Among others, an open source implementation:
http://activiti.org/download.html
http://www.bpmn.org/
Specifications are open. Open source and commercial implementations exist
(e.g., Activiti is open source and cross platform,
http://activiti.org/download.html)
The Business Process Modeling Notation (BPMN) is a graphical notation that
depicts the steps in a business process. A business process spans multiple
participants and coordination can be complex. Moreover, well-supported
standard modeling notation will reduce confusion among business and IT endusers.
Graphical process modeling
Language agnostic. Activiti written in Java
Deploying the process, Starting a process instance, completing tasks,
suspending and activating a process, query API, exception management, unit
testing, etc.
Application development tool provider
-
Adopted Standard(s)
UML compatible
Assessment
Can be easily serialized into XML
Capabilities
/Opportunities
Provides a complete specification. Facilitate integration of IoT processes into
existing business processes.
Known
Limitations/Threats
Can be too complex for simple IoT applications
ClouT – 31.07.2013
Page 86
TABLE 13- BPEL
Proposing
Partner
Name
ID Card
Owner(s)
Link to code
Project, plus link
CEA
Info
BPEL
OASIS
http://www.oracle.com/technetwork/middleware/bpel/overview/index.html
https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsbpel
OASIS standard, open specification, open source and commercial
Anatomy
implementations exist
BPEL (BPEL s.d.)is the standard for assembling a set of discrete services into an
end-to-end process flow, radically reducing the cost and complexity of process
integration initiatives.
Description
Type
Programming
language/based
technologies
Process Execution Language
Implementations in Java and .NET exist.
XML for data exchange, WSDL for service description, XPath for manipulating
XML data
A complete API from Oracle implementation can be found at:
http://docs.oracle.com/cd/E11036_01/integrate.1013/b28986/toc.htm
Exposed APIs
Target User(s)
Application development tool provider
References
Assessment
Adopted Standard(s)
Capabilities
/Opportunities
Known
Limitations/Threats
ClouT – 31.07.2013
Related to BPMN
Mature technology with many implementations:
http://en.wikipedia.org/wiki/Comparison_of_BPEL_engines
Can be too complex for simple IoT applications
Page 87
TABLE 14 – ROBUST SERVICE COMPOSITION METHOD
Proposing
Partner
NII
Anatomy
ID Card
Info
Name
Robust Service Composition Method
Owner(s)
NII
Link to code
N/A
QAS, http://grace-center.jp/research/research_projects/prj_perqashtml?lang=en
Project, plus link
exploitation
(licenses)
N/A
This method provides foundations for service composition to ensure
validity of functionality as well as optimization and constraint
satisfaction in quality. The method also takes into consideration
possibilities of failures and slight differences in functions of similar
Description
services.
Type
Programming
language/based
technologies
Method / Tool Specification and Proof-of-Concept Implementation
Exposed APIs
N/A
Target User(s)
- Developers and users who design service compositions
- System components that trigger automated composition
None
References
Adopted
Standard(s)
N/A
Effective in situations with either of the following characteristics:
Assessment
- Service functions of the same kind are actually different in interfaces
or how general inputs/outputs are required/produced
- Services are somewhat fragile and tend to change/stop in operation or
quality, due to technical aspects or dependencies on third-party
providers
- Compositions require complex workflow with careful consideration of
pre/post-conditions of each service
Capabilities
/Opportunities
- The space of possible service combinations are very large to
understand and make decisions manually
Needs input of ontology-based description of service functions and
Known
quality, by providers/mediators of involved services or developers of
Limitations/Threats service compositions
ClouT – 31.07.2013
Page 88
TABLE 15 – QOS-BASED SERVICE/CLOUD SELECTION METHOD
Proposing
Partner
NII
Anatomy
ID Card
Info
Name
QoS-based Service/Cloud Selection Method
Owner(s)
NII
Link to code
N/A
Project, plus link
exploitation
(licenses)
N/A
This method provides an algorithm for QoS-based selection of services
or clouds to use, considering network QoS in addition to service
Description
(computation) QoS.
Type
Programming
language/based
technologies
Exposed APIs
N/A
- Developers and users who select service providers/plans to use and
cloud providers/plans to deploy components service compositions
- System components that trigger automated selection of services and
clouds
Target User(s)
None
References
Adopted
Standard(s)
N/A
- There are many choices of service providers and plans to use, or cloud
providers and plans to deploy components
Assessment
- There are many quality criteria (price, availability, response time, etc.)
to consider upon selection of providers and plans
- There are end-to-end constraints (constraints on the whole
composition, such as total price) on the quality criteria and selection of
the whole combinations are difficult to satisfy the constraints
Capabilities
/Opportunities
- Attractive providers beyond continental boundaries are considered
(e.g., US cloud provider from Japan) and network latency is not neglect
able
Known
Needs concrete model of quality of services and its values for each
Limitations/Threats service
ClouT – 31.07.2013
Page 89
TABLE 16 – METADATA-BASED BEHAVIOR INSERTION FRMAEWORK
Proposing
Partner
NII
Anatomy
ID Card
Info
Name
Metadata-based Behavior Insertion Framework
Owner(s)
NII
Link to code
N/A
Project, plus link
exploitation
(licenses)
N/A
This framework provides a way to insert additional behaviors into
service compositions to achieve value-added functions often for nonfunctional requirements. It defines an algorithm to properly insert
additional behaviors according to constraints or rules, which are
Description
efficiently described by abstract metadata.
Type
Programming
language/based
technologies
Exposed APIs
N/A
Target User(s)
- Developers and users who design service compositions
- System components that trigger automated composition
None
References
Adopted
Standard(s)
WS-BPEL (Web Service Business Execution Language, OASIS,
https://www.oasisopen.org/committees/tc_home.php?wg_abbrev=wsbpel)
WS-CDL (Web Service Choreography Language, W3C,
http://www.w3.org/TR/ws-cdl-10/)
Assessment
- Non-functional requirements are achieved by inserting additional
behaviors such as logging, authentication, payment, etc.
- These behaviors are scattered in various points of execution, inside a
service composition or in multiple service compositions
Capabilities
/Opportunities
- Insertion of such behaviors is switched on or off, depending on the
environments or preferences
Known
Careful analysis and definition are necessary to define metadata,
Limitations/Threats especially to be efficiently deal with requirements that appear later
ClouT – 31.07.2013
Page 90
TABLE 17 – BPMN VERIFICATION
Anatomy
ID Card
Proposing
Partner
NII
Name
Info
Verification Framework of Time and Resource Constraints on Business
Process
Owner(s)
NII/JAIST
Link to code
N/A
Project, plus link
exploitation
(licenses)
N/A
This framework provides a capability to verify time and resource
constraints on service compositions or business processes through
model checking. It provides a small annotation syntax on BPMN
(Business Process Modeling Language) for time and resource
Description
constraints.
Type
Programming
language/based
technologies
Exposed APIs
N/A
Target User(s)
- Developers who design or verify service compositions
UPPAAL Model Checker
References
Adopted
Standard(s)
BPMN (Business Process Modeling Language 2.0, OMG,
http://www.omg.org/spec/BPMN/2.0/)
Assessment
- Services are relied by non-sharable resources including human
experts/crowds, thus resource constraints should be considered
- Time constraints such as deadlines are significant
Capabilities
/Opportunities
- State transitions are too complex to have manual analysis such as
worst case execution
Known
Some abstraction or simplification may be necessary to avoid state
Limitations/Threats explosion
ClouT – 31.07.2013
Page 91
TABLE 18 – ECA VERIFICATION
Proposing
Partner
NII
Anatomy
ID Card
Info
Name
Verification Framework of ECA Specification on Physical Interactions
Owner(s)
NII
Link to code
N/A
Project, plus link
exploitation
(licenses)
N/A
This framework provides a capability to verify complex combinations of
ECA (Event-Condition-Action) rules that have effects on shared spaces
Description
often by multiple users.
Type
Programming
language/based
technologies
Exposed APIs
N/A
Target User(s)
- Developers who design or verify service compositions
SPIN Model Checker
References
Adopted
Standard(s)
N/A
Assessment
- Services with complex preconditions and post conditions are
considered, especially physical effects on shared spaces
Capabilities
/Opportunities
- Post conditions (effects) of services work on shared spaces such as the
real places, thus requires very careful consideration of feature
interactions or undesirable situations caused by activations of multiple
services, often by multiple users
Known
Requires some knowledge on model checking and temporal logic to use
Limitations/Threats beyond the default verification items
ClouT – 31.07.2013
Page 92
7.2.
chapter 2.
TABLE 19- ESPER
Proposing
Partner
Name
ID Card
Owner(s)
Link to code
Anatomy
Project, plus link
CEA
Info
ESPER
Inc, Esper Team and EsperTech
http://esper.codehaus.org/
http://esper.codehaus.org/
GNU General Public License (GPL) (GPL v2)
The Esper engine (Inc 2012) has been developed to address the requirements
of applications that analyze and react to events. The Esper engine works a bit
like a database turned upside-down. Instead of storing the data and running
queries against stored data, the Esper engine allows applications to store
queries and run the data through. Response from the Esper engine is real-time
when conditions occur that match queries. The execution model is thus
continuous rather than only when a query is submitted.
Description
Software for complex event processing
Type
Programming
language/based
Implementations exist in Java and .NET
technologies
Reference implementation API can be found at :
Exposed APIs
Target User(s)
References
Assessment
Adopted Standard(s)
Capabilities
/Opportunities
Known
Limitations/Threats
ClouT – 31.07.2013
http://esper.codehaus.org/esper-4.6.0/doc/api/index.html
Data processing service developer, Application developer
One of rare open source complex event detection engines. It is complete and
reusable.
Compared with commercial solutions, Esper is still missing:
(http://blog.octo.com/en/the-esper-cep-ecosystem/)

more user-friendly interfaces – that is, a graphical language as an
alternative to EPL authoring

enhanced simulation and back testing features – they are expected in
the upcoming 5.0 version, with the announced event capture and
replay features

true HA functionalities – native load balancing and cluster
management – which still require custom development and
architectural design, including careful integration with an external
messaging system between checkpoints
Page 93
TABLE 20- FI-WARE GATEWAY DATA HANDLING
Proposing
Partner
CEA
ID Card
Info
Name
FI-WARE Gateway Data handling
Owner(s)
FI-WARE Project
http://catalogue.fi-ware.eu/enablers/gateway-data-handling-geesper4fastdata
Link to code
Project, plus link
Anatomy
Description
Type
Programming
language/based
technologies
Exposed APIs
Target User(s)
References
Assessment
Adopted Standard(s)
Capabilities
/Opportunities
Known
Limitations/Threats
ClouT – 31.07.2013
http://catalogue.fi-ware.eu/enablers/gateway-data-handling-gesolcep
http://catalogue.fi-ware.eu/enablers/terms-and-conditions-8
FI-Ware Gateway data handling is a complex event processing component
proposed by Orange Labs France. It is based on Esper4FastData CEP engine. It
is able to collect vast amounts of asynchronous events of different types and
correlate them into single events, called Complex Events. It can read from and
write to numerous different channels using various different protocols. It is
driven using a domain specific language called “Dolce”.
Software component
Implemented in Java (OSGi). A version for Android is also available
The Data Handling API is NGSI compliant.
Gateway Data Handling GE is fully integrated with the other enablers of FIWare, especially using the Open Mobile Alliance (OMA) Next Generation Service
Interface Context Enabler (NGSI 9 / NGSI 10)
Easy integration into OSGi environments. Restful management API needs only a
servlet container, without depending on third party framework.
Restricted access to the FI-WARE enablers. Difficulties to test and
evaluate. IPR issues can arise.
Page 94
TABLE 21- JBOSS DROOLS EXPERT
Proposing
Partner
ID Card
Name
CEA
Info
Jboss Drools Expert
Owner(s)
JBoss, RedHat
Link to code
http://www.jboss.org/drools/drools-expert.html
Project, plus link
http://www.jboss.org/drools/drools-expert


Anatomy
Description
Type
Programming
language/based
technologies
Exposed APIs
Target User(s)
References
EJ-Technologies provide licenses for JProfiler for free, for the JBoss
Drools project.
JetBrains has donated an IntelliJ license for use on open source Drools
projects
Drools Expert is a declarative, rule based, coding environment. This allows you
to focus on "what you want to do", and not the "how to do".
Rule description language
Java
Complete API on drools http://docs.jboss.org/jbpm/v5.1/javadocs/
-
Assessment
Adopted Standard(s)
Capabilities
/Opportunities
Can be integrated to BPMN,
Already existing implementations with BPMN.
-
Known
Limitations/Threats
ClouT – 31.07.2013
Page 95
TABLE 22- MONGODB
ID Card
Proposing
Partner
Name
Info
MongoDB
Owner(s)
http://www.10gen.com/
Link to code
Project, plus link
Description
Anatomy
CEA
Type
Programming
language/based
technologies
Exposed APIs
http://www.mongodb.org/
http://www.mongodb.org/
Open-source
MongoDB is an open-source document database, and the leading NoSQL
database. Data in MongoDB has a flexible schema. Collections do not enforce
document structure. Although, you may be able to use different structures for a
single data set.
Database
MongoDB distribution platform: C++, OS: Windows, Linux, OS X, Solaris
Drivers and clients: C, C++, Java, Python, etc.
http://api.mongodb.org/
Target User(s)
References
-
Adopted Standard(s)
MongoDB stores a binary form of JSON documents (BSON)
Assessment
MongoDB is an agile NoSQL database
Capabilities
/Opportunities
Auto-sharing allows MongoDB to scale from single server deployments to large,
complex multi-data center architectures.
Free cloud-based service for monitoring MongoDB deployments
Specific to documents
Known
Limitations/Threats
ClouT – 31.07.2013
Page 96
7.3.
chapter 2.
TABLE 23- REST/SOAP API’S FOR IDAS ACCESS
Proposing
Partner
Name
Owner(s)
Anatomy
ID Card
Link to code
Project, plus link
UC
Info
REST/SOAP APIs for IDAS access
https://m2m.telefonica.com/
http://forge.fi-ware.eu/plugins/mediawiki/wiki/fiware/index.php/IDAS
http://forge.fi-ware.eu/plugins/mediawiki/wiki/fiware/index.php/IDAS
Closed source. The IDAS platform has been internally developed by Telefónica
based on previously developed in-house technology. It is based on a generic
architecture for integrating heterogeneous and disperse M2M devices, sensor &
actuator technologies. TID is the owner of the platform. It will be available for
being used in the Core Platform through its API. The terms of use of the IDAS
exploitation (licenses) would need to be further discussed and agreed.
IDAS has been conceived as an end-to-end open platform intended to be used in
a broad range of Internet-of-Things application scenarios and services.
Together with providing a basic support to IoT service providers, different
types of service users are envisaged, varying from ‘individual end-users’ to ‘industrial users’, automating the acquisition and management of the information retrieved from generic wireless sensor and actuator networks.
Description
Database
Type
Programming
language/based
technologies
The IDAS platform has been built on Linux Redhat 5.5.
REST and SOAP web service interfaces
Exposed APIs
Service provider, Application developer
Target User(s)
References
Assessment
Adopted Standard(s)
Capabilities
/Opportunities
SensorML, O&M, SOS,SWE (Sensor Web Enablement), SIP communication
protocol, Open Mobile Alliance (OMA) Service Environment (OSE)
Known
Limitations/Threats
ClouT – 31.07.2013
Page 97
TABLE 24- TESTBED MANAGER
ID Card
Proposing
Partner
Name
Info
Testbed Manager
Owner(s)
http://www.smartsantander.eu/
Link to code
Project, plus link
Description
Anatomy
UC
Type
Programming
language/based
technologies
http://www.smartsantander.eu/wiki/
http://www.smartsantander.eu
To be determined during the project.
Testbed Manager module provides efficient procedures and mechanisms for
managing dense IoT deployments in a remote and efficient way.
Management
Testbed manager is developed in Java.
REST web service web-based client called wisegui and written in JavaScript
Exposed APIs
Researcher/Experimenter, Network manager
Target User(s)
References
-Wise bed project. http://wisebed.eu/site/
Assessment
Adopted Standard(s)
Capabilities
/Opportunities
Known
Limitations/Threats
ClouT – 31.07.2013
Page 98
TABLE 25 D-CASE MONITORING TOOL
Proposing
Partner
KEIO
Info
ID Card
Name
Owner(s)
Link to code
Project, plus link
D-Case Monitoring Tool
D-Case Consortium / Keio University
http://www.il.is.s.u-tokyo.ac.jp/deos/dcase/
http://www.dependable-os.net/osddeos/index-e.html
Copyright Fuji Xerox Co., Ltd. 2011-2012
exploitation (licenses) (http://www.il.is.s.u-tokyo.ac.jp/deos/dcase/Download.html)
An Eclipse-based graphical tool that uses dependability case, which is a kind of
assurance cases, to show a system is ready to avoid/address its potential faults
(risks). In order to provide tools that make city data dependable, i.e, available,
reliable, consistent, for applications accessing city data, D-Case can be used to
ensure dependability of communications between the applications themselves
and some running systems that disseminate the city data to the network.
Anatomy
Description
Type
Programming
language/based
technologies
Exposed APIs
D-Case achieves this by monitoring applications, running systems, and
networks between them.
Software
Java, Goal Structuring Notation (GSN)
N/A
Developers of ClouT applications
Target User(s)
References
Assessment
Adopted Standard(s)
Managers of ClouT applications and the platform
Matsuno, Y.; Nakazawa, J.; Takeyama, M.; Sugaya, M.; Ishikawa, Y., "Towards a
Language for Communication among Stakeholders," Dependable Computing
(PRDC), 2010 IEEE 16th Pacific Rim International Symposium on , vol., no.,
pp.93,100, 13-15 Dec. 2010
Goal Structuring Notation
Capabilities
/Opportunities
Capable of (1) authoring assurance cases to specify strategies and evidences to
achieve dependability in arbitrary systems, and (2) executing assurance cases
to collect evidences from running systems.
Known
Limitations/Threats
Current implementation does not allow us to author a complex D-Case
documents, such as those including inter-document relationship, combinations
of multiple conditions, etc.
ClouT – 31.07.2013
Page 99
TABLE 26 – SELF-HEALING FRAMEWORK FOR SENSORY DATA
Proposing
Partner
NII
Assessment
Anatomy
ID Card
Info
Name
Self-Healing Framework for Sensory Data
Owner(s)
Link to code
N/A
N/A
Project, plus link
exploitation
(licenses)
N/A
This framework provides runtime detection, classification, and
correction capabilities for sensory data faults that appear in sensory
data. It includes a fault classification model and a model-learning
Description
component based on statistical pattern matching.
Conceptual Framework/Tool Specification and Proof-of-Concept
Type
Implementation
Programming
language/based
technologies
N/A
Exposed APIs
N/A
Target User(s)
Developers who develop self-healing components in CPaaS
References
[SHF][SHF]
Adopted
Standard(s)
Capabilities
/Opportunities
No/A
・This framework provides detection, classification, and correction
capabilities for sensory data.
・This framework targets densely deployed sensor networks. It will not
Known
work well in sparsely deployed sensor networks, because it adopts fault
Limitations/Threats detection based on neighborhood voting.
ClouT – 31.07.2013
Page 100
TABLE 27 – FAULT CLASSIFICATION MODEL
Proposing
Partner
NII
Assessment
Anatomy
ID Card
Info
Name
FaultClassification Model
Owner(s)
Link to code
N/A
N/A
Project, plus link
exploitation
(licenses)
N/A
This model provides a decision tree to classify data fault sin to four
types of data fault: bias fault, drift fault, malfunction fault, and random
fault. Applied to sensory readings, this model proposes a complete and
consistent classification based on frequency and the continuity of
Description
occurrence and observable and learnable pattern.
Type
Programming
language/based
technologies
Model
Exposed APIs
N/A
Target User(s)
- Self-healing framework for sensory data
References
[CM]
Adopted
Standard(s)
Capabilities
/Opportunities
N/A
N/A
It provides a decision tree for sensory fault classification
Known
This model is only for sensory data faults from physical sensors, not for
Limitations/Threats ones from virtual sensors
ClouT – 31.07.2013
Page 101
TABLE 28– HYPERTABLE
Proposing Partner
ENG
ID Card
Info
Name
Hypertable
Owner(s)
Hypertable Inc.
Link to code
http://hypertable.com/download/0977
Project, plus link
Hypertable: http://hypertable.com/
Patents or other IPR exploitation
Open source licence: GPL Version 3
(licenses)
Description
Hypertable is a NoSQL database based on Google BigTable
technology. It is a high scalable and non-relational database
implementation. It is written in C++ language and permits to have
high performance thanks to the different distributed file systems
that can be attached to it (Hadoop HDFS or Ceph).
Type
Software
Programming language/based
technologies
C++
Assessment
Anatomy
Drivers for the following languages:
Exposed APIs
Java, PHP, Python, Perl, Ruby, C++
Target User(s)
Data Access Services of ClouT CPaaS
References
 Belvedere Trading FINANCIAL
 eBay INTERNET SEARCH
 Rediff.com INTERNET EMAIL
 Stratoshear MOBILE ADVERTISING
 Tiscali S.p.A. TELECOMMUNICATIONS
 Wirth Research AUTOMOTIVE ENGINEERING
Below a link to an exhaustive list of Hypertable implementations
and partners: http://hypertable.com/customers/
Adopted Standard(s)
Google File System (GFS), Hadoop MapReduce , Google Bigtable ,
Sawzall
Capabilities /Opportunities
Very high performance if used on top of Hadoop HDFS or Ceph
storage data. It provides the data versioning (which for example is
not a feature of object storage such as CEPH) based on time stamp.
It could be used to archive small data such as single sensor
measurements and as search engine for this kind of data. Well
suited for analyzing log data.
Not completely SQL support. Only a subset of SQL language is
implemented.
No security schemes are implemented on top of the system.
Security is demanded to the lower layer (File system or object
storage).
Known Limitations/Threats
ClouT – 31.07.2013
As to implement the security access it is necessary to implement a
system on top of Hypertable.
Page 102
TABLE 29 – HBASE
Proposing Partner
ENG
ID Card
Info
Name
Hadoop HBase
Owner(s)
Apache Foundation
Link to code
http://svn.apache.org/viewvc/hbase/trunk
Project, plus link
Apache HBase: http://hbase.apache.org/
Apache Software License, Version 2.0
http://hbase.apache.org/license.html
Anatomy
Hbase is a NoSQL database implementation, part of Apache Hadoop
project. It is a distributed database modeled on Google BigTable
technology and written in Java language. This database permits to
manage, in random access, billions of data. The goal of the project is to
have very large scale tables (billions of rows X millions of columns).
Apache HBase provides Bigtable capabilities to Hadoop and HDFS.
Description
Hbase is not a RDBMS, but it is a column oriented database, of course it
doesn’t support SQL query. Each table is based on columns and rows, where each row must have a primary key and the access must be
performed using that key.
Type
Software
Programming
language/based
technologies
Java
Exposed APIs
RESTful API’s
Target User(s)
Systems that interacts with RESTful APIs
References
 Adobe;
 Facebook;
 Twitter;
Below a link to an exhaustive list of HBase implementations:
http://wiki.apache.org/hadoop/Hbase/PoweredBy
Adopted Standard(s)
RESTful API’s, HDFS, MapReduce paradigm
Assessment
High performance and data throughput. Hbase is designed to be used in
case of a big number of data accessible in real time. It is possible, as
Adobe implemented in his system using cheapest hardware, to access
data in 50ms.
Hadoop MapReduce is implemented as native in Hbase. This means
that the elaborations are distributed into the servers. The MapReduce
permits to increase performance simply adding hardware, clusters.
Designed for large scale data analysis. Not suited for real time data
analysis.
ClouT – 31.07.2013
The columns are not sotred (It does not exist the concept of sorted
columns).
Page 103
TABLE 30 – HADOOP HDFS
Proposing Partner
ENG
ID Card
Info
Name
Hadoop HDFS
Owner(s)
Apache
Link to code
http://mirror.nohup.it/apache/hadoop/common/hadoop-1.2.0/
Project, plus link
Apache Hadoop: http://hadoop.apache.org/
Apache Software License, Version 2.0
http://projects.apache.org/projects/hadoop.html
The Hadoop Distributed File System (HDFS) is a distributed and
general purpose file system designed to run on commodity
hardware which enables the real time streaming in efficient mode.
It has a very fault tolerant file system architecture and can run on
low cost hardware. It is designed to support file of big dimension: a
typical file stored on HDFS is bigger than gigabytes and can reach
the dimension of terabytes.
Assessment
Anatomy
In a typical corporate installation, HDFS is distributed on hundreds
of thousands machines; this permits to reduce at minimum the
hardware failure. Thanks to Java build, Hadoop HDFS can be ported
on all operating systems.
Description
The architecture of HDFS is based on one node that contains the
Metadata and separated node(s) containing the data files. The data
blacks are also replicated in more than one node for fault tolerance.
Rebalance nodes: in case the free space in one node reach a define
threshold, the data are rebalanced between the nodes.
Type
Software Component
Programming language/based
technologies
Java
Exposed APIs
The Hadoop HDFS API, in PERL, python, Ruby and PHP, are exposed
through Thrift(http://wiki.apache.org/hadoop/HDFS-APIs)
Target User(s)
Software systems that need to storage in safe mode big quantity of
data.
References
 Cloudera;
 IBM;
 Talend;
Below a link to an exhaustive list of Hadoop HDFS implementations:
http://wiki.apache.org/hadoop/PoweredBy
Adopted Standard(s)
http, HTTPS, FTP, POSIX
High performance, extremely scalable and fault tolerant storage. It
could be used alone as back-end storage for Hypertable (a standalone deployment) and as storage layer for Map Reduce framework
for data processing.
ClouT – 31.07.2013
Page 104
Hadoop HDFS is the correct choose if you need high performance in
the file storage scenarios.
In scenarios where it is necessary to storage data in a NoSql
database associated to big data (for example big pictures). Hadoop
HDFS and Hbase are the correct choose because the high
interoperability of the systems. These combination, Hadoop HDFS
and Hbase, permits to create a system with high performance and
high scalability using low cost hardware.
It could be also used as a file system implementation for OpenStack
Swift object store, though this feature is currently under
development9.
HDFS is inefficient for handling small size files. Another limitation
lies in its single-node namespace server architecture. In fact, since
the name-node is a single container of the file system metadata, this
turns in to a problem for file system growth. For example in order
to store 100 million files a name-node should have at least 60GB of
RAM
9https://issues.apache.org/jira/browse/HADOOP-8545
ClouT – 31.07.2013
Page 105
TABLE 31 – OPEN STACK SWIFT
Proposing Partner
ENG
ID Card
Info
Name
Open Stack SWIFT
Owner(s)
OpenStack Foundation
Link to code
https://github.com/openstack/swift
Open Stack Swift: http://www.openstack.org/software/openstack-storage/
Project, plus link
Anatomy
exploitation (licenses) Swift is an open source software release under the Apache 2 license.
Open Stack Swift, a cloud storage platform part of the Open Stack project, is the
most notable Cloud Object Storage platforms as well as….. It allows to store or retrieve objects data using virtual containers. The architecture of Open Stack
Swift is modular and scalable, since it is possible to add and remove the
container nodes dynamically to improve performance. The access to data is
controlled by the proxy server that balances the requests and its security can be
managed either by the Open Stack Identity Service (code-named Keystone) or
Swauth sub system. The data are distributed and crypt with the algorithm
Description
chosen by the end-user (among AES, DES, RSA, end some others).
Type
Programming
language/based
technologies
Software
Exposed APIs
RESTful APIs provided by OpenStack services
Target User(s)
Python
All end user applications and ClouT architectural components that need to use
ClouT storage and are able to interact with RESTful APIs
Several commercial storage services are based on Swift, including: Rackspace10
(from which Swift originated), Coudscaling11, KTucloud Services12 ,HP13,
Internap14.
References
Among companies supporting the OpenStack Foundation15 are Ubuntu, HP,
RedHat, Suse
Adopted Standard(s)
RESTful; HTTP; MD516
The goal of Open Stack Swift is the capability to store peta-byte of data
in a distributed, standard, infrastructure of cluster servers.
Assessment
Open Stack Swift is not a file based storage system, but a very
distributed, with redundancy capabilities, cloud storage infrastructure.
Capabilities
/Opportunities
Thanks to the replication of data (stored and replicated in more than
one server), Open Stack Swift has an high level of secure in case of
failure of one server.
10http://www.rackspace.com/cloud/
11http://www.cloudscaling.com
12https://en.ucloudbiz.olleh.com/portal/ktcloudportal.epc.productintro.ss.info.html
13https://www.hpcloud.com/sites/default/files/Right-storage-for-you.pdf
14http://www.internap.com/agile/flexible-cloud-hosting-solutions/cloud-storage/
15http://www.openstack.org/foundation/companies/
16http://en.wikipedia.org/wiki/MD5
ClouT – 31.07.2013
Page 106
Known
Limitations/Threats
ClouT – 31.07.2013
The object stored must be less than 5 GB, nevertheless there is no limit in data
stream Its model is not fully CDMI complaints it doesn’t support multi-level
container structure and it currently doesn’t provide CDMI APIs, ( as CDMI layer on top of SWIFT, see Object Storage GE - FI-WARE Implementation).
Page 107
TABLE 32 – CEPH
Proposing Partner
ENG
ID Card
Info
Name
CEPH
Owner(s)
Inktank, Inc.
Link to code
https://github.com/ceph/ceph
Project, plus link
CEPH Storage, www.ceph.com
Ceph is provided under LGPL version 2.1 license
Ceph Storage is a free cloud storage designed to present objects of all
types (for example block of data or file of any type) from a single
distributed node cluster. Ceph storage architecture is completely
distributed; this means that there aren’t points of failure with a high level of availability.
Anatomy
CEPH supports up to Exabyte object size. The data replication
mechanism, in the distributed nodes, makes its architecture very
robust and fault tolerant. It is possible to scale the object storage
environments using economic hardware and to replace it easily when
a malfunction occurs.
Description
Ceph system provides to the clients a rich library to access the
storage; the libraries are available in C, C++, Java, Python and PHP.
Type
Software
Programming
language/based
technologies
C++
Exposed APIs
Ceph exposes API in C, C++, Java, Python and PHP languages.
Target User(s)
Software that needs to store big objects.
References
Assessment
Adopted Standard(s)
ClouT – 31.07.2013
 Ceph: Reliable, Scalable, and High-Performance Distributed
Storage (http://ceph.newdream.net/papers/weil-thesis.pdf)
 RADOS: A Fast, Scalable, and Reliable Storage Service for Petabytescale Storage Clusters (http://ceph.newdream.net/papers/weilrados-pdsw07.pdf)
 RESTful;
High scalable storage platform, support Exabyte of data. It could be
used as back-end storage to be attached to CDMI Proxy.
Ceph storage platform can be used as a backend system of Hypertable
database. This means the if there is the need to store big files and
associated metadata, this solution combined for example with a
Cloud DB (for storing metadata) such as Hypertable or HBASE, can be
desirable because permits to have high performance with a low cost
hardware.
Page 108
ClouT – 31.07.2013
The main limitation of CEPH is its single name-node architecture,
tracking where data is stored on the data nodes in the cluster.
Currently, it doesn’t provide CDMI interfaces, however this feature is
in the roadmap.
Page 109
TABLE 33 – OBJECT STORAGE GE - FI-WARE IMPLEMENTATION
Proposing Partner
ENG
Info
ID Card
Name
Object Storage GE - FI-WARE Implementation
Owner(s)
Link to code
https://github.com/osaddon/cdmi
Project, plus link
FP7 EU Research Project FI-WARE. This component is included in the
Fi-WARE Generic Enabler cataloguehttp://catalogue.fi-ware.eu/
The code is under Apache LicenseVersion 2.0.
Anatomy
It is an implementation of the FI-WARE ObjectStorage Specification,
which is a Copyright © 2013 INTEL. Usage terms are specified in FIWARE Open Specifications Interim Legal Notice17.
The implementation of this Generic Enabler is available to FI-PPP
partners according to the terms and conditions specified in the FI-PPP
Collaboration Agreement18
Description
This is an implementation of the FI-WARE Object Storage Specification,
developed in the context of FP7 EU Research Project FI-WARE and
included in the Fi-WARE Generic Enabler (GE) catalogue . This GE
provides an implementation of CDMI interfaces for OpenStack Swift.
Type
Software component
Programming
language/based
technologies
Python
Exposed APIs
RESTful APIs
Target User(s)
All ClouT services, end-user applications and software components at
sensor level that need to store and retrieve data from the Cloud
Assessment
References
Adopted Standard(s)
CDMI
This component allows attaching OpenStack SWIFT to a CDMI layer
(for example the one provided by CDMI Proxy), since this object
storage solution currently doesn’t provide such a standard interfaces for accessing its object and containers.
17https://forge.fi-
ware.eu/plugins/mediawiki/wiki/fiware/index.php/Open_Specifications_Interim_Legal_Notice
18http://www.fi-ppp.eu/guidance-notes-for-proposers/
ClouT – 31.07.2013
Page 110
CDMI allows container being nested, that is, a container can have other
containers inside. CDMI containers and data objects make a tree like
structure.
ClouT – 31.07.2013
Instead, this CDMI implementation, in the current version, does only
support container, data object and capability specification. According
to the OpenStack container and data object model, this implementation
only allows a user to create a container at the top level; once a
container is created, it can only hold data objects inside. This flat model
could represent a limitation in the data organization.
Page 111
TABLE 34 – CDMI PROXY
Proposing Partner
ENG
ID Card
Info
Name
CDMI Proxy
Owner(s)
Ilja Livenson, KTH PDC
Link to code
https://github.com/livenson/vcdm
Project, plus link
FP7 European Research Project VENUS-C, http://resources.venusc.eu/cdmiproxy/docs/intro.html
The terms of use of the software are governed by the Apache 2 license.
Copyright (c) 2011, Ilja Livenson, KTH PDC. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
- Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
- Neither the name of the KTH PDC nor the names of its contributors
may be used to endorse or promote products derived from this
software without specific prior written permission.
Anatomy
CDMI Proxy is an open source implementations of SNIA CDMI
specifications. CDMI (Cloud Data Management Interface) defines a
standard interface to access object (creation, retrieving, updating and
deleting) in cloud storages.
CDMI Proxy, in the last implementation, permits to manage objects in
the following types of cloud storages:


Description

Local disk
AWS S3
Azure Blob
Type
Software component
Programming
language/based
technologies
CDMI-Proxy has been written with Twisted network engine
(http://twistedmatrix.com/trac/wiki/TwistedProject)in Python
(v2.6+) and has been tested on a number of platforms: Linux, Windows
and MacOS X. Required components: CouchDB (v. 1+), PyCrypto
(https://www.dlitz.net/software/pycrypto/), OpenSSL,
Exposed APIs
CDMI-Proxy is a server exposing CDMI-compliant interface (see CDMI
table)
Target User(s)
All ClouT services, end-user applications and software components at
sensor level that need to store and retrieve (large and big amount of)
data from the Cloud.
ClouT – 31.07.2013
Page 112
Assessment
References
Adopted Standard(s)
CDMI
Other storage back-ends could be attached to CDMI Proxy to address
different ClouT storage needs: (a) open source Cloud Object Storage
(see for example Open Stack SWIFT and CEPH)or a Distributed File
System (see for example Hadoop HDFS)for storing big amount of data;
(b) a powerful Cloud noSQL Data Base which could be more suitable for
storing small data (such as single measurement from sensor devices)
that need to be accessed very frequently (for analytics purposes) and
for which a this type of DB offer more search and analysis support (see
for example Hypertable and Hadoop HBase component description)
than an object store allows to do.
Currently neither open-source object storage nor Distributed File
System back-ends are supported. Furthermore, it should be updated in
order to support more recent version of CouchDB (the metadata store).
ClouT – 31.07.2013
Page 113
TABLE 35 – SOA3
Proposing Partner ENG
ID Card
Info
Name
SOA3
Owner(s)
ENG
Link to code
N/A
N/A
Project, plus link
Patents or other IPR exploitation (licenses)
Anatomy
Description
Type
Programming language/based technologies
Web Applications/Java API
Exposed APIs
RESTful APIs
Target User(s)
References
Adopted Standard(s)
Assessment
N/A
A set of services for User Management, Authentication, Authorization, Accounting, Identity Federation and Delegation.
Java /J2EE
End Users (for User Management and Policies definition) and Applications (for Authentication and Authorization queries)
Vision Cloud (www.visioncloud.eu), Imarine (www.imarine.eu)
 RESTful;
 HTTP;
 SAML
 XACML
SOA3 provides a set of services which totally manage Authentication, Authorization and Accounting, supporting CRUD operation on users, roles, groups, and authorization policies. It supports SAML based Identity Federation and Access Delegation as well.
Doesn't support X509 Authentication and ACL based authorization
ClouT – 31.07.2013
Page 114
References
IoT. Strategic Research Agenda. Available online: http://www.internet-of-thingsresearch.eu/pdf/IoT_Cluster_Strategic_Research_Agenda_2011.pdf
Cloud Computing. The future of Cloud Computing, Opportunities for European Cloud
Computing beyond 2010 Online: cordis.europa.eu/fp7/ict/ssai/events-20100126-cloudcomputing_en.html.
World Future Council. Uexküll, Jakob. Shaping our future: Creating the World Future Council.
Foxhole, Devon, United Kingdom. UNT Digital Library.
http://digital.library.unt.edu/ark:/67531/metadc13722/. Accessed November 14, 2012.
Drools-Camel Server. (n.d.). Retrieved from
http://docs.jboss.org/drools/release/6.0.0.Beta3/droolsjbpm-integrationdocs/html_single/index.html
EUROTECH. (n.d.). Retrieved 07 02, 2013, from
http://www.eurotech.com/en/products/software+services/everyware+device+cloud/edc+wha
t+it+is
Fiware Data Handeling. (n.d.). Retrieved 07 05, 2013, from http://catalogue.fiware.eu/enablers/gateway-data-handling-ge-solcep
FI-WARE PUB/SUB. (n.d.). Retrieved 07 03, 2013, from https://forge.fiware.eu/plugins/mediawiki/wiki/fiware/index.php/FIWARE.OpenSpecification.Data.PubSub
Fiware-training Webinars (12 2012).
IBM. (2012). Building Smarter Planet Solutions with MQTT and IBM WebSphere MQ Telemetry.
IBM redbooks.
Esper. Inc, E. T. (2012). Esper Reference. Version 4.7.0.
iPojo. (n.d.). Retrieved 07 03, 2013, from
http://felix.apache.org/documentation/subprojects/apache-felix-ipojo.html
jBPM. (2013). 03: 07.
JSON. (n.d.). Retrieved 07 03, 2013, from http://www.json.org/
JSON ENGINE. (n.d.). Retrieved 07 03, 2013, from http://code.google.com/p/jsonengine/
BPEL. Leymann, P. D. (2010). BPEL vs. BPMN 2.0 should you care? Germany.
M2M.IO. (n.d.). Retrieved 07 02, 2013, from http://help.m2m.io/entries/21577233-connectingto-m2m-io
MongoDB. (n.d.). Retrieved 07 03, 2013, from http://www.mongodb.org/
MongoDB Overview. (n.d.). Retrieved from http://www.10gen.com/products/mongodb
ClouT – 31.07.2013
Page 115
Mosquitto. (n.d.). Retrieved 07 02, 2013, from http://mosquitto.org/
Mosquitto. (n.d.). Retrieved 07 02, 2013, from http://mosquitto.org/
MQTT. (n.d.). Retrieved 07 02, 2013, from http://mqtt.org/
Native JSON. (n.d.). Retrieved 07 03, 2013, from https://developer.mozilla.org/enUS/docs/Using_native_JSON
NGSI API. (n.d.). Retrieved 07 03, 2013, from https://forge.fiware.eu/plugins/mediawiki/wiki/fiware/index.php/Publish/Subscribe_Context_Broker__Context_Awareness_Platform_-_User_and_Programmer_Guide
NGSI10. (n.d.). Retrieved 07 03, 2013, from
http://www.openmobilealliance.org/Technical/release_program/docs/NGSI/V1_0-20101207C/OMA-TS-NGSI_Context_Management-V1_0-20100803-C.pdf
BPEL. (2009). ORACLE BPEL Process Manger. ORACLE DATA SHEET.
QEST. (n.d.). Retrieved 07 02, 2013, from http://qest.me/
Really Small Message Broker. (n.d.). Retrieved 07 02, 2013, from
https://www.ibm.com/developerworks/community/groups/service/html/communityview?co
mmunityUuid=d5bedadd-e46f-4c97-af89-22d65ffee070
RFC4627. (n.d.). Retrieved 07 03, 2013, from http://www.ietf.org/rfc/rfc4627.txt
BPMN. Stephen A. White, I. C. Introduction to BPMN.
UBJSON. (n.d.). Retrieved 07 02, 2013, from http://ubjson.org/
WSBPEL. (n.d.). Retrieved 07 02, 2013, from https://www.oasisopen.org/committees/tc_home.php?wg_abbrev=wsbpel
XIVELY. (n.d.). Retrieved 07 02, 2013, from http://xively.com/
Event-Driven Application Servers, Alexandre Vasseur, T. B. (2007).
MQTTs. Andy Stanford-Clark, H. L. (2007). MQTT For Sensor Networks (MQTTs). Protocol
Specification Version 1.0.
DDS and CEP. Angelo Corsaro, P. (n.d.). Stream processing with DDS and CEP .
BPEL. (n.d.). Retrieved 07 05, 2013, from
ttp://www.oracle.com/technetwork/middleware/bpel/overview/index.html
BPEL. Definition. (n.d.). Retrieved from
http://en.wikipedia.org/wiki/Business_Process_Execution_Language
BPMN. (n.d.). Retrieved 07 02, 2013, from http://www.bpmn.org/
BPMN. (2010). BPMN 2.0 by Example. OMG.
IOT. Butler email Discussion on IOT model.
Context Broker. (n.d.). Retrieved 07 03, 2013, from http://catalogue.fiware.eu/enablers/publishsubscribe-context-broker-context-awareness-platform
ClouT – 31.07.2013
Page 116
IOT-A. (IoT-A (257521)). D1.2 Internet-of-Things Architecture IOT-A. Project Deliverable D1.2 –
Initial Architectural Reference Model for IoT.
Drools Expert. (n.d.). Retrieved 07 03, 2013, from http://www.jboss.org/drools/drools-expert
Drools Expert . (n.d.). Retrieved 07 05, 2013, from
http://docs.jboss.org/drools/release/6.0.0.Beta3/drools-expert-docs/html_single/index.html
DoW. FP7 EU Research Project “Cloud of Things for empowering the citizen clout in smart cities ", Grant agreement no: 608641, Version date: 2013-03-26 - Annex I - "Description of Work"
DoW [D2.1]. FP7 EU Research Project “Cloud of Things for empowering the citizen clout in
smart cities", Grant agreement no: 608641, Version date: 2013-03-26 - Deliverable D2.1 Reusable components, techniques and standards for the CIaaS
ROBUSTCOMP. Florian Wagner, Benjamin Kloepper, Fuyuki Ishikawa, Shinichi Honiden,
Towards Robust Service Compositions in the Context of Functionally Diverse Services, The 21st
International World Wide Web Conference (WWW 2012), pp.969-978, April 2012
QoSSEL. Adrian Klein, Fuyuki Ishikawa, Shinichi Honiden, Towards Network-aware Service
Composition in the Cloud, The 21st International World Wide Web Conference (WWW 2012),
pp.959-968, April 2012
BPMNVeri. Kenji Watahiki, Fuyuki Ishikawa, Kunihiko Hiraishi, Formal Verification of Business
Processes with Temporal and Resource Constraints, The 2011 IEEE International Conference on
Systems, Man, and Cybernetics (IEEE SMC 2011), pp.1173-1180, October 2011
ECAVeri. Fuyuki Ishikawa, Basem Suleiman, Kayoko Yamamoto, Shinichi Honiden, Physical
Interaction in Pervasive Computing: Formal Modeling, Analysis and Verification, The ACM
International Conference on Pervasive Services (ICPS 2009), pp.133-140, July 2009
FUSE. File System in User Space, http://en.wikipedia.org/wiki/Filesystem_in_Userspace
OCCI. Open Cloud Computing Interface,http://occi-wg.org/
S3. Amazon Simple Storage Service, http://aws.amazon.com/documentation/s3/
QoSSEL. Adrian Klein, Fuyuki Ishikawa, Shinichi Honiden, Towards Network-aware Service
Composition in the Cloud, The 21st International World Wide Web Conference (WWW 2012),
pp.959-968, April 2012
INSERT. Ryuichi Takahashi, Fuyuki Ishikawa, Kenji Tei, Yoshiaki Fukazawa, Intention-based
Automated Composition Approach for Coordination Protocol, The IEEE 11th International
Conference on Web Services (ICWS 2013 Application & Experience Track), June 2013 (to
appear)
SHF. Valentina Baljak, Kenji Tei, and Shinichi Honiden, Fault Classification and Model Learning
from Sensory Readings Framework for Fault Tolerance in Wireless Sensor Networks, IEEE
Eighth International Conference on Intelligent Sensors, Sensor Networks and Information
Processing (IEEE ISSNIP 2013), 2013
SFC. Valentina Baljak, Tei Kenji, and Shinichi Honiden, Faults in Sensory Readings: Classification
and Model Learning, Sensors & Transducers Journal, vol.18, pp.177-187, 2013.
ClouT – 31.07.2013
Page 117
LBC. Ehsan Ullah Warriach, Kenji Tei, and Marco Aiello, A Machine Learning Approach for
Identifying and Classifying Faults in Wireless Sensor Networks, the 10th IEEE/IFIP International
Conference on Embedded and Ubiquitous Computing (EUC12), pp. 618-625, 2012
CM. Valentina Baljak, Kenji Tei, and Shinichi Honiden, Classification of Faults in Sensor Readings
with Statistical Pattern Recognition, The Sixth International Conference on Sensor Technologies
and Applications (SENSORCOMM 2012), pp.270-276 , 2012.
ClouT – 31.07.2013
Page 118

D3.1 - ClouT

Transcription

Similar documents

(ZJTABLITE2) Toastabag LITE

Girls talk about menstrual hygiene in Malawi

Using The Mashup Tool to Add Online Media to Tests and Surveys

What Is a Mashup? - Facet Publishing

ap bank fes

Spring 2016 Newsletter

Service-Oriented Architecture for High

Dr. Florian Schreiner, Fraunhofer FOKUS

Master Product Catalogue