schemalogic

Volume 1 | Issue 1
4 May 2016
Under the Bonnet is a
briefing document
Under the Bonnet
Welcome to the first issue of Under the Bonnet, the briefing
document series issued by HATDeX (HAT Data Exchange) to
members of the HAT network. This document series will provide
series issued by
briefings on different aspects of the HAT; its tech, business,
HATDeX to members
markets as well as product updates.
of the HAT network
In this issue:
Page 1 - Delivering the HAT: We explain the technical solution of how we are
rolling out HATs and how organisations can get involved in offering HATs to
their customers
Page 4 - Beta Services coming in July: We explain the two important services
that are coming out with your HAT in July
Page 5 - Sneak Peek: A preview of your HAT profile page
Delivering the HAT
Copyright@ 2016 HAT Data Exchange
Ltd. All rights reserved. No part of this
publication may be reproduced or
utilised in any form or by any electronic,
mechanical, or other means, or stored in
a retrieval system or transmitted, in any
form without prior written permission of
the copyright holders. For more
information contact
[email protected].
As the operator of the HAT ecosystem, we have always maintained that the
HAT, as an open-sourced community-based network, is open to anyone taking
the HAT schema, logic and APIs to become a HAT Platform Provider (HPP)
on their own hardware or cloud infrastructure to build and commercialise for
free. This is available at https://github.com/Hub-of-all-Things/HAT2.0. This is
our primary provisioning model: supporting such organisations to develop and
provide interoperable personal data exchange platforms admitted as HATs to
the HAT ecosystem through HATDeX offering a process of certification to
deliver assurance that the HATs operate in secure, private and trusted
environments.
While organisations can take the HAT database schema, logic and APIs and
become a HPP, we have however, been asked to build the services to deploy
HATs. This follows our successful Indiegogo campaign. Hence, the HATDeX
tech team has been busy working on getting a particular HPP solution for Beta
HATs ready for rollout in July, so providing our secondary provisioning
model.
Well, it took us a bit of time (took us ages to draw this!!!), but we’re happy to
now be able to illustrate what will be coming out with the HAT Beta release
(see diagram below).
With your HAT in a
Container, it will be
lightweight and will
always run on a wide
range of
infrastructures –
meaning it can run
on any computer, on
any infrastructure
and in any Cloud.
The HAT
First of all, the HAT is at the centre of the diagram. This is the dedicated HAT
Database sitting in a Docker Container (https://www.docker.com/what-docker)
that every HAT user will have. Each database contains a data schema,
allowing for the storing of an individual’s data from any source without losing
the structure specific to the source, while at the same time enabling the
individual to relate their data to the context of their personal life and provide a
common semantic structure for third parties to use such data.
We have always emphasised the need for our solution to keep “personal data
personal”. Namely, a user’s HAT data should not be stored in an accountbased system, where each user is a (series of) record in a database hosted by a
service provider, but ideally stored in a (virtual) private server that is
accessible only to the user. Our solution for this is the use of virtual machines
(VM). However, it is very expensive to offer each user a full VM just to run
their HAT and related light services, when a VM is designed to support
various operating systems with much broader application scopes.
Our solution for the HAT rollout is to deploy Containers, and in this case
Docker Containers. While Containers are not fully-scoped servers, they wrap
up a complete system that contains everything it needs to run: code, runtime,
system tools and system libraries. With your HAT in a Container, it will be
lightweight and will always run on a wide range of infrastructures – meaning
it can run on any computer, on any infrastructure and in any Cloud. Each
Container runs a complete and independent HAT, and the only way to
communicate with a HAT is via designated APIs. The Container-based
solution also provides an added layer of protection by localising impact of any
2
The Milliner
orchestrates all
HATs and their
databases, including
creating / deleting
HATs, backing them
up, putting them to
sleep and waking
them up when
needed, as well as
elastically adjusting
the resource
required.
security issues and eliminating the possibility of system administrators
enabling unauthorised access to the data of large numbers of users.
APIs
HAT APIs were developed to exercise user-managed control of personal data.
REST APIs for the HAT are the only mechanism through which a HAT may
be accessed by the web, mobile and other clients that interact with the HAT,
allowing the user to control their data and applications to benefit from it. API
documentation can be found at http://hub-of-all-things.github.io/doc. We
implement End-to-End encryption for HAT data via APIs – encryption of data
at rest, encryption of databases and SSL-encrypted data transport.
Hosting HATs on Amazon Web Services (AWS)
To make it possible for corporations or even advanced users to host HATs
and/or offer HATs to their customers, we will run them on Amazon Web
Services (AWS). Each HAT will run in a Container and will talk to a separate
database server to which no other service has access. Each HAT will
communicate with the outside world via APIs only, and can be accessed only
by its owner/user and authorised applications from the outside world. The
HAT’s database will run on HAT Database Servers, which will also operate as
isolated Containers across a number of Elastic Compute Cloud EC2 instances
for reliability. HAT data is Containerised personal data storage per user for the
proposed solution has never been scaled before. Hence, the HATDeX team
has developed the unique Milliner Service to manage automated deployment,
operations and scaling of Containerised HATs. Learning from the best, we
created the Milliner as a layer on top of Google’s open-source Kubernetes,
leveraging its true freedom to work on various Cloud infrastructures. The
Milliner orchestrates all HATs and their databases, including creating /
deleting HATs, backing them up, putting them to sleep and waking them up
when needed, as well as elastically adjusting the resource required.
HxPP: The adoption of Kubernetes and Docker Containers means that instead
of being a HAT Platform Provider (HPP) provisioning everything themselves
and writing code from scratch around the HAT database, companies can
license the Milliner service from HATDex and be a HATDeX Platform
Provider (HxPP) . They can then host HATs pretty much anywhere Docker
Containers and Kubernetes can run (we have even seen people doing so on a
Raspberry Pi), fitting their needs and on their preferred infrastructure. By
licensing the Milliner Service from HATDeX, HxPP will also have access to
future backend services offered through the Milliner.
HSP: For organisations that do not have the infrastructure capability to
become a HxPP, but with some software capability, we have developed
Milliner APIs to enable them to become HAT Service Providers (HSPs).
This means that HSPs can offer HATs to their customers by using Milliner
APIs on a Platform-as-a-service basis. HSPs can deploy any applications
(web / mobile) to provision, run and manage their customers’ HATs in a
Cloud environment provided by HATDeX (we currently use AWS). Being a
HSP is useful for corporations who wish to reduce personal data storage and
support costs as well as to mitigate reputational risk from privacy and security
management of personal data, outsourcing hardware and software
3
The HAT-onDemand
provisioning model
is useful for
academic projects,
SMEs and
organisations
without any software
or infrastructure
capability for
personal data.
maintenance of HATs to HATDeX. Our HSP provisioning model is useful for
software developers, SMEs and manufacturers of IoT products with their own
software capability around personal data.
HoD: HAT-on-demand (HoD) is a provisioning model for organisations or
individuals with no infrastructure and no software capability. It is a step up
from the HSP provisioning model in that the HAT, as well as the Rumpel
HyperData browser, is provided on a software-as-a-service basis. This
provisioning model is provided to organisations with a full white-labelled
solution. The HoD provisioning model is useful for academic projects, SMEs
and organisations without any software or infrastructure capability for
personal data.
Our HATDeX provisioning models ensure that organisations can be involved
in offering HATs to their customers from a small-scaled pilot to fully-scaled
HAT hosting on the chosen infrastructure. If your organisation is interested to
be involved, please email us at [email protected].
Interacting with your HAT - Beta Services coming
in July
Hyperdata Browser: Rumpel
Rumpel is the world’s first HyperData Browser for HAT users to view their
own personal data. To understand what Rumpel really is, remember that the
full collection of your personal data is very different from the data that you
would give to an organisation with whom you interact; the latter is generally
“siloed” (data of one type such as identity or card numbers, or your email or
photos), whereas the totality of your personal data is a combination of
information from many sources. This data is also dynamic and will include
data from public and private sources, including your own records – diary,
location, weather, pollution at your location, your social media postings and
postings by others, photos, etc. However, we have never seen all of this data in
one place, and therefore cannot appreciate its value to us.
Rumpel fulfils this important function for personal data, making it a truly
personal HyperData Browser with data visualisations and allowing individuals
to organise it into their own bundles and collections. It can be always close to
the individual, being available on both the web and mobile, so Rumpel
benefits the individual, providing personal data in context and on demand.
And in so doing, we would like Rumpel to ‘spin straw to gold’ – to make
individuals realise that their personal data has immense value.
Now that the individual is sitting on a goldmine, s/he must also be able to
exchange some bits of it! A Data exchange app comes within Rumpel,
offering what others wish to trade for your data and allowing marketers to buy
it from you with vouchers, discount codes, freebies or even cash. Even then,
individuals get to control what part of the data they allow access to without a
chance of anyone seeing anything else.
4
MarketSquare – Marketplace for HAT Data and community building
You can also be sociable with your personal data, as we are building the
MarketSquare for you and your data. In the MarketSquare, HAT users can get
together to talk about the best way to share our data; hang out and chat with
people like ourselves; compare the market and see what cool data offers are
out there; compete with friends on who’s earned the most from personal data;
exchange usage data and make collaborative consumption possible (i.e. offer
your vacuum cleaner available to your neighbours when you are not using it);
create guilds and groups or rally your HAT friends to your own social
movement to claim personal data back from corporates!
Sneak Peek into: Your HAT profile page
Your HAT address is where and how you sign in to your HAT. In July, this
would be name.hubofallthings.net. As you can probably see, it looks like a
URL, which it is. So if you actually type name.hubofallthings.net on your web
browser, it will take you to your public-facing HAT page.
For the beta version in July, this HAT page is just a placeholder page with
limited customisability. Users will tick the attributes on their HAT profiles
(accessible on Rumpel) that they are willing to make public, or they can
decide to keep their HAT page fully private. The two images on the left show
you how the front page of a HAT would look like.
HAT users would eventually be able to replace this page with any information
they wish to broadcast (eg a personal website, a blog page, a storefront, a
video channel, art, video or media created by the user etc). The information on
these pages are what users have designated to be public, but they form a part
of their HAT data which means they can be indexable, searchable and even
shared!
Customisation of these pages is in the development plan. If you are a
developer, you can help expedite this by joining the open source HAT
community and help us build on the code.
Comments? Views?
Please contact [email protected] or go to http://forum.hatdex.org/ to discuss further!
5