DDNA-based Embedded Systems - Nova

Transcription

Experience-Oriented Smart
Embedded System
By
Haoxi Zhang
MEng (Software Engineering)
A Thesis submitted in fulfilment of the requirements for the degree of
Doctor of Philosophy
The University of Newcastle
Faculty of Engineering and Built Environment
School of Engineering
Newcastle, Australia
March 2013
STATEMENT OF ORIGINALITY
This thesis contains no material which has been accepted for
an award of any other degree or diploma in any university or
other tertiary institution and, to the best of my knowledge and
belief, contains no material previously published or written by
another person, except where due reference has been made in the
text. I give consent to this copy of my thesis, when deposited in
the University Library**, to be made available for loan and
photocopying subject to the provisions of the Copyright Act 1968.
**Unless an Embargo has been approved for a determined period.
(Signed):__________________________________
I
STATEMENT OF AUTHORSHIP
I hereby certify that components of the work embodied in this
thesis are from published papers of which I am a joint author.
My contribution to these papers warrants inclusion of their parts
in the body of my thesis.
Signed (PhD Candidate):__________________________________
Endorsed (Supervisor):____________________________________
II
ACKNOWLEDGEMENTS
This research could not have been possible without the help
and support of some very important and special people. First and
foremost, I would like to take this opportunity to gratefully
acknowledge my supervisors: Professor Edward Szczerbicki and
Dr Cesar Sanin for patiently taking me through this difficult task
of research. Their expertise and knowledge improved my research
skills and prepared me for future challenges. I am also very
appreciative of the generous assistance, support, care and time
that they provided and devoted to my studies during the past
four years.
Deepest gratitude to the University of Newcastle, School of
Engineering, Faculty of Engineering and Built Environment, the
Office Graduate Studies, and Australian Academy of Science for
all their assistances and support provided in different ways
throughout my studies. Thanks to the Vicomtech Institute in
Spain, for allowing me to experience alternative ways of doing
research, as well as to all of the members of Vicomtech for
providing me a warm environment and pleasant experience.
I am grateful for the friendship of those who brought their own
unique talents and perspectives to my personal and professional
life. Thanks to Dr Carlos Toro, Dr Luis Unzueta, and Dave Jones
for their time, kindness, patience, and advice. Special thanks also
to my fellow PhD students, Leonardo Mancilla Amaya and Peng
Wang for their friendship, and fresh ideas. Thanks to all my other
III
friends and all the great people that became part of my life during
the last few years.
Finally, I want to express my love and gratitude to my family.
Thanks to my parents, because without their unconditional
support and love I could never achieve this. And I wish to thank
my parents in law for their understanding and support. I must
thank my wife, Lulu, for her continued support and motivation.
She kept me sane during this effort and tolerated me using her as
my sporadic but repeated debate partner. Thanks to my son and
daughter for bringing me endless happiness and joy. I love you
all.
IV
TABLE OF CONTENTS
STATEMENT OF ORIGINALITY ..................................................................................................... I
STATEMENT OF AUTHORSHIP ................................................................................................... II
ACKNOWLEDGEMENTS ............................................................................................................... III
TABLE OF CONTENTS.................................................................................................................... V
LIST OF FIGURES ........................................................................................................................ VIII
LIST OF TABLES............................................................................................................................... X
LIST OF ACRONYMS ..................................................................................................................... XI
LIST OF PUBLICATIONS DURING PHD CANDIDATURE .................................................XIV
ABSTRACT .................................................................................................................................... XVII
PREFACE ........................................................................................................................................XIX
1 INTRODUCTION: RESEARCH FOUNDATIONS ....................................................................... 23
1.1 EMBEDDED SYSTEMS............................................................................................................... 24
1.1.1
Application Areas of Embedded Systems................................................................. 24
1.1.2
State-of-the-art Embedded Systems.......................................................................... 27
1.2 KNOWLEDGE AND EXPERIENCE ............................................................................................... 30
1.2.1
Knowledge Management .......................................................................................... 31
1.2.2
Smart Knowledge Management System, SOEKS, and DDNA ................................ 34
1.3 RESEARCH OBJECTIVES ........................................................................................................... 35
1.4 SUMMARY ............................................................................................................................... 37
2 EXPERIENCE-ORIENTED SMART EMBEDDED SYSTEM ..................................................... 38
2.1 OVERVIEW OF EOSES ............................................................................................................. 41
2.2 MAIN FEATURES OF EOSES .................................................................................................... 43
2.3 CONCEPTUAL ARCHITECTURE OF EOSES ............................................................................... 45
2.4 CONCEPTUAL MODELS OF EOSES .......................................................................................... 48
2.4.1
The Server-based Model ........................................................................................... 48
V
2.4.2
The Stand-alone Model ............................................................................................ 51
2.5 SUMMARY ............................................................................................................................... 53
3 EXPERIENTIAL KNOWLEDGE MANAGEMENT IN EOSES .................................................. 54
3.1 DECISIONAL DNA: STANDARD KNOWLEDGE REPRESENTATION IN EOSES ............................ 55
3.2 EXPERIENTIAL KNOWLEDGE MANAGEMENT IN EOSES’S 4-LAYER ARCHITECTURE ............... 57
3.3 CONCEPTUAL ARCHITECTURE OF EOSES IN DETAILS ............................................................. 59
3.3.1
Application Layer ..................................................................................................... 60
3.3.2
Interface Layer .......................................................................................................... 62
3.3.3
Management Layer ................................................................................................... 64
3.3.4
Knowledge Repository Layer ................................................................................... 66
3.4 INITIAL TESTS AND RESULTS ................................................................................................... 67
3.4.1
Experiment Goal and Design .................................................................................... 67
3.4.2
Experimental Results ................................................................................................ 69
3.5 SUMMARY ............................................................................................................................... 71
4 A SERVER-BASED APPLICATION: EOSES APPLIED TO ROBOTICS .................................. 73
4.1 BACKGROUND AND MOTIVATION ............................................................................................ 75
4.1.1
What is a robot and what is robotics? ....................................................................... 75
4.1.2
Experience-based Robot Learning ............................................................................ 76
4.2 EOSES APPLIED TO ROBOTICS ................................................................................................ 78
4.2.1
Application Description ............................................................................................ 78
4.2.2
Robot Hardware ........................................................................................................ 79
4.2.3
System Design and Implementation ......................................................................... 81
4.3 EXPERIMENTS AND RESULTS ................................................................................................... 84
4.3.1
Experiment Goal and Design .................................................................................... 85
4.3.2
Experimental Results ................................................................................................ 87
4.4 SUMMARY ............................................................................................................................... 91
5 A STAND-ALONE APPLICATION: EOSES APPLIED TO DIGITAL TV ................................. 93
5.1 BACKGROUND AND MOTIVATION ............................................................................................ 95
VI
5.1.1
Digital TV ................................................................................................................. 95
5.1.2
Interactive Television ............................................................................................... 98
5.2 EOSES APPLIED TO DIGITAL TV .......................................................................................... 100
5.3 DDNA DTV PROTOTYPE 1.0 ................................................................................................ 103
5.3.1
Application Description .......................................................................................... 103
5.3.2
Simulation and Experiments ................................................................................... 105
5.4.1
5.4.2
System Design ........................................................................................................ 111
5.4.3
Implementation and Experiments ........................................................................... 112
5.5.1
Fuzzy Logic ............................................................................................................ 118
5.5.2
5.5.3
Implementation and Initial Tests ............................................................................ 120
5.6 SUMMARY ............................................................................................................................. 122
6 CONCLUSIONS AND FUTURE WORK .................................................................................... 125
6.1 EXPERIENCE-ORIENTED SMART EMBEDDED SYSTEM (EOSES) ............................................ 126
6.2 ACHIEVEMENTS OVERVIEW ................................................................................................... 127
6.3 FUTURE WORK ...................................................................................................................... 130
ANNEXES ......................................................................................................................................... 132
ANNEX 1.
CLASS DIAGRAM FOR LINE FOLLOWER................................................................. 132
ANNEX 2.
SOEKS CLASS DIAGRAM FOR LINE FOLLOWER ................................................. 133
ANNEX 3.
A SAMPLE SOEKS FOR LINE FOLLOWER ............................................................ 134
ANNEX 4.
CLASS DIAGRAM FOR DDNA DTV 2 .................................................................... 135
ANNEX 5.
API CLASS DIAGRAM FOR DDNA DTV 2 ............................................................ 136
ANNEX 6.
SOEKS CLASS DIAGRAM FOR DDNA DTV 2 ..................................................... 137
ANNEX 7.
A SAMPLE SOEKS FOR DDNA DTV 2 ................................................................ 138
REFERENCES ................................................................................................................................ 140
VII
LIST OF FIGURES
FIGURE 1. OVERVIEW OF THE EXPERIENCE-ORIENTED SMART EMBEDDED SYSTEM ... 42
FIGURE 2. ARCHITECTURE OF THE EXPERIENCE-ORIENTED SMART EMBEDDED SYSTEM
............................................................................................................................ 46
FIGURE 3. CONCEPTUAL SERVER-BASED MODEL FOR EOSES .................................... 49
FIGURE 4. CONCEPTUAL STAND-ALONE MODEL FOR EOSES ..................................... 52
FIGURE 5. RELATION BETWEEN EOSES LAYERS AND KM CONCERNS ....................... 58
FIGURE 6. ARCHITECTURE OF EOSES IN DETAILS ...................................................... 60
FIGURE 7. THE TEST MAP ........................................................................................... 68
FIGURE 8. TIME CONSUMPTION OF THE TEST PROGRAM IN BEST CASES ..................... 70
FIGURE 9. TIME CONSUMPTION OF THE TEST PROGRAM IN WORST CASES ................. 71
FIGURE 10. SYSTEM COMPOSITION OF THE LINE FOLLOWER APPLICATION ................ 80
FIGURE 11. SYSTEM ARCHITECTURE OF THE LINE FOLLOWER APPLICATION .............. 82
FIGURE 12. SIMPLIFIED CLASS DIAGRAM FOR LINE FOLLOWER .................................. 83
FIGURE 13. EXPERIMENT MAPS FOR LINE FOLLOWER................................................. 86
FIGURE 14. TIME CONSUMPTION COMPARISON ON MAP 1 .......................................... 88
FIGURE 15. TIME CONSUMPTION OVERALL COMPARISON ON MAP 1 .......................... 88
FIGURE 16. TIME CONSUMPTION COMPARISON ON MAP 2 .......................................... 89
FIGURE 17. TIME CONSUMPTION OVERALL COMPARISON ON MAP 2 .......................... 89
FIGURE 18. SYSTEM ARCHITECTURE OF DDNA DTV............................................... 101
FIGURE 19. SIMPLIFIED CLASS DIAGRAM FOR DDNA DTV PROTOTYPE 1.0 ............ 104
FIGURE 20. SCREENSHOT OF THE DEFAULT MOVIE SUGGESTION INTERFACE ........... 105
FIGURE 21. A SOEKS OF A WATCHED MOVIE .......................................................... 106
VIII
FIGURE 22. A NEWLY GENERATED MOVIE RECOMMENDATION LIST FOR TOM ........ 109
FIGURE 23. SIMPLIFIED CLASS DIAGRAM FOR DDNA DTV PROTOTYPE 2.0 ............ 112
FIGURE 24. IMPLEMENTATION OF THE DDNA DTV PROTOTYPE 2 ........................... 113
FIGURE 25. THE NEXT-12-HOUR PROGRAMME GUIDE .............................................. 115
FIGURE 26. SCREENSHOT OF THE INTERFACE OF TVHGUIDE WITH A CHANNEL LIST
LOADED ............................................................................................................ 116
FIGURE 27. A PROGRAMME NOTIFICATION SENT BY DDNA DTV WHILE LIVE TV IS
PLAYING ........................................................................................................... 117
FIGURE 28. THE ARCHITECTURE OF A FUZZY INFERENCE SYSTEM ........................... 121
FIGURE 29. RESULTS OF THE FUZZIFICATION TESTS.................................................. 122
IX
LIST
OF
TABLES
TABLE 1. RESPONSE TIME OF LINE FOLLOWER ........................................................... 91
TABLE 2. RESPONSE TIME COMPARISON ..................................................................... 91
TABLE 3. TOM’S MOVIE VIEWING RECORDS ............................................................. 107
X
LIST OF ACRONYMS
ABS
Anti-Braking Systems
AI
Artificial Intelligence
API
Application Programming Interface
ARIB
Association of Radio Industries and Business
ATSC
Advanced Television Systems Committee
DMB
Digital Multimedia Broadcasting
DNA
Deoxyribonucleic Acid
DSL
Digital Subscriber Line
DSS
Decision Support System
DTT
Digital Terrestrial Television
DTV
Digital television
DVB
Digital Video Broadcasting Project
DVB-H
Digital Video Broadcasting - Handheld
EM
Experience Management
EOSES
Experience-Oriented Smart Embedded System
ERPP
Experience-based Randomized Path Planner
ES
Embedded System
ETSI
European Telecommunication Standard Institute
FIS
Fuzzy Inference System
HDTV
High-definition Television
IC
Integrated Circuit
IPTV
Internet Protocol TV
XI
ISDB
Integrated Services Digital Broadcasting Standards
IT
Information Technology
JCTEA
Japan Cable Television Engineering Association
JVM
Java Virtual Machine
KBS
Knowledge-based System
KBES
Knowledge-based Embedded System
KE
Knowledge Engineering
KM
Knowledge Management
KMS
Knowledge Management System
KR
Knowledge Representation
MHP
Multimedia Home Platform
MMDS
Multichannel Multipoint Distribution Service
OWL
Web Ontology Language
POS
Point of Sale
RPP
Random Path Planner
SCTE
Society of Cable Telecommunications Engineers
SDTV
Standard Definition Television
SKMS
Smart Knowledge Management System
SoC
System-on-Chip
SOE
Set of Experience
SOEKS
Set of Experience Knowledge Structure
SOEKS-OWL
Set of Experience Knowledge Structure OWL Configuration
SOEKS-XML
Set of Experience knowledge structure XML Configuration
UAV
Unmanned Aerial Vehicle
W3C
World Wide Web Consortium
XII
WWW
World Wide Web
XML
Extensible Markup Language
XSD
XML Schema Definition
XIII
LIST OF PUBLICATIONS DURING PHD
CANDIDATURE
Journal Publications
1.
Haoxi Zhang, Cesar Sanín, & Edward Szczerbicki (2013): Implementing
Fuzzy Logic to Generate User Profile in Decisional DNA Television: The
Concept and Initial Case, Cybernetics and Systems: An International Journal,
44:2-3, pp. 275-283.
2.
Haoxi Zhang, Cesar Sanín, & Edward Szczerbicki (2012): Making Digital TV
Smarter: Capturing and Reusing Experience in Digital TV, Cybernetics and
Systems: An International Journal, 43:2, pp. 127-135.
3.
Haoxi Zhang, Cesar Sanin, & Edward Szczerbicki (2010): Gaining
Knowledge Through Experience: Developing Decisional DNA Applications
in Robotics, Cybernetics and Systems: An International Journal, 41:8, pp.
628-637.
4.
Haoxi Zhang, Cesar Sanin & Edward Szczerbicki (2010): Decisional DNAbased Embedded Systems: A New Perspective, Systems Science, Vol. 36, pp.
21-26.
5.
Cesar Sanin, Carlos Toro, Haoxi Zhang, Eider Sanchez, Edward Szczerbicki,
Eduardo Carrasco, Peng Wang, & Leonardo Mancilla-Amaya (2012):
Decisional DNA: A Multi-technology Shareable Knowledge Structure for
Decisional Experience, Neurocomputing, 88:0, pp. 42-53.
6.
Cesar Sanin, Leonardo Mancilla-Amaya, Haoxi Zhang, and Edward
Szczerbicki (2012): Decisional DNA: The Concept and Its Implementation
XIV
Platforms, Cybernetics and Systems: An International Journal, 43:2, pp. 6780.
Book Chapters
7.
Haoxi Zhang, Cesar Sanín, & Edward Szczerbicki (2012): Applying fuzzy
logic to Decisional DNA digital TV. L. Borzemski et al. (Eds.): Information
Systems Architecture and Technology, Part II, Oficyna Wydawnicza
Politechniki Wroclawskiej, Wroclaw, pp. 107-115.
8.
Haoxi Zhang, Cesar Sanín, & Edward Szczerbicki (2011): Gaining
Knowledge through Experience in Digital TV. L. Borzemski et al. (Eds.):
Information Systems Architecture and Technology, Part II, Oficyna
Wydawnicza Politechniki Wroclawskiej, Wroclaw, pp. 95–104.
9.
Haoxi Zhang, Cesar Sanín, & Edward Szczerbicki (2010): Testing the
Usability of Decisional DNA in Robotics. L. Borzemski et al. (Eds.):
Information Systems Architecture and Technology, Part II, Oficyna
Wydawnicza Politechniki Wroclawskiej, Wroclaw, pp.177-187.
10. Cesar Sanín, Leonardo Mancilla-Amaya, Haoxi Zhang, and Edward
Szczerbicki
(2010):
Towards
a Software Platform
for Experience
Administration: Decisional DNA Manager. In Information Systems
Architecture and Technology: IT Models In Management Process, Z.
Wilimowska, L. Borzemski, A. Grzech and J. Swiatek (Eds.). Wroclaw:
Oficyna Wydawnicza Politechniki Wrocławskiej, pp. 19-29.
Conference Publications
11. Haoxi Zhang, Cesar Sanín, & Edward Szczerbicki (2012): The Development
of Decisional DNA Digital TV, Proceedings of Knowledge-Based Intelligent
Information and Engineering Systems 16th International Conference KES
XV
2012, in Advances in Knowledge-Based and Intelligent Information and
Engineering Systems: Manuel Grana, Carlos Toro, Jorge Posada, Robert J.
Howlett, Lakhmi C. Jain (Eds.), San Sebastian, Spain, September, 2012, pp.
1500-1508.
12. Haoxi Zhang, Cesar Sanín, & Edward Szczerbicki (2011): Decisional DNA
applied to digital TV, Lecture Notes in Artificial Intelligence Proceedings of
Knowledge-Based Intelligent Information and Engineering Systems 15th
International Conference KES 2011, Robert J. Howlett Lakhmi C. Jain (Eds.),
Germany, September, 2011, Springer-Verlag Berlin Heidelberg, LNAI 6882,
pp. 667-676.
Digital TV: Concept and Initial Experiments, P. Jedrzejowicz at al. (Eds):
LNCS 6922, Proceedings of ICCCI 2011 3rd International Conference on
Computational Collective Intelligence - Technologies and Applications, 2123 September 2011, Gdynia, Poland, Springer-Verlag Berlin Heideberg (2011)
pp. 507-516.
applied to robotics, Lecture Notes in Artificial Intelligence Proceedings of
Knowledge-Based Intelligent Information and Engineering Systems 14th
International Conference KES 2010, Robert J. Howlett Lakhmi C. Jain (Eds.),
UK, September 8-10, 2010, Springer-Verlag Berlin Heidelberg, LNAI6277,
pp. 563-571.
15. Haoxi Zhang, Cesar Sanín, & Edward Szczerbicki (2010): Towards
Decisional DNA-based Cognitive Embedded Systems, The 2nd International
Conference on Computer Engineering and Technology (ICCET 2010), April
16 - 18, 2010, Chengdu, Sichuan, China, pp. 277-281.
XVI
ABSTRACT
Embedded systems have been in use since the 1970’s. For most of their history
embedded systems were seen simply as small computers designed to accomplish one
or a few dedicated functions; and they were usually working under limited resources
i.e. limited memories, limited processing power, and limited energy sources. As
such, embedded systems have not drawn much attention from researchers, especially
from those in the artificial intelligence area.
However, things have changed as today the drive for innovation is stronger than
ever. Thanks to the efforts of scientists over recent years, great progress has been
made in both computer hardware and software, which enables us to have much more
powerful computers in very small sizes and with many more functions.
Consequently, new needs and expectations for Embedded Systems (ESs) have
increased dramatically. The current market demands embedded systems to be built
smart so that they can finish tasks automatically, and assist users to make decisions
more efficiently and effectively. Thus, how to make embedded systems smart is
becoming one of researchers’ new challenges. Knowledge Management (KM) is a
discipline that promotes a systematic approach to capturing, storing, distributing, and
reusing information of an organization in order to make it available, actionable, and
valuable to others. The prospects for applying KM technologies to embedded
systems to meet these demands are very promising.
The Experience-Oriented Smart Embedded System (EOSES) is proposed as a new
XVII
technological platform providing a common knowledge management approach that
allows mass embedded systems for experiential knowledge capturing, storage,
involving, and sharing. Knowledge in the EOSES is represented as SOEKS, and
organized as Decisional DNA. The platform is mainly based on conceptual principles
from Embedded Systems and Knowledge Management. The objective behind this
research is to offer large-scale support for intelligent, autonomous, and coordinated
KM on various embedded systems.
Several conceptual elements of this research have been implemented in testing
prototypes, and the experimental results that were obtained show that the EOSES
platform can provide active knowledge management to different embedded systems,
and it can also enable various systems to learn from their daily operations in many
different fields to gather valuable knowledge, assist decision making, reduce human
workers’ workload, and improve the system itself. Consequently, the EOSES has
great potential for meeting today’s demands for embedded systems, and providing a
universe knowledge management platform for mass autonomous mechanisms.
XVIII
PREFACE
Computers and software are becoming an inescapable part of our daily life: we
use them to write thesis (like this one), communicate through the Internet, search for
information on the web, play PC games and store data. Aside from these most visible
uses, the vast majority of computers and software in use, however, are much less
visible. They control your microwave oven, dishwasher, and refrigerator. They run
trains, ships, and aircrafts. They command traffic lights in a city, power generation in
a power plant, and elevators in a building. They control the brakes, seatbelts, airbags,
air conditioner and engine in your car. They run your digital camera, enabling you to
take photos without any knowledge of photography. They run network switches,
printers, and cashier machines. They administrate man-made satellites to handle
hundreds of billions of voice, video, picture and data transmission tasks across all
countries and continents. These less visible computers are called embedded systems,
and the software they run is called embedded software.
Embedded systems have been in use since the 1970’s. For most of their history
embedded systems were seen simply as small computers designed to accomplish one
or a few dedicated functions; and they were usually working under limited resources
i.e. limited memories, limited processing power, and limited energy sources. As
such, embedded systems have not drawn much attention from researchers, especially
from those in the artificial intelligence area. However, things have changed. Thanks
to the efforts of scientists over recent years, great progress has been made in both
computer hardware and software, which enables us to have much more powerful
XIX
computers in very small sizes and with many more functions.
Thus, how to make embedded systems smart is becoming one of researchers’ new
challenges as IT technology progresses. One approach for making embedded systems
smart is based upon experiential knowledge management: by collecting and reusing
experiences, this approach enables embedded systems to capture and/or store their
experiential knowledge through day-to-day tasks, and allows them to index relevant
pieces of knowledge and apply such knowledge to the concerned circumstance for
further use, such as supporting decision making.
To achieve experiential knowledge management in embedded systems requires
new methodologies and tools to be developed. The work I am presenting here was
carried out during my Ph.D. in Mechanical Engineering at The University of
Newcastle, and under the supervision of Professor Edward Szczerbicki and Doctor
Cesar Sanin.
My research was applied to the development of the Experience-Oriented Smart
Embedded System (EOSES). In this thesis, I shall show (1) the need for building
smart embedded systems; (2) a smart experiential knowledge administration platform
for embedded systems; (3) what is Experience-Oriented, and why experienceoriented? ; (4) an effective implemented knowledge representation structure
applicable for managing experiential knowledge for embedded systems; (4) a
procedure for the integration, solving, and selection of a holistic knowledge
representation structure; and (5) two case studies of the Experience-Oriented Smart
Embedded System. To this end, I divided the plan of this document into four parts,
which structures a total of six chapters:
XX
The first part is a guided tour of relevant literature.
Chapter 1 – Introduction: Research Foundations. This chapter consists of three
parts. In the first part, I shall describe embedded systems. In the second part, I shall
discuss knowledge, knowledge management and intelligent technologies. In the third
part, I shall show the need for building smart embedded systems.
The second part concentrates on the Experience-Oriented Smart Embedded
System (EOSES). It is presenting the design rationale of the whole EOSES, as well
as a general perspective of its mechanisms in one chapter. It consists of two chapters.
Chapter 2 – Experience-Oriented Smart Embedded System (EOSES). This
chapter identifies the features, architecture and conceptual models of the EOSES. It
also shows that its requirements can match with the established needs for smart
embedded systems. I shall describe and justify the overall solution envisioned and
the implementation choices by explaining every main stage.
Chapter 3 – Experiential Knowledge Management in EOSES. This chapter
presents
EOSES’s
conceptual
architecture
and
knowledge
management
methodologies in details. It also introduces the knowledge representation used in
EOSES – the SOEKS and Decisional DNA; and why I chose them for EOSES.
The third part shows the case studies of EOSES. It consists of two chapters.
Chapter 4 – A Server-based Application: EOSES Applied to Robotics. In this
chapter, I apply EOSES to a robot. With the help of EOSES, this robot is capable of
learning maps, and reusing its knowledge about these maps to make decisions.
XXI
Chapter 5 – A Stand-alone Application: EOSES Applied to Smart TV. In this
chapter, I integrate smart TV with EOSES in order to allow the smart TV to capture
and store its user’s watching habits, and re-use these habits in the user’s future
watching.
The fourth part outlines the lessons learned and the perspectives. It discusses
the evaluation and return on experience of the EOSES, describing improvements and
future work.
Chapter 6 – Conclusions and Future Work. This chapter outlines the current
research and implementation status of the EOSES. I shall present the result of my
work, and discuss the work remaining to be done in the EOSES, some of them have
already started to be implemented and tested, while others are shallow specifications
of future work. I provide considerations on extensions that could be imagined to go
towards a complete real-world solution and I give long-term perspectives that can be
considered as my future research interests.
Finally, I have to say that this document does not fully develop all the aspects of the
platform. However, it establishes the theoretical concerns underpinning it.
Additionally, due to the numerous topics involved in this research and the
development of the platform, a general literature review was included in chapter one;
nevertheless, more specific literature reviews and state-of-the-art advances in related
areas are included in some chapters in order to offer a better understanding of the
concepts discussed.
Haoxi Zhang
XXII
1
Introduction: Research Foundations
Today the drive of innovation is stronger than ever. Novel technologies and
applications are reaching into every corner of our lives. Consequently, new needs
and expectations for Embedded Systems (ESs) have increased dramatically (Akhras
2000). The current market demands embedded systems to be built smart so that they
can finish tasks automatically, and assist users to make decisions more efficiently
and effectively. The booming market for “smart products” is the evidence for this
like Smart TV, Smart phones, and even Smart House. However, these ‘smart
23
products’ are not really smart, or not smart enough. Knowledge Management (KM)
is a discipline that promotes a systematic approach to capturing, storing, distributing,
and reusing information of an organization in order to make it available, actionable,
and valuable to others (Davenport 1994; Duhon 1998). The prospects for applying
KM technologies to embedded systems to meet these demands are very promising.
This chapter introduces the general background for the Experience-Oriented
Smart Embedded System, starting with the concept and state of the art advances in
embedded systems. Then, the definition and theories of experience and knowledge
are presented. Subsequently, some of the KM technologies are reviewed, and a novel
knowledge management platform, the Smart Knowledge Management System, is
introduced. Finally, research objectives and a brief summary on this chapter are
presented.
1.1
Embedded Systems
Embedded systems are computer systems created to implement one or a few
specific functions (Steve 2002), such as the control system in an elevator, or the ABS
(anti-lock braking systems) in a car. They are usually embedded as part of a complete
system. Nowadays, any electronic product that includes a microprocessor (one or
more) and software to carry out some constituent function within a larger entity can
be regarded as an embedded system (Kleidermacher & Kleidermacher 2012).
1.1.1
Application Areas of Embedded Systems
Embedded Systems are present in quite diverse areas: from household appliances
like microwave ovens and washing machines, to industrial applications like
24
automatic production lines and network switches; from portable devices such as MP3
players to very big equipment like nuclear power plants. The following list outlines
the main areas in which embedded systems are used (Marwedel 2010).
• Consumer electronics: As a very important sector of the electronics industry,
consumer electronics integrated with Information Technology (IT) is steadily
growing. Better quality and new features are implemented using advanced
digital techniques. Many game consoles, digital TV sets, digital cameras, and
tablets have powerful high-performance processors and memory systems.
• Telecommunication: Mobile phones have been a very fast growing as well as
rapidly changing market in the recent years. Novel cyber technologies are
having a remarkable effect on this market. Actually, on all forms of embedded
systems.
• Transport: Embedded systems are widely used in all kinds of transport systems.
The Anti-Braking Systems (ABS) is a very common example of embedded
systems used in small vehicles. While the flight control systems, pilot
information systems, and anti-collision systems are embedded systems used on
airplanes. Furthermore, there are similar systems for railway transport and
sailing.
• Security: Security is always critical for all kinds of systems. There are more and
more embedded systems being developed and used in improving security. For
example, with finger print sensors or face recognition systems, we can identify
people much quicker and more precisely.
25
• Robotics: Robotics is a very typical application of embedded systems.
Embedded processors, sensors, and other control units work together to run the
robots. The improvement of embedded system technology directly affects
robotics.
• Health sector: It is very obvious that health is essential and vitally important for
us. Healthcare products have drawn a lot attention from researchers because
there is a huge potential for improving healthcare services by taking advantage
of embedded systems and information technology. There are a number of
techniques that can be applied to this area.
• Military applications: Embedded systems have been used in military
equipment for many years. They control the latest missiles, fly Unmanned Aerial
Vehicles (UAV), and they run military radar systems. Today, the importance of
embedded system technology for military purpose is increasing.
• Smart buildings: Sensors, processors and computers have been used in some
buildings to make the building ‘intelligent’. At the entrances, there are cameras,
sensors and other equipment that work together to ensure safety and security of
the building. Inside, there are many other embedded systems taking care of the
comfort level of the building, and helping to reduce the energy consumption
within these buildings. There is a trend towards integrating access control,
lighting, air-conditioning, distribution of information and accounting into a
single system. Sensors can easily detect empty rooms, and for those empty
rooms, air conditioning subsystems can be either turned off or increased the
tolerance levels, and the same applies to lighting systems. ‘Smart’ use of blinds
26
can also optimise air-conditioning and lighting in order to reduce energy
consumption.
1.1.2
State-of-the-art Embedded Systems
Embedded systems have been linked with poor capability and low functionality
for a long time. However, recent advances in microelectronics, Integrated Circuit
(IC), communications, computing, software and other information technologies are
now forcing embedded systems to be composed of a large set of processing elements,
and the trend is towards significant enhanced capability, complexity, and
functionality. Also through wireless or wired networks more and more embedded
systems are being connected together to create large-scale distributed embedded
systems. Such embedded information processing and computing technology has
become at the same time a transformer of organizations and markets as well as a
cornerstone for future manufacturing enterprises (Pereira & Carro 2007).

Hardware
Microprocessors, memories and many other hardware components are becoming
smaller and extremely powerful. According to the Moore’s law (Pereira & Carro
2007), the capacity of microprocessors doubles about every two years. Let’s take
Apple’s iPhone for example: in 2010 customers could buy an iPhone 3GS with an
ARM Cortex A8 600 MHz processor inside (Miller 2010), while in 2012 the
processor in the new released iPhone 5 has been upgraded to 1.3GHz (Powell 2012).
In fact, microprocessors with more than 1 GHz clock rates like the PowerPC G3
(1.1GHz) have been used in embedded systems for a few years (Perls 2009).
Likewise, memory’s performance is progressing rapidly too. For instance, SanDisk
27
released its 32 GB SDHC Plus cards in January 2008, and four years later, the 128
GB SanDisk Extreme® SDXC™ UHS-I card is introduced in 2012 (SanDisk 2012).
Apart from innovation of single hardware elements, another trend for embedded
systems is to build all the constituents of a computer or other electronic system into a
sole integrated circuit (microchip). This is called System-on-Chip (SoC)
(Bergamaschi et al. 2001). A SoC allows engineers to contain analog, digital, mixedsignal, and often radio-frequency functions onto a microchip, and even building one
or several different processors on a single SoC. For instance, the portable phone is a
typical application of such technology (Reinaldo et al. 2001; Rajsuman 2000).
In addition, reconfigurable hardware, as a novel evolution, provides another level
of
programmability that
could
significantly improve
embedded
systems’
performance. Moreover, the innovation of wireless networking enables seamless
communication and Internet connection for embedded systems. Due to these
improvements of hardware technology, embedded systems are growing into more
complex, smaller, and powerful elements. Meanwhile, the power consumption and
price of embedded systems are reducing.

Software and Programming Language
In comparison with embedded hardware technology, embedded software/firmware
and middleware technologies are making notable progress too. As the core
component, embedded operating systems are making big progresses on reliability,
configurability, portability, adaptability, and robustness (Pereira & Carro 2007). For
example, μC/OS II, as a classic embedded real-time operating system, has
successfully been applied in multiple high level safety-critical devices, including
28
those certified for avionics (EUROCAE ED-12B and DO-178B Level A), medical
devices (Pre-Market Approval – PMA, medical FDA pre-market notification 510(k)), and for nuclear systems and transportation (SIL 3/SiL4 IEC) (Micrium
2012).
Embedded Linux is another example. Mobile devices running a Linux operating
system are rapidly increasing these days. This is mainly because embedded Linux is
an open source, free, robust and stable operating system supported by a large
community of developers. Google Android (Android 2012) is one of the most
successful embedded software in the history that benefits from the advantages of
embedded Linux. Android is an open-source software stack that consists of a Linuxbased operating system, a set of key applications and middleware (Meier 2012).
Moreover, the functionality of Android devices can be easily extended by installing
applications ("apps"). There is a large community of developers writing applications
for Android devices. Those applications are primarily written in a customized
version of Java (Shankland 2007). Even though Android is initially designed for
mobile devices such as smartphones and tablets, its features of being open and
customizable enable Android to be used on many other electronics and to provide a
consistent platform for application development across an increasingly wide range of
devices (Petrovan 2012; Meier 2012).
Programming languages for embedded software development is currently focused
on the C and C++ according to the survey (VDC 2011). Due to the huge progress in
both microprocessors and memories, it has become practical to apply more mature
approaches onto embedded systems, like using a regular C or C++ compiler, which
enables very fast adaption to any task, also it makes the deploying of new
29
applications an easier job (Pereira & Carro 2007). However, the Java programming
language is rapidly gaining momentum in this field. Devices with embedded Java
such as Smartphones, tablets, and other electronics have increased dramatically.
Hugo Bara, Android's director of product management, said, “500 million devices
activated globally, and over 1.3 million added every single day." (Shankland 2012).
In fact, at first Java was created for embedded systems (Cadenhead & Lemay 2002).
A key concept in Java is the Java Virtual Machine (JVM), which encapsulates all
platform-specific services, such as system I/O, networking, etc. It enables Java byte
codes to be run on any architecture for which a customized JVM is available.
To sum up, recent advances in computer hardware, software, and microelectronics
are enhancing embedded systems to be increasingly powerful, robust, smaller and
functional. These advances not only enable embedded systems to comprise a large
set of computing elements and run complex programs, but also allow embedded
systems to be custom made to meet all kinds of demands from our daily life.
1.2
Knowledge and Experience
The Oxford Dictionary (2010) defines Knowledge as “facts, information, and
skills acquired by a person through experience or education; the theoretical or
practical understanding of a subject, or awareness or familiarity gained by
experience of a fact or situation”. While Drucker P. F. (2003) defines it as
“information that changes something or somebody - either by becoming grounds for
actions, or by making an individual (or an institution) capable of different or more
effective action”. Knowledge is not knowledge until the information inside itself has
30
been taken and used by people (O'Dell & Hubert 2011).
Experience, as a general concept, comprises previous knowledge or a skill
obtained through daily life (Sun and Finnie 2003, Sun and Finnie 2004). Usually
experience is understood as a type of knowledge that one has gained from practice
rather than books (Sharma, Singh, and Goyal 2012). In this way, experience or
experiential knowledge can be regarded as a specialization of knowledge that
includes information and strategies obtained from performing previous tasks.
In a business context, knowledge is what a company knows about its mistakes,
successes, products, processes, customers, and competitors. The knowledge has thus
been taken and recognized as a valuable asset (Schreiber et al. 1999). The
globalisation of the economy has put tremendous pressure on companies for
increasing efficiency and adaptability, improving products, innovation, and creating
competitive advantages. The awareness of a company’s own knowledge and the
management of this knowledge are becoming steadily important as they cope with
these pressures (Myers 2012), and companies can greatly benefit from managing
their knowledge (O'Dell & Hubert 2011).
1.2.1
Knowledge Management
Knowledge Management (KM) is a term and a concept that arose approximately
twenty years ago. Davenport (1994) created the still widely cited definition at KM’s
early development stage: “Knowledge management is the process of capturing,
distributing, and effectively using knowledge”. A few years later, another frequently
quoted definition of KM was built by the Gartner Group as “Knowledge
management is a discipline that promotes an integrated approach to identifying,
31
capturing, evaluating, retrieving, and sharing all of an enterprise's information
assets. These assets may include databases, documents, policies, procedures, and
previously un-captured expertise and experience in individual workers” (Duhon
1998).
Both definitions share a very similar idea, which is that KM is a systematic effort
to capture, store, distribute and reuse the information of an organization to make it
available, actionable, and valuable to others. The discipline of KM is about building
up and administrating the processes to deliver the right information to the right
people at the right time, and help people act on information and share this
information in order to improve the performance of organizations (O'Dell & Hubert
2011). This information should be accessed from more than just documents and
databases, but also the experiences of individuals and teams through their day-to-day
work, collaboration and communication.
Knowledge management infrastructure is a development of the digital nervous
system within an organization, which integrates the organization at a deeper level. In
order to accomplish this goal, the tools are required to be involved in every aspect of
an organization, such as:
•
E-mail system
•
Intranet working / administration system
•
Database
•
Documents and resources
32
•
Knowledge repositories
•
Expertise access tools
•
Meeting, discussion, and chat
•
Communications
•
Work flow, procedures
Thus it is obvious that KM would involve many different organizational
processes, and KM can take on various forms. A survey carried out by Liao (2003)
found that there were generally seven categories of KM technologies and
applications developed until 2002. In another study (Matayong & Mahmood 2012),
after analysing 30 published articles between 2003 and 2010 from high quality
journals, found nine core theories in the KM area. These KM technologies,
applications, and theories enable enterprises to manage their knowledge from
different perspectives. However, there are limitations to these technologies. Most of
them are designed for one specific kind of product; they don’t have standard
knowledge presentation; most systems lack the capability for information sharing
and exchange and most of these systems only focus on supporting a particular stage
of a product lifecycle (Li, Xie, & Xu 2011).
Furthermore, due to the advances in IT and cyber technologies, our working and
living environments have been greatly changed. For instance, tablets and
smartphones are changing how we live, study, and work. People now are switching
from desktops to these small computers. These small devices are reaching into every
corner of our lives - even restaurants are getting rid of menus and are using tablets to
33
order dishes). Traditional KM technologies are not designed for this situation. For
example, Expert Systems are too slow for these devices (Mazilescu 2011).
Therefore, it is very urgent and necessary to develop new tools to cope with these
issues in order to establish a knowledge-rich environment that makes seamless
knowledge connection among individuals, organizational processes, and enterprises
possible.
1.2.2
Smart Knowledge Management System, SOEKS, and
DDNA
Smart Knowledge Management System (SKMS), as a novel application of KM
technology, is offering a new approach into AI field. Unlike other existing KBSs,
SKMS is more focused on extracting and evolving explicit knowledge through
experience, and reusing this knowledge to support decision-making (Sanin and
Szczerbicki 2009). Instead of directly using human expertise, the SKMS defines a set
of four macro processes and components required to capture, store, improve, reuse,
and transmit experience through day-to-day tasks (Sanin and Szczerbicki 2008).
The SKMS relies on the concepts of Set of Experience Knowledge Structure
(SOEKS) and Decisional DNA (DDNA). The Decisional DNA is a knowledge
repository approach that stores, organizes and manages formal decision events stored
in the Set of Experience Knowledge Structure (SOEKS) (Sanin et al. 2009).
SOEKS has been developed to acquire and store formal decision events in an explicit
way (Sanin and Szczerbicki 2009). It is a model based upon available and existing
knowledge, which must adapt to the decision event it is built from (i.e. it is a
dynamic structure that depends on the information provided by a formal decision
34
event) (Sanin et al. 2009); besides, it can be represented in XML or OWL as an
ontology in order to make it transportable and shareable (Sanin and Szczerbicki
2006; Sanin and Szczerbicki 2007). The SOEKS has been successfully applied to
industrial applications, specifically for maintenance purposes, in the areas of energy
and finances research (Sanin et al. 2009), and in conjunction with augmented reality
techniques (Toro et al. 2007).
As a promising approach for the next generation of manufacturing systems, KBS
has continuously been expanding, both in depth of knowledge as well as in the area
of applications (Zhou, Wang, & Lou 2010). Consequently, industry, organizations,
and individuals will benefit a lot from smart solutions, improved productivity,
intelligent management and more stable and increased product quality.
The subject of this thesis is focused on applying such technology to ubiquitous
embedded systems to help create a smart cyber environment for our society.
1.3
Research Objectives
This chapter has briefly pointed out that embedded systems are reaching into
every corner of our lives, as well as the importance of knowledge for individuals,
organizations and industry. In order to adapt and react properly to today’s dynamic
environment, new methodologies and tools are required. Consequently, the proposal
presented in this thesis undertakes to systematically capture, create, reuse, and share
decisional experience of work processes in embedded systems, helping the users of
these embedded systems to perform more quickly and efficiently in our daily life,
and supporting a path towards appropriate automation for frequently recurring tasks.
35
There is one overall goal for this research:
Develop a Smart Platform for Embedded Systems that would support decisionmaking through gaining knowledge by experiencing from the day-to-day
operation.
The major significance of this research is highlighted as the creation of such a
platform for different embedded systems to capture, evolve, and share the experience
of every decision made. When this goal is reached, the platform will allow building
up an intelligent cyber environment via the electronics surrounding us, and enable
everyone to benefit from this smart platform.
As the developing of the proposed platform, several significant and innovative
sub-aims will be achieved. In particular, they are:
•
Develop a converter that is able to transform data of decision events within
embedded systems to knowledge (i.e. SOEKS),
•
Design the protocols that allow standard knowledge exchange among
diverse embedded
systems
(they may use different
knowledge
presentations),
•
Establish the application frame to support different embedded systems
using the platform, and enable mass customization approaches through
distribute deployment,
•
Implement DDNA and SOEKS in the integration of different intelligent
and decision-making embedded systems,
•
Create automatic methods to select and analyse SOEKS according to the
36
priorities of the system, and suggest the best solution,
•
Implement
techniques
for
capturing,
generating,
retrieval,
and
management of experienced decision events during the system operation.
•
Establish techniques for manipulation, administration and sharing the
collected experiences among embedded systems.
1.4
Summary
In summary, I propose in my thesis a multi-technology smart platform that is a
combination of Embedded Systems, Knowledge Management Systems, Smart
Knowledge Management Platform, Decisional DNA, Set of Experience Knowledge
Structure, and other techniques.
This proposed smart platform allows embedded systems to capture, store, evolve,
reuse, and share experiential knowledge, and improves embedded systems’
performance in many ways, like reducing the analyzing time, quick start via learning
from similar systems, always up-to-date through knowledge sharing, as well as
decreasing energy consumption.
37
2
Experience-Oriented Smart
Embedded System
As introduced in the previous chapter, the ability to learn from past experience
and reuse such learned knowledge has become a critical factor for individuals’ and
organizations’ success. Experience, as one kind of knowledge, comprises previous
knowledge or skill obtained from practice, and it carries information of the tasks
handled with the results of different handling strategies in the past.
38
Thanks to technological advances in computer processing, storage, and
communication capabilities, together with improving manufacturing techniques,
embedded systems have been made smaller, lower-cost, and more powerful; which
allows the spread of an almost infinity of these embedded systems. The area of
embedded systems and their interconnection is nowadays considered as
fundamentally important (Almeida 2008). In fact, the world we are living in relies on
these small cyber devices.
From a Knowledge Management (KM) perspective, knowledge is useful only if it
is accessible to all users, and can be used to solve problems and make decisions (Lao,
Xiao, Wang, and Qin 2008). Additionally, the active KM is required to cover all
organizational processes. The wide use of embedded systems has thus turned them
into a physical basic for most of knowledge-related activities, because embedded
systems are largely used in organizational processes. By learning from practical
processes such as problem solving and decision-making, experiential knowledge can
be obtained. Subsequently, through proper reusing of knowledge, related embedded
systems can help or complete these processes more efficiently or even automatically.
Hence, embedded systems can be built smart. Most importantly, the acquiring of
knowledge is the creating of asset for such smart embedded systems’ domain; which
allows the domain/organization to know about itself from a systematic view, and
keep its asset (i.e. the knowledge) once employees retire or machines are replaced by
newer models.
Several approaches have been developed to support KM using different
technologies such as expert systems, decision support systems, knowledge-based
systems, and data mining (Matayong & Mahmood 2012; Liao 2003). However, there
39
are three common problems for applying current KM technology to embedded
systems, i.e. 1) they are too specific, 2) they do not have standard knowledge
representation, and 3) they lack information sharing and exchange capabilities.
Therefore, improving KM by means of supporting mass customization autonomous
mechanisms and the use of a domain-independent knowledge representation, still
remains as a research area in need of exploration.
technological platform based on knowledge management and is designed to provide
a common knowledge management approach that allows mass embedded systems for
experiential knowledge capturing, storage, involving, and sharing. Knowledge in the
EOSES is represented as SOEKS, and organized as Decisional DNA. The platform is
mainly based on conceptual principles from Embedded Systems and Knowledge
Management. The objective behind such combination of elements is to offer support
for intelligent, autonomous, and coordinated KM on a large scale.
This chapter introduces the general elements proposed for the ExperienceOriented Smart Embedded System, starting with its overall view and main features.
Then, the conceptual models and architecture are presented, describing the different
layers and services that comprise the platform. Finally, summary and brief
conclusions on this chapter are presented. The main goal of this chapter is to provide
a general outline of the Experience-Oriented Smart Embedded System. The details of
the platform’s implementation and case studies of using two conceptual models will
be unfolded in the following chapters.
40
2.1
Overview of EOSES
Based upon the principle of KM, the KM infrastructure is a development of the
digital nervous system within an organization, which integrates the organization at a
deeper level (Frid 2000). In an organization, there are many processes with the
purpose of fulfilling a collection of well-defined organizational aims. Due to the
diversity of embedded systems, the hardware and software of digital devices
involved in these processes could be various. Moreover, different companies could
use different devices for similar processes. As a consequence, the ExperienceOriented Smart Embedded System is designed to handle these issues, providing the
means for organizations and individuals to have a common systematic KM tool
within diverse embedded systems.
Figure 1 shows the overview of the Experience-Oriented Smart Embedded
System. The platform integrates different organizations and their devices as
illustrated in three major levels: Device, Process, and Knowledge.
41
Figure 1. Overview of the Experience-Oriented Smart Embedded System
At the bottom level of the platform, devices representing different embedded
systems in knowledge-related organizational processes capture knowledge for the
activities they are involved in. Each embedded system at the bottom level may be
different, and may be involved in different activities. A knowledge-related process
may have one or more activities. By integrating knowledge captured from a process’s
activities, experiential knowledge grows and evolves for that given organization
process. Moreover, devices can interact with each other in order to share knowledge.
At a higher level, there are processes within an organization. In a similar way, by
collecting every process’s knowledge, the knowledge for that organization is formed
and gathered. Also, processes at the middle level can interact with each other to share
42
knowledge. At the top, there is the knowledge level. Cutting-edge technologies, for
example Cloud Computing, may be used to store and enable interaction and
knowledge sharing among organizations. Organizations may create their own clouds
for a specific purpose, like data security. In addition, a large-scale knowledge market
could be formed, in which knowledge is treated as the main asset (Mancilla-Amaya
2010b).
2.2
Main Features of EOSES
In order to enable different embedded systems to capture, reuse, evolve, and share
knowledge in an easier and more standard way, the Experience-Oriented Smart
Embedded System shall provide the following features:
• Experience-Oriented: experience, as one kind of knowledge learned from
practice, is the ideal source for improving performance of processes, in which a
lot of practical activities involve. By reusing experiential knowledge, decision
makers can make decisions faster, and more efficiently base their current
decisions on experiences obtained from previous similar situations. The EOSES
is designed to use such experiential knowledge in supporting individuals and
organizations to make decisions better and more efficiently.
• Adaptability and Cross-platform Portability: Since most embedded systems are
special-purpose designed, the hardware and software for each of them could be
distinctly different from others; thus, adaptability and cross-platform portability
are critical issues for EOSES. For this reason, which causes the first problems of
these three, i.e. they are too specific. In order to work with custom embedded
43
software, most knowledge-related tools are specifically designed only for given
embedded systems or even for a particular stage of a product lifecycle.
Adaptability and cross-platform portability are rarely considered during the
design and development of these tools. For solving this problem, EOSES is
designed to work with specific designed software (applications) without redesigning or re-building. Java is used during the development of EOSES. And
even though Java may be a bit slower than some other programming languages
like C or C++ (Jelovic 2009), its inherent cross-platform capability can make
significant advantages on deployment and maintenance. Moreover, as processing
power soars, the evolution of Java language, and optimizations of JVM
(Georges, Buytaert, and Eeckhout 2007), is not a problem anymore for running
Java applications on embedded systems (Mulchandani 1998).
• Compactness and Efficiency:
Another important element concerning the
application of EOSES is that embedded systems are usually resource-limited,
which means our applications have to be designed in a compact, efficient shape.
As a flexible knowledge representation structure, Decisional DNA can be
customized compactly according to certain system goals. Moreover, it has been
shown in previous case studies (Sanin et al. 2009) that SOEKS is an efficient
knowledge representation structure, which can indeed enhance knowledge
retrieval and experience reusing applications. Thus, SOEKS is used in EOSES as
an efficient compact knowledge representation.
• Configurability: It is common for a system to run under different situations. For
example, the same system may work under different power supply conditions:
connected to a power line or by using batteries. If the system performs always in
44
the same way regardless of its power supply, it will soon run out of power and
shut down when using batteries. Also, a system may work in different modes,
for example it can be trained in training mode and work in an automatic selfcontrol mode. For developers, it may have debugging mode. Thus,
configurability is very desirable as it allows systems to perform properly in
various scenarios according to different settings.
• Security and trust: it is clear that a secure environment is a key requirement for
any distributed system these days, especially when the Internet is used as the
primary communication channel. Also, knowledge, and knowledge sources,
must be reliable to make the right decisions. Therefore, the concept of decisional
trust presented by Sanin and Szczerbicki (2009b) is extended to include more
features that reflect human-like behaviour.
2.3
Conceptual Architecture of EOSES
For carrying these five main features introduced above, the conceptual four-layer
architecture is designed for EOSES. These four layers are: Application Layer,
Interface Layer, Management Layer, and Knowledge Repository Layer. The platform
is conceptualized on the top of embedded systems, to extend the capabilities of the
latter by using DDNA and SOEKS for knowledge representation and exchange.
Figure 2 illustrates the conceptual architecture proposed for the Experience-Oriented
Smart Embedded System.
45
Figure 2. Architecture of the Experience-Oriented Smart Embedded System
Each one of the layers in the conceptual architecture is engaged with a set of
responsibilities and capabilities, as follows:
• Application Layer: This layer offers the platform’s whole functionality access to
the end-user. Mobile applications (APPs) may be used to help facilitate the
interaction among employees or groups to solve problems, make decisions, and
feed the system with information based on their daily activities. Additionally,
customization and complementing technologies are fully supported in the
application layer in order to fulfil different job requirements, which enable
EOSES to capture experiential data from different sources.
46
• Knowledge Repository Layer: Experiential knowledge, as the most valuable
source, is stored and managed at this layer. In EOSES, a single decision event is
captured and represented as a SOE, and a set of SOE is organized as DDNA
carrying the decisional fingerprint of the decision maker. This layer provides
functionality of access, storage, and administration of knowledge.
• Management Layer: This layer is the control central of EOSES. In order to
achieve knowledge capturing, reusing, evolving, storage, and sharing, a range of
functionality and capability are attached to this layer, such as inference, analysis,
knowledge extraction, knowledge retrieval, knowledge exchange, dynamic
process management, inter and intra-system interactions, global policies, and
cooperation amongst other mechanisms.
• Interface Layer: Interface is the component that connects the platform with its
outer environment; which provides a range of interaction, communication, and
knowledge-related services. These services provided by this layer are: system
configuration, input/output, application programming interface (API) service,
query service, knowledge sharing service, and updating service.
• Embedded system hardware and operating system: None of these two sections
is a part of EOSES. However, they comprise the fundamental base for EOSES.
On the top, there is the embedded operating system which manages the
embedded system and runs EOSES and applications. And at the bottom, there is
embedded hardware underlying, which is comprised of memory, computing
elements, and peripherals.
47
All the layers in the conceptual architecture make extensive use of experienceoriented knowledge management to provide appropriate autonomous capabilities to
embedded systems. More detailed discussion about the conceptual architecture will
be presented in Chapter 3.
2.4
Conceptual Models of EOSES
According to knowledge-related activities, different embedded systems may be
involved. For some activities, a single device would be enough to finish the job,
while for some other activities, having a number of devices working together is very
necessary, for example, smart buildings. In order to suit these requirements, make the
EOSES adaptable, and easy-to-apply to different situations, two major conceptual
models are designed for the EOSES. These two models are: the Server-based Model
and the Stand-alone Model.
2.4.1
The Server-based Model
This conceptual server-based model is designed for those systems that have more
than one embedded system working together to acquire information and experience
to perform as one single knowledge-based intelligent system, such as a robot. In this
conceptual mode, involved devices can be divided into three categories: sensors,
terminals, and clients. EOSES integrates data, information, and knowledge from
each of these involved sensors, terminals, and clients.
Figure 3 illustrates the conceptual server-based model. In this conceptual model, a
server is introduced to integrate all the data, information, and knowledge transmitted
from different embedded systems participating in a knowledge related process, and
48
extracting experiential knowledge from integrated information and finally, storing
the knowledge in the server for future reusing.
Figure 3. Conceptual Server-based Model for EOSES
• Sensors: are the simplest input elements in this conceptual model. They have no
capability to run any EOSES applications so that the only action the sensors
perform is to transmit proper signal/data to the server. For example, transmitting
current temperature is the only mission a temperature sensor accomplishes. In
many cases, sensors are used to measure the environment, and by collecting
measurement data from these sensors, the server can relate a decision event with
the environmental situation while the decision is being taken.
49
• Terminals: are sub-embedded systems having input and output capabilities and
can be perceived as sensors-actuators. For example, a garage door controller
receives inputs (i.e. ‘open’ or ‘close’ command) and executes the input as its
outputs.
• Clients: are sub-embedded systems that can run EOSES client applications and
have human-machine interfaces. Users can finish sub-tasks through these clients,
and can have interactions with the server through these clients. For example, in a
smart building, the user may turn on/off a room’s lights outside this building
through an APP running on an iPad (i.e. the client). Also, the server may inform
the user with various forms of information like sending a message to the user’s
smartphone saying ‘Running out of eggs!’ As sub-embedded systems, there may
be sub-knowledge repositories on them. In addition to processing local
information, clients may communicate with each other via wire and/or wireless
connections.
• Server: is the brain for this conceptual model. It is running EOSES server
applications aiming at integrating all the involved embedded systems into an
organic single system. Knowledge-related services provided by the server are
similar to those from the stand-alone application, but the server may run on a PC
rather than on embedded systems, and the algorithms in server-based models
could be more complicated than those in stand-alone models. A more detailed
case study of the conceptual server-based model will be given in Chapter 4.
50
2.4.2
The Stand-alone Model
This conceptual stand-alone model is designed for embedded systems that have
capability to finish tasks alone, like tablets. The difference between a stand-alone
model and a server-based model is: there could be many different embedded systems
involved in one single task in a server-based model, for example, smart building. In
other words, in a server-based model, there must be server software to control the
whole system. Also, knowledge for a given task is captured, and stored by the server
in server-based models. Thus, powerful devices such as computers are recommended
to run the server. However, computers may not available in some special cases, for
instance, robots. While in stand-alone model, there is always only one embedded
system that accomplishes the given tasks.
In recent years, mobile devices like tablets and smartphones have been built
increasingly powerful. They have fast processing units, large memory, and network
capability. Due to their capability, together with Web 2.0 technology and mobile
apps, these embedded systems have been largely used in knowledge-related
activities. For example, a real estate agent might use an iPad to evaluate the state of a
property: The agent checks the apartment, and uses the iPad to fill out forms for
property inspection, and makes decisions on items in need of maintenance, repair, or
replacement. Therefore, tools that can enable active KM on these embedded systems
are demanded. Figure 4 depicts the conceptual stand-alone model for EOSES.
51
Figure 4. Conceptual Stand-alone Model for EOSES
In this conceptual model, user interacts with apps running on embedded systems
to fulfil the tasks. EOSES service runs behind the apps, and supports the apps to
cooperate with knowledge-related activities and remember users’ behaviour. Every
time when the user makes a decision through the app, EOSES catches it, extracts it,
and transmits it into experiential knowledge, and stores it as a SOE. After learning
from the user, this system is able to participate in decision-making processes, and
give suggestions. Moreover, based on decisions the user made, EOSES can create the
user’s decisional fingerprint and the user’s profile, which can be used further in
activities such as knowledge sharing. For this model, a detailed case study is given in
Chapter 5.
52
2.5
Summary
In this chapter, the concept of the Experience-Oriented Smart Embedded System
is introduced at the beginning. The EOSES is a conceptual platform designed
towards smart knowledge management on embedded systems. The EOSES is based
upon the principles of different computing technologies, namely: knowledge
management, embedded systems, and Cloud computing.
Then, five major features of this platform are presented in the second section of
this chapter as the main concern in the work presented in this thesis followed by the
conceptual architecture of the EOSES. At the end, two conceptual models of EOSES
and their difference are addressed. And more detailed case studies for these two
models will be presented in Chapter 4 and Chapter 5 respectively.
53
3
Experiential Knowledge
Management in EOSES
As mentioned in previous chapters, the Experience-Oriented Smart Embedded
System (EOSES) is built to have five main features in order to solve three common
problems for current knowledge-based embedded systems, i.e. 1) too specific, 2)
does not have standard knowledge representation, and 3) lacking of information
sharing and exchange capabilities. These five built-in main features of EOSES are:
54
Experience-Oriented, Adaptability and Cross-platform Portability, Compactness and
Efficiency, Configurability, and Security and Trust. Moreover, the conceptual fourlayer architecture is designed for EOSES based on these five main features. Each of
the layers in the conceptual architecture copes with some of the three problems to
provide appropriate autonomous capabilities to embedded systems.
This chapter first introduces the novel knowledge representation approach –
Decisional DNA. Then it illustrates relation between layers of the EOSES conceptual
architecture and four main KM concerns, followed by details of the EOSES
conceptual architecture and how it solves these three problems of current knowledgebased systems. At the end of this chapter, initial tests and a brief summary on this
chapter are presented. The main goal of this chapter is to provide a detailed
discussion on how the proposed EOSES solves these three problems mentioned
above, and how it handles data, information, and experiential knowledge, in
embedded systems. Two case studies of the platform’s implementation using
different models will be unfolded in the following chapters.
3.1
Decisional
DNA:
Standard
Knowledge
Representation in EOSES
As mentioned at the beginning of this chapter, there are three common problems
remaining for KM tools. In order to solve these three problems and apply KM to
embedded systems, a standard knowledge representation is essential. The Decisional
DNA as a standard, flexible, and domain-independent, experiential knowledge
representation solution, that allows for knowledge to be acquired, reused, evolved,
55
and shared easily.
The Decisional DNA is a knowledge repository that organizes and manages
formal decision events stored in the Set of Experience Knowledge Structure
(SOEKS) (Sanin et al. 2009). The SOEKS has been developed to acquire and store
formal decision events in an explicit way (Sanin and Szczerbicki 2009a). It is a
model based upon available and existing knowledge, which must adapt to the
decision event it is built from (i.e. it is a dynamic structure that depends on the
information provided by a formal decision event) (Sanin et al. 2009); besides, it can
be represented in XML or OWL as an ontology in order to make it transportable and
shareable (Sanin and Szczerbicki 2006; Sanin and Szczerbicki 2007).
SOEKS is composed of variables, functions, constraints, and rules associated in a
DNA shape, permitting the integration of the Decisional DNA of an organization
(Sanin et al. 2009). Variables normally implicate representing knowledge using an
attribute-value language (i.e. by a vector of variables and values) (Sanin and
Szczerbicki 2009a), and they are the centre root of the structure and the starting point
for the SOEKS. Functions represent relationships between a set of input variables
and a dependent variable; moreover, functions can be applied for reasoning optimal
states. Constraints are another way of association among the variables. They are
restrictions of feasible solutions, limitations of possibilities in a decision event, and
factors that restrict the performance of a system. Finally, rules are relationships
between a consequence and a condition linked by the statements IF-THEN-ELSE.
They are conditional relationships that control the universe of variables (Sanin et al.
2009).
56
Additionally, SOEKS is designed similarly to DNA in some important features.
First, the combination of the four components of the SOE gives uniqueness, just as
the combination of four nucleotides of DNA does. Secondly, the elements of SOEKS
are connected with each other in order to imitate a gene, and each SOE can be
classified, and acts like a gene in DNA (Sanin et al. 2009). As the gene produces
phenotypes, the SOE brings values of decisions according to the combined elements.
Then a decisional chromosome storing decisional “strategies” for a category is
formed by a group of SOE of the same category. Finally, a diverse group of SOE
chromosomes comprise what is called the Decisional DNA (Sanin et al. 2009).
In short, Decisional DNA and SOEKS provide an ideal approach which not only
can be very easily applied to various embedded systems (domain-independent), but
also enable standard knowledge communication and sharing among these embedded
systems.
3.2
Experiential Knowledge Management in EOSES’s
4-layer Architecture
As mentioned before, based on the principle of KM, there are four main concerns
for KM tools: capturing, storage, distribution and reusing knowledge. In the EOSES,
the conceptual four-layer architecture is designed to solve these concerns. These four
layers are: Application Layer, Interface Layer, Management Layer and Knowledge
Repository Layer. The relation between EOSES’s four layers and these four main
KM concerns are as follows (see Figure 5):
57
Figure 5. Relation between EOSES Layers and KM Concerns
• Capturing: Knowledge capturing is the central core of the EOSES. It basically
involves three layers: Application Layer, Interface Layer, and Management
Layer. First, when the user uses an APP to make a decision, a decision event is
triggered from Application Layer. Data and information of this decision event
are then sent to Interface Layer to be collected and sorted. Finally, in
Management Layer, experiential knowledge of this decision event is captured.
• Storage: In order to save captured experiential knowledge, the EOSES relies on
the theory of Set of Experience Knowledge Structure (SOEKS) and Decisional
DNA (DDNA). As introduced in section 1.2.2, the Decisional DNA is a
knowledge repository approach that stores, organizes, and manages formal
decision events stored in the Set of Experience Knowledge Structure (SOEKS).
58
SOEKS has been developed to acquire and store formal decision events in an
explicit way. SOEKS can be represented in XML or OWL as ontology in order
to make it transportable and shareable. Also, by saving decision events in
SOEKS, experiential knowledge inside these events becomes readable and
actionable. In EOSES conceptual architecture, the Knowledge Repository Layer
stores experiential knowledge represented as SOEKS.
• Distribution: Knowledge sharing and distribution are key elements for making
knowledge available to others. In EOSES, XML format is chosen for saving
experiential knowledge represented in SOEKS. By using SOEKS and DDNA,
the EOSES provides standard knowledge sharing and distribution. In addition,
the OWL is also supported by SOEKS and DDNA. This involves Interface
Layer and Knowledge Repository Layer in EOSES conceptual architecture.
• Reusing: Only through knowledge reusing, the information inside knowledge
itself will be taken and used by people. Hence, knowledge becomes valuable. In
EOSES conceptual architecture, the Management Layer and Application Layer
are in charge of reusing knowledge. When the user needs to make a decision,
EOSES searches its knowledge repository, and compares current situations with
previous situations saved in SOEKS, then gives a suggestion or makes a decision
for the user based upon its experience gained through previous similar situations.
3.3
Conceptual Architecture of EOSES in Details
As proposed in section 2.3, the conceptual 4-layer architecture is designed for
EOSES. These four layers are: Application Layer, Interface Layer, Management
59
Layer, and Knowledge Repository Layer. Each one of these layers in the conceptual
architecture is engaged with a set of responsibilities and capabilities. This section
discusses the implementation and technique details of this proposed 4-layer
architecture (see Figure 6).
Figure 6. Architecture of EOSES in Details
3.3.1
Application Layer
The Application Layer offers EOSES’s whole functionality access to the end-user.
Also, this layer provides the Adaptability and Cross-platform Portability of EOSES.
60
At this layer, there are applications developed to fulfil different tasks. Instead of
generic applications, custom applications are largely used in embedded systems due
to the diversity of embedded systems’ hardware and software, which causes the first
problems of these three mentioned at the beginning of this chapter, i.e. too specific –
in order to work with custom applications, most knowledge-based tools are
specifically designed only for given embedded systems or even for a particular stage
of a product lifecycle. Adaptability and cross-platform portability thus are rarely
considered during the design and development of these tools. For solving this
problem, EOSES is designed to be capable of working with a range of differentdesigned applications running in various environments without re-designing or redeveloping them. In general, there are three kinds of applications at the Application
Layer:
1) Applications that use EOSES as a third-party software component
By calling EOSES APIs provided at the Interface Layer, any custom application is
able to easily accept knowledge-related services from EOSES which saves a lot of
time and money for companies and allows developers to have ‘knowledge’ and
‘knowledge management’ inside their various applications. A case study using
EOSES as a third-party component in a Smart TV application will be introduced in
Chapter 5.
2) EOSES Applications
Even though applications can access EOSES services via APIs, it is better to mix
EOSES code with applications’ code in some cases. For example, some applications
may have complicated logic, and need to access the EOSES services very frequently;
61
or applications may have restricted limitations, like small memory size. In these
situations, to design and build applications with EOSES code from very beginning of
development may be more efficient and valid. These applications that have been
integrated with EOSES code can be considered as EOSES applications. A case study
of EOSES applications will be presented in Chapter 4.
3) Applications that access EOSES service via Internet.
Apart from calling local APIs, applications can also access EOSES through a set
of Cloud services providing easy-to-use APIs of knowledge capturing, storage,
sharing, and reusing. Developers can easily apply EOSES technology to their own
mobile apps, websites, and products. Also, by using Cloud-computing technology,
applications benefit from the expansion of data processing capability, storage, and
security.
In short, the Application Layer makes accessing EOSES easy and flexible, which
enables various applications (event specifically designed) to have EOSES services
without re-design and re-developing. This layer also brings the Adaptability and
Cross-platform Portability of EOSES.
3.3.2
Interface Layer
The Interface Layer is composed of User Interface, EOSES API, System I/O, and
Integrator. These four components work together to form the interface of EOSES
connecting the platform with its outer environment. This layer not only provides
EOSES APIs of knowledge capturing, storage, sharing, and reusing to applications, it
also offers a range of interaction and communication functionalities. These functions
62
offered by Interface Layer are: system configuration, input/output, query, and system
updating.
• User Interface: The User Interface is developed to enable the interaction
between EOSES and the user. In particular, the user can control, set and
configure the system enabling the Configurability of EOSES, select EOSES
services, give feedback, and interact with the platform by using the User
Interface.
• EOSES API: The EOSES API is designed to allow EOSES to be accessed as a
third-party software component or a Cloud server providing services of
knowledge capturing, storage, sharing, and reusing. Through EOSES API,
knowledge-related services provided by EOSES can be easily applied to mobile
apps, websites, and products. Furthermore, the EOSES API is the key element to
allow the Adaptability and Cross-platform Portability of EOSES provided at the
Application Layer.
• System I/O: The System I/O allows EOSES to communicate with its domain of
operation (i.e. where it is running on). Similar to a PC I/O system, the EOSES
System I/O is designed to enable system input and output to its domain of
operation. Interruptions and memory reading and writing are engaged with
System I/O.
• Integrator: The Integrator works as a data-hub that gathers and organizes data at
the Interface Layer, and then transmits data to the Management Layer for further
processing. There are two kinds of data flowing in the Integrator:
63
(i) Scenario Data. This kind of data is used for capturing experiential
knowledge. In this case, we link each experience with a certain scenario
describing the circumstances under which experience is acquired. Normally, a
scenario comprises values such as the system time, name of a selected service,
ID of the undertaken process/task, user input, outcome, feedback, and other
service information. When a decisional event happens, the Integrator gathers and
organizes the scenario data, and then sends them to the Prognoser for further
processing;
(ii) Messages. Messages are used for communication purposes. When the user
configures the system, the application requests knowledge-related services, the
EOSES responses to the application, the user gives feedback, and all other inner
and outer EOSES interactions are processed as messages. Additionally, the
message may contain scenario data, for example, the Integrator sends scenario
data through messages.
3.3.3
Management Layer
The Management layer is designed as the control central of the EOSES. It is
designed to analyse and route data, manage experiential knowledge, cooperate with
other mechanisms, and administrate the EOSES platform. This layer comprises
EOSES Manager, Configuration Manager, Plug-In Manager, Protocol Manager,
and Prognoser.
• EOSES Manager: It is designed to deal with four main tasks, namely Data
Analysis, Knowledge Exchange Management, Cooperation Management, and
EOSES Platform Management.
64
(i) Data Analysis. It works as a switch that analyses, sorts and routes data. It
transmits data to where data are meant to be according to the type of data (i.e.
Scenario data or Messages), data content, process ID, and system settings.
(ii) Knowledge Exchange Management. It is in charge of knowledge exchange
among different EOSES applications. Cutting-edge technologies, like Cloud
computing, enable different applications to share experiential knowledge.
(iii) Cooperation Management. In order to allow for cooperation among threads
and applications, the cooperation management is designed. For example, in some
knowledge-related activities, there might be more than one thread or one
application involving. The cooperation management is designed to handle this
situation, and enable proper cooperation among threads and applications.
(iv) EOSES Platform Management. It is designed to manage the EOSES
platform including inter and intra-system interaction management, and dynamic
process management. It makes all aspects of EOSES work together.
• Configuration Manager: It is designed to read and write configurations of the
EOSES platform. By working with the User Interface, the Configurability of
EOSES is provided.
• Plug-In Manager: In order to support various applications to use EOSES
services, and expand EOSES functionality, plug-in technology is allowed by
Plug-In Manager, which manages the plug-ins and selects proper plug-ins for
applications.
• Prognoser: The Prognoser is in charge of analysis, inference, knowledge
65
extraction, knowledge creating, knowledge retrieval, and knowledge reusing.
For knowledge capturing, Prognoser captures the decisional event when a
decisional event has occurred, then inferences and extracts the experiential
knowledge in this decisional event, finally creates knowledge and sends the
knowledge to the Knowledge Repository Layer for storage. While in
supporting decision-making by reusing the knowledge, there are three
situations that must be addressed to applications: a) The Prognoser gets the
same scenario in the knowledge repository, and then it will reuse the decision
made previously for its current circumstance; b) The Prognoser does not get the
same scenario, but it gets similar ones. For this situation, the Prognoser will
perform some analyses and then derive a solution to the current circumstance,
apply it and save it as an experience into the knowledge repository; and c) The
Prognoser does not get the same nor similar scenario from the knowledge
repository. It can then ask other similar systems for help if a connection is
available, otherwise, it will return a default result.
3.3.4
Knowledge Repository Layer
Knowledge as the most valuable source is stored and managed at this layer. The
SOEKS and Decisional DNA technology is used to store and organize knowledge of
EOSES. This layer provides functionality of access, storage, and administration of
experiential knowledge of EOSES; also the SOEKS and Decisional DNA technology
offer the features of Compactness and Efficiency, and Security and Trust of EOSES.
There are Repository Manager, Decisional DNA Repository, and Convertor at this
layer:
66
• Repository Manager: The Repository Manager manages the Decisional DNA
Repository, including reading, saving, searching, removing, editing, and
moving experiential knowledge in Decisional DNA Repository. Moreover, it
provides access interface responding queries from the Management Layer.
• Decisional DNA Repository: The Decisional DNA Repository is designed to
store user’s experiential knowledge. It is composed of numerous Decisional
Chromosomes. The Decisional Chromosome is developed to gather and store
Decisional DNA genes within each category, capturing the user’s decisional
“strategies”. A Decisional DNA gene carries a single decisional event (i.e. a
SOE) and can be represented by a set of XML tags introduced by Maldonado
Sanin (2007) to store Decisional DNA in XML files.
• Convertor: In EOSES, as mentioned, experiential knowledge is stored as
SOEKS in XML files. The Convertor is designed to translate knowledge stored
in XML files into SOEKS knowledge statements in knowledge retrieval
activities.
3.4
Initial Tests and Results
This section presents the initial experimental prototype of EOSES and the results
obtained in this first set of experiments.
3.4.1
Experiment Goal and Design
The goal of the initial experiments is to validate the three of five main features of
the EOSES, i.e. 1) Adaptability, 2) Compactness and Efficiency, and 3)
67
Configurability.
A test application was developed using the Server-based Model, and implemented
on a LEGO® Mind-storms® NXT 2.0 robot (LEGO 2012), equipped with a 48 MHz
ARM7 microcontroller, a 256KBytes FLASH, a 64KBytes RAM (McNally et al.
2007), and colour sensor. Since the Decisional DNA is defined in Java (Maldonado
Sanin 2007), LeJOS is burnt into NXT 2.0 as the firmware. The LeJOS is a Java
programming environment for NXT® robots (LEJOS 2012).
In this experiment, the assumption is that there is a map with three lines drawn in
different colours, namely green, blue, and black, and each line ends at a terminal spot
drawn in different colours - yellow, red, yellow respectively (see Figure 7). The
robot should to gain its own experiential knowledge about these three lines after
following these three lines one by one, and save the knowledge gained in SOEKS.
Figure 7. The Test Map
68
To test the idea, a compact tailor version of SOEKS is designed to store the
experience, which shows that the EOSES can be easily customized in order to adapt
to different applications. In comparison with normal SOEKS, there are only three
Variables inside, and no Functions, Constrains, and Rules. These three Variables
store each line’s information of: 1) line colour, 2) time consumption to finish the
line, and 3) colour of the terminal spot.
For these experiments, two questions are designed to test the Compactness and
Efficiency: 1) what is the file size of each SOEKS, which is related to Compactness;
2) how much time it takes to query the experience, which is related to Efficiency.
While for Configurability, which is going to be tested by switching between two
modes by user configuration: First, the robot is set on Training Mode to learn the
lines one by one: It follows every line and gathers information of these lines. After
training, by configuring the robot through a human interface, the user can switch the
robot’s operation mode to Testing Mode, in which a test program runs automatically,
and prints out Query Time on the screen of the robot.
3.4.2
Experimental Results
As presented, the Adaptability and the Configurability were tested through tailor
version SOEKS and two different configurable running modes respectively. The test
results of Compactness and Efficiency are as following:
• Compactness: Each compact version SOEKS takes 1,082 bytes of memory
space, which is less than a simple full version of SOEKS (3,425 bytes). In the
full version of SOEKS, there are only 2 Variables, 2 Functions, 2 Constraints,
and 1 Rule.
69
• Efficiency: To test the efficiency of EOSES, the test program queries the
SOEKSs looking for the experience storing information about the red terminal
spot. Due to the memory limitation of the robot, a maximum of ten SOEKSs
are stored in the robot. In the best case, the first SOEKS is the one that the test
program is looking for, while in the worst case it is the last SOEKS.
Figure 8, shows the time consumption in best cases, including loading and parsing
XML files. The test program ran 10 times in best cases, and got the average time
consumption for best cases of 270.6 milliseconds.
Figure 8. Time Consumption of the Test Program in Best Cases
Figure 9, shows the time consumption in worst cases, including loading and
parsing XML files. The test program was also run 10 times with the result for the
worst cases of 2693.5 milliseconds on average. This was nearly 10 times the best
case, because in the worst cases, the test program loads, parses, and compares all 10
SOEKS to find the meant experience.
70
Figure 9. Time Consumption of the Test Program in Worst Cases
3.5
Summary
In this chapter, a detailed introduction of the Experience-Oriented Smart
Embedded System conceptual architecture with initial experiments is presented.
Starting with the knowledge representation solution – Decisional DNA, then the
relation between EOSES conceptual architecture layers and knowledge-related
services of knowledge capturing, storage, distribution, and reusing is presented at the
beginning. Followed by a discussions of EOSES conceptual architecture in this
section, a system design diagram for EOSES 4-layer architecture illustrates the
implementation and technique details of EOSES, and explains how each of the
EOSES four layers carry five proposed main features of EOSES, and how EOSES
solves three current problems for knowledge-based systems.
At the end of this chapter, a set of initial experiments and results are presented, in
which three of the five main features of the EOSES are examined. These three
EOSES’s features are: 1) Adaptability, 2) Compactness and Efficiency, and 3)
71
Configurability. By using the standard, flexible, easy-to-share, and domainindependent experiential knowledge representation solution – Decisional DNA, the
EOSES is adaptable to any embedded systems, which also solves the three common
problems current knowledge-based embedded systems are facing. And through a
customized version of SOEKS, the compactness and efficiency of EOSES is tested.
The results are acceptable, especially considering its test platform: a 48 MHz ARM7
microcontroller, a 256KBytes FLASH, plus a 64KBytes RAM. Also, the
configurability was tested through switching the robot’s operation between modes by
user configuration.
72
4
A Server-based Application: EOSES
Applied to Robotics
As mentioned in section 2.4 of chapter 2, the Server-based Model is one of the
two conceptual models designed to handle different situations in the ExperienceOriented Smart Embedded System (EOSES). The conceptual Server-based Model is
designed for enabling smart experiential knowledge management on systems that
have more than one embedded system/peripheral/sensor working together,
73
functioning as one single system, such as smart buildings and robots. In this
conceptual mode, a server is required as the core part that controls the whole system.
Apart from managing the system, the server extracts and captures experiential
knowledge from all kinds of data, information, and knowledge transmitted from
different peripherals, sensors, and sub embedded systems participated in the
knowledge related processes in the system. Also, the server is in charge of saving,
sharing, and reusing knowledge.
Robots are playing a more and more important role in today's industry. They are
designed to assist, or replace human beings in the execution of tasks, in regards to
both physical activities and decision-making. One of the long-standing goals of
research in robotics is to build a robot that captures experience through day-to-day
tasks, and reuses this experience to improve the robot’s task performance ability in
performing similar tasks (Berenson et al. 2012). The EOSES is designed to enable
its’ domain to capture, store, evolve, reuse, and share experience through daily tasks.
Also, the Robots are built with embedded computer systems, numerous actuators and
sensors, which are perfect potential platforms for the EOSES Server-based Model.
Therefore, this application of EOSES applied to robotics is proposed.
This chapter presents the case study of applying EOSES to robotics with Serverbased Model. Firstly, a theoretical background on robotics is presented, along with
some practical robotic applications in the fields of healthcare, autonomous control,
and manufacturing. Secondly, the application of EOSES applied to robotics is
discussed, in which the details of the application’s design and implementation are
illustrated. At the end of this chapter, experimental tests and brief summary are
drawn.
74
4.1
Background and Motivation
Nowadays, robots have been widely used in many different fields, such as
manufacturing, space exploration, military, and healthcare, just to mention a few.
Robots are not only capable of performing numerous different operations and tasks;
also they can work under special, risky, rough, and dangerous conditions. As robots
are reaching into new aspects of our life by providing close physical assistance to
humans, a strong demand for building autonomous robots arises. The successful
development of autonomous robots apparently requires capabilities of making
decisions and acting properly by the robot itself. The ability for robots to learn task
knowledge from previous task experience is expected (Medina et al. 2011).
4.1.1
What is a robot and what is robotics?
Yet there is no single common-accepted definition of robots, different countries
may define the concept of robot differently. In general, a robot is an electromechanical machine, which is designed and meant to be controlled by a computer
running some type of programs or a similar device. In American standards, a robot
must be a device that can be easily reprogrammable (Niku 2010; Wikipedia Robot
2012). Therefore, by simply changing the program, a robot should be able to perform
many different tasks and operations without having to be redesigned. The robots can
be classified into two different types based on a robot’s fundamental feature, i.e.
mechanical structure: robot manipulators for those with a fixed base, and mobile
robots for those with a mobile base (Siciliano et al. 2011).
Robotics, in different forms, has been on human minds for a long period (Mason
2012; Garcia et al. 2007). Robotics is considered as the research of those machines
75
that can replace human beings in both the operation of physical activities, and
decision-making. In other words, robotics can be recognised as an interdisciplinary
subject concerning the cultural areas of computers, electronics, mechanics, and
control. It is the science of studying the intelligent connection between action and
perception (Siciliano et al. 2011). Robotic systems involve not only robots, but also
other systems and devices cooperated with the robots. “Robotics is the art,
knowledge base, and the know-how of designing, applying, and using robots in
human endeavours” (Niku 2010).
4.1.2
Experience-based Robot Learning
A long-standing goal of research in robotics is to build a robot that captures
experience through day-to-day tasks, and reuses this experience to improve the
robot’s task performance ability in performing similar tasks. In recent years, more
and more efforts have been made by researchers to make learning-by-doing robots.
Several experience-based researches and theories in robotics can be found in
literature.
Berenson, Abbeel, and Goldberg (2012) proposed a framework that is able to
learn and improve with experience, called Lightning. This framework is designed for
planning paths in applications ranging from domestic assistance to robot-assisted
surgery, with the aim of reducing computation time. There are two modules in this
framework running in parallel: The Planning-from-Scratch module, and the Retrieveand-Repair module. The key part of the Lightning framework is the Retrieve-andRepair module, which retrieves and repairs paths stored in a path library. After a path
is generated for a new query, a library manager decides whether to store the path
76
based on the generated path’s similarity to the retrieved path and computation time.
Two heuristics that exploit two key aspects of retrieving an appropriate path from the
library are used in their framework: (i) A correlation between the amount a path
violates constraints and the amount of time needed to repair that path, and (ii) the
implicit division of constraints into those that vary across environments in which the
robot operates and those that do not. After evaluation and tests, Berenson et al. found
that the module reusing previous experience (i.e. the Retrieve-and-Repair module)
was faster than the other module not reusing experience in path planning. Nemec,
Vuga, and Ude (2011) presented an approach expanding and refining the database of
sensorimotor knowledge by exploiting previous experience. There are three stages in
their approach: At the first stage, an initial control policy suitable for a new situation
is realized from generalization of previous trained movements. This first
approximation can be improved at the second stage by using learning on the
manifold defined by the previously acquired training data. This enables for
reinforcement learning in a state space of reduced dimensionality. Finally, at the
third stage, the desired control policy is accomplished by exploring the solutions
outside of this training manifold in the full state space. Their proposed approach was
evaluated in simulation as well as with a real robot in a ball throwing experiment.
The experimental results showed that their approach could result in faster learning
rates. Pastor et al. (2011) proposed a general framework that uses sensor experience
form previous successful trials to achieve very contact-reactive motions for
autonomous robotic grasping and manipulation. By reusing previous experience,
their approach is able to systematically compensate for unforeseen perturbations in
future trials, which significantly increases the region of successful grasps. It also
allows for creating very compliant behaviours for contact interaction with the
77
environment. Median et al. (2011) proposed a cognition-enabled control framework
that enables a robotic assistant to enrich its own experience by acquisition of human
task knowledge during joint manipulation. Their framework was implemented on a
full- scale robot and evaluated in a classic-car restoration scenario where a human
carried a car’s bumper multiple times from a starting position to a final position in
three semantically different ways. The results showed great promise for the
applicability of their approach to a wider selection of more complex tasks. For more
examples of reusing experience in robotics, please refer to works from Sheh, Hengst,
and Sammut (2011), Nicolescu and Mataric (2001), Goel et al. (1993), Beetz et al.
(2004), and Stulp et al. (2009).
4.2
EOSES Applied to Robotics
This section presents the application of EOSES applied to robotics. It comprises
three parts: 1) Application description, 2) Robot hardware, 3) System design and
implementation.
4.2.1
Application Description
As mentioned above, the EOSES Server-based Model is designed for enabling
smart experiential knowledge management on systems that have more than one
embedded system/peripheral/sensor working together functioning as one single
system. Robots are usually composed of embedded computer systems, controllers,
processors, numerous actuators and sensors. For mobile robots, they can have wheels
and legs. They are ideal platforms for applying EOSES Server-based Model. In order
to test the usability of EOSES, the EOSES Server-based Model is applied to robotics.
78
In this application, an initial map learning and path planning EOSES application,
called Line Follower, is designed based on the test prototype introduced in section
3.4: the EOSES applied robot is capable of gaining experiential knowledge about
maps through integrated and organized sensor data, and then reuses its knowledge for
path choosing in the future. The Line Follower application is designed as the EOSES
‘Server’, and it can run the robot in two different modes:
• Training Mode: The robot follows every path on a map one by one in order to
learn and gain experience of these paths under the supervision of human beings.
After training, the robot will have experiential knowledge about the map’s paths.
• Testing Mode: In this mode, the robot’s knowledge about the map is tested. An
assumption destination will be given to the robot, and the robot should pick the
right path to follow down to its destination by reusing previous knowledge
learned through the Training Mode.
4.2.2
Robot Hardware
As the hardware platform, the LEGO® Mind-storms® NXT 2.0 robot is chosen as
the platform for implementation of the Line Follower application. The robot
comprises one LEGO Brick, two Servo Motors, one Ultrasonic Sensor, and one
Colour Sensor (LEGO 2012) (see Figure 10).
79
Figure 10. System Composition of the Line Follower Application
• LEGO Brick: The LEGO NXT Intelligent Brick is the computing central of the
robot working as the ‘brain’. This is also where the EOSES ‘Server’ (i.e. the
Line Follower application) is running. Inside the Brick, there is a powerful 32bit 48 MHz ARM7 microcontroller, a 256KBytes FLASH, and a 64KBytes
RAM (McNally et al. 2007). The Brick supports Bluetooth wireless
communication, which brings the possibility of knowledge sharing among
similar robots. Outside the Brick, there is one USB 2.0 port, four input ports,
three output ports, and a human interface comprising of four function buttons,
and one LCD display. In addition, the Brick is powered by 6 AA (1.5v) batteries.
• Servo Motor: The Servo Motor has a built-in rotation sensor that measures
distance and speed, and communicates with the NXT Intelligent Brick. It allows
for precise steps and total motor control within one degree of accuracy. In this
application, two Servo Motors are used to drive the robot’s movements.
• Ultrasonic Sensor: The ultrasonic sensor helps the robot judge distances and
80
"sees" where objects are. It is able to detect an object and measure its proximity
in inches or centimetres. In this application, the ultrasonic sensor is used to
detect the border of the map in experiments.
• Colour Sensor: The colour sensor enables the robot to distinguish a range of
bright and pastel colours. In order to mark every path and spot to look different
to the robot, five unique colours are used to draw the paths and spots on the
maps. With the help of the colour sensor, the robot is capable of reading and
distinguishing paths and spots drawn on the maps. Also, the colour sensor is
used to detect, and follow the path.
4.2.3
System Design and Implementation
As introduced above in section 4.2.2, the LEGO NXT Robot has a limited
computing resource. In order to reduce resource consumption, and optimize
application performance, direct EOSES access is used rather than calling EOSES
APIs. In other words, the EOSES is customised and integrated with other codes of
the Line Follower application. This means the Line Follower application is designed
as the second of these three EOSES application types, i.e. the EOSES Application. In
addition, the SOEKS API was used in order to manipulate the SOEKS structures.
The SOEKS API is a Java library developed by the Knowledge Engineering
Research Team of The University of Newcastle, Australia (KERT 2012).
Since EOSES is developed in Java, the Line Follower application is implemented
using Java 6 (Oracle 2012). In order to support running Java applications (i.e. the
Line Follower application), the LeJOS, a Java programming environment for the
NXT® robot, is burnt into NXT 2.0 as the firmware (LEJOS 2012). The system
81
architecture of Line Follower application is illustrated in Figure 11.
Figure 11. System Architecture of the Line Follower Application
First, through the Human Interface (i.e. four buttons and one LCD on the Brick)
the user can configure the application, set running mode, and read feedbacks. Then,
the application will function according to the user’s settings. By using data gathered
from sensors, the application captures operation experience in training mode, and
reuses this experience in testing mode. There are four main classes in Line Follower
application, namely Line Follower, Follower, FinderUsingSOEKS, and Finder.
Figure 12 presents a simplified class diagram for the application. The diagram shows
the details for the four classes.
82
Figure 12. Simplified Class Diagram for Line Follower
• LineFollower class: It is the main class that manages the whole application. It
communicates with its operation environment, and deals with the user input sent
from Human Interface. The LineFollwer class can be considered as an
implementation combining Application Layer and Interface Layer according to
the conceptual architecture of EOSES. Through LineFollower class, the user can
set system date, set map, choose operation mode, and test the application. Also,
it is the communication centre of the application, and where the application
starts execution.
83
• Follower class: This class is in charge of running the robot in Training Mode. It
guides the robot to find a path and follow that path until it ends. Then it records
the information of the path just followed, and translates the information into
SOEKS for storage. This includes some of the work from the Management
Layer in the EOSES conceptual architecture.
• FinderUsingSOEKS class: When the robot has been trained, the user can
switch the Line Follower from Training Mode to Testing Mode. In Testing
Mode, there are two execution manners: using experience or not using. The
FinderUsingSOEKS is developed to test whether the robot can pick the right
path and follow by using previous experience stored as SOEKS in the memory.
It also performs some functionality of the Management Layer, plus some
functionality of the Knowledge Repository Layer in the EOSES conceptual
architecture.
• Finder class: The Finder class is developed to test how long it takes for the
robot to reach the destination (i.e. the red spot on maps) without using previous
experience. It basically guides the robot in scanning the map, until the red spot is
detected. To avoid the robot running off map, walls along the borders of the map
are built. By using an ultrasonic sensor, the robot can measure the distance to
wall so that it detects the border of the map.
4.3
Experiments and Results
The Line Follower application is developed based on the initial experimental
prototype introduced in section 3.4. In previous experiments, three of the five main
84
features of the EOSES (i.e. Adaptability, Compactness and Efficiency, and
Configurability) were first examined. However, the initial prototype was not fully
developed, as it only printed out experience query time on LCD, but did not reuse
this experience in action. This section presents a set of experiments implemented on
the fully developed Line Follower application to test the usability and performance
of the EOSES to robotics.
4.3.1
Experiment Goal and Design
The goal of the initial experiments is to validate the usability of the EOSES to
robotics. Also, the performance of using EOSES services in self-learning and selfcontrol are tested.
Two 70cm×100cm maps are made for the experiments, namely Map 1, and Map 2
(see Figure 13). There are three paths drawn in different colours (green, blue, and
black) on each map, and every path finishes at a terminal spot. In the experiments,
the assumption is that the robot starts at position A, with spot B as the potential
destination for the robot in Testing Mode. To distinguish different terminal spots,
spot B is drawn in red (the destination), while the other two spots are drawn in
yellow (the normal terminal spots).
85
Figure 13. Experiment Maps for Line Follower
By following the first tests, this experiment is designed to mainly focus on two
questions: 1) Whether the EOSES application can help the robot make decisions
based on previous experience; and 2) What is the performance of reusing previous
experience. For answering the second question, the performance between using
previous experience and not using previous experience in the same application (i.e.
the Line Follower application) was compared. Also, another performance
comparison between the Line Follower and two similar systems on response time is
illustrated as a reference at the end of this section.
To compare the performance between using previous experience and not using
previous experience: First, the robot is set on Training Mode to learn the paths the
map one by one. After training, the robot is set on Test Mode, in which two options
are set for choosing - using previous experience or not using previous experience:
i)
Using previous experience. When using previous experience, the robot will
query its Decisional DNA Repository first before taking any action. Then the
robot finds out which path reaches the destination from querying its
86
experience. Finally, it detects the paths’ colour in order to choose the right one
and once the robot finds the right path, it will follow that path and reach the
destination.
ii) Not using experience. This option is designed to test the robot’s performance
when not using experience. With test results from previous option and this
option, the comparison of the difference between using and not using
experience can be made. A comprehensive map-scanning program was
developed for supporting this option. When the robot does not use experience
or there is no proper experience available, the robot will scan the map until it
finds the red spot (i.e. the destination).
4.3.2
Experimental Results
A) Results of using previous experience and not using previous experience:
A set of experiments testing the Line Follower’s performance of using experience
and not using experience were run 30 times on both map 1 and map 2. Figure 14 and
Figure 15 illustrate the details of time consumption for the tests on map 1. As we can
see, the time consumption is much less and more stable when using previous
experience in comparison with not using previous experience.
87
Using Experience
Not Using Experience
450000
Time Consumption
(milliseconds)
400000
350000
300000
250000
200000
150000
100000
50000
0
Figure 14. Time Consumption Comparison on Map 1
450000
Time Consumption
(milliseconds)
400000
350000
300000
250000
Using Experience
200000
150000
100000
50000
0
Min Time
Avg.Time
Max Time
Figure 15. Time Consumption Overall Comparison on Map 1
Figure 16 and Figure 17 show the tests result on map 2. By using previous
experience the robot could reach its destination much quicker than when not using
previous experience. Also, the robots’ performance was more stable in finding its
destination when previous experience was used.
88
Using Experience
Time Consumption
(milliseconds)
300000
250000
200000
150000
100000
50000
0
Figure 16. Time Consumption Comparison on Map 2
Time Consumption
(milliseconds)
300000
250000
200000
Using Experience
150000
100000
50000
0
Min Time
Avg.Time
Max Time
Figure 17. Time Consumption Overall Comparison on Map 2
B) Results of comparison with a similar system on response time
Response time is always one of the most important issues when referring to
experience-based applications. In order to evaluate the performance of the EOSES
89
applications, the comparison on response time was made between the Line Follower
and two similar systems.
The Random Path Planner (RPP) developed by Barraquand and Latombe (1991)
creates a potential function based on the destination configuration and particular
running environment. This function is then used for searching a path in the
configuration space. Caselli and Reggiani (2000) use RPP in their new approach, the
Experience-based Randomized Path Planner (ERPP) is used for faster path planning.
This new approach stores and reuses the best paths found so far by RPP in previous
planning tasks. In this study (Caselli and Reggiani, 2000), the performance of ERPP
and RPP are compared on an 8 processor Silicon Graphics Onyx2 Infinite-Reality, a
CC-NUMA machine with 650 MB/sec memory bandwidth and 711MB/sec
interprocessor communication bandwidth. Each Onyx2 processor is a 195 MHz
MIPS R10000 with 4 MB cache coupled in processor pair with 1 GB of memory.
For Line Follower application, the response time was measured from the
beginning of a query to the end of the query: First, a timer was set to count time
when a query starts and then the timer stopped counting time when the query was
processed and returned with query result. In this way, the response time was tested
and measured in two different application settings and in two different cases
respectively. The two application settings are: Query the First, and Query the Best.
When the application is set for ‘Query the First’, it returns the result immediately
once it finds the first similar experience can solve the current problem. While for
‘Query the Best’ it explores all experience stored, and comes up with the most
suitable experience for solving the current problem. Moreover, for both settings,
there are best cases and worst cases – in best cases, the first experience explored is
90
the one needed, and in worst cases, it is the last experience.
The tests were
undertaken in both settings and in both cases, 10 times each. Finally, the response
times for Line Follower application was taken as 2087.75 milliseconds, which is the
average time consumption from these two different settings (see Table 1).
Table 1. Response Time of Line Follower
Query the First
Query the best
Response Time
Best Case
264 ms
2682 ms
1473 ms
Worst Case
2705 ms
2705 ms
2705 ms
Average
1482 ms
2693.5 ms
2087.75 ms
For another two similar systems, the response times are testing results reported in
the study (Caselli and Reggiani, 2000). However, due to task complexity difference
and platform difference, this comparison is used for reference only (see Table 2).
Table 2. Response Time Comparison
Approach
RPP
ERPP
Line Follower
Average Response
Time
12.39 sec
4.68 sec
2.09 sec
Processor
195 MHz R10000
x7
195 MHz R10000
x7
48 MHz ARM7
x1
4.4
Summary
In this chapter, the case study of applying EOSES to robotics is presented.
According to the features of robotics, an EOSES application, called Line Follower, is
91
designed and developed under the EOSES Server-based Model, and the performance
of Line Follower robot is tested.
Firstly, a theoretical background on robotics is presented at the beginning of this
chapter, along with some practical robotic applications in the fields of healthcare,
autonomous, and manufacturing. Secondly, the case study of EOSES applied robot,
the Line Follower, is introduced in detail: starting with the application description,
followed by the robot hardware composition, and finally, the application design and
implementation are discussed.
At the end of this chapter, a set of experimental tests and results are outlined in
figures and tables. In the experiments, two questions about the case study, the
EOSES applied robot, are examined and answered. These two questions are: 1)
Whether the EOSES application can help the robot make decisions based on previous
experience; and 2) What is the performance of reusing previous experience. Through
a set of well-designed experiments the test results are gained, which shows that the
EOSES does help the robot make decisions by capturing and reusing experience
through its operation. Also, the test results show that by reusing previous experience,
the Line Follower can perform much quicker and more stable than when it does not
reuse previous experience. In addition, another comparison on response time is
illustrated in tables between the Line Follower and two similar systems as a reference
for the Line Follower’s performance.
92
5
A Stand-alone Application: EOSES
Applied to Digital TV
As mentioned in section 2.4 of chapter 2, the Stand-alone Model is another model
of the two conceptual models designed to handle different situations in the
Experience-Oriented Smart Embedded System (EOSES).
Unlike the conceptual
Server-based Model, in which one single task is accomplished based on cooperation
of several different embedded systems, the conceptual Stand-alone Model is
93
designed for situations that given tasks are finished in a single embedded system, for
example, watching a movie on a TV set.
In recent years, digital devices have been built increasingly powerful. They have
fast processing units, large memory, and network capability. Due to their capability,
together with Web 2.0 technology and mobile apps, these embedded systems are
widely used in our lives. Through collecting and processing data from activities
when using these devices, valuable information and knowledge can be obtained. By
reusing this knowledge, organizations and individuals can learn from past activities
so that to improve their performance in future similar situations. Moreover, devices
also can reuse this knowledge to assist its user to make better decisions, and quickly
adapt to new situations. Thus, developing new tools for enabling knowledge
management among these devices becomes an interesting and exciting research
opportunity.
This chapter presents the case study of applying EOSES to digital TV with Standalone Model. At the beginning of this chapter, a theoretical background on digital TV
and Interactive TV is introduced, along with some platforms and approaches aiming
for TV services enhancement. Then, the application of EOSES applied to digital TV
is explained with its system architecture. Following in section 2, there are three
sections illustrating three experimental prototypes and test results for improving
movie-viewing service, live TV viewing service, and the use of fuzzy logic to
generate TV user profiles for better user experience, respectively. At the end of this
chapter, brief summaries are drawn.
94
5.1
Background and Motivation
As the progress of digitalization and computerization affects our daily life, the
digital TV is becoming intelligent and interactive, and even becoming a computer.
Many companies and organizations have involved into the development and
implementation of intelligent and interactive TV, like Sun Microsystems, the Digital
Video Broadcasting Project (DVB), Google, and Apple, just to mention a few. In the
meantime, several technologies, such as Java TV (Morris 2002), Google TV (Google
2012), and Multimedia Home Platform (MHP) (Vrba, Cvrk, & Sykora 2006), have
been provided.
Thanks to these existing platforms and solutions, developers can add functions
enabling enhanced digital TV services and better user experience. By capturing the
user’s TV viewing habits and preferences, the TV content provider can offer the user
customized programme suggestions, and tailored advertisements.
5.1.1
Digital TV
Digital television (DTV) is the television broadcasting system that uses digital
signals to transmit program contents. DTV not only delivers distortion-free audio and
video signals; more importantly, it offers much higher radio spectrum efficiency than
analog television does. DTV can also seamlessly integrate with other digital media,
computer networks, and communication systems, enabling multimedia interactive
services and data transmission (Wu et al. 2006).
A) Formats and Bandwidth
Digital television supports a range of different picture formats defined by the
95
combination of interlacing, size, and aspect ratio (width to height ratio). With digital
terrestrial television broadcasting to the world, the range of formats can be broadly
divided into two categories: Standard Definition Television (SDTV) and Highdefinition Television (HDTV). These terms by themselves are not very precise, and
many subtle intermediate cases exist (Wikipedia - Digital Television 2012).
SDTV, by comparison, may use one of several different formats taking the form
of various aspect ratios depending on the technology used in the country of
broadcast. For 4:3 aspect-ratio broadcasts, the 640 × 480 format is used in NTSC
countries, while 720 × 576 is used in PAL countries. For 16:9 broadcasts, the 704 ×
480 format is used in NTSC countries, while 720 × 576 is used in PAL countries.
However, broadcasters may choose to reduce these resolutions to save bandwidth
(e.g., many DVB-T channels in the United Kingdom use a horizontal resolution of
544 or 704 pixels per line) (Latest snapshots - Freeview/DTT bitrates 2012).
HDTV, one of several different formats that can be transmitted over DTV, uses
different formats, amongst which: 1280 × 720 pixels in progressive scan mode
(abbreviated 720p) or 1920 × 1080 pixels in interlace mode (1080i). Each of these
utilizes a 16:9 aspect ratio. (Some televisions are capable of receiving an HD
resolution of 1920 × 1080 at a 60 Hz progressive scan frame rate — known as
1080p.) HDTV cannot be transmitted over current analog channels.
B) Standards
Currently, there are three main DTV standard groups (Wu et al. 2006):
1) The Digital Video Broadcasting Project (DVB), a European based standards
96
organization, which developed the DVB series of DTV standards, standardized by
the European Telecommunication Standard Institute (ETSI) (DVB 2012).
2) The Advanced Television Systems Committee (ATSC), a North American
based DTV standards organization, which developed the ATSC terrestrial DTV
series of standards. In addition, the North American digital cable TV standards now
in use were developed separately, based on work done by Cable Television
Laboratories (Cable Labs) and largely codified by the Society of Cable
Telecommunications Engineers (SCTE) (ATSC 2012).
3) The Integrated Services Digital Broadcasting standards (ISDB), a series of
DTV standards developed and standardized by the Association of Radio Industries
and Business (ARIB) and by the Japan Cable Television Engineering Association
(JCTEA) (ARIB 2012).
C) Reception
There are various ways to receive digital television. One of the oldest means of
receiving DTV (and TV in general) is using an antenna. This way is known as
Digital Terrestrial Television (DTT) (Wikipedia - Digital Terrestrial Television
2012). With DTT, viewers are limited to whatever channels the antenna picks up.
Signal quality will also vary.
Other ways have been devised to receive digital television. Among the most
familiar to people are digital cable and digital satellite. In some countries where
transmissions of TV signals are normally achieved by microwaves, digital
Multichannel Multipoint Distribution Service (MMDS) (IEEE 2012) is used. Other
97
standards, such as Digital Multimedia Broadcasting (DMB) (World DMB 2012) and
Digital Video Broadcasting - Handheld (DVB-H) (Reimers, U. H. 2006), have been
devised to allow handheld devices such as mobile phones to receive TV signals.
Another way is Internet Protocol TV (IPTV) (Yarali & Cherry 2005), which is
receiving TV via Internet Protocol, relying on Digital Subscriber Line (DSL) or
optical cable line.
5.1.2
Interactive Television
Interactive television (generally known as iTV) describes a number of techniques
that allow viewers to interact with television content and services; it is an
evolutionary integration of the Internet and DTV (Schwalb 2004).
The most exciting aspect of an interactive TV is the ability to run applications that
have been downloaded as part of the broadcast stream: this is the major difference
between a basic digital TV box and an interactive TV system. In order to support and
enable interactive applications, the receiver is required to support not only the
implementation of APIs needed to run the applications, but also the infrastructure
needed to inform the receiver which applications are available and how to run them.
Interactive TV has drawn attention from many researchers, organizations, and
companies, and there have been a few efforts and solutions offered by them. Java TV
and Multimedia Home Platform are the two most popular and vibrant techniques in
this field (Morris 2002; Vrba, Cvrk, & Sykora 2006).
A) Java TV
The Java TV is a Java-based software framework designed for supporting digital
98
TV platforms from Sun Microsystems. It brings together a number of the common
elements that are needed in a digital TV platform. These include the core application
model and lifecycle, access to broadcast services (either via Java TV itself or via the
Java Media Framework) and access to service information (Morris 2002).
Most importantly, Java TV is not bound to a specific set of standards for digital
TV. Java TV is explicit, pure, and independent. Because of this, it works equally well
with many solutions for digital TV, such as ATSC solutions, OpenCable solutions,
and DVB-based systems. It gives Java TV a very strong advantage in that
applications written to use Java TV APIs will work on any platform that supports it,
rather than being tied to a specific broadcast system (Morris 2002).
B) Multimedia Home Platform
Multimedia Home Platform (MHP) is an open standard middleware system
designed by the DVB Project for enhanced and interactive digital television (Vrba,
Cvrk, & Sykora 2006).
The MHP enables the reception and execution of interactive, Java-based
applications on a TV-set. Interactive TV applications can be delivered over the
broadcast channel, together with video and audio streams. These applications can be,
for instance, games, e-mail, information services, interactive voting, shopping or
SMS.
MHP defines a generic interface between interactive digital applications and the
terminals, on which those applications execute. This interface decouples different
applications of a provider from specific hardware and software details of different
99
MHP terminal implementations. It enables digital content providers to address all
types of terminals ranging from low-end to high-end set top boxes, integrated digital
TV sets, and multimedia PCs. The MHP extends the existing DVB open standards
for broadcast and interactive services in various broadcasting networks, like satellite,
cable, or terrestrial.
5.2
EOSES Applied to Digital TV
Nowadays, digital TV is rolling on in full force. Thanks to its capability of
transmitting digital data along with the audiovisual contents, the TV program
providers can interact with their viewers by offering customized applications, which
run at their viewers’ set-top boxes or inside the TV. In order to capture, reuse, and
share viewers’ TV-viewing experience, and improve digital TV products’ user
experience, the EOSES is applied to digital TV, called the Decisional DNA Digital
TV (DDNA DTV). This also examines and demonstrates the five main features of
the EOSES. In addition, this application is designed and developed based on the
EOSES conceptual 4-layer architecture, using Stand-alone Model.
The DDNA DTV is designed as an experiential knowledge-based framework that
allows digital TV sets, set-top boxes, and Smart TV systems to capture, reuse, and
share users’ TV viewing experiences. Generally, the DDNA DTV deals with two
kinds of content on TV: movies, and programmes. It uses the SOEKS and Decisional
DNA to store the users’ viewing experience. The DDNA DTV consists of the User
Interface, the System I/O (input/output), the Integrator, the Prognoser, the Convertor
and the Decisional DNA Repository (see Figure 18).
100
Figure 18. System Architecture of DDNA DTV
• User Interface: User Interface is developed to interact with the user. In
particular, the user can control, set and configure the system, select services,
give feedback, and interact with the service providers by using User Interface.
• System I/O: System I/O allows DDNA TV to communicate with its domain of
operation (i.e. a TV player or a Smart TV). The System Input tells DDNA TV
which service is selected, for example, what movie is playing, what feedback
was given, etc, while the System Output sends information to its domain, like
101
programme suggestions based upon the user’s viewing experience.
• Integrator: Integrator is the place where the scenario data is gathered and
organized. In our case, we link each experience with a certain scenario
describing the circumstances under which experience is acquired, such as the
system time, name of a selected service, user input, and other service
information. Integrator gathers and organizes the scenario data, and then sends
them to the Prognoser for further processing.
• Prognoser: Prognoser is in charge of sorting, analysing, organizing, creating
and reusing experience. It sorts data received from the Integrator, and then
analyses and organizes the data. Finally, it interacts with the Decisional DNA
Repository and the Convertor in order to store or reuse experience according to
different tasks.
• Convertor: Convertor translates organized information into SOEKS knowledge
statements. Also, Convertor retrieves knowledge from SOEKS, and sends
retrieved knowledge to Prognoser for reusing.
• Decisional DNA Repository: Decisional DNA Repository is the core software
component that stores and manages user’s experiences. It is composed of a
Repository Manager and numerous Chromosomes:
a) Repository Manager: Repository Manager is the interface of the Decisional
DNA Repository. It answers operational commands sent by Prognoser and
manages Chromosomes.
b) Chromosomes: Chromosome gathers and stores Decisional DNA genes within
102
each category, capturing the user’s decisional “strategies”. A Decisional DNA
gene carries a single decisional event (i.e. a SOE) and can be represented by a
set of XML tags introduced by Maldonado Sanin (2007) to store Decisional
DNA in XML files.
5.3
DDNA DTV Prototype 1.0
This section presents the initial version of the prototype for the DDNA DTV
(DDNA DTV 1.0), which captures and reuses the user’s movie-viewing experience
to provide smart movie recommendations. Also, the results obtained in the first set of
experiments are illustrated at the end of this section.
5.3.1
In DDNA DTV 1.0, the EOSES is applied to Java TV in order to capture, and
reuse the TV user’s movie-viewing experience. As introduced in section 5.1.2, Java
TV, as a Java-based software framework, is designed for supporting digital TV
platforms from Sun Microsystems. A number of common elements that are needed in
a digital TV platform are combined into Java TV. Most importantly, Java TV is not
bound with specific digital TV standards - it is explicit, pure, and independent, which
enables Java TV to work equally well with a number of different solutions for digital
TV. As a result, Java TV is thus been chosen as the platform for this prototype.
As the initial prototype version, the DDNA DTV 1.0 was developed using the
Java TV SDK working with NetBeans 6.8 on a DELL Latitude ES400 laptop. In
addition, the SOEKS API was used in order to manipulate the SOEKS structures. As
presented in Chapter 2, the EOSES defines four basic layers: Application Layer,
103
Interface Layer, Management Layer, and the Knowledge Repository Layer. The
initial version of the application includes basic features and functionality for the
EOSES. Figure 19 shows the initial class structure for this application.
Figure 19. Simplified Class Diagram for DDNA DTV Prototype 1.0
The UserInterface provides the user with the movie recommendation interface that
shows movie suggestions, and allows the user to choose movies to watch. As the user
keeps selecting and watching movies, Integrator starts to collect the user’s movie
104
choices, and sends this information to Prognoser. The Prognoser then abstracts and
captures the user’s movie-viewing experience. Once this system has more than a
certain amount (which is configurable) of the user’s movie-viewing experience, it
reorganises the movie recommendation interface to suggest movies according to the
previous gained experience.
5.3.2
Simulation and Experiments
At this very initial stage, the prototype is designed to capture the viewer’s
watching preference for movie type only, and reuse captured experience for movie
suggestions. To test this prototype, a set of experiments was made in a simulation
environment in Java TV SDK. The assumption is that there are five types of movies,
namely action, adventure, animation, comedy, and crime. Each movie is recognised
by its type and an ID number, for example Action1, Comedy2 and there are 20
movies for each type. At the beginning, the system recommends movies by movie
type equally: two movies for each movie type (see Figure 20).
Figure 20. Screenshot of the Default Movie Suggestion Interface
105
Figure 21. A SOEKS of a Watched Movie
As the viewer watches movies, the system captures the viewer’s movie-viewing
experience which is carried by seven variables: Movie Name, Director, Watch Date,
Watch Time, Ranking, Type, and Viewer. Movie Name and Director are used to
indicate which movie the viewer watched. Watch Day and Watch Time store date and
time when the movie is watched. Ranking shows how the viewer liked the movie.
106
Type illustrates what kind of movie it is. Viewer saves the name of user. These seven
variables are gathered and organized by the Integrator and then sent to the Prognoser;
and finally, they are stored as a SOEKS in XML format (see Figure 21). Once the
system has more than ten SOEKSs (this number is configurable), it begins to analyse
the
viewer’s
movie-viewing
preference,
and
gives
the
viewer
better
recommendations according to analysed settings.
In the experiments, a viewer is set as: Tom who likes to watch action movies
every Saturday night as shown in Table 3.
Table 3. Tom’s Movie Viewing Records
Movie Name
Watch Day
Watch Time
Ranking
Type
Action1
Saturday
19:35
7
Action
Action2
Saturday
20:02
9
Action
Action3
Saturday
20:13
8.5
Action
Action4
Saturday
19:42
8.7
Action
Action5
Saturday
21:07
8.6
Action
When the Prognoser recommends new movies to Tom, it retrieves Tom’s previous
movie-viewing experience from the Decisional DNA Repository, and analyses the
experience according to system settings. In this experiment, the system is set to
recommend a movie based on the preference order of movie types, and what day in a
week the viewer usually watches them.
As mentioned above, every time, the system recommends 10 movies to Tom.
After learning from Tom’s movie-viewing experience, the system starts to reallocate
the quota of each type of movie for the new 10-movie recommendation list: it adds
107
more movies in from Tom’s preferred movie types, while reducing the amount of
movies from those types that Tom does not like. Equation (1) demonstrates how the
system allocates the quota for each movie type in the new movie recommendation
list.
N = ( T × 100 / D + 5 ) / 10
(1)
N represents how many movies should be recommended from a certain movie
type; T represents how many movies have been watched from a certain movie type
on a specific weekday; D represents the total amount of watched movies on that
weekday. For example, Tom watched 11 movies in total on Saturdays, and 5 of these
movies were action movies. Therefore, there should be five action movies in the next
recommendation list: (5 × 100 / 11 + 5) / 10 = 5.
As assumed, during a few weeks of capturing experience, the system learns and
knows that Tom watched 5 action movies, 2 adventure movies, 1 animation movie, 2
comedy movies, and 1 crime movie on Saturdays. As a result, the system will
recommend 5 action movies, 2 adventure movies, 1 animation movie, 2 comedy
movies, and 1 crime movie to him for the coming Saturdays. Figure 22 shows a
screenshot of this newly generated movie recommendation list for Tom.
As the experiment results show, the DDNA DTV prototype 1.0 is capable of
capturing and reusing its viewer’s movie-viewing experiences. Most importantly, the
prototype shows that the EOSES can be applied to digital TV, and enable better user
experience of watching TV, as well as potential enhanced TV services.
108
Figure 22. A Newly Generated Movie Recommendation List for Tom
Since the concept of EOSES research is at its early stage, and the main concern of
developing this prototype is to test and demonstrate EOSES’s five main features.
Therefore, this application is very initial, and only examined in a simulation
environment. In the next version, the EOSES is applied to live TV, which makes the
prototype more practical.
5.4
Based on the initial experimental prototype introduced in section 5.3, the DDNA
DTV prototype 2.0 was developed. In previous experiments, the five main features of
the EOSES were first examined. However, the initial prototype was not fully
developed, which only supported very basic movie-viewing experience management,
and did not apply to live TV viewing. This section presents the further developed
version of the prototype called DDNA DTV 2.0 along with its implementation and
109
experiments.
5.4.1
In this version of the prototype, EOSES is applied to live TV viewing, which
provides live TV viewers smart assistance while watching TV. It can suggest
interesting channels to the viewer according to the viewer’s past TV viewing
experience; also it reminds the viewer to watch upcoming programmes/shows that
the viewer usually watches. This research extension explores the possibilities and the
practical implementations for applying EOSES to digital live TV on Android based
embedded systems (Android 2012).
Android is a software stack that consists of a Linux-based operating system, a set
of key applications, and middleware (Android 2012). While Android is primarily
designed for mobile devices such as smartphones and tablets, its nature of open and
customizable architecture allows Android to be used on many other electronics
systems and products (Android Authority 2012). Smart TV is a very good example
of the above usage (Steve 2012). Since Google launched its smart TV product –
Google TV (Google 2012), increasingly more TV manufacturers have entered this
field with their products: Samsung (2012), LG (2012), and Panasonic (2012), just to
mention a few.
However, none of these smart TV products applies intelligent techniques to
support viewer’s decision-making process, and most of them even do not assist live
TV playback. Actually they are more like larger-sized Android tablets rather than TV
sets. To change the above perception the DDNA DTV 2.0 is thus designed and
developed, which represents the first step in an attempt of adding “intelligence” into
110
this field.
5.4.2
System Design
Following the previous prototype, the second version was mainly developed on
functionality for smart management of users’ TV-viewing experiences. In order to
make the DDNA DTV compatible with various TV systems, two new classes were
designed and introduced into the DDNA DTV; these two classes are DDNATVServer
class and DDNATVClient class. With these two classes, the prototype 2.0 can work
as an open API that enables developers to apply DDNA DTV technology to their TV
products, offering knowledge management services to their users. Moreover, the
DDNARManager class was defined and implemented in this version of the
prototype.
Figure 23 presents a simplified class diagram that contains new functionality for
the new experimental prototype. The features for smart experiential knowledge
management were implemented in the EOSES’s four layers defined in Chapter 2.
The DDNATVClient class is the interface for accessing DDNA DTV services. By
importing DDNATVClient class and creating an instance of it, third-party software is
ready to use the services provided by the DDNA DTV. For reminding the user to
watch preferred programmes/shows, the third-party software can also create an
instance of the DDNATVServer class, which will send a programme suggestion
notification to the user’s device when an upcoming programme was often previously
watched.
111
Figure 23. Simplified Class Diagram for DDNA DTV Prototype 2.0
Through DDNATVClient and DDNATVServer classes, the third-party software is
able to access DDNA DTV’s experiential knowledge management services. As a
consequence, the third-party software can capture its users’ TV-viewing experience,
and reuse this experience for smart programme suggestions.
5.4.3
Implementation and Experiments
The second prototype was developed using Java 6 with Android NDK (Native
Development Kit) in Eclipse, and implemented on a Toshiba Shriving Android tablet
(PCWorld 2012). Currently, there are some technique difficulties for receiving and
playing live TV directly on Android devices (Sven 2012). Some complex API is
essential for using DTV tuners, but API is not standardized for Android devices;
also, there are no DTV features included in Android Java API (Vidakovic et al.
112
2012). Thus, instead of plugging a DTV receiver directly in an Android device, we
use a laptop to work with the DTV receiver. By running DTV streaming server
software, the received TV stream is processed and broadcasted to a local network.
Finally, the DTV streaming client running on the Android tablet receives and plays
the TV stream through the WIFI (see Figure 24).
Figure 24. Implementation of the DDNA DTV Prototype 2
In this implementation, a Hauppauge Nova-T USB TV stick (Hauppauge 2012) is
used to receive the digital TV signal. It works with TVHeadend (2012) DTV
streaming server on a laptop. The TVHeadend streams the received digital TV
content to the local network. At the Android side, we use the DDNA DTV-combined
TVHGuide (2012) to receive and play DTV streams on the Toshiba Shriving tablet
running the Android 3.2 operating system. The TVHGuide is a client for the DTV
streaming server – TVHeadend. It is an open-source Android application developed
in Java language. Because both DDNA DTV and TVHGuide are developed in Java
language, it is not hard to make them work together: by importing the DDNA DTV
into TVHGuide project, and instantiating the objects of DDNATVClient class and
DDNATVServer class of DDNA DTV in TVHGuide, the TVHGuide is ready to
communicate with the DDNA DTV.
113
It took several weeks to capture the viewer’s TV watching experience during the
experiment, and more than 60 SOEKSs were initially captured. The DDNA DTV
captured the viewer’s TV watching experience by storing each single TV watching
event. A single TV watching event was taken as the viewer watched a channel for
more than ten minutes. And there were nine variables recorded for each single TV
watching event: Channel ID, Channel Name, Programme, Watch Day, Start Time,
End Time, Type, Description, and Viewer. Channel ID and Channel Name indicates
which channel was watched. Programme stored the name/title of the program. Watch
Day recorded the day of week the channel was watched. Start Time and End Time
stored the start and stop time of the program. Type illustrated the type of program.
Description stored the introduction of the program. Viewer saved the name of the
user.
Once a single TV watching event was taken, those variables were gathered and
sent to the Prognoser by the Client from TVHGuide; then, Prognoser generated
SOEKS statements, and sent the SOEKS statements to the DDNA Repository
Manager; eventually, this TV watching event was stored as a SOEKS in XML format
similar to the SOEKS introduced in section 5.3.
When there is SOEKS in the repository, the DDNA DTV begins to analyse the
viewer’s watching habits, and suggests programs that the viewer usually watches. In
particular, Prognoser retrieves those stored watching experiences from the Decisional
DNA Repository according to Watch Day and it collects all the SOEKS with the
same Watch Day value. Then, it analyses and sorts those experiences by Start Time.
After that, it calculates the similarities of the Start Time between each SOEKS, and
finds out the most watched program for a certain time period. The similarity is a
114
value used to illustrate how similar two Variables are; if the value is closer to zero,
the two Variables are more similar (Sanin & Szczerbicki 2009a). At the end,
according to current time, the Prognoser generates a next-12-hour programme guide
consisting of five most watched programs to the viewer (see Figure 25).
Figure 25. The Next-12-hour Programme Guide
This guide only appears when DDNA DTV is starting up. In the meantime, the
TVHGuide is loading a channel list from local network; hence the background is
black. When the channel list is loaded, the viewer is capable of choosing any channel
he/she wants to watch. Figure 26 shows a screenshot of the interface of TVHGuide
with a channel list loaded.
115
Figure 26. Screenshot of the Interface of TVHGuide with a Channel List Loaded
116
Furthermore, the DDNA DTV 2.0 also enables TVHGuide to remind the viewer
to watch the suggested upcoming programmes. Figure 27 shows a notification sent
by the DDNA DTV to remind the viewer that there will be a preferred programme on
another channel while live TV is playing.
Figure 27. A Programme Notification Sent by DDNA DTV While Live TV Is Playing
5.5
In the latest version of the prototype, the DDNA DTV is combined with the
TVHGuide, and implemented on a Toshiba Shriving Android tablet. This prototype
enables TVHGuide to capture the user’s live TV viewing experience, and reuse such
experience to suggest programmes to the user. Based on the prototype 2.0, in the
third version of the prototype for the DDNA DTV (DDNA DTV 3.0), functionality
117
of uncertainty modelling and human reasoning is developed. Fuzzy logic deals with
reasoning that is approximate rather than exact and fixed. By applying fuzzy logic
technology to DDNA DTV, human reasoning can be introduced to DDNA DTV’s
knowledge-related processes enabling uncertainty modelling and providing better
inference results. This section presents the DDNA DTV 3.0 along with its initial
implementation that generates the television user’s profile utilising of fuzzy logic. A
user profile refers to the user’s basic information, such as gender, age principles, and
profession.
By using the generated user profile, DDNA DTV services can be
improved in many ways, like customized and adaptable programme suggestions,
classified experience sharing, and tailored advertising.
5.5.1
Fuzzy Logic
The concept of fuzzy logic began with the 1965 proposal of fuzzy sets by Zadeh
(1965). Rather than exact and fixed values used in traditional logic theory, where
binary sets have only two values (i.e. true and false), fuzzy logic may have many
values in a range between 0 and 1. Therefore, fuzzy logic is most suitable for
modelling the uncertainty in human reasoning, e.g. warm, small.
There are three fundamental concepts in fuzzy logic theory, namely fuzzy sets,
linguistic variables, and possibility distributions (Du & Swamy 2006). Let U be a
collection of values between 0 and 1, which could be discrete or continuous. U is
called the universe of discourse, or simply the universe, and u represents the generic
element of U.
Fuzzy Set: a fuzzy set F in a universe U is determined by its membership function
(MF) μF where μF (x) ∈ [0, 1].
118
F = � �u, µF (u)�� u ∈ U }
(2)
A fuzzy set can also be represented by
F=�
∑ni=0
∫U
µF (ui )
ui
µF(u)
u
,
,
if U is discrete
(3)
if U is continuous
Linguistic Variable: a linguistic variable is a variable whose value can be defined
quantitatively using an MF and qualitatively using a linguistic term (Du & Swamy
2006). For example, the speed of a car is a linguistic variable, whose value can be
slow, moderate, and fast.
Possibility Distribution: a possibility distribution is the elastic constraint on the
possible values of the variable imposed when a linguistic variable is assigned to a
fuzzy set.
Fuzzy logic is an effective tool for modelling the uncertainty in human reasoning.
In fuzzy logic, the knowledge of human beings is codified by means of linguistic IFTHEN rules that build up the fuzzy inference systems (FISs). FISs can be used in
many areas such as in data analysis, control and signal processing.
5.5.2
Based on the previous version, two new concepts, i.e. User Profile and User
Group Classification, are introduced to the DDNA DTV 3.0 in order to enable
knowledge sharing among users with similar viewing interests, and assist users to
adapt to new DTV systems (such as when users move from one place/country to
another). As already discussed, the user profile refers to the user’s basic information,
119
such as gender, age, and profession. As Comstock et al. (1978) claimed that users
with a similar profile tend to have similar viewing interests and behaviours.
Reversing this logic, the DDNA DTV creates the user’s profile from their viewing
patterns. For example, if a user who watches TV usually on weekdays between 9:00
a.m. and 4:00 p.m., this user can be determined to be a househusband/housewife,
because it looks like this user doesn’t need to work outside the house. Thus, the user
group classification can be generated.
Once the users have been classified, their viewing patterns can be compared with
other users with similar profiles, and viewing suggestions can be made. For instance,
if User A is classified as a housewife, User A will contribute her TV viewing
experience to the Housewife group; additionally, User A will get TV programme
suggestions from the Housewife group. Furthermore, the User Profile and the User
Group Classification can help new users quickly adapt to a TV network through
group experience sharing. Aside from these, there is also potential for tailored
advertising.
5.5.3
Implementation and Initial Tests
To apply fuzzy logic technology to the DDNA DTV 3.0, the fuzzy inference
system (FIS) is introduced. The FIS is a systematic framework that uses fuzzy rules
to map between inputs and outputs, in which the data flow usually goes through three
procedures, namely Fuzzification, Fuzzy Inference, and Defuzzification (see Figure
28) (Du & Swamy 2006).
120
Figure 28. The Architecture of a Fuzzy Inference System
First, the input data is converted into suitable linguistic values or fuzzy sets in
Fuzzification. Then, the Fuzzy Inference process performs simulation of human
reasoning based on its Rule Base and Data Base. The Rule Base defines the mapping
relationship between inputs and an output using linguistic terms; also, it describes
reasoning policy by a set of linguistic IF-THEN logic. While in the Data Base,
necessary definitions of linguistic control rules and fuzzy data manipulation are
addressed (Du & Swamy 2006; Zadeh 1965). Finally, the Defuzzification produces a
crisp output for further use.
The core task of fuzzification is constructing an accurate membership function
(Du & Swamy 2006). In the DDNA DTV 3.0, TV audience related studies
(Comstock et al. 1978) are used to make the first attempt. By measuring when, what,
and how long per week a user usually watches TV, the users can be divided into four
groups: Children, Teenagers, Men, and Women.
In the implementation of the DDNA DTV 3.0, the jFuzzyLogic (2012), a fuzzy
logic tool that allows for intuitive use of fuzzy logic in Java based projects is used as
the FIS. After setting up the membership function according to studies carried out by
Comstock et al. (1978), the DDNA DTV 3.0 is capable of generating the fuzzy User
121
Profile and creating the User Group Classification for its user.
In the initial tests, an assumed user, Jerry, mimics how a child watches TV. After
capturing Jerry’s TV viewing experience for several weeks, the DDNA DTV
generates a fuzzy User Profile for Jerry. Figure 29 shows one of the results of the
fuzzification tests. As the charts illustrates, according to Jerry’s viewing patterns, the
DDNA DTV ascribes Jerry to the ‘Children’ category. And Jerry’s User Profile
could be like: Children = 20.26; Teenagers = 9.71; men = 3.99; women = 3.99.
Figure 29. Results of the Fuzzification Tests
5.6
Summary
In this chapter, the case study of applying EOSES to digital TV is presented.
According to the features of digital TV, an EOSES application, called Decisional
DNA DTV, is designed and developed under the EOSES Stand-alone Model, and
based on the EOSES conceptual 4-layer architecture. Three prototypes are introduced
122
during the development of the DDNA DTV, and for each prototype, there are
implementation and experiments details outlined.
Firstly, a theoretical background on digital TV and Interactive TV is introduced at
the beginning of this chapter, along with some platforms and approaches aiming for
TV services enhancement. Secondly, the application of EOSES applied to digital TV
is explained with its system architecture. Then, by following section 2, three sections
illustrating three experimental prototypes and test results for DDNA DTV are
presented:
In the prototype one, the DDNA DTV is initially implemented and simulated
with Java TV, called DDNA DTV 1.0. This prototype captures and reuses the user’s
movie-viewing experience to provide smart movie recommendations.
In prototype two, the DDNA DTV is developed to be more practical: it is able to
work with third-party TV products, and provide them with TV programme
suggestion capability by capturing and reusing users’ live TV viewing experience.
This prototype is implemented and tested on a Toshiba Shriving tablet with the thirdparty software TVHGuide in live TV playing environment.
In the prototype three, fuzzy logic technology is introduced to enable human
reasoning and uncertainty modelling for DDNA DTV. In the experiential
implementation, two new concepts, i.e. User Profile and User Group Classification,
are added to the DDNA DTV 3.0 in order to enable fast user adaptability and grouporiented knowledge sharing: by using fuzzy logic methods, this prototype can
generate user profile and group the user according to its profile. On the other hand,
by using user profile, the user can adapt to new TV systems quickly by getting TV
123
programme suggestions from users with similar user profiles.
In short, this chapter presents the case study of the EOSES Stand-alone Model,
and illustrates the EOSES’s features of Adaptability and Cross-platform Portability,
and Configurability through different prototype implementations.
124
6
Conclusions and Future Work
Knowledge as a valuable asset is becoming steadily important for organizations
and individuals. It helps organizations and individuals adapt rapidly and react
properly to today’s dynamic environment. Knowledge Management (KM) is the
discipline to enable organizations, and individuals more collectively and
systematically capture, store, apply and share their knowledge, to achieve their goals.
In recent years, embedded systems have been built increasingly powerful, and
have become widely involved in knowledge-related activities. Through collecting
125
and processing data from activities while we use these devices, valuable information
and knowledge can be obtained. By reusing this knowledge, organizations and
individuals can learn from past activities to improve their performance in future
similar situations. Moreover, these devices also can reuse knowledge to assist its user
in making more accurate and effective decisions, by adapting quickly to new
situations.
However, as mentioned in chapter 1 and chapter 2, there are three common
problems for applying previous proposed KM technologies to embedded systems, i.e.
1) they are too specific, 2) they do not have standard knowledge representation, and
3) they lack information sharing and exchange capabilities. Therefore, a new solution
is required to improve existing solutions and support mass customized autonomous
mechanisms on embedded systems. In this document, I propose the ExperienceOriented Smart Embedded System (EOSES) as a technology capable of achieving
this purpose.
6.1
Experience-Oriented Smart Embedded System
(EOSES)
alternative to support knowledge management for daily operations, especially on
embedded systems. This is achieved by incorporating with the Smart Knowledge
Management System (SKMS) proposed by Sanin and Szczerbicki (2008a).
The EOSES is designed as a new KM technology platform that allows its domain
of operation to capture experiential knowledge from day-to-day operations, and then
reuse the knowledge to improve the domain’s performance in similar situations.
126
Knowledge in the EOSES is stored as SOEKS, and organized as Decisional DNA.
This experience-oriented platform is mainly based on principles of Embedded
Systems, IT, and Knowledge Management. The objective behind this research is to
offer large-scale support for intelligent, autonomous, and coordinated KM on various
embedded systems.
As introduced in chapter 2 and chapter 3, the EOSES is engaged with five main
features in order to archive KM on different embedded systems. These five main
features are Experience-Oriented, Adaptability and Cross-platform Portability,
Compactness and Efficiency, Configurability, and Security and trust. All these
features are carried by a conceptual four-layer architecture of EOSES, which is
composed of Application Layer, Interface Layer, Management Layer, and
Knowledge Repository Layer. Through these four layers in the conceptual
architecture, the EOSES is able to integrate different organizational processes and
various devices as an organic system, and make extensive use of experience-oriented
knowledge management to provide appropriate autonomous capabilities to embedded
systems.
6.2
Achievements Overview
In order to bring ‘intelligence’ to embedded systems, and make a contribution to
the fields of KM and IT, a series of objectives are proposed in chapter 1 of this
thesis. The main goal of the EOSES is to develop a smart platform that would
support decision-making activities for embedded systems through gaining knowledge
by experiencing day-to-day operations.
The major contribution of this research is highlighted as the creation of such a
127
platform for different embedded systems to capture, evolve, and share the experience
of every decision made, as well as proposing an approach which can solve the
previous mentioned three common problems encountered by applying
previous existing KM solutions to embedded systems: 1) EOSES is not bound
to a specific kind of embedded system, it supports KM on various embedded
systems; 2) EOSES uses standard knowledge representation, i.e. SOEKS and
Decisional DNA proposed by Maldonado Sanin (2007); and 3) EOSES
supports knowledge sharing and knowledge exchange among similar or
different embedded systems.
A key difference with traditional approaches is that ‘knowledge’ inside the
EOSES is based upon the idea of Formal Decision Events presented by Sanin &
Szczerbicki (Sanin and Szczerbicki 2008a), which is extracted from capturing the
constant interaction between users and the software applications they use on a daily
basis. In other words, ‘knowledge’ in the EOSES is obtained from embedded
systems’ own experiences rather than from experts or from documents. That is why it
is called ‘experience-oriented’. The feature of learning-by-itself from own
experience also allows EOSES to be built independent, general, and applicable for
most embedded systems because most embedded systems have their own processes
from which knowledge can be extracted.
But, how does the EOSES support embedded systems for capturing, storing, and
evolving experience? This experience-oriented approach follows the macroprocesses defined by the SKMS (Sanin and Szczerbicki 2008a) to extract experiential
knowledge from a variety of embedded and mobile applications based on daily
128
operations, and integrates it into the system. All the experiences are stored in the
respective knowledge bases, as shown by the experimental prototypes outlined in
chapter 4 and chapter 5.
Moreover, thanks to the domain independence and flexibility provided by the
SOEKS (Sanin, Mancilla-Amaya, Szczerbicki, and CayfordHowell 2009), the
SOEKS could be tailored so that mass customization becomes possible in the
EOSES. As presented in chapter 4 and chapter 5, EOSES is applied to two totally
different embedded systems that have different design, different hardware, and
different software, pursuing different goals.
Another achievement of the research presented in this thesis is the proposal of two
conceptual models of EOSES. Based on the variety of embedded systems, the
EOSES has two conceptual models, i.e. server-based model and stand-alone model;
which allow the EOSES to be applied to different embedded systems, and meet with
different system requirements. In chapter 4, a case study of applying EOSES to
robotics is given. By using the server-based model, the EOSES can work with a set
of embedded systems and integrate information sent by these embedded systems to
the server for further processing. While in chapter 5, the stand-alone model is
presented, which deals with applications running on single embedded systems. By
using these two conceptual models, the EOSES is applicable for most embedded
systems in most situations.
In addition, the introduction of fuzzy logic technology brings to EOSES the
capability of human reasoning and uncertainty modelling. This can potentially
improve the inference performance of the EOSES. In section 5.5 of chapter 5, an
129
initial experimental prototype is presented, in which fuzzy logic is used to stereotype
and classify the user so that a series of services is enabled.
6.3
Future Work
Throughout this section, the guidelines for future work have been sketched by
analysing the platform’s initial goal and the platform’s current achievements. As a
result, a series of future research, improvements, and extensions can be completed.
The first step towards the improvement of the EOSES is development of the
remaining elements for the EOSES: as proposed in chapter 3, there should be four
Managers in the Management Layer, i.e. Configuration Manager, Plug-In Manager,
Protocol Manager, and EOSES Manager. However, in the initial stage of
development of the EOSES, these four elements are yet to be created. Continuous
development will allow for a complete deployment of such platform on a large-scale,
which can bring mass autonomous functionality to various embedded systems
without significant modification of embedded software. Also, the development of
these four elements will allow for mass customization and flexibility of applying the
EOSES technology to other platforms and other technologies. With the help of these
four managers, developers can define their own protocols, create new plug-ins,
expand the functionality of the EOSES, and configure the EOSES more easily and
flexibly.
Additionally, given the prospect of advancement of mobile devices, wireless
networking, and Cloud Computing, another idea for future work is the connection
and integration of the EOSES applications using wireless networking and Cloud
130
Computing technologies. This will improve the platform by facilitating knowledge
gathering and sharing among every possible source, and will allow users to access
the platform’s functionality anywhere and anytime. For example, a normal user
nowadays could have many devices that provide multimedia services: the user could
watch online videos through smartphones, watch TV show episodes on tablets, and
watch live TV programmes on TV sets. By using Cloud Computing technology, the
knowledge about the user’s TV viewing preference on different devices could be
shared among each other so that more precise and sufficient knowledge about the
user’s TV viewing preference can be gathered. Also, this knowledge then could be
very easily used on the user’s newly bought devices. Furthermore, as knowledge is
one kind of valuable asset, therefore, a market that trades knowledge as goods could
form in the future. Work on this topic of knowledge sharing is currently being
developed at the University of Newcastle (Mancilla-Amaya, Sanin, & Szczerbicki,
2010a), and the integration of such approach with the EOSES will make a greater
contribution for the development of decision-support systems.
Finally, as the EOSES is a technology approach that involves multiple
technological aspects, the previously presented ideas are just a guideline for the
future development of the EOSES, and are not intended to restrict the development
of the platform. Hopefully, other researchers will find more opportunities and
challenges for future work other than the ones exposed in this thesis.
131
ANNEXES
Annex 1.
Class Diagram for Line Follower
132
Annex 2.
SOEKS Class Diagram for Line Follower
133
Annex 3.
A Sample SOEKS for Line Follower
134
Annex 4.
Class Diagram for DDNA DTV 2
135
Annex 5.
API Class Diagram for DDNA DTV 2
136
Annex 6.
SOEKS Class Diagram for DDNA DTV 2
137
Annex 7.
A Sample SOEKS for DDNA DTV 2
138
139
REFERENCES
1. Akhras, G., 2000, ‘Smart Materials and Smart Systems for the Future’, Canadian
Military Journal, Vol.1, No. 3, pp. 25-32.
2. Almeida, C. (2008). Practical Experience Teaching Embedded Systems. ACM
SIGBED Review, 5(3), 3.
3.
Android
(2012).
Discover
Android.
Retrieved
December
2012
from
http://www.android.com/about/
4.
Android Authority (2012). Android Everywhere: 10 Types of Devices That
Android
Is
Making
Better.
Retrieved
December
2012
from
http://www.androidauthority.com/android-everywhere-10-types-of-devices-thatandroid-is-making-better-57012/
5.
ARIB (2012). The Association of Radio Industries and Business. Retrieved
December 2012 from http://www.arib.or.jp/english/
6. ATSC (2012), the Advanced Television Systems Committee. Retrieved December
2012 from http://www.atsc.org/cms/
7.
Barraquand, J. and Latombe, J. C. (1991). Robot motion planning: a distributed
representation approach. Robotics Research, 10, 628-649.
8. Bellinger, Gene. (1997). "Knowledge Management: Bah Humbug!" (Presentation
at Documation '97, San Jose, CA, February 27, 1997), Internet Whitepaper,
Retrieved
December
2012
from
http://www.systems-
thinking.org/kmbh/kmbh.htm
9.
Berenson, D., Abbeel, P., & Goldberg, K. (2012, May). A robot path planning
framework that learns from experience. In Robotics and Automation (ICRA),
2012 IEEE International Conference on (pp. 3671-3678). IEEE.
140
10. Bergamaschi, R. A., Bhattacharya, S., Wagner, R., Fellenz, C., Muhlada, M.,
White, F., et al., 2001, ‘Automating the Design of SOCs Using Cores’, In IEEE
Design & Test of Computers, pp. 32–45.
11. Cadenhead, R., & Lemay, L., 2002, ‘Sams Teach Yourself Java 2 in 21 Days’,
Sams Publishing.
12. Caselli S. and Reggiani, M. (2000). ERPP: an experience-based randomized path
planner, the IEEE International Conference on Robotics and Automation, 10021008.
13. Comstock G., Chaffee S., Katzman N., McCombs M., and Roberts D. (1978).
Television and Human Behavior. Columbia University Press.
14. Davenport, Thomas H., 1994, ‘Saving IT's Soul: Human Centered Information
Management’, Harvard Business Review, March-April, 72 (2) pp. 119-131.
15. Drucker, P. F., 2003, ‘The New Realities’, Transaction Pub.
16. Duhon, Bryant, 1998, ‘It's All in our Heads’, Inform, September 12 (8).
17. DVB (2012). The Digital Video Broadcasting Project. Retrieved December 2012
from http://www.dvb.org/
18. Du K. -L., Swamy M. N. S. (2006). Neural Networks in a Softcomputing
Framework. Springer London.
19. Frid, Randy J., (2000). ‘Infrastructure for knowledge Management’, iUniverse.
20. Garcia, E., Jimenez, M. A., De Santos, P. G., & Armada, M. (2007). The
evolution of robotics research. Robotics & Automation Magazine, IEEE, 14(1),
90-103.
21. Georges, A., Buytaert, D., and Eeckhout, L., (2007). “Statistically Rigorous Java
Performance Evaluation,” Proceedings of the 2007 OOPSLA conference, Vol.
42, No. 10, Oct. 2007, pp. 57 – 76.
141
22. Goel, A., Donnellan, M., Vazquez, N., & Callantine, T. (1993, May). An
integrated experience-based approach to navigational path planning for
autonomous mobile robots. In Robotics and Automation, 1993 Proceedings,
1993 IEEE International Conference on (pp. 818-825). IEEE.
23.
Google
(2012).
Google
TV,
Retrieved
December
2012
from
http://www.google.com/tv/
24. Hauppauge (2012). Hauppauge Nova-T stick. Retrieved December 2012 from
http://www.hauppauge.co.uk/site/products/data_novatstick.html
25. IEEE (2012). Multichannel Multipoint Distribution Service. Retrieved December
2012 from http://grouper.ieee.org/groups/802/16/
26. Jelovic, D., (2009). Why Java Will Always Be Slower than C++, Retrieved
October 2012 from http://www.jelovic.com/articles/why_java_is_slow.htm
27. jFuzzyLogic
(2012).
Retrieved
December
2012
from
http://jfuzzylogic.sourceforge.net/html/index.html
28. KERT. (2012). The Knowledge Engineering Research Team. Retrieved
December
2012
from
http://www.newcastle.edu.au/school/engineering/research/KERT/
29. Kleidermacher, D., & Kleidermacher, M., 2012, ‘Embedded Systems Security:
Practical Methods for Safe and Secure Software and Systems Development’.
Newnes.
30. Lao, Guoling, Xiao, Luyuan, Wang, Qinyun, and Qin, Zheng. (2008). Research
on organizational knowledge sharing framework based on CAS theory. Paper
read at 2008 International Conference on Service Systems and Service
Management, at Melbourne, Australia, pp. 1-6.
31. Latest snapshots - Freeview/DTT bitrates (2012). Latest snapshots. Retrieved
December 2012 from http://dtt.me.uk/
142
32.
LEGO,
(2012).
What
is
NXT,
Retrieved
December
2012
from
2012
from
http://mindstorms.lego.com/en-us/whatisnxt/default.aspx
33.
LEJOS,
(2012).
Introduction,
Retrieved
December
http://lejos.sourceforge.net/nxt/nxj/tutorial/Preliminaries/Intro.htm
34. Liao, S.H., 2003, 'Knowledge Management Technologies and Applications—
Literature Review from 1995 to 2002', Expert Systems with Applications, vol.
25, pp. 155-164.
35. Li, B. M., Xie, S. Q., & Xu, X., 2011, ‘Recent development of knowledge-based
systems, methods and tools for one-of-a-kind production’, Knowledge-Based
Systems, 24(7), 1108-1119
36. LG
(2012).
Smart
TV.
Retrieved
December
2012
from
http://www.lg.com/global/smarttv/
37. Luciano Floridi (2010). Information: A Very Short Introduction. Oxford
University Press.
38. Maldonado Sanin, C. A., (2007). Smart Knowledge Management System, PhD
Thesis, Faculty of Engineering and Built Environment - School of Mechanical
Engineering, University of Newcastle, E. Szczerbicki, Doctor of Philosophy
Degree, Newcastle, 2007, 177.
39 Mancilla-Amaya, L., Sanín, C., & Szczerbicki, E. (2010a, January). The EDecisional Community: an integrated knowledge sharing platform. In Seventh
Asia-Pacific Conference on Conceptual Modelling (APCCM 2010), Brisbane,
Australia.
40. Mancilla-Amaya, L., Sanin, C., & Szczerbicki, E. (2010b). Smart knowledgesharing platform for e-decisional community. Cybernetics and Systems: An
International Journal, 41(1), 17-30.
41. Marwedel, P., 2010, 'Embedded System Design: Embedded Systems Foundations
of Cyber-Physical Systems', Springer.
143
42. Mason, M. T. (2012). Creation Myths: The Beginnings of Robotics Research.
Robotics & Automation Magazine, IEEE, 19(2), 72-77.
43. Matayong, S., & Mahmood, A. K., 2012, ‘The studies of Knowledge
Management System in organization: A systematic review’, In Computer &
Information Science (ICCIS), 2012 International Conference, IEEE, Vol. 1, pp.
221-226
44. Mazilescu, V., 2011, ‘A Real-Time Algorithm for Intelligent Control Embedded
in Knowledge based Systems’, International Conference on Risk in
Contemporary Economy, ISSN 2067-0532 XIIth Edition, 2011, Galati, Romania.
45. Medina, J. R., Lawitzky, M., Mortl, A., Lee, D., & Hirche, S. (2011, September).
An experience-driven robotic assistant acquiring human knowledge to improve
haptic cooperation. In Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ
International Conference on (pp. 2416-2422). IEEE.
46. McNally M., Klassner F., and Continanza C., (2007). Exploiting MindStorms
NXT: Mapping and Localization Projects for the AI Course, FLAIRS-20 Key
West, FL May, 7–9.
47. Meier, R., 2012, ‘Professional Android 4 application development’, Wrox.
48. Micrium,
2012,
µC/OS-II
Kernel.
Retrieved
October
2012
from
http://www.micrium.com/page/products/rtos/os-ii
49. Miller M., 2010, Top 10 Smartphones of 2010 ... for now. Retrieved October
2012,
from
http://www.zdnet.com/blog/cell-phones/top-10-smartphones-of-
2010-for-now/3854
50. Morris S., (2002). Java TV Tutorial. Retrieved December 2012 from
http://www.interactivetvweb.org/tutorials/javatv/
51. Mulchandani, D., (1998). “Java for Embedded Systems”, IEEE Internet
Computing, Vol. 2, No. 3, May/June 1998, pp. 30-39.
144
52. Myers, P. S., 2012, ‘Knowledge management and organisational design’,
Routledge.
53. Nemec, B., Vuga, R., & Ude, A. (2011, October). Exploiting previous experience
to constrain robot sensorimotor learning. In Humanoid Robots (Humanoids),
2011 11th IEEE-RAS International Conference on (pp. 727-732). IEEE.
54. Nicolescu, M. N., & Mataric, M. J. (2001). Experience-based representation
construction: learning from human and robot teachers. In Intelligent Robots and
Systems, 2001. Proceedings. 2001 IEEE/RSJ International Conference on (Vol.
2, pp. 740-745). IEEE.
55. Niku, S. (2010). Introduction to Robotics. Wiley.
56. O'Dell C., and Hubert C., 2011, ‘The New Edge in Knowledge: How Knowledge
Management Is Changing the Way We Do Business’, WILEY.
57. Oracle. (2012). Java SE 6 Documentation. Retrieved December 2012 from
http://docs.oracle.com/javase/6/docs/
58. Oxford Dictionaries, 2010, ‘knowledge’, Oxford University Press. Retrieved
October
2012
from
http://oxforddictionaries.com/definition/american_english/knowledge
59. Panasonic
(2012).
Smart-Viera.
Retrieved
December
2012
from
http://www.panasonic.com/promos/learn/smart-viera/
60. Pastor, P., Righetti, L., Kalakrishnan, M., & Schaal, S. (2011, September).
Online movement adaptation based on previous sensor experiences. In
Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International
Conference on (pp. 365-371). IEEE.
61. Pereira C. E., Carro L., 2007, ‘Distributed Real-time Embedded Systems: Recent
Advances, Future trends and Their Impact on Manufacturing Plant Control’,
Annual
Reviews
in
Control,
doi:10.1016/j.arcontrol.2007.02.005.
145
vol.
31,
No.
1,
pp.
81-92,
62. Perls P., 2009, PowerPC Overview. Retrieved October 2012, from
http://titancity.com/articles/ppc.html
63. Petrovan B., 2012, Android Everywhere: 10 Types of Devices That Android Is
Making
Better.
Retrieved
October
2012
from
http://www.androidauthority.com/android-everywhere-10-types-of-devices-thatandroid-is-making-better-57012/
64. PCWorld (2012). Toshiba Thrive 16GB. Retrieved December 2012 from
http://www.pcworld.com/article/235696/toshiba_thrive_review_a_tablet_edges_
closer_to_the_ideal.html
65. Powell M. JC, 2012, iPhone 5 review. ABC Technology and Games,
Retrieved
October
from
2012,
http://www.abc.net.au/technology/articles/2012/10/04/3603570.htm
66. Rajsuman R., 2000, ‘System-on-a-Chip: Design and Test’, Artech House, Inc.,
Norwood, MA.
67. Reimers, U. H. (2006). DVB-The Family of International standards for Digital
Video broadcasting. Proceedings of the IEEE, Vol. 94, Issue 1, pp 173-182,
ISSN: 0018-9219
68. Reinaldo A., Bergamaschi, S. Bhattacharya, R. Wagner, C. Fellenz, M. Muhlada,
William R. Lee, F. White, Jean-Marc Daveau, 2001, ‘Automating the Design of
SOCs Using Cores’, IEEE Design and Test of Computers, vol. 18, No. 5, pp. 3245, doi:10.1109/54.953270
69. Samsung (2012). Let the Smart TV experience begin. Retrieved December 2012
from http://www.samsung.com/us/2012-smart-tv/
70. SanDisk,
About
SanDisk.
Retrieved
October
2012,
from
http://www.sandisk.com/about-sandisk.aspx
71. Sanin, C., Mancilla-Amaya, L., Szczerbicki, E., and CayfordHowell, P., 2009,
‘Application of a Multi-domain Knowledge Structure: The Decisional DNA’, In
146
Intelligent Systems for Knowledge Management, edited by N. T. Nguyen and E.
Szczerbicki: Springer Berlin / Heidelberg, Vol. 252. Original edition, pp. 65-86
72. Sanin, C., and Szczerbicki, E., 2006, ‘Extending Set of Experience Knowledge
Structure into a Transportable Language Extensible Markup Language’,
International Journal of Cybernetics and Systems, Vol. 37, No. 2-3, pp. 97-117
73. Sanin, C., and Szczerbicki, E., 2007, ‘An OWL ontology of Set of Experience
Knowledge Structure’, Journal of Universal Computer Science, Vol. 13, No. 2,
pp. 209-223
74. Sanin, C., and Szczerbicki, E., (2008a). ‘Decisional DNA and the Smart
Knowledge Management System: A process of transforming information into
knowledge’, In Techniques and Tools for the Design and Implementation of
Enterprise Information Systems, edited by A. Gunasekaran. New York: IGI, pp.
149-175
75. Sanin, C., and Szczerbicki, E., (2008b). A Decisional Trust Implementation on a
Maintenance System by the Means of Decisional DNA and Reflexive
Ontologies. Paper read at WI-IAT '08. IEEE/WIC/ACM International
Conference on Web Intelligence and Intelligent Agent Technology, 2008, at
Sydney, Australia, pp. 5-8.
76. Sanin C., and Szczerbicki E., (2009a), ‘Experience-Based Knowledge
Representation: SOEKS’, Cybernetics and Systems 40 (2): 99-122
77. Sanin, C., and Szczerbicki, E., (2009b). Implementing Decisional Trust: A First
Approach For Smart Reliable Systems. Cybernetics and Systems: An
International Journal 40 (2):85 - 98.
78. Schreiber, G., Akkermans, H., Anjewierden, A., de Hoog, R., Shadbolt, N., Van
de Velde, W., & Wielinga, B., 1999, ‘Knowledge engineering and management:
the CommonKADS methodology’, MIT press.
147
79. Schwalb, E. (2004). iTV Handbook: Technologies & Standards. Prentice Hall
PTR, ACM Computers in Entertainment, Vol. 2, Number 2, Article 7, ISBN
0131003127
80. Shankland, S., 2012, Google: 500 million Android devices activated. Retrieved
October 2012, from http://news.cnet.com/8301-1035_3-57510994-94/google500-million-android-devices-activated/
81. Shankland, S. 2007, Google's Android parts ways with Java industry group.
Retrieved October 2012 from http://news.cnet.com/8301-13580_3-981549539.html.
82. Sharma, N., Singh, K., & Goyal, D. P., 2012, ‘Is Technology Universal Panacea
for Knowledge and Experience Management? Answers from Indian IT Sector’,
Information Systems, Technology and Management, 187-198.
83. Sheh, R., Hengst, B., & Sammut, C. (2011, September). Behavioural cloning for
driving robots over rough terrain. In Intelligent Robots and Systems (IROS),
2011 IEEE/RSJ International Conference on (pp. 732-737). IEEE.
84. Siciliano, B., Sciavicco, L., Villani, L., & Oriolo, G. (2011). Robotics: modelling,
planning and control. Springer.
85. Stefik, M. J., Bobrow, D., Bell, A., Brown, H., Conway, L., & Tong, C., 1982,
‘The partitioning of concerns in digital system design’, Department of Computer
Science, Stanford University.
86. Steve, H., 2002, ‘Embedded Systems Design, EDN series for design engineers’,
2nd ed., Elsevier Science, p.2
87. Steve Kovach (2012). What Is a Smart TV? Retrieved December 2012 from
http://www.businessinsider.com/what-is-a-smart-tv-2010-12/
88. Sun, Z., and Finnie, G., 2003, ‘Brain-like architecture and experience based
reasoning’, In: Proceedings of the 7th JCIS, Cary, North Carolina, USA, pp.
1735–1738
148
89. Sun, Z., and Finnie, G., 2004, ‘Intelligent Techniques in E-Commerce: A Casebased Reasoning Perspective’, Springer, Heidelberg
90. Sven Killig (2012). Connect USB peripherals to your Nexus One. Retrieved
December 2012 from http://sven.killig.de/android/N1/2.2/usb_host/
91. Toro C., Sanin C., Vaquero J., Posada J., and Szczerbicki E., 2007, ‘Knowledge
based industrial maintenance using portable devices and augmented reality’, In
Knowledge-Based Intelligent Information and Engineering Systems. 11th
International Conference, KES 2007, XVII Italian Workshop on Neural
Networks, Vietri sul Mare, Italy, September 12-14, 2007. Proceedings, Part I,
edited by B. Apolloni, R. J. Howlett and L. Jain: Springer Berlin / Heidelberg,
Vol. 4692/2009. Original edition, pp. 295- 302
92. Tvheadend
(2012).
Overview.
Retrieved
December
2012
from
https://www.lonelycoder.com/tvheadend/
93. TVHGuide (2012). About. Retrieved December 2012 from http://johntornblom.github.com/TVHGuide/index.html
94. VDC Research Group, 2011, Embedded Engineer Survey Results – Programming
languages used to develop software. Retrieved October 2012, from
http://blog.vdcresearch.com/embedded_sw/2011/06/2011-embedded-engineersurvey-results-programming-languages-used-to-develop-software.html
95. Vidakovic, M., Teslic, N., Maruna, T. & Mihic, V (2012). Android4TV: A
proposition for integration of DTV in Android devices. 2012 IEEE International
Conference on Consumer Electronics ICCE, pp. 437-438
96. Vrba, V., Cvrk, L., and Sykora, M. (2006). Framework for digital TV
applications. Proceedings of the International Conference on Networking,
International Conference on Systems and International Conference on Mobile
Communications and Learning, P. 184, ISBN: 0-7695-2552-0
97. Wikipedia. Application Programming Interface (2012). Retrieved December
2012 from http://en.wikipedia.org/wiki/Application_programming_interface
149
98. Wikipedia, Digital Television (2012).
Retrieved December 2012 from
http://en.wikipedia.org/wiki/Digital_television/
99. Wikipedia, Digital Terrestrial Television (2012). Retrieved December 2012 from
http://en.wikipedia.org/wiki/Digital_terrestrial_television
100.
Wikipedia,
Robot
(2012).
Retrieved
October
2012
from
http://en.wikipedia.org/wiki/Robot
101.World DMB (2012). Digital Multimedia Broadcasting. Retrieved December
2012 from http://www.worlddab.org/
102.Wu, Y., Hirakawa, S., Reimers, U., and Whitaker, J. (2006). Overview of digital
television development worldwide. Proc. IEEE, vol. 94, no. 1, pp. 8–21
103.Yarali, A., Cherry A. (2005). Internet Protocol Television (IPTV). TENCON
2005 IEEE Region 10, pp. 1-6, ISBN: 0-7803-9311-2
104.Zadeh L. (1965). “Fuzzy sets”, Information and Control, Volume 8, Issue 3, pp.
338–353
105.Zhou, Z., Wang, H., & Lou, P., 2010, ‘Knowledge-Based System’, In Z. Zhou,
H. Wang, & P. Lou (Eds.), Manufacturing Intelligence for Industrial
Engineering: Methods for System Self-Organization, Learning, and Adaptation,
pp. 13-46. Hershey, PA: Engineering Science Reference. doi:10.4018/978-160566-864-2.ch002
150

DDNA-based Embedded Systems - Nova

Transcription

Similar documents

computer - UniMAP Portal

Skulls Röyksopp Campaign type : Track debut with embedded

new Brochure EN Front v2

Sistemi Embedded Introduzione

7 Reasons to Choose SSDs Over Hard Drives for

our faculty hiring brochure

La cuarta revolución industrial: Internet de las cosas y sistemas

The role of Cito in the Dutch education

Look. Donʼt touch.

Kamailio World