Simulation network exploitation SEDR tool suite: system design

Transcription

Simulation network exploitation
SEDR tool suite: system design
document
Oliver Schoenborn, Bassem Mikhael and Bobby Chawla
The scientific or technical validity of this Contract Report is entirely the responsibility of the Contractor
and the contents do not necessarily have the approval or endorsement of Defence R&D Canada.
CONTRACT REPORT
DRDC Ottawa CR 2008-311
April 2009
Simulation network exploitation
SEDR tool suite: system design document
Oliver Schoenborn
Bassem Mikhael
Bobby Chawla
CAE Professional Services
Prepared By:
1135 Innovation Dr. Suite 300
Ottawa, ON K2K 3G7
Contract Project Manager: Leo Roberts, 613-293-8993
W8475-06BM04
CSA: Nacer Abdellaoui, Defence Scientist, 613-998-4582
The scientific or technical validity of this Contract Report is entirely the responsibility of the Contractor and the
contents do not necessarily have the approval or endorsement of Defence R&D Canada.
Defence R&D Canada – Ottawa
Contract Report
April 2009
Contract Scientific Authority
Original signed by Nacer Abdellaoui
Nacer Abdellaoui
Defence Scientist
Approved by
Original signed by Julie Tremblay-Luther
Julie Tremblay-Luther
SH/CARDS
Approved for release by
Original signed by Pierre Lavoie
Pierre Lavoie
DRP Chair
© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2009
© Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense nationale,
2009
Abstract ……..
This document presents the design of the three tools created for the Simulation Network
Exploitation (SNE) project. Together they form the Synthetic Environment Distributed Resources
(SEDR) Tool Suite. It is intended for the developers in charge of maintaining or extending the
system. The system consists of three tools: the SEDR Access (SEDRA) portal for accessing
stored knowledge regarding simulation resources; the SEDR Control (SEDReC) station for
executing system control recipes for re-use of complex distributed simulations; and the SEDR
Data Entry (SEDReDE) portal, which allows the user to add simulation resources knowledge to a
SEDRA-compatible database.
Résumé ….....
Ce document présente la conception du kit d’outils créé pour le projet d'Exploitation du Réseau
de Simulation. Ensemble, ces outils forment l’instrument des ressources distribuées des
environnements synthétiques. Ce kit est destiné aux développeurs en charge du maintien ou de
l’extension du système. Le kit d’outils se compose de trois outils: la portail d’accès pour l'accès a
l’information sur les différentes ressources de simulation, la station de contrôle pour l'exécution et
le démarrage du système de simulations distribuées ainsi que la réutilisation de ces simulations
complexes, et le portail de saisie de qui permet à l'utilisateur d'ajouter des ressources de
simulation à la base de données appropriée.
i
This page intentionally left blank.
ii
Executive summary
Simulation network exploitation: SEDR tool suite: system design
document
O. Schoenborn, B. Mikhael, B. Chawla; DRDC Ottawa CR 2008-311; Defence R&D
Canada - Ottawa; April 2009.
Introduction or background: This document presents the design of the three tools created for
the Simulation Network Exploitation (SNE) project. Together they form the Synthetic
Environment Distributed Resources (SEDR) Tool Suite. It is intended for the developers in
charge of maintaining or extending the system. As such it assumes familiarity with software
design concepts and principles, with various programming languages (Python, Java) and data
representations (XML, RelaxNG).
Results: The system consists of three tools:
•
SEDRA portal, the Synthetic Environment Distributed Resources Access portal: this is a
desktop application from which the user can discover, browse, and download simulation
resources by connecting to a database of meta-data;
•
SEDReC station, the Synthetic Environment Distributed Resources Control station: this is
a desktop application from which the user can browse and execute “control recipes” that detail
how to start a distributed simulation system;
•
SEDReDE portal, the Synthetic Environment Distributed Resources Data Entry portal:
this is a web-based application (run from a web-browser) that allows the user to add, edit and
remove simulation resource meta-data from a MySQL database.
iii
Sommaire .....
Simulation network exploitation: SEDR tool suite: system design
document
O. Schoenborn, B. Mikhael, B. Chawla; DRDC Ottawa CR 2008-311; R & D pour la
la défense Canada – Ottawa; Avril 2009.
Introduction ou contexte: Ce document présente la conception des trois outils créés pour le
projet d'exploitation de réseau de simulation (SNE). Ensemble, ces trois applications forment le
kit d’outils des ressources distribuées des environnements synthétiques (SEDR). Ce kit est destiné
aux développeurs en charge de maintenir ou d'étendre le système de simulations. Cependant, ce
document assume une certaine familiarité avec les principes de conception de logiciels, les divers
langages de programmation (Python, Java) et les différentes formes de représentations de données
(XML, RelaxNG).
Résultats: Les trois outils composants ainsi ce système sont:
• SEDRA, portail d'accès des ressources distribuées des environnements synthétiques: il s'agit
d'une application à partir de laquelle l'utilisateur peut découvrir, naviguer et télécharger des
ressources de simulation en se connectant à une base de données métadonnées;
• SEDReC, station de contrôle des ressources distribuées des environnements synthétiques: il
s'agit d'une application à partir de laquelle l'utilisateur peut naviguer et d'exécuter « des recettes
de contrôle », qui détaillent la façon de démarrer un système de simulation distribuée;
• SEDReDE, portail d'entrée de données des ressources distribuées des environnements
synthétique: il s'agit d'une application Web, exécutée à partir d'un navigateur Web, qui permet à
l'utilisateur d'ajouter, de modifier et de supprimer des métadonnées des ressources de simulation,
et ce à partir d'une base de données MySQL.
iv
Table of contents
Table of contents ............................................................................................................................. v
List of figures ................................................................................................................................ vii
1 ....Introduction............................................................................................................................... 1
2 ....System Overview...................................................................................................................... 2
3 ....Design Considerations .............................................................................................................. 3
3.1
General .......................................................................................................................... 3
3.2
Constraints..................................................................................................................... 4
3.3
Dependencies................................................................................................................. 4
3.3.1
SEDRA portal and SEDReC station ............................................................... 4
3.3.2
SEDRA portal only ......................................................................................... 5
3.3.3
SEDReC station only ...................................................................................... 5
3.3.4
SEDReDE ....................................................................................................... 5
4 ....Architectural Strategies............................................................................................................. 6
4.1
Across system................................................................................................................ 6
4.2
Common to SEDRA and SEDReC................................................................................ 6
4.3
Specific to SEDRA........................................................................................................ 6
4.4
Specific to SEDReC ...................................................................................................... 7
4.5
Specific to SEDReDE.................................................................................................... 7
5 ....System Architecture.................................................................................................................. 9
5.1
SEDRA portal................................................................................................................ 9
5.2
SEDReC station............................................................................................................. 9
5.3
SEDReDE portal ......................................................................................................... 10
5.4
Repositories ................................................................................................................. 10
5.5
Databases..................................................................................................................... 10
5.6
Simulation Hosts ......................................................................................................... 10
6 ....Detailed System Design.......................................................................................................... 11
6.1
SEDRA portal.............................................................................................................. 11
6.1.1
Application Core ........................................................................................... 11
6.1.2
Application Document .................................................................................. 13
6.1.3
Application GUI............................................................................................ 14
6.1.4
Kid templates ................................................................................................ 15
6.1.5
Database plug-ins .......................................................................................... 15
6.1.6
Launcher plug-ins ......................................................................................... 16
6.2
SEDReC station........................................................................................................... 16
6.2.1
Application Core ........................................................................................... 18
6.2.2
Document: Action definitions ....................................................................... 19
v
6.3
6.4
6.2.3
Execution of actions...................................................................................... 20
6.2.4
GUI................................................................................................................ 22
SEDReDE portal ......................................................................................................... 24
6.3.1
Module 1: arpMain.java ................................................................................ 25
6.3.2
Module 2: dbTreeNode ................................................................................. 26
6.3.3
Module 3: summaryField .............................................................................. 27
6.3.4
Module 4: summaryPanel ............................................................................. 27
6.3.5
Module 5: dbUtils ......................................................................................... 27
6.3.6
Module 6: fieldData ...................................................................................... 27
Schema for relational databases .................................................................................. 28
7 ....User Interface Design ............................................................................................................. 31
7.1
SEDRA portal.............................................................................................................. 31
7.2
SEDReC station........................................................................................................... 32
7.3
SEDReDE portal ......................................................................................................... 33
References ..... ............................................................................................................................... 35
Annex A .. Tables for Relational Databases .................................................................................. 37
List of symbols/abbreviations/acronyms/initialisms ..................................................................... 47
vi
List of figures
Figure 1: Dependency graph of modules in SEDRA application.................................................. 11
Figure 2: SEDRA modules related to core/control (orange shapes).............................................. 12
Figure 3: SEDRA modules related to document. .......................................................................... 13
Figure 4: SEDRA modules related to GUI.................................................................................... 14
Figure 5: Dependency graph of modules in SEDReC application. ............................................... 17
Figure 6: Sequence diagrams of important SEDReC objects........................................................ 18
Figure 7: SEDReC modules (yello) related to application core. ................................................... 18
Figure 8: Modules defined for the document and controller aspects of SEDReC......................... 19
Figure 9: Relationship of action classes defined in ActionDefs module....................................... 20
Figure 10: Relationship of classes defined in ExecActions modules. ........................................... 21
Figure 11: This is the caption for the figure shown above. ........................................................... 22
Figure 12: Action Details panel of SEDReC station. .................................................................... 23
Figure 13: Execution Failures panel of SEDReC station. ............................................................. 23
Figure 14: Four simulation host widgets in SEDReC station........................................................ 24
Figure 15: Java classes in SEDReDE. ........................................................................................... 25
Figure 16: The dbTreeNode class in SEDReDE. .......................................................................... 26
Figure 17: The fieldData class in SEDReDE portal. ..................................................................... 28
Figure 18: SEDReC station GUI, top section................................................................................ 31
Figure 19: SEDReC station GUI, top section................................................................................ 32
Figure 20: SEDReC station GUI, bottom section.......................................................................... 33
Figure 21: The SEDReDE web portal. .......................................................................................... 34
Figure 22: DB schema tables related to sys_group (SystemGroup).............................................. 42
Figure 23: DB schema tables related to sim_sys (SimulationSystem) .......................................... 43
Figure 24: DB schema tables related to comp_group (ComponentGroup) ................................... 44
Figure 25: DB schema tables related to sim_comp (SysComponent) ........................................... 45
Figure 26: DB schema tables related to doc_group (DocumentGroup) ........................................ 45
vii
viii
1
Introduction
Exploitation (SNE) project. Together they form the Synthetic Environment Distributed Resources
(SEDR) Tool Suite. It is intended for the developers in charge of maintaining or extending the
system. As such it assumes familiarity with software design concepts and principles, with various
programming languages (Python, Java) and data representations (XML, RelaxNG). Other
documentation pertaining to the system is given in the Bibliography.
In section 2 the system as a whole is introduced. In section 3 some design considerations are
discussed. Section 4 outlines some architectural strategies used, and section 5 discusses the
individual components of each tool in the system. Section 6 touches on some detailed design
aspects and section 7 on some of the GUI design aspects.
1
2
System Overview
The system consists of three tools:
• SEDRA portal, the Synthetic Environment Distributed Resources Access portal: this is a
desktop application from which the user can discover, browse, and download simulation
resources by connecting to a database of meta-data;
• SEDReC station, the Synthetic Environment Distributed Resources Control station: this is a
desktop application from which the user can browse and execute “control recipes” that detail
how to start a distributed simulation system;
• SEDReDE portal, the Synthetic Environment Distributed Resources Data Entry portal: this
is a web-based application (run from a web-browser) that allows the user to add, edit and
remove simulation resource meta-data from a MySQL database;
The tools do not exchange data, but are loosely coupled in the following ways:
• SEDRA can download a “control recipe” pointed to by a database, start SEDReC, and give
the recipe to SEDReC for handling.
• SEDRA supports, amongst others, connection to MySQL databases. Hence SEDReDE and
SEDRA support the same database schema for MySQL databases. Information entered via
SEDReDE can be used in SEDRA.
2
3
Design Considerations
3.1
General
The tool suite is completely new and unrelated to any previous toolsets found at FFSE. As such it
is not affected by any legacy applications or existing policies. The design is as open-ended as is
possible within the constraints of the development project, to allow for user-initiated changes
once the system is in use. No user at FFSE has had to chance to use such system, and therefore to
make recommendations as to function and other requirements.
Some important guidelines were followed across the system:
• Modularity: the system is broken down into three tools; each tool is broken down into layers
(e.g. document, controller, view); each layer is broken down into object-oriented
components (e.g. a panel for the database connection part of the GUI is a separate class) or
services (e.g. all network-related utilities are in one module).
• Encapsulation: the implementation details are hidden behind interfaces; public attributes are
only used for data structures that aggregate related attributes without behaviour.
• Reliance on solid tools or utility libraries: the system makes use of existing tools or libraries
as much as possible, but uses tools and libraries that are stable, mature and well
documented, such that their use is relatively straightforward, their use improves the stability
and robustness of the system (since re-implementing their equivalent would like have far
more bugs given the much smaller user base), and their use speeds up development.
Runtime environments:
• For SEDRA portal and SEDReC station: run from Desktop. Desktop applications have
access to the full power of the CPU; without much effort they are very responsive, have a
native look and feel (blend in with Operating System), and can be arbitrarily complex. This
is important given the novelty of these applications: it is not yet known exactly how they
will affect the practices in the organization.
• For SEDReDE: run from web browser
The main development method used is:
1. Perform some preliminary design
2. Implement. Use GUI editor for GUI design: a GUI editor, wxGlade, is used to create the GUI
and generate the Python source code which can then be extended to instantiate the designed
GUI. This speeds up development and provides immediate visual feedback on the GUI. The
code is generated as one file per developed GUI component, i.e. typically per stand-alone
panel.
3. Test; use regression testing as much as possible using well known test suites (like nosetests in
Python or CppUnit in C++)
3
4. Commit to version control
5. Refactor and re-test
6. Repeat from #1
3.2
Constraints
SEDRA portal:
1. Support FolderTree database: It seems quite likely that users would first define simulation
resources in their immediate surroundings, i.e. their desktop. The filesystem could therefore
be used as the initial database for meta-data as they develop a new system, with added bonus
that this type of database can also store the simulation resources proper, not just their metadata. The data could later be migrated to the organization’s main database either manually or
via a script (not currently available).
2. Support MySQL: There are many different kinds of databases. MySQL is a free one that has
an extensive user and competes with the like of Oracle and D2. To use it from Python the
Open-Source package called SQLAlchemy is used. This makes interacting with the database
tables very easy.
3. Support HTTP: This communication protocol is relatively straightforward and makes for a
database that is easy to populate and test with. It is very similar to the FolderTree database in
that it is capable of storing simulation resources, not just their meta-data. However, search
and edit capabilities must be developed as they are not standard part of web servers.
3.3
3.3.1
Dependencies
SEDRA portal and SEDReC station
• Python: Python is an interpreted language that is very expressive, stable, clean, and has an
extensive library. This improves development time, robustness (less code to implement) and
maintainability. The system is known to work with version 2.4.3 and 2.4.4 [1].
• wxPython: This is a GUI library available from Python. It sits atop lower level GUI
libraries such as MFC on Windows and GTK on Linux. It is very stable, has a very large
user community and has a very large variety of widgets and tools. The system is known to
work with version 2.6.x [2]. In addition, a tool called WxGlade was used to generate the
GUI in a WYSIWYG editing environment. Version 0.4.1 was used [3].
• XML: For data representation and transfer, XML is a de-facto standard due to its popularity.
A large number of tools exist to help editing, generating and parsing data represented using
XML. The elementtree and lxml packages were used for working with XML. The former is
very popular due to its simplicity and Pythonic feel. However it does not support validation
of XML using schemas. The much more advanced, but more complicated to install and use
and not as mature lxml library supports several schema validation languages, such as
RelaxNG. ElementTree version 1.2.6 [4] and lxml version 1.1.1 [5] were used.
4
• RelaxNG: It is preferable to validate any XML received or generated from data in a
database. RelaxNG, especially the compact notation, is an example of a clean and simple
language to describe data representations in XML. The RelaxNG compact notation files
were translated into RelaxNG XML files using a schema conversion tool called “trang” [6].
The latest version is 20030619.
3.3.2
SEDRA portal only
In addition to the dependencies listed in section 3.3.1, the following dependencies exist:
• Kid: Version 0.9.3 was used [7].
• SQLAlchemy: Version 0.2.8 was used [8].
• MySQL-Python: Version 1.2.1 was used [9].
3.3.3
SEDReC station only
In addition to the dependencies listed in section 3.3.1, the following dependencies exist:
• RPyC: Version 2.60 was used [10].
• Ctypes: Version 3.0.1 was used [11].
3.3.4
SEDReDE
All development was done using the following configuration:
• J2SDK: The Java virtual machine, compiler and programming language. Version 1.5.0_01
was used [12].
• MySQL DBMS: The database management system for MySQL databases. Version 5.0.45
was used [13].
• Apache Web Server: The server for the Java Applet. Version 2.2.6 was used [14].
• FireFox: Mozilla’s web browser. Version 2.0.0.9 was used [15] on Windows XP SP2 and
Linux RHE.
• Internet Explorer: Microsoft’s web browser. Version 6.0.2900.2180.xpsp_sp2 was used on
MS Windows XP.
Normal functionality of the portal was observed using but an exhaustive test of the features needs
to be completed. The web server and browser are not true dependencies, but are required to serve
and host the Java applet. Note that problems were observed when Internet Explorer was used as
browser on MS Windows Vista platform.
5
4
4.1
Architectural Strategies
Across system
Use Object-Oriented design: this is a powerful software design paradigm that facilities
modularization, encapsulation, and extension.
4.2
Common to SEDRA and SEDReC
The following architectural strategies were used:
1. Use Model-View-Controller strategy: Separation of concerns and responsibilities helps
contain the impact of unforeseen future changes to the system and helps with system testing.
Both SEDRA and SEDReC are therefore divided into a document object (model), a view, and
a controller.
2. Divide and conquer: Each of the document, view and controller modules are further
subdivided, as necessary, into several sub-modules to improve cohesion (decrease the number
of responsibilities of modules). E.g. the controller aspect may be divided into the main
controller and several smaller controllers that are instantiated only as necessary based on the
logical state of the application; similarly, the GUI is divided into stand-alone components
(panels) that are put together hierarchically.
3. Inherit from generated code, do not modify it: Any file genenrated by wxGlade is left
untouched. To incorporate some GUI behaviour and specifics into the GUI, the classes
defined in the generated files are derived from and methods implemented or overridden. This
eliminates the possibility that the code generation could clobber some developer code, helps
with encapsulation (the generated classes are treated as white boxes which minimizes the
dependency on implementation details).
4.3
Specific to SEDRA
In addition to the strategies mentioned in the previous section, the following architectural
strategies were used in SEDRA:
1. Use plug-in strategy: there are many possibilities for how the organisation’s meta-data will be
stored. To accommodate future decisions about databases, a plug-in mechanism is designed
into the application. It uses inheritance to implement a base interface for the new database.
2. Use HTML: Many tools exist to read, transform, generate and represent HTML. The
presentation of meta-data to the user should be pleasing to the eye, making HTML an ideal
candidate. WxPython includes an HTML rendering window.
3. Kid templates: The structured nature of the summary meta-data of simulation resources
makes templating an obvious choice for formatting the data for presentation to the user. This
6
template engine is written in Python and is very easy to use. Moreover, the template file is
easy to edit and decreases the likelihood that changes to the application would be necessary if
changes to the presentation were mandated.
4. Use pipes and filters: The strategy of dividing the presentation (view) of data (document)
through a series of pipes and filters (controller) allows for future modifications via the
insertion of new pipes and filters or modification of existing ones. E.g. SEDRA takes the
document object model of the meta-data received from a database, transforms it into an
abstract tree that keeps track of state (collapsed/expanded nodes, etc), creates a filtered tree to
allow for search items in the meta-data, and transforms the result further into a tree widget in
the GUI.
4.4
Specific to SEDReC
In addition to the strategies mentioned in Section 4.2, the following architectural strategies were
used in SEDReC:
1. Use RPyC: The execution of arbitrary commands on a remote host is greatly facilitated by
having a Python interpreter running on the remote host and listening for commands to
execute. This is provided by the RPyC (Remote Python Calls) package available as OpenSource.
2. Use tlslite: The execution of arbitrary commands on a remote host poses a severe security
risk, mandating the use of SSL or TLS. The tlslite Python library provides Transport Layer
Security and integrates with RPyC to make the use of RPyC secure.
3. Use blind invocation by events: a publish-subscribe strategy is used to further decouple the
View from the Controller. The use of wx.lib.pubsub allows modules to publish messages
without having to hold a reference to each recipient, or to even have to take special action if
there are no recipients. Similarly, it allows event handlers, termed listeners in the context of
pubsub, to not have to hold references to event generators, or to even have to take special
action if there are no event generators for a particular type of event. Blind invocation
completely separates View from Controller, thereby increasing modularity and encapsulation.
4.5
Specific to SEDReDE
A number of architectures were investigated:
• JAVA Swing Applet solution
• C# ASP.NET 2.0 Web Form solution
• Python-based web application solution
• Open-Laszlo web application platform solution
• Python-based desktop solution
7
In choosing from these options, the primary determining factors were the familiarity with the
technology and the time required to implement a solution using the selected technology while
providing the GUI features similar to the SEDRA portal.
The architecture chosen was that of a JAVA Swing Applet solution. Initial development was
attempted using the Open-Laszlo web application platform and its related tool suite but the
expressive power of the programming language and the inability to manipulate GUI widgets
satisfactorily prevented this architecture from being used.
The JAVA Applet solution was chosen primarily due to the first requirement that the system be
web-based. By developing the SEDReDE portal using a JAVA Applet, full access to the JAVA
Swing API was made available. This greatly simplified the development of the GUI portion of
the portal as the Swing API contains most of the widgets that are common to modern windowing
systems. In addition, the power of the JAVA language itself made it possible to manipulate these
widgets to advantage.
The final criterion used in selecting a JAVA based solution was the nature of web-based
applications in general. The underlying protocol used on the World Wide Web is the hyper text
transfer protocol (HTTP). This protocol is stateless and follows a “request-response” paradigm
where information about previous requests is not maintained. Interaction with databases through
a web interface however requires that some connection and state information be maintained
across requests. This is typically handled by storing this information as part of the request and
then sending it back in the response. While this strategy would indeed permit the user to view
and edit information in the database, it would detract from the desired consistent view of the user
interface in that all interactions with the database would require the user to navigate back and
forth between web pages.
Since the JAVA Applet behaves as a regular application residing within a web browser, its user
interface and state information are maintained as long as the user does not navigate to another
web page. Database interaction is handled by the Applet directly and changes are reflected
immediately in the user interface. When coupled with the inherent security constraints that are
imposed on JAVA Applets, this became the architecture of choice in view of time constraints of
the project.
8
5
5.1
System Architecture
SEDRA portal
The SEDRA portal can be divided into the following components. Note that not all are Python
code.
1. Core: all functionality that is not GUI-related and not plug-in related; this includes the
Document, the Controller, various utilities, etc.
2. GUI: the graphical user interface, which includes the wxGlade project file, the Python files
generated by wxGlade, and the Python files for classes deriving from the wxGlade
components.
3. Kid templates: used for generating any “document-like” representation in HTML, such as the
content of the About box, the summary description (meta-data) of simulation resources, etc.
4. Database plug-ins: this includes the plug-in framework for adding new database proxies, as
well as the three individual proxies currently defined (FolderTree, HTTP, and MySQL) and
the schema for XML representation of data received via the database proxies, in RelaxNG
format.
5. Launcher plug-ins: this includes the plug-in framework for adding new launcher proxies, as
well as the one proxy currently defined, the SEDReC proxy to start SEDReC from SEDRA
using a control recipe downloaded from a URL held as simulation resource meta-data in
database.
5.2
SEDReC station
The SEDReC station can be divided into the following components.
1. Core: all functionality that is not in one of the other components. This includes the recipe
manager, logging utilities, etc.
2. Document: the action tree loaded from an XML file and validated using a RelaxNG schema.
The resulting DOM is the document as seen by the application proper.
3. GUI: the graphical user interface, which includes the wxGlade project file, the Python files
generated by wxGlade, and the Python files for classes deriving from the wxGlade
components.
4. Action definitions: the classes representing the information for all types of actions supported
by SEDReC. Note that this does not include execution algorithms for each type of action, just
the “static” definition of the action and how to represent it in a text view.
9
5. Execution of actions: the classes that can execute the actions as defined in the document.
These action execution classes are thread objects that monitor children actions and send status
messages.
5.3
SEDReDE portal
The SEDReDE portal requires the interaction of three primary components:
• The SEDReDE Applet
• Web Server
• Database Server
The database server for this release of the SEDReDE portal is the MySQL DBMS. The web
server can be of any type although all development and testing was done using the Apache Web
Server. Due to the security constraints imposed upon Applet-based solutions, all of these
components must reside physically on a single machine. The design of the SEDReDE Applet is
described in detail in section 6.3.
5.4
Repositories
Repositories are storage areas for simulation resources, accessible via HTTP requests1. There are
no repositories of simulation resources included with the system, except example repositories
used for testing and demonstrations. The deployment of the system does, however, involve the
creation or, at the very least, the identification, of one or more repositories of simulation
resources available via HTTP access.
5.5
Databases
Databases store meta-data regarding simulation resources stored in repositories. There are no
databases included with the system, except example databases used for testing and
demonstrations. The deployment of the system does, however, involve the creation of one or
more databases of simulation resources meta-data.
5.6
Simulation Hosts
Simulation hosts are computers running an RPyC server and containing executable simulation
resources (programs). There is no simulation host included with the system. They are identified in
the actions defined in the “Control Recipes” stored in databases.
1
In fact, any protocol supported by URL formats and associated Python library is supported. This includes
FTP.
10
6
Detailed System Design
6.1
SEDRA portal
The startup script for SEDRA portal is src/portal.pyw. This simply imports the application as a
package and calls its main loop function. The complete dependency graph of the modules used to
create the SEDRA application is shown in Figure 1. This diagram was automatically generated by
running a Python script2 called depgraph, and therefore the layout cannot be controlled. It is also
fairly quite large, therefore parts of it are shown zoomed in figures in this section. The zoomed-in
parts are represented by three dashed boxes.
1
3
2
Figure 1: Dependency graph of modules in SEDRA application.
The core/control part of SEDRA is in box 1 (expanded in Figure 2), the document part in box 2
(expanded in Figure 3) and the GUI part in box 3. The following subsections give more details.
6.1.1
Application Core
The application core is in the src/portal folder. The module dependencies are shown in Figure
2, which is a copy from box 1 in Figure 2.
2
From Command shell, cd to src/gna_launcher and run: depgraph.py -e
id,email,_socket,_ssl,BasicLogging,wx,xml,AppOptions,delayedresult srapGUI.py
11
Figure 2: SEDRA modules related to core/control (orange shapes).
1. __init__.py: (not shown) makes the core into a “Python package” so that it can be imported
(via srapGUI) from the startup script “src/portal.pyw”.
2. srapGUI.py: Main application module. Provides the PortalAppGUI class that derives from
wxPython’s wx.App class. PortalAppGUI simply overrides wx.App’s OnInit() method to
create the controller (via AppController) and the GUI (via wxgui.PortalMainWin), set up
logging so that any messages get displayed in the logging window (via utils.BasicLogging
and wxgui.LoggingWin modules), and to trap all exceptions raised during startup and main
GUI event loop.
3. AppController.py: Implements the application control logic and enforces the application’s
state machine. It loads the appropriate database plug-in (according to what the user selected
in GUI) to manage connections to databases, and creates the documents (instance of
SimResourcesDoc).
4. AppOptions.py: (not shown) Provides the AppOptions singleton class, which is where all
application settings are stored. This makes use of wx.Config, which is a platformindependent interface to platform-dependent storage of settings (registry on Windows, files
on UNIX, etc). The only settings currently saved are the database definitions. It is not shown
because many modules use it.
12
Some of the core modules are in a subpackage called utils:
5. utils.AbstractTreeNode.py: Provides an abstraction of a tree node of the document, for
keeping track of expanded/collapsed state and filter-related state (e.g. do any children nodes
satisfy the user-selected filter, etc, as this affects the representation of the node in GUI).
6. utils.BasicLogging.py: Makes possible the output of multi-level log messages. It is used
by most modules since most modules provide some form of debug output. For this reason it is
not shown in the figures.
7. utils.delayedresult.py: Provides asynchronous request/answer. Used for accessing
network resources such as database and simulation resources stored in repositories or
database.
8. utils.Filtering.py: Provides filter-related functionality, such as a tree node coloring of
wx.TreeCtl nodes.
9. utils.NetServices.py: Provides network-related class and functions, for downloading files
from simulation resource repositories, handling authentication when accessing the files in
secured repositories, etc.
6.1.2
Application Document
Figure 3: SEDRA modules related to document.
13
1. SimResourcesDoc.py: Represents the application document. It uses a database proxy
(provided by the AppController module) to get its content, and provides an interface to access
its nodes their various properties (URL, name, etc).
6.1.3
Application GUI
Figure 4: SEDRA modules related to GUI.
The portal.wxgui package, located in the portal folder, defines the GUI panels and GUI
windows for the application. The wxGlade tool was used to create the GUI in a WYSIWYG
environment. The project file is gui.wxg. Files auto-generated from wxGlade end with the word
“Glade” (e.g. HostWidgetGlade.py) and merely provide GUI “look and feel”. Files with same
root name but no “Glade” are implementation files: they provide application-specific GUI
behaviour to the base class, as explained below.
1. PortalMainWinGlade.py: Provides the main window of the application, including menus. It
2. PortalMainWin.py: Provides application-specific GUI behaviour to the base class and an
interface for the AppController module to access user input and propagate application state.
E.g., activating the simulation resources panel of GUI only when GUI connected; asking the
user for a question as required by AppController, and returning answer; etc.
3. ConnectionPanel.py: Provides the GUI for the connection panel in the main window.
4. SimResPanel.py: Provides the simulation resources tree and description pane. It uses the
utils.Filtering to change the appearance of tree, and uses the kid/resDocPanel.kid
template to generate the HTML to be displayed in the description pane.
5. StatusbarMgr.py: Provides the status bar of the main application window and an interface to
easily set fields within the status bar.
14
6. DlgAppOptionsGlade.py: Provides the GUI for setting the application options (preferences).
7. DlgAppOptions.py: Provides the behaviour of application options setting, such as defining a
new database definition, new user, cancelling the settings, etc.
8. AboutBox.py:
Provides the window showing the html generated from
template using the Kid module (an external dependency).
the
portal/kid/about.kid
9. DlgAddNewDBGlade.py: Provide the window allowing user to enter new database links
(definitions).
10. DlgAddNewDB.py: Provides GUI-specific logic for base class (e.g. store state so that new
settings available when user clicks OK).
11. DlgGetUserPwdGlade.py: Provides the window to get the user’s password (hides text, allows
cancel/ok etc).
12. DlgGetUserPwd.py: Provides the GUI-specific logic for base class (e.g. store state so that
new settings available when user clicks OK).
13. LoggingWin.py: Extends the wx.PyLog window for use as a text output window for status
message by the application. This window is mostly useful to the developers and can be
hidden by closing it.
6.1.4
Kid templates
The Kid templates are used by the wxgui.AboutBox and wxgui.SimResPanel. The templates are
in the kid subfolder (src/portal/kid):
1. about.kid: the template used to generate the html shown in the “About” window accessed
from the GUI’s Help å About menu item.
2. resNodeDesc.kid: template used to generate the html shown in the simulation resources
panel. This converts the sequence of field name, field content pairs obtained from
SimResourcesDoc into an html table.
6.1.5
Database plug-ins
The portal supports any database for which a plug-in exists. All plug-ins stored in the
portal.db_plugins package (src/portal/db_plugins):
1. __init__.py: Makes db_plugins a package that can be imported from other modules.
2. BaseDBProxy.py: Provides the interface and logic that all database proxies must adhere to.
Actual proxies derive from the BaseDBProxy.BaseDBProxy class and implement abstract
methods defined in the base class. All plug-in modules must be named Ext*.py.
15
3. ExtFolderTree.py: Provides the plug-in for a database stored on the local host filesystem.
4. ExtHTTP.py: Provides the plug-in for a database stored on a remote host filesystem served via
HTTP. Supports basic and digest versions of authenticated access.
5. ExtMySQL.py: Provides the plug-in for a MySQL database on a remote host. Supports
authenticated access via the MySQL server’s User Management access control lists.
6.1.6
Launcher plug-ins
The launcher plug-ins allows control recipes from the simulation resources tree to be given to
external applications for processing. All are stored in the portal.launcher_plugins package
(src/portal/launcher_plugins):
1. __init__.py: Makes launcher_plugins a package that can be imported from other
modules.
2. BaseLauncherProxy.py: Provides the interface and logic that all launcher proxies must
adhere to. Actual proxies derive from the BaseLauncherProxy.BaseLauncherProxy class
and implement abstract methods defined in the base class. All plug-in modules must be
named Ext*.py.
3. ExtGNA.py: Launcher plug-in to process SEDReC control recipes.
6.2
SEDReC station
The complete dependency graph of the modules used to create the SEDReC application is shown
in Figure 5. This diagram, as for that of the SEDRA portal, was generated by running the
3
depgraph script .
The core part of SEDReC is limited to box 1 (expanded in Figure 7) and the ExecActions part of
box 2 (expanded in Figure 8 – olive green shapes). The document-related modules are the two
light-green shapes in box 2, and the GUI-related modules are the darker green shapes in box 3
(expanded in Figure 11). The following subsections give more details.
3
From Command shell, cd to src/gna_launcher and run: c:\Python24\Scripts\depgraph.py -e
BasicLogging,wx._core,Rpyc,tlslite,wx.py,wx.__version__,_ctypes,ctypes,_socket,_ssl,win32p
ipe,win32api,win32con,select main.py
16
1
3
2
Figure 5: Dependency graph of modules in SEDReC application.
Note that the wx.lib.pubsub module is used to send messages between the control modules of
SEDReC and its GUI modules. All messages coming from GUI as a result of the user clicking on
something start with the keyword ‘UI’. Similarly, all messages generated by the control modules
as a result of executing the actions start with ‘exec’. Various GUI panels listen for both ‘exec’
and ‘UI’ types of messages.
Note also that each action execution object (instance of ExecActions.*) is a Python
threading.Thread object. The application defines a “global message queue” for all action
executions to post messages to in a thread-safe manner. In addition, each action group provides a
“local message queue” for its children to post messages to for updates on their execution status.
The global message queue is for messages of “global” interest, whereas the “parent local message
queue” is for message specifically for an action’s parent. At the root of the action hierarchy is an
instance of the class derived from RecipeMgr.Launcher. The latter has access to the global queue
and the local queue for the root action of the recipe. The main GUI loop causes
RecipeMgr.Launcher to take messages off the two queues and sends them to the two methods of
the gna_launcher.main class derived from it. This class translates those messages into a form
that can be sent via wx.lib.pubsub as ‘exec.actionState’ and ‘exec.failureReason’ messages.
These messages will be received by the various GUI panels to update the application window.
This process is sketched in Figure 6 as a UML sequence diagram.
17
Global Queue
RecipeMgr.Launcher
wx.lib.pubsub
GUI
Action Exec 1
Action Exec 2
Action Exec 3
status 3a
status 1a
status 3b
status 2a
get status messages
generate pubsub messages
status 1b
send messages
status 2b
get status messages
status 1c
generate pubsub messages
send messages
Figure 6: Sequence diagrams of important SEDReC objects.
6.2.1
Application Core
Figure 7: SEDReC modules (yello) related to application core.
The modules in src/gna_launcher folder, related to core (control) of application, are:
18
1. __init__.py: this makes gna_launcher a package that can be imported from Python scripts
in src, such as recipe_station.pyw (which is just a start-up script for the application).
2. main.py: this module provides classes and functions that read the command line for run-time
parameters, read the application document (XML for the control recipe) into memory, define
a controller based on the class RecipeMgr.Launcher, instantiate the GUI, and create the
wxPython application manager. The controller merely provides an implementation for the
two abstract methods declared in RecipeMgr.Launcher. The implementation takes messages
generated by RecipeMgr.Launcher and sends them via the wx.lib.pubsub message channel.
Most of those messages will be used by the GUI.
3. RecipeMgr.py: this module provides the Launcher and InvalidXML classes. The latter is an
exception raised by Launcher when the document given to it is not valid XML (both
syntactically and semantically). Launcher is an abstract class for application control: it can
start and stop the control recipe and send recipe execution status messages. It provides two
methods which must be implemented in a derived class, for the main module to decide how
messages should be propagated to the rest of the system.
4. BasicLogging.py: this module makes possible the output of multi-level log messages. It is
used by most modules since most modules provide some form of debug output. For this
reason it is not shown in the figures.
6.2.2
Document: Action definitions
Figure 8: Modules defined for the document and controller aspects of SEDReC.
The document aspect of the application is represented by two modules in the
src/gna_launcher/ActionDefs folder:
19
1. __init__.py: makes the gna_launcher/ActionDefs folder into a Python sub-package that
can be accessed from the gna_launcher package.
2. Procedure.py: provides one class for every type of action supported, and a factory for each
one so that the action classes can be instantiated from XML document. All action types derive
from ActionDefs.ActionDef, as shown in Figure 9. This diagram shows the important
methods of the hierarchy: the class method fromXML() is a factory method that creates
instances of the concrete action classes; the executor() creates an instance of the class in
ExecActions module corresponding to the action definition class, and is used by Launcher
when the document is executed; the getActionDetails() returns the action details to be
used by the ActionDefView module to create a pretty-printed text rendition of the action
parameters.
ActionDef
+fromXML()
+executor()
+getActionDetails()
Sequence
TransferFiles
+fromXML()
+executor()
+fromXML()
Upload
+executor()
RunBase
Download
+executor()
+fromXML()
Run
+executor()
Start
+executor()
Figure 9: Relationship of action classes defined in ActionDefs module.
3. ActionDefView.py: this module provides classes that convert an action definition from
ActionDefs into a nicely formatted rendition of the action parameters. This is used as the
content generator for the action details GUI panel defined in wxgui.ActionDetails.
6.2.3
Execution of actions
The gna_launcher.ExecActions package, in the gna_launcher folder, defines modules that
make action definitions into threads of execution within the application.
1. __init__.py: makes the gna_launcher/ExecActions folder into a Python sub-package that
can be accessed from the gna_launcher package.
2. BaseExec.py: provides the BaseExec.Action class, as well as several exception classes, and
messaging classes and functions, to be used by all action executions defined in the sibling
20
modules (ExecActions.Group, ExecActions.Run, etc). The BaseExec.Action class is base
class for action classes defined in sibling modules, as shown in Figure 10. The most
important method it defines is the abstract _run() method which subclasses must implement
to execution the action (e.g., transfer files or start a program on a remote host).
Figure 10: Relationship of classes defined in ExecActions modules.
3. Group.py: provides the Group.Group class, to act as base class for any type of action which
represents a “group of child actions”. It is derived into Sequence.Sequence and eventually
will be derived into Parallel.Parallel (not yet implemented).
4. Run.py: provides the Run.Run and Run.Start action execution classes, which both derive
from Run.Process, which itself inherits from BaseExec.Action, as shown in Figure 10. The
Run.Run and Run.Start are two types of concrete action executions. A “run” is successful
when the program being run exits; a “start” is successful when a program can start. Hence a
program that started successfully but then crashed would fail a “run” but would succeed a
“start”.
5. Sequence.py: provides the Sequence.Sequence class which executes a sequence of “subactions” in order. This is in contrast to Parallel.Parallel (not yet implemented) which
executes sub-actions in parallel.
6. Upload.py: provides the Upload.Upload and Upload.Download classes to execution file
transfer operations and send progress status updates to the application.
21
6.2.4
GUI
Figure 11: This is the caption for the figure shown above.
The gna_launcher.wxgui package, located in the gna_launcher folder, defines the GUI panels
and GUI windows for the application. As for the SEDRA portal, the wxGlade tool was used to
create the GUI in a WYSIWYG environment. The project file is gui.wxg. Files auto-generated
from wxGlade end with the word “Glade” (e.g. HostWidgetGlade.py) and merely provide GUI
“look and feel”. Files with same root name but no “Glade” on end are implementation files: they
provide application-specific GUI behaviour, as explained below.
1. __init__.py: makes the gna_launcher/ExecActions folder into a Python sub-package that
can be accessed from the gna_launcher package
2. ActionDetailsGlade.py: wxGlade-generated file that defines a wx.Panel for the action
details (Figure 11). The action details are obtained from the module
gna_launcher.ActionDefs.ActionDefView.
22
Figure 12: Action Details panel of SEDReC station.
3. ActionDetailsPanel.py: sets the labels and text field content of the action details panel
when an actionDefDetails message is received. This message is generated by the
MainFrame.MainFrame window when a user has selected a new action from one of the other
panels (Host widgets panel, failure reasons panel, or action tree panel).
4. FailureReasonsGlade.py: wxGlade-generated file that defines a wx.Panel for the action
failures that occur during recipe document execution. This panel shows a list of most recent
failure messages.
Figure 13: Execution Failures panel of SEDReC station.
5. FailureReasons.py: derives from FailureReasonsGlade.FailureReasonsGlade and
implements the behaviour of populating the “reasons for execution failures” panel with
information received via ‘exec.reasonFailed’ messages. These messages are generated by
the application controller, i.e. by gna_launcher.main which gets its data from
RecipeMgr.Launcher.
6. HostWidgetGlade.py: defines the simulation host widget panels which will be instantiated,
once for every simulation host reachable, by the wxgui.SimHostsPanel module. The host
23
widget shows the actions defined in the recipe, referring to the given host. These action labels
change color when the recipe is being executed, to indicate the status of the execution.
7. HostWidget.py: implements the font change behaviour to highlight whichever action has
been selected by user in one of the other panels (via the ‘ui.actionSelected’ message), and
the color behaviour to indicate action execution status (via the ‘exec.actionState’
messages). See Figure 14.
Figure 14: Four simulation host widgets in SEDReC station.
8. LoggingWin.py: this extends the wx.PyLog window for use as a text output window for status
message by the application. This window is mostly useful to the developers and can be
hidden by closing it.
9. MainFrameGlade.py: defines the main frame (window) of the application and the tree panel.
10. MainFrame.py: implements GUI behaviour for the window and panel: what aspects of GUI to
change when user has started the execution or is closing the application.
11. SimHostsPanelGlade.py: defines the wx.Panel on which a grid of available simulation hosts
appears via instances of HostWidget.HostWidget class. See Figure 14.
12. SimHostsPanel.py:
implements
the
GUI
behaviour,
i.e.
classes for each simulation host reachable.
instantiating
HostWidget.HostWidget
13. SortImages.py: provides bitmaps for images of arrows to be used by the columns of the
FailureReasons panel.
6.3
SEDReDE portal
The SEDReDE portal user interface, its functionality and its data model have been implemented
in the form of six JAVA classes as illustrated in Figure 15 in UML notation. They are explained
in the following subsections.
24
1
arpMain
1
1
*
1
dbUtils
dbTreeNode
1
summaryPanel
1
*
summaryField
Figure 15: Java classes in SEDReDE.
6.3.1
Module 1: arpMain.java
The main class of the SEDReDE portal, arpMain.java, is implemented as a Swing Applet, ie. it
is derived from the javax.swing.JApplet. The Swing Applet class is itself derived from the
java.applet.Applet class. The choice of using a Swing Applet removed the limitation of using
the original abstract window toolkit and provided access to a larger number of convenient GUI
widgets and thereby made data abstraction and manipulation easier. The primary function of
arpMain.java is the presentation and management of the user interface. As shown in Figure 15,
the arpMain.java class serves as a top level container for all of the other classes of the system
which will be described in detail below.
In addition to it being a sub-class of javax.swing.JApplet, the arpMain class also implements a
number of event listener interfaces. Each of these is described briefly in the following table.
CellEditorListener
MouseListener
TreeSelectionListener
–
This interface is used to detect changes widgets that contain
text cells;
–
Used in arpMain.java, to manage name changes to nodes in
the simulation resource tree.
–
This interface is used to handle a number of mouse events
that are separate from GUI buttons;
–
Used in arpMain.java, to detect and handle the right-click
event on the simulation resource tree.
–
This interface is used to detect changes in the selected value
25
of widgets that contain selectable nodes/values;
ActionListener
DocumentListener
6.3.2
–
Used in arpMain.java, to detect changes in the selected
node of the simulation resource tree.
–
This interface is generic listener that captures user
interaction events from the GUI.
–
Used in arpMain.java, to detect buttons being pressed as
well as selections made from popup menus.
–
This interface is used to detect changes in the value of a text
field or text area.
–
Used in arpMain.java, to detect changes in both the name
and location of the selected node of the simulation resource
tree.
Module 2: dbTreeNode
The dbTreeNode class is used to abstract the details of a singular simulation resource in the
simulation resource tree. It extends the javax.swing.tree.DefaultMutableTreeNode class of
the JAVA 2 API. The javax.swing.tree.DefaultMutableTreeNode class represents the
default tree node object provided by the JAVA 2 API for adding nodes and leaves to the JTree
GUI component of the JAVA Swing API. The JTree component class is used in SEDReDE to
represent the hierarchy of all simulation resources. As such, the default tree node object proved
insufficient for representing simulations resources. By extending this base class and adding a
number of attributes and methods, manipulation and modification of simulation resources became
greatly simplified and more efficient. The relationship of the dbTreeNode class to its predecessor
is shown in Figure 16.
Figure 16: The dbTreeNode class in SEDReDE.
26
According to the data model of the SEDReDE system, every simulation resource has a name and
a location. These values are always reflected in the Node Information section (see below) of the
user interface when a specific node is selected. The other attributes are used primarily to identify
the specific node within its appropriate table in the database as well as to track its relationship to
its parent and children.
6.3.3
Module 3: summaryField
The data model of the SEDReDE system permits the addition of any number of summary fields to
a given simulation resource. A summary field is a non-standard meta-data field that is added to a
simulation resource at the discretion of the user. It represents additional information about the
simulation resource that is not captured in the attributes of the dbTreeNode class (discussed
above). A summary field contains a name and a value associated with that name. In addition to
data abstraction, the purpose of the summaryField class is to graphically represent the
information it stores. As such, the class itself is implemented as a generic container and extends
the javax.swing.JPanel class of the JAVA Swing API. Apart from the name and value fields,
no special attributes were added to this class but this implementation provided a means of treating
the graphical and informational portions of the summary field as a single unit which could be in
turn be added to another graphical container (i.e. summaryPanel class – see below). The only
functionality contained within the summaryField class was the removal of itself from its
container (via the “del” button).
6.3.4
Module 4: summaryPanel
The summaryPanel class is implemented as a simple graphical container that is used to
store/display all summary fields for a given simulation resource. It is also derived from the
javax.swing.JPanel. Each instance of the arpMain applet has a single instance of the
summaryPanel class. The summaryPanel class is drawn in the Summary information section of
the user interface (see below). As simulation resources are selected by the user in the resource
tree, their respective complements of summary fields are displayed in the summaryPanel. The
summaryPanel class is designed in such a way that it has methods to add, remove and clear
summary fields for the selected simulation resource.
6.3.5
Module 5: dbUtils
The dbUtils class is used to abstract the data model of the SEDReDE database schema used by
the MySQL DBMS. It serves as an API with methods that manage connection to the database as
well data manipulation. This class is instantiated and used exclusively by the arpMain class as a
proxy between the user and the database.
6.3.6
Module 6: fieldData
The fieldData class serves as a “data currency” container for summary field information.
Whenever a summary field is added or removed from a simulation resource, the data portion of
the summary field is moved to or from the database using the fieldData class. Figure 17 shows
the attributes and methods of the fieldData class.
27
Figure 17: The fieldData class in SEDReDE portal.
6.4
Schema for relational databases
The System Specification Document describes the data structure used to describe simulation
resources. This data model supports multiple group entities (e.g. Simulation Systems and System
Groups) containing the meta-data of one resource. E.g. both Project A, represented as a
Simulation System, and Project B, represented as a Component Group, could refer to Simulation
Document C, or System Component D. This means that many of the simulation resources involve
many-to-many relationships. A schema was designed that implements the data model discussed in
the Specification document and should be valid for any relational database. An implementation
exists for MySQL database types, accessible from the SEDRA and SEDReDE portals.
The hierarchical nature of the data model, coupled with the many-to-many relationships, leads to
a large number of tables. Since these tables will be accessed programmatically from SEDRA and
SEDReDE portals, it is important to use a naming convention that facilitates algorithmic
processing of the database schema.
The following words have been abbreviated in the naming of the database tables:
• System
–
sys
• Simulation
–
sim
• Component
–
comp
• Document
–
doc
All table names are purely lower case, do not contain plurals, and use the above abbreviations.
When a table contains several words, the words are separated by one underscore ‘_’ character.
Tables that relate two types of entities within the database (so-called many-to-many tables, e.g.
the table that identifies which documents are in which document groups) use the table names for
the entities, separated by two underscores. E.g. sys_group__sim_doc is the table that identifies
which system groups refer which simulation documents.
28
The following tables were created, as per the data model described below:
• comp_group (for ComponentGroup resources)
• comp_group__comp_group
• comp_group__doc_group
• comp_group__sim_c2
• comp_group__sim_comp
• comp_group__sim_doc
• comp_group__summary_field
• doc_group (for DocumentGroup resources)
• doc_group__doc_group
• doc_group__sim_doc
• doc_group__summary_field
• sim_c2 (for LaunchConfig resources)
• sim_c2__summary_field
• sim_comp (for SysComponent resources)
• sim_comp__doc_group
• sim_comp__sim_c2
• sim_comp__sim_doc
• sim_comp__summary_field
• sim_doc (for Document resources)
• sim_doc__summary_field
• sim_sys (for SimSystem resources)
• sim_sys__comp_group
• sim_sys__doc_group
• sim_sys__sim_c2
• sim_sys__sim_comp
• sim_sys__sim_doc
• sim_sys__summary_field
• sim_url
• summary_field
• sys_group (for SystemGroup resources)
29
• sys_group__doc_group
• sys_group__sim_doc
• sys_group__sim_sys
• sys_group__summary_field
• sys_group__sys_group
There are two types of tables in the above list that are worth describing further: simulation
resources tables (with the element names as described in Annex A), and resource relationship
tables. The former simply list the various simulation resources catalogued, whereas the latter
defines what resource is defined within which other resource. The sys_group, sim_sys,
comp_group, sys_comp, doc_group, sim_doc, and sim_c2 tables are all simulation resources
tables.
The table columns were named according to the following rules:
• Primary key: name of table + “_id”. E.g. if the table name is “sim_sys” then the primary key
column name for that table would be “sim_sys_id”.
• Foreign Key: name of table + “_fk”. E.g. if a table has a foreign key column referring to the
primary key column of table “sim_sys”, then the foreign key column name would be
“sim_sys_fk”.
• Constraint name: table name abbreviation + “__” + primary key + “__constraint”. The
abbreviation is the first letter of words forming the table name, i.e. “sg” for sys_group, etc.
• Foreign key constraint name: same as for constraint, but ends with “__fk” instead of
“__constraint”
Annex A describes the database schema implementing the above naming conventions and the
data model from the Specification document.
30
7
7.1
User Interface Design
SEDRA portal
The GUI of the SEDRA portal is shown in Figure 18. The individual parts of the GUI were
discussed in section 6.1.3.
Connection
panel
Simulation
resources
panel
Filter subpanel
Simulation
resources
tree
Selected
simulation
resource
description
(Summary)
Figure 18: SEDReC station GUI, top section.
31
7.2
SEDReC station
The GUI of the SEDReC station is shown in Figure 19 and Figure 20. The individual parts of the
GUI were discussed in section 6.2.4.
Operations
using recipe
Action hosts
available
Recipe as
Action Tree
Details of
selected
action
Figure 19: SEDReC station GUI, top section.
32
Reasons for
execution
failures
Interactive
Python
interpreter
Figure 20: SEDReC station GUI, bottom section.
7.3
SEDReDE portal
The user interface of the SEDReDE portal is designed to resemble the original SEDRA portal and
to provide an intuitive means to view and manipulate data in the SEDReDE database. It is
divided into three main sections:
1.
The Simulation Resource Tree Section
2.
The Resource Information Section
The simulation resource tree section of the user interface contains a tree structure representing the
current hierarchy of the contents of the database. Each simulation resource in the database is
represented by a single node on the tree. The resource information section represents the data and
meta-data of a simulation resource. As simulation resources in the tree are selected, their
respective data and meta-data are displayed in the resource information section of the user
interface. The resource information section itself is further divided into two sub-sections: node
information and summary information. These relationships are presented in Figure 21.
33
Node
Information
Simulation
Resource
Tree
Summary
Information
Figure 21: The SEDReDE web portal.
By interacting directly with the simulation resource tree, the user can add, remove or rename a
simulation resource. This is done right-clicking on a tree node and selecting the desired option
from the resulting popup menu. The user can also modify the name of a simulation resource or its
location by typing in either field of the node information section of the user interface. These
changes are reflected immediately but they are not committed to the database until the user
presses the “Apply” button. This button is only enabled when there are uncommitted changes
pending. The user can also add a summary field to the current simulation resource by pressing
the “Add” button. A single summary field can be removed from a simulation resource by
pressing its respective “del” button in the summary section of the user interface. Similarly, all
summary nodes for the currently selected simulation resource can be removed by pressing the
“Clear” button.
The SEDReDE portal interface has been developed in such a way that the user is never in danger
of doing anything that will compromise the integrity of the database or violate its schema. When
adding a simulation resource to a parent container, the user is only presented with resources that
are in keeping with the relationships that are dictated by the database schema. In addition, if the
user attempts to navigate to a different simulation resource on the tree while uncommitted
changes exist for the current resource, appropriate warnings are given and the use is given the
choice of whether or not to commit these changes before continuing.
34
References .....
[1] Python (online), http://www.python.org (Access date: 12 Nov. 2008)
[2] wxPython (online), http://www.wxpython.org (Access date: 12 Nov. 2008)
[3] wxGlade (online), http://wxglade.sourceforge.net/ (Access date: 12 Nov. 2008)
[4] elementTree (online), http://effbot.org/zone/element-index.htm (Access date: 12 Nov. 2008)
[5] lxml (online), http://pypi.python.org/pypi/lxml/ (Access date: 12 Nov. 2008)
[6] trang (online), http://www.thaiopensource.com/relaxng/trang.html (Access date: 12 Nov.
2008)
[7] Kid (online), http://www.kid-templating.org/ (Access date: 12 Nov. 2008)
[8] SQLAlchemy (online), http://www.sqlalchemy.org/ (Access date: 12 Nov. 2008)
[9] MySQL-Python (online), http://mysql-python.sourceforge.net/ (Access date: 12 Nov. 2008)
[10] RPyC (online), http://rpyc.wikispaces.com/ (Access date: 12 Nov. 2008)
[11] Ctypes (online), http://python.net/crew/theller/ctypes/ (Access date: 12 Nov. 2008)
[12] J2SDK (online), http://java.sun.com (Access date: 12 Nov. 2008)
[13] MySQL DBMS (online), http://dev.mysql.com/downloads/mysq/ (Access date: 12 Nov.
2008)
[14] Apache Web Server (online), http://www.apache.org (Access date: 12 Nov. 2008)
[15] Mozilla FireFox (online), http://www.mozilla.org (Access date: 12 Nov. 2008)
35
36
Annex A
Tables for Relational Databases
The following list describes each table:
1. comp_group table:
comp_group_id
name
dir
xinclude
INTEGER UNSIGNED
VARCHAR(80)
VARCHAR(255)
BOOLEAN
NOT NULL AUTO_INCREMENT
NOT NULL
NULL
DEFAULT TRUE
2. comp_group__comp_group table:
comp_group_id_fk
comp_group_id_fk2
pred_code
name_override
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
3. comp_group__doc_group table:
comp_group_id_fk
doc_group_id_fk
pred_code
name_override
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
4. comp_group__sim_c2 table:
comp_group_id_fk
sim_c2_id_fk
pred_code
name_override
5. comp_group__sim_comp table:
comp_group_id_fk
sim_comp_id_fk
pred_code
name_override
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
6. comp_group__sim_doc table:
comp_group_id_fk
sim_doc_id_fk
pred_code
name_override
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
7. comp_group__summary_field table:
comp_group_id_fk
INTEGER UNSIGNED
NOT NULL
37
summary_field_id_fk INTEGER UNSIGNED
name_override
VARCHAR(80)
NOT NULL
NULL
8. doc_group table:
doc_group_id
name
dir
xinclude
INTEGER UNSIGNED
VARCHAR(80)
VARCHAR(255)
BOOLEAN
NOT NULL
AUTO_INCREMENT
NOT NULL
NULL
DEFAULT TRUE
9. doc_group__doc_group table:
doc_group_id_fk
doc_group_id_fk2
pred_code
name_override
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
10. doc_group__sim_doc table:
doc_group_id_fk
sim_doc_id_fk
pred_code
name_override
11. doc_group__summary_field table:
doc_group_id_fk
INTEGER UNSIGNED
name_override
VARCHAR(80)
NOT NULL
NOT NULL
NULL
12. sim_c2 table:
sim_c2_id
name
app_type
actions_defn
sim_url_id_fk
INTEGER UNSIGNED
VARCHAR(80)
VARCHAR(255)
VARCHAR(255)
INTEGER UNSIGNED
NOT NULL
NULL
NULL
NULL
13. sim_c2__summary_field table:
sim_c2_id_fk
INTEGER UNSIGNED
name_override
VARCHAR(80)
NOT NULL
NOT NULL
NULL
14. sim_comp table:
sim_comp_id
name
dir
xinclude
38
INTEGER UNSIGNED
VARCHAR(80)
VARCHAR(255)
BOOLEAN
NOT NULL
NULL
DEFAULT TRUE
15. sim_comp__doc_group table:
sim_comp_id_fk
doc_group_id_fk
pred_code
name_override
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
16. sim_comp__sim_c2 table:
sim_comp_id_fk
sim_c2_id_fk
pred_code
name_override
17. sim_comp__sim_doc table:
sim_comp_id_fk
sim_doc_id_fk
pred_code
name_override
18. sim_comp__summary_field table:
sim_comp_id_fk
INTEGER UNSIGNED
name_override
VARCHAR(80)
NOT NULL
NOT NULL
NULL
19. sim_doc table:
sim_doc_id
name
dir
sim_url_id_fk
INTEGER UNSIGNED
VARCHAR(80)
VARCHAR(255)
INTEGER UNSIGNED
NOT NULL
NULL
NULL
20. sim_doc__summary_field table:
sim_doc_id_fk
INTEGER UNSIGNED
name_override
VARCHAR(80)
NOT NULL
NOT NULL
NULL
21. sim_sys table:
sim_sys_id
name
dir
xinclude
INTEGER UNSIGNED
VARCHAR(80)
VARCHAR(255)
BOOLEAN
NOT NULL
NULL
DEFAULT TRUE
22. sim_sys__comp_group table:
39
sim_sys_id_fk
comp_group_id_fk
pred_code
name_override
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
23. sim_sys__doc_group table:
sim_sys_id_fk
doc_group_id_fk
pred_code
name_override
24. sim_sys__sim_c2 table:
sim_sys_id_fk
sim_c2_id_fk
pred_code
name_override
25. sim_sys__sim_comp table:
sim_sys_id_fk
sim_comp_id_fk
pred_code
name_override
26. sim_sys__sim_doc table:
sim_sys_id_fk
sim_doc_id_fk
pred_code
name_override
27. sim_sys__summary_field table:
sim_sys_id_fk
INTEGER UNSIGNED
name_override
VARCHAR(80)
NOT NULL
NOT NULL
NULL
28. sim_url table:
sim_url_id
url_address
INTEGER UNSIGNED
VARCHAR(1024)
NOT NULL
INTEGER UNSIGNED
VARCHAR(255)
VARCHAR(255)
NOT NULL
NULL
29. summary_field table:
summary_field_id
name
content
40
30. sys_group table:
sys_group_id
name
dir
xinclude
INTEGER UNSIGNED
VARCHAR(80)
VARCHAR(255)
BOOLEAN
NOT NULL
NULL
DEFAULT TRUE
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
31. sys_group__doc_group table:
sys_group_id_fk
doc_group_id_fk
pred_code
name_override
32. sys_group__sim_doc table:
sys_group_id_fk
sim_doc_id_fk
pred_code
name_override
33. sys_group__sim_sys table:
sys_group_id_fk
sim_sys_id_fk
pred_code
name_override
34. sys_group__summary_field table:
sys_group_id_fk
summary_field_id_fk
name_override
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(80)
NOT NULL
NOT NULL
NULL
35. sys_group__sys_group table:
sys_group_id_fk
sys_group_id_fk2
pred_code
name_override
INTEGER UNSIGNED
INTEGER UNSIGNED
VARCHAR(20)
VARCHAR(80)
NOT NULL
NOT NULL
NULL
NULL
The column names from the above tables, following the naming conventions mentioned above,
indicate relationships that are more easily seen with diagrams. Those are given next.
41
Figure 22: DB schema tables related to sys_group (SystemGroup)
Figure 22 shows:
• The SystemGroup data model shows the relationship between the sys_group table and other
simulation resource tables that it is linked to.
• The sys_group table has a one-to-many relationship with the sys_group__doc_group table
and the doc_group table also has a one-to-many relationship with the
sys_group__doc_group table to form the sys_group__doc_group table.
• The sys_group__sim_sys table comprises of foreign keys from the sys_group table and the
sim_sys table.
• The sys_group__sys_group table has two foreign keys both from the sys_group table to
form a table of system groups that can contain other system groups.
• The sys_group__summary_field table comprises of foreign keys from the sys_group and
summary_field tables.
• The sys_group__sim_doc table comprises of foreign keys from the sys_group and sim_doc
tables.
42
Figure 23: DB schema tables related to sim_sys (SimulationSystem)
Figure 23 shows:
• The SimSystem data model shows the relationship between the sim_sys table and other
simulation resource tables that it is linked to.
• The sim_sys table has a one-to-many relationship with the sim_sys__sim_c2 table and the
sim_c2 table has a one-to-many relationship with the sim_sys__sim_c2 to form the
sim_sys__sim_c2 table.
• The sim_sys__sim_comp table comprises of foreign keys from the sim_sys and sim_comp
table.
• The sim_sys__sim_doc table comprises of foreign keys from the sim_sys and sim_doc table.
• The sim_sys__summary_field table comprises of foreign keys from the sim_sys and
summary_field table.
• The sim_sys__doc_group table comprises of foreign keys from the sim_sys and doc_group
table.
• The sim_sys__comp_group table comprises of foreign keys from the sim_sys and
comp_group table.
43
Figure 24: DB schema tables related to comp_group (ComponentGroup)
Figure 24 shows:
• The ComponentGroup simulation model shows the relationship between the comp_group
table and other simulation resource tables.
• The comp_group__comp_group table has two foreign keys both from the com_group table
to form a table of component groups that can contain other component groups.
• The comp_group__sim_c2 table comprises of foreign keys from the comp_group and
sim_c2 tables.
• The comp_group__sim_doc table comprises of foreign keys from the comp_group and
sim_doc tables.
• The comp_group__doc_group table comprises of foreign keys from the comp_group and
doc_group tables.
• The comp_group__summary_field table comprises of foreign keys from the comp_group
and summary_field tables.
• The comp_group__sim_comp table comprises of foreign keys from the comp_group and
sim_comp tables.
44
Figure 25: DB schema tables related to sim_comp (SysComponent)
Figure 25 shows:
• The SysComponent data model shows the relationship between the sim_comp table and
other simulation resource tables that it is linked to.
• The sim_comp__doc_group table comprises of foreign keys from the sim_comp and the
doc_group tables.
• The sim_comp__sim_c2 table comprises of foreign keys from the sim_comp and the sim_c2
tables.
• The sim_comp__summary_field table comprises of foreign keys from the sim_comp and
summary_field tables.
• The sim_comp__sim_doc table comprises of foreign keys from the sim_comp and sim_doc
tables.
Figure 26: DB schema tables related to doc_group (DocumentGroup)
45
Figure 26 shows:
• The DocumentGroup data model shows the relationship between the doc_group table and
other simulation resource tables that it is linked to.
• The doc_group__doc_group table has two foreign keys both from the doc_group table to
form a table of document groups that can contain other document groups.
• The doc_group__sim_doc table comprises of foreign keys from the doc_group table and
sim_doc table.
46
List of symbols/abbreviations/acronyms/initialisms
DND
Department of National Defence
DRDC
Defence Research & Development Canada
DRDKIM
Director Research and Development Knowledge and Information
Management
R&D
Research & Development
SE
Synthetic Environments
SEDR
Synthetic Environment Distributed Resources
SEDRA
Synthetic Environment Distributed Resources Access
SEDReC
Synthetic Environment Distributed Resources Control
SEDReDE
Synthetic Environment Distributed Resources Data Entry
SNE
Simulation Network Exploitation
WYSIWYG
What-you-see-is-what-you-get
47
This page is intentionally left blank
48
DOCUMENT CONTROL DATA
(Security classification of title, body of abstract and indexing annotation must be entered when the overall document is classified)
1.
ORIGINATOR (The name and address of the organization preparing the document.
Organizations for whom the document was prepared, e.g. Centre sponsoring a
contractor's report, or tasking agency, are entered in section 8.)
2.
UNCLASSIFIED
1135 Innovation Dr. Suite 300
Ottawa, ON K2K 3G7
3.
SECURITY CLASSIFICATION
(Overall security classification of the document
including special warning terms if applicable.)
TITLE (The complete document title as indicated on the title page. Its classification should be indicated by the appropriate abbreviation (S, C or U)
in parentheses after the title.)
Simulation network exploitation: SEDR tool suite: system design document
4.
AUTHORS (last name, followed by initials – ranks, titles, etc. not to be used)
Schoenborn, O., Mikhael, B., Chawla, B.
5.
DATE OF PUBLICATION
(Month and year of publication of document.)
April 2009
7.
6a. NO. OF PAGES
6b. NO. OF REFS
(Total containing information,
(Total cited in document.)
including Annexes, Appendices,
etc.)
60
0
DESCRIPTIVE NOTES (The category of the document, e.g. technical report, technical note or memorandum. If appropriate, enter the type of report,
e.g. interim, progress, summary, annual or final. Give the inclusive dates when a specific reporting period is covered.)
Contract Report
8.
SPONSORING ACTIVITY (The name of the department project office or laboratory sponsoring the research and development – include address.)
Defence R&D Canada – Ottawa
3701 Carling Avenue
Ottawa, Ontario K1A 0Z4
9a. PROJECT OR GRANT NO. (If appropriate, the applicable research
and development project or grant number under which the document
was written. Please specify whether project or grant.)
13jb
10a. ORIGINATOR'S DOCUMENT NUMBER (The official document
number by which the document is identified by the originating
activity. This number must be unique to this document.)
W8475-06BM04
9b. CONTRACT NO. (If appropriate, the applicable number under
which the document was written.)
W8475-06BM04
10b. OTHER DOCUMENT NO(s). (Any other numbers which may be
assigned this document either by the originator or by the sponsor.)
11. DOCUMENT AVAILABILITY (Any limitations on further dissemination of the document, other than those imposed by security classification.)
Unlimited
12. DOCUMENT ANNOUNCEMENT (Any limitation to the bibliographic announcement of this document. This will normally correspond to the
Document Availability (11). However, where further distribution (beyond the audience specified in (11) is possible, a wider announcement
audience may be selected.))
Unlimited
13. ABSTRACT (A brief and factual summary of the document. It may also appear elsewhere in the body of the document itself. It is highly desirable
that the abstract of classified documents be unclassified. Each paragraph of the abstract shall begin with an indication of the security classification
of the information in the paragraph (unless the document itself is unclassified) represented as (S), (C), (R), or (U). It is not necessary to include
here abstracts in both official languages unless the text is bilingual.)
Exploitation (SNE) project. Together they form the Synthetic Environment Distributed
Resources (SEDR) Tool Suite. It is intended for the developers in charge of maintaining or
extending the system. The system consists of three tools: the SEDR Access (SEDRA) portal for
accessing stored knowledge regarding simulation resources; the SEDR Control (SEDReC)
station for executing system control recipes for re-use of complex distributed simulations; and
the SEDR Data Entry (SEDReDE) portal, which allows the user to add simulation resources
knowledge to a SEDRA-compatible database.
Ce document présente la conception du kit d’outils créé pour le projet d'Exploitation du Réseau
de Simulation. Ensemble, ces outils forment l’instrument des ressources distribuées des
environnements synthétiques. Ce kit est destiné aux développeurs en charge du maintien ou de
l’extension du système. Le kit d’outils se compose de trois outils: la portail d’accès pour l'accès
a l’information sur les différentes ressources de simulation, la station de contrôle pour
l'exécution et le démarrage du système de simulations distribuées ainsi que la réutilisation de
ces simulations complexes, et le portail de saisie de qui permet à l'utilisateur d'ajouter des
ressources de simulation à la base de données appropriée.
14. KEYWORDS, DESCRIPTORS or IDENTIFIERS (Technically meaningful terms or short phrases that characterize a document and could be
helpful in cataloguing the document. They should be selected so that no security classification is required. Identifiers, such as equipment model
designation, trade name, military project code name, geographic location may also be included. If possible keywords should be selected from a
published thesaurus, e.g. Thesaurus of Engineering and Scientific Terms (TEST) and that thesaurus identified. If it is not possible to select
indexing terms which are Unclassified, the classification of each should be indicated as with the title.)
Modeling & Simulation, Synthetic Environments, Distributed Simulation, Data Management,
Knowledge Management, Remote Access, Software

Simulation network exploitation SEDR tool suite: system design

Transcription

Similar documents

George Middle School / Homepage

Sim Card leaflet AW Fin.indd

http://kucrl.org/sim - Training and Technical Assistance Center (T/TAC)

iSIM iSIM

Power devices - Crosslight Software

convergys - Govan Brown

Designing a Simulation Lab that`s Right For You

the envoy - The LIFT System

opportunities - SIM New Zealand