Roadmap for the MoD Cyber Test Range

Transcription

Roadmap for the MoD Cyber Test Range
Date
Status
10 September 2012
Final
FINAL | Roadmap for the MoD Cyber Test Range | 10 September 2012
Colophon
IT Agency Group
Operations
Van der Burchlaan 31
Postbus 90822
2509 LV Den Haag
Contact
M.P. Lindgreen
Projectmanager
M 06 516 423 79
[email protected]
Commisioned by
Author(s)
Delft TopTech
M.P. Lindgreen
No content of this thesis is to be used without the permission of the author.
The views expressed in this thesis are those of the author and do not
necessarily represent the views of the Netherlands Ministry of Defence.
Page 2 of 78
Table of contents
Colophon ................................................................................................... 2
Preface ...................................................................................................... 5
Executive summary.................................................................................... 6
1
1.1
Introduction .................................................................................... 8
Problem area .................................................................................. 8
1.2
Research goal ................................................................................. 9
1.3
Research scope .............................................................................. 10
1.4
Results to be achieved .................................................................... 10
1.5
Research methodology .................................................................... 11
2.1
Cyber operations ........................................................................... 12
Introduction .................................................................................. 12
2.2
Context and terminology ................................................................. 12
2.3
Cyber operations in perspective ........................................................ 14
2.4
Developments in cyber test ranges ................................................... 18
2.5
Summary ...................................................................................... 23
3.1
MoD Cyber Test Range ................................................................... 24
Introduction .................................................................................. 24
3.2
Methodology .................................................................................. 24
3.3
Cyber Test Range business functions ................................................. 25
3.4
Cyber Test Range services ............................................................... 26
3.5
Cyber Test Range requirements ........................................................ 38
3.6
Summary ...................................................................................... 41
4.1
Roadmap for the Cyber Test Range ................................................ 42
Introduction .................................................................................. 42
4.2
Priority Cyber Test Range business functions ...................................... 42
4.3
Cyber Test Range maturity model ..................................................... 45
4.4
Cyber Test Range roadmap .............................................................. 53
4.5
Validation ...................................................................................... 56
4.6
Summary ...................................................................................... 57
5.1
Conclusions and recommendations ................................................ 58
Conclusions ................................................................................... 58
5.2
Recommendations .......................................................................... 60
2
3
4
5
6
Reflection ...................................................................................... 61
Page 3 of 78
7
Bibliography .................................................................................. 63
Appendix I.
Detailed overview CTR business functions ........................ 65
Appendix II.
Interview protocol ............................................................ 68
Appendix III.
Interviews expectations towards a cyber test range ...... 70
Appendix IV.
Calculations of priorities per CTR business function .......... 71
Appendix V.
Calculations of CTR requirements’ complexity ................... 73
Appendix VI.
Priority of CTR business functions ..................................... 74
Appendix VII.
Requirements for delivering business functions ............. 77
Page 4 of 78
Preface
This master thesis is the last and main deliverable for the study Master of IT
Management at Delft Toptech, a subsidiary of the Delft University of Technology.
This thesis explores the phenomena of cyber test ranges in support of cyber
operations and the applicability of a cyber test range to the Netherlands Ministry of
Defence (MoD). Although cyber has been around for some time, it is my personal
belief that we are at the forefront and many more developments are to be expected.
The same applies to cyber test ranges. It is certainly a new topic for the MoD but it
is also a fairly new topic in general where much more research and development is
needed. Therefore, I am delighted to have been able to conduct research in such an
exciting area.
This research is the result of contributions, insights and fruitful discussions with
many professionals. I would like to take the opportunity to express my sincere
gratitude to Jan van den Berg, who is an associate professor at the Delft University
of Technology, for his time, suggestions and inspiring comments during the process
of research. These comments kept me motivated and craving to take this research
to a next level. Furthermore, I would like to thank Colonel Hans Folmer for
suggesting and granting the opportunity to research this topic and I also would like
to thank my colleagues at the MoD for taking the time to share their views on all
topics related to the cyber test range and for their essential information.
In alphabetical order:
Mr. R. Hinfelaar
Warrant Officer R. Hubertse
Lieutenant J. Kan
Lieutenant Colonel M. Verhagen
Major P. ‗t Hoen
Major G. Wens
Colleagues from the Military Intelligence and Security Service
Moreover, I would like to thank the following persons for reviewing this research and
offering suggestions.
In alphabetical order:
Mr. I. Lamit
Mrs E. Lindgreen
Last but certainly not least, I would like to thank my wife, Katelijn and my son,
Merlijn for their patience and support during the worktimes in the weekends and
evening hours.
Marko Lindgreen
Voorburg, September 2012
Page 5 of 78
Executive summary
The Netherlands Ministry of Defence (MoD) is strengthening its ambition in the field
of digital resilience and cyber operations. The Cyber Test Range (CTR) will be one of
the functionalities that will support the new cyber operations. A CTR can be
considered as a ‗digital shooting range‘, comparable to a shooting range in the
physical world, wherein military personnel can conduct military exercises, train their
drills and test their skills.
Cyber operations is a relatively new topic for the MoD and will play an increasing
role in its future weapons arsenal. In the slipstream of these new developments
comes the CTR which is one of the functionalities that needs to be defined, designed
and subsequently implemented. Therefore, the following research goal is
formulated:
“The goal is to design a roadmap for the development of a cyber test range.”
The research methodology aims to (a) understand the context of cyber operations
and cyber test ranges, (b) perform an analysis in order to determine the business
functions and requirements applicable to the MoD CTR and (c) design a roadmap for
the CTR based on the outcome of the analysis phase.
Through cyber operations the military have the capability, along with other military
capabilities, to defend and/ or attack (or gather intelligence) in order to achieve
political or military objectives or effects. Cyber operations consist of offensive,
defensive and intelligence capabilities in which (a) cyber defense is aimed at
protecting its own networks and systems, (b) cyber attack is aimed at disrupting,
denying, degrading or destroying networks and systems and (c) cyber intelligence
enables intelligence collection through networks and systems.
CTR‘s are virtual environments that are used for research and development,
evaluation and training purposes within the cyber domain. From a military
perspective, cyber test ranges can be seen as an environment that offer
stakeholders the ability to - more effectively - defend and attack (or gather
intelligence regarding) critical infrastructures or military capabilities. Several
countries possess CTR‘s or are in the process of implementing cyber test ranges.
The MoD business expectations towards the CTR are categorized into CTR business
functions. The generic business functions are positioned on the level of cyber
operations. These business functions offer the ability to support daily operations and
research and development. To support daily operations, the CTR can deliver
business functions that (a) enable personnel to act in the cyber domain, (b) assess
the effectiveness of current capabilities and (c) respond to cyber attacks or to
conduct cyber attacks or intelligence. To enable research and development, the CTR
can deliver business functions that offer a platform to (a) conduct applied research
into future cyber solutions or to (b) research whether external solutions add value
to the MoD. The specific business functions are positioned at the level of cyber
operations capabilities, being defensive, offensive, or intelligence capabilities. The
specific business functions are specifically tailored to support one of the three
capabilities.
Page 6 of 78
Through this approach, the CTR delivers an added value for each capability and for
each activity within a capability making it an important supporting instrument for
conducting cyber operations.
The delivery of CTR business functions depend on technical and organizational
requirements. Technical requirements include the ability to simulate ITenvironments with all its characteristics but also to be scalable in assets and flexible
in deploying configurations. Security is crucial and security requirements should be
high. The organizational requirements consist of the ability to maintain and
configure the CTR, to support training or exercises and to conduct experiments.
The roadmap offers a timeline for the CTR for the delivery of business functions and
the implementation of technical and organizational requirements during the next
five years.
1. The business functions that enable personnel to act in the cyber domain should
be the main priority. Implementation is proposed through a level 3 maturity that
evolves to a level 5 maturity, with a total lead-time between 2 and 3 years.
2. The second priority is seen for the business function that researches external
solutions to enhance cyber operations. The realization is also proposed through
a phased approach by reaching a level 3 maturity and subsequently a level 5
maturity. The aim is to achieve level 5 between 4 and 5 years.
3. The ability to support a response to cyber attacks or to prepare for performing
cyber attacks or intelligence is graded as a third priority. It is suggested to
achieve a level 5 maturity between 4 and 5 years.
4. The business functions that enable assessments of the current means and
conduct research into future cyber solutions are deemed least important. It is
suggested to set the ambition for this business functions to a maturity level 3,
to be achieved between 4 and 5 years.
The following main recommendations are provided:
1.
Establish an approach for the realization of the CTR under supervision of the
Taskforce Cyber within the NL MoD. Liaise with the UK MoD in regard of their
experience with their CTR and with NATO regarding possible developments in
the CTR area. Collaborate with knowledge institutions and partners in order to
validate the proposed CTR roadmap. Combine these views along with the
proposed roadmap as a baseline for the CTR approach within the MoD.
2.
Develop the CTR business function under the supervision of the Taskforce Cyber
and in cooperation with the three cyber operations capabilities. Liaise with the
NATO Cooperative Cyber Defence Centre of Excellence in regard of their
experience in cyber defense exercises supported by a cyber lab.
3.
Realize the requirements needed to deliver business functions. Acquire, through
the forthcoming Defence Cyber Expertise Centre, resources for conducting
trainings and exercises and research and development. Determine whether the
maintenance and configuration of the CTR will be conducted within the MoD or if
it is suitable for sourcing. Define research questions in order to research
complex technical requirements in cooperation with knowledge institutes and
businesses.
Page 7 of 78
1
Introduction
This chapter presents a brief overview of the problem area in which the cyber
developments within the Ministry of Defence (MoD) are described and the cyber test
range is introduced. Based on the problem area, the research goal is presented
along with its limitations and the results to be achieved. Lastly, an overview of the
research methodology is given.
1.1
Problem area
In April 2011 the Netherlands Ministry of Defence (MoD) revealed (1) its largest cost
cutting operation in its history. Despite the massive cost reduction, the MoD
announced its ambition in the field of digital resilience and cyber operations. An
estimated €50 million will be invested till 2015 in order to deploy cyber capabilities
that will supplement the kinetic weapon arsenal in 2016.
The MoD cyber strategy (2) presents the course for the forthcoming years. The
strategy consists of six objectives:
1. realize an cohesive approach within the MoD towards cyber;
2. strengthen the digital resilience of the MoD;
3. development of military capabilities to execute cyber operations;
4. strengthen the intelligence position in the cyber domain;
5. strengthen the knowledge position and innovative capability in the cyber
domain;
6. intensify international and national cooperation.
The future governance framework structure is revealed in a MoD study (3) into
cyber operations (see Figure 1). The first entity, a Cyber Command, will govern
cyber operations. The second entity consists of cyber operations capabilities with the
abilities to defend networks and systems of the MoD, to attack other networks and
systems and to gather intelligence through other networks and systems. The last
entity is a Cyber Expertise Centre. Its role is to enhance the strategic, tactical, and
operational knowledge and skills concerning cyber operations within the MoD. The
Cyber Expertise Centre will therefore support Cyber Command and the three cyber
operations capabilities with services in the field of knowledge development and
knowledge dissemination and will, in that context, provide a cyber test range (CTR).
Figure 1: MoD governance framework
Page 8 of 78
The CTR will be one of the functionalities that supports cyber operations. A CTR can
be considered as a ‗digital shooting range‘, comparable to a shooting range in the
physical world, wherein military personnel can conduct military exercises, train their
drills and test their skills.
The implementation of the CTR starts in 2013 and the functionalities delivered will
be gradually extended. The need for a CTR is first mentioned in a MoD policy paper
(3) in which the need for a CTR is briefly quoted but the CTR function is not further
elaborated in terms of goals, objectives or specifications.
Current IT test environments within the MoD are mainly used in support of primary
IT processes. In general, these environments are used for availability and capacity
testing purposes as part of ITIL processes (4), which is used to deliver IT-services.
Therefore, the current environments are hardly comparable to a future CTR used in
support of cyber operations, which in turn is deployed as a weapon.
Several countries possess CTR‘s or are in the process of implementing cyber test
range. However, it is difficult to understand the goals and possibilities of CTR‘s and
the progress in the realization. The information through open sources is limited. In
addition, it is difficult to compare these CTR‘s because they have different areas of
interest. Moreover, different terms are used to indicate a CTR functionality, e.g.
cyber testbed, cyber lab or cyber range. In general, it can be stated that the
development of CTR has started a few years ago, mainly in the United States and
the concept is still being developed and implemented in different countries.
1.2
Research goal
Cyber operations is a relatively new topic for the MoD and will play an increasing
role in its future weapons portfolio, as indicated in paragraph 1.1. Some parts of
cyber operations are in the development or implementation stage, others still have
to be designed. The CTR is one of the functionalities that has to be defined,
designed, and subsequently implemented. Therefore, the following research goal is
formulated:
“The goal is to design a roadmap for the development of a cyber test range.”
Derived from the research goal are the following sub goals:
1.
Create an understanding of cyber operations
The goal is to deliver the definition for cyber operations, its capabilities and to
establish the activities that are conducted within cyber operations capabilities.
2.
Describe the current state of CTRs
The goal is to acquire knowledge about the use and developments in cyber test
ranges and to provide CTR business functions.
3.
Define the MoD CTR business functions
This sub questions researches the MoD‘s expectations regarding the CTR from a
defensive, offensive and intelligence capabilities viewpoint. The goal is to
determine the CTR business functions that support offensive, defensive and
Page 9 of 78
intelligence capabilities. Business functions are seen as a series of logically
related services performed together to obtain a defined set of results.
1.3
4.
Define the necessary requirements for delivering CTR business functions
The goal is to identify the technical and organizational requirements for
delivering the CTR business functions.
5.
Design a roadmap for the development of a cyber test range
The goal is to design a timeline for the implementation of business functions and
the technical and organizational requirements needed to deliver business
functions. The roadmap delivers the necessary input for the change
management for the implementation of a CTR.
Research scope
NATO uses the acronym DOTMPLFI1 for building capabilities to indicate which
measures need to be implemented before a capability can acquire the status of
being fully operational. As stated in the problem area, there is limited (re)use of
current test environments in relation to the CTR which implies that: doctrine,
organization, training, material, leadership, and education, personnel, facilities and
interoperability surrounding the CTR have to be (re)defined, designed, developed
and implemented.
This study focuses primarily on organizational (including personnel) and technical
requirements (as part of material). The reason is that these requirements are a first
necessity for a CTR to operate. Without a solid IT-environment, there is no CTR.
Without the proper organization to maintain the CTR or to work with one there is no
CTR or CTR function.
The other measures are: doctrine, training, leadership, facilities and education and
interoperability are in this case supportive to the organization and technical
requirements. These measures are needed for a CTR to run effectively and
efficiently but are given, due to the initial phase of research into CTR‘s, a second
priority and thus not included in this research.
1.4
Results to be achieved
This research is aimed at determining which business functions can be incorporated
into the CTR during the next five years and what technical and organizational
requirements are necessary to deliver these functions.
The deliverable of this thesis is to provide a roadmap that offers a timeline for the
delivery of business functions and an implementation timeline of technical and
organizational requirements.
Based on the recommendations given through this thesis, it is possible to start
additional research into the design, development, and realization of the technical
and organizational requirements in order to be able to deliver all CTR business
functions.
1
Doctrine, organization, training, material, leadership and education, personnel, facilities and interoperability
Page 10 of 78
1.5
Research methodology
This paragraph gives an overview of the research methodology based on the
research goal and its sub goals. The research is conducted through the following
methodology:
1. Understand the context of cyber operations and cyber test ranges.
2. Perform an analysis in order to determine the business functions and
requirements applicable to the MoD CTR.
3. Design a roadmap for the CTR based on the outcome of the analysis phase.
This methodology is presented in Figure 2.
Introduction
- Chapter 1 Introduction
Research
methodology
Research goal
Context
- Chapter 2 What are cyber
operations?
What’s a Cyber Test
Range?
Desk research
Analysis
- Chapter 3 What are the expectations towards a Cyber
Test Range?
What are necessary requirements for
meeting the expectations?
Interview/ survey
Design and Validation
- Chapter 4 What is the roadmap for development
of the CTR?
Validate outcome
Conclusions and recommendations
- Chapter 5 Provide conclusions and
recommendations
CTR roadmap
Figure 2: Research methodology
Page 11 of 78
Workshop
Expert panel
2
Cyber operations
2.1
Introduction
The first step in order to design a CTR roadmap is to understand the context in
which the CTR will operate. This chapter therefore researches the context of cyber
operations and cyber test ranges. The goal is to provide:
1. The definition of cyber operations and its capabilities, so as to get an
understanding of the future users of the cyber test range;
2. The activities that are conducted within the cyber operations capabilities, so as
to get an understanding of the business conducted within these capabilities;
3. The characteristics and goals of cyber test ranges, so as to get an understanding
of the kind of cyber test ranges and its purposes.
2.2
Context and terminology
Cyber consists of many elements that in addition are being viewed differently (5). In
addition, some elements are not officially named and are labelled for the purpose of
this thesis. Therefore, this paragraph positions these ‗cyber elements‘ in a context
as applied in this thesis. This context is presented in the following figure. The
elements within the context are described in the subsequent paragraphs.
Figure 3: Cyber elements in context
Page 12 of 78
2.2.1
Cyber operations
Cyber operations are deployed in order to achieve a certain political or military
effect. This effect could for example be the degradation of the opponents‘ air
defences, or the disruption of the ability for the opponent to command and control
its forces, or the ability to assure the possibility for the MoD to command and
control its forces. Therefore, cyber operations support the ability to attack, to
defend, or to gather intelligence. For completeness, the deployment of cyber
operations is one of many options a military commander has in order to achieve a
certain effect. A commander can deploy land, air, or sea operations in order to, for
example degrade the opponent‘s air defences.
2.2.2
Cyber operations capabilities
Cyber operations consist of three capabilities that enable the ability to attack,
defend, and gather intelligence. Capabilities, in a military context, are used to
indicate that one possesses the ability or power to achieve a certain political or
military objective or effect. The three capabilities are:
1. A defensive capability to protect its networks and systems;
2. An offensive capability to disrupt, deny, degrade or destroy other networks
and systems;
3. An intelligence capability to enable intelligence collection of other networks
and systems in order to defend its own networks and systems or to support
the offensive capability.
The cyber operations capabilities are researched in paragraph 2.3.2.
2.2.3
Cyber operations activities
Within each of the three cyber operations capabilities, multiple activities are
performed that enable the delivery of a cyber operations capability. These activities
are dedicated to one capability. The cyber operations activities are researched in
paragraph 2.3.3.
2.2.4
Cyber test range
Cyber test ranges are virtual environments that simulate live environments and are
used in support of cyber operations. The key benefit of the CTR is the ability to test
and practice without the risk of damaging live networks or disruption of live
operations. Cyber test ranges are researched in paragraph 2.4.
2.2.5
Cyber test range business functions
The cyber test range delivers business functions. These business functions are seen
as a series of logically related services performed together to obtain a defined set of
results, such as the ability to train, research and develop or assess. By providing
these business functions, the CTR provides support to the defensive, offensive and
intelligence capabilities. The business functions can be generic which means that
these are aimed at supporting cyber operations or the business functions can be
specific which means that these are aimed at supporting cyber operations
capabilities. Cyber test range business functions are researched in paragraph 3.3.
2.2.6
Cyber test range services
Cyber test range business functions consist of cyber test range services. These
services focus on specific elements within the business function. Cyber test range
business services are researched in paragraph 3.4.
Page 13 of 78
2.2.7
Cyber test range requirements
Implemented cyber test range requirements provide a platform on which cyber test
range services can be delivered. Requirements can consist of requirements in the
field of doctrine, organization, training, material, leadership and education,
personnel, facilities and interoperability. This study focuses primarily on
organizational (including personnel) and technical requirements (as part of
material). Cyber test range requirements are researched in paragraph 3.5.
2.3
Cyber operations in perspective
This paragraph delivers the definition of cyber operations and its capabilities.
Additionally, it establishes the activities that are conducted within cyber operations.
2.3.1
Defining Cyber
Cyber is an expression that is widely used to form new internet-related
manifestations of existing concepts. But what does it all mean?
Cyber- is a prefix derived from "cybernetic," which comes from the Greek adjective
kybernetes meaning skilled in steering or governing (Liddell and Scott, GreekEnglish Lexicon). This word was introduced by Norbert Wiener (6) for his work in
electronic communication and control science.
William Gibson first used Cyberspace, as a noun, in a science fiction novel
Neuromancer. Gibson was quoted by Wired.com (7) in describing ―cyberspace‖ as an
―evocative and essentially meaningless‖ buzzword. However, cyberspace has
evolved from a catchy word to a term describing something that influences the
global society.
Many attempts have been made to define this concept so elusive but nevertheless
so present. As Dr Dan Kuehl (8) explains mankind used to operate in four physical
domains: ―land, sea, air, and space in which each of these four domains feature
radically different and unique physical characteristics, and they are usable only
through the use of technology to exploit those characteristics.‖ Cyberspace was
added as a fifth domain (9) and contrary to the other domains cyberspace is a virtual
domain and can reside simultaneously in both physical and virtual domains.
Although many have defined cyberspace, for the purpose of this study only the
definitions within the military context are presented. The US Department of Defence
(DoD) had a definition of cyberspace dating to the early 2000s (10) which evolved
over time. The current definition describes cyberspace (11) as: ―a global domain
within the information environment consisting of the interdependent network of
information technology infrastructures, including the Internet, telecommunications
networks, computer systems, and embedded processors and controllers.‖
Threats in cyberspace are diverse. The latest study into the status of cyber security
within the Netherlands (12) delivers a detailed overview of the actors that impose
threats. The most significant threats are carried out by states and criminals.
Although different studies indicate a high rise in incidents (13) (14), not many
studies are available regarding the impact and damage caused by those incidents
(15).
However, the general consensus is that the impact of cyber incidents poses threats
to the society or continuity of organizations. In the Chatman House Report ‗On
Cyber Warfare‘ (16), it is stated that: ―Cyber warfare is arguably at the most serious
Page 14 of 78
end of the spectrum of security challenges posed by – and within – cyberspace. Just
like the tools of conventional warfare, cyber technology can be used to attack the
machinery of state, financial institutions, the national energy and transport
infrastructure and public morale.‖ Moreover, the study argues that: ―the most
distinctive feature of cyber warfare (and cyber security more generally) is the
rapidity with which threats can evolve. The pace of change can be so abrupt as to
render the action/reaction cycle of traditional strategy out of date before it has
begun.‖
2.3.2
Cyber operations capabilities
In comparison to the other four domains, it is also possible to conduct military
activities within Cyberspace. The US DoD has labelled its cyberspace operations and
defined (17) it as: ―the employment of cyberspace capabilities where the primary
purpose is to achieve military objectives or effects in or through cyberspace.‖
Cyberspace operations are conducted through offensive, defensive and intelligence
capabilities. Capabilities, in a military context, are used to indicate that one
possesses the ability or power to achieve a certain political or military effect.
Both the US DoD (11) and NATO (18) use the same terminology and similar
definitions for these capabilities, in which ‗computer network‘ is nowadays often
replaced by ‗cyber‘.
NATO uses the following definitions to describe the capabilities within Cyber
Operations:
Computer Network Operations (CNO) - Computer Network Operations
(consisting of Computer Network Attack, Exploitation, and Protection) seek to
gain access to computer networks to disrupt, deny, degrade or destroy their
capability, or alternatively to intercept and utilize their capability, whilst
protecting the capability of the joint task force. Success in this aspect is directly
proportional to the adversary‘s dependence on such systems.
Computer network defense (CND) - Actions to protect against disruption, denial,
degradation, or destruction of information resident in computers and computer
networks and the networks themselves.
Computer network attack (CNA) - Action taken to disrupt, deny, degrade, or
destroy information resident in a computer and/or computer network, or the
computer and/or computer network itself.
Computer network exploitation (CNE) - Action taken to make use of a computer
or computer network, as well as the information hosted therein, in order to gain
advantage.
Terminology
The use of ‗computer network‘ is becoming limited in its meaning, i.e. it excludes
information-, control-, and weapon systems (12). Therefore, to indicate defensive,
offensive and intelligence capabilities, this research uses the following terminology:
Cyber Operations instead of Computer network operations (CNO);
Cyber Defense instead of Computer network defense (CND);
Cyber Attack instead of Computer network attack (CNA);
Cyber Intelligence instead of Computer network exploitation (CNE).
Page 15 of 78
2.3.3
Cyber operations activities
The previous paragraph described the capabilities within cyber operations. This
paragraph researches the activities performed within the three capabilities.
Frameworks provide an in-depth look into these activities performed within the
different cyber operations capabilities and their interdependence and relationships.
The most exhaustive frameworks exist for cyber defence and recently more research
is conducted for cyber attack. The activities for conducting cyber intelligence are in
general the same as the activities used for cyber attack (19). There is one major
difference; the activities are performed for another purpose. Wherein cyber attack is
focused on the deception, disruption, denial, degradation, or destruction of a target
system, cyber intelligence is aimed at creating a specific picture of the target
environment in preparation for a cyber attack or as a basis to refine cyber defense
capabilities. Therefore, the activities within cyber operations are researched through
defensive and offensive cyber frameworks.
2.3.3.1
Cyber attack and intelligence activities
D. Dreijer (20) and Grant, Burke and Van Heerden (21) have performed extensive
research into the activities for performing offensive cyber operations. In addition, J.
Andress and S. Winterfeld have described a cyber attack process in their work (22).
All these models are fairly comparable. The area in which Dreijer and Grant differ
from Andress and Winterfeld is the use of supporting processes such as composing
target lists or evaluations. Therefore, their frameworks are more comprehensive.
For this research, the identification of the core activities of a cyber attack is
sufficient and therefore the model of Andress and Winterfeld is used. They identify 8
phases in conducting an attack. Their model follows the steps in which specific
information is gathered from the system, an attempt is made to access the system,
the planned effect is achieved and along the way the tracks are covered, as shown
in Figure 4.
The eight activities within this framework are:
1.
Recon is the ability to gain specific information through which access to a target
system is allowed i.e. through social engineering. There is a relationship with
reconnaissance within cyber intelligence. The authors argue that the difference
lies between a general reconnaissance through cyber intelligence and a specific
reconnaissance through cyber attack.
2.
Scan is the ability to gain detailed insight into potential vulnerabilities of the
target system provided by information delivered from the recon.
3.
Access is the ability to gain access to the target system using a variety of tools
and methods.
4.
Escalate is the ability to exploit the initial access by gaining additional or
different privileges.
5.
Exfiltrate is the ability to find valuable information and exfiltrate to a location
that is accessible or move it to own systems.
6.
Assault is the ability to create the effect of a deception, disruption, denial,
degradation, or destruction within the target system.
Page 16 of 78
7.
Sustain is the ability to ensure the possibility of future access to the target
systems by creating multiple points of entry.
8.
Obfuscate is the ability to cover not only the methods that might be used to
cover up or erase evidence of the intrusion, but also to point any potential
investigators to another source entirely. Obfuscate is conducted throughout the
entire cyber attack process.
Figure 4: Cyber attack and intelligence activities. Source: Andress and Winterfeld
2.3.3.2
Cyber defense activities
Cyber defence is by definition one‘s first priority and has been around the longest.
Back in 2003, Carnegie Mellon University published the second edition of the
Handbook for CSIRTS (23). The goal was to provide guidance on forming and
operating a computer security incident response team (CSIRT) and it is still valid
today. NATO also drafted a cyber defence framework. Due to the fact that the NATO
frameworks aims to improve cooperation between member states by creating an
unambiguous language, the cyber defence activities within this thesis will be based
on the NATO framework.
In December 2010, NATO released the second version of their Cyber Defence
Capability Framework (24). The goal is ―to provide NATO and its Nations a
foundation for cyber defence in order to increase multinational cooperation in the
field of development of cyber defence capabilities, coordination of cyber defence
activities and establishing interfaces to address federated cyber defence‖.
The framework is based on four chronological activities that roughly follow the
lifecycle of an incident and two that are in fact preconditions for dealing with
incidents. Figure 5 gives an overview of the NATO framework.
The six activities within the NATO framework are:
1.
Malicious activity detection is the ability to detect malicious activity by collecting
sensor information, assessing entities and the situation, and providing
visualization of the entities and activities for human analysts.
2.
Attack termination/prevention/mitigation is the ability to terminate, prevent, or
mitigate an attack.
3.
Dynamic risk, damage, and attack assessment is the ability to assess risks to
the operation from the dependency on the CIS services, by projecting the
Page 17 of 78
current situation into the future, assess the damage incurred from an attack,
and improve the understanding of threats by assessing ongoing attacks.
4.
Cyber attack recovery is the ability to recover from an attack by restoring the
security properties and the original state of the system and the information.
5.
Timely decision-making is the ability to decide on the actions to be implemented
in a timely manner.
6.
Cyber defence information management is the ability to collect and share
information in a way that enables rapid and trusted information exchange with
various different sources.
Figure 5: Overview of the cyber defence capability breakdown. Source: NATO/ NC3A
2.4
Developments in cyber test ranges
This paragraph concludes the conceptual base by determining the characteristics
and goals of cyber test ranges.
2.4.1
Introduction
Cyber Test Ranges are virtual environments that are, for instance, used for research
and development, evaluation and training purposes within the cyber domain. From a
military perspective, soldiers are trained to hone their skills, participate in military
exercises, and to get acquainted with weapons or information and communications
systems. The same principle applies to (military) personnel that are involved in
cyber operations. Buxbaum (25) argues: ―They require a digital environment in
which to train, evaluate, and develop offensive and defensive capabilities. They, too,
wish to simulate attacks to assess information assurance capabilities, and measure
incident response procedures.‖ As Buxbaum explains: ―'Cyber ranges' are the virtual
environments which have been created for cyber-warfare training and exercises.
These constructs provide tools for strengthening the security, stability, and
performance of vital government, military, and intelligence cyber infrastructures.‖ In
addition, Sabo (26) offers another viewpoint of the purpose of a range, which is to:
―create a secure, flexible, seamless environment enabling Combatant Commanders
Page 18 of 78
to visualize non-kinetic weapons effects, thus achieving the same confidence and
expertise in employing Information Operations capabilities as with kinetic weapons.‖
Also from a military perspective but from a defensive viewpoint, Powell et al. (27)
state: ―the pervasive interconnectivity of the Global Information Grid (GIG) (28)
makes cyber attacks an increasingly attractive prospect for first, second, and third
generation threats and adversaries‖. Therefore: ―the Information Assurance range
provides DoD stakeholders with an avenue to strengthen the GIG security posture
by supporting operational exercises, training network defenders, and testing and
evaluating new information capabilities.‖
From military perspective, cyber test ranges can therefore be summarized as an
environment that offers stakeholders the ability to - more effectively - defend and
attack (or gather intelligence regarding) infrastructures or military capabilities.
Critical infrastructure and military capabilities
Live IT-environments
Energy
Water
Cyber
Transport
Banking
Land
Space
Telecommunicatio
ns
Sea
...
Infrastructures and capabilities
depend on systems. Systems are
supported through IT-environments
Air
Ability to - more effectively - defend and
attack (or gather intelligence regarding)
infrastructures or military capabilities
Live IT-environments are
simulated in the CTR
Cyber test range
Cyber operations
Cyber
Defense
Cyber
Intelligence
Cyber
Attack
CTR supports cyber operations
Figure 6: Cyber test range in context
As a result, the requirements for cyber test ranges are demanding. CTR‘s should be
able to replicate networks and computer systems and emulate business operations
and generate associated traffic to mimic real world situations (25) (29) in order to
conduct realistic tests or exercises without threatening real world environments.
Therefore, it needs to emulate configurations that consist of the components used in
the real world. Additionally, the test ranges‘ configuration needs to be adaptable
within a reasonable amount of time to conduct experiments or perform exercises
(29) (30). In case of large-scale experiments or exercises, the test range should be
able to extend to multiple other test ranges (30) (31). While conducting these
experiments the range should ensure that the underlying conditions are valid in
order to prevent false results (31). During experiments and exercises, both control
and safety need to be retained due to the possible disruptive nature of these
experiments or exercises i.e. releasing malicious software (31).
The difficulty with cyber test ranges lies in the ability to replicate a realistic
environment. To generate the appropriate traffic, test ranges were made up of
physical equipment. This is problematic for two reasons as research points out (32):
Page 19 of 78
―The equipment can be very expensive to acquire, configure, and maintain.
Instrumentation and experimentation can be very challenging. It is difficult to
correlate traffic events that move across the computer information systems and,
as a result, difficult to roll up studies and generate system-level information.‖
2.4.2
Case studies
A number of cyber test ranges are operational or being implemented. These CTR‘s
present an overview of current (or future) characteristics and goals and therefore,
contribute to a better understanding of the kind of cyber test ranges and its
purposes.
2.4.2.1
The United States
The US is in the phase of implementing a National Cyber Test Range (NCR). This
range (33) will provide ―the architecture and software tools for a secure, selfcontained testing capability to rapidly emulate large-scale complex networks that
match the depth and diversity of real-world networks.‖ The implementation started
in 2008 with the publication of a public tender (34). The range will service (35) both
researchers and operational users:
Experimental Researchers will have:
The ability to measure the
progress of their experiment in
detail
The appropriate classified or
unclassified environment
Experiments against realistic
threats
Assurance of timeliness and
accuracy
The use of scientific methodology
to track and trace experiments and
results
Operational users will have:
Proper test and evaluation of military
and government net-centric systems
to ensure current and future defense
from cyber attacks.
Rapid assessment of the Nation‘s
current and future cyber research
programs
Cyber security experimentation
technologies for all ranges and
communities
Decreased time/cost for experiments
In addition to the NCR, the development of an information operations (IO) range
started in 2006 (26). The goal is to deliver an ―environment composed of processes
and structures which establish a realistic test, training, and exercise environment for
developing and operationalizing IO capabilities and their associated tactics,
techniques, and procedures.‖ The IO range supports ―Combatant Commanders to
visualize non-kinetic weapons effects, thus achieving the same confidence and
expertise in employing IO capabilities as with kinetic weapons.‖ The IO range
therefore represents ―actual combat targets, systems, and situations, allowing users
to conduct technical and performance assurance testing for IO capability system
certification.‖
The Information Assurance (IA) range is a third range within the US DoD (27). The
IA range ―provides an operational representation of the Global Information Grid
(GIG) (28) in order to ―strengthen the GIG security posture by supporting
operational exercises, training network defenders, and testing and evaluating new
information capabilities.‖ The goal of the IA range is to:
―Improve cyber security workforce operational performance,
Validate capabilities and services provided by CND tools and mechanisms,
Validate and improve CND tactics, techniques, and procedures,
Page 20 of 78
Validate acceptable level of service of Computer Network Defense Service
Providers (CNDSPs), and
Validate IA mitigation strategies for programs of record‖.
The NCR, IO, and IA range have different characteristics (36), as presented in the
following table.
Characteristics
NCR range
IO range
IA range
Classification
Multiple levels of
security
Multiple levels of security
Unclassified
Audience
Researchers
Combatant
Commanders, Services,
Interagency, Coalition
and test & evaluate
organizations
DoD IA and CND
practitioners
Environment
Tomorrow‘s
environment
Closed loop fully meshed
network and
management among
connected nodes
Today‘s Global
Information Grid
Provides a simulated,
emulated, and
replicated research
environment to
support experiments
Provides secure
connectivity, resource
allocation, event
coordination and access
to actual and modelled
tools and targets
Provides test, training,
and exercising
environment, modelled
after the GIG and joint
services architecture
Proof of concept for
Provides security,
Test and evaluate near-
emerging / future
capabilities
connectivity and network
management to event
participants capabilities
term tools; pilot shrinkwrapped products prior
to acquisition
Research of
tomorrow‘s cyber
issues and
capabilities focuses
at the national level
Access to IO tools and
targets to provide
exposure to and
validation of IO
capabilities in test,
training and exercises
Exercise and assess
personnel and tactics,
techniques and
procedures
Business
functions
Table 1: DOD Range initiatives. Source: Defense Information Systems Agency.
DETER2 (37) is a US research project aimed at ―developing the techniques and
methods for transforming experimental cyber security into a rigorous, scientifically
grounded research discipline.‖ Furthermore, DETER runs a cyber testbed (38) in
order ―to conduct rigorous, repeatable experiments with new security technologies,
and test their effectiveness in a realistic environment similar to the public and
private networks in which new cyber defenses are needed‖ and is ―used by
researchers to perform experiments on worm propagation, distributed denial of
service attacks, and routing and infrastructure attacks‖.
2 cyber-DEfense Technology Experimental Research
Page 21 of 78
2.4.2.2
NATO
NATO Cooperative Cyber Defence Centre of Excellence (NATO CCD COE) runs a
cyber lab, as stated by the Director of CCD COE in an email correspondence. The
cyber lab aimed at operational users in support of technical training courses (39)
and technical exercises (40).
2.4.2.3
The United Kingdom
The UK opened its cyber range in 2010 (41) that ―will be able to emulate large
infrastructures and global threats and evaluate how these networks, whether
military, civilian or commercial, respond to an attack in order to develop capabilities
that will make these networks more secure‖. Northrop Grumman delivers the test
range facilities (42). The cyber range has four common uses:
1. Training aimed at preventing falling victim to cyber attacks and response
training aimed at improving the handling of cyber attacks.
2. To getting and understanding of the robustness of the IT-architecture and to
understand the consequences of additions or changes to the IT-architecture.
3. To test and to benchmark IT-components.
4. Research and development.
This Federated Cyber Range (FCR), as it is called, is designed to allow
interoperability with other cyber ranges to enable large-scale experiments beyond
the scope of a single facility.
2.4.2.4
Suppliers
Northrop Grumman runs a commercial cyber test range facility (43) that is ―an
internet environment for emulating, attacking, and evaluating information
technology, network operations, and cyber security defense.‖ A Northrop Grumman
study (29) presents a picture for future use of cyber ranges for the military. This
study describes to possibilities to emulate Navy networks in a ―USS Millersville,‖
which is the location of the cyber test range in order for the Navy to ‗obtain
increased resiliency by first evaluating the current systems and personnel to
determine their effectiveness and combat readiness‘. BreakingPoint is another
supplier and delivers technology for replicating conditions across the internet in a
realistic closed environment (44).
Page 22 of 78
2.5
Summary
Cyber (space) operations are defined (17) as: ―the employment of cyberspace
capabilities where the primary purpose is to achieve military objectives or effects in
or through cyberspace.‖ Cyber operations are conducted through offensive,
defensive and intelligence capabilities in which:
Cyber defense is aimed at protecting own networks and systems.
Cyber attack is aimed at disrupting, denying, degrading, or destroying networks
and systems.
Cyber intelligence enables intelligence collection through networks and systems.
The activities performed within cyber attack and intelligence are similar and are
aimed at accessing the system and achieving a planned effect. These activities
consist of: recon, scan, access, escalate, exfiltrate, assault, sustain, and obfuscate.
The activities performed within cyber defense roughly follow the life cycle of an
incident and consist of: malicious activity detection, attack termination, -prevention,
-mitigation, dynamic risk damage and attack assessment, cyber attack recovery,
timely decision making, cyber defence information management.
Cyber Test Ranges are virtual environments that are used for research and
development, evaluation and training purposes within the cyber domain. The test
ranges are hereby aimed at reproducing real world situations without threatening
real world networks. From a military perspective, cyber test ranges can be seen as
an environment that offers stakeholders the ability to - more effectively - defend
and attack (or gather intelligence regarding) infrastructures or military capabilities.
The requirements for cyber test ranges are demanding. CTR‘s are expected to be
able to replicate networks and computer systems and emulate business operations
and generate realistic traffic in order to conduct tests or exercises without
threatening real world environments. In addition, they need to be flexible in order to
adapt their configuration or federated with other test ranges in order to support
large-scale experiments or exercises. When using the CTR both control and safety
need to be retained due to the possible disruptive nature of these experiments or
exercises, i.e. releasing malicious software.
Page 23 of 78
3
MoD Cyber Test Range
3.1
Introduction
The goal of the previous chapter was to understand the context in which the CTR
will operate in order to be able to determine the MoD CTR expectations. Based on
the context of the previous chapter, this chapter aims to determine the possibilities
of CTR‘s and their business functions. Subsequently, derive the CTR business
functions that are specifically tailored to the MoD. Lastly, identify the necessary
technical and organizational requirements to deliver MoD CTR business functions.
3.2
Methodology
The methodology for establishing the MoD business expectations and the necessary
technical and organizational requirements consists of two phases. The first phase is
to establish the CTR business functions. The second phase consists of determining
specific MoD business expectations towards the CTR and the necessary
requirements.
3.2.1
Process for determining CTR business functions
The process for determining the CTR business functions is displayed in Figure 7. This
process is based on three steps:
1. The case studies in paragraph 2.4.2 reveal the possibilities offered by CTR‘s and
provide an overview of the individual business functions, which are shown in
Appendix I.
2. Subsequently, similar business functions from the case studies were grouped
into similar categories. These similar categories presented the generic CTR
business functions. These generic CTR business functions provide an
understanding in the way, the CTR contributes to cyber operations. However,
these generic business functions are not specific enough to understand how the
CTR contributes to a specific cyber operations capability, whether it is cyber
defense, cyber offence or cyber intelligence.
3. Therefore, the generic CTR business functions were translated into specific
business functions that apply to defensive, offensive and intelligence cyber
capabilities.
Figure 7: Process for determining CTR business functions
Page 24 of 78
The results were subsequently validated through interviews that where held for
determining the MoD expectations.
3.2.2
Process for determining the CTR services
Interviews were held to research the expectations from the different cyber
capabilities‘ point of view and to determine the expected MoD cyber test range
business functions.
The interviews are based on a qualitative approach. This approach is chosen
because it offers the possibility to focus in-depth on the topic as it provides the
chance to ask follow up questions which is necessary due to the new and complex
nature of the CTR.
The information is provided by MoD representatives from the three cyber
capabilities. The number of interviewees is six. Every cyber capability is represented
by two interviewees in order to receive a more thorough view from one capability.
The total number is however small due to two reasons. The target group is still
rather small within the MoD, which creates a limited base of people with knowledge
of cyber. In addition, the fact that detailed interviews were held meant that the
research time was limited.
The objective of the interviews was to:
1. Validate the outcome from the desk research on the purposes of test ranges as
described in paragraph 3.3;
2. Determine to which extent the CTR should support the 8 activities of cyber
attack and cyber intelligence as described in paragraph 2.3.3.1;
3. Determine to which extent the CTR should support the 6 activities of cyber
defense as described in paragraph 2.3.3.2;
4. Identify the necessary technical and organizational consequences for meeting
the expectations.
The interview method by which the research is conducted is described in Appendix
II. The complete set of interviews is presented in Appendix III.
3.3
Cyber Test Range business functions
This paragraph delivers the CTR business functions that are applicable to cyber
defense, cyber attack and cyber intelligence. These business functions are used to
determine which possibilities are relevant to the MoD CTR. The results consist of
generic CTR business functions that apply to cyber operations independent of a
defensive, offensive or intelligence capability. In addition, the results consist of
specific CTR business functions that are aimed at one capability being a defensive,
offensive or intelligence capability. The result is presented in Table 2. Appendix I
provides a detailed overview of the individual CTR business functions.
The business functions of test ranges are segmented into two areas. Test ranges
can support operations by enabling personnel to act in the cyber domain, assess if
current means deployed by the organization are effective and additionally, used in
response to cyber attacks or in preparation for conducting cyber attacks or
intelligence.
Secondly, the test range can support research and development within the cyber
domain. This consists of applied research into future cyber solutions that are aimed
at developing solutions to enhance cyber operations through finding answers to
unsolved problems or development of new products, technologies or services.
Page 25 of 78
Moreover, research can be conducted to assess if external solutions add value to the
MoD through enhancing cyber operations.
Each of the five business functions can be translated into specific business functions
that apply specifically to the defensive, offensive and intelligence capability.
Generic CTR
business functions
supporting operations
Business functions aimed at
1.
Enable personnel to act in
the cyber domain
Specific CTR
business functions
Train personnel to withstand cyber attacks
Train personnel to execute cyber attacks
Train personnel to gather intelligence through cyber
2.
Assess current means
deployed by own
organization
Assess if current defensive means are sufficient
Assess if current offensive means are sufficient
Assess if current intelligence means are sufficient
Respond to cyber attacks
3.
Respond to or conduct cyber
attacks or intelligence
Conduct cyber attacks
Conduct cyber intelligence
research and development
Business functions aimed at
1.
Applied research into future
cyber solutions
Applied research into future cyber defensive capabilities
Applied research into future offensive cyber capabilities
Applied research into future cyber intelligence capabilities
2.
Research external solutions
to enhance cyber operations
Research external solutions to enhance cyber defense
Research external solutions to enhance cyber attack
Research external solutions to enhance cyber intelligence
Table 2: Overview of CTR business functions
3.4
Cyber Test Range services
The goal of this paragraph is to research the MoD expectations, seen from the three
cyber operations capabilities, towards the CTR in order to determine the CTR
business functions that support offensive, defensive and intelligence capabilities.
This paragraph answers the sub research goal: ―Define the MoD business
expectations towards a CTR.‖
The business functions are presented on two levels, first a high-level overview from
all capabilities followed by specific expectations from each of the three cyber
capabilities.
Page 26 of 78
3.4.1
High level overview
The high-level overview presents the consolidated expectations from all cyber
operations capabilities. The high-level overview consists of three elements:
1. The generic business functions that represent the main purposes of the CTR.
The business functions are based on the conceptual base as described in
paragraph 3.3.
2. The business functions are further specified into CTR services and describe
which services are to be offered in support of activities within the cyber
operations capabilities. The services are derived from the output of interviews.
3. The target audience describes which capability uses the services. These are also
derived from the output of the interviews.
Cyber Test Range
Business functions
Cyber Test Range
Services
Cyber Test Range
Target audience
Train general competences
Perform exercises
Enable personnel to act in the
cyber domain
Assess current means deployed
by own organization
Cyber defense
Cyber attack
Cyber intelligence
Train cyber defense execution
Cyber defense
Cyber intelligence
Train cyber attack execution
Cyber attack
Cyber intelligence
Train cyber intelligence execution
Cyber intelligence
Train toolhandling
Cyber defene
Determine effectiveness or optimize
current defensive means
Cyber defense
Cyber intelligence
offensive means
Cyber attack
Cyber intelligence
intelligence means
Respond to or conduct cyber
Analyze, assess and recover from
attacks
Cyber defense
Cyber intelligence
Assess most effective means to
attack a target system
Cyber attack
Cyber intelligence
Assess most effective means to
exploit a target system
Cyber intelligence
Research new means
cyber solutions
Research areas for improvement
Research aimed at assessing
current external solutions
applicability to own organization
Determine added value of external
solutions for the MoD
Figure 8: high-level overview business expectations
Page 27 of 78
Cyber defense
Cyber attack
Cyber intelligence
Cyber attack
Cyber intelligence
Support operations
The business functions that are aimed at supporting operations are deemed most
valuable. These business functions consist of enabling personnel to act in the cyber
domain, to assess current cyber means and to respond to or conduct cyber attacks.
1. Enable personnel to act in the cyber domain
This business function is aimed at readying personnel to perform activities within
cyber operations, whether it is defensive, offensive, or intelligence. The CTR can
support personnel in achieving or maintaining a general level of knowledge and
skills. The CTR is even employable to test the skills of potential personnel. In
addition, the CTR can be used to train personnel in the processes, procedures, and
methods surrounding cyber operations activities. These processes are aimed at
executing the specific activity or at the cooperation between different cyber
operations activities in order to work as one supply chain. Getting personnel
acquainted or improving their skills in tool handling is another opportunity for the
CTR. The last service consists of supporting cyber exercises. Exercises are an
important element in cyber operations as it combines every aspect of cyber
operations into a near real live activity. Every capability can be trained in their
respective specialty. Cyber defense can be trained in every aspect from detection to
cyber attack recovery. Personnel within the cyber attack and intelligence capabilities
can be trained in performing their missions. The three capabilities can also train
each other when taking part in an integral exercise with red teams (attackers) and
blue teams (defenders) (45).
Test skills &
competences
Maintain skills &
competences
Train processes &
Conduct
procedures
cyber exercises
‗Mission ready‘
‗Basic level‘
2. Assess current means deployed
The goal of assessing current means deployed is to answer the question, to what
extent are the means deployed by the organization sufficient and effective? Means
in this context are seen as instruments deployed by the organization, like hardware,
software, networks but also the standing operating procedures that are used to
execute an activity.
This business function can be used for every activity that is conducted within cyber
operations. For example from a cyber defense point of view, it can assess if the
intrusion detection systems deployed by the organization detect known and
unknown malicious activity. From a cyber attack or cyber intelligence point of view,
it can help to understand to what extent scanning activities are traceable and what
actions are risky in terms of getting compromised.
3. Respond to or conduct cyber attacks
The service ‗respond to or conduct operations‘ was added during the interviews. The
respondents indicated a need to use the CTR during defensive actions against cyber
attacks or in preparation for conducting cyber attacks or intelligence. When struck
by malware, the CTR offers the opportunity to research the malware to understand
its modus operandi, the impact on the IT-infrastructure and to develop solutions to
prevent (further) damage. From another perspective, the CTR offers the opportunity
to understand to what extent measures are traceable and what actions increase the
chance getting compromised prior to conducting cyber intelligence or cyber attack.
Page 28 of 78
Research and development
The business functions, which are aimed at research and development, are deemed
valuable but not as valuable as the business functions that support operations.
The business functions consist of applied research into future cyber solutions and
research aimed at assessing current external solutions applicability to own
organization.
1. Applied research into future cyber solutions
This business function is aimed at developing solutions to unsolved problems or the
development of products, technologies, or services. This business function applies to
all three capabilities within cyber operations. From a defensive point of view, the
CTR can be used to develop for example a new type of intrusion detection system to
detect advanced persistent threats (43). From an attack or intelligence point of
view, the CTR can be used to develop new means to scan a target system while
obfuscating its activities.
2. Research external solutions to enhance cyber operations
This business function is aimed at determining if solutions offered by external
organizations or that are publicly available are useful to the MoD. The CTR offers a
platform to field test these external solutions whereby the MoD is able to decide if a
solution works as designed or has side effects on the current IT-infrastructure and if
so to what extent. This business function is mainly aimed at the defensive side of
cyber operations. In general, it is considered that the external solutions for cyber
attack and cyber intelligence are limited. The respondents deemed it more plausible
that these solutions will be developed in house.
3.4.2
Detailed overview
The detailed overview presents the specific expectations from each of the three
cyber operations capabilities. The structure of these expectations is based on the
same structure as presented in the high-level overview in the previous paragraph.
3.4.2.1
Cyber defense
This paragraph describes the expectations towards the CTR from a defensive point
of view. The detailed overview of the cyber defense expectations consists of three
elements.
1. The specific business functions aimed at supporting cyber defense.
2. A further specification of the specific business functions into CTR services aimed
at supporting cyber defense.
3. A breakdown of the CTR services into CTR service components aimed at
supporting cyber defense.
The figure below gives the graphic breakdown of the CTR expectations for cyber
defense.
Page 29 of 78
Cyber Test Range
Cyber defense business functions
Cyber Test Range
Cyber defense services
Train cyber defense execution
Train personnel to withstand
cyber attacks
Develop necessary skills and knowledge
Maintain necessary skill and knowledge
Train ability to detect malicous activity
Train ability to stop an attack
Train attack assessments
Train risk assessments
Train damage assessments
Train recovery methods
Train forensic evidence collection
Train recovery procedures between organisation
Train decision making process
Train information dissemenation ability
Train toolhandling
Train detection toolhandling
Train attack termination and prevention toolhandling
Perform exercises
Train defensive skills and learn cyber attack methods
Train collaboration between defensive activities
Determine effectiveness of current
means
Assess if current defensive
means are effective
Optimize cyber defense process
Perform attack analysis
Determine effectiveness of detection means
Determine ability to stop an attack
Determine risks for the IT-environment
Determine ability to deliver assessments
Determine effectiveness of recovery means
Determine ability to test integrity
Determine ability to provide needed information
Identify information needs for decision making
Establish decision making process
Determine ability to deliver information
Determine methods of an attack
Determine impact of an attack
Determine mitigation means
Perform damage assessment
Determine mitigation means
Assess system integrity after an attack
Assess data integrity after an attack
Assess data loss after an attack
Determine impact after an attack
Perform attack recovery
Conduct system integrity recovery
Conduct information integrity recovery
Conduct service integrity recovery
Support decision making
Determine which decision should be made
Decide who should make decision
Determine the prioritization of decisions
Research new means
Research new means to detect malicious activity
Research new means to mitigate attack patterns
Research new means for information
dissementation
Establish areas for improving attack termination,
prevention and mitigation
cyber defensive capabilities
Cyber Test Range
Cyber defense service components
Determine usability of external solutions
Determine quality of external solutions
Determine side effects of external solutions
Figure 9: Cyber defense expectations of the CTR
Support operations
The business functions within this category are aimed at training personnel to
withstand cyber attacks, to assess if defensive means are sufficient and to respond
to cyber attacks.
1. Train personnel to withstand a cyber attack
This business function is aimed at readying personnel to respond to cyber attacks in
order to prevent or limit damage sustained from cyber attacks. The CTR offers the
Page 30 of 78
possibility to develop and maintain the needed basic knowledge and skills based on
a hands-on approach in a simulated but safe realistic environment.
Once personnel are trained in the general competences, the personnel can be
trained for executing cyber defense activities. This is where the CTR delivers the real
added value by offering a platform in which real live networks are simulated.
Personnel can be confronted with real live situations in which they will have to put
theory into practice. Hereby personnel will be trained in executing every cyber
defense activity from ‗Malicious activity detection‘ to ‗Cyber defence information
management‘ as is presented in the following table.
Cyber defense activities
CTR services aimed at training personnel
1
Malicious activity detection
Train personnel‘s ability to detect and analyze malicious activity
and train personnel in responding on alerts
Train personnel in correlating multiple detection feeds into a
coherent understanding of an attack
2
Attack termination/
prevention/ mitigation
Determine the time needed by personnel to terminate, prevent or
mitigate cyber attacks
Train personnel‘s ability to mitigate the effects of an attack
3
Dynamic risk, damage, and
attack assessment
Train personnel‘s ability to assess the effects of cyber attacks
Train personnel‘s ability to recognize risks
Train the ability to draft an assessment which includes the effects
of an attack and its countermeasure
Train personnel‘s ability to prioritize their response to multiple
attacks based on the assumed risk of each attack (triage)
4
Cyber attack recovery
Train personnel‘s ability to conduct a system, service or
information integrity recovery after an attack
Train personnel‘s ability to conduct forensic evidence collection
after an attack
Train recovery procedures between the CERT and IT-organization
5
Timely decision making
Train personnel‘s ability to deliver the right information at the right
time to the right person
Train strategic, tactical and operational decision making
Determine the consequences of a decision or the consequences
when no decision is made
6
Cyber defence information
management
Train personnel‘s ability to collect and disseminate information
Table 3: CTR cyber defence training possibilities
In addition, personnel can be trained with handling tools that are deployed for cyber
defense. As technology evolves, new tools are becoming available and are deployed
as part of the cyber defense arsenal. The CTR offers the opportunity to get
acquainted with these tools by deploying them in a safe environment so that
personnel can experiment freely.
Lastly, the real proof of the pudding in withstanding cyber attacks is achieved
through exercises. Theoretical trainings are important but putting the knowledge
into practice by conducting exercises will enable personnel to defend as they are
taught. The CTR should support these exercises by offering simulated real-live
networks in a safe environment in order to test defensive skills and to learn the
methods and techniques used in a cyber attack through red and blue team exercise.
Page 31 of 78
2. Assess current defensive measures
This business function determines if the defensive measures taken by the
organization are sufficient and effective in order to withstand cyber attacks.
Measures are to be seen in a broad context. These include IT-security tools but also
standing operating procedures (processes, procedures, and methods) that are used
to execute an activity.
By simulating real world environments, the CTR offers the prospect to assess the
defensive means to a degree that is almost similar to live situations. Deploy
malware into the CTR and see if it is detected and if it is detected, is the
organization able to terminate the attack? Answering these questions will determine
if the measures taken are sufficient and effective.
However, the CTR offers more possibilities, it also helps to understand the risks to
the IT-environment. When using again the example of deploying malware into the
CTR, the organization will get an understanding of the extent to which the ITenvironment is affected and thus will learn the risks to the IT but, moreover, the
business will get an understanding of their dependency on the IT-environment for
their businesses processes (like conducting military operations). In addition, the
resilience can be determined. The CTR can simulate the breakdown of parts or of
the whole IT-environment in order to determine if recovery measures are effective,
to understand what needs to be done to recover from a cyber attack and the time
needed to recover to a the last correct stage.
In order to withstand or limit the effects of cyber attacks, timely decision-making is
crucial as cyber attacks can inflict damage on a large scale in a small amount of
time. By simulating real-live situations in a near-real environment, the decision
making process can be tested in practice. For example, given a specific situation,
how does the decision-making process look like and who will need to make a
decision? Moreover, what are the consequences if a decision is not taken, what are
the effects for the IT-environment? The CTR is therefore a platform that can support
the optimization of current processes or procedures.
The same holds for information dissemination, which is an important factor during
all phases of defending against cyber attacks. By testing real-live situations, the
organization is able to establish which information is needed for decision-making
and if the right information can be delivered on the right time, in the right format,
from the right source to the right person.
3. Respond to cyber attacks
This business function is used when the IT-environment is affected by a cyber attack
in order to minimize its effects. Due to the ability of the CTR to simulate real-live
networks, the CTR can be used to test or research cyber attacks and to develop
solutions which are first tested and decided upon in the CTR before committing it
into the live environment. These tests support every step in the life-cycle of an
incident.
Analysis is done by researching the malicious software in the CTR in order to
determine its modus operandi, its components, the impact on the IT-infrastructure
and possible solutions which the local IT staff can implement to prevent (further)
damage. If the malicious software is having an impact on the IT-environment then a
damage assessment is performed by ‗replaying‘ the malicious software in the CTR to
Page 32 of 78
determine the system and data integrity and to assess the loss or compromised key
data. In addition to the gained insight to the IT-environment, insight is provided into
the possible damage to the constituency both when the attack succeeds, as well as
when it is mitigated.
After the assessment is done, scenarios for recovering the various levels of the
system are tested and improved in the CTR to make sure the real-live recovery goes
as smooth as possible. Based on understanding the malware encountered, its
target, the damage it causes and the options for mitigation or recovery, the CTR is
able to support the decision making process. This is done by determining which
decision needs to be taken (and the priority amongst these decisions) and to
determine on which level decisions should be made based on assumed risk and
potential damage.
The business functions within this category are aimed at researching, developing, or
testing new defensive means.
1. Applied research into future cyber defensive capabilities
This business function is aimed at developing solutions to enhance cyber defense
through finding answers to unsolved problems or developing new products,
technologies, or services.
The CTR should offer testbed capacity in which research and development can be
done. In relation to cyber defense there is a need to research new means to detect
malicious activity whether from an external or internal source. In addition, there is a
need to research new means to mitigate attack patterns or conduct research to
understand shortfalls in the ability to withstand a cyber attack and formulate
research goals aimed at improving this ability.
2.
Research aimed at assessing current external solutions applicability to enhance
defensive capabilities
This business function is aimed at determining if solutions intended to enhance
cyber defense are useful to the MoD. There are many solutions offered by (non)
profit organizations or freely available from which the MoD could benefit. Testing
these solutions can be done in the CTR. Deploying external products in a simulated
real-live environment offers the opportunity to determine if the solutions works as
designed within the MoD IT-landscape or if it has side effects and if so to what
extent and if the solution delivers the functionality which is sought by the MoD.
Hereby the MoD is able to determine if an external solution adds value to the cyber
defense toolkit.
3.4.2.2
Cyber attack
This paragraph describes the expectations towards the CTR from an offensive point
of view. The detailed overview of the cyber attack expectations consists of three
elements.
1. The specific business functions aimed at supporting cyber attacks.
at supporting cyber attacks.
supporting cyber attacks.
Page 33 of 78
The figure below gives the graphic breakdown of the CTR expectations from a cyber
attack perspective.
Cyber Test Range
Cyber attack business functions
Train personnel to execute cyber
attacks
Cyber Test Range
Cyber attack services
Train cyber attack execution
Train ability to recon a target system
Train ability to scan a target system
Train ability to gain access
Train ability to gain additional privileges
Train ability to exfiltrate information
Train ability to execute an attack
Train ability to sustain access
Train ability to obfuscate
Train offensive skills
Train collaboration between offensive activities
Perform exercises
Assess if current offensive means
are effective
Cyber Test Range
Cyber attack service components
means
Support cyber attack process
Determine effectiveness of recon activities
Determine effectiveness of scanning activities
Determine effectiveness of access activities
Determine effectiveness of escalate activities
Determine effectiveness of exfiltrate activities
Determine effectiveness of assault activities
Determine effectiveness of sustaining activities
Determine effectiveness of obfuscating activities
Assess most effective means to recon a target system
Assess most effective means to access a target system
Determine vulnerabilities in a target system
Assess most effective means to gain access into a target system
Assess most effective means to escalate in a target system
Assess most effective means to exfiltrate information
Assess most effective means to assault a target system
Assess most effective means to sustain access
Assess most effective means to cover the tracks throughout the operation
Research new means to recon a target system
Research new means to scan a target system
Research new means to access a target system
Research new means to gain additional privileges
Research new means to exfiltrate information
Research new means to assault a target system
Research new means to sustain access
Research new means to obfuscate during an attack
cyber attack capabilities
Research new means
Figure 10: Cyber attack expectations of the CTR
Support operations
perform cyber attacks, to assess if current offensive capabilities are effective and to
support cyber attack operations.
1. Train personnel to execute a cyber attack
This business function is aimed at readying personnel to perform cyber attacks in
order to achieve military objectives or effects. The CTR offers the necessary training
platform through simulated real world environments. This enables a hands-on
training approach.
Furthermore, personnel with basic skills receive advanced training in the execution
of cyber attacks. These trainings consist of technical aspects of a particular cyber
attack activity as well as the methods and procedures to conduct one. The CTR
supports these advanced trainings by offering a safe environment that is similar to
real live environments in which personnel can practice or simulate real events.
These specific training opportunities are presented in the following table.
Page 34 of 78
Cyber attack activities
CTR services aimed at training personnel
1
Recon
Train personnel‘s ability to explore a target system using different
techniques without being discovered
Train personnel in using strategic or tactical reconnaissance information
supplied by cyber intelligence in order to perform operational
reconnaissance
Train procedural aspects of performing reconnaissance
2
Scan
Train personnel‘s ability into obtaining a more detailed overview of the
target systems hardware and software components
Train personnel‘s ability to detect known and unknown vulnerabilities in
a target system
Train procedural aspects for scanning systems
3
Access
Train personnel‘s ability in gaining access to a target system by
exploiting vulnerabilities
Train procedural aspects for accessing target systems
4
Escalate
Train or asses the personnel‘s ability to gain additional or different
privileges within a target system
Train procedural aspects for escalating privileges within a target system
5
Exfiltrate
Train personnel‘s ability to find valuable information and exfiltrate it to a
location that is accessible or move it to own systems including
procedural aspects
6
Assault
Train personnel‘s ability to create the effect of a deception, disruption,
denial, degradation or destruction of the target system
Train procedural aspects for assaulting systems
7
Sustain
Train personnel‘s ability to ensure the possibility of future access to the
8
Obfuscate
target systems by creating multiple points of entry
Train personnel‘s ability to cover not only the methods that might be
used to cover up or erase evidence of the intrusion, but also to point
any potential investigators to another source entirely
Table 4: CTR cyber attack training possibilities
Just as exercises are an important instrument for cyber defence, performing cyber
attack exercises are essential as well. The CTR should support these exercises by
offering simulated real-live networks in order to test offensive skills and to learn the
methods and techniques used by cyber defence through red and blue team
exercises in which the red team has to compromise or degrade the performance of a
system that is protected by a blue team (40).
2. Assess current means deployed
This business function determines if the offensive measures taken are sufficient and
effective in order to conduct cyber attacks. Measures include technical means but
also standing operating procedures (processes, procedures, and methods) to
execute an activity.
Taking advantage of simulating real world environments, the CTR offers the
opportunity to assess means used to conduct cyber attack activities. These means
can be tested in order to determine their effects, i.e. do they do what is expected of
them and do they achieve the desired effects. That is an important difference. The
first focuses on methodology. For example if malware is designed to alter ITcomponents, the test range can provide the ability to verify if the right ITcomponents are altered. Malware will be deployed for specific reasons and that is to
achieve an effect. The desired effect in deploying malware could for example be
Page 35 of 78
through altering IT-components to deny, destroy or disrupt the enemies command
and control capability. So, the test range can be used to determine if the desired
effect is achieved or that other (undesirable) effects are achieved as well, in military
terms speaking of a collateral damage estimate.
3. Support cyber attacks
The CTR can also be employed in support of cyber attacks. The main feature is to
determine the most effective means to achieve an intended effect. Using the target
systems configuration in the CTR, it offers an environment to test means before
they are actually deployed.
Testing also contributes to the decision making process regarding the deployment of
means. For example, when releasing specific malware you will want it to hit the
target and not a hospital. In case minimizing collateral damage is not possible then
it will support the decision making process. Knowing what the collateral damage
may be, it will offer the commander the opportunity to decide if the collateral
damage is acceptable in relation to the effect to be achieved when striking a target.
testing of new offensive measures.
1. Applied research into future cyber offensive capabilities
This business function is aimed at developing solutions to enhance cyber offensive
through finding answers to unsolved problems or developing new products,
technologies, or services. This business function is of lesser importance compared to
the business functions that support operations. The research and development are
furthermore deemed more applicable to the cyber intelligence process than to the
cyber attack process with the exception of assaulting target system.
Generally speaking, the CTR could offer testbed capacity to build new means i.e. to
recon or scan target systems or to improve the current means i.e. to create the
effect of deception, disruption, denial, degradation or destruction within a target
system.
2. Research external solutions to enhance offensive capabilities
This business function is aimed at determining if external solutions are useful to be
used in cyber attacks. This business function is applicable to a limited number of
activities that support cyber attacks. This is because external parties who develop
offensive capabilities are limited. If these capabilities are developed by non-profit
organizations, it is not plausible that these capabilities are shared. Furthermore,
there are not many profit organizations that develop these capabilities. Then there
are security reasons that also limit the cooperation with external organizations.
These offensive capabilities may have a one-time use, once released these
capabilities are known so that limits the possibility of reuse. Therefore, to minimize
the chance of information regarding capabilities being leaked, strict security
regulations should be enforced, limiting the number of external organizations.
However, this function can be used for recon or scanning target system by testing
these solutions in the CTR.
Page 36 of 78
3.4.2.3
Cyber intelligence
This paragraph describes the expectations towards the CTR from an intelligence
point of view. The detailed overview of the cyber intelligence expectations consists
of three elements.
1. The specific business functions aimed at supporting cyber attacks.
at supporting cyber attacks.
supporting cyber attacks.
The figure below gives the graphic breakdown of the CTR expectations from a cyber
intelligence perspective.
Cyber Test Range
Cyber intelligence business functions
Train personnel to gather
intelligence through cyber
Assess if current intelligence
means are effective
Cyber Test Range
Cyber intelligence services
Cyber Test Range
Cyber intelligence service components
Train cyber intelligence execution
Train ability to recon a target system
Train ability to scan a target system
Train ability to gain access
Train ability to gain additional privileges
Train ability to exfiltrate information
Train ability to sustain access
Train ability to obfuscate
Perform exercises
Train intelligence skills
Train collaboration between intelligence activities
means
Determine effectiveness of recon activities
Determine effectiveness of scanning activities
Determine effectiveness of access activities
Determine effectiveness of escalate activities
Determine effectiveness of exfiltrate activities
Determine effectiveness of sustaining activities
Determine effectiveness of obfuscating activities
Support cyber intelligence process
Determine vulnerabilities in a target system
Assess most effective means to gain access into a target system
Assess most effective means to escalate in a target system
Assess most effective means to exfiltrate information
Assess most effective means to sustain access
Assess most effective means to cover the tracks throughout the operation
Research new means to scan a target system
Research new means to access a target system
Research new means to gain additional privileges
Research new means to exfiltrate information
Research new means to sustain access
Research new means to obfuscate during an attack
cyber intelligence capabilities
Research new means
Figure 11: Cyber intelligence expectations of the CTR
Support operations
conduct cyber intelligence, to assess if current intelligence capabilities are effective
and to support cyber intelligence operations.
The possibilities for cyber intelligence share many similarities with cyber attack. The
differences are however that these activities are performed with a different purpose
and that emphasis is put on activities such as recon, scan, exfiltrate, rather than
assault. In general, the same activities can be trained, the same means can be
assessed, and the same role can be played during an operation. Therefore, the
Page 37 of 78
possibilities for cyber intelligence are not further elaborated but referred to
paragraph 3.4.2.2.
testing of new intelligence measures.
In terms of research and development possibilities for cyber intelligence, these are
similar to cyber attack. Therefore, the possibilities for cyber intelligence are not
further elaborated but referred to paragraph 3.4.2.2.
3.5
Cyber Test Range requirements
3.5.1
Introduction
This paragraph researches the requirements needed for delivering business
functions. Interviews were held to research these requirements as part of the
interview to research the CTR services, see paragraph 3.2.2.
3.5.2
Technical requirements
a) First and foremost, the CTR will need to simulate live networks. Simulation is a
key requirement in order to provide realistic but safe environments for
delivering CTR business functions. The CTR should simulate own systems within
the MoD and target systems. These simulated environments should contain the
full spectrum of assets used in a live system, whether physical or virtual. Assets
can range from network components, hardware, and software. In addition, the
CTR should also generate the network traffic and human interaction to emulate
business operations. Furthermore, the test range should be able to simulate all
vulnerabilities that are present in an IT-environment. As a hypothetical
example, there can be vulnerabilities that are only present in the physical
components in an IT-environment. However, to simulate these IT-environments,
it is plausible that they will be virtualized, due to the fact that it is inefficient to
use large numbers of physical components. However, this can lead to a
drawback because unwittingly the vulnerabilities that are only present in
physical components are removed through the use of virtualized components.
Thus, from a cyber operations perspective the opportunity to detect or exploit
an unknown vulnerability is missed.
b) The CTR infrastructure should be scalable in terms of different assets and
number of assets. This requirement is necessary to be able to adapt to
developments. Real live environments are subject to change either because of
new technologies or new business developments which leads to a decrease or
increase in users. Therefore, the CTR should be able to adapt to new
developments by adding or removing assets.
c)
Changing the configuration to mimic a real live environment should be possible
at the push of a button and the maximum time required for a complex
configuration should be no more than 1 day. This important feature enables
flexible use of the CTR by different target groups to support different scenarios.
When a live network is attacked, it is important that the CTR is quickly prepared
so preventive or recovery actions can be simulated in the test range before
Page 38 of 78
committing these actions into real live environments. Therefore, it is advisable
to have preconfigured configurations of live environments so these
configurations can be loaded when necessary or applicable. From an offensive or
intelligence perspective, the load times differ and depend on the mission. On an
operational level, the timeframe should be short. In some situations, there is a
short window of opportunity to perform the intelligence activities or to execute
offensive activities, for example to assault a network. The time to market on a
strategic level can be long. The strategic level is focused on planning offensive
or intelligence activities prior to actually performing the activity, for example
accessing a system.
d) Target groups of the CTR can be scattered across large geographical locations.
Therefore, the CTR should be accessible from different locations reducing
overhead like travel time and stimulating the use of the CTR for different
situations. In addition, the cyber test range should support the ‗train as you
fight‘ principle. This implies that if an operation is executed in the field, the
cyber test range should extend its reach so it is also accessible from the area
where the operation is conducted.
e) In terms of federating with other cyber test ranges, to enable large-scale
experiments or exercises, the CTR should be able to federate with other CTR‘s in
order to support for example, red and blue teaming exercises. However, this
feature is viewed with some suspicion due to security concerns. Therefore, this
functionality should be able to be physically turned off at all times. The
realization of this requirement is also deemed less important or in some
situations even unwanted because of the security risks.
f)
Security is crucial and security requirements should be high. The systems
simulated can be own systems but also target systems and can hold sensitive
information. When compromised a blueprint of the system could be derived.
This is not desirable. In addition, cyber weapon may have a one-time use
principle, which means that when deployed it is out there to be analysed,
neutralized, or transformed for reuse. Moreover, through the CTR, vulnerabilities
within the MoD systems can be discovered or knowledge of the modus operandi
of the MoD can be gained. Therefore, measures need to be taken to prevent the
leakage of sensitive information. This implies that the security level of the test
range should have the same security level as the live networks. So if a classified
network is deployed in the test range, the test range should also be classified.
Therefore, personnel involved in the CTR must have a top-level screening.
Additionally to safeguard missions, the CTR must be able to allow compartments
based on different classification levels in which different target systems or MoD
systems are simulated.
g) The CTR will be used for conducting risky cyber experiments for example
deploying malware into the test range. As a result, the CTR should have no
external interfaces with live environments because of the risks of malware
jumping into a live environment. In addition, the configuration in the CTR should
be completely swiped in order to ensure that new configurations are clean of
any injected malware or other risks.
Page 39 of 78
3.5.3
Organizational requirements
h) In order to run a CTR, IT-operations should be appointed for the maintenance
and configuration of the CTR. Activities consist of designing the technical
infrastructure, installing, or changing IT-assets, maintaining the current
infrastructure with focusing on reliability and availability. When preparing the
CTR for use, IT-operations should be able to load the CTR with the configuration
of the simulated live environment. This may involve loading a preconfigured
configuration or in case a new configuration is to be loaded, IT-operations will
have to design an image that can be used to load a simulated live environment.
i)
Regarding the delivery of CTR services to train personnel, instructors should be
appointed for accompanying group exercises or individual training scenarios.
Activities consist of formulating training goals and development of CTR training
scenarios derived from the formulated training goals. For conducting exercises
and training scenarios, the instructors will need to accompany these exercises in
order to instruct the participant during their exercise or training scenario.
Instructors should also be appointed for observing trainings in order to evaluate
the results of personnel and to propose improvement possibilities for individuals
or groups. In addition, an analyst should be appointed for evaluating the results
from exercises and to propose improvements for current technical capabilities,
procedures, and policies used in cyber operations in order to enhance standing
operating procedures or technical means.
j)
Researchers should be appointed in order to conduct experiments. The activities
consist of formulating the research goals and developing experiments linked to
these goals. During these experiments, the researchers should analyze the
outcome in order to evaluate the results of measures or means and to make
recommendations for improving cyber operations. Researchers should be
focused on long-term research goals for the realization of strategic goals (for
example, enhancements in the development of cyber weapons or new detection
possibilities) or short-term research goals for the realization of operational goals
(for example, mission related goals).
k) The end-user should be able and available to hone their cyber skills and
knowledge. The users are primarily representatives from the cyber operations
capability.
Page 40 of 78
3.6
Summary
The MoD business expectations towards the CTR can be categorized into CTR
business functions. The CTR business functions consist of various levels in depth.
The first level positions CTR business functions in relation to cyber operations. In
other words, these functions support the execution of cyber operations. The second
level consists of specific business functions that support one of the capabilities
within cyber operations, being defensive, offensive, or intelligence capabilities. The
cohesion of the capabilities and the CTR business functions is shown in Figure 12.
Figure 12: Cohesion cyber operations capabilities and business functions
The generic business functions consist of business functions that support daily
operations and business functions that enable research and development. To
support operations the CTR can deliver business functions that enable personnel to
act in the cyber domain, to assess if current capabilities are effective in response to
a cyber attack or to conduct a cyber attack or intelligence. To enable research and
development, the CTR can deliver business functions that offer a platform to
conduct applied research into future cyber solutions or to research if external
solutions add value to the MoD.
The specific business functions are specifically tailored to support one of the three
capabilities. For example, a generic business function such as: ―Enable personnel to
act in the cyber domain― is, in the case of cyber defense, specified into: ―Train
personnel to withstand cyber attacks.‖ In addition, these specific business functions
are further specified into services and into CTR service components. The advantage
of this approach is that each activity within one of the three capabilities is supported
by the CTR. This means that CTR can deliver an added value for each capability and
for each activity within a capability making it an important supporting instrument for
conducting cyber operations.
The delivery of CTR business functions depends on technical and organization
requirements. Technical requirements include the ability to simulate ITenvironments with all its characteristics but also to be scalable in assets and flexible
in deploying configurations. Security is crucial and security requirements should
therefore be high. The organization requirements are focused on IT-staff to maintain
and configure the CTR, as well as staff to accompany trainings or experiments.
Page 41 of 78
4
Roadmap for the Cyber Test Range
4.1
Introduction
The goal of the previous chapter was to determine the MoD business functions that
are expected from the CTR and to identify necessary technical and organizational
requirements in order to deliver these functions. This chapter aims to determine
which business functions can be incorporated during the next five years into the CTR
and what technical and organizational requirements need to be implemented in time
to deliver these functions.
The methodology for establishing a roadmap for the CTR consists of two steps. The
first step is (a) to determine the priority for CTR business functions, as viewed from
the perspective of cyber operations capabilities. It is plausible that not all business
functions can be realized at the same time. Moreover, it is assumed there is no need
from the cyber operations capabilities to make use of all the business functions at
the same time. Therefore, additional information is required to determine which
business function has to be realized first. The second step is (b) to define a maturity
model in order to obtain a more detailed view of the business functions and the
requirements necessary to deliver these business functions. The business functions,
as described in paragraph 3.4, present the possibilities for an end-state of a certain
function. It is assumed that the realization of a business function aimed at an endstate is too complex. In addition, it is also assumed that not all business functions
need to obtain an end-state. Therefore, a maturity model is proposed that offers the
possibility for incremental development of a business function.
4.2
Priority Cyber Test Range business functions
This paragraph proposes the priorities, as seen by cyber operations capabilities, for
the realization of CTR business functions
4.2.1
Methodology
The methodology for establishing the priority level of CTR business functions is
based on two variables. The first variable is the need to use the business functions
(see par 3.4) quickly (which is indicated through
‗urgency‘). The second variable consists of the
requirements needed to deliver business functions
and
the
complexity
of
realizing
these
requirements
(which
is
indicated
through
‗complexity‘). The combination of these variables
presents the priorities for the realization of
business functions. The ideal situation is when a
business function has a high urgency and a low
complexity. These business functions should be
implemented first. Business functions with a low
urgency and a high complexity should be to
implemented last. A matrix is used to categorize
the possibilities based on the combination of
urgency and complexity.
Figure 13: Matrix for determining priorities
Page 42 of 78
The above-mentioned method to prioritize business functions is done per cyber
operations capability i.e. defensive, offensive and intelligence capabilities. All three
capabilities have different dynamics i.e. what is important to one capability can be
of lesser importance to the other capability. Therefore, prioritizing business
functions is done on the level of each of the three capabilities.
Interviews were held to determine the urgency of business functions and to gain
knowledge regarding the complexity of implementing the requirements. The results
are shown in Appendix IV and Appendix V. The interviews are based on a
quantitative approach. This approach is chosen because it offers the possibility to
calculate urgency and complexity. The information is provided by MoD
representatives from the three cyber capabilities. The number of interviewees is six.
Every cyber capability is represented by two interviewees in order to receive a more
thorough view from each capability. The total number is however small due to two
reasons. The target group is still rather small within the MoD, which creates a
limited base of people with knowledge of cyber.
Results
The priority for the CTR is to be able to support personnel in order to prepare them
for cyber operations, whether these are defensive, offensive or intelligence
activities, as shown in Figure 14. This priority is based on the highest score in
urgency, based on a scale from 5 (realization as soon as possible, within 1 year) to
1 (no direct need for realization) and the number of services within this business
function. The complexity of the requirements necessary to deliver business functions
varies little from one requirement to the next. Therefore, urgency and the amount
of services are the main distinctive variables to define a priority. The second and
third priorities are seen in the ability to research external solutions that may
enhance cyber operations and the ability to use the CTR in support of responding to
cyber attacks or to conduct cyber attacks or intelligence. The descriptions for both
the complexity and the urgency scale are presented in table 5.
Priority cyber operations CTR business functions
5
Low
Possible ‘low
hanging fruit’
otherwise avoid
4
Complexity
4.2.2
cyber solutions
Most ideal
deployed by own
organization
3
the cyber domain
2
Adds value
but risky
Avoid
to enhance cyber
operations
1
Respond to or conduct
cyber attacks or intelligence
High
0
0
Low
1
2
3
4
Urgency
Figure 14: Priority CTR business functions
Page 43 of 78
High
5
The business function ―Applied research into future cyber solutions‖ is not visible in
the above-mentioned graph due to similar results as the other business function,
which positions it behind the functions: ―Assess current means deployed by own
organization‖ and ―Respond to or conduct cyber attacks or cyber intelligence.‖
Appendix VI offers a more in-depth version of the graph that shows all business
functions.
The detailed overviews for the cyber attack, defense and intelligence capabilities do
not show huge differences. Therefore, these overviews are not separately described
but rather referred to in Appendix VI.
Legend
Complexity
Urgency
Scale
Description
Scale
Description
1
Very complex. Considerable financial efforts needed.
Long time to market. Chances of success uncertain
Complex. Financial efforts needed (but less compared
to 1). Long time to market (but less compared to 1).
Chances of success limited
Normal complexity. Realization through regular
projects. Normal chances of success.
Straightforward. Experience in this field and has been
performed frequently. High probability of success.
Effortless and seen as low hanging fruit. Very high
probability of success.
1
Very long-term need, realization
after 5 years
Long-term need, realization
between 3 and 5 years
2
3
4
5
Table 5: Description of complexity and urgency scales
Page 44 of 78
2
3
4
5
Medium-term need, realization
between 2 and 3 years.
Short-term need, realization
Very short-term need, realization
within 1 year.
4.3
Cyber Test Range maturity model
This paragraph proposes a Cyber Test Range maturity model that consists of five
levels. This is the second step in building the CTR roadmap. A maturity model is
suggested because CTR business functions can be offered through a variety of
service levels ranging from very basic to very advanced. A maturity model creates
the opportunity to pinpoint the desired service level and plan the realization of the
desired service level in time.
The maturity model describes three elements:
1. A general description for each maturity level;
2. For every CTR business function, a description of the functionalities for each
maturity level;
3. For every CTR business function, the requirements to deliver the functionalities
at each maturity level.
4.3.1
Methodology
The methodology for defining a maturity model is based on the following:
1. The first step is to link the CTR requirements (paragraph 3.5) to the individual
CTR services (paragraph 3.4.2). This coupling offers insight into the necessary
requirements that have to be realized in order to deliver a specific CTR service.
2. The second step is to abstract the output from the first step to the level of CTR
business function, as the maturity model is based on business functions. This
creates an overview that is less detailed and therefore is easier to interpret. The
results are shown in Appendix VII.
3. The final step is to formulate the possible functionalities provided by business
functions based on different maturity levels. In addition, the CTR requirements
needed to deliver the functionalities on a particular maturity level were
formulated. This final step provides the CTR maturity model with different levels
for business functions and the necessary requirements for that level.
4.3.2
Description of CTR maturity levels
Level 1 Initial
CTR business functions are small-scale and focus on individuals or a single aspect
within a cyber operations activity. The level of service is basic with an ad-hoc
approach without business and IT support. The success of the business functions
depends on the competences and heroics of people in the organization.
Level 2 Managed
CTR business functions are still small-scale but in addition to level 1, the service
delivery is more controlled through activity planning, monitoring, and evaluating.
The business functions focus on individuals or single aspects within a cyber
operations activity.
Level 3 Defined
Technical analyses are more in-depth due to the possibility to simulate basic
configurations of live environments in the CTR. The CTR environment is managed
but new configurations are deployed in the CTR through projects. The business
functions are delivered for a single cyber operations activity.
Page 45 of 78
Level 4 Integrated
The CTR is flexible in terms of deploying simulated live networks and contains a
wide-range of assets. Therefore, business functions provide thorough technical and
business opportunities for testing, training or research and development. Business
functions are delivered on the level of cyber operations capabilities. The CTR
environment is managed and new configurations are loaded through standardized
approaches.
Level 5 Near real-world
CTR business functions are aimed at supporting large scale and complex exercises,
tests, research and development projects. The CTR is able to simulate exact copies
of live networks including all its characteristics and is moreover capable of
federating with other CTR‘s in order to provide heterogeneous environments. The
CTR‘s configuration is fully automated and new configurations are deployed ‗at the
push of a button‘. Multiple simultaneous tests are available at different security
levels.
4.3.3
Enable personnel to act in the cyber domain
This paragraph describes the maturity levels for the business function: ‗Enable
personnel to act in the cyber domain‘.
Functionalities
Limited practical training
possibilities for developing basic
skills and knowledge needed to
perform an activity.
Provide small-scale and
straightforward exercises to a
single target audience.
Basic training for tool handling.
Requirements
Level 2
Additional to level 1.
Possibilities for developing and
expanding skills and knowledge.
Opportunities for training
procedures regarding a cyber
operation activity.
Instructors available for formulating
training goals, developing training
scenarios, accompanying trainings
and evaluating results to identify
lessons learned.
Limited IT-support for the CTR by IToperations.
Level 3
Provide trainings to practice
collaboration between different
activities within a cyber operations'
capability.
Opportunities for training
techniques regarding a cyber
operation activity.
Simulate live networks with a basic
set of assets.
IT-operations maintains the CTR.
Level 4
Red and blue team exercises are
available to test defensive,
offensive and intelligence skills and
to learn the opponents‘ methods
and techniques.
Decision making processes are
taken into account
Simulate live networks, whether own
or target networks that contain an
extensive set of assets of the live
network.
These are scalable in terms of
diversity and numbers.
The CTR holds preconfigured
configurations of live networks.
The CTR simulates human interaction
and network traffic both in terms of
Level 1
Basic IT-infrastructure available
consisting of elementary assets.
Page 46 of 78
type and amount.
In addition, the CTR is accessible
from different locations.
IT-operations maintain and configure
the CTR.
Local staff facilities available for
support.
Level 5
4.3.4
Large-scale exercise possibilities
with multi-level scenarios and
diverse target audience.
The CTR is able to deliver realistic
training scenarios for cyber
operation execution and tool
handling in simulated real live
environments that include the ITassets and characteristics.
Full integrated decision making
process
The simulated networks contain a full
simulation of the assets of the live
network.
Different configurations of live
networks can be quickly deployed
into the CTR.
In addition, the CTR can federate
with different CTR's and is accessible
Integrated remote and local staff
facilities available for support.
Research external solutions to enhance cyber operations
This paragraph describes the maturity levels for the business function: ‗Research
external solutions to enhance cyber operations‘.
Level 1
Functionalities
Requirements
Basic technical testing of external
solutions to establish if it works as
designed.
Level 2
External solutions are tested in
terms of quality and usability to
establish applicability for the own
organization.
Researchers are available for
formulating research goals,
developing experiments and evaluate
results.
Level 3
External solutions are tested to
determine if it works as designed
when deployed in a live network.
set of assets.
The simulated network has the same
security level as required for the live
network.
Level 4
External solutions are tested to
determine if it adds value for a
cyber operations activity.
In order to conduct experiments, the
CTR simulates live networks, whether
own or target networks.
These contain an extensive set of
assets of the live network that are
scalable in diversity and numbers.
type and amount.
configurations of live networks and is
accessible from different locations.
IT-operations maintains and
configures the CTR.
Level 5
External solutions are deployed in
Page 47 of 78
simulated live networks to conduct
complex real-live experiments to
determine added value and
necessary tactics, techniques and
procedures to fit within a cyber
operations activity.
4.3.5
network.
networks can be quickly deployed
into the CTR.
The CTR federates with different
CTR's.
Respond to or conduct cyber attacks or intelligence
This paragraph describes the maturity levels for the business function: ‗Respond to
or conduct cyber attacks or intelligence‘.
Level 1
Functionalities
Requirements
In response to cyber attacks.
Ability to perform attack analysis
in response to cyber attacks.
When conducting cyber attack or
intelligence.
Gain a limited understanding of
the effects of a cyber weapon.
Level 2
Determine basic impact of
malicious activity.
evaluating results to make
recommendations for mitigation.
Level 3
Assess the methods,
consequences, and
countermeasures needed to repel
malicious activity or attack.
set of assets.
Simulated network has the same
network.
IT-operations maintain the CTR.
Replay cyber events for analyses
To conduct cyber attack or intelligence.
Assess the most effective means
to achieve an intended effect.
Support decision making by
determining if the designated
effect is achieved or that other
(undesirable) effects are achieved
as well, when releasing a cyber
weapon.
Level 4
Determine system and data
integrity after an attack and
assess the loss or compromised
key data.
Test and train system and data
recovery
Perform research to discover
vulnerabilities in a target system.
Understand the ratio between the
time needed to execute an
offensive or intelligence activity
Simulate networks, whether own or
target networks that contain an
network.
Assets scalable in diversity and
numbers.
Preconfigured configurations of live
networks are available.
and network traffic both in type and
amount.
The CTR is accessible from different
locations.
Page 48 of 78
and the time given.
Determine risks of being
compromised when executing an
activity in relation to the returns.
Level 5
Test large-scale mitigation,
system, and data recovery prior to
actually recovering a live network.
simulating major incidents in the
CTR in order to determine which
decision should be made, decide
who should make decision and
determine the prioritization of
decisions.
Assess the most effective means
to achieve an intended effect.
unknown vulnerabilities in a target
system.
determining if the designated
effect is achieved or that other
(undesirable) effects are achieved
as well, when releasing a cyber
weapon.
4.3.6
the CTR.
Possibility to employ known
vulnerabilities in live networks for
training purposes
support.
simulation of assets of the live
network.
networks are quickly deployed into
the CTR.
The CTR offers different types of
configurations with different security
classifications within separate
compartments.
The CTR federates with other CTR's.
Decision making framework and
procedures.
Applied research into future cyber solutions
This paragraph describes the maturity levels for the business function: ‗Applied
research into future cyber solutions‘.
Level
1
Level
2
Level
3
Functionalities
Applied research on single asset or
small network.
Limited development of means to
detect and mitigate malicious
activity and recon and scan target
systems.
Additional to level 1Applied
research on simulated network
Limited development of means to
detect and mitigate malicious
activity and recon and scan target
systems.
Requirements
Limited research and development
of new means to detect, terminate,
prevent, or mitigate damage from
malicious activity.
set of assets.
Basic IT-infrastructure available and
results.
Ad-hoc simulated networks in
accordance with the development
requirements
Page 49 of 78
To some degree, conduct research
into shortfalls of an ITenvironment.
Level
4
Research and development of new
means to detect, terminate,
prevent, or mitigate damage from
malicious activity with general
testing possibilities.
Research shortfalls in the ITenvironments‘ ability to withstand
cyber attacks.
Research and development with
general testing possibilities of new
means to recon and scan target
systems.
Research new means to exfiltrate
information and the ability to
create the effect of deception,
disruption, denial, degradation, or
destruction within a target system.
Perform limited research to
discover unknown vulnerabilities
and develop exploits in order to
access a target system based on a
discovered vulnerability.
Level
5
Research and development of new
defensive means with rigorous
testing possibilities.
Research and development with
rigorous testing possibilities of
offensive and intelligence means.
unknown vulnerabilities and
develop exploits in order to access
a target system based on a
discovered vulnerability.
4.3.7
network.
network.
type and amount.
the CTR.
Replay, capture, and monitoring
functions for research and
development purposes.
support.
network.
the CTR.
compartments.
CTR's.
Assess current means deployed by own organization
This paragraph describes the maturity levels for the business function: ‗Assess
current means deployed by own organization‘.
Level 1
Functionalities
Requirements
In preparation for responding to cyber
attacks.
Assess ability to detect basic
malicious activity.
In preparation for conducting cyber
Limited understanding to what
extent current offensive or
intelligence means are traceable
Page 50 of 78
and what actions increase the
chance of getting compromised.
Level 2
attacks.
Understand if core defensive means
detect, terminate, and prevent
damage from malicious activity.
Determine if current means are
able to deliver basic assessments
and indicate general risks.
results.
Determine if the current tactics and
procedures for performing attack or
intelligence activities are effective.
Determine if personnel is able to
perform basic activities and
possess the needed skills and
knowledge.
Level 3
attacks.
Assess if defensive means detect,
terminate, prevent or mitigate
damage from malicious activity and
are able to deliver assessments
and indicate risks.
Assess the effectiveness of the
decision making process by
simulating major breakdown of the
IT-environment.
set of assets.
network.
IT-operations maintain the CTR.
Monitoring and replay functions for
assessment and debrief functions.
support.
Determine if the basic techniques
for performing attack or
intelligence activities are effective
and if personnel is able to perform
these activities.
Acquire basic understanding to
what extent attack or intelligence
means are traceable and what
actions increase the chance of
getting compromised.
General understanding of the ratio
between the times needed to
execute an attack or intelligence
activity and the time given.
Level 4
In preparation to responding to cyber
attacks.
Basic understanding of the ITenvironments' resilience to
malicious activity and the recovery
time, procedures and
consequences in case the ITenvironment is struck by damaging
malicious activity.
network.
Page 51 of 78
Level 5
In preparation to conducting cyber
attacks or intelligence.
Determine if the current tactics,
techniques, and procedures for
performing attack or intelligence
activities are effective and if
personnel is able to perform these
activities.
Understand to what extent current
means are traceable and what
actions increase the chance of
getting compromised.
Understand the ratio between the
time needed to execute an activity
and the time given.
Determine if cyber weapons do
what is expected of them, assess if
the designated effects are achieved
and understand the side-effects.
type and amount.
IT-operations maintains and
configures the CTR.
attacks.
Determine the IT-environment
resilience to malicious activity and
the recovery time, procedures and
consequences in case the ITenvironment is struck by damaging
malicious activity
network.
the CTR.
compartments.
CTR's.
Determine deployment of offensive
and intelligence capabilities in near
real live situations
Determine command decision
structure in offensive capabilities
Determine full operational
capability and integration on
tactical, operational en strategically
levels
Page 52 of 78
Cyber Test Range roadmap
This paragraph proposes the roadmap for the implementation of the identified CTR business functions and the necessary requirements to
deliver these functions. The roadmap is presented in Figure 15 and explained in the next pages.
Cyber test range roadmap
(4)
(2)
Research external solutions to enhance cyber operations capability
Scope: maturity level 3
Design
Implement
Operate
Design
Design
Implement
Operate
Design
2. Organizational requirements
- Ensure availability of supporting personnel
4. Evaluate
1. Design and implement functionalities
2. Technical requirements
- Possibility to employ known vulnerabilities in live
networks for training purposes
4. Evaluate
-Assure security of CTR
-Replay cyber events for analyses
Design Design
Implement
Operate
- Able to simulate live environments
- Assure security of CTR
- Able to simulate traffic in live environments
- Ability to scale CTR
- Ability to quickly deploy configurations
- Able to federate with different CTR’s
T+1
Design
-Attain integrated and remote staff facilities for support
T+2
Figure 15: CTR roadmap
Page 53 of 78
T+3
Implement
Design
Operate
Implement
- Offer different types of configurations with different
security classifications within separate compartments.
- Decision making framework and procedures.
T+4
4. Evaluate
Operate
- Attain instructors capacity
- Attain IT-maintenance
- Attain staff facilities for support
4. Evaluate
Implement
3.
Evaluate
(5)
(3)
Operate
(1)
Design
Implement
- Attain researchers capacity
4. Evaluate
4.4
T+5
Operate
3.
Evaluate
T+6 >
The roadmap consists of several elements, all surrounding the business functions.
These elements are as follows:
Elements
Description
This element presents the different CTR business function. Each
business function has its own colour, which matches the colours
used in paragraph 4.2.
This element presents the steps for the realization of a business
function. The steps consist of: design, implement, and operate.
This element presents the maturity level for the business
function that is strived to be realized. The maturity levels are
based on the levels in paragraph 4.3.
This element presents the necessary requirements that need to
be implemented in order to deliver a business function. The
requirements are based on the requirements in the maturity
model (par 4.3)
This element presents the timeframe in years. T stands for the
moments when the implementation of the CTR is started. T+1
represents the first year after the start.
Page 54 of 78
4. Evaluate
From a cyber operations
viewpoint, the business
function aimed at training
Design
Implement
Operate
Operate
Design Design Implement
personnel is deemed most
important (see par 4.2.2).
Therefore, this function is
- Able to simulate traffic in live environments
- Able to simulate live environments
- Ability to scale CTR
- Assure security of CTR
proposed to be realized
- Ability to quickly deploy configurations
- Able to federate with different CTR’s
- Attain instructors capacity
first. In addition, the need
- Attain IT-maintenance
-Attain
integrated and remote staff facilities for support
- Attain staff facilities for support
described by the cyber
operations capabilities
fits the functionality of maturity level 5 (see par 4.3.3). As a result, it is proposed to
achieve a level 5 maturity for this business function. The aim is to achieve level 5
between 2 and 3 years. The realization is suggested to take place in two phases,
one phase aimed at achieving maturity level 3, and the second aimed at level 5. The
reason for this is that all necessary requirements will have to be designed and
implemented for the first time. Therefore it is expected that these requirements will
be technically and organizationally challenging and require many financial and
human resources. Moreover, starting small enables to use lessons learned for
implementing the next maturity level. That is the reason why an incremental
development is suggested.
4. Evaluate
4.4.1
4.4.2
4.4.3
4. Evaluate
4. Evaluate
The
cyber
operations
Research external solutions to enhance cyber operations capability
capabilities indicated that
the business function in
Design
Implement
Operate
Operate
Design
Design Implement
which external solutions
are researched is the
second priority to be
-Assure security of CTR
- Possibility to employ known vulnerabilities in live
available through the CTR
-Replay cyber events for analyses
networks for training purposes
3.
Organizational
requirements
(see par 4.2.2). The level
3.
Organizational requirements
- Attain researchers capacity
of quality should also be
aimed at achieving a level
5 maturity (see par 4.3.4). This level is proposed to be achieved between 4 and 5
years. The realization follows the same path as the previous business function. The
requirements that are necessary for this business function to operate are the same
as the requirements for the previous business function. Therefore, this business
function can take advantage of the implementations done for the realization of the
function to train personnel. However, there are still specific requirements that have
to be realized for maturity levels 3 and 5.
4. Evaluate
The ability to support response to cyber
attacks or to prepare for performing
cyber attacks or intelligence, is graded
as a third priority (see par 4.2.2). As
Design
Implement
Operate
this business function, directly support
the ability to perform cyber operations it
is suggested to achieve a maturity level
5 between 4 and 5 years (see par
- Offer different types of configurations with different
security classifications within separate compartments.
4.3.5). This business function takes
advantage of the fact that most
- Decision making framework and procedures.
requirements are implemented in order
to deliver the previous business
functions. Therefore, it is proposed to obtain a level 5. The ability to offer different
configurations types in the CTR which have different security classifications with
different compartments may prove technically challenging.
4.4.4
The business function that enables the
assessment of the current means is graded to
be one of the least important business
functions to be realized (see par 4.2.2). With
the roadmap set at realizing business
functions within the forthcoming 5 years, it is
suggested to set the ambition for this
business function to a maturity level 3 (see
par 4.3.7). To achieve this level all
requirements should by now be implemented,
making the realization of level 3 maturity less
complicated.
Page 55 of 78
Design
Implement
Operate
3.
Evaluate
4.4.5
This last business function focuses on
conducting research into future cyber
solutions and follows the same path as the
business function for assessing current
means. The priority is seen as one of the
lowest by the cyber operations capabilities
(see par 4.2.2). It is also suggested to aim at
a maturity level 3 (see par 4.3.6). The
reasons is the timeframe of this roadmap, but
more importantly cyber operations can make
use of the industry‘s ability to conduct
research, limiting the need to acquire the
capacity to conduct applied research in house.
4.5
Design
Implement
Operate
3.
Evaluate
Validation
The roadmap as presented in paragraph 4.4 is validated through the use of an
expert opinion. Major G. Wens was asked to validate the outcome. He is the staff
officer R&D within the Taskforce Cyber Operations; as such, he is responsible for the
development of the cyber test range within the MoD.
The priorities and the ambition levels of the business functions presented in the
roadmap match his perception, with two additions.
The business function for assessing current measures has a lower priority in
comparison to other business functions. This is striking. A lower priority is
understandable from a cyber attack point of view, as the research into this
capability is starting and thus, the need for assessment of current – cyber attack –
measures is expected necessary in a few years. However, cyber defense and cyber
intelligence are already operational. Therefore, one would expect there is an existing
need from these capabilities to be able to assess their current cyber defense and
intelligence measures, as this will improve the current effectiveness of both cyber
capabilities.
Developing, implementing and optimizing the cyber test range is a continual
process. Therefore, a roadmap should preferable present the same characteristics in
which there is an iterative method for the control and continuous improvement of
business functions and its ambition levels and requirements.
Page 56 of 78
4.6
Summary
A roadmap is proposed for establishing the delivery of business functions through
the CTR and the realization of the needed technical and organization requirements.
This roadmap is spread over the next five years.
The process for defining a roadmap consists of two steps. First, it has to be clear
which priorities are deemed most important by the cyber operations capabilities
towards the business functions offered by the CTR. The second step is to define the
different levels of functionalities within a business service along with the necessary
requirements.
The priorities are formulated through two variables. The first variable is the need to
use these business functions quickly (this is indicated through ‗urgency‘). The
second is the complexity for the realization of necessary requirements (which is
indicated through ‗complexity‘). The combination of these variables represents the
priorities for the realization of business functions. The business function that enables
personnel to act in the cyber domain, is deemed most important. This function is
followed by the ability to research external solutions to enhance cyber operations.
The possibility to use the CTR in response to cyber attacks or to use the CTR in
preparation for conducting cyber attacks or intelligence operations is seen as the
third priority. The business functions that assess current means and conduct
research into future cyber solutions are deemed least important.
A CTR maturity model is developed in order to be able to define different levels of
functionalities and differentiate the necessary requirements. The methodology for
defining a maturity model is based on three steps. The first step is to link the CTR
requirements to the individual CTR services. The second step is to abstract the
requirements coupled to CTR services to the level of business functions. The final
step is to split the requirements into different levels ranging and linking these
differentiated requirements to service levels for each business function. The
maturity model consists of 5 levels in which level 1 is the most basic and level 5 is
the most advanced. Each level has a general description. For each business
function, a description of the functionalities is given in each maturity level, along
with the requirements to deliver the functionalities in each maturity level.
The roadmap offers an overview of all the business functions and their timeframe.
For each business functions, an ambition level is proposed through a maturity level.
In addition, the roadmap offers an overview of the requirements that have to be
realized in order to be able to deliver the business functions based on its ambition
level.
Page 57 of 78
5
Conclusions and recommendations
This chapter ends the thesis with conclusions and recommendations of the research
that is done to design a roadmap for the development of a cyber test range. The
first section discusses the conclusions. The second section presents the
recommendations.
5.1
Conclusions
5.1.1
Understanding cyber operations
Through cyber operations the military have the capability, next to other military
capabilities, to defend and/ or attack (or gather intelligence) in order to achieve
political or military objectives or effects. Cyber operations are conducted through
offensive, defensive and intelligence capabilities in which (1) cyber defense is aimed
at protecting its own networks and systems, (2) cyber attack is aimed at disrupting,
denying, degrading or destroying networks and systems, (3) cyber intelligence
enables intelligence collection through networks and systems.
5.1.2
Current state of cyber test ranges
Cyber test ranges are virtual environments that are mostly used for research and
development, evaluation and training purposes within the cyber domain. CTR can
therefore be considered as a ‗digital shooting range‘ in which military personnel
conduct military exercises, train their drills, test their skills or test weaponry, which
is comparable to a shooting range in the physical world. CTR offers the military
stakeholder the ability to - more effectively - defend and attack (or gather
intelligence regarding) infrastructures or military capabilities (Figure 6, p19).
The requirements for cyber test ranges are demanding. The main requirement is the
ability to replicate networks and computer systems in order to conduct realistic
experiments or exercises without threatening real world environments.
5.1.3
The CTR delivers an added value for each cyber operations capability and for each
activity within a capability by offering tailor-made business functions. This makes
the CTR an important supporting instrument for conducting cyber operations.
The generic business functions (Figure 8, p27) deliver functionalities on the level of
cyber operations and support daily operations and research and development.
To support daily operations, the CTR can deliver business functions that (a) enable
personnel to act in the cyber domain, (b) to assess the effectiveness of current
capabilities and (c) to respond to cyber attacks or to conduct cyber attacks or
intelligence.
To enable research and development, the CTR can deliver business functions that
offer a platform to (a) conduct applied research into future cyber solutions or to (b)
research if external solutions add value to the MoD.
Page 58 of 78
The specific business functions are positioned on the level of cyber operations
capabilities, being defensive (Figure 9, p30), offensive (Figure 10, p34) or
intelligence capabilities (Figure 11, p37. The specific business functions are
specifically tailored to support one of the three capabilities.
5.1.4
Requirements for delivering CTR business functions
Technical requirements include the ability to simulate IT-environments with all its
characteristics. In addition, the CTR needs to be scalable in terms of different assets
and number of assets. Changing configurations of simulated live environments
should be possible by at the push of a button so as to support short times to
market. Moreover, the CTR should simulate different types of configurations
simultaneously with different security classifications within separate compartments.
Security is crucial and security requirements should be high.
The organization requirements include IT-staff to maintain and configure the CTR.
To support training or exercises, instructors should (a) formulate training goals, (b)
develop CTR training scenarios, as well as (c) accompany training or exercises and
(d) evaluate personnel, and (e) propose improvement possibilities for individuals or
groups. An analyst should propose improvements for current technical capabilities,
procedures, and policies used in cyber operations in order to enhance standing
operating procedures or technical means. To conduct experiments, researchers
should (a) formulate research goals, (b) develop experiments, and make (c)
recommendations for improving cyber operations.
5.1.5
Roadmap for the development of a cyber test range
The roadmap (Figure 15, p53) offers a timeline for the CTR for the delivery of
business functions and the implementation of technical and organizational
requirements during the next five years.
The business functions that enable personnel to act in the cyber domain should be
the main priority. Implementation is proposed through a level 3 maturity that
evolves to a level 5 maturity, with a total lead-time between 2 and 3 years.
The second priority is the business function that researches external solutions to
enhance cyber operations. The realization is also proposed through a phased
approach by reaching a level 3 maturity and subsequently a level 5 maturity. The
aim is to achieve level 5 between 4 and 5 years.
The ability to support response to cyber attacks or to prepare for performing cyber
attacks or intelligence operations is graded as a third priority. It is suggested to
achieve a maturity level 5 between 4 and 5 years.
The business functions that enable assessment of the current means and conducting
research into future cyber solutions are deemed least important. It is suggested to
set the ambition for these business functions to a maturity level 3, to be achieved
Page 59 of 78
5.2
Recommendations
5.2.1
Roadmap for the development of a cyber test range
5.2.2
5.2.3
1.
Collaborate with knowledge institutions (as TNO) and partners (as NCSC) in
order to validate the proposed CTR roadmap as to obtain a unbiased view in
regard of the CTR. Combine these views along with the proposed roadmap as a
basis for the CTR approach within the MoD.
2.
Establish an approach under supervision of the Taskforce Cyber within the NL
MoD for the realization of a CTR.
3.
Collaborate with the UK MoD in connection with the Federated Cyber Range
(42). Validate the NL MoD approach towards the CTR in terms of business
functions, requirements, and roadmap and incorporate their lessons learned into
the NL MoD approach for the CTR.
4.
Liaise with NATO, as there are also developments in the realization of a ‗cyber
test range functionality‘ and examine potential cooperation possibilities.
5.
Collaborate with the Cooperative Cyber Defence Centre of Excellence in regard
of their experience in preparing, facilitating, and conducting cyber defense
exercises supported by a cyber lab (40).
6.
Develop under the supervision of the Taskforce Cyber and in cooperation with
the three cyber operations capabilities, the CTR business function based on the
descriptions (see paragraph 3.3) and in relation to the CTR maturity levels
(4.3).
Requirements for delivering CTR business functions
7.
Formulate and acquire through the forthcoming Defence Cyber Expertise Centre,
the resources, in terms of researchers and instructors, to conduct trainings and
exercises and research and development.
8.
Determine in cooperation with the Defence Material Organisation, whether the
maintenance and configuration of the CTR will be conducted within the MoD or if
it is suitable for sourcing.
9.
Define research questions in order to and in cooperating with knowledge
institutes and businesses to research: (a) the possibilities for replicating live
networks in terms of assets and characteristics, (b) the ability for configuring
live environments to be rapidly deployed within the CTR, (c) the needed security
requirements and measures to protect sensitive information, (d) risk
management due to the possible disruptive nature of experiments or exercises
within the CTR, (e) accessibility from different locations and (f) health
management in order to guarantee that results from the CTR are trustworthy.
10. Define, design, and develop the DOTMPLFI measures needed for delivering
cyber test range functionality.
Page 60 of 78
6
Reflection
This chapter looks back on the choices made during the research, the research
process and the ideas evolved by new insights.
A broad or narrow scope?
One of the choices that had to be made concerned the research approach. It was
clear that the research area would address the potential of a cyber test range for
the MoD, as this is a new topic within the MoD and decisions about the use and need
for a cyber test range need to be made. I found it therefore important to view the
CTR from a business perspective in contrast to taking a technical approach. A
business perspective meant in my opinion to first establish in which way the CTR
contributes to a more effective and efficient cyber operations. Once established the
technical matters can be researched. Taking a technical approach first held the risk
that the outcome of the thesis would not match the business needs.
Taking the business perspective as a starting point presented the choice in the
research scope. As the business perspective for a test range is represented by cyber
operations and its capabilities, there was a choice whether to have a broad and
comprehensive or a narrow but more detailed research. I chose to have a broad
research scope which meant that I wanted to include all three capabilities instead of
one capability. The reason is that the capabilities are intertwined which presented
the opportunity to develop integral CTR business functions. For example, by
developing a business function aimed at exercises, all three capabilities can train
simultaneous with one capability aimed at defending and the others at attacking or
gathering intelligence. This approach worked well and offers a foundation for further
research for the specific business functions. The disadvantage of the broad research
scope was the additional time that was required which had consequences for some
elements.
Path to the CTR roadmap
The research methodology worked well. The methodology is aimed at understanding
the context and looking into research done in this field and based on these findings
researching the MoD CTR. This methodology provided the approach through which
the roadmap was derived. These logical steps are usable when conducting similar
research. However, the challenge was the gathering of information for
understanding the context of cyber and the CTR. It was done through desk
research. On the one hand there was loads of information, this presented a problem
when trying to answer a straightforward question as: ‗what is cyber?‘. On the other
hand, information was limited, especially regarding a CTR. Information is mainly
available regarding the technical implementations but I found it limited for
describing the use and need for a CTR. In case further research is done, it is
advisable to gather this information through case studies (par 2.4.2.). The gathering
of information for researching the added value of a CTR for the MoD was done
through interviews. This approach delivered the desired information and worked
well. The positive outcome was that it helped the organization to view the cyber
capabilities through a different structured method, which delivered interesting
discussions. The drawback was that these interview were very time-consuming for
the interviewees. In a future situation the use of workshops can be considered when
time is limited.
Page 61 of 78
The interviewees were all from the MoD. The original plan was to interview experts
from technical institutes or other organisations operating in the cyber domain, next
to the MoD colleagues. The involvement of these external experts should have
contributed to a more comprehensive understanding regarding the possibilities of a
CTR. Unfortunately due to constraints in time these experts where not interviewed.
During the development of the roadmap I discovered that there was a need to apply
differentiation in the level of functionalities of business functions. There was not
always a need to possess a ‗full blown‘ business function and in some cases less
functionalities were also sufficient. This conclusion resulted into the development of
maturity levels in CTR business functions. Although this maturity model was not
planned and it required more time, it adds value to the roadmap and contributes to
a better discussion regarding the role of a CTR within the MoD.
The end result
The main question was aimed at researching the potential of the CTR and the added
value for the MoD. Looking back at the process I am pleased with the end result.
This thesis delivers new insights and lays a solid foundation for further research.
The development of CTR business functions based on business needs contributes to
a more future proof approach as these business needs, in my opinion, are more
structural. Therefore, the focus can be laid on the technical and organizational
requirements and to research the best solutions to deliver these business functions.
In addition, the development of a CTR maturity model offers the opportunity to
discuss, decide and implement the right ambition level as desired by the MoD. This
thesis was certainly challenging not only because of the time constraints but more
due to the fact that a cyber test range is a new subject matter in general and
specifically to the MoD and therefore new theories had to be developed. Through
this thesis, I hope to make a contribution in the development of cyber and the cyber
test range within the Netherlands Ministry of Defence.
Page 62 of 78
7
Bibliography
1. Ministry of Defence. Defensie na de kredietcrisis: een kleinere krijgsmacht in een
onrustige wereld. April 2011. BS2011011591.
2. —. Defensie Cyber Strategie. Juni 2012.
3. —. Uitwerking visie Cyber Operations. May 2011. BS2011017863.
4. Cartlidge, et al. The IT Infrastructure Library; An Introductory Overview of ITIL V3. s.l. :
itSMF, 2007.
5. Cyberspace: Definitions and Implications. Ottis, Rain and Lorents, Peeter. 5th
International Conference on Information Warfare and Security, Dayton OH, US : Cooperative
Cyber Defence Centre of Excellence, 2010.
6. Wiener, Norbert. Cybernetics: or Control and Communication in the Animal and the
Machine. Cambridge : The MIT Press, 1948.
7. Thill, Scott. March 17 1948: William Gibson , Father of Cyberspace. Wired.com. [Online]
March 2011. http://www.wired.com/thisdayintech/2011/03/0317cyberspace-author-williamgibson-born/.
8. Kuehl, Dr Dan. From Cyberspace to Cyberpower: Defining the Problem. [book auth.]
Stuart H. Starr, and Larry K. Wentz Franklin D. Kramer. Cyberpower and National Security.
s.l. : Potomac Books, Inc. , 2009, Vols. in Cyberpower and National Security, ed. Franklin D.
Kramer, Stuart H. Starr, and Larry K. Wentz,.
9. US Department of Defence. The National Military Strategy for Cyberspace Operations.
December 2006.
10. —. Joint Publication 1-02, DOD Dictionary of Military and Related Terms. April 12, 2001.
11. —. Joint Publication 1-02, Department of Defense Military and Associated Terms. [Online]
November 15, 2011. http://www.dtic.mil/doctrine/dod_dictionary.
12. Ministry of Security and Justice. Cyber Security Beeld Nederland. December June 2012.
CSBN-2.
13. RSA. The Current State of Cybercrime and What to Expect in 2012. 2012.
14. Symantec. 2011 State of Security. 2011.
15. AIV. Digitale Oorlogvoering, No 77, AIV/No 22, CAVV. December 2011.
16. Paul Cornish, David Livingstone, Dave Clemente and Claire Yorke. On Cyber
Warfare. s.l. : Chatham House, November 2010.
17. US Department of Defence. Joint Publication 3-0, Joint Operations. August 2011.
18. NATO. Allied Joint Doctrine for Information Operations. November 2009. AJP 3.10.
19. van Paassen, te Paske, Smulders. Een overzicht van cyber onderzoek voor Defensie.
s.l. : TNO, 2011. 35570.
20. Dreijer. Offensieve Cyberoperaties. s.l. : Nederlandse Defensie Academie, March 2011.
21. Grant, Burke and Heerden, Van. (forthcoming) Comparing Models of Offensive Cyber
Operations. 2011.
22. Andress and Winterfeld. Cyber Warfare; Techniques, Tactics and Tools for Security
Practitioners. s.l. : Elsevier, 2011. ISBN 978-1-59749-637-7.
23. West-Brown, et al. Handbook for Computer Security Incident Response Teams (CSIRTs).
s.l. : Carnegie Mellon University, April 2003. CMU/SEI-2003-HB-002.
24. NC3A. Cyber Defence Capability Framework. December 2010.
25. BuxBaum, Peter A. Building a Better 'Cyber Range'. August 2011.
26. Sabo, Robert P. Standing Up the Information Operations Range. 2006.
27. The Information Assurance Range. Powell, Robert, Holmes, Timoty K. and Pie, Cesar
E. ITEA Journal 2010; 31: 473–477, 2010.
Page 63 of 78
28. US Department of Defence. The Global Information Grid (GIG) 2.0; Concept of
Operations. March 2009. Version 1.1.
29. Watson. Combat Readiness through Resilience in Hostile Cyber Environments.
30. Welshans. History of Cyber Testing and Evaluation - A Voice From the Front Lines. ITEA
Journal. 2010, Vol. 31, pp. 449-452.
31. Benzel, et al. Current Developements in DETER Cybersecurity Testbed Technology. 2009.
32. Van Leeuwen, Urias, Eldridge, Villamarin, Olsberg. Performing Cyber Security
Analysis using LVC Testbeds. 2010.
33. DARPA. National Cyber Range. DARPA. [Online]
http://www.darpa.mil/Our_Work/STO/Programs/National_Cyber_Range_(NCR).aspx.
34. —. Broad Agency Announcement (BAA); National Cyber Range. May 2008. DARPA-BAA-0843.
35. VanPutte. National Cyber Range; The Future of Cyber Testing & Experimentation, Test
Week 2010: Track 4. July 2010.
36. Defense Information Systems Agency. Department of Defense Information Assurance
Range: A Venue for Test and Evaluation In Cyberspace. August 2011.
37. Mirkovic, Jelena, et al. The DETER Project; Advancing the Science of Cyber Security
Experimentation and Test. s.l. : IEEE, 2010. 978-1-4244-6048-9/10.
38. Design, Deployement and Use of the DETER Testbed. Benzel, et al. DETER Community
Workshop on Cyber-Security and Test, Boston : s.n., August 2007.
39. NATO Cooperative Cyber Defence Centre of Excellence. CCD COE Training Courses.
CCD COE. [Online] http://www.ccdcoe.org/236.html.
40. —. Baltic Cyber Shield Cyber Defence Exercise 2010; After Action Report. 2010.
41. UK Ministry of Defence. Defence Minister opens UK cyber security test range. Ministry of
Defence. [Online]
http://www.mod.uk/DefenceInternet/DefenceNews/DefencePolicyAndBusiness/DefenceMinister
OpensUkCyberSecurityTestRange.htm.
42. Northrop Grumman. Federated Cyber Range; Cases studies. 2010.
43. —. Cyber Test Range. Northrop Grumman. [Online]
http://www.northropgrumman.com/performance/pdf/cybersecurity/technology/cyber_test_ran
ge.pdf.
44. BreakingPoint. Accelerating the Deployment of the Evolved Cyber Range.
45. Command Five Pty Ltd. Advanced Persistent Threats: A Decade in Review. Juni 2011.
46. Ministry of Defence. Minister Hillen over digitale frontlinie van de toekomst. Ministerie
van Defensie. [Online] April 13, 2011. http://www.defensie.nl/actueel/nieuws/
2011/04/13/46180923/Minister_Hillen_over_digitale_frontlinie_van_de_toekomst.
47. Von Hinke Kessler Scholder. DIVA: Samenwerken in samenhang. Het wat, waarom en
hoe van de Bedrijfsvoering-, Informatievoorziening- en ICT-architectuur bij Defensie. October
2006.
48. Abcouwer, Maes en Truijens. Contouren van een Generiek Model voor InformatieManagement; een landkaart van de relatie tussen een organisatie en haar
informatievoorziening' Management & Informatie. Management & Informatie, 1997, Vol. 5, 3,
pp. 92-102.
49. Design, Deployment, and Use of the DETER Testbed. Benzel, Terry, et al. DETER
Community Workshop on Cyber-Secu rity and Test, Boston : s.n., August 2007.
Page 64 of 78
Appendix I.
Detailed overview CTR business functions
Individual CTR business functions
NCR
Create a vision of the
future and derive
requirements for new
solutions
IO Range
IA Range
Basic and advanced research
and development (R&D)
Experimentation
Modelling and simulation
(M&S)
Developmental test and
evaluation (DT&E)
DETER
To conduct cyber
security experiments
such observation of
malware and
defenses, assessment
of defense
effectiveness, or
testing hypotheses
about a defense
CCD COE
Northrop
Grumman
UK Cyber range
Generic CTR
business functions
Research &
Development
Applied research into
future cyber defensive
capabilities
Applied research
into future cyber
solutions
future cyber intelligence
capabilities
Developmental test and evaluation
(DT&E)
Research external
solutions to
enhance cyber
operations
Page 65 of 78
future offensive cyber
capabilities
mechanism.
Test the applicability of
new ideas from the
research and
technology community
Specific CTR business
functions
Research external
solutions to enhance
cyber defense
Research external
cyber attack
Research external
cyber intelligence
Confirm or deny
system meets
requirements for
integration into
today’s
enterprise.
Operational
demonstrations
for and training of
operators
Operational test and evaluation
(OT&E)
Validate capabilities and
services provided by
CND tools and
mechanisms
Validate and improve
CND tactics, techniques,
and procedures
Validate acceptable level
of service of Computer
Network Defense
Service Providers
(CNDSPs)
Increase resilience
by evaluating
current systems to
determine their
effectiveness and
combat readiness
Evaluating
Network
Architectures
Component
tests
Assess if current
defensive measure are
effective
Assess current
measure deployed
by own
organization
Assess if current
intelligence measure are
Validate IA mitigation
strategies for programs
of record.
Exercises
Training certification
Studies and analysis
Improve cyber security
workforce operational
performance
Assess if current offensive
measure are effective
effective
Support technical
training courses.
Aimed at
individuals.
Support technical
training exercises.
Aimed at groups
Increase resilience
by evaluating
personnel to
determine their
effectiveness and
combat readiness
Cyber Problem
Prevention
Training
Cyber Problem
Response
Training
Train personnel to
withstand a cyber attack
Enable personnel to
act in the cyber
domain
Train personnel to
execute a cyber attack
intelligence through
cyber
Page 66 of 78
Battle lab demonstrations
Targeting and battle damage
assessment (BDA)
development
Respond to or
conduct cyber
attacks or
intelligence
Assessments of unintentional
weapon effects
Tactics, Techniques, and
Procedures (TTP) development
Rules of engagement (ROE)
approvals and authorities
Legal reviews and assessments
Tool and weapon system
operations
Joint Munitions Effectiveness
Manual (JMEM)
validation
Proof of concept
Conduct cyber
intelligence
demonstration
Weapon/capability
performance characterization
Mission rehearsal
Page 67 of 78
Appendix II.
Interview protocol
Goal
This interview protocol is intended to research the expectations from a defensive,
operational and intelligence capabilities viewpoint towards the CTR and formulates
the expected MoD cyber test range functions. Therefore interviews are held in which
the cyber capabilities activities which are derived from offensive and defensive
frameworks are mapped to the test range goals which are used by external ranges.
The information is provided by MoD representatives of defensive, operational and
intelligence capabilities. The number of participants is 6.
Result
The interview provides information on the following topics:
1. Validate the test range goals which are used by external ranges
2. Determine to which extent the CTR supports the 6 activities within the cyber
defense process;
3. Determine to which extent the CTR supports the 8 activities within the cyber
attack process from an attack and intelligence point of view;
4. Identify the necessary technical and organizational consequences for meeting
the expectations.
Method
A short introduction is given in advance to the interviewee onto the context of the
interview. In addition specific information is provided regarding the capability which
are represented by the interviewee (defensive, offensive or intelligence).
The interviewee is approached with the question whether the person wants to
participate. An appointment is made in which the context is explained. It is also
indicated that it is an open interview which lasts 1 hour. The draft report is
composed and presented to the interviewee to validate.
The security level of the thesis is unclassified.
The interview consists in three parts:
1. Introduction (15 min)
2. Open interview (30 min)
3. Completion (15 min)
The interview
Introduction
Get acquainted with each other
Appreciation for the cooperation
Check whether information is received prior to the appointment, check whether
context is clear.
Explain the interview process
Explain a draft report will be composed and presented to the interviewee
Check recording permission
Confidentiality. Classified topics will not be mentioned in the thesis and will be
adapted to be unclassified.
Page 68 of 78
Questions
1. What is your view of the possibilities of a CTR from a broad perspective?
2. What is your view of the expectations of a CTR when these are mapped to cyber
defensive?
Possible follow-up questions:
Can you elaborate on the topic?
How will the cyber <…..> capability profit from a CTR?
What is the added value compared to the current situation?
Present the overview of CTR goals. Inform interviewee on the back ground and
check whether the goals as derived from the desk research include the possibilities
and expectations from question 1 and 2. Hereby validation of the desk research is
done.
Present the defensive or offensive or intelligence activities (which is applicable in the
specific interview).
3.
How will the specific activity profit from a CTR?
Discuss the 8 activities from the cyber attack or intelligence process: recon,
scan, access, escalate, exfiltrate, assault, sustain and obfuscate.
Discuss the 6 activities from the cyber defense process: malicious activity
detection, attack termination, -prevention, -mitigation, dynamic risk damage
and attack assessment, cyber attack recovery, timely decision making, cyber
defence information management.
Possible follow-up questions:
Can you elaborate on the topic?
Who will profit if the activity is supported by a CTR?
4.
5.
What are the technical consequences for implementing the specifications?
What are the organizational consequences for implementing the specifications?
Completion
Are there any relevant issues which have not been discussed?
Check possible confidentiality of information
Draft report will be send within 5 days for approval
Thank interviewee for cooperation
Page 69 of 78
Appendix III.
Interviews expectations towards a cyber test range
Contact author for interviews
Page 70 of 78
Appendix IV.
Calculations of priorities per CTR business function
This table presents the outcome of the interviews held to establish the urgency for
CTR business functions from a cyber attack and intelligence viewpoint. There were
four respondents. Two respondents represented cyber attack and two cyber
intelligence. The urgency is based on the average.
Cyber attack and intelligence
Cyber Test
Range Business
Functions
Respondents
Cyber Test Range Services
Average
urgency
1
2
3
4
4,5
5
4
4
5
4
4
4
5
4
3,5
3
4
3
5
3
3
3
1
3
3,5
3
4
4
3
3
3
3
1
2
3,5
4
3
5
4
Train general competences, such as: develop and
maintain necessary skills and knowledge
Train cyber attack execution, such as:
train ability to recon a target system
Train personnel
to execute cyber
attacks
train ability to scan a target system
train ability to gain access to a target system
train ability to execute an attack on a target system
Perform exercises, such as:
train offensive skills
train collaboration between offensive activities
Determine effectiveness of current means, such as:
determine effectiveness of recon activities
Assess if current
offensive means
are effective
determine effectiveness of scanning activities
determine effectiveness of access activities
determine effectiveness of escalate activities
Support cyber attack process, such as:
Conduct cyber
attacks
Assess most effective means to attack a target system
Applied research
into new future
cyber offensive
capabilities
Research new means, such as:
Determine added value of external solutions, such as:
Research
external
solutions
determine usability of external solutions
determine quality of external solutions
determine side effects of external solutions
Page 71 of 78
This table presents the outcome of the interviews held to establish the urgency for
CTR business functions from a cyber defense viewpoint. There were four
respondents. Two respondents represented cyber attack and two cyber intelligence.
The urgency is based on the average.
Cyber defense
Cyber Test
Range Business
Functions
Cyber Test Range Services
Respondents
Average
Urgency
1
2
Train general competences, such as: develop and maintain
necessary skills and knowledge
4,0
5
3
4,5
4
5
4,0
4
4
3,0
3
3
2,5
2
3
2,0
2
2
3,5
3
4
3,0
4
2
Train cyber defense execution, such as:
train ability to detect malicious activity
train ability to stop an attack
Train personnel
to withstand
cyber attacks
train ability to perform attack or risk assessments
Train toolhandling
train detection toolhandling
train attack termination and prevention toolhandling
Perform exercises, such as:
train defensive skills and learn cyber attack methods
train collaboration between defensive activities
Determine effectiveness of current means, such as:
determine effectiveness of detection means
determine ability to stop an attack
Assess if current
defensive means
are sufficient
determine ability to deliver assessments
Optimize cyber defense process, such as:
Identify information needs
Establish decision making process
Improve decision making process
Support defensive capability during an attack, such as:
Perform attack analysis
Respond to cyber
attacks
Perform damage assessment
Perform attack recovery
Support decision making
Research new means, such as:
Applied research
into new future
cyber defensive
capabilities
research new means to detect malicious activity
research new means to mitigate attack patterns
Determine added value of external solutions, such as:
Research
external
solutions
determine usability of external solutions
determine quality of external solutions
determine side effects of external solutions
Page 72 of 78
Appendix V.
Calculations of CTR requirements‘ complexity
This table presents the outcome of the interviews held to establish the complexity
for implementing the requirements necessary to deliver CTR business functions.
There were six respondents. Two respondents from per each cyber operations
capabilities.
Requirements
Average
The CTR should simulate own live systems
The CTR should simulate target systems
The CTR should contain a full simulation of the assets of the network
The CTR should simulate network traffic in terms of type and amount
The CTR should simulate human interaction in terms of of type and amount
The CTR should simulate different types of configurations within separate
compartments
The CTR should simulate different types of configurations with different security
classifications within separate compartments
The CTR should be scalable in terms of different assets
The CTR should be scalable in terms of number of assets
The CTR should be scalable in terms of number of users
The CTR should have preconfigured configurations of live networks
The CTR should be able to quickly deploy a configuration of a live network into
the test range
The CTR should be accessible from different locations
The CTR should be able to federate with other CTR’s
The CTR should be simple in use for the end-user
The CTR should be able to completely wipe previous configurations to ensure
that new configurations are clean of any injected malware or other risks
The security level should have the same security level of the live networks
The CTR should not have connections with live environments
Maintenance of the cyber test range must be easy
IT-operations must be able and available for maintaining and configuring the
cyber test range
An instructor should be able and available for formulating training goals and the
development of CTR scenarios which support the training goals
An instructor should be able and available to accompany training exercises
An instructor should be able and available for observing trainings in order to
evaluate the results of personnel and to identify lessons learned
A researcher should be able and available to formulate research goals and
develop experiments linked to these goals
A researcher should be able and available for analyzing experiments in order to
evaluate the results of measures or means and to make recommendations for
improvements
The end-user should be able and available to hone their cyber skills and
knowledge
Page 73 of 78
Respondents
complexity
1
2
3
4
5
6
3,2
4
3
3
4
2
3
2,2
4
1
4
1
2
1
2,3
4
1
3
2
2
2
3,0
4
2
3
2
4
3
2,7
4
1
3
2
3
3
3,2
3
3
3
4
3
3
2,8
3
2
3
4
3
2
3,0
3
3
3
4
3
2
3,2
3
3
3
4
3
3
3,2
3
3
4
3
3
3,2
4
2
3
4
3
3
3,0
4
2
3
4
2
3
3,5
4
3
3
4
4
3
2,8
4
4
3
3
2
1
3,8
5
4
3
4
3
-
3,7
2
4
3
5
4
4
4,3
4
4
4
5
5
4
4,3
5
4
2
5
5
5
3,5
5
4
3
3
3
3
3,8
5
4
3
5
3
3
3,6
3
3
4
4
4
-
3,6
3
4
3
4
4
-
3,2
3
3
3
4
3
-
3,4
4
3
3
4
-
3,5
4
3
4
4
3
3
2,8
1
3
2
4
4
3
Appendix VI.
Priority of CTR business functions
This table and graph present the input on basis of which the priority for CTR
business functions is proposed. The average complexity and urgency are calculated
through received through interviews (appendix V and VI). The count of services is
calculated on basis of the CTR services that make a business function.
Average of
Average of
Count of
Row Labels
Complexity Urgency
services
3,24
2,64
241
3,29
2,65
364
3,23
4,17
781
3,26
3,63
165
3,29
2,82
492
Priority cyber operations CTR business functions
3,50
future cyber solutions
3,45
3,40
deployed by own
organization
the cyber domain
Complexity
3,35
3,30
3,25
Research external
solutions to enhance cyber
operations
Respond to or conduct
cyber attacks or
intelligence
3,20
3,15
3,10
3,05
3,00
0
1
2
3
Urgency
4
Page 74 of 78
5
This table presents the input on basis of which the priority for CTR specific business
functions per cyber operations capability is proposed. The average complexity and
urgency are calculated through received through interviews (appendix V and VI).
The count of services is calculated on basis of the CTR services that make a
business function.
Average of
Complexity
Row Labels
Average of
Urgency
Count of
Business
functions
Cyber Attack
Applied research into future cyber offensive capabilities
3,22
3,00
119
Assess if current offensive means are effective
3,24
3,00
128
3,28
3,50
135
Research external solutions to enhance cyber attack
3,24
3,50
36
Train personnel to execute a cyber attack
3,21
3,79
323
Applied research into future cyber defensive capabilities
3,35
3,50
48
Assess if current defensive means are sufficient
3,40
2,91
117
Research external solutions to enhance cyber defense
3,29
3,00
72
3,31
2,00
222
Train personnel to withstand a cyber attack
3,26
4,40
299
Applied research into future cyber intelligence capabilities
3,22
1,50
74
Assess if current intelligence means are sufficient
3,24
2,00
119
3,28
3,50
135
Research external solutions to enhance cyber intelligence
3,23
4,50
57
Train personnel to gather intelligence through cyber
3,21
4,50
159
Cyber Defense
Cyber Intelligence
Page 75 of 78
The follow three graphs present the priorities of the CTR specific business functions
per cyber operations capabilities.
Priority cyber attack CTR business functions
3,4
Applied research into future cyber
offensive capabilities
Complexity
3,3
3,3
Assess if current offensive means
are effective
3,3
Train personnel to execute a cyber
attack
3,3
Research external solutions to
enhance cyber attack
3,3
3,2
0
1
2
3
Urgency
4
5
Priority cyber defense CTR business functions
Complexity
3,5
3,5
defensive capabilities
3,4
Assess if current defensive means
are sufficient
3,4
Train personnel to withstand a
cyber attack
3,3
enhance cyber defense
3,3
0
1
2
3
Urgency
4
5
Priority cyber intelligence CTR business functions
3,3
intelligence capabilities
Complexity
3,3
3,3
Assess if current intelligence means
are sufficient
3,2
intelligence through cyber
3,2
enhance cyber intelligence
3,2
3,2
0
1
2
3
Urgency
4
5
Page 76 of 78
Appendix VII.
Requirements for delivering business functions
This tables presents an overview of the technical and organization requirements
needed to deliver the CTR business functions. The values in the table present the
amount of requirements that are linked to the CTR services that exists within a CTR
business function.
Count of Consequences
Applied
research
into future
cyber
solutions
Row Labels
Organizational
A researcher should be able and available for analyzing
experiments in order to evaluate the results of measures or
means and to make recommendations for improvements
A researcher should be able and available to formulate
research goals and develop experiments linked to these goals
An instructor should be able and available for formulating
training goals and the development of CTR scenarios which
support the training goals
An instructor should be able and available to accompany
training exercises
An instructor should be able and available for observing
trainings in order to evaluate the results of personnel and to
identify lessons learned
IT-operations must be able and available for maintaining and
configuring the cyber test range
The end-user should be able and available to hone their cyber
skills and knowledge
Assess
current
means
deployed by
own
organization
Enable
personnel to
act in the
cyber
domain
Research
external
solutions to
enhance
cyber
operations
16
24
9
16
24
9
Respond to
or conduct
cyber
attacks or
intelligence
40
40
40
17
24
40
9
32
40
Technical
Maintenance of the cyber test range must be easy
The CTR should be able to completely wipe previous
configurations to ensure that new configurations are clean of
any injected malware or other risks
17
24
41
9
32
17
21
41
9
29
41
9
11
24
41
9
32
21
41
9
29
9
9
The CTR should be able to federate with other CTR’s
The CTR should be able to quickly deploy a configuration of a
live network into the test range
29
The CTR should be accessible from different locations
The CTR should be scalable in terms of different assets
17
The CTR should be scalable in terms of number of assets
3
The CTR should be scalable in terms of number of users
9
2
The CTR should be simple in use for the end-user
The CTR should contain a full simulation of the assets of the
network
The CTR should have preconfigured configurations of live
networks
15
24
3
41
9
32
17
18
41
9
29
18
41
9
29
The CTR should not have connections with live environments
The CTR should simulate different types of configurations
with different security classifications within separate
compartments
1
13
Page 77 of 78
15
41
29
41
29
The CTR should simulate different types of configurations
within separate compartments
The CTR should simulate human interaction in terms of of
type and amount
The CTR should simulate network traffic in terms of type and
amount
16
18
41
9
11
16
24
41
9
32
16
24
41
9
32
The CTR should simulate own live systems
17
16
23
7
14
The CTR should simulate target systems
The security level should have the same security level of the
live networks
13
15
25
5
18
17
24
9
32
Page 78 of 78

Roadmap for the MoD Cyber Test Range

Transcription

Similar documents

ENCORE: Studio: When State Actors and Cybercriminals Join Hands

Register Now!

Web Safety

The Role of a Cyber Test Range in Stress Testing the Security of

KennethGeers

VOITTAVA KYBERSTRATEGIA Jarno Limnéll

Supertel

Cyber liability DL October 2012.indd

It is a wonderful initiative to spread the message of

West Midlands Regional Cyber Crime Unit