Gap Assessment for ASME-ITI/AWWA J100-10 Standard

Transcription

Gap Assessment for ASME-ITI/
AWWA J100-10 Standard
and Leading Vulnerability
Assessment Tools
Web Report #4358
Subject Area: Management and Customer Relations
Assessment Tools
©2011 Water Research Foundation. ALL RIGHTS RESERVED.
About the Water Research Foundation
The Water Research Foundation (formerly Awwa Research Foundation or AwwaRF) is a member-supported,
international, 501(c)3 nonprofit organization that sponsors research to enable water utilities, public health
agencies, and other professionals to provide safe and affordable drinking water to consumers.
The Foundation’s mission is to advance the science of water to improve the quality of life. To achieve this
mission, the Foundation sponsors studies on all aspects of drinking water, including resources, treatment,
distribution, and health effects. Funding for research is provided primarily by subscription payments from
close to 1,000 water utilities, consulting firms, and manufacturers in North America and abroad. Additional
funding comes from collaborative partnerships with other national and international organizations and the
U.S. federal government, allowing for resources to be leveraged, expertise to be shared, and broad-based
knowledge to be developed and disseminated.
From its headquarters in Denver, Colorado, the Foundation’s staff directs and supports the efforts of
more than 800 volunteers who serve on the board of trustees and various committees. These volunteers
represent many facets of the water industry, and contribute their expertise to select and monitor research
studies that benefit the entire drinking water community.
The results of research are disseminated through a number of channels, including reports, the Web site,
Webcasts, conferences, and periodicals.
For its subscribers, the Foundation serves as a cooperative program in which water suppliers unite to pool
their resources. By applying Foundation research findings, these water suppliers can save substantial costs
and stay on the leading edge of drinking water science and technology. Since its inception, the Foundation
has supplied the water community with more than $460 million in applied research value.
More information about the Foundation and how to become a subscriber is available on the Web at
www.WaterResearchFoundation.org.
Assessment Tools
Prepared by:
Shannon D. Spence and Corinne M. Tuozzoli
Malcolm Pirnie, the Water Division of ARCADIS
44 South Broadway, 15th Floor, White Plains, NY 10601
Jointly sponsored by:
Water Research Foundation
6666 West Quincy Avenue, Denver, CO 80235-3098
and
Association of Metropolitan Water Agencies
1620 I Street NW Suite 500, Washington, DC 20006
Published by:
DISCLAIMER
This study was funded by the Water Research Foundation (Foundation) and the Association
of Metropolitan Water Agencies (AMWA). The Foundation and AMWA assume no responsibility
for the content of the research study reported in this publication or for the opinions or statements
of fact expressed in the report. The mention of trade names for commercial products does not
represent or imply the approval or endorsement of the Foundation or AMWA. This report is
presented solely for informational purposes.
Copyright © 2011
by Water Research Foundation
ALL RIGHTS RESERVED.
No part of this publication may be copied, reproduced
or otherwise utilized without permission.
CONTENTS
TABLES ....................................................................................................................................... vii FOREWORD ................................................................................................................................. ix ACKNOWLEDGMENTS ............................................................................................................. xi EXECUTIVE SUMMARY ......................................................................................................... xiii Objectives ........................................................................................................................ xiii Background ...................................................................................................................... xiii GAP Analysis................................................................................................................... xiii
Addressing the Gaps and Quantifying Effort................................................................... xiv
Recommendations ............................................................................................................ xiv CHAPTER 1: FEATURE DEFINITION........................................................................................ 1 Introduction ......................................................................................................................... 1 Method ................................................................................................................................ 2 CHAPTER 2: GAP ANALYSIS .................................................................................................... 5 Introduction ......................................................................................................................... 5 Method ................................................................................................................................ 5 CHAPTER 3: RECOMMENDATIONS....................................................................................... 25 Introduction ....................................................................................................................... 25 Method .............................................................................................................................. 25 Conclusion ........................................................................................................................ 29 APPENDIX A: J100-10 FEATURES .......................................................................................... 31 APPENDIX B: J100-10 GAP ANALYSES ................................................................................ 41 APPENDIX C: J100-10 RECOMMENDATIONS ..................................................................... 69 v
TABLES
2.1
Comparison of J100-10 Standard and ARAM-W™ Steps ................................................17
3.1
SEMS Summary Table ......................................................................................................26
3.2
VSAT Summary Table ......................................................................................................27
3.3
ARAM-WTM Summary Table ............................................................................................28
A.1
J100-10 Standard Features Matrix .....................................................................................32
B.1
J100-10 Standard Gap Analysis Matrix - SEMS ...............................................................42
B.2
J100-10 Standard Gap Analysis Matrix - VSAT ...............................................................52
B.3
J100-10 Standard Gap Analysis Matrix – ARAM-W™ ....................................................61
C.1
J100-10 Recommendations - SEMS ..................................................................................70
C.2
J100-10 Recommendations - VSAT ..................................................................................80
C.3
J100-10 Recommendations – ARAM-W™ .......................................................................86
vii
FOREWORD
The Water Research Foundation (Foundation) is a nonprofit corporation that is dedicated
to the implementation of a research effort to help utilities respond to regulatory requirements and
traditional high-priority concerns of the industry. The research agenda is developed through a
process of consultation with subscribers and drinking water professionals. Under the umbrella of
a Strategic Research Plan, the Research Advisory Council prioritizes the suggested projects
based upon current and future needs, applicability, and past work; the recommendations are
forwarded to the Board of Trustees for final selection. The Foundation also sponsors research
projects through collaborative programs and various joint research efforts with organizations
such as the U.S. Environmental Protection Agency, the U.S. Bureau of Reclamation, and the
Association of California Water Agencies.
This publication is a result of one of these sponsored studies, and it is hoped that its
findings will be applied in communities throughout the world. The following report serves not
only as a means of communicating the results of the water industry's centralized research
program but also as a tool to enlist the further support of the nonmember utilities and individuals.
Projects are managed closely from their inception to the final report by the Foundation's
staff and large cadre of volunteers who willingly contribute their time and expertise. The
Foundation serves a planning and management function and awards contracts to other
institutions such as water utilities, universities, and engineering firms. The funding for this
research effort comes primarily from the Subscription Program, through which water utilities
subscribe to the research program and make an annual payment proportionate to the volume of
water they deliver and consultants and manufacturers subscribe based on their annual billings.
The program offers a cost-effective and fair method for funding research in the public interest.
A broad spectrum of water supply issues is addressed by the Foundation's research
agenda: resources, treatment and operations, distribution and storage, water quality and analysis,
toxicology, economics, and management. The ultimate purpose of the coordinated effort is to
assist water suppliers to provide the highest possible quality of water economically and reliably.
The true benefits are realized when the results are implemented at the utility level. The
Foundation's trustees are pleased to offer this publication as a contribution toward that end.
Roy L. Wolfe, Ph.D.
Chair, Board of Trustees
Robert C. Renner, P.E.
Executive Director
ix
ACKNOWLEDGMENTS
The authors of this report thank the Water Research Foundation (Foundation) for its
financial, technical, and administrative assistance in funding and managing this project.
Specifically, the authors thank the Foundation Project Manager, Ms. Mary Messec Smith, and
the Project Advisory Committee members:




Mr. Kevin Gertig, City of Fort Collins Utilities
Mr. George Hoke, Fairfax Water
Mr. Charles M. Murray, Fairfax Water
Mr. John P. Sullivan, P.E., Boston Water and Sewer Commission
The authors gratefully acknowledge the support and assistance of Mr. Doug Owen, P.E.,
Mr. Devesh Sinha, Mr. Ryan Zink and Mr. Joshua Ross in the completion of this project.
xi
EXECUTIVE SUMMARY
OBJECTIVES
The objective of this research project was to conduct a gap analysis between the Joint
ASME-ITI/AWWA J-100-10 Risk Analysis and Management for Critical Asset Protection
(RAMCAP®) Standard for Risk and Resilience Management of Water and Wastewater Systems
(J100-10 Standard), and the three existing water/wastewater vulnerability assessment tools –
Security and Environmental Management System (SEMS), the Vulnerability Self Assessment
Tool (VSAT), and Automated-Risk Assessment Methodology tool for Water Sector (ARAMWTM). Specifically, the project sought to achieve the following goals:


Identify critical gaps, if any, between the J100-10 Standard and the results produced through
execution of the SEMS, VSAT, and ARAM-WTM.
Propose refinements necessary to achieve compatibility with the Standard, including scope
and scale of effort required.
BACKGROUND
A primary step in improving the security of critical infrastructure is the execution of a
vulnerability assessment. To that end, critical infrastructure sectors have developed a number of
assessment tools. However, the Department of Homeland Security (DHS) determined that the
Federal Government needed to be able to compare risks both within and between sectors.
Therefore, in 2007 the DHS developed the Risk Analysis and Management for Critical Asset
Protection (RAMCAP) methodology, which provides a standardized framework for measuring
risk, both within and across critical infrastructure sectors.
The water sector has developed several vulnerability assessment tools including SEMS,
VSAT, and ARAM-W™, and in the last few years efforts were undertaken to revise these three
tools to bring them in alignment with RAMCAP. The effort to automate the Risk Assessment
Methodology for Water (RAM-W™) began in 2009, and ARAM-W™ (the automated version) is
expected to be available in mid 2011. VSAT was updated in 2010.
The committee tasked with the development of what became the J100-10 Standard began
meeting in 2008. Members of the committee included government and industry representatives,
utilities, consultants, and the developers of both VSAT and ARAM. The public comment period
on the J100-10 Standard closed on 11/30/09, and it was published in July of 2010.
GAP ANALYSIS
A detailed review found gaps between each of the three software packages and the J10010 Standard. ARAM-WTM met 54 of the 79 features of the Standard (68%); VSAT met 52 of the
79 (66%); and SEMS met 38 of the 79 (48%).
None of the software tools calculate resilience – the largest single inconsistency with the
J100-10 Standard, and one that accounts for a number of peripheral inconsistencies in each
package. Other significant gaps include the following:
xiii
xiv | Gap Assessment for ASME-ITI/AWWA J100-10 Standard and Leading Vulnerability Assessment Tools
SEMS
 Consequence calculations do not include duration and severity of service denial
 The threat likelihood is assigned only at the utility level, not at the threat-asset pair level
VSAT
 The software does not include dependency and proximity hazards
 The software does not calculate risk directly but instead uses a proxy measure, the
calculation and manipulation of which is unclear
ARAM-WTM
 The consequences for the loss of critical assets are not evaluated for each specific threat
on the asset
 The software does not include dependency and proximity hazards
ADDRESSING THE GAPS AND QUANTIFYING EFFORT
The investigators then developed recommendations to address the identified gaps and the
associated labor by analyzing – to the extent possible – the databases used by each package. For
the purposes of calculating the amount of development effort that might be required, four
categories of software upgrades were identified – small (8-16 hours), medium (16-40 hours) and
large (40-120 hours). Using these broad ranges it was estimated that it will take approximately
1250 hours of labor to address the gaps in SEMS, 1150 hours for VSAT, and 700 hours for
ARAM-WTM.
However – and very importantly – there were also a number of gaps for which the effort
to correct could not be quantified. SEMS had seven such gaps, VSAT had five, and ARAMWTM had eight of these “unknown” size upgrades. Of these unquantifiable gaps, the largest are
as follows:



SEMS – adding the ability to identify improvement packages that affect multiple threat-asset
pairs and have the greatest benefit
VSAT – adding risk calculations instead of “risk-reduction units”.
ARAM-WTM – updating the consequences to be evaluated for the loss of critical assets for
each specific threat on the asset
Clearly it was very difficult to estimate with any accuracy, the amount of effort needed to
bring any one of these software tools into compliance with the J100-10 Standard.
RECOMMENDATIONS
Focusing on addressing the discrepancy between the J100-10 Standard and the software
tools around the issue of resilience could quickly bring VSAT and/or ARAM-WTM much closer
to compliance with the Standard. In addition, clarifying with the software developers the exact
labor needed to address the larger unknown upgrades is also a logical next step.
CHAPTER 1: FEATURE DEFINITION
INTRODUCTION
The format of an industry standard is such that the high level requirements are outlined in
the body of the standard in a relatively succinct fashion and the details are outlined in the
appendices that follow. The Joint ASME-ITI/AWWA J100-10 Risk Analysis and Management
for Critical Asset Protection (RAMCAP®) Standard for Risk and Resilience Management of
Water and Wastewater Systems includes six chapters and eight appendices. A simplified outline
of the J100-10 Standard’s Table of Contents is listed below.
Foreword
1 Introduction
1.1 Origin
1.2 Evolution of RAMCAP
1.3 RAMCAP in the Water Sector
1.4 History of the Standard
1.5 ANSI Approval Dates
2 RAMCAP Overview
3 Organization of This Document
4 Comments
Committee Roster
Risk and Resilience Management of Water and Wastewater Systems
1
Scope
2
Definitions
3 Bibliography
4
Requirements
4.1 Asset Characterization
4.2 Threat Characterization
4.3 Consequence Analysis
4.4 Vulnerability Analysis
4.5 Threat Analysis
4.6 Risk and Resilience Analysis
4.7 Risk and Resilience Management
5 Process Control
6 Verification
Appendices:
 Appendix A: Guidance on the Use of this Standard
 Appendix B: Optional Use of RAMCAP Scales for Recording Consequence and
Vulnerability Estimates
 Appendix C: Glossary
 Appendix D: Expanded Bibliography
 Appendix E: RAMCAP Reference Threats
1
2 | Gap Assessment for ASME-ITI/AWWA J100-10 Standard and Leading Vulnerability Assessment Tools



Appendix F: Proxy Indicator of Terrorism Threat Likelihood for the Water Sector
Appendix G: Integrated Analysis of Natural Hazards
Appendix H: Water Sector Utility Resilience Analysis Approach
Chapter 1 of the J100-10 Standard is short with more detail regarding the required steps
provided in Chapter 4, which lays out the mandatory seven-step RAMCAP process, as follows:
1) Asset Characterization
2) Threat Characterization
3) Consequence Analysis
4) Vulnerability Analysis
5) Threat Analysis
6) Risk/Resilience Analysis
7) Risk/Resilience Management
The eight appendices (A-H) include a greater level of detail regarding each step of the
J100-10 standard and address, to a much greater extent, the intent of the Standard. It appears that
the committee that drafted the Standard worked to keep it flexible while still creating an
approach that would produce consistent results that could be used for comparison both within
and across critical infrastructure sectors. Thus, the Standard also identifies the preferred
approach to the execution of each step. Generally (but not always) these are outlined in the
appendices.
Each software product analyzed as part of this project was measured against not only how
well it adhered to the mandatory portions of the Standard, but also how well it met the Standard’s
intent.
METHOD
In analyzing the J100-10 Standard, the researchers, defined a “mandatory feature” as one
that is in a mandatory section of the Standard is written in the text with the words “shall” or
“will”.
A non-mandatory feature of the Standard is defined as a statement that is either in a nonmandatory section or is written in the text with the words “should” or “may”.
The Standard also clearly delineates which appendices are mandatory and which are nonmandatory as follows:


Mandatory:
o Appendix E: RAMCAP Reference Threats
Non-Mandatory:
o Appendix A: Guidance on the Use of this Standard
o Appendix B: Optional Use of RAMCAP Scales for Recording Consequence and
Vulnerability Estimates
o Appendix C: Glossary
o Appendix D: Expanded Bibliography
o Appendix F: Proxy Indicator of Terrorism Threat Likelihood for the Water Sector
o Appendix G: Integrated Analysis of Natural Hazards
Chapter 1: Feature Definition | 3
o
Appendix H: Water Sector Utility Resilience Analysis Approach
In various places in the Standard there are conflicts and/or cross references between
mandatory sections; and between mandatory and non-mandatory sections. In the cases where
there are conflicts between mandatory sections the authors made a note of the conflict and
analyzed the software(s) against the most conservative interpretation of the conflicting
statements.
In the cases where there are conflicts between mandatory and non-mandatory sections the
authors took a two-pronged approach. As already stated, the authors assumed that a feature was
mandatory if it met the definition as outlined above (i.e. with the words “shall” or “will”) and is
in a mandatory section of the Standard. However, if the authors felt - based on an analysis of the
cross references and wording – that the intent of the Standard was clearly indicated in a nonmandatory section or feature, they also analyzed the software(s) against that feature. For
example, the Standard clearly identifies preferred methods for approaching many of the
mandated seven steps in non-mandatory appendices including:





Appendix B for the estimation of consequences
Appendix E to identify reference threats
Appendix F to calculate the Proxy Measure for malevolent threats
Appendix G to estimate the risk of the natural hazards
Appendix H to calculate the Operational Resilience Index for resilience.
Based on this approach the researchers created a master J100-10 Standard Features
Matrix (Table A.1) that documents each section of the Standard. This matrix was used during
the analysis of each software package. The matrix lists each chapter, section and the associated
features of the Standard. Some features are found in multiple sections or appendices and are
listed as such. You will note that the matrix does not proceed strictly in either numerical or
alphabetical order. This is because the body of the Standard refers to the appendices on an ad
hoc basis. Therefore, all parts of the Standard are addressed in the matrix but not necessarily in
numerical or alphabetical order. The matrix also includes comments on software usability and
aesthetics.
Chapter 2: Gap Analysis | 5
CHAPTER 2: GAP ANALYSIS
INTRODUCTION
The J100-10 Standard was analyzed against the three currently available vulnerability
assessment tools – Security and Environmental Management System (SEMS), the Vulnerability
Self Assessment Tool (VSAT) and Automated-Risk Assessment Methodology tool for Water
Sector (ARAM-W™).
The SEMS RAMCAP Risk Assessment is a part of a larger software suite. This suite
includes Drinking Water and Wastewater Compliance, Asset Management, and Security &
Emergency Management modules. The RAMCAP Risk Assessment is a part of the Security &
Emergency Management module. There are additional advanced features that may also be
purchased which integrate the SEMS software with other commonly used software systems such
as ArcView GIS, SCADA, billing systems, and LIMS.
VSAT and ARAM-W™ are each standalone tools designed to assess a utility’s risk and
vulnerability. VSAT was developed in 2002 and updated in 2010 by the National Association of
Clean Water Agencies (NACWA), in collaboration with the PA Consulting Group and
SCIENTECH, Inc. It was funded by the U.S. Environmental Protection Agency (EPA).
In 2009, water sector stakeholders identified a need to automate the Risk Assessment
Methodology for Water (RAM-W™) and this development effort was started in January 2009.
ARAM-W™ is the automated version of the RAM-W tool and is expected to be available to the
public in mid 2011.
METHOD
To execute the gap analysis, the researchers approached each software package from the
viewpoint of an end user; an end user was defined as a water or wastewater utility staff member
not necessarily familiar with the software package. The research investigators executing the
analysis were very familiar with the J100-10 Standard but were not familiar with SEMS, VSAT,
or ARAM-W™.
For the analysis, the investigators utilized the sample vulnerability assessment included
as part of VSAT with each of the three software packages. The sample assessment was for a
small combined water and wastewater utility. The assessment identified 14 critical assets and
countermeasures, 6 man-made threats, and 24 natural disaster threats.
In addition to using the same sample assessment with all three software packages as part
of their analysis, the investigators also made changes to parameters such as countermeasures and
types of threats in order to determine if the software was executing its internal calculations in
ways that met the intent of the J100-10 Standard.
Each item listed in the J100-10 Standard Features Matrix in Appendix A was compared
to the software functionality to determine if the software met both the mandatory and nonmandatory features of the J100-10 Standard. The detailed findings were then captured in the
J100-10 Standard Gap Analysis Matrix (Gap Analysis Matrix) in Appendix B. The Gap
Analysis Matrix has been broken down as follows: Table B.1 (SEMS), Table B.2 (VSAT) and
Table B.3 (ARAM-W™) for ease of review. An overview of the gaps that were found in each
software package is described in the remainder of this chapter. These are presented in the same
5
order as the seven steps of the J100-10 Standard. The steps are as follows, listed with their
corresponding paragraphs in the Standard:
1. Asset Characterization (4.1)
2. Threat Characterization (4.2)
3. Consequence Analysis (4.3)
4. Vulnerability Analysis (4.4)
5. Threat Analysis (4.5)
6. Risk/Resilience Analysis (4.6)
7. Risk/Resilience Management (4.7)
SEMS
The SEMS Technologies’ website (www.semstechnologies.com) states that the SEMS
Technologies’ Risk Assessment Software (SEMS) is a RAMCAP-consistent program that can be
used to help perform a risk assessment of a water or wastewater utility. However, while the
software may be RAMCAP consistent (not specifically analyzed as a part of this project), the
analysis showed that there are many gaps between the SEMS software and the J100-10 Standard.
These gaps are as follows:
Paragraph 4.1: Asset Characterization. The purpose of asset characterization is to
identify critical assets to be considered in the subsequent steps.
4.1.1 Mission. – No gap identified. The software meets the requirement of the J100-10
Standard to identify the utility’s mission.
4.1.2 Critical Assets. – No gap identified. To meet the Standard, the software
practitioner must create a list of all of the utility’s assets and select those that are critical. The
SEMS software provides a dropdown list of predefined assets, e.g., chemical pumps, storage
tanks, and valves.
4.1.3 Supporting Infrastructures – Meets J100-10 Standard. Improvements suggested.
To meet the Standard, the software must allow for critical internal or external supporting
infrastructures such as financial records, legal documents, planning documents, mutual aid
agreements, etc., to be identified. The SEMS software does not provide any critical internal or
external supporting infrastructures in its predefined list, although the user can add these
manually.
4.1.4 Countermeasures and mitigation measures/features – No gap identified. The
SEMS software meets the requirement to provide predefined countermeasures and to allow the
software practitioner to be able to add additional countermeasures and descriptions and/or
details.
4.1.5 Consequence metrics – Gap identified. To meet the Standard, the software must
allow the practitioner to estimate the worst reasonable consequences for each asset without
regard to the threat. These consequence metrics include potential for fatalities, serious injuries,
major economic loss to facility or community, loss of public confidence, and inhibiting effective
function of national defense or civilian government. The SEMS software does not allow the user
to define the worst reasonable consequences without regard to threat.
4.1.6 Prioritize Assets - Gap identified. To meet the Standard, the software must allow
the practitioner to identify assets as critical and to rank them using categories of high, medium
and low. The SEMS software does this. However, the software does not allow the practitioner to
reprioritize assets based on consequences, which is also part of the Standard.
Paragraph 4.2: Threat Characterization. In the Threat Characterization step, once the
critical assets have been determined, the user must define the threats that could potentially
impact each asset.
4.2.1 Malevolent Threat Characterization – No gap identified. The software uses J10010 Standard threats.
4.2.2 Natural Hazards Threat Characterization – Gap Identified. To meet the Standard,
when selecting the natural hazard threats, the software must allow the practitioner to define the
range of magnitudes, from the smallest magnitude that could cause serious harm to the largest
reasonable magnitude, of each natural hazard. Although the SEMS software does identify what
standards should be used for selecting the magnitudes of some natural hazards to be used as part
of the analysis (e.g. FEMA flood maps), it does not provide the actual ranges. The investigator
was unable to find direct links to reference materials that would allow the practitioner to
determine the likelihood of each magnitude of each natural hazard that might impact the assets.
In addition, the software also does not include some natural hazards, such as wildfires and ice
storms although these can be added manually. However, the largest apparent gap is that in the
SEMS software, the investigator found that changing the magnitude of a natural disaster did not
seem to affect the threat probability or overall risk results from the analysis.
4.2.3 Dependency Hazards Threat Characterization – Meets J100-10 Standard.
Improvements suggested. The Standard requires the user to define any dependency threats due
to interruptions in utilities, suppliers, employees, customers and transportation; as well as threats
due to the close proximity of the utility to dangerous neighboring sites. These threats are not
predefined within the SEMS software and must be manually added to the threat list.
4.2.4 Threat-Asset Pairs – No gap identified. The Standard requires the practitioner to be
able to assign potential threats to each asset. The SEMS software does this.
4.2.5 Threat-Asset Pair Ranking – Gap identified. To meet the Standard, once all of the
potential threats have been assigned to each asset (thus creating threat-asset pairs) the
practitioner must then rank them (using professional judgment) in order from the greatest to the
least resulting consequences. The SEMS software does not allow for this ranking and instead
proceeds with the analysis of all threat-asset pairs as if each were of equal importance.
4.2.6 Critical Threat-Asset Pairs – Gap identified. To meet the Standard, the
practitioner must use the ranking developed under paragraph 4.2.5 to select the critical threatasset pairs to be further analyzed (or to treat them all as critical). The SEMS software allows the
practitioner to select critical assets but not critical threat-asset pairs.
Paragraph 4.3: Consequence Analysis. In the Consequence Analysis step, once the
critical threat-asset pairs are identified, the worst reasonable consequences that can be caused by
the specific threats on the assets are defined.
4.3.1 Threat Scenario – No gap identified. To meet the Standard, the software must
allow the practitioner to identify the worst reasonable consequences that can be caused by
specific threats on specific assets for each threat-asset pair. The SEMS software does this.
4.3.2 Estimate Consequences – Gap identified. To meet the Standard, the software must
provide the practitioner with three options for estimating fatalities, serious injuries, financial loss
to the owner/operator, financial loss to the community and duration and severity of service denial
for the affected customers of the utility. These include: a single-point estimate, a single indicator
(a bin value) and a range. The SEMS software does provide a dropdown list of J100-10 Standard
ranges for each of the different types of consequences except for duration and severity of service
denial for the affected customers of the utility. However, it does not provide the option for the
practitioner to enter point value estimates or single indicators for each consequence. As this
feature was met by only one out of three requirements in the J100-10 Standard the researchers
have identified it as a gap.
4.3.3 Estimate Consequences (other) – Gap identified. To meet the Standard, the
software must allow the practitioner to add additional consequences, if desired. The investigator
was not able to find a way to add additional consequences.
4.3.4 Document Assumptions – No gap identified. To meet the Standard, the software
must provide a field for practitioners to document their assumptions and procedures for
performing the consequence analysis. The SEMS software provides a location for the
practitioner to do this.
4.3.5 Record Consequence – Gap identified. To meet the Standard, the software must
provide both a field for consequence ranges and a field for the practitioner to insert a point
estimate. The SEMS software provides the preferred J100-10 Standard consequence ranges, but
it does not provide a field for the practitioner to insert a point estimate.
Paragraph 4.4: Vulnerability Analysis. A key component of the J100-10 Standard risk
assessment is the vulnerability analysis, which determines the likelihood that given a threat or
hazard will occur.
4.4.1 Review – No gap identified. To meet the Standard, the software must provide fields
to input pertinent details of utility/facility construction, systems and layout. The SEMS software
provides a place to do this when identifying the asset.
4.4.2 Analyze Vulnerability – Gap Identified. To meet the Standard, the software must
provide a field for the practitioner to analyze the vulnerabilities of each asset in order to estimate
the likelihood that, given the occurrence of a threat, the estimated consequences will result. The
SEMS software utilizes the vulnerability values that are listed in the J100-10 Standard in
Appendix B, Table B-5. However, the software does not allow the user to actually determine
this value using standard methods; i.e. fault, event or failure tree analysis, path analysis,
vulnerability logic diagrams, computer simulation methods, or expert judgment. In the SEMS
software, asset vulnerability is analyzed based on the number of countermeasures assigned to the
threat-asset pair (as executed under Step 4.1.4) and the ability of the countermeasure to detect,
delay, and respond to the threat. The software displays the vulnerability as a J100-10 Standard
percentage range. This calculation is performed behind the scenes, within the software.
4.4.3 Document Method – Gap identified. To meet the Standard, the software must have
a field where practitioners can define their methods of analyzing vulnerability (e.g. those listed in
paragraph 4.4.2). The SEMS software does not.
4.4.4 Record Estimates – Gap identified. To meet the Standard, the software must have
a field where practitioners can record the vulnerabilities that they have calculated for each
critical asset. The SEMS software displays the vulnerability as a J100-10 percent range, but it
does not allow the user to input the vulnerability manually.
Paragraph 4.5: Threat Analysis. In the Threat Analysis step the practitioner determines
the likelihood (or frequency) that a specific malevolent event, dependency/proximity hazard, or
natural hazard will occur to a specific critical asset.
Gap identified. To meet the Standard, the software must allow the practitioner to assign
the likelihood (or frequency) of each of the selected hazards and threats in relation to each asset.
However, the SEMS software selects (from a table) a single threat likelihood and assigns it to the
entire utility. The only explanation that the investigators could identify for the values in the table
is that they are provided by the DHS. According to the software, this scale takes into account the
population served by the utility; the amount of onsite gaseous chlorine storage; the economic
impact of the utility; and the number of critical customers served by the utility. The single threat
likelihood is a significant difference from the J100-10 Standard. See paragraphs 4.5.1 through
4.5.5 below for further explanation.
4.5.1 Malevolent Threats - Gap identified. To meet the Standard, the software must
include malevolent threat likelihood calculations using proxy measures, best estimates, or
conditional assessments. The SEMS software uses only best estimates. As this feature was met
by only one out of three methods required by the J100-10 Standard the researchers have
identified this as a gap.
4.5.2 Natural Hazards – Gap identified. To meet the J100-10 Standard, the software
must include natural hazards threat likelihood calculations. It must also allow the practitioner to
assign the likelihood or frequency of a natural hazard to the asset, based on historical data that
may be provided by the software via maps, data, or links to reference materials. The software
must then calculate the risk of each natural hazard and sum them to determine the overall risk
due to natural hazards. The SEMS software has fields for historic information and magnitudes
(text boxes). However, the investigators found that these values do not impact the calculated
results of the analysis.
4.5.3 Dependency and Proximity Hazards – Gap identified. To meet the J100-10
Standard, the software must include dependency and proximity hazards threat likelihood
calculations. It must also include historical data on dependency and proximity hazards to
determine the likelihood that the threats will occur to the assets.
4.5.4 Record Estimates – Gap identified. To meet the J100-10 Standard, the software
must allow the practitioner to determine the likelihood of each specific threat occurring to each
specific asset and record this estimate, along with the method and reasoning for the estimate.
The investigator did not find a place in the software to input this estimate.
Paragraph 4.6: Risk/Resilience Analysis. Once the consequence, vulnerability, and
threat likelihood have been determined for each threat-asset pair, the overall risk and resilience
of the utility is calculated.
Gap identified. The SEMS software does not include any Risk/Resilience Analysis.
This is a significant difference from the Standard. See paragraphs 4.6.1 through 4.6.3 below for
further explanation.
4.6.1 Calculate Risk – Gap identified. To meet the Standard, the software must allow
the practitioner to calculate the risk associated with each threat-asset pair based on the
consequence, vulnerability, and threat likelihood values selected during earlier steps. However,
the investigators did not find a place to input a specific value for vulnerability or threat
likelihood.
4.6.2 Calculate Resilience – Gap identified. To meet the Standard, the software must
determine the overall resilience of the utility, including the duration of service denial and
severity of service denial (in gallons per day), in order to determine the resilience of each threatasset pair.
4.6.2.1 Operational Resilience Asset Resilience Metric – Gap identified. To meet the
J100-10 Standard, the software must calculate the asset’s resilience.
4.6.2.2 Owner’s Economic Resilience - Gap identified. To meet the Standard, the
software must calculate the owner’s economic resilience.
4.6.2.3 Community Economic Resilience - Gap identified. To meet the Standard, the
software must calculate the community’s economic resilience.
4.6.3 Record Risk and Resilience Estimates – Gap identified. To meet the Standard, the
software must calculate the overall risk to the utility using the J100-10 Standard risk equation;
the risk to each specific threat-asset pair; and the different types of resilience as defined in
paragraphs 4.6.2.1 through 4.6.2.3. The SEMS software gives each threat-asset pair a “Risk
Score” based on a tiered binning system. The software also creates an “Overall Risk” graph,
plotting the consequence versus the vulnerability of each asset. However, the investigators could
not find any explanation on how the binning system was developed. In addition, the SEMS
software does not allow for resilience calculations.
Paragraph 4.7: Risk/Resilience Management. Once the risk of the utility and of each
threat-asset pair has been determined, the utility continues the process by deciding whether
actions are needed to enhance all-hazards security or resilience or both.
Gap identified. The SEMS software does not proceed any further in allowing the
practitioner to manage risk and resilience. See paragraphs 4.7.1 through 4.7.7 below for further
explanation.
4.7.1 Decide – Gap identified. To meet the Standard, the software must allow the
practitioner to decide what risk and resilience levels are acceptable.
4.7.2 Define – Gap identified. To meet the Standard, the software must allow the
practitioner to define new countermeasures and mitigation/resilience options to reduce
unacceptable risk to specific threat-asset pairs.
4.7.3 Estimate – Gap identified. To meet the Standard, the software must allow the
practitioner to estimate the costs of the new countermeasures and mitigation/resilience options.
4.7.4 Assess – Gap identified. To meet the Standard, the software must allow the
practitioner to assess the options by analyzing the facility or asset under the assumption that the
option has been implemented.
4.7.5 Identify – Gap identified. To meet the Standard, the software must allow the
practitioner to identify those options that have benefits to multiple threat-asset pairs. The SEMS
software does not.
4.7.6 Calculate – Gap identified. To meet the Standard, the software must allow the
practitioner to calculate the net benefits and benefit-cost ratios of the selected countermeasures.
4.7.7 Review & Rank – Gap identified. To meet the Standard, the software must allow
the practitioner to review the selected countermeasures and rank them in order to determine
which ones will be most effective in reducing the utility’s risk.
Summary. In summary, the SEMS software is a user-friendly risk assessment tool that is
one module of a larger software package, and that utilizes many of the J100-10 Standard tables
and definitions for threat and consequences. However, the analysis showed that there are many
gaps between the SEMS software and the J100-10 Standard, the largest of which are as follows:
 Paragraph 4.2.2. Natural Hazards Threat Characterization – Changing the
magnitude of a natural hazard does not appear to affect the value of the threat
probability or overall risk results.
 Paragraph 4.3.2 Estimate Consequences – The software does not include duration
and severity of service denial for the affected customers of the utility, nor does it
provide the practitioner with point value estimates or single indicators for each
consequence.
 Paragraph 4.4.2 Analyze Vulnerability - The vulnerability of a threat-asset pair is
determined by the software, not the practitioner, and the investigators could not
identify a way of changing or verifying the determination of this value.
 Paragraph 4.5 Threat Analysis - The threat likelihood is assigned only at the
utility level, not at the threat-asset pair level, and is determined using a table
instead of proxy measures, best estimates, or conditional assessments as per the
Standard.
 Paragraph 4.6 Risk/Resilience Analysis - The SEMS software does not address
resilience.
 Paragraph 4.7 Risk/Resilience Management – The SEMS software does not
provide risk or resilience management.
Vulnerability Self Assessment Tool (VSAT)
According to the Environmental Protection Agency’s website (www.vsatusers.net) the
Vulnerability Self Assessment Tool (VSAT) software is a RAMCAP-consistent risk assessment
application for water, wastewater, and combined utilities. However, the analysis executed under
this project demonstrated that, although the software may be RAMCAP consistent (a
determination outside the scope of this project), it does not meet the RAMCAP J100-10 Standard
in some key respects. The details of this analysis follow.
4.1.1 Mission – No gap identified. To meet the Standard, the software must allow the
practitioner to input the utility’s mission. The VSAT software provides a field for this.
4.1.2 Critical Assets. – No gap identified. To meet the Standard, the practitioner must
create a list of the utility’s assets and select those that are critical. At the start of a VSAT
analysis, the software requires the practitioner to define all of the utility’s assets. The software
contains a list of commonly used assets and also allows custom assets to be defined.
4.1.3 Supporting Infrastructures – No gap identified. To meet the Standard, the
software must allow for critical internal or external supporting infrastructures such as financial
records, legal documents, planning documents, mutual aid agreements, etc., to be identified. The
VSAT software does this.
4.1.4 Countermeasures and mitigation measures/features – No gap identified. To meet
the Standard, the software must provide a way for the practitioner to identify and document
existing protective countermeasures and mitigation measures/features. VSAT contains a list of
countermeasures commonly found at utilities in the United States and their associated costs. In
addition, the practitioner can also add custom countermeasures and edit the countermeasures to
add additional details.
4.1.5 Consequence metrics – Gap identified. To meet the Standard, the software must
allow the practitioner to estimate the worst reasonable consequences for each asset without
regard to the threat. These consequence metrics include potential for fatalities, serious injuries,
major economic loss to facility or community, loss of public confidence, and inhibiting effective
function of national defense or civilian government. VSAT does not allow the user to calculate
the worst reasonable consequences resulting from the destruction or loss of an individual asset,
regardless of the threat. The software skips this step and only considers the consequences
associated with specific threat-asset pairs.
4.1.6 Prioritize Assets – Gap identified. To meet the Standard, the software must allow
the practitioner to identify assets as critical and to rank them using categories of high, medium
and low. VSAT does not allow the user to rank the critical assets. In VSAT, assets are either
“critical” or “not critical”.
Paragraph 4.2: Threat Characterization. Once the critical assets have been determined,
the user must define the threats that could potentially impact each asset.
4.2.2 Natural Hazards Threat Characterization – Meets J100-10 Standard.
Improvements suggested. To meet the Standard, when selecting the natural hazard threats, the
software must allow the practitioner to define the range of magnitudes, from the smallest
magnitude that could cause serious harm to the largest reasonable magnitude, of each natural
hazard. Although the VSAT software does adequately meet this requirement for most of the
natural hazards, it does not provide the magnitude ranges for wildfires, nor does it include direct
links to reference materials to help determine the likelihood of each magnitude.
Improvements suggested. The Standard requires the user to define any dependency threats due
to interruptions in utilities, suppliers, employees, customers and transportation, as well as threats
due to the close proximity of the utility to dangerous neighboring sites. The researchers could
not locate these threats in the VSAT software, although threats can be manually added to the
threat list.
4.2.4 Threat-Asset Pairs – No gap identified. The J100-10 Standard requires the
practitioner to assign potential threats to each asset. The VSAT software does this.
4.2.5 Threat-Asset Pair Ranking – Gap identified. To meet the J100-10 Standard, once
all of the potential threats have been assigned to each asset (thus creating threat-asset pairs) the
user must then rank them (using professional judgment) in order from greatest to least resulting
consequences. The VSAT software does allow the practitioner to create threat-asset pairs, but it
does not allow for this ranking.
4.2.6 Critical Threat-Asset Pairs – Meets J100-10 Standard. Improvements
suggested. To meet the J100-10 Standard, the practitioner must use the ranking developed under
paragraph 4.2.5 to select the critical threat-asset pairs to be further analyzed (or alternately, to
treat them all as critical). The VSAT software does not rank the threat-asset pairs and does not
allow the user to remove threat-asset pairs anywhere during the analysis. This can result in a
very long list of threat-asset pairs for the user to evaluate for the remainder of the analysis.
Paragraph 4.3: Consequence Analysis. Once the critical threat-asset pairs are identified,
the worst reasonable consequences that can be caused by the specific threats on the assets are
defined.
4.3.1 Threat Scenario – No gap identified. To meet the J100-10 Standard, the software
must allow the practitioner to identify the worst reasonable consequences that can be caused by
specific threats on specific assets for each threat-asset pair. The VSAT software does this.
to the owner/operator, financial loss to the community and duration and severity of service denial
for the affected customers of the utility. These include a single-point estimate, a single indicator
(a bin value), and a range. The VSAT software utilizes the Water Health Economic Analysis
Tool (WHEAT) in addition to single-point estimates, single indicators, and ranges to measure
each of the different types of consequences. However, it does not estimate consequences for the
affected customers.
4.3.3 Estimate Consequences (other) – Gap identified. To meet the Standard, the
software would need to include the ability to add additional consequences, if desired. The VSAT
software only provides fields for four consequence types (fatalities, injuries, economic cost of
owner, and economic cost to region).
performing the consequence analysis. The VSAT software relies on the WHEAT tool to
determine the consequences. VSAT and WHEAT outputs are exported to an Excel table where
the software’s assumptions are documented.
4.3.5 Record Consequence – No gap identified. To meet the Standard, the software must
provide both consequence ranges and a field for the practitioner to insert a point estimate. In the
VSAT software, the practitioner can enter a point estimate or select from the predefined J100-10
Standard bins.
Paragraph 4.4: Vulnerability Analysis. A key component of the Standard risk
assessment is the vulnerability analysis, which determines the likelihood that a given threat or
hazard occurs.
to input pertinent details of utility/facility construction, systems, and layout. VSAT provides a
place to do this when identifying the asset.
4.4.2 Analyze Vulnerability – Gap identified. To meet the Standard, the software must
provide a field for the practitioner to analyze the vulnerabilities of each asset to estimate the
likelihood that, given the occurrence of a threat, the estimated consequences will result by
utilizing fault, event or failure trees, path analysis, vulnerability logic diagrams, computer
simulation methods, or expert judgment. The VSAT software allows the practitioner to choose
the detection level (certain, probable, possible, none), amount of delay (very, strong, limited, no
delay), and response speed (fast, variable, slow, none) relative to each threat-asset pair. The
software then uses these values to determine the vulnerability of each threat-asset pair. This
method is equivalent to expert judgment. As this feature was met by only one out of three
selection options identified in the J100-10 Standard, the researchers have identified this as a gap.
4.4.3 Document Method – No gap identified. To meet the Standard, the software must
have a field where the practitioner can define their methods of analyzing vulnerability (e.g. those
listed in paragraph 4.4.2). The VSAT software does this.
4.4.4 Record Estimates – Gap Identified. To meet the Standard, the software must have
a field where practitioners can record the vulnerabilities that have been calculated for each
critical asset. VSAT determines the likelihood of damage using countermeasure capability (very
high, high, moderate, and low). This method does not allow the practitioner to use point values.
Also, as stated in 4.4.2, the software only allows for one method of determining the vulnerability
of the threat-asset pairs.
Paragraph 4.5: Threat Analysis. In the Threat Analysis step, the practitioner determines
4.5.1 Malevolent Threats –Gap Identified. To meet the Standard, the software must
include malevolent threat likelihood calculations using proxy measures, best estimates, or
conditional assessments. The VSAT software allows the practitioner the choice to use Best
Estimate or Conditional Assessment at the start of the assessment. If Best Estimate is chosen,
then the software only allows the practitioner to determine the threat likelihood of each threat
and record them as very high, high, moderate, or low likelihoods (basically a best estimate). It
does not allow the practitioner to use proxy measures or conditional assessments. As this feature
was met by only two out of three requirements in the J100-10 Standard, the researchers have
identified this as a gap.
4.5.2 Natural Hazards – Meets J100-10 Standard. Improvements suggested. To meet
the Standard, the software must include natural hazards threat likelihood calculations. It must
also allow the practitioner to assign the likelihood or frequency of a natural hazard to the asset,
based on historical data that may be provided by the software via maps, data, or links to
reference materials. The software must then calculate the risk of each natural hazard and sum
them to determine the overall risk due to natural hazards. VSAT allows the practitioner to assign
the likelihood (or frequency) of a natural hazard to the asset based on historical data for
predefined natural hazards of floods, tornados, hurricanes, earthquakes, fire, snowstorm, and
windstorm. Although it is missing historical data for wildfires, ice storms, snow storms, and
other natural hazards, it does allow the user to manually input this information. The software
does not calculate the overall risk due to natural hazards.
4.5.3 Dependency and Proximity Hazards – Gaps identified. To meet the Standard, the
software must include dependency and proximity hazards threat likelihood calculations. It must
also include historical data on dependency and proximity hazards to determine the likelihood that
the threats will occur to the assets. The investigators were not able to find a field to input these
values.
4.5.4 Record Estimates – No gaps identified. To meet the J100-10 Standard, the
software must allow the practitioner to determine the likelihood of each specific threat occurring
to each specific asset and record this estimate, along with the method and reasoning for the
estimate. The VSAT software does this.
Gaps identified. The VSAT software does not include Resilience Analysis. This is a
significant departure from the Standard. See paragraphs 4.6.1 through 4.6.3 below for further
explanation.
4.6.1 Calculate Risk – Gaps identified. To meet the Standard, the software must allow
the practitioner to calculate the risk associated with each threat-asset pair based on the
consequence, vulnerability, and threat likelihood values selected during earlier steps. The VSAT
software does not calculate the overall risk to the utility, nor does it calculate the risk to each
threat-asset pair using the J100-10 Standard formula (Risk = Consequences x Vulnerability x
Threat Likelihood). Instead of calculating risk, VSAT calculates, displays, and generates data
and written reports of each threat-asset pair’s consequences, vulnerability, and threat likelihood.
severity of service denial (in gallons per day), in order to determine the resilience of each threatasset pair.
4.6.2.1 Operational Resilience Asset Resilience Metric – Gap identified. To meet the
Standard, the software must calculate the asset’s resilience.
4.6.2.2 Owner’s Economic Resilience - Gap identified. To meet the Standard, the
software must calculate the owner’s economic resilience.
4.6.2.3 Community Economic Resilience - Gap identified. To meet the Standard, the
software must calculate the community’s economic resilience.
4.6.3 Record Risk and Resilience Estimates – Gap identified. To meet the Standard, the
software must calculate the overall risk to the utility using the J100-10 Standard risk equation;
the risk to each specific threat-asset pair; and the different types of resilience as defined in
paragraphs 4.6.2.1 through 4.6.2.3. The VSAT software does not calculate or record the overall
or specific risk or resiliencies.
threat-asset pair has been determined, the utility continues with the process by deciding whether
actions are needed to enhance all-hazards security or resilience or both.
4.7.1 Decide – Gap identified. To meet the J100-10 Standard, the software must allow
the practitioner to decide what risk and resilience levels are acceptable. The VSAT software
does not.
4.7.2 Define – No gap identified. To meet the J100-10 Standard, the software must allow
the practitioner to define new countermeasures and mitigation/resilience options to reduce
unacceptable risk to specific threat-asset pairs. The VSAT software enables practitioners to
make improvements to their security by adding new packages of possible countermeasures and
then performing an improvement analysis of the threat-asset pairs. Practitioners are also able to
create their own countermeasure packages.
4.7.3 Estimate – No gap identified. To meet the J100-10 Standard, the software must
allow the practitioner to estimate the costs of the new countermeasures and mitigation/resilience
options. VSAT does this.
4.7.4 Assess – Gap identified. To meet the J100-10 Standard, the software must allow
the practitioner to assess the options by analyzing the facility or asset under the assumption that a
specific countermeasure or mitigation/resilience option has been implemented. Once the
analysis has been performed for each threat-asset pair with the countermeasures in place, VSAT
does not assess the improved risk levels. Instead, it again displays the consequence, vulnerability,
and threat likelihood ratings for each threat-asset pair and compares the improved ratings to the
baseline ratings.
An additional issue is that the software also calculates “risk reduction units.” The
investigators felt that the explanation as to how these units are calculated is confusing. In
addition, these units can only be used for comparison purposes inside VSAT, as “risk reduction
units” are not risk but a proxy approach for comparing packages of countermeasures. An overall
improved risk and resilience and the individual resilience of each threat-asset pair is never
calculated for the improved assessment.
4.7.5 Identify – Meets Standard. Improvements suggested. To meet the J100-10
Standard, the software must allow the practitioner to identify those options that have benefits to
multiple threat-asset pairs. Although the practitioner can create multiple packages of
countermeasures to compare and then determine which ones benefit the utility the most, the
practitioner cannot compare the reduced risk values but only the new consequence, vulnerability,
and threat likelihood ratings.
4.7.6 Calculate – Gap identified. To meet the Standard, the software must allow the
practitioner to calculate the net benefits and benefit-cost ratios of the selected countermeasures.
VSAT calculates the annualized and capital costs of each countermeasure, as well as packages of
countermeasures and compares them to the “risk reduction units” of the other countermeasures
and packages. The software does not calculate the net benefits and benefit-cost ratios.
4.7.7 Review & Rank – Meets Standard. Improvements suggested. To meet the
Standard, the software must allow the practitioner to review and rank the selected
countermeasures in order to determine which ones will be most effective in reducing the utility’s
risk. Although VSAT provides a “risk reduction unit” for each package, the means by which it
calculates the “risk reduction unit” is difficult to understand. The only explanation that the
investigators found was in the VSAT Methodology Guide which states, “The risk reduction is
quantified with a series of dimensionless numbers, which can be placed in context only when
reflecting a movement between cells.”
Summary. In summary, the VSAT software is a user-friendly risk assessment tool that
calculates consequence, vulnerability, and threat likelihood; integrates the WHEAT tool; and
utilizes many of the J100-10 Standard tables and definitions for threat.
However, the analysis showed that there are deviations between the VSAT software and
the J100-10 Standard, the largest of which are as follows:
 Paragraph 4.4.2 Analyze Vulnerability - Lack of availability to use fault, event or
failure trees, path analysis, vulnerability logic diagrams, or computer simulation
methods to determine vulnerability. VSAT only uses a form of expert judgment.
 Paragraph 4.5.1 Malevolent Threats – The software only allows the practitioner to
determine the threat likelihood of each threat and record them as very high, high,
moderate, or low, which is basically a best estimate. The software does not allow
the practitioner to use proxy measures or conditional assessments.
 Paragraph 4.5.3 Dependency and Proximity Hazards – The software does not
include dependency or proximity hazards.
 Paragraph 4.6.1 Calculate Risk – The software does not calculate risk directly but
instead uses a proxy measure, the calculation and manipulation of which is
unclear.
 Paragraph 4.6.2 Calculate Resilience – the software does not address the issue of
resiliency.
Automated Risk Assessment Methodology for Water and Wastewater Utilities (ARAM-W™)
As stated in the ARAM-W™ User’s Manual, the “Automated Risk Assessment
Methodology for Water and Wastewater Utilities (ARAM-W™) software is a RAMCAP (2007)
compliant risk assessment application for water, wastewater, and combined utilities.” While the
scope of this project did not include confirming whether the software is compliant with
RAMCAP (2007), the analysis showed ARAM-W™ did not meet all of the features of J100-10
Standard.
The ARAM-W™ software follows the Risk Assessment Methodology for Water and
Wastewater Utilities (RAM-W™) steps, which in turn are roughly equivalent to the J100-10
Standard as outlined in Table 2.1 below.
Table 2.1: Comparison of J100-10 Standard and ARAM-W™ Steps
J100-10 Standard
ARAM-W™
Paragraph 4.1 Asset Characterization
Section 1 & 2. Planning & Facility Characterization
Paragraph 4.2 Threat Characterization
Section 4.
Threat Assessment
Paragraph 4.3 Consequence Analysis
Section 3.
Consequence Assessment
Paragraph 4.4 Vulnerability Analysis
Section 5.
Vulnerability Analysis
Paragraph 4.5 Threat Analysis
Paragraph 4.6 Risk/Resilience Analysis
Section 6.
Risk Analysis
Paragraph 4.7 Risk/Resilience Management Section 7.
Risk Management & Reduction
However, while the J100-10 Standard states that the steps can be performed out of order,
trying to ensure that the Standard is being followed while using ARAM-W™ can be quite
confusing as either ARAM-W™ or the Standard need to be executed out of order. There are also
instances where the ARAM-W™ steps account for multiple parts of the J100-10 Standard. For
consistency, the analysis below follows the order of J100-10 Standard steps.
4.1.1 Mission. – No gap identified. The software meets the requirement of the J100-10
Standard to identify the utility’s mission. The ARAM-W™ software requires the practitioner to
input the utility’s missions and rank them.
4.1.2 Critical Assets. – No gap identified. To meet the Standard, the software practitioner
must create a list of all of the utility’s assets and select those that are critical. The software
requires that the practitioner select the undesired events that may impact the facility and
associate them with specific assets.
4.1.3 Supporting Infrastructures – No gap identified.
To meet the Standard, the
software must allow for critical internal or external supporting infrastructures such as financial
records, legal documents, planning documents, mutual aid agreements, etc., to be identified. The
ARAM-W™ software meets the Standard by requiring the practitioner to select infrastructure
categories when defining the asset type.
4.1.4 Countermeasures and mitigation measures/features – No gap identified. The
ARAM-W™ software meets the requirement by having the practitioner include
countermeasures, descriptions, and/or details during the creation of the Adversary Sequence
Diagrams.
4.1.5 Consequence metrics – Meets J100-10 Standard. Improvements suggested. To
meet the Standard, the software must allow the practitioner to estimate the worst reasonable
consequences for each asset without regard to the threat. These consequence metrics include
potential for fatalities, serious injuries, major economic loss to facility or community, impacts to
the environment, loss of public confidence, and inhibiting effective function of national defense
or civilian government. The ARAM-W™ software meets the Standard by providing a
“Consequence Matrix” for the practitioner to define the consequence metrics for the utility. This
allows the practitioner to select most of the consequences identified in the J100-10 Standard and
to add additional ones. To meet the full intent of the J100-10 Standard, the software could also
include impacts to the environment as a standard consequence.
4.1.6 Prioritize Assets - No gap identified. To meet the Standard, the software must
allow the practitioner to identify assets as critical and to rank them using categories of high,
medium and low. The ARAM-W™ software meets the Standard by having the practitioner rank
the facilities against each other. The facilities are then paired with the missions. The “Mission
Score” is multiplied by the “Facility Score” to determine the “Total Mission Score” of each
facility. These “Total Mission Scores” are then used to rank the facilities. The practitioners can
then select the facilities they consider critical and continue the assessment with just those
facilities. Next, using a predefined fault tree for water/wastewater treatment, collection or
distribution, source, storage, or a generic water utility, the practitioner can use or alter predefined
fault trees and select the most reasonable undesired events that will have serious consequences to
their facilities. The development of the facility-specific fault tree shows the practitioner the
possible paths that could cause the undesired events and allows the practitioner to drill down to
the asset level. Those assets are then used for the remainder of the assessment.
Paragraph 4.2: Threat Characterization. Once the critical assets have been determined,
the practitioner must define the threats that could potentially impact each critical asset.
4.2.2 Natural Hazards Threat Characterization – Meets J100-10 Standard.
Improvements suggested. To meet the Standard, when selecting the natural hazard threats the
software must allow the practitioner to define the range of magnitudes, from the smallest
magnitude that could cause serious harm to the largest reasonable magnitude, of each natural
hazard. When selecting natural hazards, the ARAM-W™ software allows the practitioner to
select from four predefined natural hazard threats (earthquakes, hurricanes, tornadoes, and
floods). Although these are the only natural hazard threats required to meet the J100-10
Standard, to meet the full intent of the Standard the software could also include wild fires, ice
storms, and the option to add additional natural threats, if desired.
Improvements suggested. The Standard requires the practitioner to define any dependency
threats due to interruptions in utilities, suppliers, employees, customers, and transportation, as
well as threats due to the close proximity of the utility to dangerous neighboring sites. Although
the ARAM-W™ software allows practitioners to manually add these threats to their lists of
malevolent threats, the J100-10 Standard reference table is not provided and might be helpful.
4.2.4 Threat-Asset Pairs – No gap identified. The Standard requires the practitioner to be
able to assign potential threats to each asset. The software meets the Standard by having the
practitioner assign threats to each undesired event/asset location pair, creating undesired
event/asset location/threat pairs (triplets).
4.2.5 Threat-Asset Pair Ranking – Gap identified. To meet the Standard, once all of the
potential threats have been assigned to each asset (thus creating threat-asset pairs), the
practitioner must then rank them (using professional judgment) in order from the worst to the
least resulting consequences. The ARAM-W™ software does not allow the practitioner to rate
or rank the pairs prior to determining vulnerability or threat likelihoods.
4.2.6 Critical Threat-Asset Pairs – Meets J100-10 Standard. Improvements suggested.
To meet the Standard, the practitioner must use the ranking developed under paragraph 4.2.5 to
select the critical threat-asset pairs to be further analyzed (or alternately to treat them all as
critical). The ARAM-W™ software does not allow the practitioner to choose critical threat-asset
pairs and does not allow the practitioner to remove pairs from the list. Allowing the practitioner
to remove threat-asset pairs would be a helpful improvement for practitioners with large numbers
of threat-asset pairs that are not critical.
Paragraph 4.3:
Consequence Analysis. Once the critical threat-asset pairs are
identified, the worst reasonable consequences that can be caused by the specific threats on the
assets are defined.
4.3.1 Threat Scenario – Gap identified. To meet the Standard, the software must allow
the practitioner to identify the worst reasonable consequences that can be caused by specific
threats on specific assets for each threat-asset pair. Although the consequences for the loss of
critical assets and the resulting undesired event occurring were previously determined, these are
not evaluated for each specific threat on the asset.
to the owner/operator, financial loss to the community, and duration and severity of service
denial for the affected customers of the utility. These include a single-point estimate, a single
indicator (a bin value) and a range. The ARAM-W™ software partially meets the Standard. The
software references the “RAMCAP Consequence Table” that lists units of consequence for
consequences. However, ARAM-W™ groups the values into more granular units (very low, low,
medium, high and very high) than the J100-10 Standard does. Thus, this portion of the J100-10
Standard is not fully met because the software does not use the same bin ranges.
4.3.3 Estimate Consequences (other) – No gap identified. To meet the Standard, the
software must allow the practitioner to add additional consequences, if desired. The ARAMW™ software meets the Standard by allowing practitioners to add any additional consequence
metrics that they feel apply to the loss of a critical asset caused by an undesired event.
performing the consequence analysis. The software meets the Standard by allowing practitioners
to document their assumptions in a text field.
4.3.5 Record Consequence – No gap identified. To meet the Standard the software must
provide both a field for consequence ranges and a field for the practitioner to insert a point
estimate. The software meets the Standard by allowing the practitioner to select consequence
ranges and by also allowing the practitioner to enter a consequence point estimate.
Paragraph 4.4: Vulnerability Analysis. A key component of the J100-10 Standard risk
assessment is the vulnerability analysis, which determines the likelihood that a given malevolent
threat or natural hazard threat occurs.
to input pertinent details of utility/facility construction, systems, and layout. The software meets
the Standard by having a Site Survey portion to enter and save data worksheets.
4.4.2 Analyze Vulnerability – Meets J100-10 Standard. Improvements suggested. To
meet the Standard, the software must provide a field for the practitioner to analyze the
vulnerabilities of each asset to estimate the likelihood that, given the occurrence of a threat, the
estimated consequences will result. The practitioner does this by utilizing fault, event or failure
trees, path analysis, vulnerability logic diagrams, computer simulation methods, or expert
judgment. The ARAM-W™ software meets the Standard by allowing the practitioner to create
Adversary Sequence Diagrams (ASD) for each of the undesired event/asset location/threat pairs.
The software could better meet the Standard by providing a means to use other methods to
analyze vulnerability.
4.4.3 Document Method – No gap identified. To meet the Standard, the software must
have a field where practitioners can define their methods of analyzing vulnerability (i.e. those
listed in paragraph 4.4.2). The ARAM-W™ software meets the Standard by allowing the
practitioner to type in a justification for the vulnerability estimate to document the method used.
4.4.4 Record Estimates – No gap identified. To meet the Standard, the software must
have a field where the practitioner can record the vulnerabilities calculated for each critical asset.
The ARAM-W™ software meets the Standard by allowing the practitioner to use either the
ASD-based analysis or a User Estimate to estimate vulnerabilities.
Paragraph 4.5: Threat Analysis. In the Threat Analysis step the practitioner determines
4.5.1 Malevolent Threats - Meets J100-10 Standard. Improvements suggested. To
meet the Standard, the software must include malevolent threat likelihood calculations using
proxy measures, best estimates, or conditional assessments. The ARAM-W™ software meets
the Standard minimum by allowing the practitioner to determine the threat likelihood using
conditional, expert judgment, or a questionnaire method. For the conditional method, the
likelihood value is automatically considered to be 100% based on the assumption that the attack
will in fact occur. For the expert judgment method, the practitioner can input a likelihood of
high, medium, or low from a drop-down menu. For the questionnaire method, the likelihood
value is determined based on the responses to approximately 10 questions. The questions align
with the threat factors outlined in the RAMCAP Framework document for estimating the
likelihood of attack, and include initial considerations of capability, history, current interest,
current surveillance, documented threats, potential consequences, ideology, and ease of attack.
The software could better meet the intent of the J100-10 Standard by also including a proxy
indicator method for determining the likelihood of a malevolent threat.
4.5.2 Natural Hazards – Meets J100-10 Standard. Improvements suggested. To meet
the Standard, the software must include natural hazards threat likelihood calculations. It must
also allow the practitioner to assign the likelihood or frequency of a natural hazard to the asset,
based on historical data that may be provided by the software via maps, data, or links to
reference materials. The software must then calculate the risk of each natural hazard and sum
them to determine the overall risk due to natural hazards. The ARAM-W™ software allows the
practitioner to assign the frequency to natural hazards, but it does not sum them to determine an
overall risk due to natural hazards. Also, although the software does allow the practitioner to
define the likelihood of the four mandatory natural hazard threats, it would better meet the intent
of the Standard if it also allowed the practitioner to assign and define other natural hazard
threats, such as wildfires and ice storms, and their probabilities.
4.5.3 Dependency and Proximity Hazards – Gap identified. To meet the Standard, the
software must include dependency and proximity hazards threat likelihood calculations. It must
also include historical data on dependency and proximity hazards to determine the likelihood that
the threats will occur to the assets. The investigators were unable to locate a field to identify
dependency and proximity hazards threat likelihoods.
4.5.4 Record Estimates – Gap identified. To meet the Standard, the software must allow
the practitioner to determine the likelihood of each specific threat occurring to each specific asset
and record this estimate, along with the method and reasoning for the estimate. The ARAMW™ software partially meets the Standard by allowing the practitioner to determine the
likelihood of the malevolent and natural threats, but it does not seem to allow the practitioner to
address dependency and proximity hazards.
4.6.1 Calculate Risk – Meets J100-10 Standard. Improvements suggested. To meet
the Standard, the software must allow the practitioner to calculate the risk associated with each
threat-asset pair based on the consequence, vulnerability, and threat likelihood values selected
during earlier steps. The ARAM-W™ software meets the Standard by calculating risk for each
individual undesired event/asset location/threat pair and displays the results as no risk, low,
medium, high, or very high. Unfortunately, the software does not display the results of
consequence, vulnerability, threat likelihood, and risk as numerical quantities but as values of
very low, low, medium, high and very high. Thus, checking the risk calculations is not possible.
severity of service denial (in gallons per day), in order to determine the resilience of each threatasset pair. The ARAM-W™ software does not execute any resilience calculations as outlined in
the following paragraphs.
4.6.2.1 Asset Resilience Metric – Gap identified. To meet the Standard, the software
must calculate the asset’s resilience.
4.6.2.2 Owner’s Economic Resilience - Gap identified. To meet the J100-10 Standard,
the software must calculate the owner’s economic resilience.
4.6.2.3 Community Economic Resilience - Gap identified. To meet the J100-10
Standard, the software must calculate the community’s economic resilience.
4.6.3 Record Risk and Resilience Estimates – Gap identified. To meet the J100-10
Standard, the software must calculate the overall risk to the utility using the J100-10 Standard
risk equation; the risk to each specific threat-asset pair; and the different types of resilience as
defined in paragraphs 4.6.2.1 through 4.6.2.3.
threat-asset pair has been determined, the utility continues with the process by deciding whether
mitigation actions are needed to enhance all-hazards security or resilience or both. As the
ARAM-W™ software does not address resilience, most of the sub-features under Paragraph 4.7
are not met.
4.7.1 Decide – Gap identified. To meet the J100-10 Standard, the software must allow
the practitioner to decide what risk and resilience levels are acceptable. The ARAM-W™
software partially meets the Standard in that it includes a field to determine if the risk for each
undesired event/asset location/threat pair is acceptable. However, the software does not include
resilience in this decision.
4.7.2 Define – Gap identified. To meet the J100-10 Standard, the software must allow
the practitioner to define new countermeasures and mitigation/resilience options to reduce
unacceptable risk to specific threat-asset pairs. The ARAM-W™ software enables the
practitioner to create potential upgrade packages to mitigate or reduce the risk to facilities, but it
does not have options to increase resiliency.
4.7.3 Estimate – No gap identified. To meet the J100-10 Standard, the software must
allow the practitioner to estimate the costs of the new countermeasures and mitigation/resilience
options. The ARAM-W™ software allows the practitioner to enter cost estimates and impacts
on operations, schedule and public opinion for each upgrade package created.
4.7.4 Assess – Gap identified. To meet the J100-10 Standard, the software must allow
the practitioner to assess the options by analyzing the facility or asset under the assumption that
the option has been implemented. The ARAM-W™ software calculates the new risk,
vulnerability, threat potential and consequence of each threat-asset pair for each upgrade
package, but it does not include resiliency in the assessment.
4.7.5 Identify – Gap identified. To meet the J100-10 Standard, the software must allow
the practitioner to identify those options that have benefits to multiple threat-asset pairs. The
ARAM-W™ software displays a report of the baseline data, including risk, vulnerability,
potential/frequency, and consequence for each threat-asset pair. The software also displays the
upgrade packages that apply to the highest risk threat-asset pair, but it does not determine which
improvements are beneficial to multiple threat-asset pairs.
4.7.6 Calculate – Gap identified. To meet the J100-10 Standard, the software must
allow the practitioner to calculate the net benefits and benefit-cost ratios of the selected
countermeasures. The ARAM-W™ software does not do this.
4.7.7 Review & Rank – Gap identified. To meet the J100-10 Standard, the software
must allow the practitioner to review the selected countermeasures and rank them in order to
determine which ones will be most effective in reducing the utility’s risk. The ARAM-W™
software does not do this.
Summary. In summary, the ARAM-W™ software is a user-friendly risk assessment tool
that utilizes many of the J100-10 Standard tables and definitions for threats and consequences. It
also has multiple points in the process where existing information can be saved to create a
comprehensive assessment of the utility. However, the fact that the software does not follow the
order of the Standard makes utilizing the software and the Standard together a challenge.
The analysis indicated that there are some inconsistencies between the ARAM-W™
software and the J100-10 Standard, the largest of which are as follows:
 Paragraph 4.3.1 Threat Scenario –The consequences for the loss of critical assets
are not evaluated for each specific threat on the asset.
 Paragraph 4.5.3 Dependency and Proximity Hazards – The software does not
include dependency or proximity hazards.
 Paragraph 4.6.2 Calculate Resilience – The software does not address the issue of
resiliency.
Chapter 3: Recommendations | 25
CHAPTER 3: RECOMMENDATIONS
INTRODUCTION
The recommendations included in this chapter were developed based on the identified
discrepancies between the J100-10 Standard and the software packages as outlined in Chapter 2.
Recommendations to address these discrepancies were developed for the three software packages
(SEMS, VSAT and ARAM-W™) and were determined by analyzing – to the extent possible –
the databases used by each package. In addition, a range of the number of hours needed to
implement the recommendations was estimated. These estimates were made assuming that they
would be implemented by experienced IT personnel.
METHOD
In order to develop recommendations to address the identified gaps in the three software
packages, the researchers first attempted to discover and understand each package’s underlying
database structure. This was done in order to determine how easily those structures could be
extended or modified to close each of the gaps found during the assessment. While SEMS and
VSAT use standard database formats, and the researchers were able to view their contents,
ARAM-W™ uses a proprietary, encrypted format that halted any further inspection on the part
of the investigators. This prevented detailed exploration of the capabilities of the software. Thus,
a more accurate estimate as to how changes could be made to SEMS and VSAT was possible.
The gaps found between the software and the requirements of the J100-10 Standard were
placed into four broad categories for the purposes of calculating the amount of development
effort that might be required to bring the software packages in line with the Standard. The four
categories of software upgrades were small, medium, large, and unknown. A general description
of each follows:




Small upgrades were generally minor issues, often a modification of the user interface in
some way. Because these changes are simple to implement, it was estimated that they should
take one to two days to resolve.
Medium upgrades were either improvements to existing functionality or the addition of
missing features; in either case, it was estimated that it would require 16 to 24 hours to
correct.
Large upgrades might require entire pieces of new functionality. However, these fixes
would also require time-intensive modifications that may take one to three weeks of work to
complete.
Unknown upgrades were those where there was some impediment in the investigator’s
analysis; thus obtaining an accurate estimate of effort needed to correct the gap was not
possible. Challenges to further analysis by the investigators included:
1. inability to determine database structure and thus the ease or difficulty of the changes
required,
2. unknown calculation details, and
3. inability to determine how the software operates when given incorrect input or when
it encounters extreme and likely incorrect results.
25
Additionally, upgrades of unknown size were further divided, where possible, based on
assumptions regarding the developers’ use of standard programming and software design
practices. This allowed many of the upgrades of unknown effort to be estimated with further
precision – small, medium, and large. While there is no way to ensure that these effort
descriptors are correct without further examination of the software structure and code, they
represent an educated guess as to the amount of effort required, assuming that the developers
followed industry best practices.
Based on the amount of effort estimated for each recommendation, an estimated number
of hours was determined to bring each lacking feature into compliance with the Standard.
SEMS
In general, SEMS has more gaps to address than either VSAT or ARAM-W™ and would
most likely require more time and resources to bring it into compliance with the J100-10
Standard. The required total number of upgrades (41) means that SEMS meets approximately
48% of the 79 Standard features (Table 3.1).
Size of
Upgrade
Small
Medium
Large
Unknown
(likely small)
Unknown
(likely medium)
Unknown
(likely large)
Unknown
(unknown)
TOTAL
Table 3.1: SEMS Summary Table
Number of Required
Range of Estimated
Upgrades
Hours
10
8 – 16
9
16 – 40
1
40 - 120
5
8 – 16
Estimated Hours of
Required Upgrades
80 – 160
144 – 360
40 – 120
40 – 80
7
16 – 40
112 – 280
2
40 - 120
80 – 240
7
∞
∞
41
1,240 - ∞
Summary of large upgrades:
1. Providing the ability to calculate the resilience of assets.
The most cost-effective upgrade (although it is a large change) might be changing SEMS
to allow for repeated analysis runs with different sets of countermeasures without starting the
analysis from the beginning. In the investigator’s opinion this would greatly improve usability.
VSAT
VSAT has fewer issues than SEMS, and because of the availability of the database
structure, the number of upgrades of indeterminate effort is the lowest of the three software
packages. In addition only 27 upgrades are needed to meet the 79 Standard features; or in other
words, VSAT is 66% compliant (Table 3.2). However, the estimated number of changes that
will likely be large should also be taken into consideration. In particular, attention should be
given to the fact that VSAT does not absolutely quantify the amount of risk to which critical
assets are exposed.
Size of
Upgrade
Small
Medium
Large
Unknown (likely
small)
Unknown (likely
medium)
Unknown (likely
large)
Unknown
(unknown)
TOTAL
Table 3.2: VSAT Summary Table
Number of Required
Range of
Upgrades
Estimated Hours
1
8 – 16
12
16 – 40
1
40 - 120
1
8 – 16
Estimated Hours of
Required Upgrades
8 – 16
192 – 480
40 – 120
8 – 16
4
16 – 40
64 – 160
3
40 - 120
120 – 360
5
∞
∞
27
1,152 - ∞
1. Calculating resiliency.
In the opinion of the investigators, the most cost-effective upgrade would be the
calculation of the net benefits of each package of countermeasures. If this relatively simple step
were done, quite a few other gaps would be resolved (in particular, the calculation of the costbenefit ratio of each package of countermeasures and the determination of the most effective
countermeasures).
ARAM-W™
ARAM-W™ has a larger proportion of issues of “unknown” size, due to the fact that its
database structure was not available to the investigators. However, this is offset by the fact that it
has the least number of overall gaps compared to the other software packages. In particular, only
25 upgrades are needed to meet the 79 Standard features; or in other words, ARAM-W™ is 68%
compliant (Table 3.3). Of the 8 recommended upgrades where the amount of effort to resolve
them is unknown, half can be resolved with the implementation of resilience calculations. In
general, it appears that ARAM-W™ meets the greatest part of the Standard and thus would
require the least amount of corrective actions to completely satisfy the requirements of the
Standard.
Size of
Upgrade
Small
Medium
Large
Unknown
(likely small)
Unknown
(likely
medium)
Unknown
(likely large)
Unknown
(unknown)
TOTAL
Table 3.3: ARAM-WTM Summary Table
Number of
Range of
Estimated Hours of
Required Upgrades Estimated Hours
Required Upgrades
5
8 – 16
40 – 80
4
16 – 40
64 – 160
2
40 - 120
80 – 240
2
8 – 16
16 – 32
4
16 – 40
64 – 160
0
40 - 120
0
8
∞
∞
25
672 - ∞
1. Incorporating the ability to account for dependency threats to the assets, as well as
including the proxy method of assessing the risk level to each facility.
2. Calculating resiliency.
The most cost-effective single upgrade might be the addition of resilience calculations.
Usability
It should be noted that while the primary focus of the analysis was the functionality of the
three software packages, other important considerations such as the software interface and
process flow could also be important when selecting a software package.
The user interfaces of the different software packages are quite different. The SEMS
RAMCAP assessment is just one component of the larger SEMS software system. Thus, its
menu-based navigation requires the user to constantly return to the main screen and review
multiple past screens to proceed with the assessment. The investigators were not able to find any
workaround for this.
On the other hand, the VSAT navigation is tab-based, which makes navigation between
components and steps in the analysis much more logical and intuitive. In contrast, ARAM-W™’s
navigation is a bit awkward, with entire tabs of the interface hidden from the practitioner until a
single check box is selected.
The flow through the different software packages also varies. SEMS and VSAT
encourage a forward flow of information from section to section. Each begins by asking for
facility information, defining assets, countermeasures, and potential threats. At this point, their
flows diverge: while SEMS performs a one-time analysis, VSAT allows for repeated iterations of
risk level analysis, followed by options for improvement. In contrast, ARAM-W™ starts the
analysis procedure by identifying harmful consequences and then works backwards from there to
determine the risk level. While both approaches have the potential to meet the Standard, the
forward flow of SEMS and VSAT are more intuitive and follow the flow of the Standard more
closely.
CONCLUSION
Of the three software packages tested for compliance with the J100-10 Standard, ARAMWTM appears to require the fewest upgrades, as it currently meets 68% of the features. VSAT
also addresses a high number of the Standard’s features (66%), while SEMS addresses the fewest
(48%). None of the software tools calculate resilience – the largest single inconsistency with the
J100-10 Standard, and one that accounts for a number of peripheral inconsistencies in each
package.
While the project investigators did their best to estimate the amount of effort needed to
upgrade each software package’s features to meet the Standard, the large number of “unknowns”
made this difficult. Focusing on addressing the discrepancy between the J100-10 Standard and
the software tools around the issue of resilience could quickly bring VSAT and/or ARAM-WTM
much closer to compliance with the Standard. In addition, clarifying with the software
developers the exact labor needed to address the larger unknown upgrades is also a logical next
step.
Appendix A: J100-10 Features | 31
APPENDIX A: J100-10 FEATURES
31
Table A.1 - J100-10 Standard Features Matrix
Feature Met
Reference Section Title
Mandatory Features
1
all-hazards risk and resilience
analysis of vulnerabilities to
man-made threats, natural
hazards, and dependencies and
proximity to hazardous sites
2.1
2.1.1
2.2
2.3
2.4
2.4.1
2.4.2
2.4.3
2.4.4
2.4.5
2.5
2.6
2.7
2.8
2.9
2.10
2.11
2.12
2.13
2.14
2.15
2.16
2.17
2.18
2.19
2.2
2.21
2.22
2.23
2.24
2.25
2.26
2.27
2.28
2.29
2.30
Scope
Definitions
Non-mandatory Features
Notes
scope must encompass the same
requirements
definitions must be the same when
provided
asset
critical asset
consequence
consequence mitigation
countermeasure
detect
deter
devalue
delay
respond
dependency
dependency hazard
event tree analysis
failure mode
fault tree analysis
frequency
hazard
incident
initiating event
likelihood
preparedness
probability
proximity hazard
response
reference threat
resilience
risk
risk analysis
risk management
scenario
system
threat
threat likelihood
vulnerability
vulnerability
assessment/vulnerability
analysis
vulnerability estimate
32
Yes
No
Some
Explanation
Feature Met
Reference Section
2.31
3
4
4.1
Title
Mission
Critical Assets
4.1.3
Supporting Infrastructures
4.1.5
4.1.5(.1)
4.1.5(.2)
4.1.5(.3)
4.1.5(.4)
4.1.5(.5)
4.1.5(.6)
4.1.6
4.2, A.4.2
Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
worst reasonable case
Notes
Bibliography
Requirements
Asset Characterization
4.1.1
4.1.2, A.4.1
4.1.4
Mandatory Features
-
identify the mission or critical
functions to determine which
assets perform or support the
mission
identify critical assets
identify critical internal and
external supporting
infrastructure
multiple fields for mission or
critical functions
multiple fields for critical assets
multiple fields for critical internal
or external supporting infrastructure
identify and document existing
Countermeasures and
protective countermeasures and refer to definition for items to
mitigation measures/features mitigation measures/features
list
estimate worst reasonable
consequences for each asset
without regard to the threat
Consequence metrics
potential for fatalities
Consequence metrics
serious injuries
Consequence metrics
major economic loss to facility
or community
Consequence metrics
impacts to the environment
Consequence metrics
loss of public confidence
Consequence metrics
inhibiting effective function of
national defense or civilian
government
Consequence metrics
Prioritize assets
Threat Characterization
Reference Threats
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Yes
prioritize critical assets using
estimated consequences
man-made hazards or accidents,
natural hazards, dependency
hazards; identify general and
specific threat scenarios to serve
as reference threats
listed as shall under body
preferred method
multiple fields for all existing
protective countermeasures and
mitigation measures/features
see below
Can group these into Human, $ and
other (i.e.. environmental) Can use
Hi, Med, Lo, V Hi, etc. field for
each consequence
field for each critical asset to assign
rank or reorder them in order of
consequence
-
field to describe the type of threat
to be considered
to be considered
to be considered
to be considered
Air attack
Land attack
Water attack
various magnitudes of attack
elements
33
No
Some
Explanation
Feature Met
Malevolent Threat
Characterization
4.2.1, Appx E
Malevolent Threat
Characterization
4.2.1, Appx E
Malevolent Threat
Characterization
4.2.1, Appx E
Malevolent Threat
Characterization
4.2.1, Appx E
Malevolent Threat
Characterization
4.2.1, Appx E
Malevolent Threat
Characterization
4.2.1, Appx E
Malevolent Threat
Characterization
4.2.1, Appx E
Malevolent Threat
Characterization
4.2.1, Appx E
Malevolent Threat
Characterization
4.2.1, Appx E
Natural Hazards Threat
4.2.2, Appx E & G Characterization
Dependency Hazards Threat
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
4.2.3, Appx G
Characterization
Mandatory Features
Notes
to be considered
to be considered
to be considered
to be considered
to be considered
to be considered
to be considered
to be considered
to be considered
Weapons types
equipment
tools
explosives
tactics
means of delivery/transport
number of adversaries
insiders
outsiders
hurricanes
floods
software must define the range of
magnitudes from the smallest that
would cause serious harm to the
largest reasonable case
tornadoes
earthquakes
wildfires
ice storms
should give the user the option of
including other threats
to be considered
to be considered
to be considered
to be considered
to be considered
to be considered
including other threats - not
included in comparisons
other
utilities
suppliers
employees
customers
transportation
proximity
other
34
Yes
No
Some
Explanation
Feature Met
4.2.4 & 4.2.5?
4.2.6
Threat-Asset Pairs
Critical Threat-Asset Pairs
Mandatory Features
Notes
check box or field to identify
selected critical threat-asset pairs
identifies the worst reasonable
consequences that can be caused
by the specific threats on the
assets as identified in 4.1
apply worst reasonable case
assumptions for each threat
scenario
Consequence Analysis
4.3.1, Appx B
Threat Scenario
4.3.2, Appx B
measured by 2 of the following:
natural units reported and
considered individually;
converted into a single,
summary economic value,
reported and considered as a
Estimate Consequences - loss single loss indicator; or in preof life to either employees or defined ranges represented by
the general public
the RAMCAP "bins" (Appx B)
4.3.2, Appx B
Estimate Consequences serious injury to either
employees or the general
public
single loss indicator; or in predefined ranges represented by
Estimate Consequences Financial loses to
owner/operator
single loss indicator; or in predefined ranges represented by
Yes
can use multiple approaches to
evaluate and rank pairs; matrix
using small, med., large or scales 110, etc. - software to have matrix or
another method of ranking pairs
evaluate and rank threat-asset
pairs
select critical threat-asset pairs
to be used going forward or use
all pairs
4.3, A.4.3
4.3.2, Appx B
-
-
optional single indicator - dollar
field for single point estimate or bin
equivalence of fatalities and
number
serious injuries in excess of
insurance
optional single indicator - dollar
equivalence of fatalities and
number
serious injuries in excess of
insurance
number
35
No
Some
Explanation
Feature Met
4.3.2, Appx B
4.3.2, Appx B
4.3.3
4.3.4
4.3.5
4.4, A.4.4
4.4.1
Mandatory Features
Notes
Estimate Consequences single loss indicator; or in preservice denial for the affected defined ranges represented by
customers
Estimate Consequences economic losses to society
and the general public
number
single loss indicator; or in predefined ranges represented by optional single indicator - value
the RAMCAP "bins" (Appx B) of a statistical life
Estimate Consequences other
if degradation in public
confidence, environmental
quality, ability of civilian or
military agencies to function,
etc. room for descriptive
analysis must be provided
Document assumptions
document specific assumptions
and procedures used for
performing the consequence
analysis, the worst reasonable
case assumptions and the results
of the consequence analysis
Record consequence
Review
field for single value
record the consequence values
using point estimates or ranges
additional consequences sociopolitical impacts, natural
security impacts, lost strategic
capability to cause harm or
including other consequences
output, detrimental effects on
brand value, public confidence,
psychological impacts, and
environmental degradation
should give users a space to save
this information possibly link files,
the documentation could include
maps and calculations - not
required
ranges in appx B
field for ranges or point estimates
-
review pertinent details of the
facility construction, systems
and layout; identify
vulnerabilities or weaknesses in
the protection system
fields to input pertinent details of
construction, systems and layout
36
Yes
No
Some
Explanation
Feature Met
4.4.2, A.4.4
Analyze Vulnerability
4.4.3
Document method
4.4.4, Appx B
Record Estimates
4.5, A.4.5
Threat Likelihood Analysis
4.5.1, Appx F
Malevolent Threats
F.3.1
Proxy Indicator - Node 1
F.3.2
F.3.3
F.3.4
F.3.5
F.3.6
F.3.7
4.5.2, G.2
4.5.2, G.3
4.5.2, G.4
4.5.2, G.5
Natural Hazards
Natural Hazards
Natural Hazards
Natural Hazards
Mandatory Features
analyze vulnerability of each
critical asset to estimate the
likelihood that, given the
occurrence of a threat, the
consequences result
document method and results of
the vulnerability analysis
may use fault or event tree
analysis, path analysis,
vulnerability logic diagrams,
computer simulation methods,
or expert judgment rules-ofthumb
Notes
Yes
-
field to document method used
use point estimates or
RAMCAP scales; if bins are
used, the midpoint is used for
Record the estimates from 4.4.2 the calculation
likelihood of malevolent event,
dependency/proximity hazard or
natural hazard
use proxy measure, best
estimate, or conditional
Proxy Measure (Appx F) is
assessment to determine
optional and preferred
field for point estimates, bin
number or ranges
-
field for threat estimate
may have a field for number of
Number of U.S. attacks per year attacks
Metro Region (RMS metro area
classes)
may have a field for likelihood
Target Type (RMS target type
analysis)
Proportion: Regional Number may have a field for likelihood
may have a field for ratio of
This Facility
capacity to metro area
This Threat-Asset Pair
(product of VxCxDetection)
calculated by multiplying each
Overall Proxy Likelihood
proxy indicator
software must have a field to enter
the risk of earthquakes for each
magnitude from historical records
earthquakes
hurricanes
tornadoes
Appx G is optional and provides
data to estimate the risk of each
natural hazard - risk calculated
by CxVxT; would be nice to
have look-up maps/tables for
each natural hazard
the risk of hurricanes for each
the risk of tornadoes from historical
records
the risk of floods for each
floods
37
No
Some
Explanation
Feature Met
4.5.2, Appx G
Mandatory Features
ice storms, extreme cold
weather, wildfires, avalanche,
tsunami, landslide, mud slide
Natural Hazards
software may have a field to enter
the risk of other natural hazards
other
4.5.2, Appx G
Natural Hazards
4.5.2, Appx G
Natural Hazards
4.5.3
Dependency and Proximity
Hazards
4.5.4
Record Estimates
4.6, A.4.6
estimates the owner's risk and
resilience and the community's
resilience relative to each threatRisk and Resilience Analysis asset pair
must have field for total natural
hazard risk
Risk
use local historical records for
frequency, severity and duration
of service denials
record the method used for
making the estimates and the
estimates themselves as single
point values or ranges
software must have a field for
likelihood
should have room for methods to be
saved
-
4.6.2, Appx H
for each threat-asset pair
calculate risk: CxVxT=R
use either threat-asset pair
Calculate the current level of resilience metric or holistic
resilience
approach in Appx H
4.6.2.1
Operational Resilience Index
(ORI) calculated by choosing
values from Table H-1 then
calculating ORI by multiplying
the indicator value by the weight
and adding all values (should
Duration x severity x
have fields for the value and
Operational Resilience Asset vulnerability x threat likelihood
weight or by pick box)
Resilience Metric
= asset resilience metric
4.6.1
4.6.2.2
4.6.2.3
Calculate Risk
Notes
software may have a field to enter
the risk of ice storms, extreme cold
tsunami, landslide, mud slide from
historical records
use midpoint of ranges from
Appx B
software must calculate risk using
the numbers input previously for C,
V and T for each threat-asset pair
-
Appx H is non-mandatory
Financial Resilience Index (FRI)
lost revenue due to the threatcalculated by choosing values
asset pair (asset resilience x unit from Table H-2 then calculating
Owner's Economic Resilience price)
FRI by multiplying the indicator
value by the weight and adding
all values (should have fields for
the value and weight or by pick
Community Economic
lost economic activity to the
box)
Resilience
community served by the utility
38
software must calculate asset
resilience metric using duration and
severity from 4.3, vulnerability
from 4.4 and threat likelihood from
4.5
software must calculate asset
resilience x unit price (have field
for unit price of asset)
software must have field for lost
economic activity to the community
(same as 4.3)
Yes
No
Some
Explanation
Feature Met
4.6.3
4.7, A.4.7
4.7.1, A.4.7.1
4.7.2, A.4.7.2
4.7.3, A.4.7.3
4.7.4, A.4.7.4
Mandatory Features
Record Risk and Resilience
Estimates
Risk and Resilience
Management
Notes
Utility Resilience Index software should calculate URI
using the values of ORI and FRI
and the weights given in Table
H-1 & H-2 (URI=ORI x w1 + software must have fields for threatasset pair resilience
FRI x w2)
-
Decide
decide what risk and resilience
levels are acceptable
field for acceptable risk and
resilience level
Define
define countermeasure and
mitigation/resilience options for
the threat-asset pairs that are not
acceptable. Include devalue,
deter, detect, delay and
response; consequence
reductions, resilience
enhancements
fields for each threat asset-pair
Estimate
estimate investment and
operating costs for each option;
include regular maintenance and
periodic overhaul; adjust to
present value
fields for costs for each option
above
Assess
4.7.5, A.4.7.5
4.7.6, A.4.7.6
Identify
Calculate
4.7.7, A.4.7.7
Calculate
4.7.9, A.4.7.8
Review & Rank
revisit 4.3 through 4.6 to
reestimate the risk and resilience
levels as if the option was
implemented; calculate the
estimated benefits of the option
identify the options that have
benefits that apply to multiple
threat-asset pairs
calculate the net benefits
field for the new value of risk
a way to highlight or mark the
options
field for value
calculate the benefit-cost ratio
rank the most cost effective
measures to implement
field for value
a field for rank
39
Yes
No
Some
Explanation
Appendix B: J100-10 Gap Analyses | 33
APPENDIX B: J100-10 GAP ANALYSES
41
Table %.1 - J100-10 Standard Gap Analysis Matrix - SEMS
Reference Section No.
1
2.1, Appx C
2.1.1
2.2
2.3
2.4
2.4.1
2.4.2
2.4.3
2.4.4
2.4.5
2.5
2.6
2.7
2.8
2.9
2.10
2.11
2.12
2.13
2.14
2.15
2.16
2.17
2.18
2.19
2.2
2.21
2.22
2.23
2.24
2.25
2.26
2.27
2.28
2.29
2.30
2.31
3, Appx D
4
4.1
Scope
Definitions
Definitions
Bibliography
Requirements
Mandatory Features
All-hazards risk and resilience
analysis of vulnerabilities to manmade threats, natural hazards, and
dependencies and proximity to
hazardous sites.
Notes/Comments
Yes
Feature Met
No
Scope must include all of the same requirements.
asset
critical asset
consequence
countermeasure
detect
deter
devalue
delay
respond
dependency
dependency hazard
event tree analysis
failure mode
fault tree analysis
frequency
hazard
incident
initiating event
likelihood
preparedness
probability
proximity hazard
response
reference threat
resilience
risk
risk analysis
risk management
scenario
system
threat
threat likelihood
vulnerability
vulnerability assessment
/vulnerability analysis
Is the list the same? If not, how is it different? Is
the content for each definition the same?
Incomplete*
X
X
Gap
The software does calculate risk, however,
resilience is not calculated, there is no input
for vulnerability, no predefined fields for
dependency or proximity hazards, threat
likelihoods, or scales for natural hazard
magnitudes.
No definitions are provided.
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
not included in software analysis
1 of 10
42
Table B.1 - J100-10 Standard Gap Analysis Matrix - SEMS
4.1.1
Mission
Mandatory Features
Notes/Comments
Software must provide multiple fields for mission or
critical functions.
functions to determine which assets
perform or support the mission
Software must provide multiple fields for critical
assets.
4.1.2, A.4.1
Critical Assets
Countermeasures and
4.1.5
4.1.5(.1)
4.1.5(.2)
Consequence metrics
Consequence metrics
Consequence metrics
4.1.5(.3)
4.1.5(.4)
4.1.5(.5)
Consequence metrics
Consequence metrics
Consequence metrics
4.1.5(.6)
X
X
Software allows user to add descriptions to
predefined assets or create new ones. User
can define asset priority (dropdown menu for
low, med., or high). Software includes two
check boxes for the following options:
including the asset in the risk assessment
(and reason for opting out) and if it is an
emergency asset.
X
Consequence metrics
protective countermeasures and
consequences for each asset without
regard to the threat
serious injuries
major economic loss to facility or
community
government
X
X
X
X
Can group these into Human, $ and other (i.e..
environmental) Can use Hi, Very Hi, Med, Lo, etc.
field for each consequence.
There is no separate field for critical
infrastructure, but the user has the ability to
create their own assets. The software enables
the user to create infrastructure as an asset
and define it as critical and have it included
in the analysis.
There is a check list of all the predefined
countermeasures. User can add descriptions
and detail to these (text box) or create your
own. Define the detection (dropdown none,
possible, probable, or certain), delay
(dropdown none, limited, strong, or very
strong), and response (dropdown none, slow,
variable, or fast).
refer to definition for types
of items
Software must estimate the worst reasonable
consequences for each asset.
Gap
Software provides drinking water and
wastewater systems’ missions to choose
from. Several generic missions are provided
(bubble checks) and an option to write your
own (text box) is included.
internal or external supporting infrastructure.
Software must provide multiple fields for all
existing protective countermeasures and mitigation
measures/features.
4.1.4
Incomplete*
identify critical internal and external
supporting infrastructure
4.1.3
Yes
Feature Met
No
X
X
X
The software does not allow the user to
define the worst reasonable consequences
without regard to threat.
X
2 of 10
43
4.1.6
4.2, A.4.2
Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
Prioritize assets
Mandatory Features
natural hazards, dependency hazards;
identify general and specific threat
scenarios to serve as reference threats
Reference Threats
Within the Standard body Reference
Threats are mandatory and Appx E is
mandatory
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Air attack
Land attack
Water attack
elements
Weapons types
equipment
tools
Notes/Comments
Yes
Incomplete*
X
Software must provide a field for each critical asset
to assign rank or re-order them in order of
consequence.
Software must provide a field to characterize the
kind of hazard/threat.
Feature Met
No
X
Gap
The user can choose if the asset is critical or
not and prioritize (using a dropdown menu of
high, med., or low) but can not reprioritize
based on consequences ( no list of asset and
consequences in comparison).
The software includes a list of most of the
J100-10 threats though no proximity,
dependency and some specific natural
hazards are included. Check boxes allow the
user to select of all that apply for each asset,
creating asset-threat pairs. The user can an
add their own threats and add descriptions
and details (text boxes).
-
Software must provide a field to describe the type of
malevolent threat to be considered and the outcome
of which is included in the software calculations.
X
X
X
X
X
X
X
Same as J100-10
3 of 10
44
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
4.2.2, Appx E & G
Malevolent Threat
Characterization
Characterization
Characterization
Characterization
Characterization
4.2.2, Appx E & G
Characterization
4.2.1, Appx E
4.2.2, Appx E & G
4.2.2, Appx E & G
4.2.2, Appx E & G
4.2.2, Appx E & G
Characterization
Characterization
4.2.3, Appx G
Characterization
4.2.2, Appx E & G
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
Mandatory Features
explosives
tactics
Notes/Comments
Yes
X
X
X
X
X
insiders
Non-mandatory in 4.2, mandatory in 4.2.1, shall be
analyzed as mandatory. Software must provide a
field to describe the type of malevolent threat to be
considered.
X
outsiders
considered.
hurricanes
Software must define the range of magnitudes from
the smallest that would cause serious harm to the
largest reasonable case.
floods
earthquakes
wildfires
Not listed in body, listed in non-mandatory
appendix, analyzed as non-mandatory.
ice storms
Undefined other
employees
Gap
The software does not provide ranges or
direct links to reference materials to
determine magnitudes. However it does say
what standards should be used for the
magnitudes. Also, changing the magnitude
of the natural disaster does not affect the
likelihood or overall risk.
X
X
X
tornadoes
suppliers
Incomplete*
Same as J100-10
utilities
Feature Met
No
X
The software does not include these as
predefined but can be added as an "other".
No ranges or direct links to reference
materials to determine magnitudes.
X
X
Software should give the user the option of
including other natural hazards or threats.
X
dependency threat to be considered.
X
X
X
predefined fields but they could potentially
be added as an "other".
4 of 10
45
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.4
Threat-Asset Pairs
4.2.5
4.2.6
4.3, A.4.3
4.3.1, Appx B
4.3.2, Appx B
4.3.2, Appx B
Evaluate and Rank ThreatAsset Pairs
Mandatory Features
customers
transportation
proximity
Notes/Comments
X
X
X
create threat-asset pairs
evaluate and rank threat-asset pairs
Software must evaluate and rank threat-asset pairs,
can use multiple approaches including a matrix
using small, med., large or scales 1-10, etc.
assumptions for each threat scenario
Threat Scenario
natural units reported and considered
individually; converted into a single,
summary economic value, reported
and considered as a single loss
indicator; or in pre-defined ranges
Estimate Consequences loss of life to either employees represented by the RAMCAP "bins"
(Appx B)
or the general public
Estimate Consequences represented by the RAMCAP "bins"
serious injury to either
employees or the general public (Appx B)
predefined fields but they could potentially
be added as an "other".
X
Software must identify the worst reasonable
consequence of a threat on assets.
X
Software must assume the worst reasonable case for
each threat.
X
X
The software does not rate or rank them
based on rough magnitude of consequences
prior to determining vulnerability or threat
likelihoods.
X
The user can select critical assets but can not
select critical threat-asset pairs.
Software must provide a field for single point
estimate of consequences or a bin number. When
reviewing bins, the values must match. Under 4.3
this is defined as an "or" under 4.3.2 it is defined as
an "and" analysis shall include "and" as a more
conservative approach.
X
optional single indicator estimate of consequences or a bin number. When
dollar equivalence of
fatalities and serious injuries this is defined as an "or" under 4.3.2 it is defined as
in excess of insurance
X
optional single indicator dollar equivalent of fatalities
and serious injuries in excess
of insurance
Gap
X
Software must provide a check box or field to
identify selected critical threat-asset pairs. (Standard
does not define critical, this is left up to the user,
i.e.. top 10, top 20?)
select critical threat-asset pairs to be
used going forward or use all pairs
consequences that can be caused by
the specific threats on the assets as
identified in 4.1
Incomplete*
including other kinds of dependency hazards or
threats.
Software must allow user to create threat-asset
pairs.
Undefined other
Yes
Feature Met
No
The software does not include single point
estimates or bin numbers. It uses a
dropdown menu of J100-10 standard ranges.
5 of 10
46
4.3.2, Appx B
4.3.2, Appx B
4.3.2, Appx B
Mandatory Features
Financial losses to
(Appx B)
owner/operator
Estimate Consequences service denial for the affected represented by the RAMCAP "bins"
customers
(Appx B)
Estimate Consequences economic losses to society and represented by the RAMCAP "bins"
(Appx B)
the general public
if degradation in public confidence,
environmental quality, ability of
civilian or military agencies to
function, etc. room for descriptive
4.3.3
4.3.4
4.3.5, Appx B
4.4, A.4.4
Estimate Consequences - other
Record consequence
document specific assumptions and
procedures used for performing the
consequence analysis, the worst
reasonable case assumptions and the
results of the consequence analysis
record the consequence values using
point estimates or ranges
Notes/Comments
Yes
optional single indicator value of a statistical life
additional consequences that
can be considered Software must provide the ability to describe
sociopolitical impacts,
primary consequence, and should give the user the
natural security impacts, lost option of including other consequences.
strategic capability to cause
harm or output, detrimental
effects on brand value,
public confidence,
Software must give users a space to document the
assumptions made in the analysis, and should have
the ability to include other documentation such as
maps and calculations.
Preferred ranges are in Appx Software must provide a field for ranges or point
B.
estimates.
Feature Met
No
Incomplete*
Gap
X
estimates or bin numbers. It uses a
dropdown menu of J100-10 standard ranges.
X
The software does not include the
consequence of service denial for the affected
customers.
X
The software does not inlcude single point
estimates or bin numbers. It uses a dropdown
menu of J100-10 standard ranges
X
The software only provides fields for 4
consequence types (fatalities, injuries,
economic cost of owner, and economic cost
to region).
Document assumptions can be typed into the
generated report.
X
X
The software only uses ranges for the
consequence value.
6 of 10
47
4.4.1
4.4.2, A.4.4
4.4.3
4.4.4, Appx B
Mandatory Features
Review
review pertinent details of the facility
construction, systems and layout;
identify vulnerabilities or weaknesses
in the protection system
analyze vulnerability of each critical
asset to estimate the likelihood that,
given the occurrence of a threat, the
consequences result
Document Method
document method and results of the
vulnerability analysis
Notes/Comments
Software must provide fields to input pertinent
details of construction, systems and layout.
may use fault, event or
failure tree analysis, path
analysis, vulnerability logic
diagrams, computer
simulation methods, or
expert judgment rules-ofthumb
4.5.1, Appx F
Malevolent Threats
use proxy measure, best estimate, or
conditional assessment to determine
F.3.1
F.3.2
F.3.3
F.3.4
Number of U.S. attacks per
year
Metro Region (RMS metro
area classes)
Target Type (RMS target
type analysis)
Proportion: Regional
Number
F.3.5
This Facility
F.3.6
F.3.7
Yes
Incomplete*
Gap
The user can add in as general information or
security information during the creation of
the asset (text boxes).
X
Software must provide a field for the vulnerability
analysis of each asset, and should use one of the
following methods: event-tree analysis, path
analysis, vulnerability logic diagrams, computer
simulations, or judgment rules-of-thumb.
X
The software does not have fields to calculate
or input values for vulnerability. Instead, it is
calculated, by the software, based on the
response time, delay, and detection
(dropdown menus) of every countermeasure
for each threat-asset pair. (Adding more
countermeasures to a pair decreases the
vulnerability of that pair)
Software must provide a field to document the
vulnerability analysis method used.
X
The software provides a vulnerability scale in
the report but no explanation of the process.
X
The software displays the vulnerability as a
J100-10 percent range and bin but does not
allow the user to input the vulnerability
manually.
RAMCAP scales; if bins are Software must provide a field for point estimates,
used, the midpoint is used
and the field should allow for utilization of
for the calculation
calculations from bins.
Record the estimates from 4.4.2
natural hazard
4.5, A.4.5
Record Estimates
Feature Met
No
Software must provide a field for an estimate of
threat severity.
X
Software must provide a field for estimate of
malevolent threats.
X
Software may have a field for number of attacks.
X
Software may have a field for likelihood.
X
X
Software may have a field for ratio of capacity to
metro area.
Software may have a field for likelihood (product of
V x C x Detection).
Software may have a field calculated by multiplying
each proxy indicator.
X
The software uses the J100-10 tier table
based on the facility (population and critical
customers). The software does not determine
likelihood for each threat.
X
X
X
7 of 10
48
4.5.2, G.2
Natural Hazards
Mandatory Features
earthquakes
4.5.2, G.3
Natural Hazards
hurricanes
4.5.2, G.4
Natural Hazards
tornadoes
4.5.2, G.5
Natural Hazards
floods
Appx G is optional and
provides data to estimate the
risk of each natural hazard risk is calculated by C x V x
T; would be nice to have
look-up maps/tables for each
natural hazard
4.5.2, Appx G
Natural Hazards
weather, wildfires,
avalanche, tsunami,
landslide, mud slide
4.5.2, Appx G
Natural Hazards
other
4.5.2, Appx G
Natural Hazards
4.5.3
Hazards
4.5.4
4.6, A.4.6
4.6.1
4.6.2, Appx H
Record Estimates
Risk and Resilience Analysis
Calculate Risk
Calculate the current level of
resilience
Risk
frequency, severity and duration of
service denials
Software must have a field to enter the risk of floods
for each magnitude from historical records.
Software may have a field to enter the risk of ice
storms, extreme cold weather, wildfires, avalanche,
tsunami, landslide, mud slide from historical
records.
Software may have a field to enter the risk of other
natural hazards.
Yes
X
X
Incomplete*
Gap
The software has fields for historic
information and magnitudes (text boxes),
however, these values do not impact the
X
X
X
X
Software must have field for total natural hazard
risk.
X
Software must have a field for predicted
dependency and proximity likelihood.
X
The software does not have a field for total
natural hazard risk. This could potentially be
done manually by performing an analysis
with only natural hazards.
X
The software provides and explains a threat
likelihood scale and displays the threat
likelihood for each threat-asset pair but does
not allow the user to determine these values
individually for each pair and explain their
reasoning.
X
The software calculates the overall risk but
not the resilience. It should also be noted
that the risk was calculated without the user
assigning their own vulnerability and threat
likelihood for each pair.
Software must calculate risk using the numbers
use midpoint of ranges from input previously for C, V and T for each threat-asset
pair.
Appx B
X
The software calculates the risk for each pair
but without the user assigning their own
vulnerability and threat likelihood for each
pair.
Some form of a resilience calculation is mandatory,
we assume that the preferred approach is to measure
the standard in two ways: threat-asset pair and
holistic approach but both of the methods described
in the standard are nonmandatory.
X
record the method used for making the
estimates and the estimates
themselves as single point values or
ranges
Software must have room for selected methods to be
documented.
Software must estimate the owner's resilience and
owner's and community's risk for each threat-asset
pair.
resilience relative to each threat-asset
pair
for each threat-asset pair calculate
risk: C x V x T = R
Notes/Comments
software must have a field to enter the risk of
earthquakes for each magnitude from historical
records.
Software must have a field to enter the risk of
hurricanes for each magnitude from historical
records.
tornadoes from historical records.
Feature Met
No
use either threat-asset pair resilience
metric or holistic approach in Appx H Appx H is nonmandatory
8 of 10
49
4.6.2.1
Operational Resilience Asset
Resilience Metric
4.6.2.2
Owner's Economic Resilience
4.6.2.3
Community Economic
Resilience
Mandatory Features
Duration x severity x vulnerability x
threat likelihood = asset resilience
metric
lost revenue due to the threat-asset
pair (asset resilience x unit price)
Notes/Comments
(ORI) calculated by
choosing values from Table
H-1 then calculating ORI by
multiplying the indicator
value by the weight and
adding all values (should
4.6.3
4.7, A.4.7
Incomplete*
Gap
X
Software may calculate asset
resilience metric using
duration and severity from
4.3, vulnerability from 4.4
and threat likelihood from
4.5.
X
Financial Resilience Index
(FRI) calculated by choosing
calculating FRI by
X
resilience x unit price (have
field for unit price of asset).
Also software may have field
for lost economic activity to
the community (same as
4.3).
Utility Resilience Index software should calculate
URI using the values of ORI
and FRI and the weights
given in Table H-1 & H-2
(URI=ORI x w1 + FRI x w2)
Estimates
Risk and Resilience
Management
Yes
Feature Met
No
X
Software may have fields for Software must have fields for both threat-asset pair
threat-asset pair resilience. resilience and holistic resilience.
The software creates a scatter plot of
consequence versus vulnerability and places
each threat-asset pair on the plot. It also
generates a report in which the risks of all of
the pairs are listed in a table and ranked from
the greatest risk to the lowest. However, no
resilience is calculated or recorded.
-
9 of 10
50
Mandatory Features
4.7.4, A.4.7.4
Assess
4.7.5, A.4.7.5
Identify
decide what risk and resilience levels
are acceptable
mitigation/resilience options for the
threat-asset pairs that are not
acceptable. Include devalue, deter,
detect, delay and response;
consequence reductions, resilience
enhancements
estimate investment and operating
costs for each option; include regular
maintenance and periodic overhaul;
adjust to present value
revisit 4.3 through 4.6 to estimate the
risk and resilience levels as if the
option was implemented; calculate the
identify the options that have benefits
that apply to multiple threat-asset
pairs
4.7.6, A.4.7.6
Calculate
4.7.1, A.4.7.1
4.7.2, A.4.7.2
4.7.3, A.4.7.3
Decide
Define
Estimate
rank the most cost effective measures
to implement
Review & Rank
4.7.7, A.4.7.7
* Incomplete can indicate that the feature was partially met or that it could use some improvements for useablility.
4.7.6, A.4.7.6
Calculate
Notes/Comments
Yes
Feature Met
No
Gap
X
The software does not provide way to set this
level or rank the threat-asset pairs. The user
is forced to include all pairs for the remaining
analysis.
X
The user can define countermeasures to lower
the risk, but only by redoing the analysis and
adding additional countermeasures.
Software must provide a field for acceptable risk
and resilience level.
Software must provide fields for countermeasure
and mitigation/resilience for each threat asset-pair.
Software must provide fields for costs for each
option above.
Incomplete*
X
Software must provide a field for the new value of
risk.
X
Software must provide a way to highlight or mark
options.
Software must include a calculation of the net
benefits.
Software must include a calculation of the benefitcost ratio.
Software must include a field for ranking cost
effective measures.
X
X
X
X
The user can determine the new risk value
but only be restarting the analysis with
additional countermeasures applied.
The software does not calculate cost for each
countermeasure or the benefits since the only
way to calculate a lower risk is to restart the
analysis.
10 of 10
51
Table B.2 - J100-10 Standard Gap Analysis Matrix - VSAT
1
2.1, Appx C
2.1.1
2.2
2.3
2.4
2.4.1
2.4.2
2.4.3
2.4.4
2.4.5
2.5
2.6
2.7
2.8
2.9
2.10
2.11
2.12
2.13
2.14
2.15
2.16
2.17
2.18
2.19
2.2
2.21
2.22
2.23
2.24
2.25
2.26
2.27
2.28
2.29
2.30
2.31
3, Appx D
4
4.1
4.1.1
Scope
Definitions
Mandatory Features
All-hazards risk and resilience analysis
of vulnerabilities to man-made threats,
natural hazards, and dependencies and
proximity to hazardous sites.
Mission
functions to determine which assets
perform or support the mission
Notes/Comments
Yes
asset
critical asset
consequence
countermeasure
detect
deter
devalue
delay
respond
dependency
dependency hazard
event tree analysis
failure mode
fault tree analysis
frequency
hazard
incident
initiating event
likelihood
preparedness
probability
proximity hazard
response
reference threat
resilience
risk
risk analysis
risk management
scenario
system
threat
threat likelihood
vulnerability
Definitions
Bibliography
Requirements
Feature Met
No
Incomplete* Gap
X
Is the list the same? If not, how is it different? Is the
content for each definition the same?
X
The software does not calculate risk or
resilience.
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Software must provide multiple fields for mission or
critical functions.
X
The software provides a field (text box).
1 of 9
52
4.1.2, A.4.1
4.1.3
4.1.4
4.1.5
4.1.5(.1)
4.1.5(.2)
4.1.5(.3)
4.1.5(.4)
4.1.5(.5)
4.1.5(.6)
4.1.6
Critical Assets
Prioritize assets
Appx E
Reference Threats
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
Countermeasures and mitigation protective countermeasures and
measures/features
Consequence metrics
Consequence metrics
serious injuries
Consequence metrics
community
Consequence metrics
Consequence metrics
Consequence metrics
government
Consequence metrics
4.2, A.4.2
4.2.1, Appx E
Mandatory Features
natural hazards, dependency hazards;
identify general and specific threat
scenarios to serve as reference threats
within the Standard body Reference
mandatory
Air attack
Land attack
Water attack
elements
Weapons types
refer to definition for types
of items
Notes/Comments
Yes
assets.
X
X
Software must provide multiple fields for all existing
protective countermeasures and mitigation
measures/features.
X
Feature Met
No
Incomplete* Gap
The software contains a list of common assets
used at most facilities. The user can also add
custom assets (text box) and edit assets to
specify details (text box).
The software provides critical internal or
external supporting infrastructures within the
predefined common assets or they may be
added manually.
The software contains a list of
countermeasures found at American utilities
and their costs. The user can add custom
countermeasures (text box) and edit to specify
details (text box).
X
X
X
X
The user can select consequences manually.
X
X
X
consequence.
Software must provide a field to characterize the
kind of hazard/threat.
X
X
prioritize assets.
The software provides a list of potentially
relevant threats for wastewater and water
utilities. All threats are placed under their
respective threat types.
malevolent threat to be considered.
X
X
X
Same as J100-10
X
X
2 of 9
53
4.2.1, Appx E
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
4.2.1, Appx E
Malevolent Threat
Characterization
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.2, Appx E & G
4.2.2, Appx E & G
4.2.2, Appx E & G
4.2.2, Appx E & G
4.2.2, Appx E & G
4.2.2, Appx E & G
4.2.2, Appx E & G
4.2.3, Appx G
4.2.3, Appx G
4.2.3, Appx G
4.2.3, Appx G
4.2.3, Appx G
Malevolent Threat
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
Mandatory Features
equipment
tools
explosives
tactics
insiders
outsiders
hurricanes
Notes/Comments
considered.
considered.
floods
earthquakes
wildfires
ice storms
Undefined other
suppliers
employees
customers
transportation
X
X
X
X
X
X
Same as J100-10
X
X
X
X
X
tornadoes
utilities
Yes
Feature Met
No
Incomplete* Gap
Software should give the user the option of including
other natural hazards or threats.
X
The software includes fires within the library
of standard natural hazards but does not
provide a range of magnitude.
X
X
The user can enter "own" risk and check it as
a natural hazard (check box). The software
includes snow and wind storms within the
library .
X
X
X
X
X
predefined fields but they could potentially be
added as an "other" under the user defined
threat section.
X
3 of 9
54
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.4
4.2.5
Threat-Asset Pairs
Mandatory Features
Undefined other
Evaluate and Rank Threat-Asset
Pairs
Yes
proximity
create threat-asset pairs
Notes/Comments
X
other kinds of dependency hazards or threats.
X
Software must allow user to creat threat-asset pairs.
X
can use multiple approaches including a matrix using
small, med., large or scales 1-10, etc.
does not define critical, this is left up to the user, i.e..
top 10, top 20?)
X
Threat Scenario
dollar equivalent of fatalities reviewing bins, the values must match. Under 4.3
and serious injuries in excess this is defined as an "or" under 4.3.2 it is defined as
of insurance
X
4.3.2, Appx B
Estimate Consequences loss of life to either employees
represented by the RAMCAP "bins"
(Appx B)
X
4.3.2, Appx B
Estimate Consequences indicator; or in pre-defined ranges
X
4.3.2, Appx B
(Appx B)
4.3, A.4.3
4.3.1, Appx B
owner/operator
X
each threat.
X
threat section.
The software does not rate or rank the threatasset pairs based on rough magnitude of
consequences prior to determining
vulnerability or threat likelihoods.
X
identified in 4.1
4.2.6
Feature Met
No
Incomplete* Gap
The user cannot remove asset-threat pairs
once they have been used in an analysis (even
if it's just the baseline analysis).
The user can use Water Health Economic
Analysis Tool (WHEAT) to determine the
consequences of each threat on an asset.
4 of 9
55
4.3.2, Appx B
4.3.2, Appx B
4.3.3
4.3.4
4.3.5, Appx B
4.4, A.4.4
4.4.1
4.4.2, A.4.4
Mandatory Features
Estimate Consequences service denial for the affected
customers
(Appx B)
Estimate Consequences economic losses to society and
the general public
(Appx B)
can be considered sociopolitical impacts,
natural security impacts, lost Software must provide the ability to describe
strategic capability to cause primary consequence, and should give the user the
harm or output, detrimental option of including other consequences.
effects on brand value, public
confidence, psychological
impacts, and environmental
degradation
Record consequence
Review
Notes/Comments
Software must give users a space to document the
assumptions made in the analysis, and should have
the ability to include other documentation such as
maps and calculations.
document specific assumptions and
procedures used for performing the
consequence analysis, the worst
reasonable case assumptions and the
results of the consequence analysis
review pertinent details of the facility
construction, systems and layout;
identify vulnerabilities or weaknesses
in the protection system
consequences result
B.
estimates.
Software must provide fields to input pertinent
details of construction, systems and layout.
diagrams, computer
Yes
Feature Met
No
Incomplete* Gap
X
customers.
X
X
economic cost of owner, and economic cost to
region).
The software results display in tables and bar
plots. VSAT can output a more detailed table
to Excel. The Water Health Economic
Analysis Tool (WHEAT) can be downloaded
and used to help determine the consequence
values and can output the results to an Excel
table.
The user can enter a specific value (text box)
or select a predefined J100 bins with their
corresponding ranges (bubble) Note: Bin 0 (025) has been broken down into 0A (0), 0B (15), and 0C (6-25).
X
X
The user can add details about facility or
assets and their locations (text boxes). The
user can choose countermeasures and add
details and locations (text box).
X
X
The software relies on the judgment of the
user and does not provide for the other
methods.
5 of 9
56
4.4.3
Document method
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete* Gap
X
X
4.4.4, Appx B
4.5, A.4.5
Record Estimates
natural hazard
Malevolent Threats
used, the midpoint is used for and the field should allow for utilization of
the calculation
threat severity.
F.3.1
F.3.2
F.3.3
F.3.4
year
area classes)
type analysis)
Number
F.3.5
This Facility
F.3.6
F.3.7
4.5.1, Appx F
4.5.2, G.2
4.5.2, G.3
Natural Hazards
Natural Hazards
earthquakes
natural hazard
X
malevolent threats.
X
X
X
X
metro area.
V x C x Detection).
X
records
records.
hurricanes
For each pair, the user has to rate the
vulnerability by "bubbling" the detection
(certain, probable, possible, none), delay
(very, strong, limited, no delay), and response
(fast, variable, slow, none). The software
then determines the likelihood as a J100-10
percentage range (with rounding) and
countermeasure capability (very high, high,
moderate, low). The user can also enter
comments (text box).
likelihood by bubbling very high, high,
moderate, or low. The user can also enter
The software allows for the option to use best
estimate or a 100% probability for all pairs
(bubble).
X
X
X
X
The software determines the probability of
EQ1 through EQ5 magnitude earthquakes
(magnitudes based on ranges of peak ground
acceleration) by the zip code of the asset. It
also determines the 50 year probability of
excedance and the annual probability of
excedance. The user must then interpret those
results to set the likelihood as very high, high,
moderate, or low.
X
The software provides maps for return periods
for category H1 through H5 on the SaffirSimpson Scale. The user must then determine
the probabilities. The user must then interpret
those results to set the likelihood as very high,
high, moderate, or low.
6 of 9
57
4.5.2, G.4
Natural Hazards
Mandatory Features
tornadoes
natural hazard
Notes/Comments
4.5.2, G.5
4.5.2, Appx G
4.5.2, Appx G
Natural Hazards
floods
Natural Hazards
weather, wildfires,
avalanche, tsunami,
other
Natural Hazards
Yes
Feature Met
No
Incomplete* Gap
X
The software determines probability of T1
through T5 magnitude tornadoes (magnitudes
based on ranges of wind speeds) by the zip
code of the asset. It also determines the
number of tornadoes in the past 50 years, the
property damage, number of injuries, and
number of fatalities. The user must then
interpret those results to set the likelihood as
very high, high, moderate, or low.
X
The software determines probability of F1
and F2 magnitude floods (user has the option
to define their own flood severities and
probabilities) by the zip code of the asset. It
provides a link to FEMA reference materials.
The user must then interpret those results to
set the likelihood as very high, high,
moderate, or low.
tsunami, landslide, mud slide from historical records.
natural hazards.
risk.
4.5.2, Appx G
Natural Hazards
4.5.3
Hazards
4.5.4
4.6, A.4.6
4.6.1
4.6.2, Appx H
Record Estimates
Calculate Risk
resilience
Risk
service denials
estimates and the estimates themselves
as single point values or ranges
pair
Software must have a field for predicted dependency
and proximity likelihood.
documented.
pair.
for each threat-asset pair calculate risk: use midpoint of ranges from Software must calculate risk using the numbers input
Appx B
previously for C, V and T for each threat-asset pair.
CxVxT=R
The software does not include a field to enter
historical data.
X
X
The software can include the likelihood of the
other threats but not based on historical data.
X
The software has a tab for natural threats
where all of the asset-threat pairs of natural
hazards are displayed, but no total risk is
displayed. Also, the added hazards, e.g.,
windstorms, snowstorms, are not displayed in
this tab.
X
The user can include comments about the
threat likelihood chosen and displays in the
report.
X
X
The software does not calculate overall risk or
resiliency.
X
X
The software includes a list of "knowledge
base" information for resiliency within the
predefined asset list but does not use them to
calculate resiliency.
7 of 9
58
4.6.2.1
4.6.2.2
4.6.2.3
Resilience Metric
Community Economic
Resilience
Mandatory Features
Notes/Comments
calculating ORI by
Yes
X
metric
4.5.
lost revenue due to the threat-asset pair Financial Resilience Index
(asset resilience x unit price)
calculating FRI by
X
X
4.6.3
4.7, A.4.7
4.7.1, A.4.7.1
4.7.2, A.4.7.2
4.7.3, A.4.7.3
4.7.4, A.4.7.4
Estimates
Risk and Resilience
Management
Decide
Define
Estimate
Assess
Feature Met
No
Incomplete* Gap
X
resilience.
X
The software does not provide a way to set
this level or rank the threat-asset pairs and the
user is forced to include all pairs for the
remaining analysis.
X
The software calculates risk reduction units.
The calculation and explination are confusing.
The calculation can only be used for
comparison purposes.
threat-asset pair resilience.
resilience and holistic resilience.
-
are acceptable
enhancements
Software must provide a field for acceptable risk and
resilience level.
X
Software must provide fields for countermeasure and
mitigation/resilience for each threat asset-pair.
option above.
risk.
X
8 of 9
59
Mandatory Features
4.7.5, A.4.7.5
Identify
that apply to multiple threat-asset pairs
4.7.6, A.4.7.6
Calculate
to implement
Review & Rank
4.7.7, A.4.7.7
* Incomplete can indicate that the feature was partially met or that it could be improved upon.
4.7.6, A.4.7.6
Calculate
Notes/Comments
options.
benefits.
effective measures.
Yes
Feature Met
No
Incomplete* Gap
The user can create multiple packages of
countermeasures and improvements to
X
compare and determine which one benefits
the utility the most.
X
X
X
The user can create upgrade packages and
compare the annualized cost, capital cost and
risk reduction units but cannot calculate net
benefit and benefit-cost ratios.
9 of 9
60
Table B.3 - J100-10 Standard Gap Analysis Matrix - ARAM-W
1
2.1, Appx C
2.1.1
2.2
2.3
2.4
2.4.1
2.4.2
2.4.3
2.4.4
2.4.5
2.5
2.6
2.7
2.8
2.9
2.10
2.11
2.12
2.13
2.14
2.15
2.16
2.17
2.18
2.19
2.2
2.21
2.22
2.23
2.24
2.25
2.26
2.27
2.28
2.29
2.30
2.31
3, Appx D
4
4.1
Scope
Definitions
Bibliography
Requirements
Mandatory Features
hazardous sites.
Notes/Comments
Yes
Feature Met
No
asset
critical asset
consequence
countermeasure
detect
deter
devalue
delay
respond
dependency
dependency hazard
event tree analysis
failure mode
fault tree analysis
frequency
hazard
incident
initiating event
likelihood
preparedness
probability
proximity hazard
response
reference threat
resilience
risk
risk analysis
risk management
scenario
system
threat
threat likelihood
vulnerability
Incomplete*
X
X
Gap
The software does not calculate resilience.
No definitions were included in the software.
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
1 of 8
61
Mandatory Features
Notes/Comments
Yes
Feature Met
No
4.1.5(.4)
Appx E
Consequence metrics
4.2.3, Appx G
Reference Threats
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
Characterization
4.2.3, Appx G
Characterization
4.2.2, Appx E & G
4.2.2, Appx E & G
4.2.2, Appx E & G
4.2.3, Appx G
4.2.3, Appx G
4.2.3, Appx G
4.2.3, Appx G
4.2.3, Appx G
4.2.5
Incomplete*
X
Gap
The user must select the undesired events,
(i.e. release of chemicals, loss of power, loss
of critical pump/valve system), from a
predefined event tree. There is a separate
tree for each type of utility and users can add
additional events. The user can then
determine the consequence of each
undesired event, which are determined by
assuming the loss of the asset which leads to
the undesired event, without regard to the
threats that may cause the loss of the asset.
Although environmental impact is not
predefined, the user has the ability to define
any new consequences that they feel apply.
within the Standard body Reference
mandatory
-
wildfires
ice storms
Undefined other
utilities
suppliers
employees
customers
transportation
proximity
Evaluate & Rank Threat-Asset
Pairs
Undefined other
Software must provide a field to describe the type
of dependency threat to be considered.
threats.
62
X
X
The software does not include any additional
natural threats or allow any other threats to
be added.
X
X
X
X
X
X
The software does not include dependency
threats, the RAMCAP labels for dependency
threats, or dependency threats within the
RAMCAP reference tables.
X
X
X
The software does not rate or rank the
undesired event/asset location/threat pairs
based on rough magnitude of consequences
prior to determining vulnerability or threat
likelihoods.
2 of 8
4.2.6
4.3, A.4.3
4.3.1, Appx B
4.3.2, Appx B
4.3.2, Appx B
4.3.2, Appx B
4.3.2, Appx B
Mandatory Features
identified in 4.1
Threat Scenario
loss of life to either employees represented by the RAMCAP "bins"
(Appx B)
employees or the general
public
(Appx B)
Financial loses to
owner/operator
(Appx B)
Estimate Consequences service denial for the affected represented by the RAMCAP "bins"
customers
(Appx B)
Notes/Comments
identify selected critical threat-asset pairs.
(Standard does not define critical, this is left up to
the user, i.e.. top 10, top 20?)
Yes
Feature Met
No
Incomplete*
X
X
each threat.
X
optional single indicator dollar equivalent of fatalities reviewing bins, the values must match. Under 4.3
and serious injuries in
excess of insurance
X
optional single indicator reviewing bins, the values must match. Under 4.3
conservative approach
X
X
X
Gap
The user cannot remove undesired
event/asset location/threat pairs once they
have been assigned in the baseline analysis.
Although the consequences for the loss of
critical assets and the resulting undesired
event occurring were previously determined,
they are not evaluated for each specific
threat on the assets.
t
i
i l d t
i d
3 of 8
63
4.3.2, Appx B
4.4, A.4.4
4.4.2, A.4.4
4.5.1, Appx F
Mandatory Features
optional single indicator summary economic value, reported
value of a statistical life
the general public
(Appx B)
Malevolent Threats
F.3.1
F.3.2
F.3.3
consequences result
diagrams, computer
Notes/Comments
Yes
Feature Met
No
Incomplete*
X
X
The user can determine the vulnerability
either by entering a user estimated single
value or by using a path analysis with the
adversary sequence diagram. The software
could better meet the Standard by providing
a means to use other methods to analyze
vulnerability.
X
For each undesired event/asset
location/threat pair, the user can choose to
determine the likelihood of attack using
conditional, expert judgment, or a
questionnaire method. For the conditional
method, the likelihood value is automatically
considered High (or 100%) based on the
assumption that the attack will occur. For
the expert judgment method, the user can
input a likelihood of high, medium or low
(drop down menu). For the questionnaire
method, the likelihood value is determined
based on the responses to the questions
asked. The questions follow the threat
factors outlined in the RAMCAP - The
Framework document for estimating
likelihood of attack and include initial
consideration of capability, history, current
interest, current surveillance, documented
threats, potential consequences, ideology and
ease of attack. The user answers the different
questions by clicking on the appropriate
circle and the threat potential is calculated
based on the responses to the questions.
malevolent threats.
year
area classes)
type analysis)
Gap
X
X
X
4 of 8
64
F.3.4
F.3.5
This Facility
F.3.6
F.3.7
metro area.
Software may have a field for likelihood (product
of V x C x Detection).
Software may have a field calculated by
multiplying each proxy indicator.
Natural Hazards
weather, wildfires,
avalanche, tsunami,
records.
4.5.2, Appx G
Natural Hazards
4.5.2, Appx G
Natural Hazards
4.5.3
Hazards
4.5.4
4.6, A.4.6
4.6.2, Appx H
Record Estimates
resilience
Mandatory Features
Notes/Comments
Number
4.5.2, Appx G
other
Risk
service denials
record the method used for making
the estimates and the estimates
ranges
pair
metric or holistic approach in Appx H Appx H is no mandatory
Yes
Feature Met
No
Incomplete*
X
X
X
X
The software only includes four natural
threats and does not allow for the addition of
any others.
X
natural hazards.
X
risk.
X
X
threats and does not allow for the addition of
any others.
The software does not include total risk of
natural hazards.
Dependency and proximity threats are not
included in the software.
X
The user can include comments about the
threat likelihood chosen or information
about the sources used and displays the
threat potential values as none, low, medium,
high, or very high in the report.
X
Software must have room for selected methods to
be documented.
pair.
we assume that the preferred approach is to
measure the standard in two ways: threat-asset pair
and holistic approach but both of the methods
described in the standard are no mandatory.
Gap
The software does not include the proxy
method.
X
5 of 8
65
Mandatory Features
Notes/Comments
Operational Resilience
Index (ORI) calculated by
4.6.2.1
Resilience Metric
metric
4.6.2.2
4.6.2.3
Community Economic
Resilience
Feature Met
No
Incomplete*
Gap
X
X
X
4.5.
(FRI) calculated by
H-2 then calculating FRI by
Yes
Also software may have
field for lost economic
activity to the community
(same as 4.3).
6 of 8
66
Mandatory Features
Notes/Comments
Yes
Feature Met
No
(URI=ORI x w1 + FRI x
w2)
4.6.3
4.7, A.4.7
4.7.1, A.4.7.1
4.7.2, A.4.7.2
4.7.4, A.4.7.4
4.7.5, A.4.7.5
4.7.6, A.4.7.6
4.7.7, A.4.7.7
Define
are acceptable
X
The software includes a field to determine if
the risk for each undesired event/asset
location/threat pair is acceptable (check
box). The software does not include
resilience in this decision.
X
develop them by redoing the ASD analysis
with the inclusion of more countermeasures
and other enhancements. The user can also
create upgrade packages which only affect
the consequences which can be entered. The
software does not include resilience
enhancement measures.
enhancements
X
The software calculates the new risk of each
pair for each upgrade package. The software
displays a report of the baseline data and
upgrade packages for the highest risk threatasset location pair. It does not calculate
benefits of the options or include resilience.
Assess
Identify
pairs
options.
benefits.
Calculate
X
The software displays the overall risk and
the risk for each undesired event/asset
location/threat pair as either none, low,
medium, high, or very high. However,
resilience is not calculated or displayed.
-
option was implemented; calculate
the estimated benefits of the option
Calculate
Gap
threat-asset pair resilience. resilience and holistic resilience.
Estimates
Risk and Resilience
Management
Decide
Incomplete*
risk.
X
compare but the software does not determine
which improvements are included in multiple
packages.
X
The software can compare packages but
there are no calculations for net benefit.
X
The software does not calculate net benefit,
nor does it compare the benefit to the cost.
7 of 8
67
Mandatory Features
Review & Rank
to implement
4.7.9, A.4.7.8
Notes/Comments
effective measures.
Yes
Feature Met
No
X
Incomplete*
Gap
The software does not consider the most
effective package, only the packages and
their affect on each pair, especially the
highest risk pair.
8 of 8
68
Appendix C: J100-10 Recommendations | 35
APPENDIX C: J100-10 RECOMMENDATIONS
69
Table C.1 - J100-10 Standard Gap Analysis Matrix - SEMS
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete* Gap
1
Scope
asset
critical asset
consequence
countermeasure
detect
deter
devalue
delay
respond
dependency
dependency hazard
event tree analysis
failure mode
fault tree analysis
frequency
hazard
incident
initiating event
likelihood
preparedness
probability
proximity hazard
response
reference threat
resilience
risk
risk analysis
risk management
scenario
system
threat
threat likelihood
vulnerability
2.1, Appx C
2.1.1
2.2
2.3
2.4
2.4.1
2.4.2
2.4.3
2.4.4
2.4.5
2.5
2.6
2.7
2.8
2.9
2.10
2.11
2.12
2.13
2.14
2.15
2.16
2.17
2.18
2.19
2.2
2.21
2.22
2.23
2.24
2.25
2.26
2.27
2.28
2.29
2.30
2.31
4.1.3
Definitions
4.1.5(.3)
4.1.5(.4)
4.1.5(.5)
Consequence metrics
Consequence metrics
Consequence metrics
Consequence metrics
government
Medium.
Requires the creation of
The software must provide the user a
an additional user
reference or documentation section in the
interface screen, but there
program, where definitions of these terms can is little to no computation
be determined and evaluated for fitness.
required to display these
definitions to the user.
X
X
X
Consequence metrics
Consequence metrics
Consequence metrics
The software should provide a method to
calculate resilience, including the supporting
user interface elements to gather the user
input required for such calculations.
Large/Unknown fix
(likely large).
A combination of user
interface elements
(another section from the
main RAMCAP menu)
and internal functionality
would be required to
support calculation of
resilience.
X
4.1.5
4.1.5(.1)
4.1.5(.2)
The software does calculate risk; however,
resilience is not calculated, there is no input
for vulnerability, no predefined fields for
dependency or proximity hazards, threat
likelihoods, or scales for natural hazard
magnitudes.
Type/Size of Fix
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
serious injuries
community
4.1.5(.6)
X
Recommendations
X
X
X
X
X
X
X
There is no separate field for critical
infrastructure, but the user has the ability to
create their own assets. The software enables The software should provide critical
infrastructure within the predefined assets.
the user to create infrastructure as an asset
and define it as critical and have it included in
the analysis.
Medium.
The functionality for
critical infrastructure
exists, but is burdensome
for the user to perform.
Additional user interface
elements would make this
Small.
Allowing the user to
define a baseline set of
consequences when
The software must determine the worst
describing each asset in
The software does not allow the user to define reasonable consequence for each asset prior
the Asset Information
to determining the threats that could affect the
the worst reasonable consequences without
page would satisfy this
regard to threat.
asset. This must be done to help prioritize the requirement. This requires
assets.
the addition of more form
fields (in particular,
dropdown menus) to the
user interface.
1 of 10
70
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete* Gap
X
4.1.6
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
4.2.1, Appx E
Prioritize assets
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Malevolent Threat
Characterization
Air attack
consequence.
X
Land attack
X
Water attack
elements
X
X
Weapons types
X
equipment
X
tools
X
explosives
X
tactics
X
X
Recommendations
The user can choose if the asset is critical or
not and prioritize (using a dropdown menu of
high, med., or low) but can not reprioritize
based on consequences (no list of asset and
consequences in comparison).
The software must have a way to rank the
assets in order of their importance after the
worst reasonable consequences have been
determined for each asset.
Same as J100-10
The software must have a way to select the
type of malevolent threat, as well as have
internal calculations that take into account
that type of threat when determining the
outcome.
Small.
The Asset Information
page should have an
additional dropdown
menu that allows the user
to select an asset
criticality based on the
consequences related to
that asset. This is a user
interface component that
displays previously
entered information, so no
additional calculation
should be required.
X
Same as J100-10
considered.
Type/Size of Fix
Small/Unknown (likely
small). Addition of a user
interface element to
choose the type of
malevolent threat requires
the addition of a combo
box or other field, but
taking into account the
impact of that choice in
calculations is of
unknkown complexity
without knowing the
internal calculations of
SEMS.
The software must have a way to select the
type of malevolent threat, as well as have
internal calculations that take into account
that type of threat when determining the
outcome.
X
insiders
2 of 10
71
4.2.2, Appx E & G
Characterization
4.2.2, Appx E & G
Characterization
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete* Gap
4.2.2, Appx E & G
4.2.2, Appx E & G
Characterization
4.2.2, Appx E & G
Characterization
4.2.2, Appx E & G
Characterization
4.2.2, Appx E & G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
hurricanes
floods
Medium/Unknown (likely
small).
The software does not provide ranges or
Missing hazards can be
direct links to reference materials to
added to the SEMS
determine magnitudes. However, it does say
database (in particular, the
what standards should be used for the
Threat table) in order to
magnitudes. Also, changing the magnitude of
not be manually entered
by the user.
the natural disaster does not affect the
Since the standards for
likelihood of overall risk.
The software must provide a range of
the magnitudes are
magnitudes that would damage the assets for
known, the software
each natural disaster type.
should be able to provide
ranges for the magnitudes
when selecting the threat.
The software's calculation
of risk should be amended
to include this magnitude,
but without knowing the
predefined but can be added as an "other."
process an estimate of
No ranges or direct links to reference
effort cannot be given.
materials exist in the software to help
X
tornadoes
X
earthquakes
X
wildfires
ice storms
Undefined other
utilities
suppliers
employees
customers
transportation
proximity
Threat-Asset Pairs
determine magnitudes.
X
other natural hazards or threats.
X
X
X
X
X
X
X
X
4.2.4 & 4.2.5?
Type/Size of Fix
X
X
Characterization
Recommendations
Medium.
The software should
provide additional choices
in the list of potential
The software should have dependency
threats to cover
hazards predefined within the list of potential
dependency hazards. This
threats; however, they can still be entered
can be accomplished
added as an "other" hazard.
manually within the user defined section.
through the addition of
these threats to the SEMS
database (specifically the
Threat table).
The software does not rate or rank pairs based
on a rough magnitude of consequences prior
to determining vulnerability or threat
likelihoods.
The software must rate and rank the pairs
based on their rough magnitude of
consequences solely based on the possible
consequences of the threat on the asset.
Unknown (likely
medium).
The software's
calculations of magnitude
of consequences are not
visible and may require
substantial reworking to
include other factors.
3 of 10
72
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete* Gap
The user can select critical assets but can not
select critical threat-asset pairs.
X
4.2.6
4.3.2, Appx B
4.3.2, Appx B
4.3.2, Appx B
4.3.2, Appx B
4.3.2, Appx B
Estimate Consequences loss of life to either employees or represented by the RAMCAP "bins"
(Appx B)
the general public
Estimate Consequences - serious
injury to either employees or the
general public
(Appx B)
Estimate Consequences - Financial represented by the RAMCAP "bins"
loses to owner/operator
(Appx B)
top 10, top 20?)
Estimate Consequences - service
denial for the affected customers
and serious injuries in excess this is defined as an "or" under 4.3.2 it is defined as
of insurance
X
X
Estimate Consequences - economic indicator; or in pre-defined ranges
losses to society and the general
public
(Appx B)
Type/Size of Fix
medium).
The addition of the ability
to select threat-asset pairs
requires the addition of
The software does not give the option of
some user interface
proceeding with an analysis of certain pairs.
elements, but revising the
Must be able to choose the pairs that must be analysis to be restricted to
analyzed for their risk.
certain subsets will
require a revision to the
underlying logic, the
extent of which is
unknown.
X
(Appx B)
Recommendations
Small.
The addition of a user
interface element (likely a
The software could provide a field for a
text field) for the user to
estimates or bin numbers. It uses a dropdown consequence point estimate in the case that an enter a value rather than
selecting from a
menu of J100-10 standard ranges.
exact number is known or desired.
predefined set of J100-10
standard ranges should be
easy to implement.
X
small).
In addition to adding the
The software must include a field to enter a user interface element for
single point estimate or bin number of the
the user to enter an
consequence of service denial for the affected estimate or bin number,
customers.
the impact that that
customers.
estimate has on the
calculations is unable to
be determined.
X
Small.
The addition of a user
interface element (likely
The software could provide a field for a
The software does not inlcude single point
text field) for the user to
estimates or bin numbers. It uses a dropdown consequence point estimate in the case that an enter a value rather than
selecting from a
exact number is known or desired.
menu of J100-10 standard ranges
predefined set of J100-10
standard ranges should be
easy to implement.
4 of 10
73
Mandatory Features
4.3.3
4.3.5, Appx B
Yes
Software must provide the ability to describe
option of including other consequences.
4.4.2, A.4.4
X
consequences result
X
B.
estimates.
X
diagrams, computer
4.4.3
Document Method
The software only uses ranges for the
consequence value.
Type/Size of Fix
The software should provide a field for a
descriptive analysis
Small.
Addition of another text
area to store a qualitative
description of the
consequences would
require an update to the
user interface of the
Consequence Assignment
page and an additional
column in the SEMS
Consequence table) for
the asset-threat pair.
The software must add a field to record
additional consequences for the analysis.
Small.
Addition of another text
area for entering
additional consequences
should be a minor change
to the user interface as
well as addition of a row
per entry into the SEMS
Consequence table).
The software should additionally provide a
field to enter a point estimate of the
consequence values.
Small.
Addition of a text field
used to enter this data
requires a modification
the user interface.
medium).
While adding a user
interface element to allow
The software must provide a field for the user for entry of a vulnerability
to assign a vulnerability to each threat-asset for each threat-asset pair
pair instead of being calculated automatically requires just a user
interface modification, the
based on the countermeasures.
changes that that entry
The software does not have fields to calculate
would make in the
or input values for vulnerability. Instead, it is
calculation method cannot
calculated based on the response time, delay,
be determined.
and detection (dropdown menus) of every
countermeasure for each threat-asset pair.
(Adding more countermeasures to a pair
decreases the vulnerability of that pair)
The software may use one of the suggested
methods for determining the vulnerability
X
Recommendations
to region).
can be considered sociopolitical impacts,
natural security impacts, lost
strategic capability to cause
harm or output, detrimental
effects on brand value, public
confidence, psychological
impacts, and environmental
degradation
Record consequence
Notes/Comments
Feature Met
No
Incomplete* Gap
Large.
The software currently
uses a determination
system that would likely
require major
modifications to
accommodate another
method, such as eventtree analysis.
Small.
The software could
simply output a
The software provides a vulnerability scale in The software must document the method used description of the method
on the same page as the
the report but no explanation of the process. to achieve the results.
results, which would
require displaying static
text in a new text field.
5 of 10
74
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete* Gap
Recommendations
Type/Size of Fix
Small.
This requires storage of
the user's selection (either
The software must allow the user to assign a from the range of J100
vulnerability to each threat-asset pair and then values or the user's
manual input) in an
record this value.
additional column in the
database, in the table
AssetThreat.
X
The software displays the vulnerability as a
J100-10 percent range and bin but does not
allow the user to input the vulnerability
manually.
The software may use J100 scales or point
estimates to record the calculated
vulnerability.
4.4.4, Appx B
4.5, A.4.5
Record Estimates
used, the midpoint is used for and the field should allow for utilization of
the calculation
threat severity.
natural hazard
X
X
F.3.2
F.3.3
F.3.4
year
area classes)
type analysis)
Number
F.3.5
This Facility
F.3.6
metro area.
V x C x Detection).
4.5.1, Appx F
F.3.1
F.3.7
Malevolent Threats
Small. Adding the ability
for the user to enter their
own values for
vulnerability would only
require a user interface
change, and the
corresponding value
would be stored in the
aforementioned database
column for each threatasset pair.
medium).
Adding the capability for
users to assign likelihood
of each threat involves the
addition of another
The software uses the J100-10 tier table based
column to the AssetThreat
The software must allow the user to assign
on the facility (population and critical
table in the SEMS
the likelihood or frequency of all hazards and
database, along with a
customers). The software does not determine
threats on a specific asset.
user interface element to
likelihood for each threat.
allow users to input and
store values in the field.
However, how this
likelihood will be
incorporated into
calculations is unknown.
malevolent threats.
X
X
X
X
X
X
X
The software could add proxy measures to
help determine the threat likelihood.
medium).
While adding a user
interface element to allow
for entry of a proxy
measures for vulnerability
requires just a user
interface modification, the
changes that that entry
would make in the
calculation method of
overall threat likelihood
cannot be determined.
6 of 10
75
4.5.2, G.2
Mandatory Features
Natural Hazards
earthquakes
4.5.2, G.3
Natural Hazards
hurricanes
4.5.2, G.4
Natural Hazards
tornadoes
4.5.2, G.5
Natural Hazards
floods
natural hazard
4.5.2, Appx G
Natural Hazards
weather, wildfires,
avalanche, tsunami,
4.5.2, Appx G
Natural Hazards
other
Notes/Comments
records
records.
records.
natural hazards.
risk.
4.5.2, Appx G
4.5.3
4.5.4
4.6, A.4.6
Natural Hazards
Hazards
Record Estimates
Yes
Feature Met
No
Incomplete* Gap
Recommendations
Type/Size of Fix
X
Unknown (likely
The software must allow the user to assign
medium). The calculation
the likelihood or frequency of a natural
hazard to the facility based on historical data. of the results of the
X
X
X
X
analysis needs to be
modified and while the
basic formula is known,
SEMS' implementation of
the calculation is not
The software may provide historical data or transparent and so the
links to data to help determine the probability amount of effort required
to implement these
that a specific magnitude of natural hazard
changes is unknown.
will occur.
X
X
Unknown (likely
medium). While it seems
like this would be simple
enough to implement (a
The software does not have a field for total
The software must calculate the risk for each summation of the
natural hazard risk. This could potentially be
natural hazard and sum them to determine the individual risks), without
done manually by performing an analysis
knowledge of the
overall risk due to natural hazards.
with only natural hazards.
calculation no
assumptions with regards
to implementation
difficulty can be made.
Risk
service denials
estimates and the estimates themselves
as single point values or ranges
pair
X
medium).
The data format of the
historical data needs to be
consistent for all entries in
The software must include historical data on order to be used for
dependency and proximity hazards to
calculations (text fields
determine the likelihood that the threats will need to be validated
according to these
occur to the assets.
parameters). If this is the
case, there must be an
update to the calculation
to reflect these user
inputs.
X
The software provides and explains a threat
likelihood scale and displays the threat
likelihood for each threat-asset pair but does
not allow the user to determine these values
individually for each pair and explain their
reasoning.
Medium.
This information should
be added as additional
The software must allow the user to
columns to the
determine the likelihood of each threat
AssetThreat table in the
occurring to an asset and record this estimate, database, and user
along with the method and reasoning for the interface elements should
be added to the
estimate.
Consequence Assignment
page to allow the user to
enter data to these fields.
X
The software must use the provided
consequence and allow the user to input the
The software calculates the overall risk but
not the resilience. It should also be noted that
threat-asset pair to calculate the owner's and
the risk was calculated without the user
community's overall risk. Must include
assigning their own vulnerability and threat
duration of service denial and severity or
likelihood for each pair.
service denial (gpd) to determine the owner's
and community's overall resilience.
documented.
pair.
Unknown.
The software contains no
current functionality
relating to resilience, so
incorporating this could
require extensive
development time.
7 of 10
76
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete* Gap
X
4.6.1
4.6.2, Appx H
Calculate Risk
resilience
risk: C x V x T = R
calculating ORI by
4.6.2.1
4.6.2.2
Resilience Metric
calculating FRI by
lost revenue due to the threat-asset pair adding all values (should
Community Economic Resilience
Type/Size of Fix
Small/Unknown.
Adding user interface
fields for the user to
choose their own values
for vulnerability and
threat likelihood is likely
a minor undertaking, but
there may be more effort
required for the updated
calculation.
X
Unknown. The software
The software must include duration of service contains no current
functionality relating to
denial and severity of service denial (gpd) to
resilience, so
determine the resilience of each threat-asset incorporating this may
pair.
require extensive
development time.
X
The software must calculate the asset
resilience.
contains no current
resilience, so
require extensive
development time.
The software must calculate the owner's
economic resilience.
contains no current
resilience, so
require extensive
development time.
The software must calculate the community
contains no current
resilience, so
require extensive
development time.
4.5.
X
Also software may have field
for lost economic activity to
the community (same as
4.3).
4.6.2.3
The software must use the provided
The software calculates the risk for each pair
consequence and allow the user to input the
but without the user assigning their own
threat-asset pair to calculate the risk for each
pair.
pair.
use midpoint of ranges from Software must calculate risk using the numbers input
Appx B
metric
Recommendations
X
8 of 10
77
Mandatory Features
Notes/Comments
Yes
4.6.3
4.7.1, A.4.7.1
4.7.2, A.4.7.2
Define
Recommendations
Type/Size of Fix
X
The software creates a scatter plot of
consequence versus vulnerability and places
each threat-asset pair on the plot. It also
The software must calculate and record the
generates a report in which the risks of all of
resilience of each threat-asset pair
the pairs are listed in a table and ranked from
the greatest risk to the lowest. However, no
resilience is calculated or recorded.
X
small).
In order to take into
account a user-defined
risk and resilience level, a
slider or some other user
interface component
must be added for
adjustments. Additionally,
The software does not provide way to set this The software must be able to record the
there should be some
level or rank the threat-asset pairs. The user is desired risk level and eliminate those threatvisual representation of
forced to include all pairs for the remaining asset pairs which have a risk that falls below which pairs will
analysis.
this desired level from the remaining analysis. accordingly be included in
the analysis. The user
interface additions should
be feasible, but there is
not enough information to
be able to determine how
SEMS will determine how
that cutoff point is used to
filter pairs.
X
Medium.
This functionality can be
addressed by having an
option to re-run the
analysis after querying the
user for additional
countermeasures, without
The software must allow users to make
The user can define countermeasures to lower
forcing them to manually
improvements to the baseline analysis by
begin the process again.
the risk, but only by redoing the analysis and
adding possible countermeasures for the next Such a change would
adding additional countermeasures.
part of the analysis.
require some user input
fields on the results page,
as well as a button that
would re-run the analysis,
so the only functional
changes are additions to
the user interface.
contains no current
resilience, so
require extensive
development time.
Estimates
Decide
Feature Met
No
Incomplete* Gap
are acceptable
enhancements
resilience level.
Software must provide fields for countermeasure and
mitigation/resilience for each threat asset-pair.
9 of 10
78
4.7.3, A.4.7.3
4.7.4, A.4.7.4
Estimate
Assess
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete* Gap
4.7.5, A.4.7.5
4.7.6, A.4.7.6
Identify
Calculate
X
X
Medium.
This functionality can be
addressed by having an
option to re-run the
analysis after querying the
user for new risk and
resilience values, without
forcing them to manually
begin the process again.
Such a change would
require some user input
fields on the results page,
as well as a button that
would re-run the analysis,
so the only functional
changes are additions to
the user interface.
option above.
Calculate
options.
benefits.
X
X
Review & Rank
to implement
4.7.7, A.4.7.7
The software must use the provided new
The user can determine the new risk value but
consequence, vulnerability, and threat
only by restarting the analysis with additional
likelihood to calculate the new risk for each
countermeasures applied.
threat-asset pair.
risk.
X
4.7.6, A.4.7.6
Type/Size of Fix
small).
Allowing the user to enter
cost information for each
countermeasure can be
achieved by adding text
fields to the
Countermeasure
Information page and
The software must allow the user to assign a
associating their inputs
cost for each countermeasure to determine the
with those
annual and capital cost for each option.
coutnermeasures. Without
knowing how the cost
calculations are
performed, however, it is
impossible to determine
how much effort it would
take to compute annual
and capital costs from
these inputs.
X
Recommendations
effective measures.
Unknown (likely large).
Without knowledge of
how the program
The software must show which improvements
structures and ranks
affect multiple threat-asset pairs and have the
benefits, it is impossible
greatest benefits.
to determine how difficult
it would be to select a
subset of the options.
Large.
Because the SEMS
system only calculates for
the currently selected
countermeasures and
The software does not calculate cost for each The software must calculate the benefit for
risks, redoing the
countermeasure or the benefits since the only each countermeasure improvement, the net
calculation to take into
way to calculate a lower risk is to restart the benefit, the benefit-cost ratios, and then rank account all of these
analysis.
them to determine the most effective measure. elements without
restarting the analysis will
require a fundamental
revision to the
calculations.
10 of 10
79
Table C.2 - J100-10 Standard Gap Analysis Matrix - VSAT
1
2.1, Appx C
2.1.1
2.2
2.3
2.4
2.4.1
2.4.2
2.4.3
2.4.4
2.4.5
2.5
2.6
2.7
2.8
2.9
2.10
2.11
2.12
2.13
2.14
2.15
2.16
2.17
2.18
2.19
2.2
2.21
2.22
2.23
2.24
2.25
2.26
2.27
2.28
Scope
Definitions
2.29
2.30
2.31
4.1.5
4.1.5(.1)
4.1.5(.2)
Consequence metrics
Consequence metrics
Consequence metrics
4.1.5(.3)
4.1.5(.4)
4.1.5(.5)
Consequence metrics
Consequence metrics
Consequence metrics
4.1.5(.6)
Consequence metrics
Mandatory Features
serious injuries
community
government
Notes/Comments
Yes
Feature Met
No
Incomplete*
asset
critical asset
consequence
countermeasure
detect
deter
devalue
delay
respond
dependency
dependency hazard
event tree analysis
failure mode
fault tree analysis
frequency
hazard
incident
initiating event
likelihood
preparedness
probability
proximity hazard
response
reference threat
resilience
risk
risk analysis
risk management
scenario
system
threat
threat likelihood
vulnerability
X
4.1.6
Prioritize assets
resilience.
Recommendations
Type/Size of Fix
Medium/Unknown (likely large)
fix.
The software should provide a method to calculate A combination of user interface
resilience, including the supporting user interface elements and internal functionality
elements to gather the user input required for such would be required to support
calculations.
calculation of resilience.
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
The software does not provide these
definitions.
The software must provide the user a reference or
documentation section in the program, where
definitions of these terms can be determined and
evaluated for fitness.
Medium.
Requires the creation of an
additional user interface screen,
but there is little to no
computation required to display
these definitions to the user.
X
X
X
X
X
X
X
The user can select consequences manually.
X
X
X
X
Gap
consequence.
prioritize assets.
The software must determine the worst reasonable
consequence for each asset prior to determining
Medium.
the threats that could affect the asset. This must
In order to configure which
be done to help prioritize the assets.
consequences apply to each asset
before determining which threats
apply, additional user interface
sections must be added to the Edit
Asset page.
Medium.
The Assets tab should have
The software must have a way to rank the assets in
another sub-tab that allows for a
order or their importance. This should be done
ranking of assets by importance.
after the worst reasonable consequences have been
This requires a user interface
determined for each asset.
change and a way to store user
selections in the database (in the
"assetlist" table).
1 of 6
80
Mandatory Features
Notes/Comments
Yes
4.2.3, Appx G
Characterization
Characterization
Characterization
Characterization
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.2, Appx E & G
4.2.3, Appx G
4.2.3, Appx G
4.2.3, Appx G
wildfires
Feature Met
No
Incomplete*
X
customers
transportation
X
X
suppliers
employees
proximity
Threat-Asset Pairs
X
X
X
4.2.6
4.3.2, Appx B
customers
(Appx B)
Medium.
The natural hazards editing page
(which appears for other natural
hazards) should be extended to
appear for wildfires. Since this
functionality is already in place
for other threats, extending it
should not require too much
modification.
threat section.
The software should have dependency hazards
predefined within the list of potential threats;
however, they could still be entered manually
within the user-defined section.
Medium.
Defining the list of what kinds of
dependency threats are possible
will require the creation of a new
table in the database. Then in
order to allow the user to enter a
selection from among these
choices, user interface elements in
the threat editing section should
be added to choose a value from
the database entries in that table.
Unknown (likely medium).
The most logical place in VSAT
to be able to display this ranking is
The software does not rate or rank the threat- The software must rate and rank the pairs based on in the Threats Assignment and
asset pairs based on rough magnitude of
their rough magnitude of consequences solely
Review page, having an option to
based on the possible consequences of the threat sort the entries in that list by a
on the asset.
variety of criteria, including
consequences. only caused by the
threat. However, it is unknown if
the system calculates such a value,
and if not it must be generated
before being displayed here.
top 10, top 20?)
medium).
Allowing the user to select a
subset of threat-asset pairs to
analyze would require a user
interface method for selection of a
The software does not give the option of
The user cannot remove asset-threat pairs
proceeding with an analysis of certain pairs. Must subset, as well as modification of
once they have been used in an analysis (even
be able to choose the pairs that must be analyzed the calculation procedure to take
if it's just the baseline analysis).
into account only a subset of the
for their risk.
threat-asset pairs. This
functionality would likely be
similar to or the same as the
functionality below (where the
user can select a level of
acceptable risk to filter the threatasset pairs for analysis).
X
Type/Size of Fix
X
X
4.2.4 & 4.2.5?
Recommendations
The software includes fires within the library
The software must provide a range of wildfire
of standard natural hazards but does not
magnitudes that would damage the assets.
provide a range of magnitude.
utilities
Gap
X
Small.
In order to allow for the user to
enter an estimate of the
The software must include a field to enter a single consequences to users, there must
consequence of service denial for the affected point estimate or bin number of the consequence be an additional field in the Edit
of service denial for the affected customers.
Assets page, as well as the
customers.
modification of the assetlist table
in the database to store the user
input.
2 of 6
81
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete*
Gap
Recommendations
The software should provide a field for a
descriptive analysis
4.3.3
4.4.2, A.4.4
4.4.4, Appx B
Record Estimates
consequences result
can be considered sociopolitical impacts, natural
security impacts, lost strategic
capability to cause harm or
output, detrimental effects on
brand value, public confidence,
Software must provide the ability to describe
option of including other consequences.
may use fault, event or failure analysis of each asset, and should use one of the
tree analysis, path analysis,
vulnerability logic diagrams,
computer simulation methods, simulations, or judgment rules-of-thumb.
or expert judgment rules-ofthumb
RAMCAP scales; if bins are
used, the midpoint is used for
the calculation
X
natural hazard
Unknown (likely large).
Since no vulnerability analysis
method is currently in place, the
The software must implement at least one other
analysis of the vulnerability of
method for running a vulnerability of each critical
each critical asset may require
asset.
significant reworking (of both
code and data structure) in order
to take into account another
analysis method.
X
X
vulnerability by "bubbling" the detection
(certain, probable, possible, none), delay
(very, strong, limited, no delay), and response
The software must allow for the user to have a
(fast, variable, slow, none). The software
greater degree of control over the vulnerability
then determines the likelihood as a J100-10 settings for each of these parameters.
percentage range (with rounding) and
countermeasure capability (very high, high,
moderate, low). The user can also enter
X
likelihood by bubbling very high, high,
moderate, or low. The user can also enter
Medium.
In order to allow for descriptive
analysis of an estimated
consequence, the software should
have an additional user input text
field on the Edit Asset page, as
well as a column to store that
information in the "assetlist" table
in the database.
Medium.
In order to allow for a description
of additional consequences to an
The software must add a field to record additional asset, the software needs to have
consequences for the analysis.
an additional user input text field
on the Edit Asset page, as well as
a column to store that information
in the "assetlist" table in the
database.
The software relies on the judgment of the
user and does not provide for the other
methods.
Software must provide a field for point estimates,
and the field should allow for utilization of
threat severity.
4.5, A.4.5
to region).
Type/Size of Fix
The software should allow the user to have more
fine-grained control of the likelihood of a threat
for each asset (3 choices is too granular).
medium).
For the user to be able to select
from a wider set of values for each
of these parameters, different user
interface elements should be
selected (such as a text field or
slider) that allow for a broader
range of values. Additionally,
these changes could imply that the
likelihood calculation, if it is
relying upon the few
predetermined levels set for each
parameter, will require reworking
to accept new kinds of values.
medium).
For the user to be able to select
from a wider set of values for the
threat likelihood, different user
interface elements should be
selected (such as a text field or
slider) that allow for a broader
range of values. Additionally, this
change could imply that the
likelihood calculation, if it is
relying upon the few
predetermined levels available for
each threat-asset pair, will require
reworking to accept a broader
range of values.
3 of 6
82
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete*
X
F.3.1
F.3.2
F.3.3
F.3.4
year
area classes)
Target Type (RMS target type
analysis)
Proportion: Regional Number
F.3.5
This Facility
F.3.6
F.3.7
4.5.1, Appx F
4.5.2, Appx G
Malevolent Threats
tsunami, landslide, mud slide
Natural Hazards
malevolent threats.
X
X
metro area.
V x C x Detection).
X
records.
X
X
X
Natural Hazards
Risk
X
4.5.3
4.6, A.4.6
Hazards
The software does not include a field to enter
historical data.
X
risk.
4.5.2, Appx G
The software could add proxy measures to help
determine the threat likelihood.
service denials
pair
The software can include the likelihood of the
other threats but not based on historical data.
X
The software has a tab for natural threats
where all of the asset-threat pairs of natural
The software must calculate the risk for each
hazards are displayed, but no total risk is
natural hazard and sum them to determine the
displayed. Also, the added hazards, e.g.,
overall risk due to natural hazards.
windstorms, snowstorms, are not displayed in
this tab.
The software must include historical data on
dependency and proximity hazards to determine
the likelihood that the threats will occur to the
assets.
pair.
X
Type/Size of Fix
Small/Unknown (likely medium).
While adding a user interface
element to allow for entry of a
proxy measures for vulnerability
requires just a slight change to the
UI, the changes that that entry
would make in the calculation
method of overall threat
likelihood cannot be determined.
The software should include historical data on
additional natural hazards to determine the
likelihood that the threat will occur to the assets. medium).
Additional user interface elements
will be necessary in the edit mode
of the Natural Threats page, in
addition to another column in the
"threats" table in the VSAT
database. Then, the values entered
by the user and stored in this
column will need to be taken into
account when calculating threat
likelihood for particular assets.
X
other
Natural Hazards
Recommendations
medium).
Adding the capability for users to
assign likelihood of each threat
The software allows for the option to use best The software should provide the user additional involves the addition of another
ways to estimate the odds of malevolent threats to column to the "threatasset" table
estimate or a 100% probability for all pairs
the asset.
(bubble).
in the VSAT database, along with
a user interface element to allow
users to input and store values in
the field. However, how this
likelihood will be incorporated
into calculations is unknown.
X
natural hazards.
4.5.2, Appx G
Gap
The software must use the provided consequence,
vulnerability, and threat likelihood to calculate the
The software does not calculate overall risk or owner's and community's overall risk. It also must
include duration of service denial and severity or
resiliency.
service denial (gpd) to determine the owner's and
community's overall resilience.
Unknown (likely small).
Because it is not evident from the
database structure how the risk is
calculated for each natural hazard,
it is impossible to deduce how to
arrive at a value for the overall
risk from natural hazards.
medium).
The data format of the historical
data needs to be consistent for all
entries in order for the data
entered there to be used for
calculations. If this is the case,
there must be an update to the
calculation to reflect these user
inputs.
Unknown. The software contains
no current functionality relating to
resilience, so incorporating this
could require extensive
development time.
4 of 6
83
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete*
Gap
4.6.2, Appx H
4.6.2.1
4.6.2.2
4.6.2.3
Calculate Risk
resilience
Resilience Metric
Community Economic
Resilience
risk: C x V x T = R
use midpoint of ranges from
Appx B
calculating ORI by multiplying
the indicator value by the
weight and adding all values
(should have fields for the
value and weight or by pick
box)
Software must calculate risk using the numbers input
4.6.3
The software must calculate the asset resilience.
The software must calculate the community
X
ili
it i (h
X
Software may have fields for
Software must have fields for both threat-asset pair
may require extensive
development time.
development time.
The software must calculate the owner's economic no current functionality relating to
resilience.
development time.
X
Utility Resilience Index software should calculate URI
using the values of ORI and
FRI and the weights given in
Table H-1 & H-2 (URI=ORI x
w1 + FRI x w2)
Estimates
The software includes a list of "knowledge
The software must include duration of service
base" information for resiliency within the
denial and severity or service denial (gpd) to
predefined asset list but does not use them to determine the resilience of each threat-asset pair.
calculate resiliency.
X
resilience metric using duration
and severity from 4.3,
vulnerability from 4.4 and
metric
threat likelihood from 4.5.
calculating FRI by multiplying
lost revenue due to the threat-asset pair the indicator value by the
weight and adding all values
(should have fields for the
value and weight or by pick
box)
X
Type/Size of Fix
Medium.
VSAT has already calculated the
three components of the risk
The software must use the provided consequence, calculation: consequence,
vulnerability, and threat likelihood to calculate the vulnerability, and threat
risk for each threat-asset pair.
likelihood. Because these
elements are already present,
calculating the risk should not be
difficult.
X
4.6.1
Recommendations
resilience.
development time.
Medium/Unknown (likely large).
In order to calculate the risk, the
The software must calculate and record the risk
software should use its alreadyand resilience estimates for both threat-asset pairs calculated values for consequence,
and for the overall utility.
vulnerability, and threat likelihood
and multiply them to find the
resulting risk. However, there is
no foundation (already calculated
values) upon which to begin the
calculation of resilience, so it is
difficult to gauge how much effort
such an implementation would
require.
5 of 6
84
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete*
4.7.4, A.4.7.4
Decide
Assess
are acceptable
resilience level.
The software calculates risk reduction units.
The calculation and explanation are
confusing. The calculation can only be used
for comparison purposes.
X
risk.
X
4.7.5, A.4.7.5
Identify
4.7.6, A.4.7.6
Calculate
4.7.6, A.4.7.6
Calculate
options.
benefits.
effective measures.
Type/Size of Fix
compare and determine which one benefits
the utility the most.
Medium.
VSAT has already calculated the
three components of the risk
The software must use the provided new
calculation: consequence,
consequence, vulnerability, and threat likelihood vulnerability, and threat
to calculate the new risk for each threat-asset pair. likelihood. Because these
elements are already present,
calculating the risk should not be
difficult.
The software should contain the functionality to
highlight which of the countermeasure packages
provides the greatest or greater benefits, to point
the user towards the countermeasures that would
be most cost-effective.
Small.
Assuming the benefit for each
countermeasure package is
already calculated and ranked (see
below), highlighting the highest
ranked packages should be trivial.
X
X
X
Review & Rank
to implement
4.7.7, A.4.7.7
Recommendations
Large.
Allowing the user to select a
subset of the threat-asset pairs
would best be done on the
Baseline Summary page, where
the user could set an arbitrary
acceptable risk level and see
which of the threat-asset pairs will
The software does not provide a way to set
The software must be able to record the desired
be included as unacceptable risks.
this level or rank the threat-asset pairs and the risk level and eliminate those threat-asset pairs
which have a risk that falls below this desired level In order for this to be
user is forced to include all pairs for the
implemented, a horizontal slider
from the remaining analysis.
remaining analysis.
or a dropdown box with different
levels of risk should be added to
the page and the asset-threat pairs
that are included in the analysis
will need to be indicated.
In order to modify the calculation,
the database entries that are
selected from the "threatasset"
table for the analysis will need to
be restricted based on the user
input.
X
4.7.1, A.4.7.1
Gap
Medium. Because the different
countermeasure upgrade packages
Must calculate the benefit for each
already have calculated costs,
compare the annualized cost, capital cost and countermeasure improvement, the net benefit, the creating a comparison between
these packages based on these
risk reduction units but cannot calculate net benefit-cost ratios, and then rank them to
determine
the
most
effective
measure.
calculations should not require too
benefit and benefit-cost ratios.
large of a change. Additionally,
the user interface should be
updated to display these rankings
of countermeasure improvement
packages.
6 of 6
85
Table C.3 - J100-10 Standard Gap Analysis Matrix - ARAM-W
Reference Section
1
2.1, Appx C
2.1.1
2.2
2.3
2.4
2.4.1
2.4.2
2.4.3
2.4.4
2.4.5
2.5
2.6
2.7
2.8
2.9
2.10
2.11
2.12
2.13
2.14
2.15
2.16
2.17
2.18
2.19
2.2
2.21
2.22
2.23
2.24
2.25
2.26
2.27
2.28
2.29
2.30
2.31
Scope
Definitions
Mandatory Features
hazardous sites.
Notes/Comments
Yes
Feature Met
No
Incomplete*
asset
critical asset
consequence
countermeasure
detect
deter
devalue
delay
respond
dependency
dependency hazard
event tree analysis
failure mode
fault tree analysis
frequency
hazard
incident
initiating event
likelihood
preparedness
probability
proximity hazard
response
reference threat
resilience
risk
risk analysis
risk management
scenario
system
threat
threat likelihood
vulnerability
X
Gap
Recommendations
Type/Size of Fix
The software does not calculate
resilience.
The software should provide a method to
calculate resilience, including the supporting
user interface elements to gather the user
input required for such calculations.
Large/Unknown (likely large)
fix.
A combination of user interface
elements and internal
functionality would be required
to support calculation of
resilience.
No definitions were included in the
software.
The software must provide the user a reference
or documentation section in the program, where
defitions of these terms can be determined and
evaluated for fitness.
Medium.
Requires the creation of an
additional user interface screen,
but there is little to no
computation required to display
these definitions to the user.
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
1 of 7
86
Reference Section
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete*
4.1.5(.4)
4.2.2, Appx E & G
4.2.2, Appx E & G
Consequence metrics
Characterization
Characterization
X
Gap
Recommendations
The user must select the undesired
events, (i.e. release of chemicals, loss of
power, loss of critical pump/valve
system), from a predefined event tree.
There is a separate tree for each type of
utility and users can add additional
events. The user can then determine the
consequence of each undesired event,
which are determined by assuming the
loss of the asset which leads to the
undesired event, without regard to the
threats that may cause the loss of the
asset. Although environmental impact is
not predefined, the user has the ability to
define any new consequences that they
feel apply.
Medium/unknown (likely small).
The software should provide the user a way to select
Requires the creation of a user
their own set of consequences and edit these
interface for the editing of
consequences to include custom consequences
consequences which will need to
outside of the predefined set.
be stored in the database.
The software does not include any
additional natural threats or allow any
other threats to be added.
The software must allow for the user to add
additional natural threats, as well as including
more default options to select from when
choosing a natural hazard. The most logical place
for this functionality to be implemented would be
on the "Natural Hazard Identification" screen,
where the user could select from a larger set of
options for the hazard type, as well as add more
threats.
X
wildfires
ice storms
X
X
4.2.2, Appx E & G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
4.2.3, Appx G
Characterization
Type/Size of Fix
Undefined other
utilities
suppliers
employees
customers
transportation
proximity
Undefined other
Medium/unknown (likely small).
interface for the addition of new
natural threats to the system, as
well as a way to add these threats
to the database. Additionally,
adding other natural threats to
the existing default threats
should be a very simple task
(adding a new row per threat to
the database).
X
X
X
X
X
X
threats.
X
The software must include another type of threats,
dependency threats. Adding this functionality would
The software does not include
dependency threats, the RAMCAP labels best be accomplished through the creation of another
subsection under the Threat Assessment heading in
for dependency threats, or dependency the left menu on the tab for each facility. In that new
threats within the RAMCAP reference section, the software will need to have the ability to
tables.
select the type of dependency threat, as well as enter
additional data about the dependency threat.
Large/Unknown (likely
medium)/Unknown.
Requires the creation of a new
threat type, along with all of the
associated changes required for
that change (user interfacecreation of additional screens,
database updates, calculation
updates).
2 of 7
87
Reference Section
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete*
X
4.2.5
Evaluate & Rank Threat-Asset
Pairs
X
identified in 4.1
4.3.1, Appx B
Threat Scenario
4.2.6
X
each threat.
X
and serious injuries in
excess of insurance
X
4.3.2, Appx B
Estimate Consequences loss of life to either employees represented by the RAMCAP "bins"
(Appx B)
optional single indicator reviewing bins, the values must match. Under 4.3
X
4.3.2, Appx B
X
4.3.2, Appx B
(Appx B)
owner/operator
Recommendations
Type/Size of Fix
The software does not rate or rank the
undesired event/asset location/threat
pairs based on rough magnitude of
The software should calculate and display the
magnitude of the consequences for each asset
before running the analysis. The most logical
place for this rating/ranking to be displayed is in
the Asset list under facility characterization.
Small/Unknown (likely small).
Displaying the rating on the asset
list would require a small
modification to the user
interface. However, the database
structure for ARAM-W is not
visible, so it is impossible to
determine how difficult this kind
of calculation would be.
The user cannot remove undesired
event/asset location/threat pairs once
they have been assigned in the baseline
analysis.
The software should allow for a subset of the
threat-asset pairs to be chosen from each analysis
result page in order to run another analysis. The
Analysis Calculation Mode section should allow
for the selection of which threat-asset pairs are
allowed to continue to be used in the next
analysis.
Although the consequences for the loss
of critical assets and the resulting
undesired event occurring were
previously determined, they are not
evaluated for each specific threat on the
assets.
The structure of the database
makes a large impact on the
The software should allow for the relationship
association of the threats and the
between the consequences and threats to be
consequences. Depending on
explored through the database to determine what how the existing structure is laid
out, this could be either a very
kind of a relationship exists between the two.
simple association (if the
This relationship should be displayed in the
relational structure is wellThreat Assessments section of the interface.
designed) or could require a
reworking of the database layout
to accomplish this change.
identify selected critical threat-asset pairs.
(Standard does not define critical, this is left up to
the user, i.e.. top 10, top 20?)
4.3, A.4.3
Gap
medium).
Allowing the user to select the
threat-asset pairs to use in the
calculation would require a
check box to be added for each
threat-asset pair displayed, that
can be toggled by the user if the
pair should be included in further
analyses. How the calculation
would be impacted by this
selection is unknown, however.
3 of 7
88
Reference Section
Mandatory Features
4.3.2, Appx B
customers
(Appx B)
4.3.2, Appx B
individually; converted into a single, optional single indicator value of a statistical life
the general public
(Appx B)
4.4.2, A.4.4
4.5.1, Appx F
Malevolent Threats
consequences result
diagrams, computer
Notes/Comments
Yes
Feature Met
No
Incomplete*
X
X
Gap
Recommendations
Although the consequences for the loss
of critical assets and the resulting
undesired event occurring were
previously determined, they are not
evaluated for each specific threat on the
assets.
The structure of the database
makes a large impact on the
The software should allow for the relationship
association of the threats and the
between the consequences and threats to be
consequences. Depending on
explored through the database to determine what how the existing structure is laid
out, this could be either a very
kind of a relationship exists between the two.
simple association (if the
This relationship should be displayed in the
relational structure is wellThreat Assessments section of the interface.
designed) or could require a
reworking of the database layout
to accomplish this change.
X
The user can determine the vulnerability
either by entering a user estimated single
value or by using a path analysis with the
adversary sequence diagram. The
software could better meet the Standard
by providing a means to use other
methods to analyze vulnerability.
The software should allow for the user to have
another means to analyze the vulnerability of
critical assets beside a user estimate or a path
analysis.
X
For each undesired event/asset
location/threat pair, the user can choose
to determine the likelihood of attack
using conditional, expert judgment, or a
questionnaire method. For the
conditional method, the likelihood value
is automatically considered High (or
100%) based on the assumption that the
attack will occur. For the expert
judgment method, the user can input a
likelihood of high, medium or low (drop
down menu). For the questionnaire
method, the likelihood value is
determined based on the responses to the
questions asked. The questions follow
the threat factors outlined in the
RAMCAP - The Framework document
for estimating likelihood of attack and
include initial consideration of
capability, history, current interest,
current surveillance, documented threats,
potential consequences, ideology and
ease of attack. The user answers the
different questions by clicking on the
appropriate circle and the threat potential
is calculated based on the responses to
the questions.
The software should allow the user to have a
greater degree of control when selecting the
threat likelihood. In particular, the "expert
judgment" options should be expanded to include
more than three different threat levels.
Type/Size of Fix
Unknown (likely medium). The
size of this fix depends entirely
upon the other method chosen to
analyze vulnerability. Depending
on its complexity, this could take
a significant amount of time.
malevolent threats.
Small/Unknown (likely small).
The user interface would need to
be adjusted to include more
values (probably 5 or more
differing levels of threat), and
those levels will need to be taken
into account into the calculation.
4 of 7
89
Reference Section
F.3.1
F.3.2
F.3.3
F.3.4
Feature Met
No
Incomplete*
Notes/Comments
year
area classes)
type analysis)
Number
X
Mandatory Features
Yes
X
The software must include another option for
This Facility
F.3.6
V x C x Detection).
X
X
records.
X
Natural Hazards
weather, wildfires,
avalanche, tsunami,
4.5.2, Appx G
Natural Hazards
method).
Large.
Addition of another calculation
method for the risk level will
require some considerable
development time, as well as
requiring modifications to the
database and user interface to
display the result of such a
calculation.
4.5.3
4.5.4
Record Estimates
The software does not include total risk
of natural hazards.
The software must display the total risk to the
assets from natural hazards. Ideally, this should
appear at the bottom of the Natural Hazard
Identification page, where the list of natural
hazards already resides.
Dependency and proximity threats are
not included in the software.
The software must display the predicted
likelihood of a proximity and dependency threat
for the assets. The ideal location for these to
display would be the Threat Assessment section
of the user interface.
X
X
Risk
X
Hazards
threats and does not allow for the
addition of any others.
The software must allow for a greater range of
flexibility when defining the natural threats to the
assets. In particular, the software needs the
functionality to add more natural hazards. This
functionality needs to exist in order to allow for
other threat types (e.g. a tsunami) or to allow for
threats of the same type but of different
magnitude (i.e. a hurricane of magnitude 2
should be considered differently than a hurricane
of magnitude 4). These changes should be
incorporated into the Natural Hazard
Identification section of the interface.
Medium/unknown (likely
medium) .
interface for the addition of new
natural hazards to the system, as
well as a way to add these
hazards to the database.
other
Natural Hazards
risk.
4.5.2, Appx G
The software does not include the proxy
assessing the risk level of a facility (the proxy
method.
X
natural hazards.
Type/Size of Fix
X
F.3.5
4.5.2, Appx G
Recommendations
X
metro area.
F.3.7
Gap
service denials
record the method used for making
the estimates and the estimates
ranges
X
Software must have room for selected methods to
be documented.
The user can include comments about
the threat likelihood chosen or
The software should include some form of
information about the sources used and
documentation to let the user know how the
displays the threat potential values as
questionnaire computes the threat likelihood.
none, low, medium, high, or very high in
the report.
Small.
Because the individual risk from
each natural hazard is known,
this change involves adding a
text field and displaying a sum of
all of the natural hazards in that
field.
medium).
The addition of a user interface
element to display the results of
these likelihood calculations is a
minor task, but the calculation of
the likelihoods may require more
effort.
Small.
The user interface should include
a brief description of how the
answers to the questionnaire
impact the overall computed
threat likelihood, which is a
minor user interface addition).
5 of 7
90
Reference Section
4.6, A.4.6
4.6.2, Appx H
4.6.2.1
4.6.2.2
resilience
Resilience Metric
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete*
pair.
pair
metric or holistic approach in Appx H Appx H is no mandatory
Operational Resilience
Index (ORI) calculated by
Duration x severity x vulnerability x multiplying the indicator
metric
calculating FRI by
we assume that the preferred approach is to
measure the standard in two ways: threat-asset pair
and holistic approach but both of the methods
described in the standard are no mandatory.
X
4.6.2.3
4.6.3
Estimates
4.7.2, A.4.7.2
Decide
Define
enhancements
resilience.
determine the resilience of each threat-asset pair.
Unknown.
The software contains no current
development time.
X
resilience.
resilience.
X
Unknown.
The software must include duration of service denial
and severity of service denial (gpd) to determine the
resilience of each threat-asset pair.
development time.
The software must include duration of service denial
and severity of service denial (gpd) to determine the
no current functionality relating
resilience of each threat-asset pair.
to resilience, so incorporating
this could require extensive
development time.
Unknown.
determine the resilience of each threat-asset pair. resilience, so incorporating this
development time.
resilience.
Unknown.
determine the resilience of each threat-asset pair. resilience, so incorporating this
development time.
y
software should calculate
X
X
4.7.1, A.4.7.1
Type/Size of Fix
X
Software must have fields for both threat-asset pair
Software may have fields for resilience and holistic resilience.
are acceptable
Recommendations
resilience.
X
Community Economic
Resilience
Gap
X
The software displays the overall risk
and the risk for each undesired
event/asset location/threat pair as either
none, low, medium, high, or very high.
However, resilience is not calculated or
displayed.
Unknown.
determine the resilience of each threat-asset pair.
The software includes a field to
determine if the risk for each undesired
event/asset location/threat pair is
acceptable (check box). The software
does not include resilience in this
decision.
Unknown.
determine the resilience of each threat-asset pair. functionality relating to
development time.
The user can create upgrade packages
and develop them by redoing the ASD
analysis with the inclusion of more
countermeasures and other
enhancements. The user can also create denial and severity of service denial (gpd) to
upgrade packages which only affect the determine the resilience of each threat-asset pair.
consequences which can be entered. The
software does not include resilience
enhancement measures.
development time.
Unknown.
development time.
6 of 7
91
Reference Section
Mandatory Features
Notes/Comments
Yes
Feature Met
No
Incomplete*
X
4.7.4, A.4.7.4
Assess
option was implemented; calculate
the estimated benefits of the option
risk.
Gap
The software calculates the new risk of
each pair for each upgrade package. The
software displays a report of the baseline The software must include duration of service
data and upgrade packages for the
highest risk threat-asset location pair. It determine the resilience of each threat-asset pair.
does not calculate benefits of the options
or include resilience.
4.7.5, A.4.7.5
Identify
multiple packages. The preferred way to indicate
this would be a highlighting of the improvements
that have been included in multiple
countermeasure packages.
options.
X
4.7.6, A.4.7.6
4.7.7, A.4.7.7
Calculate
Calculate
Review & Rank
to implement
4.7.9, A.4.7.8
The software must calculate the net benefit of
each countermeasure package and display it to
The software can compare packages but
the user. This information should appear on the
there are no calculations for net benefit. Upgrade Packages page as an additional field for
each package.
benefits.
X
The software does not calculate net
benefit, nor does it compare the benefit
to the cost.
X
The software does not consider the most
effective package, only the packages and
their affect on each pair, especially the
highest risk pair.
effective measures.
Type/Size of Fix
Unknown.
development time.
compare but the software does not
The software should make a visible distinction
determine which improvements are
Medium.
for the improvements that have been placed into
Searching through the packages
included in multiple packages.
X
pairs
Recommendations
(in the database) for their
included improvements will yield
the list of packages that contain
the same improvements, and
these packages can be displayed
with a small marker next to them
to highlight that fact.
Small.
The database contains the
information pertaining to the
benefit for each improvement in
the package, so determining the
sum and outputting it for the user
involves little calculation and
addition of a user interface
element to display the output.
The software must calculate the benefit-cost ratio
for each countermeasure upgrade package and
display it to the user. The most appropriate place
for this to occur is in the "Upgrade Packages"
page. For each entry in the list of upgrade
packages, the software must display its benefits
and cost so the user can select the most
appropriate upgrade package.
Small.
Assuming the above net benefit
for each package has been
calculated, the cost for each
improvement package is known
and the ratio is a simple
calculation that can then be
displayed to the user with a user
interface element.
Assuming that the net benefit has been
determined (see above), the software should use
that information, along with the cost of each
upgrade package, to determine the costeffectiveness of each package. This costeffectiveness should be displayed for each
package on the "upgrade packages" page.
Small.
The cost-benefit ratio is
equivalent to the cost
effectiveness, and so finding the
most effective package involves
selecting the package with the
(already calculated above)
highest cost-benefit ratio.
7 of 7
92
Gap Assessment for ASME-ITI/AWWA J100-10 Standard and Leading Vulnerability Assessment Tools
6666 West Quincy Avenue, Denver, CO 80235-3098 USA
P 303.347.6100 • F 303.734.0196 • www.WaterResearchFoundation.org
4358
WEB-ONLY-4358-04/11-RF

Gap Assessment for ASME-ITI/AWWA J100-10 Standard

Transcription

Similar documents

Just for Parents 2015 - University Counseling Center

PDF Version

The Professional Resilience Paradigm

PDF - European Defence Agency

The Psychology of the Sports Champion: What It Could Mean to

Resilience - NRN-LCEE

2014 Health Industry Threat Landscape Briefing