Here - Hydro

Transcription

Here - Hydro

Cisco and Hydro Québec
Use cases proposal for IEC 61850-90-12
Faramarz Maghsoodlou, Ph. D.
IoT Connected Industries & Energy Practice
Advanced Services
Cisco Systems, Inc.
Jean Raymond, ing., Ph. D., M. Sc. A.
Evolution du Réseau IP, STAR
Direction Ingénierie et Solutions de Télécommunications
Direction Principale Télécommunications - Groupe Technologie
Hydro-Québec
August 26, 2014
Date August 26, 2014
Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
1
Table of Contents
1.
Preface....................................................................................................................................... 5
2.
Introduction ............................................................................................................................... 6
3.
Evolution of the Utility Telecom Networks.................................................................................. 9
3.1
3.2
3.3
3.4
3.5
3.6
3.7
4.
Transmitting Data over Multiple Media .............................................................................................9
Changing and Growing with the Industry ......................................................................................... 10
Connecting Large Number of Devices .............................................................................................. 10
Maintaining Reliability .................................................................................................................... 10
Connecting Multiple Types of Systems ............................................................................................. 10
Ensuring Security ............................................................................................................................ 11
Providing Smooth Migration............................................................................................................ 11
Use Cases ................................................................................................................................. 13
4.1 Transmission Use Cases ................................................................................................................... 16
4.1.1 Tele-Protection ................................................................................................................................... 16
4.1.2 Inter-Trip Protection Scheme ............................................................................................................. 19
4.1.3 Current Differential Protection Scheme ............................................................................................. 23
4.1.4 Distance Protection Scheme ............................................................................................................... 25
4.1.5 Inter-Substation Protection Signaling ................................................................................................. 27
4.1.6 Intra-Substation Process Bus Communication.................................................................................... 30
4.1.7 Control Center Monitoring of Station Bus .......................................................................................... 33
4.1.8 Communication System Failure & Degradation Monitoring .............................................................. 35
4.1.9 High Voltage Substation SCADA RTU .................................................................................................. 38
4.1.10 Medium Voltage Substation SCADA RTU.......................................................................................... 41
4.1.11 Condition-Based Monitoring ............................................................................................................ 44
4.1.12 Transformer Dissolved Gas Analysis ................................................................................................. 47
4.1.13 Gas-Insulated Switchgear Health Monitoring .................................................................................. 49
4.1.14 Dynamic Asset Rating ....................................................................................................................... 51
4.1.15 Wide Area Monitoring & Control Systems ....................................................................................... 54
4.1.16 Substation Environmental Monitoring ............................................................................................. 58
4.1.17 Power System State Estimation........................................................................................................ 61
4.1.18 Inter Control Center Communication (ICCP) .................................................................................... 64
4.1.19 RTU File Transfer............................................................................................................................... 68
4.1.20 Weather Monitoring Stations ........................................................................................................... 70
4.2 Distribution .................................................................................................................................... 72
4.2.1 Power Quality Monitoring System – PQMS ........................................................................................ 72
4.2.2 Sensors in the Distribution Grid .......................................................................................................... 75
2
4.2.3 Pole-Top Voltage Regulators & Capacitor Bank Monitoring .............................................................. 76
4.2.4 Remote Control of Overhead or Underground Switches ................................................................... 79
4.2.5 Distribution Feeder Voltage Regulation ............................................................................................. 81
4.2.6 Distribution Volt / VAR Optimization.................................................................................................. 84
4.2.7 Substation Capacitor Monitoring & Control ....................................................................................... 87
4.2.8 Underground Cable Distributed Temperature Monitoring ................................................................ 88
4.2.9 Online Transformer Condition Monitoring ......................................................................................... 92
4.2.10 Switchgear & Transformer Partial Discharge Monitoring ................................................................ 94
4.2.11 Fault Location Isolation & Service Restoration (FLISR) ..................................................................... 96
4.2.12 Voltage Regulation............................................................................................................................ 99
4.2.13 Advanced Metering Infrastructure (AMI) ....................................................................................... 103
4.2.14 Integrated Grid-Scale Energy Storage ............................................................................................ 106
4.3 Energy Supply Use Cases ............................................................................................................... 109
4.3.1 Frequency Control / Automatic Generation Control (AGC) .............................................................. 109
4.3.2 Hydroelectric Power Plants – General Telemetry............................................................................. 112
4.3.3 Hydro Plant Dam Leakage Monitoring .............................................................................................. 115
4.3.4 Hydro Plant Gate Position Indicator ................................................................................................. 117
4.3.5 Hydro Plant Water Flow Control ....................................................................................................... 119
4.3.6 Hydro Plant Water Level Indicator.................................................................................................... 121
4.3.7 Hydro Plant Dam Over-Topping Protection ...................................................................................... 123
4.3.8 Hydro Plant Turbine Vibration Monitoring ....................................................................................... 125
4.3.9 Hydro Plant Dam Deformation Monitoring ...................................................................................... 126
4.3.10 Wind Farm Operation ..................................................................................................................... 127
4.3.11 Wind Farm Maintenance ................................................................................................................ 130
4.4 Extreme Contingencies .................................................................................................................. 133
4.4.1 H-Q Transmission System Characteristics......................................................................................... 133
4.4.2 Extreme Contingencies Criteria ........................................................................................................ 134
4.4.3 Special Protection Schemes (SPS) ..................................................................................................... 136
4.4.4 Telecom Requirements for Extreme Contingencies ......................................................................... 138
4.4.5 NPCC requirements for SPS Communication Systems...................................................................... 139
4.5 Demand Side ................................................................................................................................ 141
4.5.1 Hour-Ahead Load Optimization – Demand Response ...................................................................... 141
4.5.2 Electric Vehicle Charging................................................................................................................... 143
4.5.3 Automated Demand Response with Water Heaters ........................................................................ 145
4.5.4 Customer Premise Network Integration ........................................................................................... 150
4.6 Mobility & Collaboration ............................................................................................................... 151
4.6.1 Field Workforce Voice over IP (VoIP) ................................................................................................ 151
4.6.2 Workforce Video ............................................................................................................................... 153
4.6.3 Radio over IP (RoIP) .......................................................................................................................... 155
4.6.4 Substation Worker Access to Corporate Applications ...................................................................... 158
4.7 Physical Safety & Security ............................................................................................................. 161
4.7.1 Electronic Access Control .................................................................................................................. 161
4.7.2 Video Monitoring & Surveillance ...................................................................................................... 164
4.7.3 Remote Fire Alarms Monitoring ....................................................................................................... 167
3
4.8 Other Use Cases ............................................................................................................................ 169
4.8.1 Generic Network Management Use Case ......................................................................................... 169
4.8.2 Precision Time Protocol (PTP) ........................................................................................................... 172
5.
Network Requirements Heat Map & Dependency ................................................................... 173
5.1
5.2
6.
Network Requirements Heat Map – By Use Case ........................................................................... 174
Network Dependency Matrix – By Use Case ................................................................................... 176
Communication Trends & Best Practices ................................................................................. 178
6.1 General communication Requirements .......................................................................................... 178
6.2 Migration to Packet-Switched Network ......................................................................................... 182
6.3 MPLS technology .......................................................................................................................... 183
6.3.1 Network Virtualization ...................................................................................................................... 183
6.3.2 Support for Existing Networks & Interoperability ............................................................................ 184
6.3.3 Security in MPLS................................................................................................................................ 184
6.3.4 Utility-Grade Performance................................................................................................................ 184
6.3.5 IP/MPLS and MPLS-TP for the WAN ................................................................................................. 185
6.3.6 The Cost Efficiencies of MPLS ........................................................................................................... 185
6.4 IP Address Planning & Management .............................................................................................. 186
6.4.1 Overview ........................................................................................................................................... 186
6.4.2 IPv6 Considerations .......................................................................................................................... 187
6.4.3 Critical Issues in IP Address Assignment ........................................................................................... 187
7.
Security Trends & Best Practices ............................................................................................. 188
7.1
7.2
7.3
7.4
Current Practices & Their Limitations ............................................................................................. 188
Security Trends in Utility Networks................................................................................................ 189
Regulatory Compliance (NERC CIP) ................................................................................................ 190
General Security Requirements ..................................................................................................... 193
8.
Bibliography ........................................................................................................................... 197
9.
Glossary ................................................................................................................................. 198
4
1. Preface
The use cases that are described in this document represent an abridged version of the
content that was developed by Faramarz Maghsoodlou of Cisco Systems’ Advanced
Services, in collaboration with Jean Raymond of Hydro Québec (H-Q), as part of H-Q’s
effort to prepare for, plan, and deploy the next generation of its IP/MPLS
telecommunications network.
Cisco and Hydro Québec have jointly decided to make this document available, in a
limited distribution, to the IEC 61850-90-12 committee members to assist in their
collective work and deliberations for developing the relevant IEC standards.
The content of this document, in its entirety, including methods, diagrams, figures, and
descriptions, is the intellectual property of Cisco Systems, Inc. and it is not intended for
public distribution. Its use and distribution is limited to the collective work of the IEC
committee members for the purpose of standards development.
References to this document should be cited as:
Faramarz Maghsoodlou and Jean Raymond, “Cisco and Hydro-Québec Use cases
proposal for IEC 61850-90-12, August 2014.”
5
2. Introduction
The business and technology trends that are sweeping the utility industry will drastically transform the utility
business from the way it has been for many decades. At the core of many of these changes is a drive to
modernize the electrical grid with an integrated communications infrastructure. At Hydro-Québec, the Direction
Principale des Télécommunications group (DPT) launched a private MPLS network design and implementation
project a few years ago to provide H-Q with advanced network services. This project and this document are a
part of that effort.
This document consists of 9 chapters that cover the following topics:
•
•
•
•
•
•
•
•
•
Introduction
Evolution of the Utility Telecom Networks
Use Cases that represent eight utility operational domains
Network Requirements Heat Map & Dependencies matrix
General Communication Trends & Best Practices
General Security Trends & Best Practices
Smart Grid Standards
Bibliography
Glossary
Given the range and diversity of the requirements that should be addressed by the next generation
telecommunications infrastructure, the approach that we adopted in this project is to document the
telecommunication requirements based on a wide range of operational use cases that reflect the current and
future needs of the generation, transmission, and distribution business. These future operational needs are
themselves a reflection of the business and technology trends that are shaping the industry.
We therefore started by reviewing the top 15 industry trends and explored their relevance to Hydro Québec.
Top Business & Technology Trends
IT-OT Convergence
Field Area Network
Advanced Metering Infrastructure
Mobility & Mobile Technologies
Business Intelligence & Big Data Analytics
Social Computing
Electric Vehicles
XaaS and Cloud Computing
Figure 1.
Energy Storage
Distributed Intelligence
Transactive Energy
Aging Workforce
Aging Assets
Infrastructure Security
Renewable Energy
Top Energy Industry Busienss & Technology Trends
6
For each trend we describe the business and technology drivers and point out challenges that utilities face in
responding to these changes. We offer recommendations on how Hydro Québec could address these challenges
based on their relevance and level of impact on H-Q’s operations and business model.
The following figure shows the impact of these trends on H-Q on a variable scale of 0 to 5, with 0 indicating no
impact and 5 indicating a high level of impact.
IT-OT Convergence
5
Renewable Resources
Field Area Network
4
Advanced Metering
Infrastructure (AMI)
Infrastructure Security
3
2
Aging Assets
Mobility & Mobile
Technologies
1
0
Business Intelligence (BI)
& Big Data Analytics
Aging Workforce
Transactive Energy
Social Computing
Distributed Intelligence
Energy Storage
Figure 2.
Electric Vehicles
XaaS and Cloud
Computing
Top Industry Trends & Their Relevance to Hydro Québec
To meet this diverse set of requirements, both today and in the future, it is imperative that the next generation
utility telecom network will be based on open-standards-based IP architecture.
The future end-to-end IP/MPLS architecture will enable Hydro Québec to:
•
•
•
•
•
•
•
Facilitate interoperability across disparate networks and devices
Support data transmission over diverse media
Connect large number of devices
Maintain reliability
Connect multiple types of systems
Ensures security, and
Change and grow with the industry.
7
Focusing on the operational requirements, our starting point is set of well-defined use cases that are grouped
into the following operational domains:
Transmission
Distribution
Energy Supply
Extreme Contingencies
Figure 3.
Operational Domains
Demand Side
Mobility & Collaboration
Physical Safety & Security
Other
Operational Domains for Use Case Classification
8
3. Evolution of the Utility Telecom Networks
The business and technology trends that are sweeping the utility industry will drastically transform the utility
business from the way it has been for many decades. At the core of many of these changes is a drive to
modernize the electrical grid with an integrated communications infrastructure. However, interoperability,
concerns, legacy networks, disparate tools, and stringent security requirements all add complexity to grid
transformation. Given the range and diversity of the requirement that should be addressed by the next
generation telecommunications infrastructure utilities need to adopt a holistic architectural approach to
integrate the electrical grid with digital communication across the entire power delivery chain.
Many utilities still rely on complex environments formed of multiple application-specific, proprietary networks.
Information is siloed between operational areas. This prevents utility operations from realizing the operational
efficiency benefits, visibility, and functional integration of operational information across grid applications and
data networks. The key to modernizing grid communications is to provide a common, multi-service network
infrastructure for the entire utility organization. Such a network serves as the platform for current capabilities
while enabling future expansion of the network to accommodate new applications and services.
To meet this diverse set of requirements, both today and in the future, the next generation utility telecom
network will be based on open-standards-based IP architecture. An end-to-end IP architecture takes advantage
of nearly three decades of IP technology development, facilitating interoperability across disparate networks
and devices, as it has been already demonstrated in many mission-critical and highly secure networks.
It is imperative that utilities participate in standards development bodies to influence the development of future
solutions and to benefit from shared experiences of other utilities and vendors.
In the following sections we indicate how IP can meet every challenge that utilities will face in their grid
modernization journey.
3.1 Transmitting Data over Multiple Media
Utility operational data Smart grid data must be able to travel rapidly and reliably over a variety of different
network media, from copper cables to fiber infrastructure to wireless networks.
IP can run over any link layer network, including Ethernet, wireless radio networks, and serial lines, providing a
common and flexible way to use and manage a network composed of disparate parts.
9
3.2 Changing and Growing with the Industry
The electric power system will evolve as technological advances yield new hardware, applications, and devices.
At the same time, the utility must incorporate such advances into the network with minimal cost and difficulty.
One of the principal benefits of IP is its ability to add a capability such as a new application without having to
change IP itself. A good analogy is a highway and cars: car designs change constantly in response to emerging
consumer demands, but nonetheless can still use the same roads and traffic management. That is why IP can
run applications it was not originally designed to support, such as secure Internet commerce, voice,
collaboration, and Web 2.0 applications. And just as highways are designed to support traffic for the next 100
years, IP will be able to support new applications as they are developed for decades to come.
3.3 Connecting Large Number of Devices 1
The telecom infrastructure must enable communication and correlation of data from potentially thousands of
substations and many millions of grid and consumer devices.
One of the main challenges with connecting large numbers of devices is providing a unique identifier, or address,
for each device. Unlike the many architectures that went before it, IPv6 offers straightforward addressing and
routing for a huge network such as the future utility telecom network.
3.4 Maintaining Reliability
High network availability is absolutely critical. Network outages are costly and debilitating – and unfortunately
all too frequent these days. Ensuring uninterrupted electrical service to ratepayers is a prime challenge for any
utility. Therefore, ensuring that the utility telecom network is reliable, so that it in turn can ensure
uninterrupted electrical service to ratepayers, is crucial.
IP already has more tools and applications to help manage the network and maintain reliability than any other
communication protocol.
3.5 Connecting Multiple Types of Systems
The utility communication network must connect and exchange data freely with many different types of
hardware, ranging from smart sensors in home appliances to home energy meters to transformers and beyond.
1
A Standardized and Flexible IPv6 Architecture for Field Area Networks
10
IP is device independent. This means that it can identify any type of system to which data is addressed and
deliver it to its destination. IP can also identify the system from which the data came, so it enables the receiving
device to respond back to the sending device to let it know the data has arrived.
3.6 Ensuring Security
The unfortunate reality is that because of the critical nature of the technology and the services it provides, the
grid becomes a prime target for acts of terrorism and cyber attacks. The transformation of traditional energy
networks to smart grids requires an intrinsic security strategy and specific security mechanisms to safeguard this
critical infrastructure.
IP is as secure as you want to make it. Although IP was designed to be open and flexible, over the years more
and more tools have been built to provide security in the communications that travel over an IP network. In fact,
of all communications protocols, IP has the most tools for securing and managing the transport of data.
Therefore, while all the communications systems in the network will be able to utilize IP as a communications
pipeline, IP has state-of-the-art tools to ensure the information travels as privately as needed, sending the
information to the right destination while ensuring that it is not intercepted or accessed by unauthorized users.
IP is able to provide security on both public and private networks, and today many industries transmit their
communications over both these types of networks. For example, some parts of financial networks are public as
well as very secure, such as the retail banking section, while at the same time, many other areas of financial
networks are completely private. All of these networks utilize IP as their foundation.
Many industries with exacting security standards have embraced IP, despite initial reservations. For example,
governments, militaries, service providers for both voice and cable services, telecommunications providers, and
mainframe computer utilities were at first concerned about using IP for their operations, fearing security risks.
Now all of these industries use IP as their communications foundation. IP has adapted to meet the stringent
requirements of their networks, especially in the area of security.
3.7 Providing Smooth Migration
Utilities must be able to migrate from their current disjointed data communications networks to converged
networks in a phased approach with minimal service effect.
IP provides a way to migrate in phases from multiple monitoring and control networks to a single converged
network without disrupting service. This enables utilities to receive all the benefits of IP without having to
undergo a massive “forklift” implementation. The steps to the convergence are:
•
Encapsulation – Legacy non-routable data communications protocols are encapsulated in an IP
“wrapper,” which can then be routed over an IP network. One way to accomplish this is bisync serial
tunneling, or BSTUN, a protocol originally designed to facilitate migration from mainframe System
Network Architecture (SNA) networks to IP. Not only is this method an effective first step in IP
11
migration, it also emphasizes IP’s proven track record of flexibility and reliability. However,
encapsulation does not offer the kind of end-to-end manageability and high performance afforded by
native IP, but it is a relatively straightforward and easy to implement first step.
•
Gateways – Protocol translation devices, called gateways, are installed between legacy networks and
the IP network. The gateway maps the legacy protocol functions to IP functions. Protocol translation is
not a format conversion operation. Rather, it is similar to translating between two human languages—
not every word or phrase in one language has an equivalent in the other. Some words and phrases
cannot be translated at all, so the translator simply does its best to supply as close a translation as
possible. Therefore, like encapsulation, gateways are a useful but temporary migration step.
•
Native IP – The ultimate goal of migration is a native IP network. A native IP network provides the endto-end robust security and outstanding manageability discussed above, along with quality of service,
redundancy, scalability, and adaptability. A native IP network also delivers benefits of lower operational
expense due to easier implementation and streamlined management.
12
4. Use Cases
The list of use cases is divided into two categories of in-scope and out-of-scope based on their relevance to
Hydro Québec’s business and operational requirements.
The following tables summarize the in-scope use cases in the respective business categories.
For each use case we show three timelines:
1. The rollout timeline of the functionality at Hydro Québec with any communication technology
2. The anticipated rollout timeline of the functionality at Hydro Québec with IP / MPLS technology2
3. The anticipated rollout timeline of the functionality at industry-leading utilities
Rollout Timeline
Hydro Québec
With Any Technology With IP / MPLS
Transmission Use Cases
Tele-Protection
Inter-Trip Protection Scheme
Current Differential Protection Scheme
Distance Protection Scheme
Inter-Substation Protection Signaling
Process Bus Communication
Control Center Monitoring of Station Bus
Communication System Failure & Degradation Monitoring
High Voltage Substation SCADA RTU
Medium Voltage Substation SCADA RTU
Condition-Based Monitoring
Transformer Dissolved Gas Analysis
Gas-Insulated Switchgear Health Monitoring
Dynamic Asset Rating
Wide Area Monitoring & Control Systems
Substation Environmental Monitoring
Power System State Estimation
Inter Control Center Communication (ICCP)
RTU File Transfer
Weather Monitoring Stations
Figure 4.
2
Currently Deployed
Currently Deployed
Currently Deployed
Currently Deployed
0 - 5 Years
5 - 10 Years
5 - 10 Years
Currently Deployed
Currently Deployed
Currently Deployed
0 - 5 Years
0 - 5 Years
0 - 5 Years
Currently Deployed
0 - 5 Years
Currently Deployed
Currently Deployed
Currently Deployed
Currently Deployed
0 - 5 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
0 - 5 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
Industry Leaders
With IP / MPLS
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
Currently Deployed
Currently Deployed
Currently Deployed
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
Currently Deployed
Currently Deployed
0 - 5 Years
Currently Deployed
0 - 5 Years
0 - 5 Years
Transmission Use Cases Rollout Timeline
This rollout timeline with IP/MPLS can vary based on the orientations and the priorities of the utility.
13
Rollout Timeline
Hydro Québec
Distribution Use Cases
Power Quality Monitoring System – PQMS
Sensors in the Distribution Grid
Pole-Top Voltage Regulators & Capacitor Bank Monitoring
Remote Control of Overhead or Underground Switches
Distribution Feeder Voltage Regulation
Distribution Volt / VAR Optimization
Substation Capacitor Monitoring & Control
Underground Cable Distributed Temperature Monitoring
Online Transformer Condition Monitoring
Switchgear & Transformer Partial Discharge Monitoring
Fault Location Isolation & Service Restoration (FLISR)
Voltage Regulation
Advanced Metering Infrastructure (AMI)
Integrated Grid-Scale Energy Storage
Figure 5.
Rollout Timeline
Hydro Québec
Frequency Control / Automatic Generation Control (AGC)
Hydroelectric Power Plants – General Telemetry
Hydro Plant Dam Leakage Supervision
Hydro Plant Gate Position Indicator
Hydro Plant Water Flow Control
Hydro Plant Water Level Indicator
Hydro Plant Dam Over-Topping Protection
Hydro Plant Turbine Vibration Monitoring
Hydro Plant Dam Deformation Monitoring
Wind Farm Operation
Wind Farm Maintenance
Figure 6.
Extreme Contingencies Use Cases
Currently Deployed
Currently Deployed
Currently Deployed
Currently Deployed
Currently Deployed
Currently Deployed
Currently Deployed
Currently Deployed
Currently Deployed
Currently Deployed
5 - 10 Years
5 - 10 Years
Currently Deployed
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
Currently Deployed
0 - 5 Years
Industry Leaders
With IP / MPLS
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
Distribution Use Cases Rollout Timeline
Rollout Timeline
Hydro Québec
Currently Deployed
Figure 7.
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
Distribution Use Cases Rollout Timeline
Energy Supply Use Cases
Special Protection Schemes
0 - 5 Years
5 - 10 Years
5 - 10 Years
0 - 5 Years
Currently Deployed
Currently Deployed
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
0 - 5 Years
Currently Deployed
Currently Deployed
0 - 5 Years
Industry Leaders
With IP / MPLS
Industry Leaders
With IP / MPLS
5 - 10 Years
5 - 10 Years
Extreme Contingencies Use Cases Rollout Timeline
14
Rollout Timeline
Hydro Québec
Demand Side Use Cases
Hour-Ahead Load Optimization – Demand Response
Electric Vehicle Charging
Demand Response with Water Heaters Using OpenADR2
Customer Premise Network Integration
Figure 8.
Field Workforce Voice over IP (VoIP)
Workforce Video
Radio over IP (RoIP)
Substation Worker Access to Corporate Applications
Figure 10.
5 - 10 Years
5 - 10 Years
0 – 5 Years
5 - 10 Years
Rollout Timeline
Hydro Québec
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
0 - 5 Years
Industry Leaders
With IP / MPLS
Currently Deployed
0 - 5 Years
Currently Deployed
Currently Deployed
Mobility & Collaboration Use Cases Rollout Timeline
Physical Safety & Security Use Cases
Electronic Access Control
Video Monitoring & Surveillance
Remote Fire Alarms Monitoring
5 - 10 Years
5 - 10 Years
5 - 10 Years
5 - 10 Years
Demand Side Use Cases Rollout Timeline
Mobility & Collaboration Use Cases
Figure 9.
5 - 10 Years
0 - 5 Years
0 - 5 Years
5 - 10 Years
Industry Leaders
With IP / MPLS
Rollout Timeline
Hydro Québec
Currently Deployed
Currently Deployed
Currently Deployed
0 - 5 Years
0 - 5 Years
0 - 5 Years
Industry Leaders
With IP / MPLS
Currently Deployed
Currently Deployed
Currently Deployed
Physical Safety & Security Use Cases Rollout Timeline
15
4.1 Transmission Use Cases
This section includes use cases related to power transmission including protection, substation automation and
centralized grid monitoring and control through EMS/SCADA.
4.1.1 Tele-Protection
The key criteria for measuring Teleprotection performance are command transmission time, dependability and
security. These criteria are defined by the IEC standard 60834 as follows:
•
•
•
Transmission time (Speed): The time between the moment where state changes at the transmitter input
and the moment of the corresponding change at the receiver output, including propagation time.
Overall operating time for a Teleprotection system includes the time for initiating the command at the
transmitting end, the propagation time over the communications link and the selection and decision
time at the receiving end, including any additional delay due to a noisy environment.
Dependability: The ability to issue and receive valid commands in the presence of interference and/or
noise, by minimizing the probability of missing command (PMC). Dependability targets are typically set
for a specific bit error rate (BER) level.
Security: The ability to prevent false tripping due to a noisy environment, by minimizing the probability
of unwanted commands (PUC). Security targets are also set for a specific bit error rate (BER) level.
Additional key elements that may impact Teleprotection performance include bandwidth rate of the
Teleprotection system and its resiliency or failure recovery capacity. Transmission time, bandwidth utilization
and resiliency are directly linked to the communications equipment and the connections that are used to
transfer the commands between relays.
4.1.1.1 Latency Budget Considerations
Delay requirements for utility networks may vary depending upon a number of parameters, such as the specific
protection equipment used. Most power line equipment can tolerate short circuits or faults for up to
approximately five power cycles before sustaining irreversible damage or affecting other segments in the
network. This translates to total fault clearance time of 100ms. As a safety precaution, however, actual
operation time of protection systems is limited to 70- 80 percent of this period, including fault recognition time,
command transmission time and line breaker switching time. Some system components, such as large
electromechanical switches, require particularly long time to operate and take up the majority of the total
clearance time, leaving only a 10ms window for the communications part of the protection scheme,
independent of the distance to travel. Given the sensitivity of the issue, new networks impose requirements that
are even more stringent: IEC standard 61850 limits the transfer time for protection messages to 1⁄4 - 1⁄2 cycle
or 4 - 8ms (for 60Hz lines) for the most critical messages.
16
The following diagram shows the latency budget for a fault clearing time of a protection system, divided by the
different actors involved.
Figure 11.
Tele-Protection Clearance Time
As it can be seen, most of the delay is caused by the electromechanical actions on the circuit breaker itself,
leaving a very short delay budget for the communications portion.
4.1.1.2 Asymmetric Delay
In addition to minimal transmission delay, a differential protection communication channel must be
synchronous, i.e., experiencing symmetrical channel delay in transmit and receive paths. This requires special
attention in jitter-prone packet networks. While optimally Teleprotection systems should support zero
asymmetric delay, typical relays can tolerate discrepancies of up to 250µs. H-Q’s existing relays can tolerate up
to 750µs.
The main tools available for lowering delay variation below this threshold are:
•
•
A jitter “buffer” at the multiplexers on each end of the line can be used to offset delay variation by
queuing sent and received packets. The length of the queues must balance the need to regulate the rate
of transmission with the need to limit overall delay, as larger buffers result in increased latency. This is
the old TDM traditional way to fulfill this requirement.
Traffic management tools ensure that the Teleprotection signals receive the highest transmission
priority and minimize the number of jitter addition during the path. This is one way to meet the
requirement in IP networks.
17
•
Standard Packet-Based synchronization technologies, such as 1588-2008 Precision Time Protocol (PTP)
and Synchronous Ethernet (Sync-E), can help maintain stable networks by keeping a highly accurate
clock source on the different network devices involved.
4.1.1.3 Legacy Teleprotection vs. Next generation Teleprotection Requirements
The following table captures the main differences between legacy teleprotection and next generation
teleprotection requirements. The next generation teleprotection will be based on the IEC 61850 standard.
Network Requirements
Client Interfaces
One Way Delay
Delay Symmetry Required
Jitter
Layer 2 or Layer 3
VPN Topology
Multicast
Availability / Reliability
Precise Timing Required
Link / Node Failure Delay
Performance Management
Legacy Teleprotection
C37.94, X.21, E&M, G.703, T1/E1, V.35, RS422
4 to 10ms
Yes
<750µs
Layer 2 pseudowire
P-to-P
No
99.9999 / High
Yes
<50ms – Hitless
Yes, Mandatory
Figure 12.
Next Generation Teleprotection
Ethernet
4 - 10ms
Yes
<250µs
Layer 2 pseudowire
P-to-P, P-to-MP
No
99.9999 / High
Yes
<50ms – Hitless
Yes, Mandatory
Legacy vs. Next generation tele-protection
4.1.1.4 NPCC Requirements for Communication Systems
NPCC has established specific requirements for the communication systems that are required for teleprotection.
According to the NPCC Directory #4 3:
•
•
•
3
All elements of the bulk power system shall be protected by two protection groups, each of which is
independently capable of performing the specified protective function for that element. This
requirement also applies during energization of the element. This last sentence means that all
communications have to be operational and running for generators, transformers, and other equipment
in the crank path so the system can be energized from cold start.
Communication facilities required for teleprotection shall be designed to have a level of performance
consistent with that required of the protection system
The two protection groups shall not share the same component.
NPCC Regional Reliability Reference Directory # 4 – Bulk Power System Protection Criteria
18
•
•
•
Where each of the two protection groups protecting the same bulk power system element requires a
communication channel, the equipment and channel for each protection group shall be separated
physically and designed to minimize the risk of both protection groups being disabled simultaneously by
a single event or condition
Teleprotection equipment shall be monitored to detect loss of equipment and/or (communication)
channels to allow prompt attention by the appropriate operating authorities
The two teleprotection groups shall not share the same component. However, the use of a single
communication tower for the radio communication systems used by two protection groups protecting a
single element is permitted as long as directional diversity of the communication signals is achieved.
4.1.2 Inter-Trip Protection Scheme
Inter-tripping is the controlled tripping of a circuit breaker so as to complete the isolation of a circuit or piece of
apparatus in concert with the tripping of other circuit breakers. The main use of such schemes is to ensure that
protection at both ends of a faulted circuit will operate to isolate the equipment concerned.
Inter-tripping schemes use signaling to convey a trip command to remote circuit breakers to isolate circuits.
Main types of protection included in this category are breaker failure and differential transformer protection.
Three types of Inter-tripping are commonly encountered, and are described below:
1. Direct Tripping (Transfer Tripping)
In direct tripping applications, inter-trip signals are sent directly to the master trip relay. Receipt of the
command causes circuit breaker operation. The method of communication must be reliable and secure
because any signal detected at the receiving end will cause a trip of the circuit at that end. The
communications system design must be such that interference on the communication circuit does not
cause spurious trips. Should a spurious trip occur, it might result in unnecessary isolation of the primary
system.
2. Permissive Tripping
Permissive trip commands are always monitored by a protection relay. The circuit breaker is tripped
when receipt of the command coincides with operation of the protection relay at the receiving end
responding to a system fault. Requirements for the communications channel are less onerous than for
direct tripping schemes, since receipt of an incorrect signal must coincide with operation of the
receiving end protection for a trip operation to take place. The intention of these schemes is to speed up
tripping for faults occurring within the protected zone.
3. Blocking Scheme
Blocking commands are initiated by a protection element that detects faults external to the protected
zone. Detection of an external fault at the local end of a protected circuit results in a blocking signal
being transmitted to the remote end. At the remote end, receipt of the blocking signal prevents the
remote end protection operating if it had detected the external fault. Loss of the communications
19
channel is less serious for this scheme than in others as loss of the channel does not result in a failure to
trip when required. However, the risk of a spurious trip is higher.
While the different schemes require different communication capabilities we will use the most stringent
communication requirements, i.e., the requirements of the direct tripping scheme.
Use Case General Description
Title
Inter-trip Protection
Description
Inter-tripping is the controlled tripping of a circuit breaker so as to complete the isolation
of a circuit or piece of apparatus in concert with the tripping of other circuit breakers.
Actors
Protection relays in two substations
Business Service Category
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Data Center / Control Center
Centralized Generation Plant / Storage Facility - with capacity > 100MVA
Distributed Generation / Storage Resource with capacity < 100MVA
High Voltage Transmission Substation with voltage level > 161KV
Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV
Low Voltage Distribution Substation with voltage level - 25KV
Distribution Line / feeder at medium or low voltage levels
Customer Premise
Places in the Communication Network
Control Center
Data Center
Wide Area Network (WAN)
Substation Network (Transmission)
X
X
Communication Events & Attributes
Event
Direct trip
Mechanism
Protocol
Bandwidth
EventLegacy
64kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
X
X
X
Generation / Storage Plant Network
Field Area Network (Distribution)
Commercial / Industrial Network
Residential Premise Network
Actor 1
Relay
Latency
5ms
Jitter
N/A
Acknowledge
Y
Trans. Rate
Actor 2
Relay
Packet Loss
Security
0.1%
H
Time Synch.
Y
BER / PER
…
20
Mechanism
Eventtriggered
Redundancy
Y
Event
Permissive trip
Protocol
Legacy
Availability
H
Actor 1
Relay
Bandwidth
64kbps
Latency
5ms
Jitter
N/A
Direction
BI
Acknowledge
Y
Trans. Rate
Event
Blocking
Mechanism
Protocol
Bandwidth
EventLegacy
64kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Relay
Latency
5ms
Jitter
N/A
Acknowledge
Y
Trans. Rate
Actor 2
Relay
Packet Loss
Security
0.1%
H
Time Synch.
Y
BER / PER
…
Actor 2
Relay
Packet Loss
Security
0.1%
H
Time Synch.
Y
BER / PER
…
21
Figure 13.
Inter-Trip Protection Tabular & Graphic Information
22
4.1.3 Current Differential Protection Scheme
Current differential protection is commonly used for line protection, and is typical for protecting parallel circuits.
A main advantage for differential protection is that, compared to overcurrent protection, it allows only the
faulted circuit to be de-energized in case of a fault.
At both end of the lines, the current is measured by the differential relays, and based on Kirchhoff’s law, both
relays will trip the circuit breaker if the current going into the line does not equal the current going out of the
line.
This type of protection scheme assumes some form of communication being present between the relays at both
end of the line, to allow both relays to compare measured current values.
A fault in line 1, will cause overcurrent to be flowing in both lines, but because the current in line 2 is a through
following current, this current is measured equal at both ends of the line, therefore the differential relays on line
2 will not trip line 2. Line 1 will be tripped, as the relays will not measure the same currents at both ends of the
line.
Line differential protection schemes assume a very low communications delay between both relays, often as low
as 5ms. Moreover, as those systems are often not time-synchronized, they also assume symmetric
communications paths with constant delay, which allows comparing current measurement values taken at the
exact same time.
Title
Current Differential Protection
Description
Current differential protection scheme. The current through the line is measured at both
ends of the line by the differential relays, and based on Kirchhoff’s law, both relays will trip
the circuit breaker if the current going into the line does not equal the current going out of
the line.
Actors
Differential protection relays; Line Breakers
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
X
X
23
Customer Premise
Control Center
Data Center
X
X
Event
Sample Value of Current
Mechanism
Protocol
Bandwidth
EventProprietary
64 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Figure 14.
X
Commercial / Industrial Area Network
Actor 1
Relay
Latency
5ms
Jitter
0.2ms
Acknowledge
Y
Trans. Rate
Actor 2
Relay
Packet Loss
Security
0.1%
H
Time Synch.
Y
BER / PER
…
Current Differential Protection Tabular & Graphic Information
24
4.1.4 Distance Protection Scheme
Distance (Impedance Relay) protection scheme is based on voltage and current measurements. A fault on a
circuit will generally create a sag in the voltage level. If the ratio of voltage to current measured at the
protection relay terminals, which equates to an impedance element, falls within a set threshold the circuit
breaker will operate. The operating characteristics of this protection are based on the line characteristics. This
means that when a fault appears on the line, the impedance setting in the relay is compared to the apparent
impedance of the line from the relay terminals to the fault. If the relay setting is determined to be below the
apparent impedance it is determined that the fault is within the zone of protection. When the transmission line
length is under a minimum length distance protection becomes more difficult to coordinate. In these instances
the best choice of protection is current differential protection.
The schemes may work in different modes (trip, blocking...).
Title
Distance Protection
Description
Distance protection scheme relies on both voltage and current measurements. If the ration
of voltage over current is above a certain threshold (indicating a very small impedance
equivalent) the relay will operate and open the breaker to isolate the line.
Actors
Distance protection relays; Breakers
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
25
Event
Block / Trip Signal
Mechanism
Protocol
Bandwidth
EventProprietary
64 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Figure 15.
Actor 1
Relay
Latency
5ms
Jitter
0.2ms
Acknowledge
Y
Trans. Rate
Actor 2
Relay
Packet Loss
Security
0.1%
H
Time Synch.
Y
BER / PER
…
Distance Protection Tabular & Graphic Information
26
4.1.5 Inter-Substation Protection Signaling
This use case describes the exchange of Sampled Value or GOOSE message between IED’s in two substations for
protection and tripping coordination. The two IED’s are in a master-slave mode.
The CT/VT in one substation sends the sampled analog voltage or current value to the Merging Unit (MU) over
hard wire. The merging unit sends the time-synchronized 61850-9-2 sampled values to the slave IED. The slave
IED forwards the information to the Master IED in the other substation. The master IED makes the
determination (for example based on sampled value differentials) to send a trip command to the originating IED.
Once the slave IED/Relay receives the GOOSE trip for breaker tripping, it opens the breaker. It then sends a
confirmation message back to the master. All data exchanges between IEDs are either through Sampled Value or
GOOSE messages.
The GPS Master Clock can send 1PPS or IRIG-B format to MU through serial port, or IEEE 1588 protocol via
network.
Title
Inter-Substation Protection Signaling using 61850
Description
Exchange of GOOSE message between IED’s in two substations for protection and tripping
coordination.
Actors
CT/VT; Merging Unit; IED; Time Synchronization
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
27
Event
Monitor
Mechanism
Protocol
Bandwidth
Event61850-9-2
64 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Command
Monitor
Mechanism
Protocol
Bandwidth
Event61850-9-2
64 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Event
Confirmation
Mechanism
Protocol
Bandwidth
Event61850-9-2
64 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Actor 1
IED / Relay
Latency
Jitter
5ms
N/A
Acknowledge
Y
Trans. Rate
Actor 1
IED / Relay
Latency
Jitter
5ms
N/A
Acknowledge
Y
Trans. Rate
Actor 1
IED / Relay
Latency
Jitter
5ms
N/A
Acknowledge
Y
Trans. Rate
Actor 2
IED / Relay
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
Actor 2
IED / Relay
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
Actor 2
IED / Relay
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
28
Figure 16.
Inter-Substation Protection Signaling with 61850 Tabular & Graphic Information
29
4.1.6 Intra-Substation Process Bus Communication
This use case describes the data flow from the CT/VT to the IEDs in the substation via the merging unit (MU).
The CT/VT in the substation send the sampled value (analog voltage or current) to the Merging Unit (MU) over
hard wire. The merging unit sends the time-synchronized 61850-9-2 sampled values to the IEDs in the substation
in GOOSE message format. The GPS Master Clock can send 1PPS or IRIG-B format to MU through serial port, or
IEEE 1588 protocol via network.
Process bus communication using 61850 simplifies connectivity within the substation and removes the
requirement for multiple serial connections and removes the slow serial bus architectures that are typically
used. This also ensures increased flexibility and increased speed with the use of multicast messaging between
multiple devices.
Title
Intra-Substation 61850 GOOSE Data Flow
Description
Sampled values collected through CT/VT are sent from the process bus to IEDs on the
station bus in the IEC 61850 GOOSE message format.
Actors
CT/VT; Merging Unit; IED; Time Synchronization
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
30
Event
Monitor
Mechanism
Protocol
Bandwidth
Event61850-9-2
64 kbps
triggered
Redundancy
Availability
Direction
N
H
BI
Event
Command
Mechanism
Protocol
Bandwidth
Event61850-9-2
64 kbps
triggered
Redundancy
Availability
Direction
N
H
BI
Event
Acknowledgement
Mechanism
Protocol
Bandwidth
Event61850-9-2
64 kbps
triggered
Redundancy
Availability
Direction
N
H
BI
Actor 1
IED / Relay
Latency
Jitter
5ms
N/A
Acknowledge
Y
Trans. Rate
Actor 1
IED / Relay
Latency
Jitter
5ms
N/A
Acknowledge
Y
Trans. Rate
Actor 1
IED / Relay
Latency
Jitter
5ms
N/A
Acknowledge
Y
Trans. Rate
Actor 2
IED / Relay
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
Actor 2
IED / Relay
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
Actor 2
IED / Relay
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
31
Figure 17.
Process Bus Communication using 61850 Tabular & Graphic Information
32
4.1.7 Control Center Monitoring of Station Bus
This use case describes the process of polling substation station bus from the EMS or DMS application in the
control center where the communication between control center and the substation is via 61850-90-2 and
communication within the substation is via 61850.
Title
Control Center Monitoring of Substation Station Bus with 61850
Description
Monitoring of substation Station Bus from a control center application, e.g. EMS or DMS.
Communication within substation is via IEC 61850 and communication to the control
center is via 61850-90-2.
Actors
Control Center applications (EMS or DMS); IEC 61850 gateway; Station Bus; IED
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Event61850-9-2
64 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
X
X
X
X
Actor 1
Control Center
Latency
Jitter
< 1sec
N/A
Acknowledge
Trans. Rate
Actor 2
61850 Gateway
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
33
Event
Monitor
Mechanism
Protocol
Bandwidth
Event61850-9-2
64 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Figure 18.
Actor 1
61850 Gateway
Latency
Jitter
5ms
N/A
Acknowledge
Trans. Rate
Actor 2
IED
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
Control Center Monitoring of Station Bus using 61850 Tabular & Graphic Information
34
4.1.8 Communication System Failure & Degradation Monitoring
Due to the critical role of the communication network in the next generation monitoring, control, and protection
functions, it is important that the failure and degradation of the communication network is constantly
monitored. The IEC 62351-7 standard calls for monitoring the communication network to detect and log the
following conditions:
•
•
•
•
•
•
Network equipment permanent failures
Network equipment temporary failures or resets
Communication link failures
Communication link degradation or lower than expected throughput
Network routing degradation or lower than expected throughput
Logging equipment and communication link failures and degraded conditions
This use case describes the monitoring of the communication network, including device status, device
configuration, latency test, software management, and threshold testing by the Network Management System.
The benefits include improved visibility of communication network faults; reduced communications minutes lost
due to undetected faults, and proactive maintenance of communications assets.
Title
Communication System Failure and Degradation Monitoring
Description
Monitoring the failure or degradation of the communication system using NMS.
Actors
Network Management System; Communication Device (Router/Switch)
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
X
X
X
X
X
35
Control Center
Data Center
X
X
X
X
Event
Configuration
Mechanism
Protocol
Bandwidth
Ad hoc
SSH
64 kbps
Redundancy
Availability
Direction
N
H
BI
Event
Threshold Event
Mechanism
Protocol
Bandwidth
Ad hoc
SNMPv3
9.6 kbps
Redundancy
Availability
Direction
N
H
BI
Event
Device Status
Mechanism
Protocol
Bandwidth
Ad hoc
SNMPv3
9.6 kbps
Redundancy
Availability
Direction
N
H
BI
Event
Software Management
Mechanism
Protocol
Bandwidth
Ad hoc
SCP
64 kbps
Redundancy
Availability
Direction
N
H
BI
Event
Latency Test
Mechanism
Protocol
Bandwidth
Ad hoc
ICMP
9.6 kbps
Redundancy
Availability
Direction
N
H
BI
Actor 1
NMS
Latency
1sec
Acknowledge
N
Jitter
N/A
Trans. Rate
Actor 1
Device
Latency
1sec
Acknowledge
N
Jitter
N/A
Trans. Rate
Actor 1
Device
Latency
1sec
Acknowledge
N
Jitter
N/A
Trans. Rate
Actor 1
NMS
Latency
1sec
Acknowledge
N
Jitter
N/A
Trans. Rate
Actor 1
Device
Latency
1sec
Acknowledge
N
Jitter
N/A
Trans. Rate
X
X
Actor 2
Device
Packet Loss
Security
5%
M
Time Synch.
BER / PER
Y
…
Actor 2
NMS
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
Actor 2
NMS
Packet Loss
Security
5%
M
Time Synch.
BER / PER
Y
…
Actor 2
Device
Packet Loss
Security
5%
M
Time Synch.
BER / PER
Y
…
Actor 2
NMS
Packet Loss
Security
5%
H
Time Synch.
BER / PER
Y
…
36
Figure 19.
Communication System Failure & Degradation Monitoring Tabular & Graphic
Information
37
4.1.9 High Voltage Substation SCADA RTU
The control and supervision of high voltage substations is done via the use of Remote Terminal Units (RTU).
Today, most RTU’s use serial connection and proprietary legacy protocols. In this use case we model a generic
device to control center communication through the RTU using IEC 60870-5-104 or IEC 60870-5-101 protocols.
For high voltage substations communication network availability and performance for SCADA applications are
critical.
Title
High Voltage Substation SCADA RTU Using IEC 60870-5-101 & 104.
Description
Generic use case describing communication between substation RTU and control center
using IEC 60870-5-101 & 104.
Actors
Substation RTU; Control Center SCADA Headend
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
60870-5-101
20 kbps
& 104
Redundancy
Availability
Direction
X
X
X
X
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Actor 2
RTU
Packet Loss
Security
5%
H
Time Synch.
BER / PER
38
Y
H
BI
Event
Alert
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-101
20 kbps
& 104
Redundancy
Availability
Direction
Y
H
BI
Event
Control
Mechanism
Protocol
Bandwidth
Polled
60870-5-101
20 kbps
& 104
Redundancy
Availability
Direction
Y
H
BI
Event
Time Synch
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-101
20 kbps
& SNTP
Redundancy
Availability
Direction
Y
H
BI
Y
1 / Second or
two Seconds
Actor 1
RTU
Latency
0.5 Sec
Jitter
N/A
Acknowledge
Y
Trans. Rate
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
1 / Second or
two Seconds
Actor 1
RTU
Latency
0.5 Sec
Jitter
N/A
Acknowledge
Y
Trans. Rate
Y
…
Actor 2
Control Center
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
Actor 2
RTU
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
39
Figure 20.
High Voltage Substation SCADA RTU Tabular & Graphic Information
40
4.1.10 Medium Voltage Substation SCADA RTU
The control and supervision of medium voltage substations are done via the use of Remote Terminal Units
(RTU). Today, most RTU’s use serial connection and proprietary legacy protocols. In this use case we model a
generic device to control center communication through the RTU using IEC 60870-5-104 or IEC 60870-5-101
protocols.
While the bandwidth and latency requirements are similar to those for the high voltage substations, the
requirements for uptime and availability are less stringent for medium voltage substation networks.
Title
Medium Voltage Substation SCADA RTU Using IEC 60870-5-101 & 104.
Description
Generic use case describing communication between substation RTU and control center
using IEC 60870-5-101 & 104.
Actors
Substation RTU; Control Center SCADA Headend
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
60870-5-101
9.6 kbps
& 104
X
X
X
X
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Actor 2
RTU
Packet Loss
Security
5%
H
41
Redundancy
Y
Availability
H
Direction
BI
Event
Alert
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-101
9.6 kbps
& 104
Redundancy
Availability
Direction
Y
H
BI
Event
Control
Mechanism
Protocol
Bandwidth
Polled
60870-5-101
9.6 kbps
& 104
Redundancy
Availability
Direction
Y
H
BI
Event
Time Synch
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-101
9.6 kbps
& SNTP
Redundancy
Availability
Direction
Y
H
BI
Acknowledge
Y
Trans. Rate
1 / Second or
two Seconds
Actor 1
RTU
Latency
0.5 Sec
Jitter
N/A
Acknowledge
Y
Trans. Rate
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
1 / Second or
two Seconds
Actor 1
RTU
Latency
0.5 Sec
Jitter
N/A
Acknowledge
Y
Trans. Rate
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
Actor 2
RTU
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
42
Figure 21.
Medium Voltage Substation SCADA RTU Tabular & Graphic Information
43
4.1.11 Condition-Based Monitoring
This use case applies to both Transmission and Distribution service categories. T&D Operations can use sensors
to proactively monitor equipment in the field to make maintenance decisions based on the current conditions of
the assets. Automated analysis is performed on sensor data using rule-based algorithms to identify assets that
are potentially in need of repair or replacement.
The following are just a few examples of what can be monitored for condition-based maintenance:
•
•
•
Transformer oil temperature and oil pressure monitoring
Turbine monitoring
Backup battery monitoring
There are many benefits to condition-based monitoring including: improved SAIDI/SAIFI, higher efficiency in
operations, improved asset uptime, outage reduction, better system monitoring, increased crew safety,
increased public safety, improved power quality, reduced truck rolls, and deferred capital expenditure by
extending the useful life of the asset.
Condition based monitoring can also be used to automate certain maintenance routines by linking routine
inspection and work scheduling to the results of asset condition assessment. It can also alleviate costs
associated with mandatory regulatory requirements for visual inspection by recording the periodically collected
asset data for auditing purposes.
With orders of magnitude increase in the volume of data that are collected utilities will use Big Data query and
analysis tools to extract business intelligence and drive higher efficiency in asset utilization.
Title
Condition-Based Monitoring using IEC 60870
Description
Online monitoring of field assets’ condition to drive maintenance and repair scheduling, to
improve the efficiency of operation, and to improve asset reliability and life expectancy.
Actors
Substation RTU; Data Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
X
Other (Specify)
Places in the Grid
X
X
X
X
44
Customer Premise
Control Center
Data Center
X
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
60870-5-104
20 kbps
Redundancy
Availability
Direction
N
M
BI
Event
Control
Mechanism
Protocol
Bandwidth
Polled
60870-5-104
20 kbps
Redundancy
Availability
Direction
N
M
BI
X
X
X
Actor 1
RTU
Latency
1 Sec
Acknowledge
Jitter
N/A
Trans. Rate
1 / 4 Hours
Actor 1
Data Center
Latency
Jitter
1 Sec
N/A
Acknowledge
Trans. Rate
X
X
Actor 2
Data Center
Packet Loss
Security
5%
M
Time Synch.
BER / PER
Y
…
Actor 2
RTU
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
45
Figure 22.
Condition Based Monitoring Tabular & Graphic Information
46
4.1.12 Transformer Dissolved Gas Analysis
Transformers are designed to operate 30-50 years. Periodic analysis of transformer oil provides an indication of
the condition of the transformer, which can provide valuable information to substation designers and engineers.
The presence or changes in dissolved gas indicates internal changes in the transformer, sometimes due to
deterioration of seals between internal components. Today the analysis is done based on manual samples taken
every couple of years.
This use case describes automated monitoring of dissolved gas every four hours. The analysis typically includes
hydrogen (H2), oxygen (O2), nitrogen (N2), methane (CH4), carbon monoxide (CO), carbon dioxide (CO2),
ethylene (C2H4), ethane (C2H6), acetylene (C2H2), and propane (C3H8). The data could be transferred to the
Data Center via IEC 60870-5-104 or a legacy SCADA protocol such as DNP3, over the IP network.
Continuous collection of data on grid assets provides a rich data set that can be mined for optimizing the
utilization and life expectancy of the assets. Here is yet another opportunity for using Big Data query and
analysis tools to uncover failure patterns that once identified could be preventable by timely maintenance or a
shift in operational duty cycles.
This analysis can be done for both transmission and distribution transformers.
Title
Transformer Dissolved Gas Analysis using IEC 60870
Description
Online monitoring and analysis of transformer dissolved gas analysis to detect anomalies
and signs of wear and failure for preventive maintenance.
Actors
Gas Analysis Sensor; Data Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
X
X
Other (Specify)
Places in the Grid
Customer Premise
X
X
X
X
X
X
47
Control Center
Data Center
X
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
60870-5-104
96 kbps
or
DNP3 over
IP
Redundancy
Availability
Direction
N
M
UNI
Figure 23.
Actor 1
RTU
Latency
1 Min
Jitter
N/A
Acknowledge
Trans. Rate
1 / 4 Hours
X
Actor 2
Data Center
Packet Loss
Security
10%
M
Time Synch.
Y
BER / PER
…
Transformer Dissolved Gas Analysis Tabular & Graphic Information
48
4.1.13 Gas-Insulated Switchgear Health Monitoring
Gas-insulated switchgear could be monitored at regular intervals to increase the overall operational efficiency of
the asset by reducing the SF6 gas inspection time. Monitoring the real-time gas density value in combination
with historical trends for gas leakage rate allows the company to predict and optimize equipment maintenance
scheduling, going from time-based to condition-based maintenance.
This use case describes automated monitoring of SF6 gas every four hours. The data could be transferred to the
Data Center via IEC 60870-5-104 or a legacy SCADA protocol such as DNP3, over the IP network.
This is yet another opportunity to use Big Data analytics tools for uncovering pre-failure asset condition patterns
from volume of data that is collected on similar assets to proactively predict and prevent asset failures.
This analysis can be done for both transmission and distribution switchgear.
Title
Gas-Insulated Switchgear Health Monitoring using IEC 60870
Description
Online monitoring and analysis of gas-insulated switchgear to detect anomalies and signs
of equipment wear and failure for preventive maintenance purposes.
Actors
RTU; Data Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
49
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
DNP3 over
20 kbps
IP or
60870-5-104
Redundancy
Availability
Direction
N
M
UNI
Figure 24.
Actor 1
RTU
Latency
1 Min
Jitter
N/A
Acknowledge
Trans. Rate
1 / 4 Hours
Actor 2
Data Center
Packet Loss
Security
5%
M
Time Synch.
Y
BER / PER
…
Gas-Insulated Switchgear Health Monitoring Tabular & Graphic Information
50
4.1.14 Dynamic Asset Rating
Dynamic asset rating refers to the ability to remotely monitor transmission line conditions (with sensors that
detect conductor temperature, line sag, and wind speed and direction) to determine the maximum power
carrying capacity and loading of the line. This provides a more accurate and timely view of the line capacity
compared to static nameplate limits for the line. Armed with this information operators can push more power
through the line to alleviate transmission constraints and network congestion issues. This leads to a more
effective utilization of the asset capability.
It is conceivable that with continuous collection and monitoring of the assets using Big Data Analytics tools the
operators will be able to anticipate asset conditions under different grid operating states and automate
decisions for line loading for different grid states.
Title
Dynamic Asset Rating using IEC 60870
Description
The ability to remotely monitor transmission line conditions and determine excess capacity
of the line for carrying additional power flow.
Actors
RTU; Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
X
X
X
Actor 1
Control Center
Latency
Jitter
Actor 2
RTU
Packet Loss
Security
51
Polled
Redundancy
N
60870-5-104
Availability
M
20 kbps
Direction
BI
0.5 Sec
Acknowledge
Y
N/A
Trans. Rate
1 / Min
5%
Time Synch.
Y
H
BER / PER
…
Event
Alert
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-104
20 kbps
Redundancy
Availability
Direction
N
M
BI
Latency
0.5 Sec
Acknowledge
Y
Jitter
N/A
Trans. Rate
1 / Min
Actor 2
Control Center
Packet Loss
Security
1%
H
Time Synch.
BER / PER
Y
…
Event
Control
Mechanism
Protocol
Bandwidth
Polled
60870-5-104
20 kbps
Redundancy
Availability
Direction
N
M
BI
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
1 / Min
Actor 2
RTU
Packet Loss
Security
1%
H
Time Synch.
BER / PER
Y
…
Actor 1
RTU
Actor 2
Control Center
Packet Loss
Security
1%
H
Time Synch.
BER / PER
Y
…
Event
Time Synch
Mechanism
Protocol
Bandwidth
Polled
SNTP
20 kbps
Redundancy
Availability
Direction
N
M
BI
Actor 1
RTU
Latency
0.5 Sec
Acknowledge
Y
Jitter
N/A
Trans. Rate
1 / Min
52
Figure 25.
Dynamic Asset Rating Monitoring Tabular & Graphic Information
53
4.1.15 Wide Area Monitoring & Control Systems
The application of synchrophasor measurement data from Phasor Measurement Units (PMU) to Wide Area
Monitoring and Control Systems promises to provide important new capabilities for improving system stability.
Access to PMU data enables more timely situational awareness over larger portions of the grid than what has
been possible historically with normal SCADA data. Handling the volume and real-time nature of synchrophasor
data presents unique challenges for existing application architectures.
Wide Area management System (WAMS) makes it possible for the condition of the bulk power system to be
observed and understood in real-time so that protective, preventative, or corrective action can be taken.
The history of PMU-based WAMS at H-Q goes back more than 30 years to the mid-1970s. In 2004, H-Q
commissioned an 8-PMU WAMS system that feed EMS with GPS-synchronized angles, frequencies and harmonic
distortion measurements from key 735 KV substations. H-Q uses this system for frequency regulation reporting
and control room implementation of preventive measures against geomagnetic storm-induced contingencies.
Because of the very high sampling rate of measurements and the strict requirement for time synchronization of
the samples, WAMS has stringent communication requirements in an IP network that are captured in the
following table:
WAMS Requirement
Client Interfaces
One Way Delay
Jitter
Layer 2 or Layer 3
VPN Topology
Multicast
Figure 26.
Attribute
Ethernet, Serial, X.21
50ms
No
Not Critical
Layer 3
Multi-Point to Multi-Point
Layer 3
99.999 / High
Yes
<50ms – (if closed loop)
Yes, Mandatory
WAMS Special Communication Requirements
At present, there is a field trial that will connect several PMUs to a PDC in Montreal for monitoring purposes.
Ultimately, the plan is for PMUs to be able to communicate with other PMUs in the system to perform real-time
data analytics and take control actions.
Typically, in PMU data communication networks, the Phasor Data Collectors (PDC) aggregate and forward PMU
data to control center. In IP networks, with multicast capability and the ability for multipoint-to-multipoint
communication, the need for PDCs is eliminated, leading to streamlining and simplification of the PMU data
network.
54
Title
Wide Area Monitoring System (WAMS)
Description
This use case refers to collection and analysis of data from Phasor Measurement Units
(PMU) at very high sampling rate to perform preventive controls for transient stability,
contingency analysis, frequency regulation reporting, state estimation, and other advanced
analytical applications. WAMS is a transmission network analysis and monitoring tool.
However, PMUs could also be deployed and have been deployed for distribution grid
applications as well.
Actors
PMU; Phasor Data Collector (PDC); Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Stream
C37.118
100 kbps
Redundancy
Availability
Direction
Y
H
UNI
Latency
100 ms
Acknowledge
Y
Jitter
N/A
Trans. Rate
60 / sec
Actor 2
PDC / Control Center
Packet Loss
Security
1%
H
Time Synch.
BER / PER
Y
…
Event
Time Sync
Mechanism
Protocol
Bandwidth
Polled
IEEE 1588
20 kbps
Redundancy
Availability
Direction
Y
H
UNI
Actor 1
PMU / PDC
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
60 / sec
Actor 2
Control Center
Packet Loss
Security
1%
H
Time Synch.
BER / PER
Y
…
Actor 1
PMU
55
The following UML diagrams represent three distinct possibilities:
1. Wide Area Management System with PDCs
2. Wide Area Management System without PDCs
3. Wide Area Management System without PDCs and with control capability
Figure 27.
Wide Area Management System with PDC Tabular & Graphic Information
56
Figure 28.
Figure 29.
Wide Area Management System without PDC Graphic Information
Wide Area Management System with P2P Control Capability Graphic Information
57
4.1.16 Substation Environmental Monitoring
This use case applies to both transmission and distribution substations. Examples of environmental monitoring
sensors include: substation internal and external ambient temperature, transformer temperature, battery
temperature, humidity sensor, airflow sensor, flood sensors, fuel level, earthquake, etc.
Typically, environmental measurements are taken at 10-minute intervals and sent to the control center for
archival and analysis. However, depending on the measurement type, samples could be taken less or more
frequently, e.g. fuel levels are measured once a week.
The benefits of substation environmental monitoring are:
•
•
•
•
•
•
Avoiding equipment damage
Real-time prediction of likely outages
Reducing the impact of low probability, high impact events, such as flooding
Creating more accurate asset lifetime model
Developing more accurate transformer rating
Making informed decisions about sending field crew to locations that could be intolerably hot or humid
Title
Substation Environmental Monitoring
Description
Periodic monitoring of substation environmental conditions for better asset management
and improved system reliability.
Actors
Environmental Sensor(s); Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
X
X
58
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
DNP3
9.6 kbps
Redundancy
Availability
Direction
N
L
UNI
Actor 1
Air Flow Sensor
Latency
Jitter
1 Min
N/A
Acknowledge
Trans. Rate
N
Variable
Actor 2
Control Center
Packet Loss
Security
10%
M
Time Synch.
BER / PER
Y
…
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
DNP3
9.6 kbps
Redundancy
Availability
Direction
N
L
UNI
Actor 1
Battery Sensor
Latency
Jitter
1 Min
N/A
Acknowledge
Trans. Rate
N
Variable
Actor 2
Control Center
Packet Loss
Security
10%
M
Time Synch.
BER / PER
Y
…
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
DNP3
9.6 kbps
Redundancy
Availability
Direction
N
L
UNI
Actor 1
Humidity Sensor
Latency
Jitter
1 Min
N/A
Acknowledge
Trans. Rate
N
Variable
Actor 2
Control Center
Packet Loss
Security
10%
M
Time Synch.
BER / PER
Y
…
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
DNP3
9.6 kbps
Redundancy
Availability
Direction
N
L
UNI
Actor 1
Flood Sensor
Latency
Jitter
1 Min
N/A
Acknowledge
Trans. Rate
N
Variable
Actor 2
Control Center
Packet Loss
Security
10%
M
Time Synch.
BER / PER
Y
…
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
DNP3
9.6 kbps
Redundancy
Availability
Direction
N
L
UNI
Actor 1
Temperature Sensor
Latency
Jitter
1 Min
N/A
Acknowledge
Trans. Rate
N
Variable
Actor 2
Control Center
Packet Loss
Security
10%
M
Time Synch.
BER / PER
Y
…
59
Figure 30.
Substation Environmental Monitor Tabular & Graphic Information
60
4.1.17 Power System State Estimation
This use case describes the data acquisition process for power system State Estimation. RTUs at generation and
transmission substations are periodically polled to send analog and status measurements including voltages,
currents, phase angles, real and reactive power flows, and breaker and other logical device statuses to the
SCADA headend in the Control Center and on to the State Estimator. The State Estimator uses statistical
estimation algorithms to determine the topology of the electrical grid, to sort out “bad data”, and to estimate
the current status of all flows and devices based on a consistent snapshot of network state measurements at
appoint in time. The results of the State Estimator are used by other advanced monitoring and control
applications in the Energy Management System to optimize system performance by minimizing system losses.
Data exchange between these applications is via in-memory databases in the control center.
In many instances RTUs use serial connection and proprietary protocols. In this use case we model RTU to
Control Center communication using IEC 60870-5-104 or IEC 60870-5-101 protocols.
For transmission substation communication availability and performance of the RTU is critical.
Title
SCADA Data Collection for State Estimation
Description
Collection and transmission of data from RTUs using IEC 60870-5-104 and 60870-5-101
protocols.
Actors
Substation RTU; Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
X
61
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
61850-5-101
20 kbps
&
61850-5-104
Redundancy
Availability
Direction
Y
H
BI
Event
Alarm
Mechanism
Protocol
Bandwidth
Ad hoc
61850-5-101
20 kbps
&
61850-5-104
Redundancy
Availability
Direction
Y
H
BI
Event
Control
Mechanism
Protocol
Bandwidth
Polled
61850-5-101
20 kbps
&
61850-5-104
Redundancy
Availability
Direction
Y
H
BI
Event
Time Sync
Mechanism
Protocol
Bandwidth
Polled
61850-5-101
20 kbps
& SNTP
Redundancy
Availability
Direction
Y
H
BI
Actor 1
RTU
Latency
0.5 Sec
Jitter
N/A
Acknowledge
Y
Trans. Rate
Every 2 Sec
Actor 1
RTU
Latency
0.5 Sec
Jitter
N/A
Acknowledge
Y
Trans. Rate
Every 2 Sec
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
RTU
Latency
0.5 Sec
Jitter
N/A
Acknowledge
Y
Trans. Rate
Every 2 Sec
Actor 2
Control Center
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
Actor 2
RTU
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
62
Figure 31.
State Estimation Tabular & Graphic Information
63
4.1.18 Inter Control Center Communication (ICCP)
The Inter-Control Center Communications Protocol (ICCP) or IEC 60870-6 is used for real-time data exchange
among utility control centers, regional control centers, and non-utility generators. Through ICCP, utilities
periodically exchange system data include analog and status points, quality flags, setpoints, events, and error
conditions, etc.
This use case describes the process of updating SCADA data in one Control Server (ICCP Client) by periodically
(every 4-8 seconds) sending analog and status values representing transmission and distribution network
telemetered or calculated values and generation related data from another Control Center (ICCP Server).
Occasional non-delivery could be tolerated as long as the consuming applications in the receiving Control Center
are made aware of the non-delivery and therefore could reinitialize the database. Otherwise, inconsistent ICCP
data could lead to corruption of database in the receiving Control Center.
For ICCP, the best practice is to use “Secure ICCP” as outlined in the “Secure ICCP Integration Considerations and
Recommendations” by Sandia National Laboratories.
The Sandia report’s recommendations are restated here:
Secure ICCP Certificate Management
•
PKI Domain Design – Based on best-practice implementations, two primary PKI domain designs were
identified: a flat hierarchy and a tiered hierarchy. For control systems within a single established
domain, a flat hierarchy is recommended for the distribution of authentication certificates. This
recommendation is based on the number of endpoints sharing ICCP data. For the most part, such
networks tend to be isolated and generally small (at most a few hundred nodes) and, as such, lend
themselves better to a flat hierarchy. The advantage of a flat hierarchy is that only one CA needs to be
established for everyone on the internal domain network, reducing the complexity of the configuration.
In a tiered approach, each company would maintain its own CA, a proposition that is likely costprohibitive and more managerially complex.
•
Inter-Domain Communication – The architecture recommended for inter-domain communication is a
tiered hierarchy. This recommendation is based on the need to provide the most secure
implementation. Creating a single “root” Certificate Authority (CA) allows more restrictive security
policies to be enforced at the root while alleviating some of stringent security requirements on
subordinate CA’s.
•
Secure ICCP Application Issues – Current implementations of certificate-based schemes within ICCP
applications are primarily static in nature. This implies that any certificate update or renewal process
requires actions by an operator. This mechanism does not fit modern techniques of end node
authentication. Web-based forms of certificate authentication do not require machines (computers) to
be informed of the certificate update because the new certificate will be sent at the beginning of each
64
SSL handshake. Because a node’s certificate is sent at the beginning of each session, nodes should not
need to store local copies of anyone else’s certificate. Therefore, when a node is issued a new certificate
for any reason (expiration, key update, etc.), the operation is transparent to other nodes in the network
and they do not need to be notified. It is recommended that these techniques be designed into all
applications intended to support Secure ICCP.
Network System Design & Quality of Service
•
In any Wide Area Network (WAN), the most efficient and highly available routes will become congested
and communication between participating end nodes, e.g. SCADA control centers, may be delayed or
lost. Therefore, it is essential to create Service-Level Agreements (SLAs) for WAN that guarantee a level
of service for ICCP data streams.
Transition Strategy
•
Layer 2 and Layer 3 Protection Schemes – For some utility sites the conversion from the standard ICCP to
Secure ICCP will not be rapidly achieved. The report discusses some potential alternatives to provide the
security needed to assure ICCP data protection. IPSec and data link encryption are suggested as means
to provide the necessary data surety for the protection of in-flight ICCP data. A technique is also
described to configure a network connection to provide a mixed-mode operational scenario when both
secure and non-secure forms of ICCP co-exist on a network.
Performance
•
The report discusses measurements that were taken to characterize the impact of using different
security layers associated with securing the ICCP data. The processing and transport delays are
characterized to provide the user with a sense of the operational impact when adding protection
technologies to an ICCP network. Associated implementations, such as OpenSSL for Secure ICCP and
IPSec for a Layer-3 encryption, are documented. The overall results show that the integration of secure
protocols should have minimal effect on end-to-end application performance but the overall
management complexity will increase with each added layer of protection.
Title
Inter Control Center Communication (ICCP)
Description
SCADA data transfer between two control centers using ICCP protocol
Actors
Sending Control Center (ICCP Server); Receiving Control Center (ICCP Client)
Transmission
Distribution
Energy Supply
Demand Side
X
X
X
Other (Specify)
65
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
Event
SCADA Data
Mechanism
Protocol
Bandwidth
Polled
IEC 60870-6
> 1MB
Redundancy
Availability
Direction
Y
H
UNI
Actor 1
ICCP Server
Latency
Jitter
< 4 Sec
N/A
Acknowledge
Trans. Rate
Y
1 every 4 Sec
Actor 2
ICCP Client
Packet Loss
Security
5%
H
Time Synch.
BER / PER
Y
…
Event
Delivery Flag
Mechanism
Protocol
Bandwidth
Polled
IEC 60870-6
9.6 kbps
Redundancy
Availability
Direction
Y
H
UNI
Actor 1
ICCP Server
Latency
Jitter
< 4 Sec
N/A
Acknowledge
Trans. Rate
Y
1 every 4 Sec
Actor 2
ICCP Client
Packet Loss
Security
5%
H
Time Synch.
BER / PER
Y
…
66
Figure 32.
ICCP Tabular & Graphic Information
67
4.1.19 RTU File Transfer
All RTU maintain some record files with chronological data. This use case describes communication between a
dedicated application in the control center that uploads and downloads database files to and from the RTUs. The
RTU file is downloaded for analysis. The application uses the IEC 60870-5-101/104 SCADA protocol to perform
this operation. This application is basically an IEC 60870-5-104 file transfer to HTTP converter /gateway with two
IP interfaces. One connected to the intranet, and the other to the substation device management network. Once
a user opens an http connection with this application, that query is translated into a file transfer using IEC
60870-5-104 against a specific RTU.
There are two options:
1. Whole RTU configuration file upload/download
2. Selective, line-by-line update of RTU configuration
Title
RTU File Transfer
Description
Transferring the RTU configuration file and binary database to / from the RTU to the
Control Center.
Actors
RTU; Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
X
X
68
Event
File Transfer
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-101
64 kbps
&
60870-5-104
Redundancy
Availability
Direction
Y
H
BI
Event
Set Point Update
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-101
9.6 kbps
&
60870-5-104
Redundancy
Availability
Direction
Y
H
BI
Figure 33.
Actor 1
RTU
Latency
5ms
Jitter
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
Control Center
Latency
Jitter
5ms
N/A
Acknowledge
Y
Trans. Rate
…
Actor 2
Control Center
Packet Loss
Security
5%
H
Time Synch.
Y
BER / PER
…
Actor 2
RTU
Packet Loss
Security
5%
H
Time Synch.
Y
BER / PER
…
RTU File Transfer Tabular & Graphic Information
69
4.1.20 Weather Monitoring Stations
By monitoring weather in real-time the utility will be able to predict weather-sensitive load. Also, advance
warning of severe weather conditions can prepare the utility to cope with potential outages that are caused by
extreme weather conditions. Another advantage of having real-time weather data in the vicinity of transmission
lines is to be able to calculate their dynamic ratings. And finally, weather stations can provide valuable data on
wind speed and direction to forecast wind farm generation. For this reasons many utilities install their own
weather stations to monitor local weather conditions. The weather station acts as a sensor that can measure
and forward analog and digital data to the control center through standard SCADA protocols, either over IP or a
legacy protocol such as DNP3.
Title
Weather Monitoring Stations
Description
Collecting local weather data via utility-owned weather stations for load forecasting,
dynamic asset rating calculation, and asset condition monitoring.
Actors
Weather Sensor; Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
DNP3
9.6 kbps
X
X
X
X
X
X
Actor 1
Weather Sensor
Latency
Jitter
N/A
N/A
X
Actor 2
Control Center
Packet Loss
Security
5%
M
70
Redundancy
N
Availability
L
Figure 34.
Direction
UNI
Acknowledge
N
Trans. Rate
Once an Hour
Time Synch.
Y
BER / PER
…
Weather Monitoring Station Tabular & Graphic Information
71
4.2 Distribution
This section includes use cases related to power distribution including distribution and feeder automation and
centralized grid monitoring and control of the distribution grid through control center applications such as
SCADA, DMS, and OMS.
4.2.1 Power Quality Monitoring System – PQMS
This use case describes the installation and operation of a unified and centralized power quality monitoring
system to enable the utility to meet increasing demand for power quality benchmarking, power quality
contracts, billing and energy use verification, predictive maintenance, etc. It is assumed that a centralized head
end server communicates with various sensors, including IEDs, meters, and protective relays that collect the
relevant data at selected locations across the grid. Information from the sensors is retrieved periodically, or by
exception if there are unexpected events, by PQMS. This information is stored for archival and reporting
purposes and is made available to various applications for analysis.
Here are some of the metrics that are used in power quality monitoring. The IEEE 1159 standard defines these
metrics as follows:
•
•
•
•
•
•
•
•
•
Flicker – Impression of unsteadiness of visual sensation induced by a light stimulus whose luminance or
spectral distribution fluctuates with time.
Voltage or Current Imbalance – The ratio of the negative sequence component to the positive sequence
component, usually expressed as a percentage.
Momentary Interruption – A type of short-duration root-mean-square (RMS) voltage variation where
the complete loss of voltage (<0.1 pu) on one or more phase conductors is for a time period between
0.5 cycles and 3 seconds.
Sustained Interruption – A type of long-duration root-mean-square (RMS) voltage variation where the
complete loss of voltage (<0.1 pu) on one of more phase conductors is for a time greater than 1 min.
Temporary Interruption – A type of short-duration root-mean-square (RMS) variation where the
complete loss of voltage (<0.1 pu) on one or more phase conductors is for a time period between 3
seconds and 1 min.
Voltage Change – A variation of the root-mean-square (RMS) or peak value of a voltage between two
consecutive levels sustained for definite but unspecified durations.
Voltage Fluctuation – A series of voltage changes or a cyclical variation of the voltage envelope.
Voltage Interruption – The disappearance of the supply voltage on one or more phases. It is usually
qualified by an additional term indicating the duration of the interruption (e.g., momentary, temporary,
sustained).
Waveform Distortion – A steady-state deviation from an ideal sine wave of power frequency principally
characterized by the spectral content of the deviation.
IED meters are typically used for monitoring of sag, swell, interruptions, imbalance, flicker, harmonics, power
factor, and frequency.
72
Title
Power Quality Monitoring System (PQMS)
Description
A unified, centralized application for collecting, archiving and acting as a system of record
for power quality monitoring.
Actors
Power Quality IEDs, meters, and relays
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Event
Measurement
Mechanism
Protocol
Bandwidth
Polled
61850-90-17
64 kbps
Redundancy
Availability
Direction
N
L
BI
Actor 1
RTU (PQ Meter/IED/Sensor)
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
Variable
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
BER / PER
Y
…
Event
Alert
Mechanism
Protocol
Bandwidth
Ad hoc
61850-90-17
64 kbps
Redundancy
Availability
Direction
N
L
BI
Actor 1
RTU (PQ Meter/IED/Sensor)
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
Variable
Actor 2
Control Center
Packet Loss
Security
1%
L
Time Synch.
BER / PER
Y
…
Actor 1
Actor 2
Event
73
Mechanism
Polled
Redundancy
Y
Control
Protocol
61850-90-17
Availability
H
Bandwidth
20 kbps
Direction
BI
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
Variable
IED / Control Device
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
In recognition of the fact that the input data for PQMS will come from a wide variety of internal sources, including
distribution sensors, PMUs, smart meters, as well as external sources such as the weather service and even social networks,
we have shown an average security level of “Medium” in the UML graphic representation. In reality, the security
classification will include Low’s, Medium’s and High’s depending on the source of the specific data stream and criticality of
the information.
Figure 35.
Power Quality Monitoring System Tabular & Graphic Information
74
4.2.2 Sensors in the Distribution Grid
Smart Grid initiatives have led to the proliferation of numerous types of sensors in the medium and low voltage
network for real-time monitoring and control purposes. This use case describes a generic distribution grid sensor
to Control Center information exchange.
Unlike past practices whereby sensors and their corresponding communication network were installed for a
single purpose or were only intended to provide information to a single applications, the next generation of
sensors are multi-purpose in that their data will be consumed by multiple applications all connected to the
Enterprise Service Bus (ESB) for information sharing. Furthermore, instead of a single purpose network, a
common IP network will be used for transporting data to and from the sensor to the Control Center.
With the increasing volume of data that are collected through sensors in the distribution grid utilities will be able
to use Big Data query and analysis tools to extract business intelligence and drive higher efficiency and reliability
in the distribution grid.
Title
Sensors in the Distribution Grid
Description
Collecting data from many sensors in the distribution grid through a common IP network
and providing that information to multiple applications through a shared Enterprise Service
Bus.
Actors
Distribution Grid Sensor; Data Center / Control Center
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
75
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled or
60870-5-104
> 9.6 kbps
EventTriggered
Redundancy
Availability
Direction
N
L
UNI
Figure 36.
Actor 1
IED / Relay
Latency
Jitter
5ms
N/A
Acknowledge
N
Trans. Rate
Variable
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
Sensors in the Distribution Grid Tabular & Graphic Information
4.2.3 Pole-Top Voltage Regulators & Capacitor Bank Monitoring
76
Pole top voltage regulators and capacitor banks exist today but are not centrally monitored or controlled during
normal system activity. It is desirable to be able to centrally monitor these devices as part of active voltage
support and power factor correction activities. Centralized monitoring will enable integrated decision making for
optimal device configuration between multiple devices on the same feeder.
With the volume of data that will be available, operators will be able to detect grid operating states and
dynamically control reactive resources in contrast to traditional way of activating these resources on a fixed preprogrammed schedule.
With the increasing volume of data that are collected about voltage regulators and capacitor banks utilities will
be able to use Big Data query and analysis tools to extract business intelligence and drive higher efficiency in the
utilization of these assets.
Title
Pole Top Voltage Regulator Capacitor Bank Monitoring
Description
Centralized monitoring of pole top voltage regulator and capacitor bank monitoring for
coordinated voltage support and power factor correction decision-making.
Actors
IED; Control Center
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
X
X
X
X
Actor 1
IED / Relay
Latency
Jitter
X
Actor 2
Control Center
Packet Loss
Security
77
Polled
Redundancy
N
60870-5-104
Availability
L
Figure 37.
20 kbps
Direction
UNI
1 Sec
Acknowledge
Y
N/A
Trans. Rate
SCADA Scan
Rate
5%
Time Synch.
Y
L
BER / PER
…
Voltage Regulator / Capacitor Bank Monitoring Tabular & Graphic Information
78
4.2.4 Remote Control of Overhead or Underground Switches
Deployment of remote control overhead switches is a component of the broader fault detection, isolation and
service restoration (FLISR) initiative in utilities. Of course there are other components in FLISR, including smart
relays, smart sectionalizers / reclosers, real-time distribution feeder monitors, and smart meters. Switching
activities are more efficient when they are controlled centrally.
These components, along with various monitoring and control algorithms that run in the distribution control
center as part of an advanced distribution management system make it possible to detect and respond to faults
and restore service in a short time.
Overhead switches are largely deployed in rural areas whereas underground switches are deployed in urban
areas where feeders are normally underground.
Title
Remote Control of Overhead or Underground Switches
Description
Remote control of distribution switches from centralized application such as DMS enables
faster response to faults and therefore shorter time to service restoration.
Actors
IED; Control Center
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
79
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
60870-5-104
20 kbps
Redundancy
Availability
Direction
y
M
BI
Latency
0.5 Sec
Acknowledge
Y
Event
Control
Mechanism
Protocol
Bandwidth
Pushed
60870-5-104
20 kbps
Redundancy
Availability
Direction
y
M
BI
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Figure 38.
Actor 1
IED
Jitter
N/A
Trans. Rate
1 / 2 Seconds
Actor 2
Control Center
Packet Loss
Security
1%
L
Time Synch.
BER / PER
Y
…
Actor 2
IED
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
Remote Control of Distribution Switches Tabular & Graphic Information
80
4.2.5 Distribution Feeder Voltage Regulation
Voltage regulation requires that voltage sensors be placed on the distribution feeder at critical points to monitor
the system voltage. In the event of a voltage deviation from a pre-specified band, the voltage sensors will send
an alarm back to the control center. This may then require a scan of the voltage sensors or measurements from
the AMI meters and other relevant load meters in the affected area to compute an optimal voltage solution.
Commands could then be sent to voltage regulators (tap changers) to move up or down accordingly. The voltage
regulator accepts the new control setpoints and gradually drives the voltage back to the acceptable normal
band.
The continuous flow of data from the distribution grid and smart meters and subsequent processing of that data
for optimal voltage profile calculation demand scalable Field Area Network and Wide Area Network and the
requisite processing power in the Control Center for timely analysis of that information and issuing the
necessary control commands.
Furthermore, if there are controllable distributed energy resources or EV charging stations on the feeder, the
DMS could send new setpoints to the affected devices. The Voltage regulators play a key part in Volt/VAR
control, stabilizing and flattening feeder voltages, thereby reducing grid losses.
Title
Distribution Feeder Voltage Regulation
Description
Monitoring and regulating voltage in distribution grid using voltage regulators and other
controllable devices, such as distributed energy resources and electric vehicles and
chargers.
Actors
Voltage sensors; IED; Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
Other (Specify)
Places in the Grid
Customer Premise
X
X
X
X
81
Control Center
Data Center
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
60780-5-101
20 kbps
&
60870-5-104
Redundancy
Availability
Direction
Y
M
BI
Event
Alert
Mechanism
Protocol
Bandwidth
Ad hoc
60780-5-101
20 kbps
&
60870-5-104
Redundancy
Availability
Direction
Y
M
BI
Event
Control
Mechanism
Protocol
Bandwidth
Polled
60780-5-101
20 kbps
&
60870-5-104
Redundancy
Availability
Direction
Y
M
BI
Actor 1
Voltage Sensor
Latency
Jitter
1 Sec
5 Min
Acknowledge
Y
Trans. Rate
1 / Minute
Actor 1
Voltage Sensor
Latency
Jitter
1 Sec
5 Min
Acknowledge
Y
Trans. Rate
1 / Minute
Actor 1
Control Center
Latency
Jitter
0.5 Sec
5 Min
Acknowledge
Y
Trans. Rate
…
X
Actor 2
Control Center
Packet Loss
Security
5%
M
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
Actor 2
IED
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
82
Figure 39.
Distribution Feeder Voltage Regulation Tabular & Graphic Information
83
4.2.6 Distribution Volt / VAR Optimization
Coordinated control of voltage and reactive power is a key requirement within the power system. By reducing
the amount of reactive power (VARs) flowing on the distribution feeder, the utility can reduce electrical losses
and improve the voltage profile along the feeder. The use of capacitor banks and voltage regulators is one way
that utilities can control demand and increase system efficiency.
Voltage regulating devices are usually installed at the substations and on the feeders. The substation
transformers can have tap changers, which can adjust the feeder voltage at the substation, depending on the
loading condition of the feeders. Special transformers with load tap changers (LTC) called voltage regulators are
also installed at various locations on the feeders, providing fine-tuning capability for voltage at specific points on
the feeders. Reactive compensation devices (i.e., capacitor banks) could be located in the substation or on the
feeders. Capacitor banks can be fixed or switched.
The DMS application in the Control Center monitors real-time voltages, real and reactive power from LTCs,
regulators, capacitors, medium voltage sensors, and additional monitoring points, such as customer meters.
Using this real time set of analog measurements, the application can minimize the operational costs by
managing real time power factor and voltages as close as possible to the substation power factor and desired
voltage targets.
Today at Hydro Québec, voltage regulation equipment is implemented within some substations. All voltage
regulation is locally controlled and the substation and reaction times of 30 seconds are typical. In some
substations, this is done manually only. Today, no substations have voltage regulation controlled via the
SCADA/DMS.
Title
Distribution Volt / VAR Optimization
Description
Coordinated monitoring and control of voltage regulators and capacitor banks from DMS to
control the power factor and minimize losses and therefore reduce costs.
Actors
Voltage Sensors; Voltage Regulators; Capacitor Banks; Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
Other (Specify)
Places in the Grid
X
X
X
84
Customer Premise
Control Center
Data Center
X
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
60870-5-104
20 kbps
Redundancy
Availability
Direction
N
M
BI
Actor 1
Voltage Sensor
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
BER / PER
Y
…
Event
Alert
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-104
20 kbps
Redundancy
Availability
Direction
N
M
BI
Actor 1
Voltage Sensor
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
Control Center
Packet Loss
Security
1%
L
Time Synch.
BER / PER
Y
…
Event
Control
Mechanism
Protocol
Bandwidth
Polled
60870-5-104
20 kbps
Redundancy
Availability
Direction
N
M
BI
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
IED
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
85
Figure 40.
Distribution Volt/VAR Optimization Tabular & Graphic Information
86
4.2.7
This use case applies to both Transmission and Distribution network and applies the monitoring and control of
substation capacitor banks from the Control Center. It is assumed that the voltage is measured by an IED at the
substation that also provides a means of remotely controlling the capacitor bank upon receiving a command
from the control center.
Title
Description
Remote monitoring and controlling of capacitor banks in the substation from the Control
Center.
Actors
IED; Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
60870-5-104
20 kbps
Redundancy
Availability
Direction
Y
H
BI
X
X
X
X
Actor 1
IED / Voltage Sensor
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
Control Center
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
87
Event
Alert
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-104
20 kbps
Redundancy
Availability
Direction
Y
H
BI
Actor 1
IED / Voltage Sensor
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
Control Center
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
Event
Control
Mechanism
Protocol
Bandwidth
Pushed
60870-5-104
20 kbps
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
IED
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
Figure 41.
Substation Capacitor Monitoring & Control Tabular & Graphic Information
4.2.8 Underground Cable Distributed Temperature Monitoring
88
One method of measuring temperature in an underground oil-field cable involves emitting a laser into a fiber
optic cable that is laid down in the immediate vicinity of the electrical cable, to measure the temperature of the
fiber to very high spatial resolution and thermal accuracy. The fiber optic cable used for this purpose is passive in
nature and has no individual sensing point. The monitoring equipment is calibrated by defining thermal sections
with similar configuration and environmental conditions including surface elements and fill type. The monitoring
instrument measures the temperature based on the Raman effect by analyzing a laser pulse that has propagated
through the fiber. The temperature of the fiber is determined by measuring the Raman scattering of the light.
The position of the temperature reading is determined by measuring the arrival timing of the returning light
pulse, similar to a radio echo. The temperature limit is translated into a current (loading) limit based on
conservative assumptions of the cable configuration and the thermal resistivity of the soil, that is how quickly
heat generated by the current in the cable can be dissipated.
The benefits of monitoring cable temperature and current limits include:
•
•
•
•
Evaluating the real-time cable rating and therefore improving cable utilization
Identifying emerging issues along the cable
Improving network reliability through early detection of failures and hotspots
Saving costs associated with maintenance and repair of underground cables
Title
Underground Cable Distributed Temperature Monitoring
Description
Monitoring the temperature as a proxy for current along an underground cable using a
fiber optic cable laid in close proximity of the energized electrical cable.
Actors
Distributed Temperature Sensor (DTS); Data Center
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
89
Event
Temperature Sample
Mechanism
Protocol
Bandwidth
Polled
HTTPS or
128 kbps
DNP3 over
IP
Redundancy
Availability
Direction
N
L
UNI
Event
Current Estimate
Mechanism
Protocol
Bandwidth
Polled
HTTPS or
20 kbps
DNP3 over
IP
Redundancy
Availability
Direction
N
L
UNI
Event
Alarm
Mechanism
Protocol
Bandwidth
Ad hoc
HTTPS or
20 kbps
DNP3 over
IP
Redundancy
Availability
Direction
N
L
UNI
Actor 1
DTS Collector
Latency
Jitter
5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
DTS Collector
Latency
Jitter
5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
DTS Collector
Latency
Jitter
5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 2
Data Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
Actor 2
Data Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
Actor 2
Data Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
90
Figure 42.
Underground Cable Distributed Temperature Sensing Tabular & Graphic Information
91
4.2.9 Online Transformer Condition Monitoring
T&D Operations can proactively monitor equipment in the field to make maintenance decisions based on the
current condition of the assets. Monitoring the cooling oil, especially as transformers age, can provide critical
early warning of transformer issues. If problems are identified early, preventative measures can be taken to
prevent a likely failure and significant outages. Automated analysis is performed on sensor data using rule-based
algorithms to identify assets that are potentially in need of repair or replacement.
This use case describes the ability to monitor transformers in real-time for various reasons, e.g. to drive them to
their rated limits and beyond if necessary to meet the load. At the same time, it can warn the operators if a
transformer has been operating at or above its nominal operating limit for a prolonged period of time, which
increases the risk of asset failure. Therefore, this could serve as a real-time operational advisory tool to increase
the distribution network’s reliability and continuity of service.
We should point out that this use case applies to both Transmission and Distribution.
Title
Online Transformer Condition Monitoring
Description
Real-time decision analysis based on the current condition of a transformer to drive it
beyond its rated limit or to avoid transformer failure due to operational stress.
Actors
RTU (Transformer Sensors); Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
X
92
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
60870-5-101
64 kbps
or
60870-5-104
Redundancy
Availability
Direction
N
L
UNI
Figure 43.
Actor 1
IED / Relay
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
1 / 5 Minutes
Actor 2
IED / Relay
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
Online Transformer Condition Monitoring Tabular & Graphic Information
93
4.2.10 Switchgear & Transformer Partial Discharge Monitoring
It has been observed that transformers and switchgear at sub-transmission and distribution voltage levels
typically exhibit increased partial discharge activity prior to causing a fault that activates protection schemes.
Transformer bushings can also exhibit increased partial discharge activity prior to failure. Continuously
monitoring partial discharge, combined with other health monitoring measures, enables predictive fault
diagnostics and can keep the risk of failure and workplace safety under control.
The advantages of online continuous partial discharge testing over periodic partial discharge testing are:
•
•
•
Interval-based testing could miss PD activities since PD varies by time. Continuous monitoring
overcomes this inherent flaw.
Online monitoring is superior to off-line testing since it is done under real operating conditions.
Continuous online monitoring reduces maintenance labor costs.
Title
Switchgear & Transformer Partial Discharge (PD) Monitoring
Description
Continuous online switchgear & transformer partial discharge monitoring to maximize the
uptime and reliability of switchgear and transformers.
Actors
PD Monitor; Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
X
94
Mechanism
Polled
Redundancy
N
Event
Monitor
Protocol
60870-5-101
or
60870-5-104
Availability
L
Figure 44.
Bandwidth
64 kbps
Direction
UNI
Actor 1
PD Sensor
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
1 / Minutes
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
Partial Discharge Monitoring Monitoring Tabular & Graphic Information
95
4.2.11 Fault Location Isolation & Service Restoration (FLISR)
As the name implies, Fault Location, Isolation, and Service Restoration (FLISR) refers to the ability to
automatically locate the fault, isolate the fault, and restore service in the distribution network. It is a “selfhealing” feature whose purpose is to minimize the impact of faults by serving portions of the loads on the
affected circuit by switching to other circuits. It reduces the number of customers that experience a sustained
power outage by reconfiguring distribution circuits. This will likely be the first wide spread application of
distributed intelligence in the grid.
Secondary substations can be connected to multiple primary substations. Normally, static power switch statuses
(open/closed) in the network dictate the power flow to secondary substations. Reconfiguring the network in the
event of a fault is typically done manually on site to operate switchgear to energize/de-energize alternate paths.
Automating the operation of substation switchgear allows the utility to have a more dynamic network where the
flow of power can be altered under fault conditions but also during times of peak load. It allows the utility to
‘shift’ peak loads around the network. Or, to be more precise, alters the configuration of the network to move
loads between different primary substations.
The FLISR capability can be enabled in two modes:
•
•
Managed centrally from DMS, or
Executed locally through distributed control via intelligent switches and fault sensors.
There are 3 distinct sub-functions that are performed:
Figure 45.
Three Steps in FLISR
1. Fault Location Identification
This sub-function is initiated by SCADA inputs, such as lockouts, fault indications/location, and, also, by
input from the Outage Management System (OMS), and in the future by inputs from fault-predicting
devices. It determines the specific protective device, which has cleared the sustained fault, identifies
the de-energized sections, and estimates the probable location of the actual or the expected fault. It
96
distinguishes faults cleared by controllable protective devices from those cleared by fuses, and identifies
momentary outages and inrush/cold load pick-up currents.
This step is also referred to as Fault Detection Classification & Location (FDCL). This step helps to
expedite the restoration of faulted sections through fast fault location identification and improved
diagnostic information available for crew dispatch. Also provides visualization of fault information to
design & implement a switching plan to isolate the fault.
2. Fault Type Determination
I.
Indicates faults cleared by controllable protective devices by distinguishing between:
a. Faults cleared by fuses
b. Momentary outages
c. Inrush/cold load current
II.
Determines the faulted sections based on SCADA fault indications and protection lockout
signals
III.
Increases the accuracy of the fault location estimation based on SCADA fault current
measurements and real-time fault analysis
3. Fault Isolation & Service Restoration
Once the location and type of the fault has been pinpointed the systems will attempt to isolate the fault
and restore the non-faulted section of the network. This can have three modes of operation:
I.
Closed-loop mode – This is initiated by the Fault location sub-function. It generates a
switching order (i.e., sequence of switching) for the remotely controlled switching devices to
isolate the faulted section, and restore service to the non-faulted sections. The switching
order is automatically executed via SCADA.
II.
Advisory mode – This is initiated by the Fault location sub-function. It generates a switching
order for remotely and manually controlled switching devices to isolate the faulted section,
and restore service to the non-faulted sections. The switching order is presented to
operator for approval and execution
III.
Study mode – the operator initiates this function. It analyzes a saved case modified by the
operator, and generates a switching order under the operating conditions specified by the
operator.
With the increasing volume of data that are collected through fault sensors utilities will be to use Big Data query
and analysis tools to study outage information to anticipate and prevent outages by detecting failure patterns
and their correlation with asset age, type, load profiles, time of day, weather conditions, and other conditions to
discover conditions that lead to faults and take the necessary preventive and corrective measures.
97
Title
Fault Location, Isolation, and Service Restoration (FLISR)
Description
FLISR is a self-healing feature enabled by sensors, automated switches, and knowledge of
the power distribution in a local area thus enabling reconfiguration of the feeders to
minimize the number of impacted customers after a fault.
Actors
Actors
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
Event
Measurement
Mechanism
Protocol
Bandwidth
Polled
60870-5-104
20 kbps
Redundancy
Availability
Direction
N
M
BI
Event
Fault Event
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-104
20 kbps
Redundancy
Availability
Direction
N
M
BI
X
X
X
Actor 1
RTU
Latency
0.5 Sec
Acknowledge
Y
Jitter
N/A
Trans. Rate
1 / sec
Actor 1
RTU
Latency
0.5 Sec
Acknowledge
Y
Jitter
N/A
Trans. Rate
1 / sec
X
Actor 2
Control Center
Packet Loss
Security
1%
H
Time Synch.
BER / PER
Y
…
Actor 2
Control Center
Packet Loss
Security
1%
H
Time Synch.
BER / PER
Y
…
98
Event
Switch Control
Mechanism
Protocol
Bandwidth
Polled
60870-5-104
20 kbps
Redundancy
Availability
Direction
N
M
BI
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
1 / sec
Actor 2
RTU
Packet Loss
Security
1%
H
Time Synch.
BER / PER
Y
…
Event
Fault Event
Mechanism
Protocol
Bandwidth
Ad hoc
61850
64 kbps
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Fault Sensor
Latency
Jitter
80mSec
40mSec
Acknowledge
Trans. Rate
Y
…
Actor 2
Protection Relay
Packet Loss
Security
0.1%
H
Time Synch.
BER / PER
Y
…
Figure 46.
Fault Location, Isolation & Service Restoration Tabular & Graphic Information
4.2.12 Voltage Regulation
99
A voltage regulator is a transformer with a tap changer that can increase or decrease the voltage on a
distribution circuit to help keep the voltage within a pre-determined band. They typically monitor the voltage at
the location where they are connected, comparing it to a pre-programmed set point. If the voltage deviates too
far from the set point, the voltage regulator can adjust its output voltage by moving the tap on the secondary
side up or down.
Voltage regulators play a key component in Volt/VAR control, stabilizing and flattening feeder voltages, thereby
reducing grid losses.
This use case applies to both transmission and distribution.
Title
Voltage Regulation
Description
Voltage regulation in distribution network using LTC transformers. LTCs can change tap
under load thereby changing the voltage on the secondary windings up or down as desired.
This use case applies to transmission network as well.
Actors
Voltage Sensor; Voltage Regulator; Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
Event
Monitor
X
X
X
X
X
X
X
X
Actor 1
Voltage Monitor
X
Actor 2
Control Center
100
Mechanism
Polled
Redundancy
Y
Protocol
60870-5-101
&
60870-5-104
Availability
M
Bandwidth
20 kbps
Latency
0.5 Sec
Jitter
N/A
Packet Loss
1%
Security
M
Direction
UNI
Acknowledge
Y
Trans. Rate
1 / 2 Seconds
Time Synch.
Y
BER / PER
…
Event
Event
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-101
20 kbps
&
60870-5-104
Redundancy
Availability
Direction
Y
M
UNI
Event
Control
Mechanism
Protocol
Bandwidth
Ad hoc
60870-5-101
20 kbps
&
60870-5-104
Redundancy
Availability
Direction
Y
M
UNI
Actor 1
Voltage Monitor
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
1 / 2 Seconds
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
1 / 2 Seconds
Actor 2
Control Center
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
Actor 2
Voltage Regulator
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
101
Figure 47.
Voltage Regulation Tabular & Graphic Information
102
4.2.13 Advanced Metering Infrastructure (AMI)
This use case describes the implementation of AMI based on open industry standards. AMI refers to the
collection of systems that measure, collect, and analyze energy usage and interact with end-point devices such
as electric meters, gas meters, and water meters. A typical AMI system records customer consumption at least
once an hour and transmits those measurements at least once a day. AMI requires a fixed communication
network with stationary transmitters and receivers and must provide two-way communications.
Here are some of the AMI benefits:
•
•
•
•
•
•
•
•
•
Provide real-time and detailed information on customer consumption
Enable better customer communication and outage notification
Provide an opportunity to offer time-based rates
Improve reliability and accuracy in meter reading
Improve visibility with embedded network diagnostic capabilities
Remote connect and disconnect of power
Over the air firmware updates and upgrades for meter registration and communication
Protect revenue
Facilitate customer conservation
In this use case we assume the implementation of a wireless IEEE 802.15.4g/e mesh neighborhood area network
(NAN) based on the 6LoWPAN adaptation layer, IPv6 and IPv6 Routing Protocol for Low Power and Lossy
Networks (RPL) standard to integrate the NAN into end-to-end network architecture.
The network requirements at a high level are:
•
•
•
•
•
•
Device authentication so that no access is granted to unknown devices
Encryption on all traffic
Prioritization of critical data when sharing the WAN with more critical Distribution Automation traffic
Network management for zero-touch provisioning
Data integrity, confidentiality, and privacy across the FAN
Strong authentication
Title
Description
Automated periodic measurement of end user energy usage along with 2-way
communication with remote connects / disconnect capability.
Actors
Smart meter; AMI Concentrator; Data Center
Transmission
Distribution
103
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
Event
Meter Data
Mechanism
Protocol
Bandwidth
Polled
ANSI C12 /
20 kbps / Meter
IEEE 1377
Redundancy
Availability
Direction
Y
H
BI
Event
Meter Data
Mechanism
Protocol
Bandwidth
Polled
ANSI C12 /
20 kbps / Meter
IEEE 1377
Redundancy
Availability
Direction
Y
H
BI
Event
Command
Mechanism
Protocol
Bandwidth
Polled
ANSI C12 /
20 kbps / Meter
IEEE 1377
Redundancy
Availability
Direction
Y
H
BI
X
X
X
Actor 1
Smart Meter
Latency
Jitter
1 Min
…
Acknowledge
Y
Trans. Rate
1 / Day
Actor 1
AMI Concentrator
Latency
Jitter
1 Min
…
Acknowledge
Y
Trans. Rate
1 / Day
Actor 1
Data Center
Latency
Jitter
5 Sec
…
Acknowledge
Y
Trans. Rate
1 / Day
X
X
X
Actor 2
AMI Concentrator
Packet Loss
Security
1%
H
Time Synch.
Y/N
BER / PER
…
Actor 2
Data Center
Packet Loss
Security
1%
H
Time Synch.
Y/N
BER / PER
…
Actor 2
AMI Concentrator
Packet Loss
Security
1%
H
Time Synch.
Y/N
BER / PER
…
104
Event
Command
Mechanism
Protocol
Bandwidth
Polled
ANSI C12 /
20 kbps / Meter
IEEE 1377
Redundancy
Availability
Direction
Y
H
BI
Figure 48.
Actor 1
AMI Concentrator
Latency
Jitter
5 Sec
…
Acknowledge
Y
Trans. Rate
1 / Day
Actor 2
Smart meter
Packet Loss
Security
1%
H
Time Synch.
Y/N
BER / PER
…
AMI Tabular & Graphic Information
105
4.2.14 Integrated Grid-Scale Energy Storage
This use case is relevant to Transmission, Distribution, and Energy Supply categories. Energy storage
technologies have been used by utilities to balance the variability of renewable generation and to reinforce the
electricity grid, enabling it to accommodate the increase in peak demands without the need to invest in new
generation or T&D expansion.
Grid-scale energy storage technologies include pumped hydro
• Batteries
• Flywheels
• Superconducting magnetic energy storage
• Ultra-capacitors
• Compressed air storage, and
• Aggregated plug-in electric vehicles
The benefits of grid-scale energy storage include:
•
•
•
•
•
DMS-controllable devices in the distribution grid
Used as Demand Response instruments to shave or shift peak load
Used as dispatchable resources inside a microgrid to balance local supply and demand in a local context
Used as voltage and reactive power support instruments
In aggregate, these devices can provide both real and reactive power support at the transmission level
However, the use of these resources requires real-time communications for monitoring and control purposes.
Title
Integrated Grid-Scale Energy Storage
Description
Grid-scale energy storage resources are increasingly used by utilities in both transmission
and distribution grid for a variety of reasons including renewable integration, Demand
Response, and voltage and reactive power support.
Actors
Energy Storage Controller; Control Center; IED (current/voltage/frequency sensors)
Transmission
Distribution
Energy Supply
Demand Side
X
X
X
Other (Specify)
Places in the Grid
X
X
X
X
106
Customer Premise
Control Center
Data Center
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
60870-5-101
20 kbps
&
60870-5-104
Redundancy
Availability
Direction
N
L
BI
Event
Alert
Mechanism
Protocol
Bandwidth
Event
60870-5-101
20 kbps
&
60870-5-104
Redundancy
Availability
Direction
N
L
BI
Event
Alert
Mechanism
Protocol
Bandwidth
Event
60870-5-101
20 kbps
&
60870-5-104
Redundancy
Availability
Direction
N
L
BI
X
X
Actor 1
IED
Latency
0.5 Sec
Jitter
N/A
Acknowledge
Y
Trans. Rate
1 / 2 Seconds
Actor 1
IED
Latency
0.5 Sec
Jitter
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
1 / 2 Seconds
X
X
Actor 2
Control Center
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
Actor 2
Energy Storage Controller
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
107
Figure 49.
Integrated Grid-Scale Energy Storage Tabular & Graphic Information
108
4.3 Energy Supply Use Cases
This section includes use cases related to centralized power plants, grid scale storage, and distributed
generation and storage resources.
4.3.1 Frequency Control / Automatic Generation Control (AGC)
The system frequency should be maintained within a very narrow band. Deviations from the acceptable
frequency range are detected and forwarded to the Load Frequency Control (LFC) system so that required up or
down generation increase / decrease pulses can be sent to the power plants for frequency regulation. The trend
in system frequency is a measure of mismatch between demand and generation, and is a necessary parameter
for load control in interconnected systems.
Automatic generation control (AGC) is a system for adjusting the power output of generators at different power
plants, in response to changes in the load. Since a power grid requires that generation and load closely balance
moment by moment, frequent adjustments to the output of generators are necessary. The balance can be
judged by measuring the system frequency; if it is increasing, more power is being generated than used, and all
machines in the system are accelerating. If the system frequency is decreasing, more demand is on the system
than the instantaneous generation can provide, and all generators are slowing down.
Where the grid has tie lines to adjacent control areas, automatic generation control helps maintain the power
interchanges over the tie lines at the scheduled levels. The AGC takes into account various parameters including
the most economical units to adjust, the coordination of thermal, hydroelectric, and other generation types, and
even constraints related to the stability of the system and capacity of interconnections to other power grids.
For the purpose of AGC we use static frequency measurements and averaging methods are used to get a more
precise measure of system frequency in steady-state conditions.
During disturbances, more real-time dynamic measurements of system frequency are taken using PMUs,
especially when different areas of the system exhibit different frequencies. But that is outside the scope of this
use case.
Title
Frequency Control / Automatic Generation Control (AGC)
Description
AGC maintains close balance between total load and total generation in a control area by
tracking system frequency as a measure of load-generation imbalance and by sending
control signals to power plants to raise or lower their output accordingly.
Actors
Frequency Sensor; Generation Plant RTU; Control Center
Transmission
109
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
Event
Frequency Value
Mechanism
Protocol
Bandwidth
Event…
20 kbps
triggered
Redundancy
Availability
Direction
Y
M
BI
Event
Generator Output
Mechanism
Protocol
Bandwidth
Event60870-5-101
20 kbps
triggered
&
60870-5-104
Redundancy
Availability
Direction
Y
M
BI
Event
Alert
Mechanism
Protocol
Bandwidth
Event60870-5-101
20 kbps
triggered
&
60870-5-104
Redundancy
Availability
Direction
X
X
Actor 1
PMU
Latency
1 Sec
Jitter
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
Generation Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
1 / sec
Actor 1
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
X
Actor 2
Control Center
Packet Loss
Security
5%
M
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
5%
H
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
1%
H
Time Synch.
BER / PER
110
Y
M
BI
Event
AGC Control
Mechanism
Protocol
Bandwidth
Event60870-5-101
20 kbps
triggered
&
60870-5-104
Redundancy
Availability
Direction
Y
M
BI
Figure 50.
Y
1 / sec
Actor 1
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
1 / sec
Y
…
Actor 2
Control Center
Packet Loss
Security
1%
H
Time Synch.
Y
BER / PER
…
Automatic Generation Control Tabular & Graphic Information
111
4.3.2 Hydroelectric Power Plants – General Telemetry
There are many parameters in the hydro power generation cycle that are monitored for safety and reliability
operation of the plant and the power system. In this section we review the general communication
requirements for a hydro plant. In the following sections we will review some of the more specific parameters
and their communication requirements.
The following is a list of some of the specific parameters that are monitored:
•
•
•
•
•
•
•
Oscillographs to detect earth movements
Turbine vibration sensors
Dam Control
Seismometer
Security Cameras
Water level surveillance
Surveillance of the level of ice over the dam using limnimeters
Title
Hydroelectric Power Plants general Telemetry
Description
General pattern of telemetry for a hydroelectric plant.
Actors
Plant RTU; Control Center
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
112
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
61850-7-410
20 kbps
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
1 / 2 Seconds
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
BER / PER
Y
…
Event
Alert
Mechanism
Protocol
Bandwidth
Ad hoc
61850-7-410
20 kbps
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
1 / 2 Seconds
Actor 2
Control Center
Packet Loss
Security
1%
L
Time Synch.
BER / PER
Y
…
Event
Control
Mechanism
Protocol
Bandwidth
Pushed
61850-7-410
20 kbps
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
1 / 2 Seconds
Actor 2
Plant RTU
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
113
Figure 51.
Hydro Plant General telemetry Tabular & Graphic Information
114
4.3.3 Hydro Plant Dam Leakage Monitoring
This use case describes the dam leakage monitoring telemetry requirements.
Event
Dam Water Level
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Event
Alarm
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 2
Control Center
Packet Loss
Security
5%
H
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
5%
H
Time Synch.
Y
BER / PER
…
All other attributes are the same as for the hydro plant general telemetry use case.
115
Figure 52.
Hydro Plant Dam leakage Supervision Tabular & Graphic Information
116
4.3.4 Hydro Plant Gate Position Indicator
This use case describes the gate position indicator telemetry requirements.
Event
Gate Position
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Event
Upper- or Lower-End Position Reached
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
117
Figure 53.
Hydro Plant Gate Position Indicator Tabular & Graphic Information
118
4.3.5 Hydro Plant Water Flow Control
This use case describes the water flow control telemetry requirements.
Event
Water Flow
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Event
Maximum / Minimum Flow Reached Indicator
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
119
Figure 54.
Hydro Plant Water Flow Control Tabular & Graphic Information
120
4.3.6 Hydro Plant Water Level Indicator
This use case describes the water level indicator telemetry requirements.
Event
Water Level
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Event
Measuring Device Failure Alarm
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
121
Figure 55.
Hydro Plant Water Level Indicator Tabular & Graphic Information
122
4.3.7 Hydro Plant Dam Over-Topping Protection
This use case describes the dam over-topping protection telemetry requirements. If over-topping is detected the
protection will open one or more of the controllable gates.
Event
Over-Topping Level Reached
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Event
Gate Control
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 2
Control Center
Packet Loss
Security
5%
H
Time Synch.
Y
BER / PER
…
Actor 2
Plant RTU
Packet Loss
Security
5%
H
Time Synch.
Y
BER / PER
…
123
Figure 56.
Hydro Plant Water Over-Topping Protection Tabular & Graphic Information
124
4.3.8 Hydro Plant Turbine Vibration Monitoring
Turbine vibration monitoring is done using a series of sensors in the turbines and other equipment to identify
possible vibrations that might provide useful information for proactive maintenance and can also help identify
and avoid harmful situations that cause damage to the turbine and associated systems.
Event
Turbine Vibration Monitoring
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Figure 57.
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 2
Control Center
Packet Loss
Security
5%
H
Time Synch.
Y
BER / PER
…
Hydro Plant Vibration Monitoring Tabular & Graphic Information
125
4.3.9 Hydro Plant Dam Deformation Monitoring
Dam deformation monitoring is done to monitor the integrity of the dam structure and helps avoid possible
issues with catastrophic consequences.
Event
Dam Deformation Monitoring
Mechanism
Protocol
Bandwidth
Event61850-7-410
20 kbps
triggered
Redundancy
Availability
Direction
Y
H
BI
Figure 58.
Actor 1
Plant RTU
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
…
Actor 2
Control Center
Packet Loss
Security
5%
H
Time Synch.
Y
BER / PER
…
Hydro Plant Dam Deformation Monitoring Tabular & Graphic Information
126
4.3.10 Wind Farm Operation
There is about 1,349MW in Hydro Quebec’s generation portfolio from 15 wind farms operated by independent
power producers.
IEC standard 61400-25 defines standardized models for SCADA and data exchange and their corresponding
mapping to communication profiles for monitoring and control of wind power plants.
Title
Wind Farm Operation
Description
Monitoring and control of wind farms
Actors
RTU; Control Center
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
61400-25 &
20 kbps
60870-5-104
Redundancy
Availability
Direction
X
X
X
Actor 1
RTU
Latency
0.5 Sec
Jitter
N/A
Acknowledge
Trans. Rate
X
X
Actor 2
Control Center
Packet Loss
Security
5%
L
Time Synch.
BER / PER
127
Y
M
BI
Event
Event
Mechanism
Protocol
Bandwidth
Ad hoc
61400-25 &
20 kbps
60870-5-104
Redundancy
Availability
Direction
Y
M
BI
Event
Control
Mechanism
Protocol
Bandwidth
Event61400-25 &
20 kbps
triggered
60870-5-104
Redundancy
Availability
Direction
Y
H
BI
Y
1 / 2 Seconds
Actor 1
RTU
Latency
0.5 Sec
Jitter
N/A
Acknowledge
Y
Trans. Rate
…
Actor 1
Control Center
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Y
Trans. Rate
1 / 2 Seconds
Y
…
Actor 2
Control Center
Packet Loss
Security
1%
L
Time Synch.
Y
BER / PER
…
Actor 2
RTU
Packet Loss
Security
1%
M
Time Synch.
Y
BER / PER
…
128
Figure 59.
Wind Farm Operation Tabular & Graphic Information
In reality since H-Q does not operate these wind farms, the wind farm information is received by H-Q through an
ICCP link between the wind farm operator’s control center and the H-Q Control Center. This scenario is shown in
the following diagram.
Figure 60.
Wind Farm Operation at H-Q Graphic Information
For the ICCP link network requirements refer to the ICCP use case description in the Transmission use cases
section.
129
4.3.11 Wind Farm Maintenance
The random behavior of wind speed determines the energy produced by a wind turbine. The same factor also
influences the turbine degradation process due to the random load that the turbine is subjected to. As a result,
wind turbines undergo a degradation process more complex than the equipment that work under stationary
conditions.
Wind farms are typically instrumented to detect early signs of failure and addressing the issue to avoid disruptions
and therefore increasing the system’s uptime. Wind turbine technological diversity and geographical spread of
wind farms present significant operation and maintenance challenges. The solution is continuous remote
monitoring of all operating parameters, both electrical and mechanical, including speed, frequencies of vibration
on turbine components, oil pressure, gearbox assembly, etc.
This data is processed at the local SCADA system within the farm and also transmitted to a centralized
maintenance and monitoring center to be analyzed for preventive and corrective maintenance.
Analysis of available telemetry data using condition-monitoring systems and algorithms and dispatching crew to
address the issues before failure is critical to decreasing operating costs.
Title
Wind Farm Maintenance
Description
Continuous monitoring of the mechanical and electrical components in the wind farm for
preventive and corrective maintenance.
Actors
Sensors/IED; Local SCADA; Data Center
Transmission
Distribution
Energy Supply
Demand Side
X
Other (Specify)
Places in the Grid
Customer Premise
X
X
X
130
Control Center
Data Center
X
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
61400-25 &
20 kbps
60870-5-104
Redundancy
Availability
Direction
N
M
BI
Event
Event
Mechanism
Protocol
Bandwidth
Polled
61400-25 &
20 kbps
60870-5-104
Redundancy
Availability
Direction
N
M
BI
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
61400-25 &
20 kbps
60870-5-104
Redundancy
Availability
Direction
N
M
BI
Event
Event
Mechanism
Protocol
Bandwidth
Polled
61400-25 &
20 kbps
60870-5-104
Redundancy
Availability
Direction
N
M
BI
Actor 1
Sensors/IED
Latency
Jitter
1 Sec
N/A
Acknowledge
Trans. Rate
…
Actor 1
Sensors/IED
Latency
Jitter
1 Sec
N/A
Acknowledge
Trans. Rate
…
Actor 1
Local SCADA
Latency
Jitter
1 Sec
N/A
Acknowledge
Trans. Rate
…
Actor 1
Sensors/IED
Latency
Jitter
1 Sec
N/A
Acknowledge
Trans. Rate
…
X
X
Actor 2
Local SCADA
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
Actor 2
Local SCADA
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
Actor 2
Maintenance Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
Actor 2
Maintenance Center
Packet Loss
Security
5%
L
Time Synch.
Y
BER / PER
…
131
Figure 61.
Wind Farm Maintenance Tabular & Graphic Information
However, since H-Q does not maintain these wind farms, the wind farm information could be received by H-Q
through an ICCP link between the wind farm operator control center and the H-Q Control Center, as shown
below.
Figure 62.
Wind Farm Operation at H-Q Graphic Information
For the ICCP link network requirements refer to the ICCP description in the Transmission use cases section.
132
4.4 Extreme Contingencies
Based on a detailed analysis of data and accumulated experience from significant contingencies over the past
30+ years Hydro Québec has developed defensive plans against extreme contingencies to safeguard the
reliability of the system for continuity of service to its customers.
4.4.1 H-Q Transmission System Characteristics
Hydro Québec’s long transmission lines, harsh weather, and customers’ heavy reliance on electricity, altogether
demand the highest level of security and reliability considerations in system design. H-Q’s transmission system is
designed to have successive line of defense to counter events that are increasingly more severe but also
increasingly more rare.
H-Q’s system has no synchronous links with neighboring systems. Furthermore, the extensive 735kV
transmission network has a relatively limited number of lines located in two major corridors, each about a
1000km in length. Because of these characteristics, stability and voltage control become critical issues. The first
major corridor extends northwest up to the James Bay hydroelectric complex (15 000 MW) and the second
corridor extends northeast up to Churchill-Falls and integrates about 14,000 MW of generation. There are also
thirty-one 735kV substations, 11,200 MVAR in series compensation, one 1200-km 450-kV dc line, dynamic shunt
compensation consisting of 11 static compensators and nine synchronous compensators, and about 3,900 MW
of DC interconnections with neighboring systems. The following figure shows the layout of the 735kV
transmission system4.
4
Designing a Reliable Power System: Hydro Québec’s Integrated Approach
133
Figure 63.
Hydro Québec’s 735kV Transmission System
4.4.2 Extreme Contingencies Criteria
Disturbances that can affect the electrical and mechanical integrity of the system fall into three categories:
1. Natural causes (lightening, storms, cold, ice, forest fires, and geomagnetic storms)
2. Equipment outage or protection system failures
3. The human factor (operating errors, vandalism, design flaws, etc.)
Depending on the triggering disturbance event, H-Q’s system is exposed to a diverse set of power system
134
phenomena:
•
•
•
•
Transient instability
Dynamic instability (interregional oscillations at 0.5 Hz)
Voltage instability
Frequency instability (over- or under-frequency)
Hydro Québec’s concept of successive line of defense hinges on 4 principles:
Principle 1: Service continuity must be assured following events most likely to occur on the power
system.
Principle 2: H-Q’s power system must include ways of avoiding system-wide power failures under
extreme contingencies.
Principle 3: Strategic equipment must not sustain any damage in the event of a general outage to
ensure that system restoration is always an option.
Principle 4: H-Q’s transmission system must be designed so as to allow the system to be restored within
a reasonable period after a catastrophic event.
The following figure summarizes H-Q’s concept of successive line of defense.
Figure 64.
5
Hydro Québec’s Concept of Successive Line of Defense 5
135
The following figure shows the event(s) that constitute an extreme contingency in H-Q’s transmission system
and the performance that is required in each case.
Figure 65.
Contingencies & Corresponding Performance Requirements 6
4.4.3 Special Protection Schemes (SPS)
Hydro Québec’s has devised the following automatic actions and special protection schemes to counter the
extreme contingencies.
6
136
Special Protection Schemes
7
10
Level 1
Limited Action
Level 2
Use of Corrective
SPS Types
Level 3
Use of SPS with
massive actions
Loss of two series or parallel 735kV
lines
AC-DC event; loss of a bipolar line with
loss of one 735kV line
Loss of a generating station or
generation unit at a station
Sudden loss of a major load center
Unintended operation of an SPS
Loss of lines and bypass of all series
capacitors on the remaining lines in
the same corridor
Loss of all 735kV lines in a corridor
Loss of all 735kV lines originating from
a substation
Figure 66.
UnderFreq. Load
Shedding
(UFLS)
9
Generation Rejection (GR) and
12
Remote Load Shedding (RLS)
UnderVoltage
Load
Shedding
8
(UVLS)
Limited Generation Rejection (GR)
Shunt Reactor Tripping
Extreme Contingencies
Shunt Reactor Closing
Response Levels
Remote Tripping of Shunt Reactor
13
(RTS)
RPTC
11
MAIS
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Extreme Contingencies & Relevant Special Protection Schemes
As can be seen from the above figure, H-Q’s Special Protection Systems include:
•
Automatic Shunt Reactor Systems (MAIS) – installed in twenty-two 735kV substations that control about
7
MAIS – Automatic 735kV shunt reactor closing or tripping
UVLS – Under-Voltage Load Shedding
9
UFLS – Under-Frequency Load Shedding
10
RPTC – Combination of GR, RLS, and RTS
11
GR – Generation Reaction
12
RLS – Remote Load Shedding
13
RTS – Remote tripping of shunt reactor
8
137
15,000 MVAR. Each MAIS device relies on a high precision, local voltage measurement while
coordination between substations is done through switching-delay settings.
•
Under-Frequency Load Shedding System (UFLS) – installed in about 150 distribution substations that can
access over 13,000 MW of load. UFLS is intended to restore the generation/load balance after an
extreme contingency. Unlike other more interconnected systems, due to absence of synchronous links
between H-Q’s system and its neighbors’, UFLS can be relied on as a selective and effective measure to
detect loss of generation, up to the amount of the system’s largest power station (5,300 MW).
•
Under-Voltage Load Shedding System (UVLS) or (TDST) – able to shed 1,500 MW of load mainly in
735kV substations in Montréal area. Load shedding is considered as a last resort and a corrective
measure against voltage instability particularly when there is a severe initial drop in voltages that cannot
be tolerated for a long time. TDST is a response-based SP, due to the nature of long-term voltage
instability, and relies on voltage measurements taken in 5 substations in the Montréal area with high
prevision MAIS transformers. TDST operates in a pre-defined load shedding arrangement with its control
restricted to certain percentage of the load and the set of distribution breakers that can be opened is
pre-defined.
•
Generation Rejection, Load Shedding, and Remote Reactor Tripping at 735kV (RPTC) – designed to
detect multiple line losses or series-compensated capacitor bank tripping in 15 strategic 735kV
substations. In addition to traditional under-frequency load shedding actions, H-Q has implemented an
extensive generation rejection and remote load-shedding scheme to handle transient angular instability
problems. RPTC is an event-based SPS due to the speed of angle instability phenomenon. RPTC is an
effective measure against contingencies that cause loss of transmission capacity, without involving a loss
of load or generation that lead to rapid instability. To be effective, the corrective measures should be
taken very fast, for example generation rejections and remote load shedding must be executed no later
than 0.25 s and 0.33 s respectively after the event is detected 14. In each substation in the RPTC, there
are local programmable devices called contingency composition units (CCU) that analyze individual
losses of lines and individual losses of series compensation banks to classify the contingencies by
severity and flag those for which actions are required. GR systems and RLS systems receive their
respective orders to operate directly from the CCUs in each substation.
4.4.4 Telecom Requirements for Extreme Contingencies
Due to the fast reaction times and the critical importance of SPS to maintain power system electrical integrity,
SPS have the most stringent communication requirements in terms of availability and low latency. These
requirements are highlighted in the following figure.
Network Parameter
14
Network Requirement
Hydro Québec’s Defense Plan Against Extreme Contingencies, 1999
138
Latency
Jitter
VPN Topology
Bit Error Rate
Physical Path Redundancy
Figure 67.
4 to 10ms
Yes
<750µs
99.9999 / High
Point-to-Point and Point-to-Multipoint
Yes
10-6 under normal condition and 10-5 with a fault on the network
<50ms – Hitless
Yes, Mandatory
Yes
Special Protection Schemes Telecom Requirements
These SPSs protect the system against events that threaten the power system electrical integrity.
For extreme contingencies that threaten the power system mechanical integrity, such as damage to the physical
infrastructure caused by severe storms, maintaining voice connectivity to critical substations is vital. This
requirement calls for redundant communication systems and separate physical communication paths to ensure
network connectivity in order to restore power when the power system goes down.
In conclusion, for any extreme contingency event, at least one path of the
redundant communication systems shall always be up and running, available
and reliable, for critical electric services and voice.
4.4.5 NPCC requirements for SPS Communication Systems
NPCC, in the Regional Reliability Reference Directory # 7 – Special Protection Systems 15, has established specific
mandatory requirements for communication systems that are required for Special Protection Systems. These
requirements are summarized here:
•
Communication System Performance – Communication facilities required for teleprotection shall be
designed to have a level of performance consistent with that required of the SPS
•
Communication System Redundancy – Where the design of an SPS is composed of multiple protection
groups for redundancy and each group requires a communication channel:
o
15
The equipment for each group shall be separated physically on non-adjacent panels and
designed to minimize the risk of more than one protection group being disabled simultaneously
by a single event or condition.
NPCC Regional reliability reference Directory # 7 - Special Protection Systems
139
o
The communication medium outside the substation/plant physical perimeter for each
protection group shall be designed to minimize the risk of both protection groups being disabled
simultaneously by a single event or condition. In addition, physical separation of the
communication media outside the substation fence shall be three feet at a minimum.
•
Communication System Monitoring – Communication link availability shall be annunciated to a 24-hour
Operations center so that operating personnel can be notified and can initiate appropriate actions.
•
Separate Power Source – Teleprotection equipment shall be powered by the substation batteries or
other sources independent from the power system
•
Communication Signal Diversity – The use of a single communication tower for radio communication
systems used by the two protection groups of an SPS is permitted as long as diversity of the
communication signals is achieved
•
Communication System Ruggedness – SPSs, including intelligent electronic devices (IEDs) and
communication systems used for protection, should comply with applicable industry standards for utility
grade protection service. Utility Grade Protection System Equipment are equipment that are suitable for
protecting transmission power system elements that are required to operate reliably, under harsh
environments normally found at substations. Utility grade equipment should meet the applicable
sections of all or some of the following types of industry standards, to ensure their suitability for such
applications:
o
o
o
o
o
o
o
o
o
o
•
IEEE C37.90.1 (oscillatory surge and fast transient)
IEEE C37.90.1 (service conditions)
IEC 60255-22-1 (1 MHz burst, i.e. oscillatory)
IEC 61000-4-12 (oscillatory surge)
IEC 61000-4-4 (EFT)
IEC 60255-22-4 (EFT)
IEEE C37.90.2 (narrow-band radiation)
IEC 60255-22-3 (narrow-band radiation)
IEC 61000-4-3 (narrow-band radiation)
IEEE 1613 (communications networking devices in Electric power Substations)
Communication Medium for Teleprotection – In cases where constraints do not allow three feet
separation, this distance may be reduced if a proposed alternative design can achieve comparable
physical protection of the communication medium. If an alternative design cannot be met, then an
alternative communication path or protection scheme should be proposed.
140
4.5 Demand Side
4.5.1 Hour-Ahead Load Optimization – Demand Response
Reduce power congestion due to weather forecast (e.g. temperature drop) by triggering the need for demand
response. The demand response will typically be controlled by the supply business and may be activated using
radio signals, telephone calls or via the smart meter infrastructure.
In this use case we assume DR is activated through the AMI infrastructure.
Title
Hour-Ahead Load Optimization – Demand Response
Description
The utility needs to curtail load through the AMI infrastructure due to a sudden change in
weather changes.
Actors
Actors
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
Event
DR Request
Mechanism
Protocol
Bandwidth
Ad hoc
DLMS
9.6 kbps
Redundancy
Availability
Direction
X
X
Actor 1
DR Headend
Latency
Jitter
1 Min
N/A
Acknowledge
Trans. Rate
X
X
X
Actor 2
AMI Concentrator
Packet Loss
Security
5%
H
Time Synch.
BER / PER
141
N
L
BI
Event
DR Request
Mechanism
Protocol
Bandwidth
Ad hoc
DLMS
9.6 kbps
Redundancy
Availability
Direction
N
L
BI
Figure 68.
Y
…
Actor 1
AMI Concentrator
Latency
Jitter
1 Min
N/A
Acknowledge
Trans. Rate
Y
…
Y
…
Actor 2
Smart Meter
Packet Loss
Security
5%
H
Time Synch.
BER / PER
Y
…
Hour-Ahead Load Optimization with DR Through AMI Tabular & Graphic Information
142
4.5.2 Electric Vehicle Charging
This use case describes support for residential consumers with an EV charging device and vehicle to grid (V2G)
services. The two-way communication with the charging device enables more accurate forecasting of potential
peak demands. The solution may be as simple as an additional EV Smart Meter or it can be extended to a
sophisticated EV charging point whereby V2G services are transacted through a service provider. Once the
charge / discharge status of the vehicle is known, the second step is to enable intelligent charging based on
environmental and grid conditions. This may be driven by demand management signals such as Time of Use
(TOU) tariffs or by direct load control signals.
Title
Electric Vehicle Charging
Description
Electric vehicle charging and V2G services.
Actors
Actors
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
Event
Monitor
Mechanism
Protocol
Bandwidth
Polled
61850
64 kbps
Redundancy
Availability
Direction
N
M
BI
X
X
Actor 1
Charging Device
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
…
X
X
X
Actor 2
Control Center
Packet Loss
Security
5%
M
Time Synch.
BER / PER
Y
…
143
Event
Event
Mechanism
Protocol
Bandwidth
Polled
61850
64 kbps
Redundancy
Availability
Direction
N
M
BI
Actor 1
Charging Device
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
Control Center
Packet Loss
Security
5%
M
Time Synch.
BER / PER
Y
…
Event
Charge Request / Authorization
Mechanism
Protocol
Bandwidth
Ad hoc
Proprietary
64 kbps
Redundancy
Availability
Direction
N
M
BI
Actor 1
Charging Device
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
Control Center
Packet Loss
Security
5%
M
Time Synch.
BER / PER
Y
…
Figure 69.
Electric Vehicle Charging Tabular & Graphic Information
144
4.5.3 Automated Demand Response with Water Heaters
Demand Response (DR) is a mechanism for managing customer load in response to supply conditions, such as a
change in price, or in response to grid conditions, such as generation/load imbalance leading to potential
outages and grid reliability incidents. In the past, most DR was implemented manually through a fax or a phone
call. The utility would notify the DR participants a day in advance for lowering their consumption. Today, with
the establishment of communication links to smart loads, DR can be activated in much shorter time frames.
This use case describes the control of water heaters for demand response.
There are generally two flavors of DR:
1) Slow DR – where the DR signals are sent significantly before the events are called, e.g. day-ahead.
Typical slow DR lead times are in hours and days.
2) Fast DR – where faster than usual response times are required. These programs have lead times of
seconds (e.g. 4 sec response time), used for load balancing and frequency stabilization, for example
ancillary services and regulation services.
The current industry standard for automated demand, OpenADR216, is a flexible protocol to facilitate
information exchange between energy service providers, aggregators, and end users. It enables the
implementation of two-way signaling between service providers and automated clients, or in OpenADR
terminology the Virtual Top Node (VTN) and Virtual End Node (VEN).
The following figure 17 shows the possible relationships of VTN and VEN in OpenADR.
16
OpenADR Alliance is an industry group for the development, adoption, and compliance of the Open
Automated Demand Response (OpenADR) standards.
17
OpenADR 2.0 Profile Specification – B Profile
145
Figure 70.
Possible Relationships of VTN and VEN in OpenADR2 [Source: OpenADR2 Spec]
Operational Security Assumptions & Principles
With regard to securing demand side networks and resources, there are useful assumptions and principles
offered in the “OpenADR Security Profile”18 document that we would like to summarize here:
•
•
•
•
•
•
•
18
Risks associated with the compromise of each DR asset will be different based on the compromised DR
asset’s capabilities.
DR is not part of critical grid operations unless DR resource/asset gives full commitment to accurately
follow DR instructions.
DR resource can be used to enhance grid reliability or to facilitate market operations. However,
regulations and legal agreement require a separation between electric system operations and market
functions.
DR controlling entity (e.g. the ISO/RTO, Transmission Operator, DR Aggregator), may have little or no
control over the physical environment in which DR assets reside in.
Security controls should have minimal impact on the primary mission of DR.
OpenADR should employ different types of security measures depending on the risks associated with
different types of DR events in order to facilitate efficient operations of OpenADR applications:
o If personally identifiable information is communicated, confidentiality becomes important.
o If direct load control is introduced, integrity becomes important.
o If faster response times are required, availability and low latency become important.
All control activity on the DR system shall be auditable.
“Security Profile for OpenADR”, 2011
146
•
The integration of DR systems should not expose other utility systems to unauthorized access or attack.
DR systems should support non-repudiation of all transactions between the DR controlling entity and DR
resource/asset.
In this use case we describe the “B Profile” of OpenADR2. OpenADR2 messages are sent over the Internet using
HTTPS protocol.
Title
Demand Response using OpenADR2
Description
Demand Response is a mechanism for managing customer load in response to supply
conditions, such as a change in price, or in response to grid conditions, such as
generation/load imbalance leading to potential outages and grid reliability incidents
Actors
DR Service Provider; DR Participant
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
Service Provider
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
Event
DR Event
Mechanism
Protocol
Bandwidth
Push or Pull
HTTP
20 kbps
Redundancy
Availability
Direction
N
L
BI
X
X
Actor 1
DR Service Provider
Latency
Jitter
1 Min
N/A
Acknowledge
Trans. Rate
Y
…
X
X
Actor 2
DR Participant
Packet Loss
Security
5%
M
Time Synch.
BER / PER
N
…
147
Event
Report
Mechanism
Protocol
Bandwidth
Push or Pull
HTTP
64 kbps
Redundancy
Availability
Direction
N
L
BI
Actor 1
DR Service Provider
Latency
Jitter
1 Min
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
DR Participant
Packet Loss
Security
5%
M
Time Synch.
BER / PER
N
…
Event
Register
Mechanism
Protocol
Bandwidth
Push or Pull
HTTP
64 kbps
Redundancy
Availability
Direction
N
L
BI
Actor 1
DR Service Provider
Latency
Jitter
1 Min
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
DR Participant
Packet Loss
Security
5%
M
Time Synch.
BER / PER
N
…
Event
Opt In / Out
Mechanism
Protocol
Bandwidth
Push or Pull
HTTP
20 kbps
Redundancy
Availability
Direction
N
L
BI
Actor 1
DR Participant
Latency
Jitter
1 Min
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
DR Service Provider
Packet Loss
Security
5%
M
Time Synch.
BER / PER
N
…
148
Figure 71.
Demand Response Using OpenADR2 Tabular & Graphic Information
149
4.5.4 Customer Premise Network Integration
This use case describes integration between the utility distribution system and the customer premise network to
give the utility a better understanding of actions behind the meter, e.g. distributed generation profile, and
varying load a profiles that affect distribution operations.
For customers’ participation in grid related transactions, be it market-driven or initiated by a utility call for
reliability events is best handled through existing protocols such as OpenADR, whereby message exchanges
between all participating parties are pre-defined and roles and responsibilities of each party is spelled out.
For interactions that take place between the utility and a group of customers that are within the same local area
(a neighborhood or a community) there are two possible scenarios:
1. The customers are represented through an aggregator or an energy service provider whereby the
aggregator acts as a proxy for customers in all interactions with the utility. There are two contracts, one
between the aggregator and the utility, and another between the aggregator and the customers. This
applies for example to customers who have a leased rooftop PV system installed and operated by a
service provider that guarantees the customer protection against rate changes and rate increases.
2. The second scenario is when customers are part of a microgrid. Their overall energy usage is balanced
by existing active resources (distributed generation and storage) within the neighborhood or campus)
and any excess power (or shortage of power) is balanced by the grid. In this scenario once can think of
the possibility of an isolated microgrid whereby the system is separated from the grid and is operated
autonomously. The system can be reconnected back to the grid in a controlled process that synchronizes
frequency and angle at the point of common coupling.
To this date there are not many., if any, systems whereby the utility has direct control over smart appliances
inside the residential premise, except for those that are already covered under some type of demand response
agreement that we address in the previous section. Typically, control of in-house or on-premise devices are
entirely up to the site owner with little utility involvement. The only visible interface is either through the meter
(as discussed under AMI use case) or through some form of demand response as we have seen previously.
150
4.6 Mobility & Collaboration
This section includes mobility and collaboration use cases including voice, video, and data connectivity for the
mobile workforce, radio interoperability, and field coordination for emergency response and service restoration.
4.6.1 Field Workforce Voice over IP (VoIP)
When the mobile worker at the substation makes a phone call, a session is established to the destination phone,
via the IP PBX. Once the call session is established, VoIP packets flow directly between the IP phones.
Title
Field Workforce – IP Telephony
Description
Phone call from a substation to outside using IP Telephony.
Actors
Field Handset; Desktop Handset; Call Manager
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
Event
Voice Stream
Mechanism
Protocol
Bandwidth
Stream
RTP
80 kbps
Redundancy
Availability
Direction
X
X
X
X
X
Actor 1
Field Handset
Latency
Jitter
150ms
30ms
Acknowledge
Trans. Rate
X
X
Actor 2
Desktop Handset
Packet Loss
Security
1%
L
Time Synch.
BER / PER
151
Y
H
BI
Event
Call Control
Mechanism
Protocol
Bandwidth
On Demand
RTP
150 kbps
Redundancy
Availability
Direction
Y
H
BI
Figure 72.
Y
…
Actor 1
Call Manager
Latency
Jitter
0.5 Sec
N/A
Acknowledge
Trans. Rate
Y
…
N
…
Actor 2
Field Handset
Packet Loss
Security
1%
L
Time Synch.
BER / PER
N
…
Field Worforce Voice over IP Tabular & Graphic Information
152
4.6.2 Workforce Video
Real-time collaboration through PC’s, laptops, tablets, or smart mobile devices is a unified communications
application that is designed to make end users more productive by enabling them to communicate and
collaborate easily with others in remote locations using a range of different communication options, including
instant messaging (IM), voice, desktop sharing and video.
Title
Field Workforce Video
Description
On-demand video connection between two locations over IP network
Actors
Video Camera; Video Viewer; Video Manager
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
Event
Video Stream
Mechanism
Protocol
Bandwidth
Stream
Video RTP
384 kbps
Redundancy
Availability
Direction
Y
H
BI
X
X
X
X
X
Actor 1
Field Camera
Latency
Jitter
150ms
30ms
Acknowledge
Trans. Rate
Y
…
X
X
Actor 2
Desktop Video System
Packet Loss
Security
1%
L
Time Synch.
BER / PER
N
…
153
Event
Video Control
Mechanism
Protocol
Bandwidth
Polled
H.232
150 kbps
Redundancy
Availability
Direction
Y
H
BI
Figure 73.
Actor 1
Video Manager
Latency
Jitter
1 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
Desktop Video System
Packet Loss
Security
1%
L
Time Synch.
BER / PER
N
…
Field Workforce Video Over IP Tabular & Graphic Information
154
4.6.3 Radio over IP (RoIP)
This use case describes the transfer of land mobile radio (LMR) audio signals over the IP network. A Land Mobile
Radio (LMR) system is a collection of portable and stationary radio units designed to communicate with each
other over predefined frequencies. They are widely deployed by utilities because of the need to have instant
communication between geographically dispersed and mobile personnel. In typical LMR systems, a central
dispatch console or base station controls communications to the disparate handheld or mobile units in the field.
Within an organization, the radio systems tend to be homogenous, with most elements typically purchased from
the same manufacturer. Although the electromagnetic spectrum is rather vendor agnostic, signaling
mechanisms and other control aspects of individual radio systems can be quite proprietary. And if the utility
needs to consolidate operations that were previously using different LMR systems, issues with interoperability
could require workarounds to bridge the existing systems or ultimately require the purchase of all new
equipment.
With the LMR over IP service, standards-based VoIP technology voice gateways are used in combination with
additional LMR specific features to address interoperability, extending command and control, and other issues.
Base stations, repeaters, and dispatch consoles generally possess a wired interface that can be used to monitor
audio received from their air interface, and as input for audio to be transmitted on their air interface. Although
this wired interface may contain other control capabilities as well, as long as it has some sort of speaker output
and microphone input, it can be connected to a voice port on a router.
The audio received on the voice port is encoded with a standard audio codec, such as G.711 or G.729. Those
audio samples are packaged in standards-based Real-Time Transport Protocol (RTP) packets suitable for
transport on an IP network. At this point, the communication element is abstracted from the distinctive
characteristics of each radio system, thus providing a solution for the interoperability problem. Now, these
audio packets can be sent across the network to other LMR gateways with different brands of radio systems
either individually (unicast) or as a group (multicast).
The recipient of the audio packets need not be another LMR gateway. It can be any device capable of receiving
and decoding the RTP stream, such as an IP telephone or PC with appropriate software. The IP network and IPenabled devices can be used to allow users to monitor or transmit on a particular radio channel from a desk
without issuing another radio. This can be done locally, nationally, or internationally, assuming the IP network
has been properly designed.
Title
Field Workforce – Land Mobile Radio (LMR) over IP (RoIP)
Description
Connecting disparate land mobile radio systems over the IP network and patching radio
systems over terrain and geographies with no radio coverage.
Actors
Radio Repeater Router; RoIP Server; Radio Dispatch Console
155
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
X
X
X
Event
RoIP Stream (Voice + Control)
Mechanism
Protocol
Bandwidth
Stream
RTP
200 kbps
Redundancy
Availability
Direction
Y
H
BI
Actor 1
Radio Repeater / Router
Latency
Jitter
150ms
30ms
Acknowledge
Trans. Rate
Y
…
Actor 2
RoIP Server
Packet Loss
Security
1%
L
Time Synch.
BER / PER
N
…
Event
RoIP Stream (Voice + Control)
Mechanism
Protocol
Bandwidth
Stream
RTP
200 kbps
Redundancy
Availability
Direction
Y
H
BI
Actor 1
RoIP Server
Latency
Jitter
150ms
30ms
Acknowledge
Trans. Rate
Y
…
Actor 2
Radio Dispatch Console
Packet Loss
Security
1%
L
Time Synch.
BER / PER
N
…
156
Figure 74.
Radio over IP (RoIP) Tabular & Graphic Information
157
4.6.4 Substation Worker Access to Corporate Applications
This use case focuses on the network connectivity for substation technicians and engineers within the
substation. Currently, technicians / engineers perform work using information that is either from paper
documents or from “ruggedized” laptop computers brought to the field. These computers contain large
amounts of useful information (e.g. asset information, maps, work orders, and manuals).
However, using a standalone computer has the following limitations:
•
Information is not always current because it is not updated throughout the day in real-time. The field
worker must connect with the network at the office to obtain updated information.
•
Back office systems do not obtain updated information from the field (for example, status of current
work orders, missing or incorrect asset information, inspection results, and work orders generated in the
field) until the work is completed and the field worker returns to the office to turn in completed
paperwork or connects their laptops to the network.
This lack of automated information exchange between field workers and the back-office leads to operating
inefficiencies (errors, delays and reduced work capacity), as well as outdated asset information. Providing secure
IP network access to the devices in the substation and to the office enables:
•
•
•
•
•
Technician / engineer collects information in the field and perform asset inspection, as well as planned
and unplanned work assignments. The technician/power engineer may be using a hardened PC or a
“ruggedized” laptop.
Technician / engineer in substation accesses real-time information from the intranet (utility corporate
network) including the utility’s asset management system.
Technician / engineer in substation can access information from the intranet (utility corporate network)
regarding assets: data sheets, manuals and guides, and troubleshooting instructions based on the initial
analysis of the situation.
Technician / engineer in substation updates information in the asset management system based on the
current status of the equipment. This allows engineers/technicians to add the required details as they
work on the case, providing accurate and complete information.
Technician/engineer in substation accesses information from the Internet for third-party equipment
information.
This use case should account for both wired and wireless access within the substation. The user needs to be
authenticated and depending on the corporate application that is accesses the data may need to be encrypted.
Network management is important for assigning IP addresses and managing ruggedized PCs, access points, and
laptops. Within the Operational network these collaboration applications should be segregated from the SCADA
network traffic all the way to the enterprise head-end and the corporate network should be separated from the
SCADA network via centralized back-to-back firewalls.
158
Title
Worker access to corporate applications from a substation.
Description
The technician/engineer can access data within the substation as well as securely access
the application on the corporate network.
Actors
Field Worker PC/Laptop; Corporate Application
Transmission
Distribution
Energy Supply
Demand Side
X
X
X
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
Event
Corporate Traffic
Mechanism
Ad hoc
Redundancy
N
Protocol
TCP/UDP
Availability
M
Actor 1
PC
Bandwidth
64 kbps
Direction
BI
X
X
X
X
X
Latency
1 Sec
Acknowledge
Y
Jitter
N/A
Trans. Rate
…
X
X
Actor 2
Corporate Application
(HTTPs, Citrix, etc.)
Packet Loss
Security
5%
M
Time Synch.
BER / PER
Y
…
159
Figure 75.
Substation Worker Access to Corporate Applications Tabular & Graphic Information
160
4.7 Physical Safety & Security
This section includes use cases related to physical security of the infrastructure and safety of the workforce and
the public. Physical security use cases are motivated by the necessity to protect the grid infrastructure from
internal and external physical threats and by the need for compliance with regulatory requirements such as
North American Electric Reliability Corporation’s Critical Infrastructure Protection (NERC/CIP) standard CIP006.
For example, version 5 of CIP006 (the latest version at the time of this writing) calls for mandatory security
measures to protect those grid assets that if compromised will impact the reliability and stability of the grid.
Specific CIP006 mandates include:
•
•
•
Establishment of a physical security program
Physical access controls for high- and medium-impact grid assets
Monitoring physical access to important grid facilities such as control centers, high voltage substations,
and any other with significant impact on grid reliability
• Utilizing 2 or more physical access controls to collectively restrict access
• Logging physical access to grid facilities
• Retaining access logs
• Maintenance and testing of physical security measures to provide compliance with the regulatory
requirements
For physical safety and security we study the following use cases:
1.
2.
3.
4.
5.
Electronic access control
Video surveillance
Fire alarms
Substation environmental monitoring
A composite use case for worker safety in the case of a fault
4.7.1 Electronic Access Control
This service provides electronic monitoring and control for physical access to remote areas, building and
locations either by using badges, card readers, and keypads or remotely from a Control Center over the
communications network.
Title
Electronic Access Control
Description
Using electronic sensors and actuators, e.g. door sensors and badge readers and remote
control from a security control center to authorize access to a restricted area.
Actors
Physical Access Gateway; Security Control Center
161
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
X
X
X
X
X
X
X
X
Event
Door Position
Mechanism
Protocol
Bandwidth
Event
TCP based
5 kbps
Redundancy
Availability
Direction
N
M
BI
Actor 1
Physical Access Gateway
Latency
Jitter
1 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
Security Control Center
Packet Loss
Security
5%
M
Time Synch.
BER / PER
Y
…
Event
Entry Request
Mechanism
Protocol
Bandwidth
Event
TCP based
5 kbps
Redundancy
Availability
Direction
N
M
BI
Actor 1
Latency
Jitter
1 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
Event
Entry Request
Mechanism
Protocol
Bandwidth
Event
TCP based
5 kbps
Redundancy
Availability
Direction
N
M
BI
Actor 1
Latency
Jitter
1 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
Packet Loss
Security
1%
H
Time Synch.
BER / PER
Y
…
162
Figure 76.
Electronic Access Control Tabular & Graphic Information
163
4.7.2 Video Monitoring & Surveillance
Video surveillance provides the control center operators the capability of viewing live events as well as the
retrieval and analysis of past security events by accessing the Digital Video Recorders (DVR).
The benefits of live video surveillance are:
•
•
•
•
Detection of an actual or attempted security breach at a substation
Providing intelligence for security monitoring of critical assets
Cost savings in responding to alarms
Cost savings associated with security monitoring and random patrol fees.
Furthermore,
•
•
Infrared cameras could be used for thermal monitoring of transformers, and
Pan-Tilt-Zoom (PTZ) cameras can be used to visually monitor breakers.
The same pattern in deployment of sensors for remote monitoring applies to other monitoring such as:
•
•
•
Motion detection
Gas leak detection
Earthquake detection
Typically there is a policy-based minimum retention period for video files, e.g. 3-30 days. File storage can be
done locally at the premise. After the initial retention period files are moved to off site storage.
Title
Video Monitoring & Surveillance
Description
Video surveillance for remote monitoring of restricted premises; cost savings in truck rolls
and personnel time, and asset condition monitoring.
Actors
Video cameras and Digital Video Recorder (DVR); Security Control Center
Transmission
Distribution
Energy Supply
Demand Side
X
X
X
Other (Specify)
Places in the Grid
X
X
X
X
X
X
164
Customer Premise
Control Center
Data Center
X
X
X
X
Event
Video Control
Mechanism
Protocol
Bandwidth
On demand
RTP
256 kbps
Redundancy
Availability
Direction
N
M
BI
Event
Video Stream
Mechanism
Protocol
Bandwidth
Stream
RTP
256 kbps
Redundancy
Availability
Direction
N
M
BI
X
X
X
Actor 1
Latency
Jitter
1 Sec
N/A
Acknowledge
Trans. Rate
Y
…
Actor 2
DVR
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
Actor 1
Camera
Actor 2
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
Latency
1 Sec
Acknowledge
Y
Jitter
N/A
Trans. Rate
…
165
Figure 77.
Video Monitoring & Surveillance Tabular & Graphic Information
166
4.7.3 Remote Fire Alarms Monitoring
Remote monitoring of fire alarms enables the Control Center personnel to quickly locate and react to possible
fires in remote locations using smoke detectors and fire sensors.
The existing fire alarm panels ('FAP') within H-Q’s footprint have no IP connectivity, and alerts are
communicated via SCADA to the electrical control room. However, in circumstances where a FAP is experiencing
a fault, there is currently no way to diagnose the issue remotely. Instead, the root cause of the issue can only be
identified when the FAP maintainer reaches the site and is able to connect to the panel via a laptop. While this
can lead to significant delay, especially in rural sites, this also leads to multiple site visits, as the maintainer may
not arrive at site with the parts needed to resolve the issue. Fire Services Officers and safety staff also have no
ability to remotely audit FAPs to ensure parts of the system have not been left isolated.
Title
Remote Fire Alarms Monitoring
Description
Remote monitoring of fire alarms for early detection and timely response without having
to dispatch personnel to the site.
Actors
Actors
Transmission
Distribution
Energy Supply
Demand Side
Other (Specify)
X
Places in the Grid
Customer Premise
Control Center
Data Center
Event
Alarm
X
X
X
X
X
X
X
X
X
Actor 1
Alarm System
X
X
Actor 2
167
Mechanism
Eventtriggered
Redundancy
N
Protocol
TCP based
Bandwidth
5 kbps
Latency
1 Sec
Jitter
N/A
Packet Loss
1%
Security
M
Availability
M
Direction
UNI
Acknowledge
Y
Trans. Rate
…
Time Synch.
Y
BER / PER
…
Figure 78.
Remote Fire Alarms Monitoring Tabular & Graphic Information
168
4.8 Other Use Cases
This section is added for completeness and includes use cases that don’t fit within the other categories.
4.8.1 Generic Network Management Use Case
This use case describes the monitoring of the communication network, including device status, device
configuration, latency test, software management, and threshold testing by the Network Management System.
The benefits include improved visibility of communication network faults; reduced communications minutes lost
due to undetected faults, and proactive maintenance of communications assets.
Title
Communication System Monitoring
Description
Monitoring of the communication system
Actors
Network Management System; Communication Device (Router/Switch)
Transmission
Distribution
Energy Supply
Demand Side
X
X
Other (Specify)
Places in the Grid
Customer Premise
Control Center
Data Center
X
X
X
Event
Configuration
Mechanism
Protocol
Bandwidth
Ad hoc
SSH
64 kbps
X
X
X
X
X
Actor 1
NMS
Latency
1sec
Jitter
N/A
Actor 2
Device
Packet Loss
Security
1%
M
169
Redundancy
N
Availability
H
Direction
BI
Event
Threshold Event
Mechanism
Protocol
Bandwidth
Ad hoc
SNMPv3
9.6 kbps
Redundancy
Availability
Direction
N
H
BI
Event
Device Status
Mechanism
Protocol
Bandwidth
Ad hoc
SNMPv3
9.6 kbps
Redundancy
Availability
Direction
N
H
BI
Event
Software Management
Mechanism
Protocol
Bandwidth
Ad hoc
SCP
64 kbps
Redundancy
Availability
Direction
N
H
BI
Event
Latency Test
Mechanism
Protocol
Bandwidth
Ad hoc
ICMP
9.6 kbps
Redundancy
Availability
Direction
N
H
BI
Acknowledge
N
Trans. Rate
Actor 1
Device
Latency
1sec
Acknowledge
N
Jitter
N/A
Trans. Rate
Actor 1
Device
Latency
1sec
Acknowledge
N
Jitter
N/A
Trans. Rate
Actor 1
NMS
Latency
1sec
Acknowledge
N
Jitter
N/A
Trans. Rate
Actor 1
Device
Latency
1sec
Acknowledge
N
Jitter
N/A
Trans. Rate
Time Synch.
Y
BER / PER
…
Actor 2
NMS
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
Actor 2
NMS
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
Actor 2
Device
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
Actor 2
NMS
Packet Loss
Security
1%
M
Time Synch.
BER / PER
Y
…
170
Figure 79.
Generic Network Management Tabular & Graphic Information
171
4.8.2 Precision Time Protocol (PTP)
Hydro Québec do not use GPS clocks in generation substations. One of the main reasons is that some of the
generation plants are 30 to 50 meters deep under ground and the GPS signal can be weak and unreliable.
Instead, atomic clocks are used. There are forty Rubidium clocks and four Cesium clocks. Clocks are synchronized
amongst each other and the synchronized time signal is sent through SONET. Rubidium clocks provide clock to
the SONET network and 1ms timestamps for IRIG-B. Eventually, H-Q plans to transition to the Precision Time
Protocol (IEEE 1588), distributing the synchronization signal over the IP/MPLS network.
The Precision Time Protocol (PTP) is defined in IEEE standard 1588. PTP is applicable to distributed systems
consisting of one or more nodes, communicating over a network. Nodes are modeled as containing a real-time
clock that may be used by applications within the node for various purposes such as generating time-stamps for
data or ordering events managed by the node. The protocol provides a mechanism for synchronizing the clocks
of participating nodes to a high degree of accuracy and precision.
PTP operates based on the following assumptions 19:
•
•
•
•
It is assumed that the network eliminates cyclic forwarding of PTP messages within each communication
path (e.g., by using a spanning tree protocol). PTP eliminates cyclic forwarding of PTP messages between
communication paths.
PTP is tolerant of an occasional missed message, duplicated message, or message that arrived out of
order. However, PTP assumes that such impairments are relatively rare.
PTP was designed assuming a multicast communication model. PTP also supports a unicast
communication model as long as the behavior of the protocol is preserved.
Like all message-based time transfer protocols, PTP time accuracy is degraded by asymmetry in the
paths taken by event messages. Asymmetry is not detectable by PTP, however, if known, PTP corrects
for asymmetry.
A time-stamp event is generated at the time of transmission and reception of any event message. The timestamp event occurs when the message’s timestamp point crosses the boundary between the node and the
network.
19
IEEE 1588, Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control
Systems
172
5. Network Requirements Heat Map & Dependency
This section presents two views of the communication requirements:
1. A matrix showing color-coded representation of use case telecom requirements. The color convention
will follow the same rules that we defined for the UML diagrams. The matrix can be seen as a heat map
showing where there are stringent requirements for telecom services. We have segmented the matrix
by domain.
2. A dependency matrix indicating all places in the network that data packets for each use case traverse
viewed from the network’s perspective, one can see all use cases that should be taken into account for
the architecture and design of the network.
173
5.1 Network Requirements Heat Map – By Use Case
This section represents the main network requirements, i.e., latency, bandwidth, security, and availability, for all
use cases. The representation is in matrix format where the cells are color-coded to indicate use cases that
impose more stringent requirements on the communication network. Red indicates the most stringent
requirement; yellow and green indicate relatively less stringent requirements, respectively. The matrix offers a
visual heat map of the network requirements.
174
Figure 80.
Network Requirements Heat Map – by Use Case
175
5.2 Network Dependency Matrix – By Use Case
The following matrix shows dependencies on different places in the network, for each use case.
176
Figure 81.
Network Dependency Matrix – by Use Case
177
6. Communication Trends & Best Practices
These general communication requirements are over and above the specific requirements of the use cases that
have been addressed so far. These include both current and future communication related requirements that
should be factored into the network architecture and design.
6.1 General communication Requirements
IP Connectivity everywhere
Establish IP connectivity everywhere. Features of the IP network must meet the needs of all services in
terms of:
•
•
•
•
Security – cyber security and data integrity
Quality – bandwidth and latency
Reliability – redundancy
Availability – reduced failure rates of devices, for example by minimizing the number of devices
and by ruggedizing them as necessary
Monitoring services everywhere and from different remote centers
Due to the big large and diversity of devices and manufacturers, it is necessary to be able to remotely
connect to all of them in order to retrieve the product configuration and event log. Multisite capability is
important in this case and is one of the reasons for recommending using IP protocol everywhere.
Move services to a virtual data center
Due to the legacy application some of the communication management application do not reside in the
virtual data center. Some services are currently provided by servers outside the Virtual data centers and
are therefore out of the security framework and maintenance plans. All application should be move and
any new application must be VDC enabled.
Unify access to applications / information from the corporate network
This requirement is also related to moving all applications to the virtual Data center and ensuring that
applications can be access from anywhere in the network (corporate or operational network
infrastructures) with the appropriate level of security.
Unify services
Typically, it is common practice to install new infrastructure/solution when a new service is required
instead of integrating the new one in the already installed infrastructure. This is especially evident while
deploying remote services. This has lead to building a siloed network, which increases the complexity, is
178
less efficient than a converged network and increases management costs. The introduction of new
services as well as the integration of existing ones is significantly easier using a unified or converged
infrastructure and thus enables a unified service approach.
Unified Communications Solutions
A unified voice/video communication solution should provide connectivity to all remote sites,
substations, and generation plants. Ensuring connectivity to many of the remote sites where vehicle
access during the winter is difficult. Building a robust infrastructure for these locations is critical.
Unified Fiber Technology – SONET replacement
H-Q enhanced over time their SONET network to support DWDM and SONET-NG technologies. SONET is
reaching its end of life and can’t compete with new technologies like IP over dense wavelength-division
multiplexing (IPoDWDM) or IP/MPLS over dense wavelength-division multiplexing
(IPoMPLSoDWDM), which provide packet switching instead of circuits-based networks.
Standardize grid communication protocol to opened standard.
Multiple protocols exist for substation automation, many of them being proprietary protocols with
custom communication links. Such environment makes cross-vendor interoperability a challenge and
increase additional OPEX and CAPEX to support multiple gateways and protocol translators.
Standardizing the communication model is key to benefit from the full potential of an IP end-2-end
network. IEC 61850 is the new standard communication model developed since the last 20 years and
supported today by most of the vendors. The goals of the working group defining this protocol are
•
•
•
•
•
A single protocol for complete substation considering modeling of different data required for
substation.
Definition of basic services required for transferring data so that the entire mapping to
communication protocol can be made future proof.
Promotion of high interoperability between systems from different vendors.
A common method/format for storing complete data.
Define complete testing required for the equipment that conforms to the standard.
This will enable the migration to an end-to-end IP infrastructure from sensor to control center.
Reliable Communications for Distribution Substations
The Fault Location Isolation & Service Restoration (FLISR) algorithm and its implementation demands
reliable communications, at least with urban, and semi-urban where there is the possibility of feed from
two different primary substations. Rest of the substations requires reliable communications to the
extent that they can reliably receive and accept commands from the control center.
179
IEEE 1588 time synchronization Client / Server Capabilities
Use IEEE 1588 for time synchronization to enable a single clock source and remove the requirements to
have GPS clocks in each location. This will enable an accurate centralized timing source to be
implemented. The benefits are reduced costs and greater control over and lowering of some of the
security threats that have been seen from using remote clock sources.
The communication devices must be compatible with IEEE 1588 in order to be synchronized as a client.
In addition, these devices should be able to behave as a server in order to provide the clock to the 1588
end-points. Accurate clock synchronization is required for the current infrastructure and long-term
applications. The communication devices software and hardware must be IEEE 1588 complainant.
Currently, Time is distributed using legacy, propriety protocols and using redundant GPS in some
locations.
H-Q generation plants are located deeply underground blocking any GPS signal. To provide time
synchronization service, H-Q deployed the following architecture:
- 4 Cesium atomic clocks, each one covering one area of the province and backing up each other.
- Cesium clocks provide clock to 40 Rubidium clocks
- Rubidium clocks provide clock to the SONET network and 1ms timestamps for IRB
H-Q will use IEEE 1588 to provide time synchronization once all grid services will have been migrated to
their IP/MPLS backbone.
Integration of Multicast Design
Integrate multicast into the high level design for point to multipoint traffic, such as PMU or video.
Multicast technologies will become a requirement as we move to using real time data across multiple
locations or when streaming data from devices such as PMUs. The ability for the communications
network to effectively manage these data streams and to ensure that they get delivered to the correct
destination in a timely fashion will become critical to the operation of the grid. Multicast is a key
technology so the control center can reach thousands if not millions of devices using a single request. A
basic use case is sensor firmware upgrade.
New technology like Label Switch Multicast or LSM should be considered to transport multicast traffic
over a MPLS backbone. It will support MVPN services without the need to enable PIM in the core and
will benefit from feature like FRR.
QoS Requirements Mapping
Converging multiple types of applications into a shared multi-services infrastructure requires a strong
QoS policy. Each application must be characterized from delay / jitter / latency / bandwidth perspective
so class of services can be defined as well as the mapping policy of the application to those classes.
180
Enable Future Network Expansion
The network design should allow for expansion of the network as future applications are added. It is
hard to predict all of the services and applications that the network will be required to support over the
next 15 years so it is vital that the designers build a flexible network architecture capable of expansion
for both number of locations, number of devices, and the communications requirements such as
bandwidth and latency.
Substation Network Resilience
Define criteria for categorizing substations based on their criticality and design the substation network
resilience requirements based on the criticality of the sue cases that should be supported in each
substation category.
Fast Convergence Design
Different applications have different requirements in terms of connectivity availability. Most critical
application like Teleprotection needs the lowest convergence time. Fast convergence is not a feature
you just need to enable. It’s an architecture that requires deep understanding of all the different routing
protocols and links type used to provide end-to-end connectivity. The technical decision will be based on
the failure scenarios that should be supported and the expected convergence time.
Scalable Headend Design
Extend the Smart Grid network design into the headend and data center. The communications network
design must extend into the headed or control center architecture to ensure a full end-to-end design is
achieved. This is critical when the application has low end-2-end latency budget and need the remote
device needs to talk to the Control Center.
Define Service Level Agreements (SLA) and Enable SLA Monitoring
Within the high level architecture, build the network SLAs and define the methods of measuring them
within the network management system. It is critical that the SLAs for the network are understood. This
allows the design to not only ensure that they can be meet but also that we can use synthetic traffic
monitoring controls by a network management system to monitor that they are being achieved.
Integration of 3G/4G Technologies
Build 3G/4G technology into the High Level Design. Review the use of WiMax and radio technologies
within the architectural design. These may be used to connect to remote locations.
Ethernet Connectivity for Station Bus Architecture
181
Build Ethernet architecture for Station bus connection and remove serial connections. This will simplify
connectivity within the substation. This will also remove the requirement for multiple serial connections
and removes the slow serial bus architectures that are used. This also ensures increased flexibility for
integration into other systems in the substation such as physical security.
Station bus resilience through standards such as PRP and HSR should be examined within the high-level
design. IEC61850 Ethernet Station Bus will be required but this is a long-term goal.
Ethernet Connectivity for Process Bus Architecture
Enable Ethernet connectivity from RTUs to sensors and IEDs. This will simplify connectivity within the
substation. This removes the requirement for multiple serial connections and removes the slow serial
bus architectures that are used. This also ensures increased flexibility and increased speed with the use
of multicast messaging between multiple devices. Ethernet Technologies will be deployed for the
process bus using fiber optic technology.
Protection and teleprotection on IP
Integrate teleprotection into the IP network. Converging onto a single IP infrastructure offers
considerable cost savings. However, the challenge is the integration of new components and systems
such as remedial actions schemes and system integrity protection schemes that require integration of
many systems that has not been integrated before. Running over an IP infrastructure using standard
protocols such as IEC 61850 simplifies the deployment and management of the overall system. It also
removes the tie between the physical communications infrastructure to the system logic. This will be
increasingly important as we move to a distributed intelligence approach for grid monitoring and
control.
6.2 Migration to Packet-Switched Network
Throughout the world, utilities are increasingly planning for a future based on smart grid applications requiring
advanced telecommunications systems. Many of these applications utilize packet connectivity for
communicating information and control signals across the utility’s Wide Area Network (WAN), made possible by
technologies such as multiprotocol label switching (MPLS).
The data that traverses the utility WAN includes:
•
•
•
•
•
Grid monitoring, control, and protection data
Non-control grid data (e.g. asset data for condition-based monitoring)
Physical safety and security data (e.g. voice and video)
Remote worker access to corporate applications (voice, maps, schematics, etc.)
Field area network backhaul for smart metering, and distribution grid management
182
•
Enterprise traffic (email, collaboration tools, business applications)
WANs support this wide variety of traffic to and from substations, the distribution grid, generators, between
control centers, and between work locations and data centers. To maintain this rapidly expanding set of
applications, many utilities are taking steps to evolve present time-division multiplexing (TDM)–based and frame
relay infrastructures to packet systems. Packet-based networks are designed to provide greater functionality
and higher levels of service for applications, while continuing to deliver reliability and deterministic (real-time)
traffic support.
6.3 MPLS technology
MPLS is a proven WAN technology for network operators who need to support diverse legacy systems as well as
modernize for next-generation applications. Enabling transparent integration of traditional and smart grid
capabilities, MPLS facilitates transport of most forms of traffic, from traditional serial-based technologies such as
SCADA remote terminal units (RTUs) to today’s IEC 61850 packet-based intelligent electronic devices (IEDs).
In the utility industry, MPLS is by far the most commonly selected WAN technology for smart grid
implementations because of its:
-
Maturity and proven capabilities across large-scale industrial and enterprise networks
Ability to support both traditional applications and next-generation requirements
Ability to virtualize the WAN into independent sub-networks
Centralized management of physical infrastructure and virtualized sub-networks
Ability to enhance and become an integral part of the security framework across the WAN
Modularity for scalability and flexibility, as well the ability to protect the overall system from domain
failures
By supporting multiple applications on a converged network, our smart grid solutions provide a framework for
integrating new technologies and utility-specific applications. The modular approach enables implementation of
projects over time, allowing utilities to plan their investments and flexibly adapt to changing business
circumstances.
MPLS offers a number of features that make it especially suitable for multiservice, high-security industrial
environments. By protecting existing system investments while enabling the transition to the modernized grid,
MPLS provides a secure, flexible, and high-performance foundation for utility systems.
6.3.1 Network Virtualization
In high-security, geographically distributed environments, large flat networks are usually neither practical nor
reliable. They create a single fault domain and do not permit the kind of modularity needed for dependable,
secure system management over large distances. As a result, grid applications suffer due to congestion, poor
performance, reliability and security issues, and administrative complexity.
183
Within an MPLS infrastructure, however, applications can be logically separated and secured to support specific
business functions while remaining on a single physical network. These software-defined segments operate
securely and independently from each other, minimizing the fault domain. Within each virtual network, MPLS
supports deterministic traffic, prioritization of traffic flows, and flexible allocation of bandwidth to enable
optimal, consistent performance.
Traffic is secured by policy-based trust boundaries, with each operations system accessible only to an approved
group of users. Administrators can also create new secure segments as needed based on scalable Layer 2 and
Layer 3 virtual private networking (VPN) domains, including point-to-multipoint capabilities.
6.3.2 Support for Existing Networks & Interoperability
One of the most valued features of MPLS is that it allows utilities to perpetuate the use of existing TDM circuits,
ATM, frame relay, and other traditional communication networks on the same WAN backbone with nextgeneration packet-based systems. This is achieved either by running these legacy systems over an MPLS network
using techniques such as circuit emulation with Pseudo Wire Emulation Edge-to-Edge (PWE3) and/or by
overlaying MPLS onto an existing TDM-based network infrastructure.
Enhanced by MPLS Traffic Engineering (TE), networks can integrate virtually all forms of traffic without having to
disruptively replace still-functioning older systems. This helps to unify the network management environment,
making it significantly more cost-effective to administer. By running new applications alongside older systems on
the same network, utilities can protect their current investment while transitioning to a modernized grid.
6.3.3 Security in MPLS
MPLS virtualization greatly enhances network security. By creating logical separation of routing and data flows,
utilities are able to safeguard specific information for each segmented domain. In doing so, it assures greater
security for a variety of applications.
For example, many utilities are considering how they might improve mobile worker connectivity through Wi-Fi,
especially in remote areas where public cellular signals are not always reliable. However, a security threat exists
in the potential of Wi-Fi traffic mingling with operations traffic. This issue can be resolved by utilizing robust WiFi security mechanisms as well as MPLS to create a totally segregated virtual network for all Wi-Fi traffic. Along
the way, MPLS mechanisms such as rate shaping, priority handling, and traffic engineering mechanisms help to
guard against denial of service and other malicious attacks.
6.3.4 Utility-Grade Performance
184
Utilities have traditionally accepted SONET/SDH for its ability to deliver high-performance connectivity. By
contrast, packet solutions have sometimes been characterized as “best-effort networks,” especially in situations
where they are based on T1 or low-bandwidth connectivity. But this not true for well-designed packet networks,
especially not for high-speed MPLS networks designed with Quality of Service, traffic engineering, fault
detection, and Fast Reroute (FRR) features.
In contrast to other forms of packet solutions, which function on a hop-by-hop basis, MPLS TE steers traffic
across predetermined routes in case of a network failure. Features such as bidirectional forwarding detection
(BFD) and FRR can detect failures and reroute traffic on a par with SONET, helping to assure reliability and speed
of network traffic.
6.3.5 IP/MPLS and MPLS-TP for the WAN
The flexibility of MPLS allows utilities to transport data using a variety of static and dynamic techniques,
including fully switched or circuit-oriented connections, or quite commonly for a hybrid configuration supporting
both. The appropriate deployment approach is determined by assessing such issues as the nature of the
environment, the coverage area, the level of SONET/SDH functions, dynamic signaling, control plane policing as
well as operational business requirements.
6.3.6 The Cost Efficiencies of MPLS
A key benefit of the virtualized MPLS infrastructure is that it can help utilities to cut costs in a number of areas. It
assists in capital management by eliminating duplicate equipment and minimizing spares and inventory. As well,
asset management is improved with a less complex infrastructure and management capabilities. Utilities also
avoid early depreciation with planned, timely investments in strategic equipment.
Utilities have seen reduced operations and administrative costs of overlay networks, tools, and management
systems by as much as 50 percent, according to industry studies. Based on the single network platform,
organizations eliminate duplicate vendor support services and contracts, and optimize service provider contracts
(and circuits). They minimize downtime with less maintenance and out of service conditions and reduce the
potential for regulatory fines.
185
6.4 IP Address Planning & Management
6.4.1 Overview
IP addressing plan development and IP Addresses management are two fundamental activities for any IP based
communication network. A good IP addressing plan implemented in a well-designed network provides
scalability, predictability, and flexibility. IP address management is the process of allocating, recycling and
documenting IP addresses and subnets in a network.
IP addressing standards define subnet size, subnet assignment, network device assignments and dynamic
address assignments within a subnet range. Recommended IP address management standards reduce the
opportunity for overlapping or duplicate subnets, non-summarization in the network, duplicate IP address
device assignments, wasted IP address space, and unnecessary complexity.
The first step to successful IP address management is to understand the IP address blocks used in the network.
Once the address blocks have been defined, allocate them to areas of the network in a way that promotes
summarization. In many cases, these will have to be further subdivided based on the number and size of subnets
within the defined range. Standard subnet sizes for standard applications should be defined, such as building
subnet sizes, WAN link subnet sizes, loopback subnet size, or WAN site subnet size. Then these subnets can be
allocated for new applications out of a subnet block within a larger summary block.
The main focuses of a new IP addressing plan should be to:
•
•
•
•
•
Provide additional control over network resource allocation
Accommodate new applications requirements
Support continuing growth
Ease of route and network management
Increase application, network, and system security
Utilities also have unique challenges from an IP addressing perspective:
•
•
•
•
Need to connect many (up to millions) field devices in a plug and play way (Zero touch deployment)
Must take into account complete isolation between the station and the process buses within the
substation
In-line with NERC-CIP requirements
Support new IP based applications like physical security, mobile workforce enablement, Precise Time
distribution or IP telephony.
It is clear IPv4 can’t support all these requirements and Utilities must consider IPv6 when designing smart grid
networks.
186
6.4.2 IPv6 Considerations
IPv6 integration is a critical factor to any smart grid network that is being designed today. IPv6 for smart
metering is a must, as potentially millions of devices must be addressed. We also need to consider IPv6 within
the main communications network, and while there are few substation devices that are IPv6 enabled today, the
network is being designed to last 15 years and we do see a requirement for these devices to be deployed within
that time frame.
It’s recommended to secure a public IPv6 prefix sooner than later even if it’s not implemented right away. Utility
of the size of H-Q should request a /36 prefix.
6.4.3 Critical Issues in IP Address Assignment
Utilities typically have a repeatable plug and play approach when deploying RTUs or IEDs equipment in
substation. These devices are all "pre-conﬁgured exactly the same way which greatly simpliﬁes the maintenance
and support model. When those same devices were IP-enabled, utilities continued to use the same practice
resulting in multiple devices having the same IP address. This scheme relies heavily Network Address Translation
(NAT) mechanism to allow communication with the control centers. This model works fine when with the
control center(s). This model was adequate when the communication was mainly unidirectional and following a
hub & spoke topology.
However, this design is not recommended anymore, as it will not support new smart grid application
requirements:
•
Bidirectional Communication – If the application, hosted outside of the substation, needs to initiate a
connection, complex Port Address Translation (PAT) rules must be deployed. Furthermore, it imposes a
strong requirement at the application level, as each of them must use a unique application port.
•
Multi-point to Multi-point Communication – A substation may have multiple exit points to reach many
different sites across the utility network. If NAT is required, it must be enabled at every single edge
interface which makes the network much more difficult to manage and operate. Each time a new
application is deployed, it may require update of the NAT rules for every NAT gateway deployed across
the network.
•
Bottleneck – Most of the time NAT implementation is not stateless. It means the traffic in both
directions must go through the same NAT gateway leading to a Single Point of Failure and potential
performance bottleneck issues.
•
Application Support – Some applications are not NAT-compatible as they carry some IP information in
their payload. To support such applications, an Application Level Gateway (ALG) must be embedded into
the NAT gateway itself to apply the required translation rules to the payload of the packet.
187
7. Security Trends & Best Practices
7.1 Current Practices & Their Limitations
Grid monitoring and control devices are already targets for cyber attacks and legacy communication protocols
have many intrinsic network related vulnerabilities.
DNP3, Modbus, PROFIBUS/PROFINET, and other protocols are designed around a common paradigm of request
and respond. Each protocol is designed for a “master” device such as an HMI system to send commands to
subordinate “slave” devices to retrieve data (reading inputs) or control (writing to outputs). Because many of
these protocols lack authentication, encryption, or other basic security measures, they are prone to networkbased attacks, allowing a malicious actor or attacker to utilize the “request and respond” system as a
mechanism for “command and control” like functionality.
Specific security concerns common to most industrial control, including utility communication protocols include
the following:
•
Network or transport errors (e.g. malformed packets or excessive latency) can cause protocol failure.
•
Protocol commands may be available that are capable of forcing slave devices into inoperable states,
including powering-off devices, forcing them into a “listen only” state, disabling alarming.
•
Protocol commands may be available that are capable of restarting communications and otherwise
interrupting processes.
•
Protocol commands may be available that are capable of clearing, erasing, or resetting diagnostic
information such as counters and diagnostic registers.
•
Protocol commands may be available that are capable of requesting sensitive information about the
controllers, their configurations, or other need-to-know information.
•
Most protocols are application layer protocols transported over TCP; therefore it is easy to transport
commands over non-standard ports or inject commands into authorized traffic flows.
•
Protocol commands may be available that are capable of broadcasting messages to many devices at
once (i.e. a potential DoS).
•
Protocol commands may be available to query the device network to obtain defined points and their
values (i.e. a configuration scan).
188
•
•
Protocol commands may be available that will list all available function codes (i.e. a function scan).
Bump in the wire (BITW) solutions – A hardware device is added to provide IPSec services between two
routers that are not capable of IPSec functions. This special IPsec device will intercept then intercept
outgoing datagrams, add IPSec protection to them, and strip it off incoming datagrams. BITW can all
IPSec to legacy hosts and can retrofit non-IPSec routers to provide security benefits. The disadvantages
are complexity and cost.
These inherent vulnerabilities, along with increasing connectivity between IT an OT networks, make networkbased attacks very feasible. Simple injection of malicious protocol commands provides control over the target
process. Altering legitimate protocol traffic can also alter information about a process and disrupt the legitimate
controls that are in place over that process. A man- in-the-middle attack could provide both control over a
process and misrepresentation of data back to operator consoles.
7.2 Security Trends in Utility Networks
Although advanced telecommunication networks can assist in transforming the energy industry, playing a critical
role in maintaining high levels of reliability, performance, and manageability, they also introduce the need for an
integrated security infrastructure. Many of the technologies being deployed to support smart grid projects—
such as smart meters and sensors can increase the vulnerability of the grid to attack.
Top security concerns for utilities migrating to an intelligent smart grid communications platform center on the
following trends:
•
Integration of distributed energy resources
•
Proliferation of digital devices to enable management, automation, protection, and control
•
Regulatory mandates to comply with standards for critical infrastructure protection
•
Migration to new systems for outage management, distribution automation, condition-based
maintenance, load forecasting, and smart metering
•
Demand for new levels of customer service and energy management
This development of a diverse set of networks to support the integration of microgrids, open-access energy
competition, and the use of network-controlled devices is driving the need for a converged security
infrastructure for all participants in the smart grid, including utilities, energy service providers, large commercial
and industrial, as well as residential customers. Securing the assets of electric power delivery systems, from the
control center to the substation, to the feeders and down to customer meters, requires an end-to-end security
infrastructure that protects the myriad of communication assets used to operate, monitor, and control power
flow and measurement.
189
Cyber security refers to all the security issues in automation and communications that affect any functions
related to the operation of the electric power systems.
Specifically, it involves the concepts of:
•
•
•
•
Integrity – data cannot be altered undetectably
Authenticity – the communication parties involved must be validated as genuine
Authorization – only requests and commands from the authorized users can be accepted by the system
Confidentiality – data must not be accessible to any unauthenticated users
When designing and deploying new smart grid devices and communication systems, it´s imperative to
understand the various impacts of these new components under a variety of attack situations on the power grid.
Consequences of a cyber attack on the grid telecommunication network can be catastrophic. This is why security
for smart grid is not just an ad hoc feature or product, it’s a complete framework integrating both physical and
Cyber security requirements and covering the entire smart grid networks from generation to distribution.
Security has therefore become one of the main foundations of the utility telecom network architecture and
must be considered at every layer with a defense-in-depth approach. Migrating to IP based protocols is key to
address these challenges for two reasons:
1. IP enables a rich set of features and capabilities to enhance the security posture
2. IP is based on open standards, which allows interoperability between different vendors and products,
driving down the costs associated with implementing security solutions in OT networks.
Securing OT communication over packet-switched IP networks follows the same principles that are foundational
for securing the IT infrastructure, i.e., consideration must be given to enforcing electronic access control for
both person-to-machine and machine-to-machine communications, and providing the appropriate levels of data
privacy, device and platform integrity, and threat detection and mitigation.
7.3 Regulatory Compliance (NERC CIP)
Hydro Québec is subject to regulatory security requirements that are mandated by the North American Electric
Reliability Corporation (NERC). The most recently approved version of NERC CIP in 2014 is version 5. Compliance
with NERC CIP standards requires comprehensive cyber security solutions including segmentation,
authentication, authorization, monitoring, logging, and training; and comprehensive physical security solutions
including access control, and video surveillance.
The Utility Compliance solution should be based on the following design principles:
•
Proper segmentation of trusted zones such as the Electronic Security Perimeter (ESP) with the use of
routing, firewalling and intrusion detection/prevention technologies at each substation (CIP-005)
190
•
Secure remote access for Interactive Remote Access control of the ESP network (CIP-005)
•
Packet Routing at the control center with Multiprotocol Label Switching (MPLS ) and Flex VPN/DMVPN
support for segmentation and encryption in the core
•
Event Correlation at the control center for monitoring and logging of events for multi-vendor network
devices (CIP-007)
•
Access Control Server and Security Manager for user identity management, physical and electronic
access control (CIP-004)
•
Physical access control solutions such as badge card readers; electronic door locks, controllers and
sensors; physical access manager server; and video surveillance solutions (CIP-006)
The following table summarizes the various sections of the NERC CIP standard and their impact on the
communication network.
NERC/CIP Section
CIP-002
• Bulk Electric System (BES)
Critical Cyber Asset
Identification
• BES Cyber System Identification
• Annual Review and Approval
Solution
Monitoring, Analysis and
Reporting System
•
•
•
•
•
Relevant Features & Benefits
Network topology views, services, links
Network topology and device discovery
Network Compliance Manager
controlling visibility into network
changes and tracking compliance with
a broad variety of regulatory, IT,
corporate governance, and technology
best practices.
Network auto-discovery with
automated asset tracking including the
device line cards, serial number,
firmware version ensures correct
complete global inventory tracking
Network diagrams on demand, which
will help to gain immediate and
accurate insight into network
relationships
191
CIP-003
• Security Management Controls
• Cyber Security Policy
(documentation and
implementation)
• Leadership
• Exceptions
• Information Protection
CIP-004
• Security Awareness Program
• Security Training Program
• Personnel Risk Assessment
Program
• Access Management Program
• Access Revocation Program
CIP-005
• Electronic Security Perimeter
• Electronic Access Controls
• Monitoring Electronic Access
• Interactive Remote Access
Management
CIP 006
• Physical Security
• Physical Security Plan
• Physical Access Controls
• Monitoring Physical Access
• Logging Physical Work
• Access Log Retention
• Maintenance and Testing
• Network Admission
Control
• User authentication,
authorization and
accounting
• Enforcing security policies and
compliance
• Real-time audit trails including who,
what, when, and why
• Security Appliances
(Firewall, IPS, VPN
modules)
• Private VLANs
• Use banners (MOTD,
Login, Exec, AAA, SLIPPPP)
• VPN
• Netflow, IP ACLs
• Wireless Point of Entry
• End-to-end security portfolio can help
in building a strong Electronic Security
Perimeter at all Point of Entries,
including wired and wireless
• Control inbound and outbound access
to high security zones
• Enforce strict controls over remote
access of process control networks
• IP Cameras
• IP Gateway Encoders
• Video Surveillance
manager
• Hardened enclosure
• Physical Cable locks
• Physical security solutions with end-toend solution for physical access
monitoring from cameras, encoders to
stream managers
• Rugged enclosures
• Cable locks for physical security
• Systems to report on disconnected
devices, console access etc.
• Port security can detect unauthorized
access of ports
192
•
•
•
•
CIP-007
• System Security Management
• Test Procedures
• Maintenance and Testing
• Ports and Services
• Security Patch Management
• Malicious Code Prevention
• Security Event Monitoring
• Documentation
•
•
•
•
Host IDS
ACS
IPS
Configuration
assurance solution
Security Monitoring
Syslog
Port security
Event Correlation
• Security Monitoring
System
CIP-008
• Incident Reporting and
Response Planning
• Testing & Documentation
CIP-009
• Recovery Plans Specifications,
Implementation and Testing
• Disaster recovery
manager
• Resource Manager
• SAN products for
disaster recovery and
business continuity
• Asset Management
System
• Network Compliance
Management
• Penetration Testing
• Network encryption
technologies for data in
transit
CIP-010
• Configuration Change
Management
• Vulnerability Assessment
CIP-011
• Information Protection
• Disposal or Re-deployment
Figure 82.
• Protecting against spyware, rootkits
and zero-day attacks
• IPS allows for known signature
detection
• Security Management, Syslog can help
documenting the results of a
vulnerability assessment
• Monitoring System, Firewall, IPS logs
can help in maintaining records
• Configuration assurance solution
supports network configuration audits
and validation
• Port security disallows unwanted ports,
devices
• Help in characterizing and classifying
cyber incidents
• Monitoring System, document and
report incidents
• Resource manager can backup
configuration files
• Disaster recovery manager provides
full data backup and recovery for call
manager clusters
• Enforce the approval of a change in the
network
• Validate configurations are compliant
• Identify areas for security posture
improvement
• Protect and secure handling of BES
Cyber System Information
Network Impacts of NERC CIP v5
7.4 General Security Requirements
Categorization of Cyber Assets
All cyber assets need to be categorized based on their impact on the reliability of the bulk electric
system. This requirement is derived from NERC CIP v5.
193
Furthermore, cyber assets should be mapped to named reliability service(s) that they contribute to.
These services are:
•
•
•
•
•
•
•
•
•
Dynamic response to bulk electric system (BES) conditions
Balancing load and generation
Controlling frequency (real power)
Controlling voltage (reactive power)
Managing constraints
Monitoring & control
Restoration of BES
Situational awareness
Inter-entity real-time coordination and communication
Security Management Control
All cyber assets need to be categorized based on their impact on the reliability of the bulk electric
•
•
•
•
•
•
Detailed cyber security policy
Management and governance
Accountability and responsibility
Change management
Information protection
Exceptions
Electronic Security Perimeter
The electronic security perimeter requirements include:
•
•
•
•
•
•
Electronic access to cyber systems
Inbound and outbound access permissions
Authenticated dialup connections
Detection of malicious software
Remote access through intermediate device or encryption
Multi-factor authentication
Patch management / Firmware Upgrade
The patch management program for field devices requires:
•
•
•
Monitoring Patch requirements (every 35 days)
Locking down logical Ports & services
Disabling ports and services that are not used
194
•
•
•
•
•
•
Prevention of malicious software
Account management / log logins
Maintenance & testing
Systems Security Management
Policy-based access control
Configuration change management and vulnerability assessment
Integrated Security Operations Center (ISOC)
The integrated security operations center requirements include:
•
•
•
•
•
Monitoring and management of both cyber and physical security threats from a centralized
location
Security event monitoring
Cyber vulnerability assessment
Incident reporting and response planning
Recovery plans for bulk electric system cyber assets
Device Hardening – Intrusion Prevention (IPS) and Intrusion Detection (IDS) Capabilities
Device hardening should be done as part of the base configuration of any field device. Device hardening
best practice should be included in the high level design, e.g. the services that should be disabled and
other general good practice configuration requirements.
Intrusion detection and protection enables the identification of malformed SCADA and other data
traffic. This traffic can also be removed. However, in general, utilities implement intrusion detection to
enable them to identify the threat but not delete the packets. This would be deployed for example in
the interconnection between the Corporate and SCADA networks.
Data Manipulation Attacks
The concept of data manipulation is critical because of important role that “data” play in the overall grid
monitoring, automation, and control process and because the alteration of that data can either directly
or indirectly manipulate that process. That network protocols can be vulnerable to attacks is well
known. By sending unexpected or malformed messages that exploit bugs or inadequate defenses (e.g.,
buffer overflows) in protocol implementations, adversaries can crash or hijack victims.
The intention of Manipulation attack is not to crash or hijack the systems but to induce other behaviors
that benefit the adversaries or harm the system.
“Data” here include the values of the SCADA telemetered and calculated points from generation,
transmission, and distribution facilities that are reported to HMIs and SCADA consoles, where it is
consumed by a human operator. Manipulation of these values can influence all aspects of the
automation process.
195
Showing a human operator with misleading values from the field could cause the operator to override
the (legitimate) automation logic, effectively sabotaging their own process. While the operator’s
intentions are good, they are tricked into action through the dissemination of false data.
Manipulating values used by other controllers could prevent supplementary systems—potentially
including protection systems—from behaving properly. Data manipulation can also impact higher-level
operations and business functions including the manipulation of production data to influence energy
trading, demand-response systems, and other back-end systems that utilize real-time energy production
data. Because many of these information systems are integral to “Smart Grid” services, the
manipulation of data within the process control systems of a generation facility can cascade throughout
all areas of the grid.
This security recommendations aim to harden the network infrastructure and its services in a highly
secure environment in order to be compliant with NERC CIP Standards.
196
8. Bibliography
•
Customer Requirements Document, Faramarz Maghsoodlou, Cisco Systems Inc., and Jean
Raymond, Hydro-Québec, June 16, 2014.
•
Hydro Québec and Sony Announce the Establishment of a Joint Venture to Start Development
of a Large-Scale Energy Storage System for Power Grids, April 16, 2014.
•
Ultra Large-Scale Power System Control Architecture, A Strategic Framework for Integrating
Advanced Grid Functionality.
•
GridWise® Transactive Energy Framework, Draft Version
•
Special Publication 800-53, Security and Privacy Controls for federal Information Systems and
Organization.
•
NIST Guidelines for Smart Grid Cyber Security.
•
NPCC Regional Reliability Reference Directory # 4 – Bulk Power System Protection Criteria.
•
Designing a Reliable Power System: Hydro Québec’s Integrated Approach.
•
Hydro Québec’s Defense Plan Against Extreme Contingencies, 1999.
•
NPCC Regional reliability reference Directory # 7 - Special Protection Systems.
•
IEEE 1588, Standard for a Precision Clock Synchronization Protocol for Networked
Measurement and Control Systems
197
9. Glossary
Term
1PPS
AAA - Authentication, Authorization,
and Accounting Server
AMI
Bay
BES
CAIDI
Communications Processor
Control
Control Center
CT
Data Center
DAU
DER
DFR
DLMS
DMS
DoS
DR
DSM
EMS
EV
Description
One pulse per second
Database used for authentication of end clients. Also allows imposing
access policies based on access permissions for a user or group of users.
Collection of interrelated electric power apparatus and functions that
share geographic proximity and logical association. Likely to include
dedicated IEDs for monitoring, control, and protection and may include
a dedicated bay Ethernet switch.
Bulk Electric System (term used in NERC CIP standard)
Customer Average Interruption Duration Index
A form of next generation SCADA RTU that may also integrate
synchrophasor data, time synchronization, remote engineering access,
connection to multiple SCADA masters, local HMI interfaces, and so on.
An operational function used for changing and modifying, intervening,
switching, controlling, parameterization and optimization of a grid asset.
Hosts the applications that monitor, control, and administer the power
grid.
Current Transformer – Transducer
Hosts the enterprise applications and services required by control
center/NOC. May or may not be physically collocated with the control
center.
Data Acquisition Unit – A microprocessor-based device providing CT, PT,
and status indication functions.
Distributed Energy Resource
Digital Fault Recorder – Used to store and analyze a variety of substation
events.
Device Language Message Specification – international standard for
utility meter data exchange
Distribution Management System – A suite of applications that run in
the distribution control center for monitoring and control of the
distribution network. The monitoring and control functions are
performed through the SCADA network. Optimization is performed
through various DMS advanced applications.
Denial of Service
Demand Response
Demand Side Management
Energy Management System – A suite of applications that run in the
transmission control center, used by system operators to monitor,
control, and optimizes the performance of the transmission system. The
monitoring and control functions are performed through the SCADA
network. Optimization is performed through various EMS advanced
applications.
Electric Vehicle
198
FAN
FCAP
FEP
GPS - Global Positioning System with
Inter Range Instrumentation Group
mod B (IRIG-B)
GR
HMI
IED
IPSec
IRIG-B
LAN
LOD
MAIS
Monitoring
MU
NAT
NMS
Field Area Network
FCAPS, which stands for “fault-management, configuration, accounting,
performance, and security” is a network management framework
created by the International Organization for Standardization (ISO).
Front End Processor – SCADA master front-end processor is responsible
for polling information from and controlling remote SCADA RTUs.
Serial time code format for providing precise time-of-day clock. Typical
accuracy specification for IRIG-B: demodulated output shall be within
±100 nanoseconds (average) and ±500 nanoseconds (maximum) of UTC
time. Modulated output and serial port IRIG-B shall be ±1 microsecond
of UTC time.
A special protection scheme for Generation Rejection
Human Machine Interface – The apparatus that presents process data to
a human operator, and through which the human operator controls the
process. Typically provided through a graphical user interface.
Intelligent Electronic Device – Various devices deployed in a substation
that leverage SCADA protocols for communication. They are microprocessor-based controllers. Common types of IEDs include protective
relaying devices, load tap changer controllers, circuit breaker
controllers, capacitor bank switches, recloser controllers, voltage
regulators, and so on. Depending on functions, they may reside in the
substation control building or in a substation yard outdoor enclosure.
Internet Protocol Security
Inter-range instrumentation group time codes – are standard formats
for transferring timing information
Local Area Network
Line Opening Detection
A special protection scheme for automatic 735kV shunt reactor closing
or tripping
An operational function used for local or remote observation of a
system or a process for any changes, which may occur over time. The
term can also be used for observation of the behavior of a data value or
a group of data values.
Merging Unit – Merges data from several CTs or PTs and communicates
those values to the Process Bus.
Network Address Translation
Network Management System – Manages the network devices (routers,
switches, appliances, and so on) in the utility’s corporate network.
Typically managed by the IT department.
Maintains network element inventory and capabilities, performs
periodic data collection from network elements, perform threshold and
trend analysis.
Supports central maintenance of element configurations, firmware
management, security policies, and QoS policies.
Fields incoming asynchronous notifications triggered by out of bounds
conditions. Diagnoses root cause of a related stream of faults.
199
NOC - Network Operations Center
NPCC
OMS
PAT
PDC
PMU
Process Bus
Protocol Stack
PT
PU
PWE3
RLS
RPTC
RPTC
RTP
RTS
RTU
SAIDI
SCADA
SCADA RTU
SCB
SLA
SNTP
SOE
Station Bus
The NOC hosts applications that monitor, control, and administer the
data network. May or may not be collocated with the Data Center or
Control Center.
Northeast Power Coordinating Council
Outage Management System
Port Address Translation
Phasor Data Concentrator
Phasor Measurement Unit – Measures voltage and current values (real
and reactive) as well as frequency and angle from select points on the
power system. This data is time-stamped and sent to the control center
to detect anomalies in the grid.
Conveys, through Ethernet LAN or serial or hardwired connection,
unprocessed power system information (voltage/current samples,
device statuses, etc.) from switchyard source devices such as CTs, PTs,
DAUs, or MUs to the IEDs/relays that process the data into
measurements and control/protection decisions.
A particular software implementation of a computer networking
protocol suite.
Potential Transformer – Transducer
Per Unit
Pseudo Wire Emulation Edge to Edge
A special protection scheme for Remote Load Shedding
Generator Tripping & Load Shedding
A special protection scheme that includes Generation Rejection (GR),
Remote Load Shedding (RLS), and Remote Tripping of Shunt Reactor
(RTS)
Real-Time Transport Protocol is used for transferring audio across the
network.
A special protection scheme for Remote Tripping of Shunt Reactor
Remote Terminal Unit
System Average Interruption Duration Index
Supervisory Control And Data Acquisition
The SCADA RTU is typically a legacy system that does not support
advanced processing capabilities or Ethernet/IP interfaces.
Series Compensation Bypass
Service Level Agreement
Simple Network Time Protocol – SNTP is a networking protocol for clock
synchronization between computer systems over packet-switched,
variable latency networks.
Sequence of Events Recorder – Produces chronological list of when
monitored devices change state for post-disturbance analysis.
This requires 1ms of accuracy.
Interconnects IEDs, distributed controllers, and HMI. Provides the
connection from these devices to the WAN router. May also provide
direct connectivity for DFR, SCADA RTUs, communication processors if a
distributed controller is not present.
200
Substation Control Building
Substation Yard
TOU
UFLS
UVLS
WAN
Resides in the substation location. Used to host the Station Bus,
multiservice bus, and IEDs and other substation devices that interface to
the Station Bus.
An extension of the substation control building hosting devices that are
connected back to the Control Building IEDs via the Process Bus.
Time of Use
A special protection scheme for Under Frequency Load Shedding
A special protection scheme for Under Voltage Load Shedding
Wide Area Network – The WAN is the primary link from the substation
to the energy control center with optional backup link for redundancy.
The WAN may be leased network capacity or a private network owned
and operated by the utility.
201

Here - Hydro

Transcription

Similar documents

Training Objectives Overview 3

smart pac iii

Windmolens en warmtepompen

Fluke 1520 MegOhmMeter - Technical Diagnostic Services

PREVECTRON® key advantages Fully controlled upward streamer

amprobe - Chevrier Instruments inc.

TDA2040 - Micropik

alternative routes

SC 255-96M SC 260-96M SC 265-96M