Substation Communications Design

Transcription

Substation
Communications
Design Legacy to IEC 61850
Part 3/3:
Reliability & Security
Tim Wallaert
Chris Jenkins
Agenda
•
Substation to the Control Room Communications
−
−
−
−
•
?
How to Build a Redundant Network
−
−
−
−
−
−
•
Legacy networks
Networking today
CIP
Hardened equipment
RSTP
MRP
Routing
Router
Cellular
PRP/HSR
How to Lock it Down
−
Firewalls
−
VPN
Port Security
Authentication
−
−
© 2014 Belden Inc. | belden.com | @BeldenInc
2
Legacy Utility Networks
1200 baud
SCADA
Master
Administration
Modem Bank
Dial-in
Leased
Lines
IED
RTU
IED
IED
RTU
IED
IED
Distributed Substations
3
RTU
IED
Today’s Digital Networks
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
PBX
GW
HMI
Switch
Term Server
Video
Monitoring
Video
Storage
PoE
HMI
Wide Area
Network
PoE
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
VOIP
HMI
PoE
Enet
IED
SerialSerial
Serial
IED IED
IED
Sub A
Sub C
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
Sub B
4
New networks have to be compliant…
What does “compliance” mean?
• FERC/NERC-CIP
− Federal Energy Regulatory Commission
− North American Electric Reliability Corporation
• http://www.nerc.com/pa/Stand/Pages/CIPStandards.aspx
• Critical Infrastructure Protection (CIP)
• Version 1 enforced in 2008
• Currently enforcing Version 3
• Version 4 has been approved, April 2014 deadline
• Still to be approved Version 5 is being pushed to replace Version 4
• Version 5 has MANY changes including:
• Encryption
• Multi-Factor Authentication
•
5
Federal Regulation
• Critical Infrastructure Protection (CIP) is a group of
standards enforced by NERC
• NERC does not certify equipment
• There is no such thing as a “CIP certified router”
• Belden provides equipment that enables customers to
design and implement networks that are CIP compliant
• Constantly monitoring NERC for changes in standards
and listening to feedback from our customers
6
“I need to upgrade my network and stay
compliant, but how?”
7
Turn to a Trusted Leader
8
Solutions Portfolio
9
Important Specs to Consider
•
Experienced, Reliable
Vendor
•
Standards Based
Equipment
•
Environment
Extended Temp ranges
 Noise Immunity
 IEEE1613
 IEEE61850-3

10
IEEE 1613
IEC 61850-3
Contact
8kV
Air
15kV
ESD
Radiated RF
35V/M
I/O ports
Fast
Transient
Oscillatory
Dielectric
Strength
Operating
Temperature
4kV
Power ports
(HV and LV)
4kV
I/O ports
2.5kV
Power ports
(HV and LV)
2.5kV
HV power ports
3kV
LV power ports
2kV
-40 to +85+ C
ESD
(61000-4-2)
Contact
8kV
Air
15kV
Radiated RF
(61000-4-3)
Fast Transient
(61000-4-4)
Surge
(61000-4-5)
Conducted RF
(61000-4-6)
20V/M
I/O ports
4kV
Power ports (HV
and LV)
4kV
I/O ports
4kV
Power ports (HV
and LV)
4kV
I/O ports
10V
Power ports (HV
and LV)
10V
Magnetic Field
(61000-4-8)
Voltage Dips
& Interrupts
(61000-4-11)
30A/m
HV power ports
11
Pass
•
Redundant Power options
− Low
and High voltage options
− Dual power supply features
− Hot Swappable
PS1
PS2
12
• Redundancy features
- Serial Port
- RSTP
- MRP
- RIP, OSPF, BGP
- VRRP
- Cellular redundancy
- PRP/HSR
• Security
- Encryption
- Authentication
- Firewalls
- Detection
13
Redundancy
Features
14
Serial Redundancy
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
PBX
GW
HMI
Switch
Term Server
Video
Monitoring
Video
Storage
PoE
HMI
Wide Area
Network
PoE
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
VOIP
HMI
PoE
Enet
IED
Sub A
Sub C
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
SerialSerial
Serial
IED IED
IED
Sub B
15
Serial Redundancy
Poll
Master
Site
SCADA/
EMS
SCADA/
EMS
HMI
PoE
HMI
Wide Area
Network
VOIP
Enet
IED
Term Server
Term Server
Video
Monitoring
HMI
PoE
SerialSerial
Serial
IED IED
IED
VOIP
Sub B
RESP
RESP
SerialSerial
PoE
RESP
SerialSerial
Serial
IED IED
IED
Sub C
Serial
IED IED
IED
Enet
IED
Video
Monitoring
VOIP
Enet
IED
Sub A
Backup
Site
16
Ethernet Redundancy Protocols
Protocol
Parallel Redundancy Protocol
(PRP)
Current Standard
Typical Re-Config
Topology
Available
since
IEC 62439-3:2012-07 0mS
Any topology/mesh
2010
IEC 62439-3:2012-07 0mS
Pure Ring Only
2010
IEEE 802.1D-2004
5-20mS per switch
Any topology/mesh
2004
Media Redundancy Protocol (MRP) IEC 62439-2:2010
200mS worst case, 50
switches max
Pure Ring Only
1998/2007
Routing Information Protocol (RIP &
RIP 2)
RFC 1723
~30sec
small networks
1988/1994
Open Shortest Path First (OSPF)
RFC 2328
seamless
Any topology/mesh
1987/1998
Border Gateway Protocol (BGP)
RFC 4271
seamless
Any topology/mesh
1989/2006
High-Availability Seamless
Redundancy (HSR)
Rapid Spanning Tree Protocol
(RSTP)
17
RSTP Redundancy
18
Rapid Spanning Tree Protocol (RSTP) Bridging
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
Video
Monitoring
PBX
GW
Switch
Term Server
Video
Storage
BPDU
HMI
HMI
PoE
HMI
PoE
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
Sub A
VOIP
IED
BPDU
Enet
IED
SerialSerial
Serial
IED IED
IED
SerialSerial
Serial
IED IED
IED
Sub C
Sub B
PoE
19
Rapid Spanning Tree Protocol (RSTP) Bridging
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
Video
Monitoring
DATA
DATA
DATA
PBX
GW
Switch
Term Server
Video
Storage
BPDU
BPDU
HMI
PoE
~5 -15ms recovery
HMI
PoE
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
Sub A
HMI
VOIP
IED
BPDU
BPDU
Enet
IED
SerialSerial
Serial
IED IED
IED
SerialSerial
Serial
IED IED
IED
Sub C
Sub B
PoE
20
MRP Redundancy
21
Media Redundancy Protocol (MRP)
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
PBX
GW
Video
Storage
HELLO
HELLO
HELLO
Ring
Switch
HMI
Switch
Term Server
Video
Monitoring
Ring
Manager
PoE
*10mS recovery
VOIP
HMI
Ring
Switch
HMI
PoE
PoE
VOIP
VOIP
IED
IED
SerialSerial
Serial
IED IED
IED
Sub C
IED
SerialSerial
Serial
IED IED
IED
Term Server
IED
IED
Sub C
Sub B
* 50 switches per ring max
22
Routing Protocol
Redundancy
23
Routing Protocols RIP, OSPF, BGP
Master
Site
SCADA/
EMS
PBX
GW
Switch
Term Server
Video
Monitoring
HMI
RTU Management &
Provisioning
Video
Storage
PoE
HMI
Wide Area
Network
PoE
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
HMI
PoE
VOIP
Enet
IED
Sub A
SerialSerial
Serial
IED IED
IED
Sub C
Enet
IED
SerialSerial
Serial
IED IED
IED
Sub B
24
SCADA/
EMS
PBX
GW
Master
HMI
Substation C is
over here
Video
Term Server
Video
Monitoring
Substation A is
over here
RTU
Mgmt
Storage
PoE
PoE
HMI
HMI
PoE
PoE
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
HMI
PoE
VOIP
Enet
IED
Sub A
SerialSerial
Serial
IED IED
IED
Sub C
Enet
IED
SerialSerial B is
Substation
Serial
IED IED
IED
over here
Sub B
25
Master
Site
SCADA/
EMS
PBX
GW
Term Server
Video
Monitoring
HMI
RTU
Mgmt
PoE
Video
Storage
Wide Area Network
HMI
PoE
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
HMI
PoE
VOIP
Enet
IED
Sub A
SerialSerial
Serial
IED IED
IED
Sub C
Enet
IED
SerialSerial
Serial
IED IED
IED
Sub B
26
Router Redundancy
27
Virtual Router Redundancy Protocol (VRRP)
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
Video
Monitoring
PBX
GW
Term Server
Switch
Video
Storage
Slave
Router(s)
Master
Router
Wide Area Network
28
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
Video
Monitoring
PBX
GW
Term Server
Switch
Video
Storage
Slave
Router(s)
Master
Router
Wide Area Network
29
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
Video
Monitoring
DATA
PBX
GW
Term Server
Switch
Video
Storage
Slave
Router(s)
Master
Router
Wide Area Network
30
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
Video
Monitoring
DATA
PBX
GW
Term Server
Switch
Video
Storage
Slave
Router(s)
Master
Router
Wide Area Network
31
Cellular Redundancy
32
Before Cellular
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
Video
Monitoring
PBX
DATA
ATA
GW
Switch
Term Server
Video
Storage
Wide Area
Network
HMI
PoE
DATA
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
Sub A
33
Before Cellular
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
PBX
ATA
GW
Switch
Term Server
Video
Monitoring
Video
Storage
Wide Area
Network
HMI
PoE
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
Sub A
34
Cellular Backup
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
PBX
DATA
DATA
GW
Switch
Term Server
Video
Monitoring
Video
Storage
Verizon Wireless
Internet
Wide Area
Network
HMI
VOIP
Enet
IED
PoE
DATA
DATA
SerialSerial
Serial
IED IED
IED
Sub A
35
PRP/HSR Redundancy
36
Parallel Redundancy Protocol (PRP)Protocol)
Zero failover with Network Redundancy
“0ms” recovery
DAN Dual Attached Node
SAN Single Attached Node
SAN
SAN
SAN
SAN
DAN
DAN
SAN
SAN
Two redundant networks
By doubling the packets no data loss if one packet fails
PRP-Redundancy-Box = bidirectional splitter and combiner
37
High-Available Seamless Redundancy (HSR)
HSR
Packet
HSR
Packet
No packet loss
38
Security
39
Today’s Networks
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
PBX
GW
HMI
Switch
Term Server
Video
Monitoring
Video
Storage
PoE
HMI
Wide Area
Network
PoE
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
HMI
PoE
VOIP
Enet
IED
Sub A
SerialSerial
Serial
IED IED
IED
Sub C
Enet
IED
SerialSerial
Serial
IED IED
IED
Sub B
40
Defense in Depth
Wide Area
Network
HMI
PoE
Enet
IED
Enet
IED
Serial
Serial
Serial
IED
IEDIED
SCADA
VOIP
Enet
IED
Serial
Serial
Serial
IED
IEDIED
Serial
Serial
Serial
IED
IEDIED
Enet
IED
IT
Engineering
Sub
41
Firewalls
Public
NonSecured
Network
Private
Secured
Network
•
Integrated or standalone device that can be configured to allow
or block specific traffic and users
•
DX/10Series Routers offer integrated Firewall features
•
Eagles are standalone devices
42
Firewalls
•
Firewall with Stateful Packet Inspection (SPI)
− Both
IP and MAC address filtering supported
•
Network Address Translation (NAT)
•
VPN support
10XTS, 10ETS,
10RX
DX
Eagle
43
Eagle 20 Tofino – Firewall + DPI
•
Firewall with Stateful Packet Inspection (SPI)
•
Layer 2 Bridge with No IP Address
− No
disruption to existing network design
− VERY
•
secure
Content Inspection filters traffic at the protocol
level (Deep Packet Inspection)
− Modbus/TCP
− others
•
to follow
Simple deployment, configuration and
management
44
Tofino™ Modbus TCP Enforcer LSM:
Content Inspector for Modbus
•
Protocol ‘Sanity Check’ blocks any traffic not conforming to the
Modbus standard
•
Control engineer defines list of allowed Modbus commands,
registers and coils
•
Automatically blocks and reports any Modbus traffic that does not
match your rules
45
VPN
46
Virtual Private Networks
RTU Management &
Provisioning
Master
Site
SCADA/
EMS
PBX
GW
Switch
Term Server
Video
Monitoring
Video
Storage
Encrypted using
IPSec
HMI
PoE
Wide Area
Network
HMI
PoE
VOIP
Enet
IED
SerialSerial
Serial
IED IED
IED
HMI
PoE
Enet
IED
SerialSerial
Serial
IED IED
IED
Sub A
Sub C
Enet
IED
SerialSerial
Serial
IED IED
IED
Sub B
47
Security - VPN
•
Hardware and software encryption
•
Multiple tunnel support
•
Pre-Shared Key (PSK) or X.509 Certificates
•
IPSec
•
DX/10Series Routers offer integrated VPN features
•
Eagles are standalone devices
48
Port Security
49
Port Security
•
Default with all ports “administratively set to DOWN”
•
Some devices support “no tail ending”. Port is locked after
being unplugged. Must be enabled by administrator
•
Physical port security devices
•
Unusual port connectors provide a small level of security
50
MAC Based Port Security
•
Secures physical ports by applying a MAC based filter on a per
port basis which allows only the authorized MAC address to
forward traffic from the given port.
51
IP Based Port Security
•
Secures physical ports by applying a IP based filter on a per
port basis which allows only the authorized IP address to
forward traffic from the given port.
52
Authentication
53
Authentication
•
Switches and Routers
support RADIUS Authentication
− Protects
access to the
1
console ports
− Authenticates
4
users to
Network
the network
-
Helps satisfy CIP
authentication requirements
2
3
Radius Server
54
Authentication
•
Secure Access
Servers
− Subnet
Control Center
Engineering Access
Solutions
Secure
Access
Manager
− CrossBow
− Cooper
Power
RSA
•
Satisfies CIP access
record keeping
requirements
Network
Communications
Gateway
IED
IED
RTU
IED
IED
RTU
55
Summary
•
Substation to the Control Room Communications
−
−
−
−
•
?
How to Build a Redundant Network
−
−
−
−
−
−
•
Legacy networks
Networking today
CIP
Hardened equipment
RSTP
MRP
Routing
Router
Cellular
PRP/HSR
How to Lock it Down
−
Firewalls
−
VPN
Port Security
Authentication
−
−
56
Top Three Takeaways
•
Multiple Redundant protection schemes to pick from when
designing/upgrading a network
•
Security Features that support CIP compliant networking
requirements
•
Belden - Industry Leading Product Depth and Experience
57
Additional Resources & Assistance
Obtain further Substation Communication resources from our
website:
1.


www.belden.com/power-td/
This webpage includes substation communication diagrams and other useful tools
Contact a Belden representative for assistance:
2.

Call 510-438-9071 if you are in the U.S. or Canada

Or complete the form at www.belden.com/contact/
Thank you for your interest in this presentation!
58
Belden.com
|
@BeldenInc
© 2014 Belden

Substation Communications Design

Transcription

Similar documents

Untitled - PowerBoss Intl

EX-3403