6_Attilla de Groot

v
Routing on a host
with Openstack
Cumulus Networks
Attilla de Groot
Systems Engineer
HCIE #3494, CCNP, JNCIA
cumulusnetworks.com
1
Transformation: First Servers, Now Networking
Applications, OS and Hardware
LOCKED
OPEN
First:
Compute
Transformed
Now:
Networking
Transforms
Open Ecosystem
Agile, open, scalable with unprecedented cost savings
Open Networking Enables Platform Choice and Affordable Capacity
cumulusnetworks.com
cumulusnetworks.com
2
Great Physical Networking with the Cumulus® Linux® OS
NSX
app
app
app
operating system
hardware
APP
APP
APP
OPERATING SYSTEM
HARDWARE
Single-Vendor blob
Customer Choice
- Constrained innovation
- High margins for vendors
- Slow feature dev cycles
- Vibrant DevOps community
- Vendors must perform
- Rapid deployment
cumulusnetworks.com
Industry-Standard Network Hardware
Juniper
Cisco
Arista
Extreme
7500E
X770
Brite-Box | White Box
Nexus 3100
HCL:
Agema, Edge-Core, Penguin
Computing, Quanta, Dell, HP, Mellanox
QFX 3500
Nexus 9000
7250X
powered by
cumulusnetworks.com
4
User Space
Linux As a NOS: Version 1
Vendor Blob (Holds Master State)
Linux Kernel
Linux networking is not used at all
Routing
Tables
ARP
Table
Bridge
Table
Ethernet
Interfaces
CPU, RAM, Flash, etc.
SAI
ASIC
Driver
Switch Silicon
Front Panel Ports
cumulusnetworks.com
5
Linux As a NOS: Version 2
User Space
Apps
Vendor Blob (Holds Master State)
Partial Kernel sync
Linux Kernel
Linux kernel networking is used partially
Routing
Tables
ARP
Table
Bridge
Table
Ethernet
Interfaces
CPU, RAM, Flash, etc.
SAI
ASIC
Driver
Switch Silicon
Front Panel Ports
cumulusnetworks.com
6
Cumulus® Linux® Architecture
Linux Kernel
User Space
Quagga
Routing
Suite
VxLAN
Network Orchestration
Bridging
Routing
Tables
VXLAN
ARP
Table
Monitoring
Automation
Third Party/Customer Applications
Bridge
Table
Ethernet
Interfaces
CPU, RAM, Flash, etc.
switchd
Switch HAL
ASIC
Driver
Switch Silicon
Front Panel Ports
cumulusnetworks.com
7
Traditional Hierarchical Network Topology
§ Legacy and limitations
§ Not designed for today’s data
center running modern workloads
Core
L3
•
•
§ Numerous proprietary protocols
Aggregation
L2
Server density
Increased server-to-server traffic
•
PVST/PVST+/RPVST, VTP, HSRP, VPC
§ “This is what we’ve been taught”
Access
v1.0
Cumulus Networks and Partner Confidential
The Modern Data Center Topology
Flatter & scalable topologies, non-blocking fabric,
modern management techniques
Spine Layer
Spine Layer
40GE/100GE
Leaf Layer
Leaf Layer
10GE/25GE
Server Nodes
Storage Nodes
Cumulus ™ Linux® operating system
running on bare-metal switches
cumulusnetworks.com
9
MLAG Fabric
Spines
MLAG Setup
MLAG
§ Full MLAG Fabric
§ MLAG in backbone
Leafs
§ LACP from servers
§ Classic L2 connectivity
MLAG
§ Limited scalability
MLAG
Servers
cumulusnetworks.com
10
Layer3 to TOR
Spines
Layer3 to TOR
eBGP / OSPF
ECMP
§ CLOS Fabric
VXLAN Overlay
§ Routing protocol to TOR
Leafs
§ LACP from servers
§ One device from the server
MLAG
§ SDN/Overlay with VXLAN
MLAG
Servers
§ ML2 Provisioning
cumulusnetworks.com
11
Openstack ML2 traffic flow
VXLAN ID 1000
VXLAN ID 2000
ML2
driver
ML2
driver
swp1
eth1
ML2
driver
eth1
ML2
driver
VM2
swp1
swp3
eth2
Compute1
VM1
swp2
swp3
eth1
eth2
ML2
driver
Compute2
VM1
swp2
VM2
cumulusnetworks.com
eth2
Network
Vrouter1
Vrouter2
12
Full Layer3 fabric
Routing on the Host
Spines
§ Full Layer3 Fabric
eBGP / OSPF
§ High capacity, scalable
networking
ECMP
§ IP Fabric to the hosts
§ No more Layer 2 domains
§ Cumulus Quagga
improvements
Leafs
eBGP / OSPF
eBGP / OSPF
ECMP
ECMP
Servers
Quagga
10.1.1.1/32 Lo1
Quagga
10.1.1.2/32 Lo1
cumulusnetworks.com
Quagga
10.1.1.3/32 Lo1
Quagga
10.1.1.4/32 Lo1
13
Openstack ROH traffic flow
swp1
eth1
ML2
driver
eth1
ML2
driver
VM2
swp1
swp3
eth2
Compute1
VM1
swp2
swp3
eth1
eth2
ML2
driver
Compute2
VM1
swp2
Network
Vrouter1
VM2
eth2
Vrouter2
VXLAN ID 1000
VXLAN ID 2000
cumulusnetworks.com
14
Minimizing configuration
Interfaces
BGP Unnumbered
§ Peers on IPv6 link-local
§ Neighbor detection on RA
messages
§ RFC 5549
IPv4 over IPv6 neighbor
§ Removing unnecessary
variables
§ Troubleshooting additions
auto lo
iface lo inet loopback
address 10.100.1.1
auto lo
iface lo inet loopback
address 10.100.1.1
auto swp1
iface swp1
address 172.30.1.1/30
address 2001:DB8:1::1/64
auto swp1
iface swp1
auto swp2
iface swp2
address 172.30.1.5/30
address 2001:DB8:2::1/64
auto swp2
iface swp2
BGP configuration
router bgp 64501
bgp log-neighbor-changes
bgp router-id 10.0.0.1
!
neighbor 10.1.1.1 remote-as
neighbor 10.1.2.1 remote-as
neighbor 10.1.3.1 remote-as
neighbor 10.1.4.1 remote-as
cumulusnetworks.com
65000
65000
65000
65000
router bgp 64501
bgp log-neighbor-changes
bgp router-id lo
!
neighbor swp1 remote-as external
neighbor swp2 remote-as external
neighbor swp3 remote-as external
neighbor swp4 remote-as external
15
RoH configuration
Interfaces
auto lo
iface lo inet loopback
address 10.100.1.1
swp1
swp2
swp1
swp2
auto swp1
iface swp1
auto swp2
iface swp2
Quagga BGP
interface swp1
ipv6 nd ra-interval 5
interface swp2
ipv6 nd ra-interval 5
eth1
Node1
eth2
eth1
router bgp 65001
bgp router-id 10.100.1.1
maximum-paths 255
bgp bestpath as-path multipath-relax no-as-set
redistribute connected
eth2
Node2
neighbor swp1 interface
neighbor swp1 remote-as external
neighbor swp1 capability extended-nexthop
neighbor swp2 interface
neighbor swp2 remote-as external
neighbor swp2 capability extended-nexthop
cumulusnetworks.com
16
Cumulus VX & Openstack
cumulusnetworks.com
17
Not just a party trick…
cumulusnetworks.com
18
Bringing the Linux Revolution to Networking
§Thank You!
© 2014 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its
affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI,
the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.
cumulusnetworks.com
19