Halo. Jadi kelanjutan dari bagian pertama telah datang. Seperti yang dijanjikan, dalam artikel ini, saya ingin menyentuh opsi utama untuk mengimplementasikan fabric pada VXLAN / EVPN, dan memberi tahu mengapa kami memutuskan untuk memilih solusi ini atau itu di pusat data kami.
Memilih desain Underlay
Kata pengantar
Hal pertama yang harus Anda tangani saat membangun pabrik adalah desain Underlay - bagaimana kita ingin membangun terowongan VXLAN (lebih tepatnya, mengatur pencarian VTEP)?
Kami memiliki 3 opsi:
1. , VTEP - , , , , RSVP-TE . , . SDN.
2.Multicast - , , , .. Cumulus , Juniper resourse-intensive .
3.BGP BGP, - BGP? , - , , BGP, EVPN BGP. . iBGP eBGP, underlay .
iBGP
iBGP IGP, OSPF IS-IS ( ), .. Loopback, , , - . , iBGP , full-mesh ( Spine BGP).
Spine Route-Reflector.
2 full-mesh RR. , , , .
eBGP
, , , . eBGP, Route-Reflector (, ), IGP , .. p2p . MLAG . . MLAG AS, .. VXLAN/EVPN , AS. peerlink'e, , .. , Spine, - , .
, - , , AS. Cumulus 4.2(.. ) , .. AS, MAC, ( 32-bit AS).
Spine AS. .. Spine Leaf, . , AS Spine, Cumulus.
P.S. AS , AS . 32-bit AS .
BGP Unnumbered
, p2p , loopback . - , Unnumbered. Cumulus Cisco, . Cumulus IPv6 , eBGP . extended-nexthop ( , IPv4 Family IPv6 ).
P.S. - IPv4 /30 /31 , BGP . broadcast , p2p.
BGP+BFD
, BGP . , iSCSI, VSAN .. . BFD. Cumulus 2x50 , Cisco 233 , .
?
1. eBGP AS + iBGP MLAG
2. loopback, Unnumbered
3.BFD
1. AS Cumulus
# Cumulus AS
#P.S. Spine AS, .. AS-path
cumulus@Switch1:mgmt:~$ net add bgp autonomous-system
<1-4294967295> : An integer from 1 to 4294967295
leaf : Auto configure a leaf ASN in the 4-byte private range 4200000000 - 4294967294 based on the switch
MAC
spine : Auto configure a spine AS-number in the 4-byte private ASN range. The value 4200000000 is always
used
# "net add bgp autonomous-system leaf"
cumulus@Switch1:mgmt:~$ net add bgp autonomous-system leaf
cumulus@Switch1:mgmt:~$ net pending
+router bgp 4252968529 # BGP AS
+end2. BGP+Unnumbered
#loopback
net add loopback lo clag vxlan-anycast-ip 10.223.250.30 # MLAG IP (
net add loopback lo ip address 10.223.250.1/32
#AS+Router ID
net add bgp autonomous-system leaf
net add bgp router-id 10.223.250.1
# peer-group
net add bgp neighbor fabric peer-group # peer
net add bgp neighbor fabric remote-as external # eBGP
net add bgp neighbor fabric bfd 3 50 50 # BFD
net add bgp neighbor fabric capability extended-nexthop # IPv4 over IPv6
# (Unnumbered)
net add bgp neighbor swp2 interface peer-group fabric
net add bgp neighbor peerlink.4094 interface remote-as internal #iBGP Peerlink
net add bgp ipv4 unicast neighbor peerlink.4094 next-hop-self # next-hop iBGP
net add bgp ipv4 unicast redistribute connected # BGP
#EVPN
net add bgp l2vpn evpn neighbor fabric activate # family eBGP
net add bgp l2vpn evpn neighbor peerlink.4094 activate # family iBGP
net add bgp l2vpn evpn advertise-all-vni # BGP VNI 3.BGP Summary
cumulus@Switch1:mgmt:~$ net show bgp summary
#IPv4
show bgp ipv4 unicast summary
=============================
BGP router identifier 10.223.250.1, local AS number 4252968145 vrf-id 0
BGP table version 84
RIB entries 29, using 5568 bytes of memory
Peers 3, using 64 KiB of memory
Peer groups 1, using 64 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
Switch3(swp2) 4 4252424337 504396 484776 0 0 0 02w0d23h 3
Switch4(swp49) 4 4208128255 458840 485146 0 0 0 3d12h03m 9
Switch2(peerlink.4094) 4 4252968145 460895 456318 0 0 0 02w0d23h 14
Total number of neighbors 3
#EVPN
show bgp l2vpn evpn summary
===========================
BGP router identifier 10.223.250.1, local AS number 4252968145 vrf-id 0
BGP table version 0
RIB entries 243, using 46 KiB of memory
Peers 3, using 64 KiB of memory
Peer groups 1, using 64 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
Switch3(swp2) 4 4252424337 504396 484776 0 0 0 02w0d23h 237
Switch4(swp49) 4 4208128255 458840 485146 0 0 0 3d12h03m 563
Switch2(peerlink.4094) 4 4252968145 460895 456318 0 0 0 02w0d23h 807
Total number of neighbors 3
4.net show route
cumulus@Switch1:mgmt:~$ net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
# IPv4 IPv6 ( Cumulus weight, Cisco)
C>* 10.223.250.1/32 is directly connected, lo, 02w0d23h #Loopback
B>* 10.223.250.2/32 [200/0] via fe80::ba59:9fff:fe70:e50, peerlink.4094, weight 1, 02w0d23h
B>* 10.223.250.6/32 [20/0] via fe80::ba59:9fff:fe70:e5c, swp49, weight 1, 3d12h19m
B>* 10.223.250.7/32 [20/0] via fe80::ba59:9fff:fe70:e5c, swp49, weight 1, 3d12h19m
B>* 10.223.250.9/32 [20/0] via fe80::ba59:9fff:fe70:e5c, swp49, weight 1, 3d12h19m
C>* 10.223.250.30/32 is directly connected, lo, 02w0d23h #MLAG Loopback
B>* 10.223.250.101/32 [20/0] via fe80::1e34:daff:fe9e:67ec, swp2, weight 1, 02w0d23h
B>* 10.223.250.102/32 [20/0] via fe80::1e34:daff:fe9e:67ec, swp2, weight 1, 02w0d23h
B>* 10.223.250.103/32 [20/0] via fe80::1e34:daff:fe9e:67ec, swp2, weight 1, 02w0d23h
B>* 10.223.252.11/32 [200/0] via fe80::ba59:9fff:fe70:e50, peerlink.4094, weight 1, 02w0d06h
B>* 10.223.252.12/32 [200/0] via fe80::ba59:9fff:fe70:e50, peerlink.4094, weight 1, 02w0d06h
B>* 10.223.252.20/32 [200/0] via fe80::ba59:9fff:fe70:e50, peerlink.4094, weight 1, 02w0d06h
B>* 10.223.252.101/32 [200/0] via fe80::ba59:9fff:fe70:e50, peerlink.4094, weight 1, 02w0d06h
B>* 10.223.252.102/32 [200/0] via fe80::ba59:9fff:fe70:e50, peerlink.4094, weight 1, 02w0d06h
B>* 10.223.252.103/32 [200/0] via fe80::ba59:9fff:fe70:e50, peerlink.4094, weight 1, 02w0d06h5.traceroute + BFD
#traceroute ( TCP )
cumulus@Switch1:mgmt:~$ traceroute -s 10.223.250.1 10.223.250.6
vrf-wrapper.sh: switching to vrf "default"; use '--no-vrf-switch' to disable
traceroute to 10.223.250.6 (10.223.250.6), 30 hops max, 60 byte packets
1 10.223.250.7 (10.223.250.7) 1.002 ms 1.010 ms 0.981 ms # loopback
2 10.223.250.6 (10.223.250.6) 0.933 ms 0.917 ms 1.018 ms
# BFD
cumulus@Switch1:mgmt:~$ net show bfd
------------------------------------------------------------------------------------------
port peer state local type diag vrf
------------------------------------------------------------------------------------------
swp2 fe80::1e34:daff:fe9e:67ec Up fe80::1e34:daff:fea6:b53d singlehop N/A N/A
swp49 fe80::ba59:9fff:fe70:e5c Up fe80::1e34:daff:fea6:b510 singlehop N/A N/A
Underlay , Overlay.
Overlay
, Overlay VXLAN VTEP EVPN. . 3 SVI, .
Centralized IRB
, centralized switch. (active-active ), L3 , Leaf L2. entralized switch EVPN type-2 (MAC+IP) c Default Gateway community(0x03). centralized switch VNI .
, , , Leaf L3. .. L3 , , .
Asymmetric IRB
Symmetric IRB ( ), . VTEP, L2. , Leaf VNI+Vlan, L3VNI.
P.S. , , ( VNI). , VLAN VRF. , , c .
Symmetric IRB
L3 L3VNI, VLAN. VNI VTEP.
, , , VM3 VM4. Leaf03/04, route-lookup, VM4 Leaf05/06, nexthop L3VNI . Leaf05/06 , VXLAN SVI20. SVI , , L3VNI .
Overlay.
Overlay , VTEP .
1. L3VNI
#VRF+VNI
net add vrf Test vrf-table auto #C VRF RD+RT
net add vrf Test vni 200000 # VNI VRF
net add vxlan vniTest vxlan id 200000 # L3VNI
net add vxlan vniTest bridge learning off # , .. EVPN
net add vxlan vniTest vxlan local-tunnelip 10.223.250.1 #
#VLAN
net add vlan 2000 hwaddress 44:38:39:BE:EF:AC # MLAG, Active-Active MAC
net add vlan 2000 vlan-id 2000 #
net add vlan 2000 vlan-raw-device bridge # bridge
net add vlan 2000 vrf vniTest # VRF
net add vxlan vniTest bridge access 2000 # VLAN L3VNI2. VNI
#VNI
net add vxlan vni-20999 vxlan id 20999
net add vxlan vni-20999 bridge arp-nd-suppress on # ARP-supress - BUM
net add vxlan vni-20999 bridge learning off # , .. EVPN
net add vxlan vni-20999 stp bpduguard # bpduguard
net add vxlan vni-20999 stp portbpdufilter # bpdufilter
net add vxlan vni-20999 vxlan local-tunnelip 10.223.250.1 #
# VLAN
net add vlan 999 ip address 10.223.255.253/24 # IP VLAN
net add vlan 999 ip address-virtual 44:39:39:ff:01:01 10.223.255.254/24 # ( gateway Leaf)
net add vlan 999 vlan-id 999 #
net add vlan 999 vlan-raw-device bridge # bridge
net add vlan 999 vrf Test # VRF
net add vxlan vni-20999 bridge access 999 # VLAN VNI
#P.S. L2 only vlan ( VNI )
net add vlan 999 ip forward off
net add vlan 999 vlan-id 999
net add vlan 999 vlan-raw-device bridge
3.
# BGP VRF
net add bgp vrf Test autonomous-system 4252424337
net add bgp vrf Test router-id 10.223.250.101
net add bgp vrf Test neighbor 100.64.1.105 remote-as 35083
net add bgp vrf Test ipv4 unicast redistribute connected
net add bgp vrf Test ipv4 unicast redistribute static
net add bgp vrf Test ipv4 unicast neighbor 100.64.1.105 route-map Next-Hop-VRR_Vl997 out # MLAG+VSS BGP+SVI,
#P.S. /32 EVPN, .
net add bgp vrf Test l2vpn evpn advertise ipv4 unicast # EVPN
, :
#VNI
cumulus@Switch1:mgmt:~$ net show evpn vni
VNI Type VxLAN IF MACs ARPs Remote VTEPs Tenant VRF
20995 L2 vni-20995 5 3 1 default #VNI, IP SVI
20999 L2 vni-20999 26 24 4 Test #VNI
200000 L3 vniTest 3 3 n/a Test # L3VNI
#VNI
cumulus@Switch1:mgmt:~$ net show evpn vni 20999
VNI: 20999
Type: L2
Tenant VRF: Test
VxLAN interface: vni-20999
VxLAN ifIndex: 73
Local VTEP IP: 10.223.250.30 # MLAG, vxlan-anycast-ip
Mcast group: 0.0.0.0 #
Remote VTEPs for this VNI: # VNI
10.223.250.9 flood: HER # Head-end Replication ( Ingress replication)
10.223.252.103 flood: HER
10.223.252.20 flood: HER
10.223.250.103 flood: HER
Number of MACs (local and remote) known for this VNI: 26
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 24
Advertise-gw-macip: No #Community centralized IRB#
cumulus@Switch3:mgmt:~$ net show evpn mac vni 20999
Number of MACs (local and remote) known for this VNI: 28
Flags: B=bypass N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy
MAC Type Flags Intf/Remote ES/VTEP VLAN Seq #'s
0c:59:9c:b9:d8:dc remote 10.223.250.30 0/0
0c:42:a1:95:79:7c remote 10.223.250.30 0/0
# MAC, Interface VNI
cumulus@Switch3:mgmt:~$ net show bridge macs vlan 999
VLAN Master Interface MAC TunnelDest State Flags LastSeen
---- ------ --------- ----------------- ---------- --------- ------------ -----------------
999 bridge bridge 1c:34:da:9e:67:68 permanent 24 days, 04:35:13
999 bridge bridge 44:39:39:ff:01:01 permanent 14:26:45
999 bridge peerlink 1c:34:da:9e:67:48 permanent 31 days, 17:22:36
999 bridge peerlink 1c:34:da:9e:67:e8 static sticky 24 days, 04:14:16
999 bridge vni-20999 00:16:9d:9e:dd:41 extern_learn 19 days, 04:17:21
999 bridge vni-20999 00:21:1c:2e:86:42 extern_learn 19 days, 04:17:21
999 bridge vni-20999 00:22:0c:de:30:42 extern_learn 19 days, 04:17:21
# VTEP
cumulus@Switch3:mgmt:~$ net show evpn rmac vni all
VNI 200000 RMACs 3
RMAC Remote VTEP
44:38:39:be:ef:ac 10.223.250.30
44:39:39:ff:40:94 10.223.252.103
44:38:39:be:ef:ae 10.223.250.9
#Arp-cache
cumulus@Switch3:mgmt:~$ net show evpn arp-cache vni 20999
Number of ARPs (local and remote) known for this VNI: 28
Flags: I=local-inactive, P=peer-active, X=peer-proxy
Neighbor Type Flags State MAC Remote ES/VTEP Seq #'s
10.223.255.242 local active 1c:34:da:9e:67:68 0/0
10.223.255.13 remote active 0c:42:a1:96:d2:44 10.223.250.30 0/0
10.223.255.243 local inactive 06:73:4a:02:27:8a 0/0
10.223.255.7 remote active 0c:59:9c:b9:f8:fa 10.223.250.30 0/0
10.223.255.14 remote active 0c:42:a1:95:79:7c 10.223.250.30 0/0#
cumulus@Switch1:mgmt:~$ net show route vrf Test | grep "10.3.53"
// next-hop vlan2000, L3VNI
C * 10.223.255.0/24 [0/1024] is directly connected, vlan999-v0, 03w3d04h
C>* 10.223.255.0/24 is directly connected, vlan999, 03w3d04h
B>* 10.223.255.1/32 [20/0] via 10.223.250.30, vlan2000 onlink, weight 1, 02w5d04h
B>* 10.223.255.2/32 [20/0] via 10.223.250.30, vlan2000 onlink, weight 1, 02w5d04h
B>* 10.223.255.3/32 [20/0] via 10.223.250.30, vlan2000 onlink, weight 1, 02w5d04h
# EVPN
cumulus@Switch3:mgmt:~$ net show bgp evpn route vni 20999
BGP table version is 1366, local router ID is 10.223.250.101
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[ESI]:[EthTag]:[IPlen]:[VTEP-IP]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
*> [2]:[0]:[48]:[00:16:9d:9e:dd:41]
10.223.250.30 0 4252968145 i
RT:9425:20999 RT:9425:200000 ET:8 Rmac:44:38:39:be:ef:ac
*> [2]:[0]:[48]:[00:22:56:ac:f3:42]:[32]:[10.223.255.1]
10.223.250.30 0 4252968145 i
RT:9425:20999 RT:9425:200000 ET:8 Rmac:44:38:39:be:ef:ac
, VXLAN/EVPN . Underlay Unnumbered BGP , Overlay VXLAN/EVPN c Symmetric IRB, . , .
, Spine, . , - . , , .