MPLS L3VPN

LIST OF CONTENTS

Intoduction To VPN (Overlay and Peer to Peer VPN) ____________________________________________1

Introduction To VRF And MPLS L3VPN ________________________________________________________3

How VRF filled its routing table______________________________________________________________4

BGP Role in MPLS L3VPN___________________________________________________________________5

Route Distinguisher________________________________________________________________________6

Route Target_______________________________________________________________________________8

Why Do We Need VPN Label In MPLS L3VPN_________________________________________________10

End To End MPLS L3VPN Traffic flow________________________________________________________11

MPLS L3VPN Lab Using Cisco VIOS Image In EVE-NG Community_____________________________13

The material in this note is taken from the Cisco Service Provider course taught by Mr. M Aditya
Hananto in the Indonetworkers Community forum. This note contains material about the Basic
Theory of MPLS L3VPN along with a simple lab.
 1

Introduction To VPN

One of the very popular MPLS applications sold as a service by service providers is VPN. Before we
discuss MPLS VPN, we will first discuss what a VPN is?

For example, as a service provider, we have 3 routers, 2 Provider Edge (PE) routers, each of which is
connected to the customer and 1 core which is usually called the P router. Virtual private network
(VPN), as the name suggests "private network" means as if the customer has a private network.
Physically, this network infrastructure is shared with other customers, but virtually the traffic does
not mix with other customers. There are two types of VPN, the first is overlay VPN, where the
provider is not involved in customer routing. for example, frame relay switches.

Overlay VPN

This frame relay switch is an L2VPN service that uses frame relay. In services like this, the provider
provides virtual circuits as a substitute for dedicated point to point. it's different compared to a
 2

leased line, which we actually rent a point to point physical link, this frame relay is merely a virtual
circuit, it just emulates a point to point leased line connection. In the example above, even though it
looks like CE1 and CE2 are physically connected, because the Customer only rent VC 1 and VC2,
CE1 and CE2 cannot communicate directly, CE1 needs to go through CE3 to get to CE2, or another
way could be by the customer renting a new virtual dedicated circuit that connects CE1 and CE2.
So this is the VPN overlay, the provider really doesn't participate in routing between customers, the
provider only provides a virtual circuit to connect one site to another site. Then the second type of
VPN is peer to peer VPN.

Peer to Peer VPN

This VPN is quite different compared to an overlay VPN, in peer to peer VPN, the provider is involved
in routing the customer. We can see in the example below that between the customer and the
provider edge (PE) using a routing protocol. For example, if we use OSPF, the OSPF neighbor is
between the customer edge (CE) and the nearest PE. So later when an edge customer sends a
routing update, he sends a routing update to the edge provider, then this edge provider forwards the
routing update to another PE, until it is forwarded to another edge customer. So this routing update
is important so that the inter-ce can communicate and will automatically choose the best or
shortest path to each destination. for example from site one to site two, then it will choose the
shortest route from PE1 to PE2 so you don't need to go around to PE3 first. So between site 1 site 2
site 3, because routing protocol already installed, it will go through the shortest path between
sites. This MPLS L3VPN is categorized a peer to peer VPN.
 3

VRF And MPLS L3VPN Introduction

Here we have 2 different customers, namely customer A and customer B. Each has a different VPN,
so their traffic cannot be mixed, A cannot cross to B and vice versa. So to save resources, instead of
making a dedicated router, in the sense that customer A is provided with its own PE, customer B is
provided with its own PE, that would be too expensive, so that customer A and customer B, even
customers C, D and so on, can connect to the same PE and the traffic of one customer and another
customer is not mixed, then a virtual route and forwarding (vrf) is introduced, or simply put, a virtual
routing table. So for each customer we create a separate routing table, this is the vrf for customer A,
then for customer B we also create a separate routing table, so in the PE we will create a vrf for a
number of customers who are connected to that PE.

So if in this example, in PE1 and PE2 there are customers A and B, then two VRFs are created and
there will be 2 VRF routing tables for customers A and B. So in PE1 and PE2 there are 3 routing
tables, 2 VRF routing tables and 1 global routing table that we usually check using show ip route
command. This global routing table contains the service provider's network, like how to get to
 4

another PE, and there is no correlation with the customer network. while the customer network is in
VRF(different routing table).

How VRF filled its Routing Table.

For example, customer A has network 10.1.1.0/24. Customer A will send a routing update. Well,
later on PE1 it will record in customer A's vrf routing table, it will get a routing update containing
network 10.1.1.0/24 from customer A1. The RU obtained by PE1 will be given to PE2, so in PE2 there
must be vrf cust A too, which will also record 10.1.1.0/24 in the vrf cust A routing table and then this
routing update will be forwarded to customer A2 so that the router A2 knows to go to 10.1.1.0/24 via
PE2. Here the P router does not need to know the routing to the customer network, the P router only
needs to load the global routing table which only contains the provider's network. for example, how
to go to PE1 or PE2, and the advantage is that we can save memory or CPU resources on the P
router. P routers only focus on connecting one PE to another. The customer network is the provider
edge (PE) business. So the routing table of the P router is simpler. Then how can router P forward
the routing updates it receives from PE while it doesn't know the destination to the customer
network? This is where MPLS comes into play. Without MPLS, when the P router receives a packet
to a destination address that does not have a routing table, the packet will be dropped, but thanks
to MPLS which makes decisions based on labels, this does not happen. We will discuss the details
later. Now the question is whether B1 can also use IP 10.1.1.0/24 or not? This IP conflicts or
 5

overlaps with the IP owned by customer A. The answer is yes, because it's a different VRF.

When B sends a routing update, PE1 installs 10.1.1.0/24 into cust B's vrf routing table and this will
not conflict due to different vrfs. So vrf A and vrf B may have the same address. this is also the
advantage of MPLS L3VPN. The customer is flexible, he can use the IP he wants even if the IP is
already used by other customers. so it's not rigid, for example if it's not allowed, it's not good,
customer A used 10.1.1.0/24 first, customer B also used the 10.1.1.0/24 network and wanted to
subscribe to VPN with this provider, but B was told to change. “Wow, sorry sir, you can't use
10.1.1.0/24, you have to replace it because 10.1.1.0/24 is already being used by another customer”.
But thanks to VRF, this is possible, because VRF separates one routing table from another routing
table. Create a virtual router inside a physical router. So it can be said that the physical router has
its own routing table which is called the global routing table and the VRF has its own routing table,
and these routing tables are not related to each other.

BGP role in MPLS L3VPN.

Now how does PE1 inform about routing updates to other PEs?

Suppose we have a customer edge A1 that sends routing updates to PE1. In a PE there could be a
lot of vrfs, for example, if it has a lot of customers then there are a lot of vrfs in that PE. So the PE
 6

must inform PE2 of all routing in this vrf. PE2 requires a routing protocol that must be able to
support a lot of routing updates, even if there are thousands or hundreds of thousands of prefixes.
And secondly, it must be able to send updates to a router that is far away, meaning not to the router
next to it, because a PE and another PE are not always next to each other. And the routing protocol
that can overcome the above problem is BGP. BGP is a routing protocol that has been proven to be
able to carry even a large number of prefixes, on the internet now for IP version 4, around 800,000
more prefixes can be sent using BGP. then BGP also supports indirect neighbors, which means it
doesn't directly connect to our side. so between PE1 and PE2 using BGP. Even though PE1 goes to
PE2 via the P router, the P router does not need to use BGP, because BGP is used to carry the
customer network on the left (PE1) to the one on the right (PE2). So those who need to use BGP are
PE1 and PE2. For PE1-P -PE2 you only need to use IGP for routing between nodes so that loopbacks
can be reached and add LDP to exchange labels so that MPLS works.

How does BGP send updates containing the same 2 prefixes from different sources?

Route Distinguisher (RD)

Now let's see what updates BGP brings. From the previous example, it is said that vrf A has the
prefix 10.1.1.0/24, vrf B has 10.1.1.0/24. The two prefixes from each vrf are included in 1 BGP
update. So how does BGP differentiate between A's 10.1.1.0/24 address and B's 10.1.1.0/24
address? There is an additional attribute that makes the address prefix unique, its name is Route
Distinguisher or RD. so this RD is 64 bits long, used as a differentiator for VPN addresses. So the
customer's address is differentiated using RD.

RD Format

RD Writing format (X:X) often divided into two parts like this

RD number format RD
IP address:X 192.168.1.11:3001
AS Number:X 65123:3001

X here is just a number parameter. The RD in each vrf must be unique. So each VRF must have a
different route distinguisher. For example, for vrf A we give RD 192.168.1.11:3001, while for vrf B
192.168.1.11:3002 (assuming 192.168.1.11 is the loopback IP of router PE1).
 7

Then this RD will be combined with the IP address to construct the VPNv4 address. let's look at the
table below

VPNv4 address = RD+Ip address

Customer VPNV4 Address

A 192.168.1.11:3001+10.1.1.0
B 192.168.1.11:3002+10.1.1.0

RD is 64 bits long, while IPv4 is 32 bits. We see the IP is the same as 10.1.1.0, but because of the RD
route distinguisher which must be unique, this causes the address to be unique. This combination
of RD and IPv4 will become a new address called VPNv4 address which is 96 bits long. this is what
is included in the BGP update. So in BGP the update from one PE to another PE is a VPN version 4
address, so it's not a regular IP address, but an IP address that has been added with an RD that we
have to configure manually.

Tips for configuring RD

1. 1 VRF with 1 same RD on all PE.

RD PE1 RD PE2
VRF A 65123:3001 65123:3001
VRF B 65123:3002 65123:3002

The advantage is simplicity, you don't need too many numbers, so 1 vrf uses the same 1RD in all
PEs. For example, vrf a in any PE, the RD is this, but we cannot quickly see which vrf A customer is
in which PE.

2. 1 vrf with different RD for every PE.

RD PE1 RD PE2
VRF A 192.168.1.11:3001 192.168.1.12:3001
VRF B 192.168.1.11:3002 192.168.1.12:3002

If it's like this, the advantage is that when we look at an address, we see the RD in front of it is
192.168.1.11, we can easily know immediately that this customer's address is only on PE1. And if
the front RD is 192.168.1.12, we can easily know that this is a customer network that connects
directly to PE2.
 8

Route Target (RT)

The format for this route target is usually AS number: number. Even though the writing format is the
same as RD, RT and RD have different functions. The route distinguisher is used to differentiate the
customer's VPN address, whereas the route target is used as a VPN membership.

Here we want A1 and A2 to be one VPN and we give it the name VPN A. To indicate which VPN this
network is in, we assign the route target. We enter the route target for VPN A 65123:3001.

When PE1 receives a routing update from CE A1, PE1 will insert the RU into VRF A, then PE1
forwards the RU from the vrf it owns using MP-BGP in the form of a VPNv4 address (RD+IP address)
to PE2.

Because PE1 here is the MP-BGP update sender, PE1 needs to add an export route target
configuration, its function is to notify PE2, if the update starts with RT 65123:300, then this is VPN
A's network or customer A's. So that PE2 Also know, if the update starting with RT 65123:3001
 9

belongs to customer A, then on the PE2 side it is necessary to configure the same import route
target. After that, PE2 can enter the updates it receives into the appropriate vrf, namely vrf A. So the
target route exported by the sender and the one imported on the receiving side must match. if it
matches, then network 10.1.1.0 will be included in vrf table A, but if it doesn't match it will not be
installed in vrf table A. PE2 then forwards this update to customer A without a route distinguisher or
route target, just a normal IP address like IGP routing updates in general.

For example, there is a new customer who wants to subscribe to a VPN service provider, we give the
name customer B, customer B also has network 10.1.1.0/24 which is sent to PE1.

Here we need to define a new route target for VPN B that is different from the route target for VPN A
so that traffic between customers is not mixed. When B1 sends an update to PE1, PE1 will install
the RU from customer B1 into vrf B, PE1 must configure the export route target and then PE1 can
send the update using MP-BGP, in PE2 the condition is that it can install the prefix 10.1.1.0
belonging to customer B into VRF B's routing table, as long as the import RT on PE2 is the same as
PE1.
 10

VPN Label

there is one more information that is sent from one PE to another PE, namely the VPN label. So
there will be two labels used in MPLS L3VPN.

Let's look at the traffic flow when CE A2 wants to send a packet to the destination IP 10.1.1.1
without using a VPN Label (only a label from LDP).

When customer A2 sends a packet to PE2 with destination 10.1.1.1, PE2 gets information from VRF
A's routing table, to go to 10.1.1.0/24 via PE1. The information in the VRF A routing table received
from MP-BGP does not tell PE2 where PE1 is located. But because PE1-P1-P2-PE2 are connected to
each other with IGP and MPLS LDP has been configured, PE2 can know how to get to PE1 by looking
at the FIB (show ip cef). Here PE1 has a loopback IP 192.168.1.11. The LSR or label switch router is
tasked with generating each label to represent the network that will be advertised to LSR neighbors
 11

using LDP.

When PE2 receives a packet with destination 10.1.1.1, PE2 will see the FIB (because it received the
IP packet), PE2 will see it going to PE1 or 192.168.1.11, the nexthop is P2 and the nexthop label (NH
L) is 200, then PE2 will impose a label on the packet with L=200 and forward it to P2. Then P2
switches packets based on the label by looking at the LFIB, when it gets a labeled packet with
L=200, then the nexthop is P1 and the nexthop label is 100, then P2 swaps the label L=200 to L=100
and forwards the packet to P1. Then P1 will also see the LFIB, when it receives a labeled packet with
L=100, what it does is pop the label and forward the packet to PE1. After the packet is received by
PE1, PE1 will see that the destination address is 10.1.1.1, while on PE1 there are 2 customers who
use the IP subnet 10.1.1.0, so PE1 will have difficulty determining whether this packet is directed to
customer A or B. This is why LDP labels alone are not enough to facilitate L3VPN's MPLS service.
Therefore, there is a need for an additional label, namely the VPN label.

End to End MPLS L3VPN Traffic flow

Label Protocol Function

LDP label LDP Notifies the destination to the egress PE
VPN label MP-BGP Notifies the existence of the VPN client
 12

Here we will see end to end traffic flow from Customer A2 to A1. When PE1 sends an MP-BGP
update, it will also include VPN label information for each customer. In the example above, the VPN
label for customer A's 10.1.1.0 is 1001, and customer B's 10.1.1.0 is 1002. When PE2 receives a bgp
update from PE1, PE2 will also receive the VPN label information and will pair the VPN label with
the appropriate VRF. We see here, when CE A2 sends a packet with destination 10.1.1.1 to PE2, PE2
receives the packet on the interface assigned to vrf A, then PE2 will encapsulate the packet with
VPN label A=1001, then PE2 sees vrf A destination to 10.1.1.0 via PE1. PE2 checks the FIB (show ip
cef), to go to PE1 or 192.168.1.11, the nexthop is P2 with nexthop label=200. PE2 then imposes a
second label, namely LDP label L=200 and forwards the labeled packet to P2. There are 2 labels
assigned to the packet, the front label is the LDP label for MPLS purposes, and the VPN label for
VPN client identification. So when PE2 sends a packet to P2, P2 only reads the top label, and
switches labels based on LFIB. When the packet arrives at PE1, PE1 can determine which customer
this packet was sent to because it still has VPN label=1001. So PE1 will forward the packet to CE
A1. So we have seen the control plane that works in the topology above, there are IGP, lDP, Multi
Protocols BGP. The P router control plane only has IGP and LDP, while in each PE, the control plane
has IGP, LDP and MP-BGP.
 13

Cisco MPLS L3VPN Lab.

System image file : vios-adventerprisek9-m

Configuration steps

1. Prepare the MPLS backbone by configuring IGP on PE1, P and PE2. Make sure the loopback is
reachable.

PE1 P PE2

conf t conf t conf t

router ospf 100 router ospf 100 router ospf 100
interface g0/0 interface g0/0 interface g0/0
ip ospf 100 area 0 ip ospf 100 area 0 ip ospf 100 area 0
int lo0 interface g0/1 int lo0
ip ospf 100 area 0 ip ospf 100 area 0 ip ospf 100 area 0
end int lo0 end
ip ospf 100 area 0
end

Configure MPLS on each router.

PE1 P PE2
conf t conf t conf t
interface g0/0 interface g0/0 interface g0/0
mpls ip mpls ip mpls ip
end interface g0/1 end
mpls ip
end
 14

Verify MPLS backbone command

-show mpls ldp discovery > LDP neighbor

-show mpls ldp binding > see the LIB

-show mpls forwarding-table > see the LFIB

-show ip cef x.x.x.x/SM detail > see the FIB

Lets check LDP neighbor from PE2.

We can see that the MPLS LDP neighbor of PE2 is 2.2.2.2, namely P router. And there is PE2's router
ID information, namely 3.3.3.3. so PE2 and P have become LDP neighbors.

Then we can check the LIB which contains the local label and nexthop label for each prefix.

Let's take the prefix 1.1.1.1/32 as an example. Local label=17 and nexthop label=16 nexthop LSR is
2.2.2.2 or P router.

Lets check the LFIB

 15

Here it is shown, for the prefix 1.1.1.1/32 the local label=17 nh label=16, the outgoing interface is
via G0/0 and the nexthop is 100.1.2.1 (ip address physical interface P router which leads to PE2).

Lets check the FIB

Here we can also see the prefix 1.1.1.1/32 local label (global) = 17, nh label=16, nexthop is
100.1.2.1 via G0/0.

We assume here that when PE2 receives an IP packet with destination 1.1.1.1, what is checked is
the FIB (ip cef x.x.x.x/SM detail).

We try moving to router P and verify as in PE2.

16
 17

On the P router, if we check the LFIB (mpls forwarding-table). We can see that when router P
receives a labeled packet L=16, what it does (outgoing interface) is pop the label. The P router
performs a pop label for the destination packet 1.1.1.1/32, because the P router here is exactly 1
hop before the egress LSR, namely PE1 (the router which has the address 1.1.1.1/32).

Lets check at PE1

 18

Because PE1 receives an IP packet (without a label), PE1 will check the FIB, 1.1.1.1/32, the status is
attached, connected, it is in lo0. Then PE1 will process the packet.

After we have succeeded in ensuring that the MPLS backbone is running, next we will configure MP-
BGP between PE1 and PE2.

BGP configuration

PE1 PE2
router bgp 65123 router bgp 65123
bgp log-neighbor-changes bgp log-neighbor-changes
no bgp default ipv4-unicast no bgp default ipv4-unicast
neighbor 3.3.3.3 remote-as 65123 neighbor 1.1.1.1 remote-as 65123
neighbor 3.3.3.3 update-source Loopback0 neighbor 1.1.1.1 update-source Loopback0
address-family vpnv4 address-family vpnv4
neighbor 3.3.3.3 activate neighbor 1.1.1.1 activate
address-family ipv4 vrf VRF-CE-A address-family ipv4 vrf VRF-CE-A
redistribute ospf 1 redistribute ospf 1
exit-address-family exit-address-family
! !
address-family ipv4 vrf VRF-CE-B address-family ipv4 vrf VRF-CE-B
 19

redistribute ospf 2 redistribute ospf 2

exit-address-family exit-address-family

Verify bgp neighbor adjacency between PE1 and PE2.

On PE1 and PE2 you can see each BGP router ID and its neighbors (1.1.1.1 belongs to PE1 and
3.3.3.3 belongs to PE2). The state/prefixrecord (pfxrcd) is also a number, meaning that the BGP
neighbor adjacency between these two routers has been established.
 20

Here we can see the BGP address family VPNV4 unicast, the BGP neighbor ID is there and the state
is established.

The next step we will define VRF for each customer, namely customer A and B.

PE1 PE2
ip vrf VRF-CE-A ip vrf VRF-CE-A
rd 1.1.1.1:10112 rd 3.3.3.3:10116
route-target export 65123:11 route-target export 65123:11
route-target import 65123:11 route-target import 65123:11

int g0/1 int g0/1

desc to CE A1 desc to CE A2
ip vrf forwarding VRF-CE-A ip vrf forwarding VRF-CE-A
ip address 10.1.1.1 255.255.255.252 ip address 10.1.1.5 255.255.255.252
no shutdown no shutdown
exit exit

ip vrf VRF-CE-B ip vrf VRF-CE-B

rd 1.1.1.1:10222 rd 3.3.3.3:10226
route-target export 65123:12 route-target export 65123:12
route-target import 65123:12 route-target import 65123:12

int g0/2 int g0/2

desc to CE B1 desc to CE B2
ip vrf forwarding VRF-CE-B ip vrf forwarding VRF-CE-B
ip address 10.1.1.1 255.255.255.252 ip address 10.1.1.5 255.255.255.252
no shutdown no shutdown
exit exit

Verify VRF

-show vrf brief

-show vrf detail

 21

After setting up the L3VPN backbone, we configure it for each client and make sure the p2p ping is
successful.

CE A1

CE B1

CE A2
 22

CE B2

OSPF configuration on both Customer

A1 A2
conf t conf t
int g0/0 int g0/0
ip ospf 1 area 0 ip ospf 1 area 0
int lo0 int lo0
ip ospf 1 area 0 ip ospf 1 area 0

In this lab, the routing protocol OSPF is used between customer A and PE. So OSPF routes between
A1 and PE1 need to be redistributed to BGP PE1 and PE2. BGP routes from PE1 and PE2 need to be
 23

redistributed to OSPF between PE2 and A2. And vice versa.

Now let's configure OSPF for VRF customer A in the Provider and redistribute BGP to OSPF, then
redistribute OSPF to BGP.

PE1 PE2
conf t conf t
router ospf 1 vrf VRF-CE-A router ospf 1 vrf VRF-CE-A
redistribute bgp 65123 redistribute bgp 65123
network 10.1.1.0 0.0.0.3 area 0 network 10.1.1.4 0.0.0.3 area 0
router bgp 65123 router bgp 65123
address-family ipv4 vrf VRF-CE-A address-family ipv4 vrf VRF-CE-A
redistribute ospf 1 redistribute ospf 1
exit exit

If we now verify on PE1, there will be a route passed via BGP as a result of the redistribution, which
we can see in the state/pfxrcd column
 24

If we take a look at A1

The route to customer A2's loopback IP is visible in A1's routing table. The ping and traceroute
results also indicate that the loopback IP A2 is reachable from A1.

The next case example in this lab, customer B is connected to the Provider with eBGP.
 25

Configure eBGP at Customer B

B1 B2
conf conf
router bgp 64512 router bgp 65022
neighbor 10.1.1.1 remote-as 65123 neighbor 10.1.1.5 remote-as 65123
network 10.0.0.5 mask 255.255.255.255 network 10.0.0.6 mask 255.255.255.255

Configure eBGP at PE1 and PE2.

PE1 PE2
conf t conf t
router bgp 65123 router bgp 65123
address-family ipv4 vrf VRF-CE-B no address-family ipv4 vrf VRF-CE-A
neighbor 10.1.1.2 remote-as 64512 address-family ipv4 vrf VRF-CE-B
neighbor 10.1.1.6 remote-as 65022

We can check the routing table B1 and B2.

B1 has received prefix 10.0.0.6 (B2's loopback ip) from eBGP. We can check

The flag for 10.0.0.6/32 is valid and best (*>), it has no i (internal), meaning the update comes from
external bgp.
 26

From B1 to go to loopback B2, must go through the p2p link between PE2 and B2, which is
10.1.1.4/30, meanwhile we don't find this prefix in B1's routing table. Likewise on B2, when you
want to go to loopback B1, you have to go through the p2p link between PE1 and B1, while we check
on B2, there is no route to link 10.1.1.0/30.

If we check the ping from B1 to B2, without providing a specific source, it will fail.

This is because when B1 sends ICMP packets to 10.0.0.6, the source IP address is 10.1.1.2. while
on the B2 side, it doesn't know the route to 10.1.1.0/30. So we can try to ping and traceroute using
source B1's loopback IP (to replace the source IP address which previously used the physical
interface IP address, and now the source is replaced by using B1's loopback IP whose route B2
already knows).
 27

So that both the IP on the physical interface and the loopback IP of B1 and B2 are reachable without
providing a specific source, we can redistribute the connected network on PE1 and PE2 to bgp
65123.

PE1 PE2
conf t conf t
router bgp 65123 router bgp 65123
address-family ipv4 vrf VRF-CE-B address-family ipv4 vrf VRF-CE-B
redistribute connected redistribute connected

Now we can check

 28

B1 gets a new route to 10.1.1.4/30 from eBGP. We checked ping to IP 10.1.1.6 and 10.0.0.6 without
specific source, and it was successful.

Here we can conclude that 2 customers who use the same IP address when connecting to the
provider can still connect to the remote customer site when using the MPLS L3VPN service.