Methods of steering traffic into MPLS and Segment Routing LSP is one of the least standardized and most confusing parts of traffic engineering.
The mess of nexthop resolution
Despite some existing interoperability issues, in general, the MPLS and SR control plane is fairly well standardized and works more or less the same way across implementations.
The same cannot be said about traffic steering into MPLS or SR. RFCs don’t regulate any of this and every vendor has their own nexthop resolution logic, configuration options and default settings. This creates a lot of interesting challenges for network engineers trying to make it work in a multivendor environment, or transitioning from one vendor to another.
Describing this in detail is an opportunity to write an entire book so I’ll just mention a few examples.
Cisco IOS-XE/XR
When you enable LDP on Cisco IOS-XE or IOS-XR, they by default advertise label mappings for all IGP prefixes. So all traffic, even between router loopbacks, will be labeled. Likewise, when you enable SR, it will install label bindings for all IPs that have prefix SID.
When both LDP and SR are enabled, LDP is preferred, but there is a command “sr-prefer” to use SR labels instead of LDP.
By default, Cisco does not resolve any routes via RSVP-TE LSP, but there is a command “autoroute announce” which make a particular RSVP-TE LSP eligible for nexthop resolution, and preferred before other protocols.
One difference between IOS-XE and IOS-XR is that XR prefers BGP-LU labels over LDP.
Juniper JUNOS
On JUNOS, MPLS LSP are not used to forward all traffic between routers, instead, they are installed in the special inet.3 routing table which BGP uses for nexthop resolution. So traffic between router loopbacks will be unlabeled, but BGP or MPLS VPN between the same 2 routes will be resolved via whatever MPLS LSP available in inet.3
There is a document explaining route preference, e.g. RSVP-TE LSP is preferred before SR-TE, which is preferred before LDP, then OSPF-SR, ISIS-SR etc.
You can also create custom tables using RIB groups on JUNOS and configure BGP to resolve via a specific RIB on some condition.
Arista EOS
EOS in this particular regard is very similar to JUNOS, just with a Cisco-like config syntax. There are tunnel RIBs which BGP uses for nexthop resolution.
Since Arista docs are hidden behind paywall, I’ll paste one table here:
| Source protocol | System-defined preference |
|---|---|
| Static | 15 |
| Nexthop group | 25 |
| RSVP LER | 45 |
| LDP | 55 |
| IS-IS SR | 65 |
| BGP-LU | 85 |
The order of preferences is the same as on JUNOS. Likewise, you can create custom RIBs.
IP Infusion OcNOS
Similarly to Juniper and Arista, IP Infusion uses MPLS LSP only for BGP nexthop resolution and not for all traffic, however, similar to Cisco, they have the “sr-prefer” command to prefer SR over LDP. As for RSVP-TE, there is a special command “map-route” to make a certain prefix prefer RSVP-TE LSP.
As I said before, this is by no means exhaustive and I’m sure there are other vendors with their own unique approaches to nexthop resolution.
The point of all of the above is not to say that vendor A does it right and vendor B does it wrong, but to illustrate how messy and confusing all of this is.
SR color steering
The idea behind Segment Routing is to make MPLS simple and scalable. This does not only concern label distribution and traffic engineering, but also steering traffic into SR-TE policies.
Each SR-TE policy has a color which is just a numerical value, and BGP routes which match the nexthop and the color extended community, are mapped to the relevant SR-TE policy.
In the example below, R8 advertises an MPLS VPN route with color 101, and R1 matches it with an SR-TE policy with endpoint 8.8.8.8 and color 101. So R1 steers traffic into the SR-TE policy.
There are at least 2 big advantages of this approach:
- It’s possible to map different services on the same router to different SR-TE policies
- The color steering concept is standardized and should work the same way across all vendors
Another advantage is color-only steering which allows the router to ignore nexthop and steer traffic into an SR-TE policy using only color.
In the example below, R1 maps a BGP route with color 101 to a null-endpoint policy. Traffic Dictator, used as a controller, interprets a null-endpoint policy as Egress Peer Engineering (EPE) policy to the closest egress peer that matches affinity and bandwidth constraints.
Poor man’s solution to color steering
With any new technology, no matter how good it is, there is a period of adoption. Not all vendors implement the new standard quickly, and not everything related to color steering is even standardized yet. For example, colored SR-TE policy advertisement in PCEP is still a draft and is not supported by all PCEP implementations. BGP SR-TE has not yet been standardized either.
Let’s take a look at a solution that does not require any new protocols but allows to map different services on the same endpoint to different SR-TE policies. It works as follows:
- Endpoint (R8) has multiple “service loopbacks” that correspond to different services, just like SR-TE colors. Nexthop for service routes (BGP, MPLS VPN, EVPN etc) is changed to the relevant service loopback
- Service loopbacks are never advertised into IGP, but advertised into BGP-LU with low LOCAL_PREF and the main loopback (8.8.8.8) as a nexthop
Example from R8 (Arista EOS):
interface Loopback101 ip address 8.1.0.1/32 ! router bgp 65002 router-id 8.8.8.8 no bgp default ipv4-unicast neighbor 1.1.1.1 remote-as 65002 neighbor 1.1.1.1 update-source Loopback0 ! address-family ipv4 labeled-unicast neighbor 1.1.1.1 activate network 8.1.0.1/32
So the resolution on R1 looks as follow:
R1#sh ip bgp labeled-unicast | grep 8.1.0.1 * > L 8.1.0.1/32 8.8.8.8 0 - 100 0 i
R1#sh bgp labeled-unicast tunnel | grep 8.1.0.1 2 8.1.0.1/32 IS-IS SR IPv4 (11) - [ 3 ] Yes 0 MED 0 200 0
R1#show isis segment-routing tunnel 11 Index Endpoint Next Hop/Tunnel Index Interface Labels ----------- ---------------- --------------------------- --------------- ---------- 11 8.8.8.8/32 10.100.2.4 Ethernet2 [ 900008 ] 10.100.3.5 Ethernet3 [ 900008 ]
The nexthop resolution logic illustrated:
3. The SDN controller calculates policies towards the main loopback (8.8.8.8) which is advertised in IGP, uses its prefix SID for segment list resolution, but sends policies via BGP-LU using service loopback as a destination.
Policy config and outputs on Traffic Dictator:
TD1#sh run | sec R1_R8_BLUE_ONLY_IPV4 policy R1_R8_BLUE_ONLY_IPV4 headend 1.1.1.1 topology-id 101 endpoint 8.8.8.8 service-loopback 8.1.0.1 priority 7 7 install direct labeled-unicast 192.168.0.101 ! candidate-path preference 100 metric igp affinity-set BLUE_ONLY bandwidth 100 mbps
TD1#show traffic-eng policy R1_R8_BLUE_ONLY_IPV4 detail Detailed traffic-eng policy information: Traffic engineering policy "R1_R8_BLUE_ONLY_IPV4" Valid config, Active Headend 1.1.1.1, topology-id 101, Maximum SID depth: 6 Endpoint 8.8.8.8, service-loopback 8.1.0.1 Endpoint type: Node, Topology-id: 101, Protocol: isis, Router-id: 0008.0008.0008.00 Setup priority: 7, Hold priority: 7 Reserved bandwidth bps: 100000000 Install direct, protocol labeled-unicast, peer 192.168.0.101 Policy index: 10, SR-TE distinguisher: 16777226 Candidate paths: Candidate-path preference 100 Path config valid Metric: igp Path-option: dynamic Affinity-set: BLUE_ONLY Constraint: include-all List: ['BLUE'] Value: 0x1 This path is currently active Calculation results: Aggregate metric: 200 Topologies: ['101'] Segment lists: [900008] BGP-LU next-hop: 10.100.0.2 Policy statistics: Last config update: 2025-02-18 09:37:27,208 Last recalculation: 2025-02-18 09:37:27.517 Policy calculation took 0 miliseconds
Outputs on R1 now:
R1#show ip bgp labeled-unicast | grep 8.1.0.1
* > L 8.1.0.1/32 10.100.0.2 0 - 500 0 65001 i
* L 8.1.0.1/32 8.8.8.8 0 - 100 0 i
R1#sh bgp labeled-unicast tunnel | grep 8.1.0.1
2 8.1.0.1/32 10.100.0.2 Ethernet3 [ 900008 ] Yes 0 MED 0 200 0
The nexthop resolution logic illustrated:
The result is that traffic for different services can be mapped to different policies, and if some of the policies fail (e.g. due to a failed constraint) or the controller fails, traffic is just resolved over the IGP shortest path.
Another advantage of this approach is that we don’t have to deal with any of the nexthop resolution nonsense I mentioned in the beginning of this post. The resolution order is controlled using BGP LOCAL_PREF which works the same way across all implementations.
Service loopbacks and RSVP-TE LSP
Nothing of this is new. The book MPLS in the SDN era covers the concept of signaling RSVP-TE LSP to different loopbacks on JUNOS (see pages 170-176).
The problem is that unlike the SR solution with BGP-LU, signaling RSVP-TE LSP to different loopbacks is not a standard solution and does not work on all implementations.
When you configure a regular MPLS-TE LSP on a router, there are 2 steps to provision it:
- Calculate CSPF using the configured constraints – this gives us the Explicit Route Object (ERO)
- Signal the RSVP-TE LSP using the ERO that was calculated in step (1)
Both CSPF calculation and RSVP-TE signaling should be done towards the MPLS-TE router ID of the endpoint. On JUNOS, there is an “inter-domain” knob that allows to relax this rule. Furthermore, this rule applies just to step (1), so if the CSPF calculation is done on the controller, it’s possible to signal RSVP-TE LSP to secondary loopbacks. It works with Juniper and IP Infusion. Unfortunately this doesn’t work with Cisco IOS-XR because it refuses to signal an RSVP-TE LSP to any IP other than the MPLS-TE router ID.
Traffic Dictator supports RSVP-TE policies with service loopbacks.
Conclusion
The long term industry solution for traffic steering is color steering. There is effort to bring color advertisement to RSVP-TE and PCEP, so hopefully at some point it will work for all sorts of traffic engineering.
However, in practice we have to deal with all kinds of imperfect solutions and somehow make them work. BGP with multiple service loopbacks is a decent standard-based solution that consistently works with most MPLS and SR implementations.
References
- MPLS in the SDN Era: Interoperable Scenarios to Make Networks Scale to New Services – Antonio Sanchez Monge, Krzysztof Grzegorz Szarkowicz, ISBN-13: 978-149190545 – https://www.amazon.co.uk/MPLS-SDN-Era-Interoperable-Scenarios/dp/149190545X
- Poor man’s Traffic Engineering – https://routingcraft.net/poor-mans-traffic-engineering/
- RSVP-TE policies – https://vegvisir.ie/rsvp-te-policies/
- Understanding Route Preference Values (Administrative Distance) – https://www.juniper.net/documentation/us/en/software/junos/routing-overview/bgp/topics/concept/routing-protocols-default-route-preference-values.html
- Path Computation Element Communication Protocol (PCEP) Extensions for Segment Routing (SR) Policy Candidate Paths – https://datatracker.ietf.org/doc/html/draft-ietf-pce-segment-routing-policy-cp-21
- Advertising Segment Routing Policies in BGP – https://datatracker.ietf.org/doc/html/draft-ietf-idr-segment-routing-te-policy-26
- Path Computation Element Protocol (PCEP) Extension for Color – https://datatracker.ietf.org/doc/html/draft-ietf-pce-pcep-color-11