Category: Load Balancing

Establishing parallel EBGP sessions across parallel links between two edge routers (EBGP peers) – as displayed in the diagram below – is the most versatile form of EBGP load balancing. It does not require static routing or extra routing protocol (like the design running EBGP between routers’ loopback interfaces), device-specific tricks like configuring the same IP address on multiple interfaces) or specific layer-2 encapsulation (like Ethernet LAG or Multilink PPP).

It even allows proportional load-balancing across unequal-bandwidth links and combinations of various layer-2 technologies (for example, load-balancing between a serial line and an Ethernet interface). The only drawback of this design is the increased size of the BGP table, as every BGP prefix is received from the EBGP neighbor twice.

EIGRP was always described as the only routing protocol that can do unequal-cost load sharing. As it turns out, BGP is another one (although it's way more limited than EIGRP). For example, if you have two links into a neighbor AS, you can load-share across them proportionally to their bandwidth.

EBGP load balancing was introduced with the BGP 4 Multipath Support feature in IOS release 11.2. Initially, EBGP supported up to six maximum paths; IOS release 12.0(S) increased that value to 8, IOS release 12.3T to 16 and 12.2S (including 12.2SRC) to 16.

Unequal load-sharing with static routes is almost impossible as there is no configuration command to assign non-default traffic share count to a static route. For example, if you configure two default routes, one pointing to a low-speed interface and another one pointing to a high-speed interface, there is no mechanism to force majority of the traffic onto the high-speed link (IOS ignores interface bandwidth when calculating load sharing ratios).

You can, howerer, use a workaround: if you configure multiple routes for the same prefix pointing to the same interface, that interface will attract proportionally more outbound traffic.

One of the most commonly asked load-sharing-related questions is “can I load-share traffic across unequal-cost links?”. In general, the answer is no. In order to load-share the traffic, you need more than one path to the destination and the only way to get multiple routes toward a destination in the IP routing table is to make them equal-cost (the only notable exception being EIGRP that supports unequal-cost load-sharing with the variance parameter).

There are, however, two cases where you can force unequal traffic split across equal-cost paths toward a destination: when using inter-AS BGP with the link bandwidth parameter, and when using unequal-bandwidth traffic-engineering tunnels.

In designs with very low number of IP hosts, no per-destination load-sharing algorithm will work adequately. Consider, for example, an extranet design where a large number of IP hosts are NAT-ed to a single IP address which then accesses a single remote server.

In this design, all the traffic flows between a single pair of IP addresses, making per-destination load-sharing unusable.

Cisco Express Forwarding (CEF) can perform per-packet or per-destination (actually source/destination IP address pair) load-sharing with no performance degradation (without CEF, per-packet load-sharing requires process switching).

Even though there is no performance impact on the router, per-packet load sharing will almost always result in out-of-order packets. The packet reordering might degrade TCP throughput in high-speed environments (in low-speed/few-flows scenarios, per-packet load-sharing actually improves the per-flow throughput) or severely impact applications that cannot survive out-of-order packet delivery, such as Fast Sequenced Transport for SNA over IP or voice/video streams.

In environments with a low number of IP hosts you have to fine-tune the CEF load-sharing algorithm to ensure that the traffic is spread between all parallel paths. A typical scenario is a primary-backup data center setup with pairs of replicating servers, as shown in the figure below.

In these cases, you have to try different values of seed parameter of the CEF universal algorithm.

According to the Cisco IOS documentation, you can select between the original and the universal CEF load sharing algorithm with the ip cef load-sharing algorithm name parameter global configuration command (we'll leave the tunnel algorithm aside for the moment). Of course, they don't tell you what you select.

I had to investigate the details of CEF load sharing for one of my upcoming article and found (yet again) that the details are rather undocumented in official documentation. So, this is how it works (in case you ever need to know):