Category: CEF

One of my readers sent me an interesting question a while ago:

I reviewed one of your blog posts "Per-Destination or Per Packet CEF Load Sharing?" and wondered if you had investigated previously on how MQC QoS worked together with the CEF load-sharing algorithm (or does it interact at all)? For example, let's say I have two equal cost paths between two routers and the routing table (as well as CEF) sees both links as equal paths to the networks behind each router. On each link I have the same outbound service policy applied with a simple LLQ, BW, and a class-default queues. Does CEF check each IP flow and make sure both link's LLQ and BW queues are evenly used?

Unfortunately, packet forwarding and QoS are completely uncoupled in Cisco IOS. CEF performs its load balancing algorithm purely on source/destination information and does not take in account the actual utilization of outbound interfaces. If you have bad luck, most of the traffic ends on one of the links and the packets that would easily fit on the other link will be dropped by the QoS mechanisms.

When I was discussing the details of the BGP troubleshooting video with one of my readers, he pointed out that I should mention the need for CEF switching in EBGP multipath scenario. My initial response was “Why would you need CEF? EBGP multipath is older than CEF” and his answer told me I should turn on my gray cells before responding to emails: “Your video as well as Cisco’s web site recommends CEF for EBGP multipath design… but interestingly, it does work without CEF”.

One of my readers has noted an interesting load-balancing behavior: when he was running traceroute tests from various routers in a topology similar to the one displayed below, the traceroute outputs indicated per-packet load balancing (both paths were used) when they were initiated from R2 or R3, but used a single path when initiated from R1 or R4.

 

Peter Weymann sent me a really intriguing question:

A few days ago I started reading the Ciscopress book End-to-End Network Security: Defense-in-Depth and stumbled over the scheduler command. This one could be used to allocate time that the cpu spends on fast switching packets or process switching packets, if I understand it correctly. They also mention interrupting CPU processes but honestly I don't really understand how it works.

Cisco routers support (at least) three forms of layer-3 switching (formerly known as routing). CEF switching and fast switching are performed entirely within the interrupt context (I/O adapter interrupts a process the CPU is currently executing and all the work is done before the process resumes). Process switching is performed in two steps: packet is briefly analysed within the interrupt context and requeued into the IP Input process where it's eventually switched. Almost all I/O adapters used these days use a concept of RX/TX rings to communicate with the CPU, meaning that the CPU potentially has to handle more than one packet for each interrupt.

A while ago I wrote about cached CEF adjacencies and the impact they have on ARP caching. If you ever need to, you can debug them with the debug ip cef table command. As this command might produce a lot of output in a production network, always use it in combination with an access-list that limits the debugging to the selected address range.

Alternatively, you can use the debug arp adjacency command, but you cannot limit its output with an access-list

The "How could we figure out if any traffic uses the default route" challenge was obviously too easy; a number of readers quickly realized that the CEF accounting can do what we need (and I have to admit I've completely missed it).

However, when I started to explore the various CEF accounting features, it turned out the whole thing is not as simple as it looks. To start with, the ip cef accounting global configuration command configures three completely unrelated accounting features: per-prefix accounting (that we need), traffic matrix accounting (configured with the non-recursive keyword) and prefix-length accounting.

In a previous post I've been writing about the inability to clean the ARP cache due to cached CEF adjacencies. As it turns out, this behavior has another side effect: the router will automatically refresh all ARP entries (and CEF adjacencies) as they expire from the ARP cache. This might become a problem on high-end devices with a lot of directly connected hosts if you set the arp timeout to a low value.

Whenever a router running CEF switching has LAN interfaces (or any other multi-access interfaces), you'll find cached adjacencies for active directly attached IP neighbors in its CEF table. These adjacencies ensure the smooth traffic flow toward the LAN-attached next-hops (preventing the initial packet drop symptom once the next-hop becomes active).

One of the most commonly asked load-sharing-related questions is “can I load-share traffic across unequal-cost links?”. In general, the answer is no. In order to load-share the traffic, you need more than one path to the destination and the only way to get multiple routes toward a destination in the IP routing table is to make them equal-cost (the only notable exception being EIGRP that supports unequal-cost load-sharing with the variance parameter).

There are, however, two cases where you can force unequal traffic split across equal-cost paths toward a destination: when using inter-AS BGP with the link bandwidth parameter, and when using unequal-bandwidth traffic-engineering tunnels.

The packets that cannot be CEF-switched in a box with CEF switching enabled are punted to the next switching level (fast switching or process switching). The incoming packets can be punted for a number of reasons, for example:

• If the destination is reachable over an interface that cannot use CEF-switching due to a feature not supported by CEF (for example, X.25 link), the packet has to be fast- or process-switched.

These destinations are easily discovered by inspecting the punt adjacencies.