High Availability — LAN — NIC Bundling

The parent article on High Availability.

Switching on a LAN provides some of the most basic network connectivity options, and are often overlooked. Nonetheless most switches (Cisco, HP, Dell and others) support these configurations, but one thing I can guarantee is that you will find limitations on pretty much every platform. If you’re after inter-operability, do your testing so you can understand these limitations.

Bundle Network Links

We want to bundle network links for two reasons; to aggregate bandwidth (two links give twice the packet-passing capacity) and for failover (if one link fails a second is still running).

LACP

I discussed LACP in an earlier article, but I would like to go into a little more detail here. Make sure you review Cisco’s documentation on configuring LACP, and the Wikipedia article on link aggregation.

In my experience, I find LACP to be the best solution for link aggregation. It is a common protocol so interoperability between devices is almost always possible and the configuration is sensible enough that you can explain it to a lay-person.

In the example above, we have bundled two physical links into a single logical link between two switches.

LACP Virtual Adaptors

When we bundle network links with LACP, each host creates a virtual adaptor that represents the bundle. For example, on a Cisco switch we can create an interface called portchannel 1, that represents the two interfaces fastethernet0/1 and fastethernet0/2.

In this case, instead of making changes to or examining the configurations of the physical interfaces we can instead work with portchannel 1. Of course you can work with the physical interfaces, but you must take care to make sure all parameters match on all physical interfaces in the bundle.

LACP Load-balancing Flows

LACP is flow-aware, and it can be configured to load-balance based on MAC address or IP address; the default in Cisco switches is to load-balance based on MAC address.

Load-balancing only really works when the system is able to identify many unique flows; as each flow is established it is put on one of the bundled links and all subsequent traffic also follows that physical link.

Be aware, that load-balancing based on MAC address (the default behaviour) may not be what you want — if your traffic crosses a router the original source MAC address will be obscured.

In a routed environment (if you’re using VLANs) you will find that any traffic that crosses a routing boundary will have its source MAC address replaced by the router MAC address. This can make many hosts appear (to LACP) as if they’re coming from a single MAC address and will definitely skew the load-balancing calculations. A better approach is to use IP based LACP load-balancing, as each host will likely have a unique IP address.

A good rule of thumb is to use MAC address load-balancing if you’ve got a flat Layer 2 network. It is easier for the switch to identify MAC addresses and in a network like this, every flow should be coming from a unique MAC address.

Server Based Failover and Load Balancing

Some NIC manufacturers have provided software to accommodate NIC failover and load-balancing without using LACP. See HP’s document describing the options they offer.

These configurations do work, however they are complex (in terms of traffic flow) and are therefore harder to troubleshoot in the event of network problems. Use LACP where possible, and these server based methods where necessary.

One thought on “High Availability — LAN — NIC Bundling

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s