User Tools

Site Tools

This documentation is no longer maintained by the Turris team (although it can be used by the community to share content). The current official documentation is at docs.turris.cz.

Definition:

https://en.wikipedia.org/wiki/Link_aggregation

This documentation will use the term bond to describe an 802.3ad configuration.

Requirements:

  1. Knowledge of exact ports to be used in bond
    • Drivers loaded for adapters to be used
    • Kernel assigned devices for adapters to be used
    • Properly plumbed interfaces
  2. Local configuration of adapter
  3. Configuration of ports on remote side of bond/aggregate link.

Prerequisite work:

For the purpose of this documentation we will use the Omnia as an example. lan0, lan1, lan2, lan3 constitute the 4 adjoined ports provided by the SoC. We will assume that we want to create a 2-port bond, using the left two RJ-45 ports.

The ideal way to find the correct interface names is to watch the dmesg log, while connecting to the port. A link state change will be logged when the device is properly linked to another. This method does not work when using modems, but will work fine when using network devices.

If you have added new devices, whether by PCIe slots, USB or another method you will need to make sure your device(s) are detected.

To show PCI/PCIe devices, and the corresponding kernel module in use:

lspci -v

To show USB devices:

lsusb -v

If your device is found, with these commands, but does have an associated device, you will need to look up the chipset used in your device, and install the proper kernel module package. For example if you install a PCIe card with an Intel i350 chip, you would have to install the kmod_igb package, as the i350 chip is part of the Intel IGB family.

The linux networking stack requires a bond interface to actually use a bond By following the steps listed above, you should come to the conclusion that lan0 and lan1 will be the devices for the bond. As such we will need to install the kmod_bonding package as well to get the kernel support of the bond interface.

With the installation of the correct packages and monitoring of the link states, we should find that the interfaces we want to use are:

lan0, lan1, bond0

If we installed any kernel modules, the modules should auto-probe, and we could restart the network stack. Unless you have a compelling reason not to, it is advised to reboot the device to make sure everything probes and plumbs properly on a clean boot.

Configuration:

There are multiple ways to configure the bond. The following commands can be run directly at a prompt for a temporary bond, or they can be added to a system start script. OpenWRT which the Omnia is based on, recommends the use of /etc/rc.local, so we will use that in this example.

The following commands will be used:

  • ifconfig bond0 down
  • echo 802.3ad > /sys/devices/virtual/net/bond0/bonding/mode
  • echo fast > /sys/devices/virtual/net/bond0/bonding/lacp_rate
  • echo layer3+4 > /sys/devices/virtual/net/bond0/bonding/xmit_hash_policy
  • echo +lan0 > /sys/devices/virtual/net/bond0/bonding/slaves
  • echo +lan1 > /sys/devices/virtual/net/bond0/bonding/slaves
  • ifconfig bond0 up

Explanation:

  • We must bring the bond0 interface down, as trying to add devices to it while it is up will result in an error if we do not: write error: Resource busy
  • With the interface down, we are going to set the bonding mode to LACP.
  • Then we are going to set the LACP polling rate to fast. There are other options and in some cases Cisco devices will require this option to be set to slow.
  • Next we will set the transmit has policy to layer 3 and layer 4. There are other options for this as well, but they are very rarely used.
  • Once the bond configuration parameters are set we will have to add each device to the bond that will make the aggregate bond.
  • Finally we will bring the bond0 interface back up, and if everything is set correctly you will have an active/active aggregate bond.

For the purpose of this example we will assume that port 15, and port 16 will be used for our bond.

Extreme Networks example:

# configure vlan {vlan name} add ports 15 untagged
# configure vlan {vlan name} del ports 16
# enable sharing 15 grouping 15-16 algorithm address-based L2 lacp
# show sharing
Load Sharing Monitor
Config    Current Agg     Min    Ld Share  Flags Ld Share  Agg Link  Link Up
Master    Master  Control Active Algorithm       Group     Mbr State Transitions
================================================================================
    15     15     LACP       1    L2        A     15        Y     A       8
                                  L2              16        Y     A       3
================================================================================
Link State: A-Active, D-Disabled, R-Ready, NP-Port not present, L-Loopback
Minimum Active: (<) Group is down. # active links less than configured minimum
Load Sharing Algorithm: (L2) Layer 2 address based, (L3) Layer 3 address based
                        (L3_L4) Layer 3 address and Layer 4 port based
                        (custom) User-selected address-based configuration
Custom Algorithm Configuration: ipv4 L3-and-L4, xor
Distribution Mode Flags:
        A - All: Distribute to all members
        L - Local Slot: Distribute to members local to ingress slot
        P - Port Lists: Distribute to per-slot configurable subset of members
Number of load sharing trunks: 1

#save

In this example, if the link state is “A” (active) for both ports and the Aggregate members are both “Y” (yes) your link is setup and active. Disconnecting one cable or the other will change the states in the show command. Your connection will remain active but only across the remaining connection.

Cisco example:

Router> enable 
Router# configure terminal
Router(config)# interface port-channel 1
Router(config-if)# interface g2/0/0 
Router(config-if)# no ip address
Router(config-if)# channel-group 1 mode active 
Router(config-if)# exit
Router(config)# interface g4/0/0 
Router(config-if)# no ip address
Router(config-if)# channel-group 1 mode active 
Router(config-if)# end 
Router# copy running-config startup-config