802.3ad Network Bonding & KVM Network Bridge

**jinzishuai** · July 27th, 2009

I finally got it working with the /etc/network/interface file as

Code:

auto bond0
iface bond0 inet manual
        slaves eth0 eth1
        bond-mode 0
        bond-miimon 100

auto br0
iface br0 inet static
        address 192.168.1.23
        netmask 255.255.255.0
        network 192.168.1.0
        broadcast 192.168.1.255
        gateway 192.168.1.1
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 192.168.1.1
        dns-search example.com
        bridge_ports bond0
        bridge_fd 9
        bridge_hello 2
        bridge_maxage 12
        bridge_stp off

**brutimus** · January 29th, 2010

I replicated what jinzishuai posted and it fixed my problems! The solution appears to have been with the bridge_fd, bridge_hello, and bridge_maxage values. Thanks!

Here is my /etc/network/interfaces

Code:

auto bond0
iface bond0 inet manual
    slaves eth0 eth1
    bond-mode 4
    bond-miimon 100

auto virbr0
iface virbr0 inet static
    address 192.168.3.182
    netmask 255.255.255.0
    network 192.168.3.0
    broadcast 192.168.3.255
    gateway 192.168.3.1
    dns-nameservers 192.168.3.1
    dns-search gsi
    bridge_ports bond0
    bridge_fd 9
    bridge_hello 2
    bridge_maxage 12
    bridge_stp off

**SpiderNetUK** · February 8th, 2010

Thanks, this has helped me get my interfaces bonded and bridge so the VM's ca nbe networked. Trouble is I'm seeing a lot of kernel error messages in the syslog

Code:

Feb  8 11:46:13 PUBLIC5 kernel: [268889.602543] bond0: received packet with  own address as source address
Feb  8 11:46:14 PUBLIC5 kernel: [268890.600951] bond0: received packet with  own address as source address
Feb  8 11:46:15 PUBLIC5 kernel: [268891.599291] bond0: received packet with  own address as source address
Feb  8 11:48:13 PUBLIC5 kernel: [269009.401908] bond0: received packet with  own address as source address
Feb  8 11:48:14 PUBLIC5 kernel: [269010.400340] bond0: received packet with  own address as source address

This results in the VM being unresponsive and not replying to ping requests, it happens every 2mins.

Anyone else had this? Thinking it might eb something to do with broadcasts.

**SpiderNetUK** · March 17th, 2010

Thought I'd just bump this thread...

In the end I had to disable the bond, the bridge works fine this way.

**chienpo** · June 14th, 2010

While doing some searching I found this thread. I am also trying to set up a server to host virtual guests. This server has 4 network interfaces that I want to bond together for redundancy/availability (and bandwidth). I then want to set up bridging on the bonded interface so I can use this for my virtual guests, allowing them to appear as hosts on the same subnet as my server.

The difference between me and earlier posters appears to be that I'm running a newer version of Ubuntu Server: 10.04 (lucid lynx).

Anyhow I first set up my bonded interface successfully using the following /etc/network/interfaces:

Code:

# ...
# The bonded network interface
auto bond0
iface bond0 inet static
  address 192.168.100.102
  netmask 255.255.255.0
  gateway 192.168.100.1
  bond-slaves none
  bond-mode 802.3ad
  bond-miimon 100

auto eth0
iface eth0 inet manual
  bond-master bond0

auto eth1
iface eth1 inet manual
  bond-master bond0

auto eth2
iface eth2 inet manual
  bond-master bond0

auto eth3
iface eth3 inet manual
  bond-master bond0

The bonding alone works great. However, I can't get both bonding and bridging to work together. I've tried the configurations posted earlier that worked for others. I've also tried several variations of my own, essentially stabbing in the dark for solutions. My most recent attempt looked like the following:

Code:

# ...
# The bonded network interface
auto bond0
iface bond0 inet manual
  bond-slaves none
  bond-mode 802.3ad
  bond-miimon 100

auto eth0
iface eth0 inet manual
  bond-master bond0

auto eth1
iface eth1 inet manual
  bond-master bond0

auto eth2
iface eth2 inet manual
  bond-master bond0

auto eth3
iface eth3 inet manual
  bond-master bond0

# The bridged network interface
auto br0
iface br0 inet static
  address 192.168.100.102
  netmask 255.255.255.0
  gateway 192.168.100.1
  bridge_ports  bond0
  bridge_fd 9
  bridge_hello 2
  bridge_maxage 12
  bridge_stp off

Some of the permutations I've attempted include reordering the above stanzas in my /etc/network/interfaces file, as well as variations that look more like versions people have posted (usually for older versions of ubuntu).

Has anyone out there been able to successfully get bonding and bridging to work together on Ubuntu Server 10.04 (lucid lynx) in the manner that I am? Does anyone otherwise have any suggestions or pointers?

**Uthark** · June 25th, 2010

Originally Posted by chienpo

Has anyone out there been able to successfully get bonding and bridging to work together on Ubuntu Server 10.04 (lucid lynx) in the manner that I am? Does anyone otherwise have any suggestions or pointers?

I have Ubuntu Server 10.04 too, and I've managed to get some results.

I've set up bonding as usual, without using bond-master and bond-slave:

Code:

auto bond0
iface bond0 inet static
        address 0.0.0.0
        netmask 0.0.0.0
        bond_mode balance-rr
        bond_miimon 100
        bond_downdelay 200
        bond_updelay 200
        slaves eth0 eth1

And the bridge as:

Code:

auto br0
iface br0 inet static
        address 192.168.1.23
        netmask 255.255.255.0
        network 192.168.1.0
        broadcast 192.168.1.255
        gateway 192.168.1.1
        bridge_ports bond0
        bridge_stp on

My current issue is trying to connect br0 to KVM guests

**Uthark** · June 27th, 2010

After 12 hours straight, i've got this setup working for now:

First, enable bonding module, by appending it at /etc/modules:

Code:

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

loop
lp
rtc
bonding

Then edit /etc/network/interfaces:

Code:

# The loopback network interface
auto lo
iface lo inet loopback

auto bond0
iface bond0 inet manual
        bond-slaves none
        bond-mode balance-alb
        bond-miimon 100

auto eth0
iface eth0 inet manual
        bond-master bond0
        bond-primary eth0 eth1

auto eth1
iface eth1 inet manual
        bond-master bond0
        bond-primary eth0 eth1

auto vnet0
iface vnet0 inet static
        address 192.168.1.2
        netmask 255.255.255.0
        gateway 192.168.1.1
        bridge-ports bond0

Reboot and confirm that:

Your slave interfaces (eth0 and eth1 in my case) are UP, RUNNING and in SLAVE mode (check with ifconfig ethX)

Code:

root@kvmhost:~# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:30:48:f0:1b:ef  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:19 errors:0 dropped:0 overruns:0 frame:0
          TX packets:188 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:1372 (1.3 KB)  TX bytes:11311 (11.3 KB)
          Memory:fafe0000-fb000000 

root@kvmhost:~# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 00:30:48:f0:1b:ee  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:65 errors:0 dropped:0 overruns:0 frame:0
          TX packets:227 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:7810 (7.8 KB)  TX bytes:18041 (18.0 KB)
          Memory:faf20000-faf40000

Check if your bond (bond0 in my case) is UP, RUNNING, in MASTER mode (check with ifconfig bondX) and has no IP assigned.

Code:

root@kvmhost:~# ifconfig bond0
bond0     Link encap:Ethernet  HWaddr 00:30:48:f0:1b:ef  
          inet6 addr: fe80::230:48ff:fef0:1bef/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:145 errors:0 dropped:0 overruns:0 frame:0
          TX packets:948 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:16408 (16.4 KB)  TX bytes:62374 (62.3 KB)

Check if your bridge (vnet0 in my case) is UP, RUNNING and has the IP you've set (192.168.1.2 in my case):

Code:

root@kvmhost:~# ifconfig vnet0
vnet0     Link encap:Ethernet  HWaddr 00:30:48:f0:1b:ef  
          inet addr:192.168.1.2  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::230:48ff:fef0:1bef/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:144 errors:0 dropped:0 overruns:0 frame:0
          TX packets:75 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:14788 (14.7 KB)  TX bytes:11344 (11.3 KB)

Check if the bond is attached to the bridge. brctl show should show something like:

Code:

root@kvmhost:~# brctl show
bridge name	bridge id		STP enabled	interfaces
virbr0		8000.000000000000	yes		
vnet0		8000.003048f01bef	no		bond0

Check your ip routing table:

Code:

root@kvmhost:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     *               255.255.255.0   U     0      0        0 vnet0
192.168.122.0   *               255.255.255.0   U     0      0        0 virbr0
default         192.168.1.1     0.0.0.0         UG    100    0        0 vnet0

Check your KVM VM's XML definitions (listed at /etc/libvirt/qemu):

Code:

    <interface type='bridge'>
      <mac address='52:54:00:0a:7b:f7'/>
      <source bridge='vnet0'/>
      <model type='virtio'/>
    </interface>

I really hope all this hassle can be avoided, maybe by compiling netcf support in libvirt, so physical network interfaces can be detected and configured from virsh or virt-manager.

Don't forget to adjust bond-mode to your favorite bonding method!

**Nuesel** · January 20th, 2011

Hello,
Uthark, what packages do have installed to get the bonding/bridge running? Probably ifenslave and bridge-utils, something else what I need here?

Best regards
Nuesel

**Uthark** · January 20th, 2011

Originally Posted by Nuesel

Hello,
Uthark, what packages do have installed to get the bonding/bridge running? Probably ifenslave and bridge-utils, something else what I need here?

Best regards
Nuesel

You're right, just those two

Code:

aptitude install ifenslave bridge-utils

**Nuesel** · January 21st, 2011

Thanks for the answer.
I belive I've found the problem:
I used Ubuntu 10.10, fresh install and all update up to now. On this system your instruction doesn't work. The error message said something about an illegal argument.
But when I used ifenslave-2.6_1.1.0-14ubuntu2_amd64 (in my case 64bit) from Ubuntu 10.04 (instead of Ubuntu 10.10 package ifenslave-2.6 version 1.1.0-15) it seems to work. At least I checked all the things you mentioned and it seems good.
But if run dmesg I see alot of entries like:
vnet0: received packet on bond0 with own address as source address
similar as SpiderNetUK mentioned.
I will keep trying...

Thread: 802.3ad Network Bonding & KVM Network Bridge

Thread Tools

Display

Re: 802.3ad Network Bonding & KVM Network Bridge

Re: 802.3ad Network Bonding & KVM Network Bridge

Re: 802.3ad Network Bonding & KVM Network Bridge

Re: 802.3ad Network Bonding & KVM Network Bridge

Re: 802.3ad Network Bonding & KVM Network Bridge

Re: 802.3ad Network Bonding & KVM Network Bridge

Re: 802.3ad Network Bonding & KVM Network Bridge

Re: 802.3ad Network Bonding & KVM Network Bridge

Re: 802.3ad Network Bonding & KVM Network Bridge

Re: 802.3ad Network Bonding & KVM Network Bridge

Bookmarks

Bookmarks

Posting Permissions