Results 1 to 3 of 3

Thread: KVM Bridged Network - solutions and problems

  1. #1
    Join Date
    Apr 2007
    Beans
    32

    Wink [Solved] KVM Bridged Network - solutions and problems

    Edit: This was very easy to solve. Thanks to a quick search I found this link...

    http://blog.cynapses.org/2007/07/12/...network-setup/

    Down near the bottom

    Note that the VMs have different MAC addresses. It took me a long time to find why I couldn’t ping from one guest to another
    Somehow I'd managed to give each virtual machine the same MAC. Bridge routing cannot work then since it uses the MAC at layer2 to do the bridging.

    With this changed everything works like a charm - I even have a virtualized VPN gateway working now.
    --


    I have successfully got a couple of KVM virtual servers running on an 8-way ubuntu host with a single NIC on local physical network 192.168.100.0/24. I have a gateway router at 192.168.100.1 and a few other physical servers on the network.

    My experience so far with KVM and libvirt is pretty nice. Default networking works fine. But now I want to turn the virtual servers into production workhorses so I need to give them static IPs on a bridged host network.

    So far I've tried two configurations...

    1) Simple Bridged

    Following a combination of the Ubuntu KVM howto

    https://help.ubuntu.com/community/KVM

    and the libvirt wiki

    http://wiki.libvirt.org/page/Network...terface_config

    I've managed to give my two VMs IP addresses 192.168.100.101 and 192.168.100.102. Both machines are visible to the host. Both machines can ping the physical router. So far so good. Here's the /etc/network/interfaces configuration:

    auto lo
    iface lo inet loopback

    iface eth0 inet static
    address 192.168.100.10
    netmask 255.255.255.0
    gateway 192.168.100.1

    auto eth0

    auto br0
    iface br0 inet static
    address 192.168.100.10
    network 192.168.100.0
    netmask 255.255.255.0
    broadcast 192.168.100.255
    gateway 192.168.100.1
    bridge_ports eth0
    bridge_fd 9
    bridge_hello 2
    bridge_maxage 12
    bridge_stp off
    The hard part was figuring out that libvirt needs this configuration for a bridged network guest in the vm definition file

    Code:
     <interface type='bridge'>
         <mac address='52:54:e9:81:87:15'/>
         <source bridge='br0'/>
     </interface>
    So the now the problems.

    i) Intermittent packet loss with 'ping 192.168.100.10x' from a physical server on the physical network. Also see latency on first connection with up to 10 packets lost at the start of ping. Seem to behave fine with a flood 'ping -f'

    ii) Worse problem, neither virtual machine can see the other. Cannot ping 192.168.100.10x at all. This is a killer since these VMs will provide an n-tier application platform.

    Suggestions on how I can fix this would be appreciated. But I tried another network configuration too...

    2) VDE (Virtual Distributed Networking)

    I used the following guide to try a VDE network...

    http://tjworld.net/wiki/Linux/Ubuntu...hVDENetworking

    Have sucessfully got all virtual machines on the virtual network and each can ping the other, and the host can ping each of them. Here's my /etc/network/interfaces for this ...

    Code:
    auto lo
    iface lo inet loopback
    
    auto eth0
    iface eth0 inet static
            address 192.168.100.10
            netmask 255.255.255.0
            gateway 192.168.100.1
    
    
    # KVM/QEMU Virtual Machine Network
    auto kvm0
    iface kvm0 inet static
            address 192.168.110.254
            netmask 255.255.255.0
            pre-up /usr/bin/vde_switch --tap ${IFACE} --daemon --group vde2-net --sock /var/run/${IFACE}.ctl \
             --mod 775 --mgmtmode 770 --mgmt /var/run/${IFACE}-manage --pidfile /var/run/${IFACE}_vde.pid
            up /usr/sbin/dnsmasq --interface=${IFACE}  --except-interface=lo --bind-interfaces --user=nobody \
             --dhcp-range=kvm,192.168.110.1,192.168.110.99,255.255.255.0,192.168.110.255,8h \
             --domain=1060research.com --pid-file=/var/run/${IFACE}_dnsmasq.pid --conf-file
            up echo "`route -n | sed -n 's/^0\.0\.0\.0 .* \(.*\)$/\1/p'`" > /var/run/${IFACE}_route
            up iptables -t nat -A POSTROUTING -o `cat /var/run/${IFACE}_route` -j MASQUERADE
            down iptables -t nat -D POSTROUTING -o `cat /var/run/${IFACE}_route` -j MASQUERADE
            down kill -s TERM `cat /var/run/${IFACE}_dnsmasq.pid` && rm -f /var/run/${IFACE}_dnsmasq.pid
            post-down kill -s HUP `cat /var/run/${IFACE}_vde.pid`
            post-down rm -f /var/run/${IFACE}_route
    Problem with this

    i) My virtual network is on 192.168.110.0/24 and my physical network is on 192.168.100.0/24. I don't know how to bridge the physical network to the eth0 interface and connect up the VDE network. So the VDE network is purely logical and can't see the outside world - nor can be found from the physical network. How do I bridge it?

    My preference is to solve the bridging for config 1) since libvirt makes production management of the VMs very easy. Option 2) works great and looks like the future but VDE is not yet supported by libvirt so it is going to take a lot of work to get the VMs manageable for production.

    Hope these experiences help someone and if you've got any tips for me they'd be greatly appreciated.

    In general, KVM seems like a good option, I've used VMware server but its a pain that it requires rebuilding each time the host server gets a kernel security fix. KVM is just a kernel module so it stays in step and so is very easy to manage long term. Performance is excellent (barring the network issues I need to solve).
    Last edited by drplix; August 14th, 2008 at 03:11 PM. Reason: Solved

  2. #2
    Join Date
    Oct 2005
    Location
    sweden
    Beans
    74
    Distro
    Ubuntu 8.04 Hardy Heron

    Re: [Solved] KVM Bridged Network - solutions and problems

    Is the static configuration of eth0 required or is it just because you need it on your LAN?
    Windows belongs in /dev/null and not on /dev/hd*|/dev/sd* !

  3. #3
    Join Date
    Feb 2009
    Location
    Idaho
    Beans
    7
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: KVM Bridged Network - solutions and problems

    Well I'm seeing something similar to your scenario 1, and the mac addresses are different.
    Every so often the inside interface eth0 starts to drop out, and my remote ssh session will fail. I do routing through this box so I lose all external connectivity when this happens.

    I have 2 NICs so my /etc/network/interfaces file is:

    Code:
    # The loopback network interface
    auto lo
    iface lo inet loopback
    
    # The primary network interface
    auto eth0
    iface eth0 inet manual
    auto br0
    iface br0 inet static
            address 192.168.0.5
            netmask 255.255.255.0
            network 192.168.0.0
            broadcast 192.168.0.255
            bridge_ports eth0
            bridge_fd 0
            bridge_hello 2
            bridge_maxage 12
            bridge_stp on
    # Secondary
    auto eth1
    iface eth1 inet manual
    auto br1
    iface br1 inet static
            address 192.168.10.5
            netmask 255.255.255.0
            network 192.168.10.0
            broadcast 192.168.10.255
            gateway 192.168.10.1
            bridge_ports eth1
            bridge_fd 1
            bridge_hello 2
            bridge_maxage 12
            bridge_stp on
            # dns-* options are implemented by the resolvconf package, if installed
            dns-nameservers 192.168.0.106 192.168.0.11
            dns-search mydomain.com
    ifconfig yeilds:
    Code:
    br0       Link encap:Ethernet  HWaddr 00:e0:81:b0:fb:2c  
              inet addr:192.168.0.5  Bcast:192.168.0.255  Mask:255.255.255.0
              inet6 addr: fe80::2e0:81ff:feb0:fb2c/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:87805 errors:0 dropped:0 overruns:0 frame:0
              TX packets:47555 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0 
              RX bytes:13139783 (13.1 MB)  TX bytes:31771326 (31.7 MB)
    
    br1       Link encap:Ethernet  HWaddr 00:e0:81:b0:fb:2d  
              inet addr:192.168.10.5  Bcast:192.168.10.255  Mask:255.255.255.0
              inet6 addr: fe80::2e0:81ff:feb0:fb2d/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:102014 errors:0 dropped:0 overruns:0 frame:0
              TX packets:68153 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0 
              RX bytes:34595789 (34.5 MB)  TX bytes:11529481 (11.5 MB)
    
    eth0      Link encap:Ethernet  HWaddr 00:e0:81:b0:fb:2c  
              inet6 addr: fe80::2e0:81ff:feb0:fb2c/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:128192 errors:0 dropped:0 overruns:0 frame:0
              TX packets:124217 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:100 
              RX bytes:25193018 (25.1 MB)  TX bytes:47270423 (47.2 MB)
              Memory:d3000000-d3020000 
    
    eth1      Link encap:Ethernet  HWaddr 00:e0:81:b0:fb:2d  
              inet6 addr: fe80::2e0:81ff:feb0:fb2d/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:115937 errors:0 dropped:0 overruns:0 frame:0
              TX packets:121526 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:100 
              RX bytes:55747765 (55.7 MB)  TX bytes:24450319 (24.4 MB)
              Memory:d3020000-d3040000 
    
    lo        Link encap:Local Loopback  
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING  MTU:16436  Metric:1
              RX packets:118648 errors:0 dropped:0 overruns:0 frame:0
              TX packets:118648 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0 
              RX bytes:72029076 (72.0 MB)  TX bytes:72029076 (72.0 MB)
    
    vnet0     Link encap:Ethernet  HWaddr de:ea:ed:8a:c0:bb  
              inet6 addr: fe80::dcea:edff:fe8a:c0bb/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:45528 errors:0 dropped:0 overruns:0 frame:0
              TX packets:95438 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:500 
              RX bytes:10977464 (10.9 MB)  TX bytes:16254696 (16.2 MB)
    
    vnet1     Link encap:Ethernet  HWaddr 46:bf:c4:1f:94:27  
              inet6 addr: fe80::44bf:c4ff:fe1f:9427/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:32186 errors:0 dropped:0 overruns:0 frame:0
              TX packets:80702 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:500 
              RX bytes:11927177 (11.9 MB)  TX bytes:23459593 (23.4 MB)
    I tried to get rid of vnet0 and vnet1 thinking that they were causing the issue, but they're obviously being pulled from somewhere I haven't thought to look.

    My vm xml file has this:
    Code:
     <interface type='bridge'>
          <mac address='00:16:36:65:7f:d9'/>
          <source bridge='br0'/>
        </interface>
        <interface type='bridge'>
          <mac address='00:16:36:5f:e0:7a'/>
          <source bridge='br1'/>
        </interface>
    I'm wondering if there is a problem trying to use both nics on a vm instance. I dunno. If I remove either of them the problem goes away. However, I kind of need both in order to provide service inside and outside of the network.

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •