lost and found ( for me ? ): Open vSwitch

just referred to http://kashyapc.wordpress.com/2013/07/13/configuring-libvirt-guests-with-an-open-vswitch-bridge/

# cat /etc/fedora-release ;uname -ri

Fedora release 19 (Schrödinger’s Cat)

3.11.1-200.fc19.x86_64 x86_64

Fedora 19 box has one NIC called p10p1.

p10p1 gets IP address over DHCP.

# cat /etc/sysconfig/network-scripts/ifcfg-p10p1

DEVICE=p10p1

ONBOOT=yes

#BRIDGE=br0

BOOTPROTO=dhcp

install open vswitch via yum

# yum install openvswitch

start open vswitch

[root@localhost ~]# systemctl enable openvswitch.service

ln -s '/usr/lib/systemd/system/openvswitch.service' '/etc/systemd/system/multi-user.target.wants/openvswitch.service'

[root@localhost ~]# systemctl start openvswitch.service

[root@localhost ~]# systemctl status openvswitch.service

openvswitch.service - Open vSwitch Unit

Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled)

Active: active (exited) since 月 2013-09-30 00:09:20 JST; 5s ago

Process: 4117 ExecStart=/bin/true (code=exited, status=0/SUCCESS)

configure open vswitch bridge

# ovs-vsctl add-br ovsbr0

associate ovsbr0 to phsical interface p10p1

Please note that you will lost internet connectivity after issuing following command.

# ovs-vsctl add-port ovsbr0 p10p1

after issuing following commands, you will get internet connectivity again.

[root@localhost ~]# ifconfig p10p1 0.0.0.0

[root@localhost ~]# ifconfig ovsbr0 192.168.11.100

[root@localhost ~]# dhclient ovsbr0 &

[1] 4206

# ip addr show dev ovsbr0

8: ovsbr0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN

link/ether 78:e7:d1:7f:84:b7 brd ff:ff:ff:ff:ff:ff

inet 192.168.11.100/24 brd 192.168.11.255 scope global ovsbr0

valid_lft forever preferred_lft forever

inet 192.168.11.5/24 brd 192.168.11.255 scope global secondary ovsbr0

valid_lft forever preferred_lft forever

inet6 fe80::e8ec:5aff:fe59:8655/64 scope link

valid_lft forever preferred_lft forever

[root@localhost ~]#

[1]+ 終了 dhclient ovsbr0

[root@localhost ~]# ovs-vsctl show

6e8e87c4-700b-4556-b977-cc3fa103d893

Bridge "ovsbr0"

Port "p10p1"

Interface "p10p1"

Port "ovsbr0"

Interface "ovsbr0"

type: internal

edit VM’s xml file to use open vswitch.

# virsh dumpxml f19-vm1

start the VM

# virsh start f19-vm1

on the VM, confirm that the VM gets an IP address over DHCP.

[root@localhost ~]# ip addr show dev eth0

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 52:54:00:d0:86:2e brd ff:ff:ff:ff:ff:ff

inet 192.168.11.11/24 brd 192.168.11.255 scope global eth0

valid_lft forever preferred_lft forever

inet6 fe80::5054:ff:fed0:862e/64 scope link

valid_lft forever preferred_lft forever

[root@localhost ~]#

[root@localhost ~]# ping www.google.com -c 1

PING www.google.com (173.194.38.114) 56(84) bytes of data.

64 bytes from nrt19s18-in-f18.1e100.net (173.194.38.114): icmp_seq=1 ttl=53 time=11.2 ms

--- www.google.com ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 11.256/11.256/11.256/0.000 ms

Here’s how to use one physical interface for both the KVM host and VMs as a bridge.

just referred to
http://blog.allanglesit.com/2012/10/linux-kvm-ubuntu-12-10-with-openvswitch/
many thx :D

KVM host info

mint14-1 ~ # lsb_release -a
No LSB modules are available.
Distributor ID: LinuxMint
Description: Linux Mint 14 Nadia
Release: 14
Codename: nadia

mint14-1 ~ # uname -ri
3.5.0-21-generic x86_64

I’ve already prepared a VM called ubuntu1210-vm01.

virsh # list --all
Id    Name                           State
----------------------------------------------------
-     ubuntu1210-vm01                shut off

[ build OVS from a source ( fail ! ) ]

uninstall OVS packages.

mint14-1 ~ # apt-get remove openvswitch-controller openvswitch-brcompat openvswitch-switch openvswitch-datapath-source

before building OVS from a source , install linux-source packages.

# apt-get install linux-source linux-source-3.5.0

download an OVS source

mint14-1 ~ # wget http://openvswitch.org/releases/openvswitch-1.7.1.tar.gz

mint14-1 ~ # tar xzvf openvswitch-1.7.1.tar.gz

mint14-1 ~ # cd openvswitch-1.7.1/
mint14-1 openvswitch-1.7.1 # ./boot.sh
./boot.sh: 2: ./boot.sh: autoreconf: not found

# apt-get install autoconf -y

mint14-1 openvswitch-1.7.1 # ./boot.sh
Can't exec "libtoolize": No such file or directory at /usr/bin/autoreconf line 195.
Use of uninitialized value in pattern match (m//) at /usr/bin/autoreconf line 195.

mint14-1 openvswitch-1.7.1 # apt-get install libtool

mint14-1 openvswitch-1.7.1 # ./boot.sh
mint14-1 openvswitch-1.7.1 #

mint14-1 openvswitch-1.7.1 # make
make all-recursive
make[1]: Entering directory `/root/openvswitch-1.7.1'
Making all in datapath
make[2]: Entering directory `/root/openvswitch-1.7.1/datapath'
Making all in linux
make[3]: Entering directory `/root/openvswitch-1.7.1/datapath/linux'
make -C /lib/modules/3.5.0-21-generic/build M=/root/openvswitch-1.7.1/datapath/linux modules
make[4]: Entering directory `/usr/src/linux-headers-3.5.0-21-generic'
CC [M] /root/openvswitch-1.7.1/datapath/linux/datapath.o
/root/openvswitch-1.7.1/datapath/linux/datapath.c:42:24: fatal error: asm/system.h: No such file or directory
compilation terminated.
make[5]: *** [/root/openvswitch-1.7.1/datapath/linux/datapath.o] Error 1
make[4]: *** [_module_/root/openvswitch-1.7.1/datapath/linux] Error 2
make[4]: Leaving directory `/usr/src/linux-headers-3.5.0-21-generic'
make[3]: *** [default] Error 2
make[3]: Leaving directory `/root/openvswitch-1.7.1/datapath/linux'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/root/openvswitch-1.7.1/datapath'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/root/openvswitch-1.7.1'
make: *** [all] Error 2

seen from datapath.c , after 3.3 kernels are not supported..

mint14-1 openvswitch-1.7.1 # less datapath/linux/datapath.c
#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,18) || \
   LINUX_VERSION_CODE >= KERNEL_VERSION(3,4,0)
#error Kernels before 2.6.18 or after 3.3 are not supported by this version of Open vSwitch.
#endif

I’ll install OVS via apt-get and give up building OVS from a source.

[ install OVS via apt-get ( success ! ) ]

before installing OVS , disable the network called “default” ( virbr0 ) which have been installed by default when installing KVM.

virsh # net-list
Name                 State      Autostart
-----------------------------------------
default              active     yes

virsh # net-destroy default
Network default destroyed

virsh # net-autostart --disable default
Network default unmarked as autostarted

virsh # net-list --all
Name                 State      Autostart
-----------------------------------------
default              inactive   no

install OVS related packages via apt-get.

mint14-1 ~ # apt-get install openvswitch-controller openvswitch-brcompat openvswitch-switch openvswitch-datapath-source

remove ebtables ( Ethernet bridge frame table administration )

mint14-1 ~ # apt-get purge ebtables

install OVS related packages via apt-get.

mint14-1 ~ # apt-get install openvswitch-controller openvswitch-brcompat openvswitch-switch openvswitch-datapath-source

confirm that openvswitch bridge compatibility is disabled.

mint14-1 ~ # cat /etc/default/openvswitch-switch
# This is a POSIX shell fragment                -*- sh -*-

# FORCE_COREFILES: If 'yes' then core files will be enabled.
# FORCE_COREFILES=yes

# BRCOMPAT: If 'yes' and the openvswitch-brcompat package is installed, then
# Linux bridge compatibility will be enabled.
#BRCOMPAT=yes

restart openvswitch-controller / switch

mint14-1 ~ # /etc/init.d/openvswitch-switch restart
* ovs-brcompatd is not running
* Killing ovs-vswitchd (1705)
* Killing ovsdb-server (1695)
* Starting ovsdb-server
* Configuring Open vSwitch system IDs
* Starting ovs-vswitchd

mint14-1 ~ # /etc/init.d/openvswitch-controller restart

[ interface configuration on the KVM host ]

before configuring network setting for OVS.
There’s one NIC called eth0 and eth0 get an IP address from DHCP Server.

mint14-1 ~ # cat /etc/network/interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback

I’m using networking script not network-manager.

mint14-1 ~ # sysv-rc-conf --list networking
networking   2:on 3:on 4:on 5:on

mint14-1 ~ # sysv-rc-conf --list network-manager
network-mana 2:off 3:off 4:off 5:off

after editing the interfaces file.
use one physical NIC ( eth0 ) for both the KVM and VMs.

mint14-1 ~ # cat /etc/network/interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback

# for eht0
auto eth0
iface eth0 inet manual
up ifconfig $IFACE 0.0.0.0 up
down ifconfig $IFACE down

# for OVS
auto ovsbr0p1
iface ovsbr0p1 inet dhcp

do not reboot the OS. continue to configure OVS.

After configuring OVS , reboot the OS.

[ configure OSV ]

create a bridge called ovsbr0

mint14-1 ~ # ovs-vsctl add-br ovsbr0

connect the bridge “ovsbr0” to “eth0”
This enables VMs access to the internet via eth0.

# ovs-vsctl add-port ovsbr0 eth0

associate ovsbr0p1 with the bridge ovsbr0

# ovs-vsctl add-port ovsbr0 ovsbr0p1 -- set interface ovsbr0p1 type=internal

When configuring OVS , the KVM host lost Internet connectivity.
However , after rebooting the OS , the KVM host gets Internet connectivity.

edit /etc/init/failsafe.conf to avoid hanging networking when booting the OS.

from

    31         $PLYMOUTH message --text="Waiting for network configuration..."      31 || :
    32         sleep 40
    33
    34         $PLYMOUTH message --text="Waiting up to 60 more seconds for netw     34 ork configuration..." || :
    35         sleep 59

to

    31         $PLYMOUTH message --text="Waiting for network configuration..."      31 || :
    32         #sleep 40
    33         sleep 1
    34
    35         $PLYMOUTH message --text="Waiting up to 60 more seconds for netw     35 ork configuration..." || :
    36         #sleep 59
    37         sleep 1

reboot the OS

# init 6

after rebooting the OS.

mint14-1 ~ # ovs-vsctl show
cd925df8-427b-46c3-9acd-28a712e9ba15
   Bridge "ovsbr0"
       Port "eth0"
           Interface "eth0"
       Port "ovsbr0p1"
           Interface "ovsbr0p1"
               type: internal
       Port "ovsbr0"
           Interface "ovsbr0"
               type: internal
   ovs_version: "1.4.3"

interface configuration is like this:

mint14-1 ~ # ifconfig
eth0      Link encap:Ethernet HWaddr
         inet6 addr: fe80::7ae7:d1ff:fe7f:84b7/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
         RX packets:7177 errors:0 dropped:0 overruns:0 frame:0
         TX packets:6704 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:2320311 (2.3 MB) TX bytes:2091563 (2.0 MB)

lo        Link encap:Local Loopback
         inet addr:127.0.0.1 Mask:255.0.0.0
         inet6 addr: ::1/128 Scope:Host
         UP LOOPBACK RUNNING MTU:16436 Metric:1
         RX packets:456 errors:0 dropped:0 overruns:0 frame:0
         TX packets:456 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:55924 (55.9 KB) TX bytes:55924 (55.9 KB)

ovsbr0p1 Link encap:Ethernet HWaddr
         inet addr:192.168.11.18 Bcast:192.168.11.255 Mask:255.255.255.0
         inet6 addr: fe80::4c4f:5bff:fe6c:3afc/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
         RX packets:7144 errors:0 dropped:0 overruns:0 frame:0
         TX packets:6654 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:2310240 (2.3 MB) TX bytes:2085980 (2.0 MB)

confirm the KVM host has internet connectivity.

mint14-1 ~ # ping -c 1 www.google.com
PING www.google.com (74.125.235.148) 56(84) bytes of data.
64 bytes from nrt19s11-in-f20.1e100.net (74.125.235.148): icmp_req=1 ttl=54 time=9.73 ms

--- www.google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 9.730/9.730/9.730/0.000 ms

[ configure VM ]

edit ubuntu1210-vm01 configuration to use OSV.

mint14-1 ~ # virsh edit ubuntu1210-vm01

from

   <interface type='bridge'>
     <mac address='52:54:00:ac:a8:cb'/>
     <source bridge='virbr0'/>
     <model type='virtio'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
   </interface>

to

   <interface type='bridge'>
     <mac address='52:54:00:ac:a8:cb'/>
     <source bridge='ovsbr0'/>
     <virtualport type='openvswitch'>
     </virtualport>
     <model type='virtio'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
   </interface>

start the VM

virsh # start ubuntu1210-vm01 --console

confirm the VM has Internet connectivity.

ubuntu-vm1 login: root
Password:
Last login: Wed Jan 9 00:29:00 JST 2013 on ttyS0
Welcome to Ubuntu 12.10 (GNU/Linux 3.5.0-21-generic x86_64)

* Documentation: https://help.ubuntu.com/
root@ubuntu-vm1:~# ping www.google.com -n -c 1
PING www.google.com (74.125.235.83) 56(84) bytes of data.
64 bytes from 74.125.235.83: icmp_req=1 ttl=54 time=12.7 ms

--- www.google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, tim

after editing the xml file , librit will create a port “vnet0” on OSV for the VM.

mint14-1 ~ # ovs-vsctl show
cd925df8-427b-46c3-9acd-28a712e9ba15
   Bridge "ovsbr0"
       Port "eth0"
           Interface "eth0"
       Port "ovsbr0p1"
           Interface "ovsbr0p1"
               type: internal
       Port "ovsbr0"
           Interface "ovsbr0"
               type: internal
       Port "vnet0"
           Interface "vnet0"
   ovs_version: "1.4.3"

get things done !

I need to study OVS more..

lost and found ( for me ? )

lost and found ( for me ? )

openvswitch ovsdbmonitor

Fedora 19: install open vswitch

Linux Mint 14 : how to install / use Open vSwitch in combination with KVM