lost and found ( for me ? )

Ubuntu 11.04 : KVM : live migration w/o shared storage


# kvm –version
QEMU emulator version 0.14.0 (qemu-kvm-0.14.0), Copyright (c) 2003-2008 Fabrice Bellard

# uname -r
2.6.38-8-server


[ network topology ]

 VM   ---> live migration
ubuntu-1       ubuntu-2
   |             |
   -------L2SW  ---

There’s no shared storage.
Llive migration w/o shared storage has been implemented on kvm 0.12.1

Assumption:

VM called ubuntu-vm3 is running on ubuntu-1
do live migration from ubuntu-1 to ubuntu-2.

[ procedure ]

on ubuntu-2 , create virtual disk w/ qemu-img and open TCP socket w/ kvm command
on ubuntu-1 , do live migration

[ logs including trial and error ]

When doing live migration w/o shared storage , you need to specify either –copy-storage-all or –copy-storage inc option.
--copy-storage-all indicates
          migration with non-shared storage with full disk copy,
--copy-storage-inc indicates
          migration with non-shared storage with
          incremental copy (same base image shared between source and
          destination).


There’s a VM called ubuntu-vm3 which is running on ubuntu-1.
I’ll do live migration of ubuntu-vm3 from ubuntu-1 to ubuntu-2 w/o shared storage.

on ubuntu-1
VM called ubuntu-vm3 is running.
root@ubuntu-1:~# LANG=C virsh list --all | grep run
 1 ubuntu-vm3           running


on ubuntu-2
There’s no ubuntu-vm3.
root@ubuntu-2:~# LANG=C virsh list --all
Id Name                 State
----------------------------------
 - BT5                  shut off
 - SL6-vm1              shut off
 - SL6-vm2              shut off
 - SL6-vm3              shut off


move ubuntu-vm3 from ubuntu-1 to ubuntu-2
oh ,,, error.
root@ubuntu-1:~#virsh migrate --live --copy-storage-all ubuntu-vm3 qemu+ssh://ubuntu-2/system --verbose
root@ubuntu-2's password:
エラー: unable to set user and group to '102:107' on '/var/disk1/images/ubuntu-vm3.img': No such file or directory


There’s a user called libvirt-qeme whose user ID and group ID are 102 and 107 on both ubuntu-1 and ubuntu-2

on ubuntu-1
root@ubuntu-1:~# egrep 102 /etc/passwd
libvirt-qemu:x:102:107:Libvirt Qemu,,,:/var/lib/libvirt:/bin/false


on ubuntu-2
root@ubuntu-2:~# egrep 102 /etc/passwd
libvirt-qemu:x:102:107:Libvirt Qemu,,,:/var/lib/libvirt:/bin/false


Seen from logs on ubuntu-2 , I can find the following message.
This problem seems to ubuntu-2’s side.
# tail -f  /var/log/libvirt/qemu/ubuntu-vm3.log
time: shutting down


nnn, iptables issue ??
live migration uses TCP 49125 – 49126.
So I allowed 49125 – 49126 access ( all ports ) on both ubuntu-1 and ubuntu-2.

on ubuntu-1
root@ubuntu-1:~# iptables –F
root@ubuntu-1:~# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination


on ubuntu-2
root@ubuntu-2:~# iptables -F
root@ubuntu-2:~# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination


try again.same error.
root@ubuntu-1:~# virsh migrate --live --copy-storage-all ubuntu-vm3 qemu+ssh://ubuntu-2/system --verbose
root@ubuntu-2's password:
エラー: unable to set user and group to '102:107' on '/var/disk1/images/ubuntu-vm3.img': No such file or directory


on ubuntu-2 , create virtual disk image for ubuntu-vm3 w/ touch command under /var/disk1/images directory.
root@ubuntu-2:.../images# touch /var/disk1/images/ubuntu-vm3.img
root@ubuntu-2:.../images# chown 102.107 /var/disk1/images/ubuntu-vm3.img


tray again. another error
root@ubuntu-1:~# virsh migrate --live --copy-storage-all ubuntu-vm3 qemu+ssh://ubuntu-2/system --verbose
root@ubuntu-2's password:
エラー: operation failed: failed to retrieve chardev info in qemu with 'info chardev'


log on ubuntu-2
root@ubuntu-2:~# less /var/log/libvirt/qemu/ubuntu-vm3.log

time: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 128 -smp 1,sockets=1,cores=1,threads=1 -name ubuntu-vm3 -uuid e97d49e3-93e2-dec3-d075-bd3b093080ef -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/ubuntu-vm3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot c -drive file=/var/disk1/images/ubuntu-vm3.img,if=none,id=drive-ide0-0-0,format=qcow2 -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=18,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:36:4b:66:fb,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -k ja -vga cirrus -incoming tcp:0.0.0.0:49169 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
char device redirected to /dev/pts/3
kvm: -drive file=/var/disk1/images/ubuntu-vm3.img,if=none,id=drive-ide0-0-0,format=qcow2: could not open disk image /var/disk1/images/ubuntu-vm3.img: Invalid argument
time: shutting down


Okay. create virtual disk w/ qemu-img command.
creating virtual disk w/ touch command seems to be unacceptable.
root@ubuntu-2:~# qemu-img create -f qcow2 /var/disk1/images/ubuntu-vm3.img 10G
Formatting '/var/disk1/images/ubuntu-vm3.img', fmt=qcow2 size=10737418240 encryption=off cluster_size=0


open TCP socket.
root@ubuntu-2:~# /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 128 -smp 1,sockets=1,cores=1,threads=1 -name ubuntu-vm3 -uuid e97d49e3-93e2-dec3-d075-bd3b093080ef -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/ubuntu-vm3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot c -drive file=/var/disk1/images/ubuntu-vm3.img,if=none,id=drive-ide0-0-0,format=qcow2 -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=18,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:36:4b:66:fb,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -k ja -vga cirrus -incoming tcp:0.0.0.0:49169 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
char device redirected to /dev/pts/3
kvm: -netdev tap,fd=18,id=hostnet0: TUNGETIFF ioctl() failed: Bad file descriptor
TUNSETOFFLOAD ioctl() failed: Bad file descriptor
kvm: -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:36:4b:66:fb,bus=pci.0,addr=0x3: pci_add_option_rom: failed to find romfile "pxe-virtio.bin"


do live migration
root@ubuntu-1:~# virsh migrate --live --copy-storage-all ubuntu-vm3 qemu+ssh://u
buntu-2/system --verbose
root@ubuntu-2's password:
Migration: [  0 %]


log on ubuntu-2
time: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 128 -smp 1,sockets=1,cores=1,threads=1 -name ubuntu-vm3 -uuid e97d49e3-93e2-dec3-d075-bd3b093080ef -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/ubuntu-vm3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot c -drive file=/var/disk1/images/ubuntu-vm3.img,if=none,id=drive-ide0-0-0,format=qcow2 -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=18,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:36:4b:66:fb,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:2 -k ja -vga cirrus -incoming tcp:0.0.0.0:49171 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
char device redirected to /dev/pts/5
kvm: -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:36:4b:66:fb,bus=pci.0,addr=0x3: pci_add_option_rom: failed to find romfile "pxe-virtio.bin"
Receiving block device images
Completed 27 %


complete !

on ubuntu-1
root@ubuntu-1:~# virsh migrate --live --copy-storage-all ubuntu-vm3 qemu+ssh://u
buntu-2/system --verbose
root@ubuntu-2's password:
Migration: [100 %]


on ubuntu-2
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 128 -smp 1,sockets=1,cores=1,threads=1 -name ubuntu-vm3 -uuid e97d49e3-93e2-dec3-d075-bd3b093080ef -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/ubuntu-vm3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot c -drive file=/var/disk1/images/ubuntu-vm3.img,if=none,id=drive-ide0-0-0,format=qcow2 -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=18,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:36:4b:66:fb,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:2 -k ja -vga cirrus -incoming tcp:0.0.0.0:49171 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
char device redirected to /dev/pts/5
kvm: -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:36:4b:66:fb,bus=pci.0,addr=0x3: pci_add_option_rom: failed to find romfile "pxe-virtio.bin"
Receiving block device images
Completed 100 %


ubuntu-vm3 is running on ubuntu-2 !

on ubuntu-1
root@ubuntu-1:~# LANG=C virsh list --all
Id Name                 State
----------------------------------
 - SL6-vm2              shut off
 - ubuntu-vm1           shut off
 - ubuntu-vm2           shut off
 - ubuntu-vm3           shut off


on ubuntu-2
root@ubuntu-2:~# LANG=C virsh list --all
Id Name                 State
----------------------------------
21 ubuntu-vm3           running
 - BT5                  shut off
 - SL6-vm1              shut off
 - SL6-vm2              shut off
 - SL6-vm3              shut off


It works well.
root@ubuntu-2:~# virsh console ubuntu-vm3
Connected to domain ubuntu-vm3
エスケープ文字は  ^] です

root@ubuntu-vm3:~# echo hello
hello

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.