lost and found ( for me ? )

Ubuntu 13.04: install ganeti 2.6.2 within virtualbox host by using vagrant ( fail )

error logs..

I tried to deploy ganeti nodes within virtualbox host, however I can’t do that, because virtualbox does not support nested VMs..

here are environments I tried.

- one virtualbox host ( support VT ). ubuntu 13.10
- install ganeti within virtualbox host by using vagrant.
- ganeti nodes use KVM as its hypervisor . ( but virtualbox does not support nested VM, so ganeti nodes can not use KVM.)

virtualbox host
# tail -1 /etc/lsb-release ;uname -ri
DISTRIB_DESCRIPTION="Ubuntu 13.10"
3.11.0-13-generic x86_64

referred to

I have been using KVM as a hypervisor.
To install ganeti with vagrant, you need virtualbox.
So I unloaded KVM related kernel modules to avoid conflicts between KVM and virtualbox and then start virtualbox to install ganeti within virtualbox.
# rmmod kvm_amd
# rmmod kvm

install virtualbox and vagrant
# apt-get install virtualbox vagrant

# /etc/init.d/virtualbox start

# lsmod | grep vbox*
vboxpci                23194  0
vboxnetadp             25670  0
vboxnetflt             27613  0
vboxdrv               320455  3 vboxnetadp,vboxnetflt,vboxpci
#
# lsmod | grep kvm
#

When I want to switch back to KVM from virtualbox, stop virtualbox and then load KVM kernel modules as below.
# /etc/init.d/virtualbox stop

# modprobe kvm
# modprobe kvm_amd
# lsmod | grep kvm
kvm_amd                59958  0
kvm                   431315  1 kvm_amd

# rmmod kvm_amd
# rmmod kvm
# /etc/init.d/virtualbox restart

Let’s install ganeti!

# git clone https://github.com/ramereth/vagrant-ganeti.git
# cd vagrant-ganeti/
# git submodule update --init

no VMs are running.
# vagrant status
Current machine states:

node1                    not created (virtualbox)
node2                    not created (virtualbox)
node3                    not created (virtualbox)

start the node1.
error..
# vagrant up node1

Notice: /Stage[main]/Ganeti_tutorial::Ganeti::Initialize/Exec[initialize-ganeti]/returns: Corrupt configuration file: The configuration denotes node packer-ubuntu-12.04 as master, while my hostname is node1.example.org; opening a foreign configuration is only possible in accept_foreign mode
Notice: /Stage[main]/Ganeti_tutorial::Ganeti::Initialize/Exec[initialize-ganeti]/returns: Aborting.
Notice: /Stage[main]/Ganeti_tutorial::Ganeti::Initialize/Exec[initialize-ganeti]/returns: Failure: prerequisites not met for this operation:
Notice: /Stage[main]/Ganeti_tutorial::Ganeti::Initialize/Exec[initialize-ganeti]/returns: Cluster not initialized or this machine is not part of a cluster
Error: /vagrant/modules/ganeti_tutorial/files/scripts/initialize-ganeti returned 1 instead of one of [0]
Error: /Stage[main]/Ganeti_tutorial::Ganeti::Initialize/Exec[initialize-ganeti]/returns: change from notrun to 0 failed: /vagrant/modules/ganeti_tutorial/files/scripts/initialize-ganeti returned 1 instead of one of [0]
Notice: Finished catalog run in 86.96 seconds
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

cd /tmp/vagrant-puppet/manifests && puppet apply --modulepath '/etc/puppet/modules:/tmp/vagrant-puppet/modules-0' node1.pp --detailed-exitcodes || [ $? -eq 2 ]


the node1 is running, but seen from the above error message, it seemed to fail to initialize ganeti cluster.
# vagrant status
Current machine states:

node1                    running (virtualbox)
node2                    not created (virtualbox)
node3                    not created (virtualbox)

start node2 and node3 as well.

# vagrant up node2
# vagrant up node3

three nodes, node1, node2, and node3 are running.
# vagrant status
Current machine states:

node1                    running (virtualbox)
node2                    running (virtualbox)
node3                    running (virtualbox)

log onto the node1 over SSH.
root password is vagrant.
# vagrant ssh node1
Welcome to Ubuntu 12.04.3 LTS (GNU/Linux 3.8.0-29-generic x86_64)

* Documentation:  https://help.ubuntu.com/
Last login: Mon Nov  4 22:39:17 2013 from 10.0.2.2
vagrant@node1:~$ su -
Password:
#

root@node1:~# egrep -v ^# /etc/hosts
127.0.0.1 localhost
33.33.33.11 packer-ubuntu-12.04 packer-ubuntu-12

::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
33.33.33.13 node3.example.org node3
33.33.33.32 instance2.example.org instance2
33.33.33.33 instance3.example.org instance3
33.33.33.31 instance1.example.org instance1
33.33.33.34 instance4.example.org instance4
33.33.33.10 ganeti.example.org ganeti
33.33.33.12 node2.example.org node2

root@node1:~# ping -c 1 node2
PING node2.example.org (33.33.33.12) 56(84) bytes of data.
64 bytes from node2.example.org (33.33.33.12): icmp_req=1 ttl=64 time=1.54 ms

--- node2.example.org ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.547/1.547/1.547/0.000 ms

root@node1:~# ping -c 1 node3
PING node3.example.org (33.33.33.13) 56(84) bytes of data.
64 bytes from node3.example.org (33.33.33.13): icmp_req=1 ttl=64 time=1.60 ms

--- node3.example.org ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.603/1.603/1.603/0.000 ms





about the error “Error: /vagrant/modules/ganeti_tutorial/files/scripts/initialize-ganeti returned 1 instead of one of [0]” while issuing “vagrant up node1”...

look into the script file to find the root cause.

on the node1, find initialize-ganeti file.
#  updatedb
#  locate initialize-ganeti

# locate initialize-ganeti
/vagrant/modules/ganeti_tutorial/files/scripts/initialize-ganeti
/vagrant/modules/ganeti_tutorial/nodes/node1/initialize-ganeti.pp

# cd /vagrant/modules/ganeti_tutorial/files/scripts/

here is the script file.
seen from the script, the VM(node1) try to use kvm as its hypervisors
but I unloaded kvm and kvm_amd modules on the virtualbox host...
# cat initialize-ganeti
#!/bin/bash
GANETI_VERSION="$(gnt-cluster --version | awk '{print $4}')"
EXTRA_OPTS=

/bin/sed -i -e 's/node.*localhost /localhost /' /etc/hosts
/bin/sed -i -e 's/node.*localhost\t/localhost /' /etc/hosts

if [[ "$GANETI_VERSION" > "2.6.0" ]] ; then
   EXTRA_OPTS="--disk-parameters drbd:metavg=ganeti $EXTRA_OPTS"
fi

/usr/local/sbin/gnt-cluster init \
   --vg-name=ganeti -s 33.33.34.11 \
   --master-netdev=br0 \
   -I hail \
   -H kvm:kernel_path=/boot/vmlinuz-kvmU,initrd_path=/boot/initrd-kvmU,root_path=/dev/sda2,nic_type=e1000,disk_type=scsi,vnc_bind_address=0.0.0.0,serial_console=true \
   -N link=br0 --enabled-hypervisors=kvm \
   $EXTRA_OPTS \
   ganeti.example.org

/usr/local/sbin/gnt-cluster modify --reserved-lvs=ganeti/swap

on the node1
# gnt-node list
Failure: prerequisites not met for this operation:
Cluster not initialized or this machine is not part of a cluster

how about initializing cluster?
on the node1
already initialized..
root@node1:~# gnt-cluster init cluster01
Failure: prerequisites not met for this operation:
error type: wrong_state, error details:
Cluster is already initialised

add the nodes.
on the node1

root@node1:~# gnt-node add -s 33.33.34.12 node2
Failure: prerequisites not met for this operation:
Cluster not initialized or this machine is not part of a cluster
root@node1:~#

root@node1:~# gnt-node add -s 33.33.34.13 node3
Failure: prerequisites not met for this operation:
Cluster not initialized or this machine is not part of a cluster

shutdown three nodes and load kvm modules and then run three node3 again.
I am not sure whether or not there are conflicts between kvm modules and virtualbox modules..

on the virtualbox host.
# vagrant halt node1
# vagrant halt node2
# vagrant halt node3

# modprobe kvm
# modprobe kvm_intel

root@ubuntu:~# lsmod | grep kvm
kvm_amd                59958  0
kvm                   431315  1 kvm_amd

root@ubuntu:~# lsmod | grep vbox
vboxpci                23194  0
vboxnetadp             25670  0
vboxnetflt             27613  2
vboxdrv               320455  4 vboxnetadp,vboxnetflt,vboxpci
# vagrant up node1
# vagrant up node2
# vagrant up node3

log onto the node1.
nnn, can’t add nodes in the cluster..
root@ubuntu:/var/disk1/Vagrant_works/vagrant-ganeti# vagrant ssh node1
Welcome to Ubuntu 12.04.3 LTS (GNU/Linux 3.8.0-29-generic x86_64)

* Documentation:  https://help.ubuntu.com/
Last login: Sat Nov 16 15:51:40 2013 from 10.0.2.2
vagrant@node1:~$ su -
Password:

root@node1:~# gnt-node add -s 33.33.34.12 node2
Failure: prerequisites not met for this operation:
Cluster not initialized or this machine is not part of a cluster

root@node1:~# grep svm /proc/cpuinfo
root@node1:~#

root@node1:~# gnt-cluster info
Failure: prerequisites not met for this operation:
Cluster not initialized or this machine is not part of a cluster

root@node1:~# kvm-ok
INFO: Your CPU does not support KVM extensions
KVM acceleration can NOT be used

I guess that I need nested VM in this environment, however virtualbox does not seem to support nested VM even though virtualbox host supports VT.

it might work if I use Xen as ganeti node’s hypervisor instead of KVM.

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.