error logs..
I tried to deploy ganeti nodes within virtualbox host, however I can’t do that, because virtualbox does not support nested VMs..
here are environments I tried.
- one virtualbox host ( support VT ). ubuntu 13.10
- install ganeti within virtualbox host by using vagrant.
- ganeti nodes use KVM as its hypervisor . ( but virtualbox does not support nested VM, so ganeti nodes can not use KVM.)
virtualbox host
# tail -1 /etc/lsb-release ;uname -ri
DISTRIB_DESCRIPTION="Ubuntu 13.10"
3.11.0-13-generic x86_64
|
referred to
I have been using KVM as a hypervisor.
To install ganeti with vagrant, you need virtualbox.
So I unloaded KVM related kernel modules to avoid conflicts between KVM and virtualbox and then start virtualbox to install ganeti within virtualbox.
# rmmod kvm_amd
# rmmod kvm
|
install virtualbox and vagrant
# apt-get install virtualbox vagrant
|
# /etc/init.d/virtualbox start
|
# lsmod | grep vbox*
vboxpci 23194 0
vboxnetadp 25670 0
vboxnetflt 27613 0
vboxdrv 320455 3 vboxnetadp,vboxnetflt,vboxpci
#
# lsmod | grep kvm
#
|
When I want to switch back to KVM from virtualbox, stop virtualbox and then load KVM kernel modules as below.
# /etc/init.d/virtualbox stop
# modprobe kvm
# modprobe kvm_amd
# lsmod | grep kvm
kvm_amd 59958 0
kvm 431315 1 kvm_amd
# rmmod kvm_amd
# rmmod kvm
# /etc/init.d/virtualbox restart
|
Let’s install ganeti!
# git clone https://github.com/ramereth/vagrant-ganeti.git
# cd vagrant-ganeti/
# git submodule update --init
|
no VMs are running.
# vagrant status
Current machine states:
node1 not created (virtualbox)
node2 not created (virtualbox)
node3 not created (virtualbox)
|
start the node1.
error..
# vagrant up node1
Notice: /Stage[main]/Ganeti_tutorial::Ganeti::Initialize/Exec[initialize-ganeti]/returns: Corrupt configuration file: The configuration denotes node packer-ubuntu-12.04 as master, while my hostname is node1.example.org; opening a foreign configuration is only possible in accept_foreign mode
Notice: /Stage[main]/Ganeti_tutorial::Ganeti::Initialize/Exec[initialize-ganeti]/returns: Aborting.
Notice: /Stage[main]/Ganeti_tutorial::Ganeti::Initialize/Exec[initialize-ganeti]/returns: Failure: prerequisites not met for this operation:
Notice: /Stage[main]/Ganeti_tutorial::Ganeti::Initialize/Exec[initialize-ganeti]/returns: Cluster not initialized or this machine is not part of a cluster
Error: /vagrant/modules/ganeti_tutorial/files/scripts/initialize-ganeti returned 1 instead of one of [0]
Error: /Stage[main]/Ganeti_tutorial::Ganeti::Initialize/Exec[initialize-ganeti]/returns: change from notrun to 0 failed: /vagrant/modules/ganeti_tutorial/files/scripts/initialize-ganeti returned 1 instead of one of [0]
Notice: Finished catalog run in 86.96 seconds
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!
cd /tmp/vagrant-puppet/manifests && puppet apply --modulepath '/etc/puppet/modules:/tmp/vagrant-puppet/modules-0' node1.pp --detailed-exitcodes || [ $? -eq 2 ]
|
the node1 is running, but seen from the above error message, it seemed to fail to initialize ganeti cluster.
# vagrant status
Current machine states:
node1 running (virtualbox)
node2 not created (virtualbox)
node3 not created (virtualbox)
|
start node2 and node3 as well.
# vagrant up node2
# vagrant up node3
|
three nodes, node1, node2, and node3 are running.
# vagrant status
Current machine states:
node1 running (virtualbox)
node2 running (virtualbox)
node3 running (virtualbox)
|
log onto the node1 over SSH.
root password is vagrant.
# vagrant ssh node1
Welcome to Ubuntu 12.04.3 LTS (GNU/Linux 3.8.0-29-generic x86_64)
* Documentation: https://help.ubuntu.com/
Last login: Mon Nov 4 22:39:17 2013 from 10.0.2.2
vagrant@node1:~$ su -
Password:
#
root@node1:~# egrep -v ^# /etc/hosts
127.0.0.1 localhost
33.33.33.11 packer-ubuntu-12.04 packer-ubuntu-12
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
33.33.33.13 node3.example.org node3
33.33.33.32 instance2.example.org instance2
33.33.33.33 instance3.example.org instance3
33.33.33.31 instance1.example.org instance1
33.33.33.34 instance4.example.org instance4
33.33.33.10 ganeti.example.org ganeti
33.33.33.12 node2.example.org node2
root@node1:~# ping -c 1 node2
PING node2.example.org (33.33.33.12) 56(84) bytes of data.
64 bytes from node2.example.org (33.33.33.12): icmp_req=1 ttl=64 time=1.54 ms
--- node2.example.org ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.547/1.547/1.547/0.000 ms
root@node1:~# ping -c 1 node3
PING node3.example.org (33.33.33.13) 56(84) bytes of data.
64 bytes from node3.example.org (33.33.33.13): icmp_req=1 ttl=64 time=1.60 ms
--- node3.example.org ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.603/1.603/1.603/0.000 ms
|
about the error “Error: /vagrant/modules/ganeti_tutorial/files/scripts/initialize-ganeti returned 1 instead of one of [0]” while issuing “vagrant up node1”...
look into the script file to find the root cause.
on the node1, find initialize-ganeti file.
# updatedb
# locate initialize-ganeti
# locate initialize-ganeti
/vagrant/modules/ganeti_tutorial/files/scripts/initialize-ganeti
/vagrant/modules/ganeti_tutorial/nodes/node1/initialize-ganeti.pp
# cd /vagrant/modules/ganeti_tutorial/files/scripts/
|
here is the script file.
seen from the script, the VM(node1) try to use kvm as its hypervisors
but I unloaded kvm and kvm_amd modules on the virtualbox host...
# cat initialize-ganeti
#!/bin/bash
GANETI_VERSION="$(gnt-cluster --version | awk '{print $4}')"
EXTRA_OPTS=
/bin/sed -i -e 's/node.*localhost /localhost /' /etc/hosts
/bin/sed -i -e 's/node.*localhost\t/localhost /' /etc/hosts
if [[ "$GANETI_VERSION" > "2.6.0" ]] ; then
EXTRA_OPTS="--disk-parameters drbd:metavg=ganeti $EXTRA_OPTS"
fi
/usr/local/sbin/gnt-cluster init \
--vg-name=ganeti -s 33.33.34.11 \
--master-netdev=br0 \
-I hail \
-H kvm:kernel_path=/boot/vmlinuz-kvmU,initrd_path=/boot/initrd-kvmU,root_path=/dev/sda2,nic_type=e1000,disk_type=scsi,vnc_bind_address=0.0.0.0,serial_console=true \
-N link=br0 --enabled-hypervisors=kvm \
$EXTRA_OPTS \
ganeti.example.org
/usr/local/sbin/gnt-cluster modify --reserved-lvs=ganeti/swap
|
on the node1
# gnt-node list
Failure: prerequisites not met for this operation:
Cluster not initialized or this machine is not part of a cluster
|
how about initializing cluster?
on the node1
already initialized..
root@node1:~# gnt-cluster init cluster01
Failure: prerequisites not met for this operation:
error type: wrong_state, error details:
Cluster is already initialised
|
add the nodes.
on the node1
root@node1:~# gnt-node add -s 33.33.34.12 node2
Failure: prerequisites not met for this operation:
Cluster not initialized or this machine is not part of a cluster
root@node1:~#
root@node1:~# gnt-node add -s 33.33.34.13 node3
Failure: prerequisites not met for this operation:
Cluster not initialized or this machine is not part of a cluster
|
shutdown three nodes and load kvm modules and then run three node3 again.
I am not sure whether or not there are conflicts between kvm modules and virtualbox modules..
on the virtualbox host.
# vagrant halt node1
# vagrant halt node2
# vagrant halt node3
# modprobe kvm
# modprobe kvm_intel
root@ubuntu:~# lsmod | grep kvm
kvm_amd 59958 0
kvm 431315 1 kvm_amd
root@ubuntu:~# lsmod | grep vbox
vboxpci 23194 0
vboxnetadp 25670 0
vboxnetflt 27613 2
vboxdrv 320455 4 vboxnetadp,vboxnetflt,vboxpci
# vagrant up node1
# vagrant up node2
# vagrant up node3
|
log onto the node1.
nnn, can’t add nodes in the cluster..
root@ubuntu:/var/disk1/Vagrant_works/vagrant-ganeti# vagrant ssh node1
Welcome to Ubuntu 12.04.3 LTS (GNU/Linux 3.8.0-29-generic x86_64)
* Documentation: https://help.ubuntu.com/
Last login: Sat Nov 16 15:51:40 2013 from 10.0.2.2
vagrant@node1:~$ su -
Password:
root@node1:~# gnt-node add -s 33.33.34.12 node2
Failure: prerequisites not met for this operation:
Cluster not initialized or this machine is not part of a cluster
root@node1:~# grep svm /proc/cpuinfo
root@node1:~#
root@node1:~# gnt-cluster info
Failure: prerequisites not met for this operation:
Cluster not initialized or this machine is not part of a cluster
root@node1:~# kvm-ok
INFO: Your CPU does not support KVM extensions
KVM acceleration can NOT be used
|
I guess that I need nested VM in this environment, however virtualbox does not seem to support nested VM even though virtualbox host supports VT.
it might work if I use Xen as ganeti node’s hypervisor instead of KVM.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.