lost and found ( for me ? )

install proxmox with Fedora KVM ( nested VM )

Here’s an explanation of how to install proxmox within KVM including trial and error logs.

KVM host info ( fedora 18 )

I’m using virt-preview repository.

[root@f18 ~]# qemu-kvm --version
QEMU emulator version 1.4.0, Copyright (c) 2003-2008 Fabrice Bellard

[root@f18 ~]# libvirtd --version
libvirtd (libvirt) 1.0.3

[root@f18 ~]# cat /etc/fedora-release
Fedora release 18 (Spherical Cow)

[root@f18 ~]# uname -ri
3.8.2-206.fc18.x86_64 x86_64

[root@f18 ~]# qemu-kvm --version
QEMU emulator version 1.4.0, Copyright (c) 2003-2008 Fabrice Bellard
[root@f18 ~]# libvirtd --version
libvirtd (libvirt) 1.0.3
[root@f18 ~]# cat /etc/fedora-release
Fedora release 18 (Spherical Cow)
[root@f18 ~]# uname -ri
3.8.2-206.fc18.x86_64 x86_64
[root@f18 ~]# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 16
model : 4
model name : AMD Phenom(tm) II X4 925 Processor
stepping : 2
microcode : 0x10000db
cpu MHz : 800.000
cache size : 512 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt hw_pstate npt lbrv svm_lock nrip_save
bogomips : 5599.67
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

processor : 1
vendor_id : AuthenticAMD
cpu family : 16
model : 4
model name : AMD Phenom(tm) II X4 925 Processor
stepping : 2
microcode : 0x10000db
cpu MHz : 800.000
cache size : 512 KB
physical id : 0
siblings : 4
core id : 1
cpu cores : 4
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt hw_pstate npt lbrv svm_lock nrip_save
bogomips : 5599.67
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate


[ install proxmox with Fedora KVM ]

Here is an xml file for proxmox

don’t use virtio
[root@f18 ~]# virsh dumpxml proxmox-vm
<domain type='kvm'>
 <name>proxmox-vm</name>
 <uuid>8265cd7a-7307-340f-538f-d6aa69f79cee</uuid>
 <memory unit='KiB'>1048576</memory>
 <currentMemory unit='KiB'>1048576</currentMemory>
 <vcpu placement='static'>1</vcpu>
 <os>
   <type arch='x86_64' machine='pc-1.2'>hvm</type>
   <boot dev='hd'/>
 </os>
 <features>
   <acpi/>
   <apic/>
   <pae/>
 </features>
 <cpu mode='custom' match='exact'>
   <model fallback='allow'>Opteron_G3</model>
   <vendor>AMD</vendor>
   <feature policy='require' name='skinit'/>
   <feature policy='require' name='vme'/>
   <feature policy='require' name='mmxext'/>
   <feature policy='require' name='fxsr_opt'/>
   <feature policy='require' name='cr8legacy'/>
   <feature policy='require' name='ht'/>
   <feature policy='require' name='3dnowprefetch'/>
   <feature policy='require' name='3dnowext'/>
   <feature policy='require' name='wdt'/>
   <feature policy='require' name='extapic'/>
   <feature policy='require' name='pdpe1gb'/>
   <feature policy='require' name='osvw'/>
   <feature policy='require' name='ibs'/>
   <feature policy='require' name='cmp_legacy'/>
   <feature policy='require' name='3dnow'/>
   <feature policy='force' name='svm'/>
 </cpu>
 <clock offset='utc'/>
 <on_poweroff>destroy</on_poweroff>
 <on_reboot>restart</on_reboot>
 <on_crash>restart</on_crash>
 <devices>
   <emulator>/usr/bin/qemu-kvm</emulator>
   <disk type='file' device='disk'>
     <driver name='qemu' type='raw'/>
     <source file='/var/lib/libvirt/images/proxmox-vm.img'/>
     <target dev='hda' bus='ide'/>
     <address type='drive' controller='0' bus='0' target='0' unit='0'/>
   </disk>
   <disk type='block' device='cdrom'>
     <driver name='qemu' type='raw'/>
     <target dev='hdc' bus='ide'/>
     <readonly/>
     <address type='drive' controller='0' bus='1' target='0' unit='0'/>
   </disk>
   <controller type='usb' index='0'>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
   </controller>
   <controller type='ide' index='0'>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
   </controller>
   <interface type='network'>
     <mac address='52:54:00:c6:ac:dd'/>
     <source network='network1'/>
     <model type='e1000'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
   </interface>
   <serial type='pty'>
     <target port='0'/>
   </serial>
   <console type='pty'>
     <target type='serial' port='0'/>
   </console>
   <input type='mouse' bus='ps2'/>
   <graphics type='vnc' port='-1' autoport='yes' keymap='ja'/>
   <video>
     <model type='cirrus' vram='9216' heads='1'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
   </video>
   <memballoon model='virtio'>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
   </memballoon>
 </devices>
</domain>

[root@f18 ~]#



install proxmox VM using an ISO file
[root@f18 ~]# ls /home/hattori/ISO_images/proxmox-ve_2.2-7f9cfa4c-28.iso


boot the proxmox VM
[root@f18 ~]# virsh start proxmox-vm
Domain proxmox-vm started


launch browser and access to the VM


[ eploy a qcow2 image within the proxmox VM ]

To deploy an existing qcow2 image , create a blank VM.

click “create VM”
on the “CD/DVD” tab , choose “do not use any media “

check a path to the VM image
on the proxmox VM
root@proxmox:/var# cat /etc/pve/nodes/proxmox/qemu-server/100.conf
bootdisk: virtio0
cores: 1
ide2: none,media=cdrom
memory: 512
name: ubuntu1204-vm1
net0: virtio=72:37:2E:C2:92:D7,bridge=vmbr0
ostype: l26
sockets: 1
virtio0: local:100/vm-100-disk-1.raw,size=10G


Where is the path to vm-100-disk-1.raw ?

check the local storage path.
Folder View -> DataCenter -> Storage.
The path to the local storage seems to be /var/lib/vz.

The path to the above VM image seems to be “/var/lib/vz/images/100/vm-100-disk-1.raw”
root@proxmox:/var/lib# file /var/lib/vz/images/100/vm-100-disk-1.raw
/var/lib/vz/images/100/vm-100-disk-1.raw: data


copy the qcow2 image to the proxmox VM over scp.

on the KVM host.
[root@f18 KVM_images]# scp ubuntu-1204-vm1.qcow2 root@192.168.20.152:/var/lib/vz/images/100


on the proxmox VM
root@proxmox:/var/lib/vz/images/100# pwd
/var/lib/vz/images/100
root@proxmox:/var/lib/vz/images/100# ls
ubuntu-1204-vm1.qcow2  vm-100-disk-1.raw
root@proxmox:/var/lib/vz/images/100# cp vm-100-disk-1.raw vm-100-disk-1.raw.bak
root@proxmox:/var/lib/vz/images/100# mv ubuntu-1204-vm1.qcow2 vm-100-disk-1.raw


start the VM via proxmox GUI
nn error.
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
No accelerator found!
TASK ERROR: start failed: command '/usr/bin/kvm -id 100 -chardev 'socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/100.vnc,x509,password -pidfile /var/run/qemu-server/100.pid -daemonize -name ubuntu1204-vm1 -smp 'sockets=1,cores=1' -nodefaults -boot 'menu=on' -vga cirrus -k ja -m 512 -usbdevice tablet -drive 'if=none,id=drive-ide2,media=cdrom,aio=native' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=/var/lib/vz/images/100/vm-100-disk-1.raw,if=none,id=drive-virtio0,aio=native,cache=none' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,vhost=on' -device 'virtio-net-pci,mac=72:37:2E:C2:92:D7,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300'' failed: exit code 1


proxmox VM does not load KVM module..
load kvm_amd with modprobe.
nn , error.
root@proxmox:~# lsmod | grep kvm
kvm                   318658  0

root@proxmox:~# lsmod | grep kvm
kvm                   318658  0
root@proxmox:~# modinfo kvm_amd
filename:       /lib/modules/2.6.32-16-pve/kernel/arch/x86/kvm/kvm-amd.ko
license:        GPL
author:         Qumranet
srcversion:     43D8067144E7D8B0D53D46E
depends:        kvm
vermagic:       2.6.32-16-pve SMP mod_unload modversions
parm:           npt:int
parm:           nested:int
root@proxmox:~# modprobe kvm_amd
FATAL: Error inserting kvm_amd (/lib/modules/2.6.32-16-pve/kernel/arch/x86/kvm/kvm-amd.ko): Operation not supported
root@proxmox:~#


check the vCPU info of proxmox VM
smv flag is enabled on proxmox VM..
Fedora KVM’s xml file for the proxmox VM is wrong ?
root@proxmox:~# modinfo kvm_amd
filename:       /lib/modules/2.6.32-16-pve/kernel/arch/x86/kvm/kvm-amd.ko
license:        GPL
author:         Qumranet
srcversion:     43D8067144E7D8B0D53D46E
depends:        kvm
vermagic:       2.6.32-16-pve SMP mod_unload modversions
parm:           npt:int
parm:           nested:int
root@proxmox:~# modprobe kvm_amd
FATAL: Error inserting kvm_amd (/lib/modules/2.6.32-16-pve/kernel/arch/x86/kvm/kvm-amd.ko): Operation not supported
root@proxmox:~#
root@proxmox:~# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 6
model name : AMD Opteron 23xx (Gen 3 Class Opteron)
stepping : 1
cpu MHz : 2799.998
cache size : 512 KB
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb lm 3dnowext 3dnow up rep_good extd_apicid unfair_spinlock pni cx16 popcnt hypervisor cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw
bogomips : 5599.99
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:


shutdown the proxmox VM
root@proxmox:~# init 0


on the Fedora KVM
check cpu_map.xml file to check what type of CPU map  we can define in the xml file.
[root@f18 ~]# cat /usr/share/libvirt/cpu_map.xml

or

[root@f18 ~]# qemu-kvm -cpu \?
x86           qemu64  QEMU Virtual CPU version 1.4.0                  
x86           phenom  AMD Phenom(tm) 9550 Quad-Core Processor         
x86         core2duo  Intel(R) Core(TM)2 Duo CPU     T7700  @ 2.40GHz
x86            kvm64  Common KVM processor                            
x86           qemu32  QEMU Virtual CPU version 1.4.0                  
x86            kvm32  Common 32-bit KVM processor                     
x86          coreduo  Genuine Intel(R) CPU           T2600  @ 2.16GHz
x86              486                                                  
x86          pentium                                                  
x86         pentium2                                                  
x86         pentium3                                                  
x86           athlon  QEMU Virtual CPU version 1.4.0                  
x86             n270  Intel(R) Atom(TM) CPU N270   @ 1.60GHz          
x86           Conroe  Intel Celeron_4x0 (Conroe/Merom Class Core 2)   
x86           Penryn  Intel Core 2 Duo P9xxx (Penryn Class Core 2)    
x86          Nehalem  Intel Core i7 9xx (Nehalem Class Core i7)       
x86         Westmere  Westmere E56xx/L56xx/X56xx (Nehalem-C)          
x86      SandyBridge  Intel Xeon E312xx (Sandy Bridge)                
x86          Haswell  Intel Core Processor (Haswell)                  
x86       Opteron_G1  AMD Opteron 240 (Gen 1 Class Opteron)           
x86       Opteron_G2  AMD Opteron 22xx (Gen 2 Class Opteron)          
x86       Opteron_G3  AMD Opteron 23xx (Gen 3 Class Opteron)          
x86       Opteron_G4  AMD Opteron 62xx class CPU                      
x86       Opteron_G5  AMD Opteron 63xx class CPU                      

Recognized CPUID flags:
 pbe ia64 tm ht ss sse2 sse fxsr mmx acpi ds clflush pn pse36 pat cmov mca pge mtrr sep apic cx8 mce pae msr tsc pse de vme fpu
 hypervisor rdrand f16c avx osxsave xsave aes tsc-deadline popcnt movbe x2apic sse4.2|sse4_2 sse4.1|sse4_1 dca pcid pdcm xtpr cx16 fma cid ssse3 tm2 est smx vmx ds_cpl monitor dtes64 pclmulqdq|pclmuldq pni|sse3
 3dnow 3dnowext lm|i64 rdtscp pdpe1gb fxsr_opt|ffxsr mmxext nx|xd syscall
 perfctr_nb perfctr_core topoext tbm nodeid_msr tce fma4 lwp wdt skinit xop ibs osvw 3dnowprefetch misalignsse sse4a abm cr8legacy extapic svm cmp_legacy lahf_lm
[root@f18 ~]#

or

[root@f18 ~]# virsh capabilities | head -10
<capabilities>

 <host>
   <uuid>ed1acc3c-5fd6-11df-bc36-8c970d9d0b55</uuid>
   <cpu>
     <arch>x86_64</arch>
     <model>Opteron_G3</model>
     <vendor>AMD</vendor>
     <topology sockets='1' cores='4' threads='1'/>
     <feature name='wdt'/>




edit the xml file. how about this ?
[root@f18 ~]# virsh dumpxml proxmox-vm
<domain type='kvm' id='4'>
 <name>proxmox-vm</name>
 <uuid>8265cd7a-7307-340f-538f-d6aa69f79cee</uuid>
 <memory unit='KiB'>1048576</memory>
 <currentMemory unit='KiB'>1048576</currentMemory>
 <vcpu placement='static'>1</vcpu>
 <os>
   <type arch='x86_64' machine='pc-1.2'>hvm</type>
   <boot dev='hd'/>
 </os>
 <features>
   <acpi/>
   <apic/>
   <pae/>
 </features>
 <cpu mode='custom' match='exact'>
   <model fallback='allow'>phenom</model>
 </cpu>


start the VM
[root@f18 ~]# virsh start proxmox-vm
Domain proxmox-vm started


log onto the proxmox VM
okay , the proxmox VM has loaded kvm_amd module.
root@proxmox:~# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : AMD Phenom(tm) 9550 Quad-Core Processor
stepping : 3
cpu MHz : 2799.998
cache size : 512 KB
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb lm 3dnowext 3dnow up rep_good extd_apicid unfair_spinlock pni cx16 popcnt hypervisor lahf_lm svm abm sse4a npt
bogomips : 5599.99
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

root@proxmox:~#
root@proxmox:~# lsmod | grep kvm
kvm_amd                41454  0
kvm                   318658  1 kvm_amd
root@proxmox:~#



start the ubuntu VM via proxmox GUI
the ubuntu VM is running within the proxmox VM which is running within Fedora KVM :D

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.