Here are logs when setting up Openstack with Juju and MAAS with one physical server.
Juju 2.0.0
MAAS 2.1.2
one physical server
install MAAS and juju within LXD container.
run nodes for MAAS within KVM.
Assume you already setup MAAS.
http://lost-and-found-narihiro.blogspot.jp/2016/11/ubuntu-1604-install-maas-within-ubuntu.html
[ MAAS network ]
all nodes(Maas node, m-node01..) have two subnets, 192.168.40.0/24 and 192.168.41.0/24.
192.168.40.0/24 is private network and 192.168.41.0/24 is external network
- 192.168.40.0/24
enable DHCP
MAAS DHCP leases IP addresses to nodes, but does not provide a default gateway address.
This network is used for PXE boot.
Gateway is blank.
- 192.168.41.0/24
disable DHCP
[ network configuration of MAAS nodes ]
eth0 : 192.168.40.0/24 ( auto-assign, assign a static IP without DHCP )
eth1 : 192.168.41./24 ( auto-assign, assign a static IP without DHCP )
[ install juju on MAAS node ]
$ lxc exec maas01 bash
root@maas01:~# apt install juju
|
Register MAAS as cloud provider.
When I specified 127.0.0.1 as an endpoint, I failed to bootstrap juju controller.
root@maas01:~# cat maas-clouds.yaml
clouds:
my-maas:
type: maas
auth-types: [oauth1]
endpoint: http://192.168.40.10/MAAS
|
root@maas01:~# juju add-cloud my-maas maas-clouds.yaml
|
get an API key and add that credentials
root@maas01:~# maas-region apikey --username=admin
root@maas01:~# juju add-credential my-maas
|
Prepare five or more “Ready” state nodes on MAAS.
nodes has two disks, vda, vdb.
bootstrap controller.
One of nodes managed by MAAS will be picked up for “my-maas-controller”
root@maas01:~# juju bootstrap my-maas my-maas-controller --constraints "tags=virtual"
|
juju-gui will be installed in this controller.
Confirm that you can access to Juju GUI.
root@maas01:~# juju gui --no-browser
https://192.168.42.116:17070/gui/90178a27-bd4a-40d3-85a1-8f5217313579/
root@maas01:~# juju show-controller --show-password
|
Reference
https://jujucharms.com/openstack-base/
download oepnstack charm
root@maas01:~# wget https://api.jujucharms.com/charmstore/v5/openstack-base/archive
root@maas01:~# unzip archive
|
edit bundle.yaml
I changed /dev/sdb to /dev/vdb for ceph storage.
# grep vdb bundle.yaml
osd-devices: /dev/vdb
|
eth1 is used for an external network.
# grep eth bundle.yaml
data-port: br-ex:eth1
|
deploy openstack.
root@maas01:~# juju deploy bundl.yaml
|
done.
# juju status
Model Controller Cloud/Region Version
default my-maas-controller my-maas 2.0.0
App Version Status Scale Charm Store Rev OS Notes
ceph-mon 10.2.3 active 3 ceph-mon jujucharms 6 ubuntu
ceph-osd 10.2.3 active 3 ceph-osd jujucharms 238 ubuntu
ceph-radosgw 10.2.3 active 1 ceph-radosgw jujucharms 245 ubuntu
cinder 9.0.0 active 1 cinder jujucharms 257 ubuntu
cinder-ceph 9.0.0 active 1 cinder-ceph jujucharms 221 ubuntu
glance 13.0.0 active 1 glance jujucharms 253 ubuntu
keystone 10.0.0 active 1 keystone jujucharms 258 ubuntu
mysql 5.6.21-25.8 active 1 percona-cluster jujucharms 246 ubuntu
neutron-api 9.0.0 active 1 neutron-api jujucharms 246 ubuntu
neutron-gateway 9.0.0 active 1 neutron-gateway jujucharms 232 ubuntu
neutron-openvswitch 9.0.0 active 3 neutron-openvswitch jujucharms 238 ubuntu
nova-cloud-controller 14.0.1 active 1 nova-cloud-controller jujucharms 292 ubuntu
nova-compute 14.0.1 active 3 nova-compute jujucharms 259 ubuntu
ntp unknown 4 ntp jujucharms 0 ubuntu
openstack-dashboard 10.0.0 active 1 openstack-dashboard jujucharms 243 ubuntu
rabbitmq-server 3.5.7 active 1 rabbitmq-server jujucharms 54 ubuntu
Unit Workload Agent Machine Public address Ports Message
ceph-mon/0* active idle 1/lxd/0 192.168.40.68 Unit is ready and clustered
ceph-mon/1 active idle 2/lxd/0 192.168.40.74 Unit is ready and clustered
ceph-mon/2 active idle 3/lxd/0 192.168.40.69 Unit is ready and clustered
ceph-osd/0 active idle 1 192.168.40.63 Unit is ready (1 OSD)
ceph-osd/1* active idle 2 192.168.40.64 Unit is ready (1 OSD)
ceph-osd/2 active idle 3 192.168.40.65 Unit is ready (1 OSD)
ceph-radosgw/0* active idle 0/lxd/0 192.168.40.66 80/tcp Unit is ready
cinder/0* active idle 1/lxd/1 192.168.40.71 8776/tcp Unit is ready
cinder-ceph/0* active idle 192.168.40.71 Unit is ready
glance/0* active idle 2/lxd/1 192.168.40.75 9292/tcp Unit is ready
keystone/0* active idle 3/lxd/1 192.168.40.70 5000/tcp Unit is ready
mysql/0* active idle 0/lxd/1 192.168.40.76 Unit is ready
neutron-api/0* active idle 1/lxd/2 192.168.40.73 9696/tcp Unit is ready
neutron-gateway/0* active idle 0 192.168.40.62 Unit is ready
ntp/3 unknown idle 192.168.40.62
nova-cloud-controller/0* active idle 2/lxd/2 192.168.40.67 8774/tcp Unit is ready
nova-compute/0 active idle 1 192.168.40.63 Unit is ready
neutron-openvswitch/2 active idle 192.168.40.63 Unit is ready
ntp/2 unknown idle 192.168.40.63
nova-compute/1* active idle 2 192.168.40.64 Unit is ready
neutron-openvswitch/0* active idle 192.168.40.64 Unit is ready
ntp/0* unknown idle 192.168.40.64
nova-compute/2 active idle 3 192.168.40.65 Unit is ready
neutron-openvswitch/1 active idle 192.168.40.65 Unit is ready
ntp/1 unknown idle 192.168.40.65
openstack-dashboard/0* active idle 3/lxd/2 192.168.40.72 80/tcp,443/tcp Unit is ready
rabbitmq-server/0* active idle 0/lxd/2 192.168.40.77 5672/tcp Unit is ready
Machine State DNS Inst id Series AZ
0 started 192.168.40.62 snntex xenial default
0/lxd/0 started 192.168.40.66 juju-2e35f2-0-lxd-0 xenial
0/lxd/1 started 192.168.40.76 juju-2e35f2-0-lxd-1 xenial
0/lxd/2 started 192.168.40.77 juju-2e35f2-0-lxd-2 xenial
1 started 192.168.40.63 b76s6q xenial default
1/lxd/0 started 192.168.40.68 juju-2e35f2-1-lxd-0 xenial
1/lxd/1 started 192.168.40.71 juju-2e35f2-1-lxd-1 xenial
1/lxd/2 started 192.168.40.73 juju-2e35f2-1-lxd-2 xenial
2 started 192.168.40.64 pgrb77 xenial default
2/lxd/0 started 192.168.40.74 juju-2e35f2-2-lxd-0 xenial
2/lxd/1 started 192.168.40.75 juju-2e35f2-2-lxd-1 xenial
2/lxd/2 started 192.168.40.67 juju-2e35f2-2-lxd-2 xenial
3 started 192.168.40.65 wa8m3e xenial default
3/lxd/0 started 192.168.40.69 juju-2e35f2-3-lxd-0 xenial
3/lxd/1 started 192.168.40.70 juju-2e35f2-3-lxd-1 xenial
3/lxd/2 started 192.168.40.72 juju-2e35f2-3-lxd-2 xenial
Relation Provides Consumes Type
mon ceph-mon ceph-mon peer
mon ceph-mon ceph-osd regular
mon ceph-mon ceph-radosgw regular
ceph ceph-mon cinder-ceph regular
ceph ceph-mon glance regular
ceph ceph-mon nova-compute regular
cluster ceph-radosgw ceph-radosgw peer
identity-service ceph-radosgw keystone regular
cluster cinder cinder peer
storage-backend cinder cinder-ceph subordinate
image-service cinder glance regular
identity-service cinder keystone regular
shared-db cinder mysql regular
cinder-volume-service cinder nova-cloud-controller regular
amqp cinder rabbitmq-server regular
cluster glance glance peer
identity-service glance keystone regular
shared-db glance mysql regular
image-service glance nova-cloud-controller regular
image-service glance nova-compute regular
amqp glance rabbitmq-server regular
cluster keystone keystone peer
shared-db keystone mysql regular
identity-service keystone neutron-api regular
identity-service keystone nova-cloud-controller regular
identity-service keystone openstack-dashboard regular
cluster mysql mysql peer
shared-db mysql neutron-api regular
shared-db mysql nova-cloud-controller regular
cluster neutron-api neutron-api peer
neutron-plugin-api neutron-api neutron-gateway regular
neutron-plugin-api neutron-api neutron-openvswitch regular
neutron-api neutron-api nova-cloud-controller regular
amqp neutron-api rabbitmq-server regular
cluster neutron-gateway neutron-gateway peer
quantum-network-service neutron-gateway nova-cloud-controller regular
juju-info neutron-gateway ntp subordinate
amqp neutron-gateway rabbitmq-server regular
neutron-plugin neutron-openvswitch nova-compute regular
amqp neutron-openvswitch rabbitmq-server regular
cluster nova-cloud-controller nova-cloud-controller peer
cloud-compute nova-cloud-controller nova-compute regular
amqp nova-cloud-controller rabbitmq-server regular
neutron-plugin nova-compute neutron-openvswitch subordinate
compute-peer nova-compute nova-compute peer
juju-info nova-compute ntp subordinate
amqp nova-compute rabbitmq-server regular
ntp-peers ntp ntp peer
cluster openstack-dashboard openstack-dashboard peer
cluster rabbitmq-server rabbitmq-server peer
root@maas01:~/MAAS_works#
|
install openstack clients tools
source novarc.
root@maas01:~/MAAS_works# apt -y install python-novaclient python-keystoneclient python-glanceclient python-neutronclient python-openstackclient
root@maas01:~/MAAS_works# source novarc
root@maas01:~/MAAS_works# keystone catalog
root@maas01:~/MAAS_works# openstack service list
|
add an image.
root@maas01:~/images# curl http://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-disk1.img -o xenial-server-cloudimg-amd64-disk1.img
root@maas01:~/images# openstack image create --disk-format qcow2 --container-format bare --public --file ./xenial-server-cloudimg-amd64-disk1.img xenial
root@maas01:~/images# openstack image list
+--------------------------------------+--------+--------+
| ID | Name | Status |
+--------------------------------------+--------+--------+
| 1ee4452f-4a4b-425e-9c58-a794ef3ad9af | xenial | active |
+--------------------------------------+--------+--------+
|
Configure network
There are two scripts, neutron-ext-net and neutron-tenant-net, which are included in charm bundle.
Create an external network.
root@maas01:~/MAAS_works# ./neutron-ext-net -g 192.168.41.1 -c 192.168.41.0/24 -f 192.168.41.30:192.168.41.60 ext_net
INFO:root:Configuring external bridge
INFO:root:Creating new external network definition: ext_net
INFO:root:New external network created: 3d84978f-9307-40b9-bc67-703cdce3db05
INFO:root:Creating new subnet for ext_net
INFO:root:New subnet created: 4565f32a-18ad-4f44-ac31-74c69d4b9fca
INFO:root:Creating provider router for external network access
INFO:root:New router created: 08be5112-1846-4e0f-b40c-bb7617070c54
INFO:root:Plugging router into ext_net
INFO:root:Router connected to ext_net
root@maas01:~/MAAS_works#
|
Create a tenant network for admin.
root@maas01:~/MAAS_works# ./neutron-tenant-net -t admin -r provider-router -N 8.8.8.8 internal 10.5.5.0/24
INFO:root:Creating network: internal
INFO:root:Creating subnet for internal
INFO:root:Updating dns_nameservers (8.8.8.8) for subnet internal_subnet
INFO:root:Adding interface from provider-router to internal_subnet
root@maas01:~/MAAS_works#
|
root@maas01:~/MAAS_works# openstack network list
+--------------------------------------+----------+--------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+----------+--------------------------------------+
| 3d84978f-9307-40b9-bc67-703cdce3db05 | ext_net | 4565f32a-18ad-4f44-ac31-74c69d4b9fca |
| 8756b5d2-2cef-43b8-8b46-5f9ffecd3530 | internal | 42683930-be5f-4b51-b7e0-5adad7e03533 |
+--------------------------------------+----------+--------------------------------------+
root@maas01:~/MAAS_works#
|
Boot an instance
root@maas01:~# nova keypair-add --pub-key ~/.ssh/id_rsa.pub mykey
root@maas01:~# nova boot --image xenial --flavor m1.small --key-name mykey --nic net-id=$(neutron net-list | grep internal | awk '{ print $2 }') xenial-test
ERROR (CommandError): No flavor with a name or ID of 'm1.small' exists.
|
error.
root@maas01:~# openstack flavor list
root@maas01:~#
|
create a flavor
root@maas01:~# openstack
(openstack) flavor create --public --vcpus 1 --ram 2048 --disk 20 m1.small
+----------------------------+--------------------------------------+
| Field | Value |
+----------------------------+--------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| disk | 20 |
| id | 0b71fd72-cdd6-40cc-81eb-ec8492cb22ea |
| name | m1.small |
| os-flavor-access:is_public | True |
| ram | 2048 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 1 |
+----------------------------+--------------------------------------+
(openstack)
|
boot an instance
root@maas01:~# nova boot --image xenial --flavor m1.small --key-name mykey --nic net-id=$(neutron net-list | grep internal
| awk '{ print $2 }') xenial-test
|
error
(openstack) server show xenial-test
| fault | {u'message': u'Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance a8386797-1216-447a-9cef-f03c700fcf46. Last exception: invalid argument: could not find capabilities for domaintype=kvm ', u'code': 500, u'details': u' File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 480, in build_instances\n filter_properties, instances[0].uuid)\n File "/usr/lib/python2.7/dist-packages/nova/scheduler/utils.py", line 184, in populate_retry\n raise exception.MaxRetriesExceeded(reason=msg)\n', u'created': u'2017-01-07T03:58:54Z'} |
|
login to nova-compute.
problem is caused by KVM configuration on nova compute VMs which are running within KVM.
root@maas01:~# juju ssh nova-compute/0
ubuntu@m-node08:~$ sudo kvm-ok
INFO: Your CPU does not support KVM extensions
KVM acceleration can NOT be used
|
on the KVM host
$ cat /sys/module/kvm_intel/parameters/nested
Y
|
root@maas01:~# juju ssh nova-compute/0
ubuntu@m-node08:~$ sudo systemctl poweroff
|
edit XML
$ virsh edit m-node08
$ virsh dumpxml m-node08 | grep host-pass
<cpu mode='host-passthrough'/>
$ virsh start m-node08
|
root@maas01:~# for i in `seq 0 2`;do juju ssh nova-compute/$i kvm-ok;done
INFO: /dev/kvm exists
KVM acceleration can be used
Connection to 192.168.40.63 closed.
INFO: /dev/kvm exists
KVM acceleration can be used
Connection to 192.168.40.64 closed.
INFO: /dev/kvm exists
KVM acceleration can be used
Connection to 192.168.40.65 closed.
root@maas01:~# juju status nova-compute
Model Controller Cloud/Region Version
default my-maas-controller my-maas 2.0.0
App Version Status Scale Charm Store Rev OS Notes
ceph-osd 10.2.3 active 3 ceph-osd jujucharms 238 ubuntu
neutron-openvswitch 9.0.0 active 3 neutron-openvswitch jujucharms 238 ubuntu
nova-compute 14.0.1 active 3 nova-compute jujucharms 259 ubuntu
ntp unknown 3 ntp jujucharms 0 ubuntu
|
boot an instance.
root@maas01:~# nova boot --image xenial --flavor m1.small --key-name mykey --nic net-id=$(neutron net-list | grep internal
| awk '{ print $2 }') xenial-test
|
root@maas01:~# openstack server show 04989262-8522-4650-9fe9-9c9e922627c6
+--------------------------------------+----------------------------------------------------------+
| Field | Value |
+--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | m-node11 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | m-node11.maas |
| OS-EXT-SRV-ATTR:instance_name | instance-00000002 |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2017-01-07T04:14:28.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | internal=10.5.5.14 |
| config_drive | |
| created | 2017-01-07T04:14:21Z |
| flavor | m1.small (0b71fd72-cdd6-40cc-81eb-ec8492cb22ea) |
| hostId | 7f16269230c396200c2cdb3849f1ac849cf888692d0ede4a16ec749f |
| id | 04989262-8522-4650-9fe9-9c9e922627c6 |
| image | xenial (1ee4452f-4a4b-425e-9c58-a794ef3ad9af) |
| key_name | mykey |
| name | xenial-test |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| project_id | 5215dfbd6f294fba8d3164911b7acb6c |
| properties | |
| security_groups | [{u'name': u'default'}] |
| status | ACTIVE |
| updated | 2017-01-07T04:14:29Z |
| user_id | 88cffdb9e48f4729896e9ad8c042fa36 |
+--------------------------------------+----------------------------------------------------------+
|
allow access vi SSH and ping.
(neutron) security-group-rule-create --protocol icmp --direction ingress default
(neutron) security-group-rule-create --protocol tcp --port-range-min 22 --port-range-max 22 --direction ingress default
|
send ping to the instance from neutron gateway.
root@maas01:~# juju ssh neutron-gateway/0 ip netns
qdhcp-8756b5d2-2cef-43b8-8b46-5f9ffecd3530 (id: 4)
qrouter-08be5112-1846-4e0f-b40c-bb7617070c54 (id: 3)
root@maas01:~# juju ssh neutron-gateway/0 sudo ip netns exec qrouter-08be5112-1846-4e0f-b40c-bb7617070c54 ip -4 a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: qg-c63ed59f-0a@if25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link-netnsid 0
inet 192.168.40.33/24 brd 192.168.40.255 scope global qg-c63ed59f-0a
valid_lft forever preferred_lft forever
inet 192.168.40.30/32 brd 192.168.40.30 scope global qg-c63ed59f-0a
valid_lft forever preferred_lft forever
3: qr-a0fc15b8-65@if27: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc noqueue state UP group default qlen 1000 link-netnsid 0
inet 10.5.5.1/24 brd 10.5.5.255 scope global qr-a0fc15b8-65
valid_lft forever preferred_lft forever
root@maas01:~# juju ssh neutron-gateway/0 sudo ip netns exec qrouter-08be5112-1846-4e0f-b40c-bb7617070c54 ping -c 3 10.5.5.14
PING 10.5.5.14 (10.5.5.14) 56(84) bytes of data.
64 bytes from 10.5.5.14: icmp_seq=1 ttl=64 time=0.893 ms
64 bytes from 10.5.5.14: icmp_seq=2 ttl=64 time=0.712 ms
64 bytes from 10.5.5.14: icmp_seq=3 ttl=64 time=0.822 ms
|
floating ip
(neutron) noroot@maas01:~# nova list
+--------------------------------------+-------------+--------+------------+-------------+--------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------------+--------+------------+-------------+--------------------+
| 04989262-8522-4650-9fe9-9c9e922627c6 | xenial-test | ACTIVE | - | Running | internal=10.5.5.14 |
+--------------------------------------+-------------+--------+------------+-------------+--------------------+
root@maas01:~# neutron port-list -- --device_id 04989262-8522-4650-9fe9-9c9e922627c6
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------+
| 1da4aec5-3362-4c43-9ba5-6fae6c50282e | | fa:16:3e:83:1b:7f | {"subnet_id": "42683930-be5f-4b51-b7e0-5adad7e03533", "ip_address": "10.5.5.14"} |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------+
root@maas01:~# neutron floatingip-create ext_net
root@maas01:~# neutron floatingip-list
+--------------------------------------+------------------+---------------------+---------+
| id | fixed_ip_address | floating_ip_address | port_id |
+--------------------------------------+------------------+---------------------+---------+
| 90351e78-53aa-4d28-b9d6-26996256c882 | | 192.168.40.30 | |
| ed54e29d-f5f0-4a34-a4b6-7950d604fcf4 | | 192.168.40.35 | |
+--------------------------------------+------------------+---------------------+---------+
root@maas01:~# neutron floatingip-associate 90351e78-53aa-4d28-b9d6-26996256c882 1da4aec5-3362-4c43-9ba5-6fae6c50282e
Associated floating IP 90351e78-53aa-4d28-b9d6-26996256c882
root@maas01:~# neutron floatingip-show 90351e78-53aa-4d28-b9d6-26996256c882
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| created_at | 2017-01-07T04:57:34Z |
| description | |
| fixed_ip_address | 10.5.5.14 |
| floating_ip_address | 192.168.40.30 |
| floating_network_id | 3d84978f-9307-40b9-bc67-703cdce3db05 |
| id | 90351e78-53aa-4d28-b9d6-26996256c882 |
| port_id | 1da4aec5-3362-4c43-9ba5-6fae6c50282e |
| project_id | 5215dfbd6f294fba8d3164911b7acb6c |
| revision_number | 2 |
| router_id | 08be5112-1846-4e0f-b40c-bb7617070c54 |
| status | ACTIVE |
| tenant_id | 5215dfbd6f294fba8d3164911b7acb6c |
| updated_at | 2017-01-07T05:01:05Z |
+---------------------+--------------------------------------+
|