Reference
https://greatbsky.github.io/kolla-for-openstack-in-docker/en.html
https://docs.openstack.org/kolla-ansible/latest/quickstart.html
Here are logs when I set up OpenStack multi nodes with kolla.
Build docker images from source, push images to docker private registry, deploy OpenStack with kolla-ansible.
Prepare four nodes
one for docker private registry
one for controller node.
the other two are for compute
All nodes are based on Ubuntu 16.04.
[ network ]
all nodes are belong to 192.168.100.0/24
- hostname : docker-reg
one NIC.
eth0 : 192.168.100.10
- hostname : controller
two NICs
ens3 : 192.168.100.11
ens7 : interface is up, but no ip address
root@controller:~# grep -v ^# /etc/network/interfaces.d/50-cloud-init.cfg
auto lo
iface lo inet loopback
auto ens3
iface ens3 inet static
address 192.168.100.11
netmask 255.255.255.0
gateway 192.168.100.1
dns-nameservers 8.8.8.8 8.8.4.4
auto ens7
iface ens7 inet manual
|
- hostname : node01
tow NICs
ens3 : 192.168.100.12
ens7 : interface is up, but no ip address
- hostname : node02
two NICs
ens3 : 192.168.100.13
ens7 : interface is up, but no ip address
all nodes have a network reachability with hostname. ( edit /etc/hosts )
[ set up docker private registry ]
install docker and run docker private registry.
root@docker-reg:~# docker run -d -p 5000:5000 --restart=always -v /data/Iaas/Docker/Volumes/Registry:/var/lib/registry --na
me registry registry:2
|
root@docker-reg:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
97cd11afc792 registry:2 "/entrypoint.sh /etc/" About a minute ago Up About a minute 0.0.0.0:5000->5000/tcp registry
|
[ enable password-less ssh access among nodes (controller, node01, node02)]
on the controller
root@controller:~# ssh-keygen
root@controller:~# ssh-copy-id -i .ssh/id_rsa.pub root@controller
root@controller:~# ssh-copy-id -i .ssh/id_rsa.pub root@node01
root@controller:~# ssh-copy-id -i .ssh/id_rsa.pub root@node02
|
do the same thing on both node01 and node02.
[ install docker on controller, node01, node02 ]
root@controller:~# curl -ksSL https://get.docker.io | bash
root@node01:~# curl -ksSL https://get.docker.io | bash
root@node02:~# curl -ksSL https://get.docker.io | bash
|
[ uninstall lxd on controller, node01, node02 ]
# apt purge lxd -y
|
[ edit systemd config for docker. ( controller, node01, node02 ) ]
root@controller:~# systemctl edif --full docker
|
[Service]
ExecStart=/usr/bin/dockerd -H fd:// --insecure-registry 192.168.100.10:5000
MountFlags=shared
|
restart docker
root@controller:~# systemctl restart docker
|
[ install ntp on controller, node01, node02 ]
root@node02:~# apt install ntp -y
|
[ install ansible, kolla-* on controller node ]
root@controller:~# apt install python-pip -y
root@controller:~# pip install ansible
root@controller:~# apt-get install python-dev libffi-dev gcc libssl-dev python-selinux
root@controller:~# pip install kolla-ansible
|
# Note that you also need to install docker-py on node01, and node02.
# do the following on node01 and node02.
# apt install python-pip, pip install docker-py
[ disable libvirt on controller, node01, node02 if it’s running ]
25 systemctl stop libvirt-bin
26 systemctl disable libvirt-bin
|
[ install docker-py on node01, node02 ]
# apt install python-pip
# pip install docker-py
|
[ build docker images on the controller node and push images to the private registry ]
root@controller:~# cp -r /usr/local/share/kolla-ansible/etc_examples/kolla /etc/kolla/
root@controller:~# cp /usr/local/share/kolla-ansible/ansible/inventory/* .
root@controller:~# cp /etc/kolla/globals.yml /etc/kolla/globals.yml.orig
root@controller:~# cp multinode multinode.orig
root@controller:~# ls
all-in-one multinode multinode.orig
|
edit globals.yml
root@controller:~# grep -v ^# /etc/kolla/globals.yml | grep -v ^$
---
kolla_base_distro: "ubuntu"
kolla_install_type: "source"
kolla_internal_vip_address: "192.168.100.11" # controller node IP
enable_haproxy: "no"
docker_registry: "192.168.100.10:5000"
network_interface: "ens3"
neutron_external_interface: "ens7"
designate_backend: "bind9"
designate_ns_record: "sample.openstack.org"
tempest_image_id:
tempest_flavor_ref_id:
tempest_public_network_id:
tempest_floating_network_name:
|
edit multimode file.
root@controller:~# grep -v ^# multinode | grep -v ^$
[control]
controller
[network]
controller
[compute]
node01
node02
[monitoring]
controller
[storage]
controller
[baremetal:children]
control
network
compute
storage
monitoring
[chrony-server:children]
control
[chrony:children]
control
network
compute
storage
monitoring
[collectd:children]
compute
[grafana:children]
monitoring
[etcd:children]
control
[influxdb:children]
monitoring
[karbor:children]
control
[kibana:children]
control
[telegraf:children]
compute
control
monitoring
network
storage
[elasticsearch:children]
control
[haproxy:children]
network
[mariadb:children]
control
[rabbitmq:children]
control
[mongodb:children]
control
[keystone:children]
control
[glance:children]
control
[nova:children]
control
[neutron:children]
network
[cinder:children]
control
[cloudkitty:children]
control
[freezer:children]
control
[memcached:children]
control
[horizon:children]
control
[swift:children]
control
[barbican:children]
control
[heat:children]
control
[murano:children]
control
[solum:children]
control
[ironic:children]
control
[ceph:children]
control
[magnum:children]
control
[sahara:children]
control
[mistral:children]
control
[manila:children]
control
[ceilometer:children]
control
[aodh:children]
control
[congress:children]
control
[panko:children]
control
[gnocchi:children]
control
[tacker:children]
control
[trove:children]
control
[tempest:children]
control
[senlin:children]
control
[vmtp:children]
control
[watcher:children]
control
[rally:children]
control
[searchlight:children]
control
[octavia:children]
control
[designate:children]
control
[placement:children]
control
[glance-api:children]
glance
[glance-registry:children]
glance
[nova-api:children]
nova
[nova-conductor:children]
nova
[nova-consoleauth:children]
nova
[nova-novncproxy:children]
nova
[nova-scheduler:children]
nova
[nova-spicehtml5proxy:children]
nova
[nova-compute-ironic:children]
nova
[nova-serialproxy:children]
nova
[neutron-server:children]
control
[neutron-dhcp-agent:children]
neutron
[neutron-l3-agent:children]
neutron
[neutron-lbaas-agent:children]
neutron
[neutron-metadata-agent:children]
neutron
[neutron-vpnaas-agent:children]
neutron
[ceph-mon:children]
ceph
[ceph-rgw:children]
ceph
[ceph-osd:children]
storage
[cinder-api:children]
cinder
[cinder-backup:children]
storage
[cinder-scheduler:children]
cinder
[cinder-volume:children]
storage
[cloudkitty-api:children]
cloudkitty
[cloudkitty-processor:children]
cloudkitty
[freezer-api:children]
freezer
[iscsid:children]
compute
storage
ironic-conductor
[tgtd:children]
storage
[karbor-api:children]
karbor
[karbor-protection:children]
karbor
[karbor-operationengine:children]
karbor
[manila-api:children]
manila
[manila-scheduler:children]
manila
[manila-share:children]
network
[manila-data:children]
manila
[swift-proxy-server:children]
swift
[swift-account-server:children]
storage
[swift-container-server:children]
storage
[swift-object-server:children]
storage
[barbican-api:children]
barbican
[barbican-keystone-listener:children]
barbican
[barbican-worker:children]
barbican
[heat-api:children]
heat
[heat-api-cfn:children]
heat
[heat-engine:children]
heat
[murano-api:children]
murano
[murano-engine:children]
murano
[ironic-api:children]
ironic
[ironic-conductor:children]
ironic
[ironic-inspector:children]
ironic
[ironic-pxe:children]
ironic
[magnum-api:children]
magnum
[magnum-conductor:children]
magnum
[sahara-api:children]
sahara
[sahara-engine:children]
sahara
[solum-api:children]
solum
[solum-worker:children]
solum
[solum-deployer:children]
solum
[solum-conductor:children]
solum
[mistral-api:children]
mistral
[mistral-executor:children]
mistral
[mistral-engine:children]
mistral
[ceilometer-api:children]
ceilometer
[ceilometer-central:children]
ceilometer
[ceilometer-notification:children]
ceilometer
[ceilometer-collector:children]
ceilometer
[ceilometer-compute:children]
compute
[aodh-api:children]
aodh
[aodh-evaluator:children]
aodh
[aodh-listener:children]
aodh
[aodh-notifier:children]
aodh
[congress-api:children]
congress
[congress-datasource:children]
congress
[congress-policy-engine:children]
congress
[panko-api:children]
panko
[gnocchi-api:children]
gnocchi
[gnocchi-statsd:children]
gnocchi
[gnocchi-metricd:children]
gnocchi
[trove-api:children]
trove
[trove-conductor:children]
trove
[trove-taskmanager:children]
trove
[multipathd:children]
compute
[watcher-api:children]
watcher
[watcher-engine:children]
watcher
[watcher-applier:children]
watcher
[senlin-api:children]
senlin
[senlin-engine:children]
senlin
[searchlight-api:children]
searchlight
[searchlight-listener:children]
searchlight
[octavia-api:children]
octavia
[octavia-health-manager:children]
octavia
[octavia-housekeeping:children]
octavia
[octavia-worker:children]
octavia
[designate-api:children]
designate
[designate-central:children]
designate
[designate-mdns:children]
designate
[designate-worker:children]
designate
[designate-sink:children]
designate
[designate-backend-bind9:children]
designate
[placement-api:children]
placement
root@controller:~#
|
Generate passwords ( /etc/kolla/passwords.yml )
root@controller:~# kolla-genpwd
|
build images on the controller node and push images to the docker private registry.
root@controller:~# kolla-build --base ubuntu --type source --registry 192.168.100.10:5000 --push
|
Deploy OpenStack.
root@controller:~# kolla-ansible deploy -i ./multimode
TASK [Gathering Facts] ****************************************************************************************************
ok: [controller]
TASK [octavia : include] **************************************************************************************************
skipping: [controller]
PLAY RECAP ****************************************************************************************************************
controller : ok=225 changed=48 unreachable=0 failed=0
node01 : ok=60 changed=23 unreachable=0 failed=0
node02 : ok=60 changed=23 unreachable=0 failed=0
|
on controller node.
root@controller:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7253548dbeef 192.168.100.10:5000/kolla/ubuntu-source-horizon:4.0.2 "kolla_start" 3 minutes ago Up 3 minutes horizon
ddf85fd52511 192.168.100.10:5000/kolla/ubuntu-source-heat-engine:4.0.2 "kolla_start" 3 minutes ago Up 3 minutes heat_engine
95ae0c6a5e10 192.168.100.10:5000/kolla/ubuntu-source-heat-api-cfn:4.0.2 "kolla_start" 3 minutes ago Up 3 minutes heat_api_cfn
c404c7b0dd54 192.168.100.10:5000/kolla/ubuntu-source-heat-api:4.0.2 "kolla_start" 3 minutes ago Up 3 minutes heat_api
81e91bb064b3 192.168.100.10:5000/kolla/ubuntu-source-neutron-metadata-agent:4.0.2 "kolla_start" 4 minutes ago Up 4 minutes neutron_metadata_agent
c6ad418b90b0 192.168.100.10:5000/kolla/ubuntu-source-neutron-l3-agent:4.0.2 "kolla_start" 4 minutes ago Up 4 minutes neutron_l3_agent
42db0499ea32 192.168.100.10:5000/kolla/ubuntu-source-neutron-dhcp-agent:4.0.2 "koll
|
on node01
root@node01:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
27629bfbf3c1 192.168.100.10:5000/kolla/ubuntu-source-neutron-openvswitch-agent:4.0.2 "kolla_start" 4 minutes ago Up 4 minutes neutron_openvswitch_agent
aeaa732e4a72 192.168.100.10:5000/kolla/ubuntu-source-openvswitch-vswitchd:4.0.2 "kolla_start" 5 minutes ago Up 4 minutes openvswitch_vswitchd
17bbef6a2838 192.168.100.10:5000/kolla/ubuntu-source-openvswitch-db-server:4.0.2 "kolla_start" 5 minutes ago Up 5 minutes openvswitch_db
5e59bf46dfd9 192.168.100.10:5000/kolla/ubuntu-source-nova-compute:4.0.2 "kolla_start" 10 minutes ago Up 10 minutes nova_compute
|
on node02
root@node02:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e8128ddf26f1 192.168.100.10:5000/kolla/ubuntu-source-neutron-openvswitch-agent:4.0.2 "kolla_start" 5 minutes ago Up 5 minutes neutron_openvswitch_agent
108c2dcfcf63 192.168.100.10:5000/kolla/ubuntu-source-openvswitch-vswitchd:4.0.2 "kolla_start" 5 minutes ago Up 5 minutes openvswitch_vswitchd
09b7ceabd1e6 192.168.100.10:5000/kolla/ubuntu-source-openvswitch-db-server:4.0.2 "kolla_start" 5 minutes ago Up 5 minutes openvswitch_db
f6f534c61b22 192.168.100.10:5000/kolla/ubuntu-source-nova-compute:4.0.2 "kolla_start" 10 minutes ago Up 10 minutes nova_compute
|
root@controller:~# kolla-ansible post-deploy
root@controller:~# ls /etc/kolla/admin-openrc.sh
/etc/kolla/admin-openrc.sh
|
root@controller:~# pip install python-openstackclient
|
root@controller:~# source /etc/kolla/admin-openrc.sh
root@controller:~# openstack service list
+----------------------------------+-------------+----------------+
| ID | Name | Type |
+----------------------------------+-------------+----------------+
| 0b727b6a11fc456495814b4381b38021 | nova_legacy | compute_legacy |
| 222bc4a9269d44adad625574acbeb94d | heat-cfn | cloudformation |
| 2714d8f44c8d4391b4f2329ff1cd0e16 | heat | orchestration |
| 66c1c2f6a2cd42c9a67647f61defc8d6 | neutron | network |
| 67b757a4b3934a3cb7e9ef1ab65d8356 | glance | image |
| 696e48b1c7294bd28fa0017d69e26b88 | nova | compute |
| c4d8e948d9c14c9f8541d4f77bc361f9 | placement | placement |
| ef384109b76b4932bab8a43af38d4d40 | keystone | identity |
+----------------------------------+-------------+----------------+
root@controller:~#
|
add an image, create a network with a script.
root@controller:~# cd /usr/local/share/kolla-ansible/
root@controller:/usr/local/share/kolla-ansible# cp init-runonce init-runonce.orig
root@controller:/usr/local/share/kolla-ansible# diff init-runonce init-runonce.orig
11,13c11,13
< EXT_NET_CIDR='192.168.100.0/24'
< EXT_NET_RANGE='start=192.168.100.120,end=192.168.100.150'
< EXT_NET_GATEWAY='192.168.100.1'
---
> EXT_NET_CIDR='10.0.2.0/24'
> EXT_NET_RANGE='start=10.0.2.150,end=10.0.2.199'
> EXT_NET_GATEWAY='10.0.2.1'
|
run the script.
root@controller:/usr/local/share/kolla-ansible# source /etc/kolla/admin-openrc.sh
root@controller:/usr/local/share/kolla-ansible# ./init-runonce
Done.
To deploy a demo instance, run:
openstack server create \
--image cirros \
--flavor m1.tiny \
--key-name mykey \
--nic net-id=abf504e6-761d-4abc-b0a9-fd884c7122ca \
demo1
|
access to the dashboard. ( controller’s IP )
you can find a user name and credentials at /etc/kolla/admin-openrc.sh.
on the controller node.
bring up br-ex.
root@controller:~# ip a s br-ex
728: br-ex: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1
link/ether 3a:10:1d:8a:44:48 brd ff:ff:ff:ff:ff:ff
root@controller:~# ip link set br-ex up
root@controller:~# ip a s br-ex
728: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1
link/ether 3a:10:1d:8a:44:48 brd ff:ff:ff:ff:ff:ff
inet6 fe80::3810:1dff:fe8a:4448/64 scope link
valid_lft forever preferred_lft forever
root@controller:~#
|
launch two instances.
on the controller.
root@controller:~# source /etc/kolla/admin-openrc.sh
root@controller:~# openstack server list
+--------------------------------------+------------+--------+--------------------+--------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------------+--------+--------------------+--------+---------+
| 0d3228af-722d-4728-a86e-a826f3f3cb82 | instance02 | ACTIVE | demo-net=10.0.0.6 | cirros | m1.tiny |
| 0dad6a28-29f8-411f-b0af-853cc67849d7 | instance01 | ACTIVE | demo-net=10.0.0.12 | cirros | m1.tiny |
+--------------------------------------+------------+--------+--------------------+--------+---------+
root@controller:~#
|
on the node01
root@node01:~# docker ps | grep libvirt
64deaef784ea 192.168.100.10:5000/kolla/ubuntu-source-nova-libvirt:4.0.2 "kolla_start" 18 hours ago Up 18 hours nova_libvirt
root@node01:~#
root@node01:~# docker exec -t -i -u root 64deaef784ea virsh list
Id Name State
----------------------------------------------------
1 instance-00000002 running
root@node01:~#
|
on the node02
root@node02:~# docker ps | grep libvirt
e845a1b8ea93 192.168.100.10:5000/kolla/ubuntu-source-nova-libvirt:4.0.2 "kolla_start" 18 hours ago Up 18 hours nova_libvirt
root@node02:~# docker exec -t -i -u root e845a1b8ea93 virsh list
Id Name State
----------------------------------------------------
1 instance-00000001 running
root@node02:~#
|
access to the neutron l3 agent.
on the controller
root@controller:~# docker ps | grep l3
c6ad418b90b0 192.168.100.10:5000/kolla/ubuntu-source-neutron-l3-agent:4.0.2 "kolla_start" 21 hours ago Up 21 hours neutron_l3_agent
root@controller:~# docker exec -i -t -u root c6ad418b90b0 bash
(neutron-l3-agent)[root@controller /]# ip netns list
qrouter-804b79fc-426c-425b-8c71-6ba99db33b11
qdhcp-abf504e6-761d-4abc-b0a9-fd884c7122ca
(neutron-l3-agent)[root@controller /]#
root@controller:~# docker exec -i -t -u root c6ad418b90b0 bash
(neutron-l3-agent)[root@controller /]# apt install iputils-ping
(neutron-l3-agent)[root@controller /]# ip netns exec qrouter-804b79fc-426c-425b-8c71-6ba99db33b11 ping 10.0.0.6
64 bytes from 10.0.0.6: icmp_seq=1 ttl=64 time=2.02 ms
64 bytes from 10.0.0.6: icmp_seq=2 ttl=64 time=0.700 ms
|
add floating IPs
associate those floating IPs with two instances.
send pings to floating IPs.
root@controller:~# ping 192.168.100.129 -c 1
PING 192.168.100.129 (192.168.100.129) 56(84) bytes of data.
64 bytes from 192.168.100.129: icmp_seq=1 ttl=63 time=2.32 ms
--- 192.168.100.129 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.320/2.320/2.320/0.000 ms
root@controller:~#
root@controller:~# ping 192.168.100.125 -c 1
PING 192.168.100.125 (192.168.100.125) 56(84) bytes of data.
64 bytes from 192.168.100.125: icmp_seq=1 ttl=63 time=2.24 ms
--- 192.168.100.125 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.246/2.246/2.246/0.000 ms
root@controller:~#
|
SSH access via floating IP.
root@controller:~# ssh 192.168.100.125 -l cirros
$ hostname
instance01
$ ping 10.0.0.6 -c 3
PING 10.0.0.6 (10.0.0.6): 56 data bytes
64 bytes from 10.0.0.6: seq=0 ttl=64 time=1.728 ms
64 bytes from 10.0.0.6: seq=1 ttl=64 time=1.124 ms
64 bytes from 10.0.0.6: seq=2 ttl=64 time=0.964 ms
--- 10.0.0.6 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.964/1.272/1.728 ms
$
$ exit
Connection to 192.168.100.125 closed.
root@controller:~#
|
To destroy an environment, enter:
root@controller:~# kolla-ansible destroy --yes-i-really-really-mean-it
|
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.