lost and found ( for me ? )

Ubuntu 16.04 : set up Openstack all-in-one with kolla ( Openstack in Docker )

Reference
https://docs.openstack.org/developer/kolla-ansible/quickstart.html
http://qiita.com/lychee3/items/e0a57c833450654006a5

I was able to set up Openstack with the following versions.

Host OS : Ubuntu 16.04
Docker version 17.06.1-ce
OpenStack Docker images : CentOS docker images
kolla-ansible 4.0.0
ansible 2.3.2.0

gateway   ---------------- VM ( two NICs, ens3, ens7 )
192.168.153.1

ens3 and ens7 belong to the same network ( 192.168.153.0/24)
ens3 has an IP, ens7 does not have an IP.

This VM has two interfaces, ens3 and ens7.

ens3 has an ip address and ens7 does not have an IP.
ubuntu@kolla:~$ ip a s ens3 | grep inet
   inet 192.168.153.10/24 brd 192.168.153.255 scope global ens3
   inet6 fe80::5054:ff:fe53:507c/64 scope link

ubuntu@kolla:~$ ip r g 8.8.8.8
8.8.8.8 via 192.168.153.1 dev ens3  src 192.168.153.10
   cache

bring up ens7 ( no ip address )
$ sudo ip link set ens7 up

ubuntu@kolla:~$ sudo apt install python-pip -y

become a root user.
ubuntu@kolla:~$ sudo su -
root@kolla:~#
root@kolla:~# pip install -U pip
root@kolla:~# apt install -y python-dev libffi-dev gcc libssl-dev
root@kolla:~# pip install ansible

root@kolla:~# ansible --version
ansible 2.3.2.0
 config file =
 configured module search path = Default w/o overrides
 python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

install docker
root@kolla:~# curl -sSL https://get.docker.io | bash

root@kolla:~# docker --version
Docker version 17.06.1-ce, build 874a737

root@kolla:~# mkdir -p /etc/systemd/system/docker.service.d

root@kolla:~# tee /etc/systemd/system/docker.service.d/kolla.conf <<-'EOF'
> [Service]
> MountFlags=shared
> EOF
[Service]
MountFlags=shared
root@kolla:~#
root@kolla:~# cat /etc/systemd/system/docker.service.d/kolla.conf
[Service]
MountFlags=shared
root@kolla:~#

root@kolla:~# systemctl daemon-reload
root@kolla:~# systemctl restart docker

root@kolla:~# pip install -U docker-py
root@kolla:~# apt install -y ntp

stop libvirt if it is running
root@kolla:~# service libvirt-bin stop
Failed to stop libvirt-bin.service: Unit libvirt-bin.service not loaded.
root@kolla:~# update-rc.d libvirt-bin disable
update-rc.d: error: cannot find a LSB script for libvirt-bin
root@kolla:~# sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd
File /etc/apparmor.d/usr.sbin.libvirtd not found, skipping...
root@kolla:~#

uninstall lxd, lxc
root@kolla:~# apt purge lxd lxc -y
root@kolla:~# reboot
I could not set up openstack with kola-ansible 4.0.1/4.0.2.
root@kolla:~# pip install kolla-ansible\==4.0.0

root@kolla:~# cp -r /usr/local/share/kolla-ansible/etc_examples/kolla /etc/kolla/
root@kolla:~# cp /usr/local/share/kolla-ansible/ansible/inventory/* .
root@kolla:~# ls
all-in-one  multinode
root@kolla:~#

edit global.yaml.
I could not set up openstack with Ubuntu docker images, so I used centos images.
root@kolla:~# grep -v ^# /etc/kolla/globals.yml | grep -v ^$
---
kolla_base_distro: "centos"
kolla_install_type: "binary"
kolla_internal_vip_address: "192.168.153.10"
enable_haproxy: "no"
network_interface: "ens3"
neutron_external_interface: "ens7"
designate_backend: "bind9"
designate_ns_record: "sample.openstack.org"
tempest_image_id:
tempest_flavor_ref_id:
tempest_public_network_id:
tempest_floating_network_name:

generate passwords
root@kolla:~# kolla-genpwd
root@kolla:~#

confirm your CPU supports virtualization assist.
root@kolla:~# egrep -c '(vmx|svm)' /proc/cpuinfo
4

ok.
root@kolla:~# kolla-ansible prechecks -i ./all-in-one

PLAY RECAP ****************************************************************************************************************
localhost                  : ok=126  changed=0    unreachable=0    failed=0

pull images
# kolla-ansible pull

deploy
root@kolla:~# kolla-ansible deploy -i ./all-in-one

root@kolla:~# docker ps -a
CONTAINER ID        IMAGE                                                 COMMAND             CREATED             STATUS              PORTS               NAMES
0a5861451eef        kolla/centos-binary-horizon:4.0.0                     "kolla_start"       15 hours ago        Up 15 hours                             horizon
cfd88266848e        kolla/centos-binary-heat-engine:4.0.0                 "kolla_start"       15 hours ago        Up 15 hours                             heat_engine
ca27b4950479        kolla/centos-binary-heat-api-cfn:4.0.0                "kolla_start"       15 hours ago        Up 15 hours                             heat_api_cfn
3308fe21492b        kolla/centos-binary-heat-api:4.0.0                    "kolla_start"       15 hours ago        Up 15 hours                             heat_api

Generate admin rc
root@kolla:~# kolla-ansible post-deploy

you can find username and password to login the dashboard.
root@kolla:~# cat /etc/kolla/admin-openrc.sh

# pip install python-openstackclient

root@kolla:~# . /etc/kolla/admin-openrc.sh

# openstack service list
+----------------------------------+-------------+----------------+
| ID                               | Name        | Type           |
+----------------------------------+-------------+----------------+
| 0cc53d757d8640ad93e54b6e435b4127 | neutron     | network        |
| 1b030881656f425d8894072f6e61be3e | placement   | placement      |
| 344f1198ca2642208438ff30dedf76ef | nova_legacy | compute_legacy |
| 3aeba6a5500d4da2a5b3672bd3166798 | heat-cfn    | cloudformation |
| 65f56634c2c3403a88dd2b12358e6901 | glance      | image          |
| 9b2d72a773eb42e2a0645966b607fd19 | keystone    | identity       |
| 9c05d7d777e243529db945074a862f66 | nova        | compute        |
| 9dc38157cead489bb721d846d88ed543 | heat        | orchestration  |

editi a script to meet your environment.
This script will download an iamge, create a network etc.
root@kolla:~# cd /usr/local/share/kolla-ansible

root@kolla:/usr/local/share/kolla-ansible# diff init-runonce init-runonce.orig
11,13c11,13
< EXT_NET_CIDR='192.168.153.0/24'
< EXT_NET_RANGE='start=192.168.153.100,end=192.168.153.130'
< EXT_NET_GATEWAY='192.168.153.1'
---
> EXT_NET_CIDR='10.0.2.0/24'
> EXT_NET_RANGE='start=10.0.2.150,end=10.0.2.199'
> EXT_NET_GATEWAY='10.0.2.1'

run the script
root@kolla:/usr/local/share/kolla-ansible# source  /etc/kolla/admin-openrc.sh
root@kolla:/usr/local/share/kolla-ansible# ./init-runonce

Done.

To deploy a demo instance, run:

openstack server create \
   --image cirros \
   --flavor m1.tiny \
   --key-name mykey \
   --nic net-id=ff0d6fef-bbd2-4b79-8c84-6d14d11d654c \
   demo1
root@kolla:/usr/local/share/kolla-ansible#

Config files are stored under /etc/kola directory.
root@kolla:/etc/kolla# pwd
/etc/kola

root@kolla:/etc/kolla# ls
admin-openrc.sh  heat-api       mariadb                    neutron-server    nova-novncproxy        placement-api

Access to the dashboard.

Launch an instance

root@kolla:~# /usr/local/bin/openstack server list
+--------------------------------------+------------+--------+--------------------+--------+---------+
| ID                                   | Name       | Status | Networks           | Image  | Flavor  |
+--------------------------------------+------------+--------+--------------------+--------+---------+
| 3723fe8c-fe63-4a7b-879c-f16829589291 | instance02 | ACTIVE | demo-net=10.0.0.11 | cirros | m1.tiny |
| 4e616f77-8101-4e0d-ae04-e3d9c28794bb | instance01 | ACTIVE | demo-net=10.0.0.7  | cirros | m1.tiny |
+--------------------------------------+------------+--------+--------------------+--------+---------+
root@kolla:~#

access to the neutron-l3-agent and send ping to the VMs.
root@kolla:~# docker ps | grep neutron | grep l3
4ea44804c1b6        kolla/centos-binary-neutron-l3-agent:4.0.0            "kolla_start"       4 hours ago         Up 4 hours                              neutron_l3_agent
root@kolla:~#

root@kolla:~# docker exec -it -u root 4ea44804c1b6 bash
(neutron-l3-agent)[root@kolla /]#

(neutron-l3-agent)[root@kolla /]# ip netns list
qrouter-14b8034b-d2b5-4b2b-8202-34d88b232d5a
qdhcp-ff0d6fef-bbd2-4b79-8c84-6d14d11d654c
(neutron-l3-agent)[root@kolla /]#

(neutron-l3-agent)[root@kolla /]# ip netns exec qrouter-14b8034b-d2b5-4b2b-8202-34d88b232d5a ping -c 3 10.0.0.11
PING 10.0.0.11 (10.0.0.11) 56(84) bytes of data.
64 bytes from 10.0.0.11: icmp_seq=1 ttl=64 time=2.56 ms
64 bytes from 10.0.0.11: icmp_seq=2 ttl=64 time=0.897 ms
64 bytes from 10.0.0.11: icmp_seq=3 ttl=64 time=0.352 ms

nova-compute
root@kolla:~# docker ps -a | grep nova-compute
b115428354df        kolla/centos-binary-nova-compute:4.0.0                "kolla_start"       4 hours ago         Up 4 hours                              nova_compute

root@kolla:~# docker exec -it -u root b115428354df virsh list
Id    Name                           State
----------------------------------------------------
1     instance-00000001              running
2     instance-00000002              running

root@kolla:~#

[ access to instances from an external network ]

bring up br-ex.
root@kolla:~# docker exec -it -u root b115428354df bash
(nova-compute)[root@kolla /]# ip a s br-ex
6: br-ex: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1
   link/ether f6:fc:c2:22:cc:40 brd ff:ff:ff:ff:ff:ff
(nova-compute)[root@kolla /]# ip link set br-ex up
(nova-compute)[root@kolla /]# ip a s br-ex
6: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1
   link/ether f6:fc:c2:22:cc:40 brd ff:ff:ff:ff:ff:ff
   inet6 fe80::f4fc:c2ff:fe22:cc40/64 scope link
      valid_lft forever preferred_lft forever
(nova-compute)[root@kolla /]#


add floating IPs

associate a floating IP


root@kolla:~# openstack server list
+--------------------------------------+------------+--------+-------------------------------------+--------+---------+
| ID                                   | Name       | Status | Networks                            | Image  | Flavor  |
+--------------------------------------+------------+--------+-------------------------------------+--------+---------+
| 3723fe8c-fe63-4a7b-879c-f16829589291 | instance02 | ACTIVE | demo-net=10.0.0.11, 192.168.153.106 | cirros | m1.tiny |
| 4e616f77-8101-4e0d-ae04-e3d9c28794bb | instance01 | ACTIVE | demo-net=10.0.0.7, 192.168.153.100  | cirros | m1.tiny |
+---------------

send ping
root@kolla:~# ping 192.168.153.100 -c 3
PING 192.168.153.100 (192.168.153.100) 56(84) bytes of data.
64 bytes from 192.168.153.100: icmp_seq=1 ttl=63 time=1.09 ms
64 bytes from 192.168.153.100: icmp_seq=2 ttl=63 time=1.06 ms
64 bytes from 192.168.153.100: icmp_seq=3 ttl=63 time=0.636 ms

SSH accesss to the instance
root@kolla:~# ssh -i .ssh/id_rsa cirros@192.168.153.100
$ hostname
instance01

$ ping www.google.com
PING www.google.com (216.58.197.228): 56 data bytes
64 bytes from 216.58.197.228: seq=0 ttl=50 time=3.960 ms
64 bytes from 216.58.197.228: seq=1 ttl=50 time=6.124 ms
^C
--- www.google.com ping statistics