Here are trial and error logs when running multiple coreOS with vagrant.
I’m new to CoreOS, Docker..
Reference
https://coreos.com/blog/coreos-clustering-with-vagrant/
Host OS
$ vagrant --version
Vagrant 1.7.4
$ virtualbox --help | head -1
Oracle VM VirtualBox Manager 5.0.0
$ tail -1 /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"
|
1. run one CoreOS ( failed to run multiple coreOS )
$ git clone https://github.com/coreos/coreos-vagrant.git
$ cd coreos-vagrant/
$ cp user-data.sample user-data
$ cp config.rb.sample config.rb
|
start coreOS
$ vagrant up
$ vagrant status
Current machine states:
core-01 running (virtualbox)
|
I would like to run multiple coreOS, so shutdown core-01.
$ vagrant halt
|
2. run multiple coreOS
- edit user-data ( cloud-config )
get a token which is used for discovery URL for ectd.
https://coreos.com/os/docs/latest/cluster-discovery.html
t$ curl -w "\n" 'https://discovery.etcd.io/new?size=3'
https://discovery.etcd.io/6cd496a33df40cfecae0e905a1d99cae
|
I will use etcd2 instead of etcd.
details are described here.
hattori@ubuntu:/media/hattori/HDD/Vagrant_works/CoreOS_works/coreos-vagrant$ grep -v '#' user-data
coreos:
etcd2:
https://discovery.etcd.io/6cd496a33df40cfecae0e905a1d99cae
advertise-client-urls: http://$public_ipv4:2379
initial-advertise-peer-urls: http://$private_ipv4:2380
listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
listen-peer-urls: http://$private_ipv4:2380,http://$private_ipv4:7001
fleet:
public-ip: $public_ipv4
flannel:
interface: $public_ipv4
units:
- name: etcd2.service
command: start
- name: fleet.service
command: start
- name: docker-tcp.socket
command: start
enable: true
content: |
[Unit]
Description=Docker Socket for the API
[Socket]
ListenStream=2375
Service=docker.service
BindIPv6Only=both
[Install]
WantedBy=sockets.target
|
- set up CoreOS ( config.rb )
hattori@ubuntu:/media/hattori/HDD/Vagrant_works/CoreOS_works/coreos-vagrant$ grep -v ^# config.rb | grep -v ^$
$num_instances=3
$new_discovery_url="https://discovery.etcd.io/new?size=#{$num_instances}"
$update_channel='stable'
|
start three CoreOS
hattori@ubuntu:/media/hattori/HDD/Vagrant_works/CoreOS_works/coreos-vagrant$ vagrant up
Bringing machine 'core-01' up with 'virtualbox' provider...
Bringing machine 'core-02' up with 'virtualbox' provider...
Bringing machine 'core-03' up with 'virtualbox' provider...
==> core-01: Clearing any previously set forwarded ports...
==> core-01: Clearing any previously set network interfaces...
==> core-01: Preparing network interfaces based on configuration...
core-01: Adapter 1: nat
core-01: Adapter 2: hostonly
==> core-01: Forwarding ports...
core-01: 22 => 2222 (adapter 1)
==> core-01: Running 'pre-boot' VM customizations...
==> core-01: Booting VM...
==> core-01: Waiting for machine to boot. This may take a few minutes...
core-01: SSH address: 127.0.0.1:2222
core-01: SSH username: vagrant
core-01: SSH auth method: private key
core-01: Warning: Connection timeout. Retrying...
core-01: Warning: Authentication failure. Retrying...
core-01: Warning: Authentication failure. Retrying...
core-01: Warning: Authentication failure. Retrying…
Timed out while waiting for the machine to boot. This means that
Vagrant was unable to communicate with the guest machine within
the configured ("config.vm.boot_timeout" value) time period.
If you look above, you should be able to see the error(s) that
Vagrant had when attempting to connect to the machine. These errors
are usually good hints as to what may be wrong.
If you're using a custom box, make sure that networking is properly
working and you're able to connect to the machine. It is a common
problem that networking isn't setup properly in these boxes.
Verify that authentication configurations are also setup properly,
as well.
If the box appears to be booting properly, you may want to increase
the timeout ("config.vm.boot_timeout") value.
|
nnn, authentication failure..
how about an output of ‘vagrant status’?
only one CoreOS(core-01) is running..
hattori@ubuntu:/media/hattori/HDD/Vagrant_works/CoreOS_works/coreos-vagrant$ vagrant status
Current machine states:
core-01 running (virtualbox)
core-02 not created (virtualbox)
core-03 not created (virtualbox)
|
nnn, authentication failure..
how about issuing ‘vagrant reload’?
nnn, same error.
hattori@ubuntu:/media/hattori/HDD/Vagrant_works/CoreOS_works/coreos-vagrant$ vagrant reload
==> core-01: Attempting graceful shutdown of VM...
core-01: Guest communication could not be established! This is usually because
core-01: SSH is not running, the authentication information was changed,
core-01: or some other networking issue. Vagrant will force halt, if
core-01: capable.
==> core-01: Forcing shutdown of VM...
==> core-01: Clearing any previously set forwarded ports...
==> core-01: Clearing any previously set network interfaces...
==> core-01: Preparing network interfaces based on configuration...
core-01: Adapter 1: nat
core-01: Adapter 2: hostonly
==> core-01: Forwarding ports...
core-01: 22 => 2222 (adapter 1)
==> core-01: Running 'pre-boot' VM customizations...
==> core-01: Booting VM...
==> core-01: Waiting for machine to boot. This may take a few minutes...
core-01: SSH address: 127.0.0.1:2222
core-01: SSH username: vagrant
core-01: SSH auth method: private key
core-01: Warning: Connection timeout. Retrying...
core-01: Warning: Authentication failure. Retrying...
core-01: Warning: Authentication failure. Retrying...
|
shutdown core-01 forcefully.
hattori@ubuntu:/media/hattori/HDD/Vagrant_works/CoreOS_works/coreos-vagrant$ vagrant destroy
==> core-03: VM not created. Moving on...
==> core-02: VM not created. Moving on...
core-01: Are you sure you want to destroy the 'core-01' VM? [y/N] y
==> core-01: Forcing shutdown of VM...
==> core-01: Destroying VM and associated drives...
==> core-01: Running cleanup tasks for 'file' provisioner...
==> core-01: Running cleanup tasks for 'shell' provisioner...
|
run ‘vagrant up’ again.
I was able to boot three CoreOS.
hattori@ubuntu:/media/hattori/HDD/Vagrant_works/CoreOS_works/coreos-vagrant$ vagrant reload
==> core-01: VM not created. Moving on...
==> core-02: VM not created. Moving on...
==> core-03: VM not created. Moving on…
hattori@ubuntu:/media/hattori/HDD/Vagrant_works/CoreOS_works/coreos-vagrant$
hattori@ubuntu:/media/hattori/HDD/Vagrant_works/CoreOS_works/coreos-vagrant$ vagrant up
Bringing machine 'core-01' up with 'virtualbox' provider...
Bringing machine 'core-02' up with 'virtualbox' provider...
Bringing machine 'core-03' up with 'virtualbox' provider...
==> core-01: Box 'coreos-stable' could not be found. Attempting to find and install...
core-01: Box Provider: virtualbox
core-01: Box Version: >= 0
==> core-01: Loading metadata for box 'http://stable.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json'
core-01: URL: http://stable.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json
==> core-01: Adding box 'coreos-stable' (v717.3.0) for provider: virtualbox
core-01: Downloading: http://stable.release.core-os.net/amd64-usr/717.3.0/coreos_production_vagrant.box
core-01: Calculating and comparing box checksum...
==> core-01: Successfully added box 'coreos-stable' (v717.3.0) for 'virtualbox'!
==> core-01: Importing base box 'coreos-stable'...
==> core-01: Matching MAC address for NAT networking...
==> core-01: Checking if box 'coreos-stable' is up to date...
==> core-01: Setting the name of the VM: coreos-vagrant_core-01_1438274131155_59902
==> core-01: Clearing any previously set network interfaces...
==> core-01: Preparing network interfaces based on configuration...
core-01: Adapter 1: nat
core-01: Adapter 2: hostonly
==> core-01: Forwarding ports...
core-01: 22 => 2222 (adapter 1)
==> core-01: Running 'pre-boot' VM customizations...
==> core-01: Booting VM...
==> core-01: Waiting for machine to boot. This may take a few minutes...
core-01: SSH address: 127.0.0.1:2222
core-01: SSH username: core
core-01: SSH auth method: private key
core-01: Warning: Connection timeout. Retrying...
==> core-01: Machine booted and ready!
==> core-01: Setting hostname...
==> core-01: Configuring and enabling network interfaces...
==> core-01: Running provisioner: file...
==> core-01: Running provisioner: shell...
core-01: Running: inline script
==> core-02: Box 'coreos-stable' could not be found. Attempting to find and install...
core-02: Box Provider: virtualbox
core-02: Box Version: >= 0
==> core-02: Loading metadata for box 'http://stable.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json'
core-02: URL: http://stable.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json
==> core-02: Adding box 'coreos-stable' (v717.3.0) for provider: virtualbox
==> core-02: Importing base box 'coreos-stable'...
==> core-02: Matching MAC address for NAT networking...
==> core-02: Checking if box 'coreos-stable' is up to date...
==> core-02: Setting the name of the VM: coreos-vagrant_core-02_1438274164229_15686
==> core-02: Fixed port collision for 22 => 2222. Now on port 2200.
==> core-02: Clearing any previously set network interfaces...
==> core-02: Preparing network interfaces based on configuration...
core-02: Adapter 1: nat
core-02: Adapter 2: hostonly
==> core-02: Forwarding ports...
core-02: 22 => 2200 (adapter 1)
==> core-02: Running 'pre-boot' VM customizations...
==> core-02: Booting VM...
==> core-02: Waiting for machine to boot. This may take a few minutes...
core-02: SSH address: 127.0.0.1:2200
core-02: SSH username: core
core-02: SSH auth method: private key
core-02: Warning: Connection timeout. Retrying...
==> core-02: Machine booted and ready!
==> core-02: Setting hostname...
==> core-02: Configuring and enabling network interfaces...
==> core-02: Running provisioner: file...
==> core-02: Running provisioner: shell...
core-02: Running: inline script
==> core-03: Box 'coreos-stable' could not be found. Attempting to find and install...
core-03: Box Provider: virtualbox
core-03: Box Version: >= 0
==> core-03: Loading metadata for box 'http://stable.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json'
core-03: URL: http://stable.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json
==> core-03: Adding box 'coreos-stable' (v717.3.0) for provider: virtualbox
==> core-03: Importing base box 'coreos-stable'...
==> core-03: Matching MAC address for NAT networking...
==> core-03: Checking if box 'coreos-stable' is up to date...
==> core-03: Setting the name of the VM: coreos-vagrant_core-03_1438274202199_33649
==> core-03: Fixed port collision for 22 => 2222. Now on port 2201.
==> core-03: Clearing any previously set network interfaces...
==> core-03: Preparing network interfaces based on configuration...
core-03: Adapter 1: nat
core-03: Adapter 2: hostonly
==> core-03: Forwarding ports...
core-03: 22 => 2201 (adapter 1)
==> core-03: Running 'pre-boot' VM customizations...
==> core-03: Booting VM...
==> core-03: Waiting for machine to boot. This may take a few minutes...
core-03: SSH address: 127.0.0.1:2201
core-03: SSH username: core
core-03: SSH auth method: private key
core-03: Warning: Connection timeout. Retrying...
==> core-03: Machine booted and ready!
==> core-03: Setting hostname...
==> core-03: Configuring and enabling network interfaces...
==> core-03: Running provisioner: file...
==> core-03: Running provisioner: shell...
core-03: Running: inline script
|
hattori@ubuntu:/media/hattori/HDD/Vagrant_works/CoreOS_works/coreos-vagrant$ vagrant status
Current machine states:
core-01 running (virtualbox)
core-02 running (virtualbox)
core-03 running (virtualbox)
|
connect to ‘core-01’
$ vagrant ssh core-01 -- -A
CoreOS stable (717.3.0)
core@core-01 ~ $
core@core-01 ~ $ cat /etc/lsb-release
DISTRIB_ID=CoreOS
DISTRIB_RELEASE=717.3.0
DISTRIB_CODENAME="Red Dog"
|
‘-- -A’ : ssh agent forwarding ?
on the core-01
core@core-01 ~ $ fleetctl list-machines
MACHINE IP METADATA
251871b7... 172.17.8.103 -
66623272... 172.17.8.101 -
b026bad8... 172.17.8.102 -
|
I was able to run three coreOS with vagrant, I will look into this further for my next post.