Docker swarm is cluster management and orchestration tool for Docker container. Docker engines participating in a cluster are running in swarm mode. You enable swarm mode for an engine by either initializing a swarm or joining an existing swarm.
A swarm is a cluster of Docker engines, or nodes, where you deploy services. The Docker Engine CLI and API include commands to manage swarm nodes (e.g., add or remove nodes), and deploy and orchestrate services across the swarm.
When you run Docker without using swarm mode, you execute container commands. When you run the Docker in swarm mode, you orchestrate services. You can run swarm services and standalone containers on the same Docker instances.
Read more about Docker swarm here
Docker Swarm terminology
Node - A node is an instance of the Docker engine participating in the swarm. You can nodes in one physical machine or distributed across multiple machine Manager node - A Manager node manages services on a cluster Worker node - A worker node executes task dispatched by manager node
Prerequisite
To use swarm you need Docker version v1.12 or higher. Install docker on all nodes, follow instructions below:
Login as a user with sudo permission
$su - divine
$ uname -r //check kernel version it should be 3.10 or higher
3.10.0-229.el7.x86_64
$ sudo yum update
run Docker installation script
$ curl -fsSL https://get.docker.com/ | sh
Logout and log back in
$logout
$su - divine
Enable the service
$ sudo systemctl enable docker.service
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service
Start Docker daemon
$ sudo systemctl start docker
check if Docker is running
$docker -v
Docker version 1.12.6, build 78d1802
Create Docker group
$ sudo groupadd docker
Add user to Docker group
$sudo usermod -aG docker $(whoami)
Logout and log back in
$logout
$su - divine
[divine@localhost ~]$ docker --version
Docker version 1.12.6, build 78d1802
[divine@localhost ~]$
In this lab I have three physical machines, one machine acting as Manager and two as Worker. Make sure there is IP connectivity among nodes (perform ping test)
Manager= 192.254.211.167
Worker_1= 192.254.211.166
Worker_1= 192.254.211.168
Procedure
- Create swarm on machine acting as Manager
[divine@Manager ~]$ docker swarm init --advertise-addr 192.254.211.167 Swarm initialized: current node (9ra9h1uopginqno78zi8rq9ug) is now a manager. To add a worker to this swarm, run the following command: docker swarm join \ --token SWMTKN-1-4ok62ytnftqgeyvzlz82zlbuzmbx3soqhyfjsoiylsfx7o3tnh-8b5cyqbg6ukpl9oxap0ntrl7t \ 192.254.211.167:2377
Swarm process is running and listening on port 2377
[divine@Manager ~]$ netstat -pan | grep :2377 (No info could be read for "-p": geteuid()=1002 but you should be root.) tcp 0 0 192.254.211.167:40892 192.254.211.167:2377 ESTABLISHED - tcp 0 0 127.0.0.1:53116 127.0.0.1:2377 ESTABLISHED - tcp6 0 0 :::2377 :::* LISTEN - tcp6 0 0 192.254.211.167:2377 192.254.211.167:40892 ESTABLISHED - tcp6 0 0 127.0.0.1:2377 127.0.0.1:53116 ESTABLISHED - [divine@localhost ~]$ [divine@Manager ~]$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS 9ra9h1uopginqno78zi8rq9ug * Manager Ready Active Leader [divine@Manager ~]$
2. Add worker_1 to swarm
//run this command to get worker join command [divine@Manager ~]$ docker swarm join-token worker To add a worker to this swarm, run the following command: docker swarm join \ --token SWMTKN-1-4ok62ytnftqgeyvzlz82zlbuzmbx3soqhyfjsoiylsfx7o3tnh-8b5cyqbg6ukpl9oxap0ntrl7t \ 192.254.211.167:2377 //run out of above command on worker node [divine@Worker_1 ~]$ docker swarm join \ > --token SWMTKN-1-4ok62ytnftqgeyvzlz82zlbuzmbx3soqhyfjsoiylsfx7o3tnh-8b5cyqbg6ukpl9oxap0ntrl7t \ > 192.254.211.167:2377 This node joined a swarm as a worker.
3.Add worker_2 to swarm using same command. Run same command on Worker_2
[divine@Worker_2 ~]$ docker swarm join \ > --token SWMTKN-1-4ok62ytnftqgeyvzlz82zlbuzmbx3soqhyfjsoiylsfx7o3tnh-8b5cyqbg6ukpl9oxap0ntrl7t \ > 192.254.211.167:2377 This node joined a swarm as a worker.
Note:All nodes date/time needs to be in-sync otherwise worker join will fail
4. Check node status in manager node. Run these commands on manager node
MANAGER STATUS:<empty> – worker node, leader – primary manager
AVAILABILITY: active – node is active scheduler can assign task, pause – don’t assign new task but old task remain , drain – shutdown task and don’t assign new task
[divine@Manager ~]$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
0pye1jeusjpyypcesrjpwsdgg Worker_1 Ready Active
9iz0pjzudzvv2ujqyh0dgzhfg Worker_2 Ready Active
9ra9h1uopginqno78zi8rq9ug * Manager Ready Active Leader
[divine@localhost ~]$
//inspect manager node
[divine@Manager ~]$ docker node inspect self
[
{
"ID": "9ra9h1uopginqno78zi8rq9ug",
"Version": {
"Index": 10
},
"CreatedAt": "2017-01-19T23:12:49.552590718Z",
"UpdatedAt": "2017-01-19T23:12:49.812466748Z",
"Spec": {
"Role": "manager",
"Availability": "active"
},
"Description": {
"Hostname": "Manager",
"Platform": {
"Architecture": "x86_64",
"OS": "linux"
},
"Resources": {
"NanoCPUs": 4000000000,
"MemoryBytes": 12412542976
},
"Engine": {
"EngineVersion": "1.12.6",
"Plugins": [
{
"Type": "Network",
"Name": "bridge"
},
{
"Type": "Network",
"Name": "host"
},
{
"Type": "Network",
"Name": "null"
},
{
"Type": "Network",
"Name": "overlay"
},
{
"Type": "Volume",
"Name": "local"
}
]
}
},
"Status": {
"State": "ready"
},
"ManagerStatus": {
"Leader": true,
"Reachability": "reachable",
"Addr": "192.254.211.167:2377"
}
}
]
[divine@Manager ~]$
5. Inspect worker node. Run below command on manager node
[divine@Manager ~]$ docker node inspect Worker_2
[
{
"ID": "9iz0pjzudzvv2ujqyh0dgzhfg",
"Version": {
"Index": 16
},
"CreatedAt": "2017-01-19T23:19:56.795050481Z",
"UpdatedAt": "2017-01-19T23:19:56.954890419Z",
"Spec": {
"Role": "worker",
"Availability": "active"
},
"Description": {
"Hostname": "Worker_2",
"Platform": {
"Architecture": "x86_64",
"OS": "linux"
},
"Resources": {
"NanoCPUs": 8000000000,
"MemoryBytes": 12412542976
},
"Engine": {
"EngineVersion": "1.13.0",
"Plugins": [
{
"Type": "Network",
"Name": "bridge"
},
{
"Type": "Network",
"Name": "host"
},
{
"Type": "Network",
"Name": "macvlan"
},
{
"Type": "Network",
"Name": "null"
},
{
"Type": "Network",
"Name": "overlay"
},
{
"Type": "Volume",
"Name": "local"
}
]
}
},
"Status": {
"State": "ready"
}
}
]
[divine@Manager ~]$ docker node inspect Worker_1
[
{
"ID": "0pye1jeusjpyypcesrjpwsdgg",
"Version": {
"Index": 21
},
"CreatedAt": "2017-01-19T23:32:47.763436319Z",
"UpdatedAt": "2017-01-19T23:32:47.916593695Z",
"Spec": {
"Role": "worker",
"Availability": "active"
},
"Description": {
"Hostname": "Worker_1",
"Platform": {
"Architecture": "x86_64",
"OS": "linux"
},
"Resources": {
"NanoCPUs": 4000000000,
"MemoryBytes": 12412542976
},
"Engine": {
"EngineVersion": "1.13.0",
"Plugins": [
{
"Type": "Network",
"Name": "bridge"
},
{
"Type": "Network",
"Name": "host"
},
{
"Type": "Network",
"Name": "macvlan"
},
{
"Type": "Network",
"Name": "null"
},
{
"Type": "Network",
"Name": "overlay"
},
{
"Type": "Volume",
"Name": "local"
}
]
}
},
"Status": {
"State": "ready"
}
}
]
[divine@Manager ~]$
6.Start a service in swarm. You can create service either by pulling image from docker hub or local image. In case of local image you need to manually load image on all worker and manager node.
//no service active on swarm [divine@Manager ~]$ docker service ls ID NAME REPLICAS IMAGE COMMAND //run this command on manager node. This will create 20 containers in cluster named http_server using image nginx. Docker will pull this image from Docker hub and create 20 containers. As you can see containers are equally distributed on 3 nodes by scheduler [divine@Manager ~]$ docker service create --replicas 20 --name httpd_server nginx brscnmwefdbxopz6hk2nnxebc [divine@Manager ~]$ docker service ls ID NAME REPLICAS IMAGE COMMAND brscnmwefdbx httpd_server 0/20 nginx [divine@Manager ~]$ docker service ls ID NAME REPLICAS IMAGE COMMAND brscnmwefdbx httpd_server 0/20 nginx [divine@Manager ~]$ docker service ls ID NAME REPLICAS IMAGE COMMAND brscnmwefdbx httpd_server 0/20 nginx [divine@Manager ~]$ docker service ps httpd_server ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR etg7ng9e738kde0cffknfaymx httpd_server.1 nginx Worker_2 Running Preparing less than a second ago d7o4irupasdflk9dqq955rjil httpd_server.2 nginx Manager Running Preparing 26 seconds ago a0px1ll6rla2nfjfenfro3ax1 httpd_server.3 nginx Worker_2 Running Preparing less than a second ago 5lx61fb15p2otfmnz0azyk4c1 httpd_server.4 nginx Worker_2 Running Preparing less than a second ago buvz6ndhjb3o7mdbua9y4st3l httpd_server.5 nginx Manager Running Preparing 26 seconds ago 5zdum3ef3qo2vakppwnjt9t6n httpd_server.6 nginx Worker_2 Running Preparing less than a second ago 7pt4s3fl9z41hhcqwlo2llefa httpd_server.7 nginx Worker_1 Running Running less than a second ago c5jt346vkr5rcwkk72hfc3if0 httpd_server.8 nginx Manager Running Preparing 26 seconds ago 6wr6r9zz0hfy1zc4lg49i108m httpd_server.9 nginx Worker_1 Running Running less than a second ago cmzrng2t23udi4i71k0vhf0jl httpd_server.10 nginx Worker_2 Running Preparing less than a second ago 1zagdr2zbwvaz5sd7tnrlucks httpd_server.11 nginx Worker_1 Running Running less than a second ago dbap94coizealad8clzi1d779 httpd_server.12 nginx Worker_2 Running Preparing less than a second ago 57hvlhnd942fnsr6b2wvhp0al httpd_server.13 nginx Worker_1 Running Running less than a second ago 51ic2pk2eoq9hqvx2kp2u60n1 httpd_server.14 nginx Worker_1 Running Running less than a second ago 3myxunl1h2dk14zne2hci83av httpd_server.15 nginx Worker_1 Running Running less than a second ago b7smowk7getaxnc3xilty2vez httpd_server.16 nginx Worker_2 Running Preparing less than a second ago dlmagnn74o3mqpo61fb9cchtq httpd_server.17 nginx Manager Running Preparing 26 seconds ago 0t3j2u3wkl4ym3w66rvplgn65 httpd_server.18 nginx Worker_1 Running Running less than a second ago 0smn62ybwbdlrwtysfxax3yyl httpd_server.19 nginx Manager Running Preparing 26 seconds ago 6hk3ge3t5xhyemd70vyw6ymvk httpd_server.20 nginx Manager Running Preparing 26 seconds ago [divine@Manager ~]$ docker service ls ID NAME REPLICAS IMAGE COMMAND brscnmwefdbx httpd_server 16/20 nginx [divine@Manager ~]$ docker service ls ID NAME REPLICAS IMAGE COMMAND brscnmwefdbx httpd_server 20/20 nginx [divine@Manager ~]$
7.Check containers in worker node. Run below commands on worker node
//as you can see this worker node instantiated 7 containers. [divine@Worker_1 ~]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0e7c1a499cac nginx:latest "nginx -g 'daemon ..." About a minute ago Up About a minute 80/tcp, 443/tcp httpd_server.15.3myxunl1h2dk14zne2hci83av b2d474cd78f3 nginx:latest "nginx -g 'daemon ..." About a minute ago Up About a minute 80/tcp, 443/tcp httpd_server.9.6wr6r9zz0hfy1zc4lg49i108m b4e9323c2303 nginx:latest "nginx -g 'daemon ..." About a minute ago Up About a minute 80/tcp, 443/tcp httpd_server.7.7pt4s3fl9z41hhcqwlo2llefa fa2b01136a2d nginx:latest "nginx -g 'daemon ..." About a minute ago Up About a minute 80/tcp, 443/tcp httpd_server.11.1zagdr2zbwvaz5sd7tnrlucks 0a1dfde8ba0f nginx:latest "nginx -g 'daemon ..." About a minute ago Up About a minute 80/tcp, 443/tcp httpd_server.18.0t3j2u3wkl4ym3w66rvplgn65 0f33c50d86aa nginx:latest "nginx -g 'daemon ..." About a minute ago Up About a minute 80/tcp, 443/tcp httpd_server.14.51ic2pk2eoq9hqvx2kp2u60n1 9c66bcd8e981 nginx:latest "nginx -g 'daemon ..." About a minute ago Up About a minute 80/tcp, 443/tcp httpd_server.13.57hvlhnd942fnsr6b2wvhp0al [divine@Worker_1 ~]$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE nginx latest a39777a1a4a6 2 days ago 182 MB [divine@Worker_1 ~]$
8. You can scale up or down the service. Run below commands on manager node
//here I am reducing number of containers from 20 to 10 [divine@Manager ~]$ docker service scale httpd_server=10 httpd_server scaled to 10 [divine@Manager ~]$ docker service ls ID NAME REPLICAS IMAGE COMMAND brscnmwefdbx httpd_server 10/10 nginx //10 services moved from running to shutdown [divine@Manager ~]$ docker service ps httpd_server ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR etg7ng9e738kde0cffknfaymx httpd_server.1 nginx Worker_2 Shutdown Shutdown less than a second ago d7o4irupasdflk9dqq955rjil httpd_server.2 nginx Manager Shutdown Shutdown 18 seconds ago a0px1ll6rla2nfjfenfro3ax1 httpd_server.3 nginx Worker_2 Running Running 47 seconds ago 5lx61fb15p2otfmnz0azyk4c1 httpd_server.4 nginx Worker_2 Running Running 41 seconds ago buvz6ndhjb3o7mdbua9y4st3l httpd_server.5 nginx Manager Running Running 4 minutes ago 5zdum3ef3qo2vakppwnjt9t6n httpd_server.6 nginx Worker_2 Running Running 41 seconds ago 7pt4s3fl9z41hhcqwlo2llefa httpd_server.7 nginx Worker_1 Shutdown Shutdown less than a second ago c5jt346vkr5rcwkk72hfc3if0 httpd_server.8 nginx Manager Running Running 4 minutes ago 6wr6r9zz0hfy1zc4lg49i108m httpd_server.9 nginx Worker_1 Shutdown Shutdown less than a second ago cmzrng2t23udi4i71k0vhf0jl httpd_server.10 nginx Worker_2 Shutdown Shutdown less than a second ago 1zagdr2zbwvaz5sd7tnrlucks httpd_server.11 nginx Worker_1 Running Running about a minute ago dbap94coizealad8clzi1d779 httpd_server.12 nginx Worker_2 Shutdown Shutdown less than a second ago 57hvlhnd942fnsr6b2wvhp0al httpd_server.13 nginx Worker_1 Shutdown Shutdown less than a second ago 51ic2pk2eoq9hqvx2kp2u60n1 httpd_server.14 nginx Worker_1 Shutdown Shutdown less than a second ago 3myxunl1h2dk14zne2hci83av httpd_server.15 nginx Worker_1 Running Running about a minute ago b7smowk7getaxnc3xilty2vez httpd_server.16 nginx Worker_2 Shutdown Shutdown less than a second ago dlmagnn74o3mqpo61fb9cchtq httpd_server.17 nginx Manager Shutdown Shutdown 18 seconds ago 0t3j2u3wkl4ym3w66rvplgn65 httpd_server.18 nginx Worker_1 Running Running about a minute ago 0smn62ybwbdlrwtysfxax3yyl httpd_server.19 nginx Manager Running Running 4 minutes ago 6hk3ge3t5xhyemd70vyw6ymvk httpd_server.20 nginx Manager Running Running 4 minutes ago //scale up containers from 10 to 15 [divine@Manager ~]$ docker service scale httpd_server=15 httpd_server scaled to 15 [divine@Manager ~]$ docker service ls ID NAME REPLICAS IMAGE COMMAND brscnmwefdbx httpd_server 15/15 nginx //5 new containers started. Looks like Docker doesn't move containers from shutdown to running state instead it creates new containers [divine@Manager ~]$ docker service ps httpd_server ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR 4fs7ekdmlsd7ljo4blssyho35 httpd_server.1 nginx Worker_2 Running Running less than a second ago etg7ng9e738kde0cffknfaymx \_ httpd_server.1 nginx Worker_2 Shutdown Shutdown less than a second ago 4kjcflhtq8t5m39syol75pbgx httpd_server.2 nginx Worker_2 Running Running less than a second ago d7o4irupasdflk9dqq955rjil \_ httpd_server.2 nginx Manager Shutdown Shutdown 59 seconds ago a0px1ll6rla2nfjfenfro3ax1 httpd_server.3 nginx Worker_2 Running Running about a minute ago 5lx61fb15p2otfmnz0azyk4c1 httpd_server.4 nginx Worker_2 Running Running about a minute ago buvz6ndhjb3o7mdbua9y4st3l httpd_server.5 nginx Manager Running Running 4 minutes ago 5zdum3ef3qo2vakppwnjt9t6n httpd_server.6 nginx Worker_2 Running Running about a minute ago cezj4m9x16dtow9yet4wovi6e httpd_server.7 nginx Worker_1 Running Running less than a second ago 7pt4s3fl9z41hhcqwlo2llefa \_ httpd_server.7 nginx Worker_1 Shutdown Shutdown less than a second ago c5jt346vkr5rcwkk72hfc3if0 httpd_server.8 nginx Manager Running Running 4 minutes ago 2iyv3fjibrc53k4aoti5sl115 httpd_server.9 nginx Worker_1 Running Running less than a second ago 6wr6r9zz0hfy1zc4lg49i108m \_ httpd_server.9 nginx Worker_1 Shutdown Shutdown less than a second ago 7pi63gh79o9sbm1bqy4ouw24j httpd_server.10 nginx Manager Running Running 8 seconds ago cmzrng2t23udi4i71k0vhf0jl \_ httpd_server.10 nginx Worker_2 Shutdown Shutdown less than a second ago 1zagdr2zbwvaz5sd7tnrlucks httpd_server.11 nginx Worker_1 Running Running 2 minutes ago dbap94coizealad8clzi1d779 httpd_server.12 nginx Worker_2 Shutdown Shutdown less than a second ago 57hvlhnd942fnsr6b2wvhp0al httpd_server.13 nginx Worker_1 Shutdown Shutdown less than a second ago 51ic2pk2eoq9hqvx2kp2u60n1 httpd_server.14 nginx Worker_1 Shutdown Shutdown less than a second ago 3myxunl1h2dk14zne2hci83av httpd_server.15 nginx Worker_1 Running Running 2 minutes ago b7smowk7getaxnc3xilty2vez httpd_server.16 nginx Worker_2 Shutdown Shutdown less than a second ago dlmagnn74o3mqpo61fb9cchtq httpd_server.17 nginx Manager Shutdown Shutdown 58 seconds ago 0t3j2u3wkl4ym3w66rvplgn65 httpd_server.18 nginx Worker_1 Running Running 2 minutes ago 0smn62ybwbdlrwtysfxax3yyl httpd_server.19 nginx Manager Running Running 4 minutes ago 6hk3ge3t5xhyemd70vyw6ymvk httpd_server.20 nginx Manager Running Running 4 minutes ago [divine@Manager ~]$ docker service inspect --pretty httpd_server ID: brscnmwefdbxopz6hk2nnxebc Name: httpd_server Mode: Replicated Replicas: 15 Placement: UpdateConfig: Parallelism: 1 On failure: pause ContainerSpec: Image: nginx Resources: [divine@localhost ~]$
9. Delete a service. Run below commands on manager node
[divine@Manager ~]$ docker service ls ID NAME REPLICAS IMAGE COMMAND brscnmwefdbx httpd_server 15/15 nginx [divine@Manager ~]$ docker service rm httpd_server httpd_server [divine@Manager ~]$ docker service ls ID NAME REPLICAS IMAGE COMMAND [divine@Manager ~]$ docker service ps httpd_server Error: No such service: httpd_server [divine@localhost ~]$ //check worker [divine@Worker_1 ~]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
10. Leave swarm. Remove worker node from swarm
[divine@Manager ~]$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS 0pye1jeusjpyypcesrjpwsdgg Worker_1 Ready Active 9iz0pjzudzvv2ujqyh0dgzhfg Worker_2 Ready Active 9ra9h1uopginqno78zi8rq9ug * Manager Ready Active Leader //try this on worker node [divine@Worker_1 ~]$ docker swarm leave Node left the swarm. [divine@fpm4richdev ~]$ //check manager. Worker_1 shows Down state [divine@Manager ~]$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS 0pye1jeusjpyypcesrjpwsdgg Worker_1 Down Active 9iz0pjzudzvv2ujqyh0dgzhfg Worker_2 Ready Active 9ra9h1uopginqno78zi8rq9ug * Manager Ready Active Leader [divine@localhost ~]$
11. Join swarm again. Worker rejoin the swarm
//run this command on manager to get worker node join command [divine@Manager ~]$ docker swarm join-token worker To add a worker to this swarm, run the following command: docker swarm join \ --token SWMTKN-1-4ok62ytnftqgeyvzlz82zlbuzmbx3soqhyfjsoiylsfx7o3tnh-8b5cyqbg6ukpl9oxap0ntrl7t \ 192.254.211.167:2377 //try above join command on worker node [divine@Worker_1 ~]$ docker swarm join \ > --token SWMTKN-1-4ok62ytnftqgeyvzlz82zlbuzmbx3soqhyfjsoiylsfx7o3tnh-8b5cyqbg6ukpl9oxap0ntrl7t \ > 192.254.211.167:2377 This node joined a swarm as a worker. [divine@Worker_1 ~]$ //check node status in manager. Docker adds node again it doesn't move down node to ready [divine@Manager ~]$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS 0pye1jeusjpyypcesrjpwsdgg Worker_1 Down Active 8pzfy2447ox4c2ay8we1l35su Worker_1 Ready Active 9iz0pjzudzvv2ujqyh0dgzhfg Worker_2 Ready Active 9ra9h1uopginqno78zi8rq9ug * Manager Ready Active Leader [divine@localhost ~]$ //remove down node [divine@Manager ~]$ docker node rm 0pye1jeusjpyypcesrjpwsdgg 0pye1jeusjpyypcesrjpwsdgg [divine@Manager ~]$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS 8pzfy2447ox4c2ay8we1l35su Worker_1 Ready Active 9iz0pjzudzvv2ujqyh0dgzhfg Worker_2 Ready Active 9ra9h1uopginqno78zi8rq9ug * Manager Ready Active Leader [divine@localhost ~]$
Welcome to cloudNet 