{"id":2317,"date":"2016-10-04T11:18:54","date_gmt":"2016-10-04T08:18:54","guid":{"rendered":"https:\/\/upcloud.com\/global\/us\/resources\/tutorials\/docker-swarm-orchestration\/"},"modified":"2016-10-04T11:18:54","modified_gmt":"2016-10-04T08:18:54","slug":"docker-swarm-orchestration","status":"publish","type":"tutorial","link":"https:\/\/upcloud.com\/global\/resources\/tutorials\/docker-swarm-orchestration\/","title":{"rendered":"How to get started with Docker Swarm orchestration"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Docker has been at the leading edge of containerized technologies with its open-source Community Edition and is steadily building features around it. Docker Swarm is the Docker native clustering solution that turns a pool of Docker hosts into a single virtual server allowing clustering with the built-in Swarm orchestration.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/signup.upcloud.com\/\">Try UpCloud for free!<\/a><\/div>\n<\/div>\n\n\n\n<p class=\"wp-block-paragraph\">The docker CE is designed around four core principles: simple yet powerful with a \u201cjust works\u201d user experience, resilient zero single-point-of-failure architecture, secure by default with automatically generated certificates, and backward compatibility with existing components. Thanks to the ease of use, this introductory guide is suitable for anyone looking to get started with Docker Swarm orchestration. No prior knowledge of containers is necessary, but if you wish, you can get acquainted with Docker by checking out a starter&#8217;s guide for <a href=\"https:\/\/upcloud.com\/global\/resources\/tutorials\/wordpress-with-docker\/\">how to install WordPress with Docker<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Deploying CoreOS&nbsp;nodes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">One of the easiest ways of running docker is deploying a CoreOS host in which docker comes pre-installed. CoreOS itself is a container-optimised operating system that employs many of the docker&#8217;s features out of the box. Deploying a CoreOS server is fast and simple using a short\u00a0<a href=\"https:\/\/upcloud.com\/global\/docs\/guides\/initialization-scripts\/\" target=\"_blank\" rel=\"noreferrer noopener\">Initialization Script<\/a>\u00a0called cloud-config, which is commonly used to bootstrap together a CoreOS cluster along with the required services. However, in this case, docker does not require any specific settings to start, but you may still wish to learn about the operating system itself with the article\u00a0<a href=\"https:\/\/upcloud.com\/global\/blog\/the-rise-of-coreos\/\">the rise of CoreOS<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For this guide, you will need to deploy three cloud servers: a manager and two worker nodes. Each of the nodes in the swarm cluster can act in either role and may be&nbsp;promoted or demoted even during runtime. If you are ready to get started, log into your <a rel=\"noopener noreferrer\" href=\"https:\/\/hub.upcloud.com\/deploy\" target=\"_blank\">UpCloud control panel<\/a> and select <em>Deploy server<\/em> in the Servers menu.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Select the location for your node.<\/li>\n\n\n\n<li>Pick a configuration, the smallest is fine.<\/li>\n\n\n\n<li>Under the operating system section, use the CoreOS template.<\/li>\n\n\n\n<li>Include any SSH keys you may wish to add to the root user.<\/li>\n\n\n\n<li>Use a simple initialization script to input a cloud-config file to customise the server at deployment. Add the following to the text field. Replace the <span style=\"color: #ff0000;\">ssh_key<\/span> with the key you wish to use for login.&nbsp;The first key is added to the default username <tt>core<\/tt>. In the <tt>users<\/tt> section, you can also define your own username like the example <tt>docker<\/tt> user below.\n<pre>#cloud-config\nssh_authorized_keys:\n  - \"<span style=\"color: #ff0000;\">ssh_key<\/span>\"\nusers:\n  - name: \"docker-user\"\n    groups:\n      - \"sudo\"\n      - \"docker\"\n    ssh-authorized-keys:\n      - \"<span style=\"color: #ff0000;\">ssh_key<\/span>\"<\/pre>\n<\/li>\n\n\n\n<li>Once all settings are ready, start the node by clicking the Deploy server button at the bottom of the page.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Repeat until you have three CoreOS nodes up and running.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Log into each node over SSH using the keys you included or with the root user and password provided by the deployment process.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Configuring Docker Swarm on CoreOS<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The Docker CE comes with an integrated cluster management solution. It allows the creation of a swarm of docker nodes that can deploy application services. As the first-party solution, no additional software is needed to use Swarm orchestration to create and manage a cluster.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">CoreOS usually handles updates automatically, but in this case, use the command below to run the update manually.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">sudo update_engine_client -update\n...\nUpdate succeeded -- reboot needed.<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Once the update is completed, reboot the nodes to start using the new version.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">sudo reboot\n...\nCoreOS stable (2079.3.0)<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The Docker Swarm follows a decentralised design where nodes can handle any role in the cluster. The node specialisations for managers and workers are chosen at runtime. As the cluster must have at least one manager, the first node initializing the cluster is assigned as such. Subsequent nodes joining the cluster are usually added as workers but can be assigned as either. The flexibility means that the entire swarm can be built from a single disk image with little differentiation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Create a new swarm cluster with the initialization command below. Replace the <span style=\"color: #ff0000;\">manager_private_ip<\/span> with the private IP of the host you are creating the cluster on.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker swarm init --advertise-addr&nbsp;<span style=\"color: #ff0000;\">manager_private_ip<\/span><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">When the swarm starts successfully, you can see a docker swarm join command in the output, as shown in the example below. Use the command on your other nodes to join them to the swarm.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker swarm join --token <span style=\"color: #ff0000;\">SWMTKN_token<\/span> <span style=\"color: #ff0000;\">manager_private_ip<\/span><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">When all nodes are connected, use the following command on the manager to check that the nodes are ready to accept tasks.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker node ls\n\nID                          HOSTNAME  STATUS  AVAILABILITY  MANAGER STATUS\n3h39l1ex8c1i5oo2xn4qnwkj6 * node0     Ready   Active        Leader\n4bdtzjfa0qw799e6npb4dxvl1   node1     Ready   Active\ncu0jd5hb2l56swx9siq42f7li   node2     Ready   Active<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">If you run into problems joining nodes to the swarm, you can have the problematic node leave the cluster with the command below.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker swarm leave<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Then try to join again using the token and manager IP address as shown on your manager node. If every node of the cluster leaves, the swarm is disbanded and any node can start a new cluster with the initialization command.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Running services on Docker Swarm<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Services are deployed on the cluster with simple requests for a number of containers of a certain application image. For example, if you deploy 5 replicas of a web host container, the&nbsp;manager takes the request and instructs the worker nodes to run more containers until the number of replicas is fulfilled.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The swarm nodes enforce TLS authentication and encryption to secure communication between nodes. All of this is done by default and requires no additional attention. It is also possible to use self-signed root certificates, but for most cases, it is fine to go with the default implementation.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/upcloud.com\/media\/Swarm-automatic-TLS-1-2.png\" alt=\"Swarm automatic TLS\" class=\"wp-image-2492\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Prepare the cluster for&nbsp;containerized services by creating a network overlay for the containers to attach to.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker network create \n--driver overlay \n--subnet 10.0.0.0\/12 \n--opt encrypted \nservices\n<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Then start a pair of web hosts with the following command.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker service create \n--replicas 2&nbsp;\n--name nginx \n--network services \n--publish 80:80 \nnginx<\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\">ldm0yem039fj9eeet1s26gckt\noverall progress: 2 out of 2 tasks\n1\/2: running   [==================================================&gt;]\n2\/2: running   [==================================================&gt;]\nverify: Service converged\n<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">You can see all currently deployed services with the command below.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker service ls<\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\">ID            NAME   MODE        REPLICAS  IMAGE         PORTS\nldm0yem039fj  nginx  replicated  2\/2       nginx:latest  *:80-&gt;80\/tcp<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">If you want to take a closer look at a specific service, use the next command.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker service ps nginx<\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\">ID            NAME     IMAGE         NODE   DESIRED STATE  CURRENT STATE          ERROR  PORTS\nmem2a0tbqx04  nginx.1  nginx:latest  node1  Running        Running 9 seconds ago\n4sbg87lfh6fo  nginx.2  nginx:latest  node2  Running        Running 9 seconds ago<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The output shows which nodes are running the containers.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Networking in the Swarm cluster<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The swarm manages multi-host networking which supports overlay network services. The cluster manager automatically assigns virtual IP addresses to the containers that join the overlay. The swarm runs an embedded DNS server, which makes it possible for the swarm to also assign each container a unique DNS name that is resolvable from any container within the overlay network.&nbsp;This greatly simplifies service discovery and allows load balancing from the get-go.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/upcloud.com\/media\/Docker-Swarm-Networking-2.png\" alt=\"Docker Swarm networking\" class=\"wp-image-2495\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Docker creates a number of network configurations by default. You can see the currently active networks using the <em>ls<\/em> command in the <em>network<\/em> scope.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker network ls<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Next to test the connection between containers, create a simple <em>busybox<\/em> service with the following command. The sleep parameter ensures the container remains alive long enough for you to have time to connect to it, as without having anything to do the container would simply exit.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker service create \n--name busybox \n--network services \nbusybox \nsleep 3000<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">When the service is running, check which node it is on.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker service ps busybox\n\nID                         NAME       IMAGE    NODE   DESIRED STATE  CURRENT STATE\neh2iznb6b8x8fm79eclmwbv7h  busybox.1  busybox  node1  Running        Running 9 seconds ago\n<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Then, connect to that node through SSH and check the name of the container.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker ps\n\nCONTAINER ID  IMAGE           COMMAND       CREATED        STATUS       PORTS  NAMES\ne5329caea747  busybox:latest  \"sleep 3000\"  2 minutes ago  Up 2 minutes        busybox.1.4czfvm7771qy8lmbiz2bv4nbwdocker<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Use the full container name to open it in a new terminal. For example as below but by using the container name shown in your terminal.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker exec -it busybox.1.4czfvm7771qy8lmbiz2bv4nbwdocker \/bin\/sh<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The following <em>nslookup<\/em> commands only work within a container connected to the overlay. Try to look up the web host service.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">nslookup nginx\n\nServer: 127.0.0.11\nAddress 1: 127.0.0.11\n\nName: nginx\nAddress 1: 10.10.1.2<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Service <em>Address 1<\/em> is the IP address for the load balancer. You can test the connectivity using wget within the busybox.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">wget -O-&nbsp;10.10.1.2<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Alternatively, you can also look up the specific container addresses.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">nslookup tasks.nginx\n\nServer: 127.0.0.11\nAddress 1: 127.0.0.11\n\nName: tasks.nginx\nAddress 1: 10.10.1.4 nginx.1.288l03z4xzjrpfy53aokk1msd.services\nAddress 2: 10.10.1.3 nginx.2.ajyko9rdg8b7oiomd400y240g.services<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Similarly, the direct container addresses are reachable from within the busybox container.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">wget -O- 10.10.1.3<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">In both cases, you should see the default nginx welcome page in the reply.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Service scaling and management<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Services running on the cluster are easily scalable through a single parameter. You can declare the number of replicas a service should have and the swarm manager automatically adapts by adding or removing containers to maintain the desired state.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The swarm also provides resilience through the reconciliation of the desired states. The manager continuously monitors the state of the nodes in the cluster. If a node goes offline, the manager will reconcile the difference in the desired state and the current state by redeploying the lost services on the available nodes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Inspect the web host service currently running on your cluster. Using the <em>\u2013pretty<\/em> parameter formats the output into a more human readable list.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker service inspect --pretty nginx<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Create an additional container for the web host by updating the scale of the service with the following command.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker service scale nginx=3<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Then, check which node was assigned to run the third task. Note that a single node can host multiple containers of the same service.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker service ps nginx<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">If you ever have to scale down the entire cluster or even just temporarily&nbsp;shut down a node, you can prepare the swarm in advance by updating the node availability. Empty one of the nodes in your cluster by changing its availability to <em>drain<\/em> with the command below.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker node update --availability=drain&nbsp;&lt;node hostname&gt;<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The swarm manager will then migrate any containers running on the drained node elsewhere in the cluster. Scale the service back down again.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker service scale nginx=2<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Lastly, return the node availability back to active, therefore allowing new containers to run on it as well.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">docker node update --availability=active &lt;node hostname&gt;<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The cluster is then ready for anything. You may still want to remove the web host containers when you are done with them to free up the port reservations.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Summary<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">That concludes this guide to Docker Swarm for getting started. The inclusion of native Swarm orchestration is a big update, but it is very easy to learn the basics. As a result, managing your swarm is greatly simplified thanks to first-party support.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The automatic TLS configuration enables a secure connection between nodes, along with the already secure UpCloud private network in which only cloud servers on your account are connected. The Swarm networking allows containers to communicate easily using service names through the secure overlay; however, remember to take care when publishing container ports to the public network.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Next, you might wish to look into the load balancing provided by the swarm orchestration. You can read more about the <a rel=\"noopener noreferrer\" href=\"https:\/\/docs.docker.com\/engine\/swarm\/ingress\/\" target=\"_blank\">Swarm mode routing mesh<\/a> in the docker documentation.&nbsp;Together with an external proxy, the swarm can manage an idea backend for distributed services.<\/p>\n","protected":false},"author":3,"featured_media":24573,"comment_status":"open","ping_status":"closed","template":"","community-category":[223],"class_list":["post-2317","tutorial","type-tutorial","status-publish","has-post-thumbnail","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/tutorial\/2317","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/tutorial"}],"about":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/types\/tutorial"}],"author":[{"embeddable":true,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/comments?post=2317"}],"version-history":[{"count":0,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/tutorial\/2317\/revisions"}],"wp:attachment":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/media?parent=2317"}],"wp:term":[{"taxonomy":"community-category","embeddable":true,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/community-category?post=2317"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}