chore(): Publishes a blog post about Swarm

This commit is contained in:
Elia el Lazkani 2022-08-25 01:15:30 +02:00
parent 10fdf0cc0f
commit 795a44f37e
2 changed files with 515 additions and 0 deletions

View file

@ -2559,6 +2559,259 @@ with the /templating feature/, offers us another flexible way to serve a big
variety of /static sites/. Using them in combination open a wide range of variety of /static sites/. Using them in combination open a wide range of
possibilities. possibilities.
*** DONE Raspberry Pi, Container Orchestration and Swarm right at home :docker:linux:arm:swarm:traefik:raspberry_pi:
:PROPERTIES:
:EXPORT_HUGO_LASTMOD: 2022-08-24
:EXPORT_DATE: 2022-08-24
:EXPORT_FILE_NAME: raspberry-pi-container-orchestration-and-swarm-right-at-home
:CUSTOM_ID: raspberry-pi-container-orchestration-and-swarm-right-at-home
:END:
When I started looking into solutions for my home container orchestration, I
wanted a solution that runs on my 2 Raspberry Pis. These beasts have 4 virtual
CPUs and a whoping 1GB of memory each. In other words, not a lot of resources to
go around. What can I run on these? I wonder!
#+hugo: more
**** Consideration
If we look at the state of /container orchestration/ today, we see that
/Kubernetes/ domates the space. /Kubernetes/ is awesome, but will it run on my
Pis ? I doubt it.
Fret not ! There are other, /more lightweight/, solutions out there. Let's
discuss them briefly.
***** K3s
I have experience with /K3s/. I even wrote a blog [[#building-k3s-on-a-pi][post]] on it. Unfortunately, I
found that /K3s/ uses almost half of the memory resources of the Pis to run.
That's too much overhead lost.
***** MicroK8s
/MicroK8s/ is a Canonical project. It has similarities to /K3s/ in the way of
easy deployment and lightweight focus. The end result is also extremly similar
to /K3s/ in resource usage.
***** Nomad
/Nomad/ is a /HashiCorp/ product and just all their other products, it is very
well designed, very robust and extremly versatile. Running it on the Pis was a
breeze, it barely used any resources.
It shoulds great so far, doesn't it ? Well, sort of. The deployment and
configuration of /Nomad/ is a bit tricky and requires a bit of moving
components. Those can be automated with /Ansible/ eventually. Aside that,
/Nomad/ requires extra configuration to install and enable CNI and service
discovery.
Finally, it has a steep learning curve to deploy containers in the cluster and
you have HCL to deal with.
***** Swarm
I was surprised to find that not only /Docker Swarm/ is still alive, it also
became a mode which comes preshipped with /docker/ since a few years ago.
I also found out that /Swarm/ has great /Ansible/ integration, for both
initializing and creating the cluster and deploying /stacks/ and /services/ into
it. After all, if you are already familiar with /docker-compose/, you'll feel
right at home.
**** Setting up a Swarm cluster
I set up to deploy my /Swarm Cluster/ and manage it using /Ansible/. I didn't
want to do the work again in the future and I wanted to go the IaC
(/Infrastructure as Code/) route, as should you.
At this stage, I have to take a few assumptions. I assume that you already have
at least 2 machines with a Linux Distribution installed on them. I, also, assume
that /docker/ is already installed and running on both machines. Finally, all
the dependencies required to run /Ansible/ on both hosts (~python3-docker~ and
~python3-jsondiff~ on /Ubuntu/).
There are *two* types of /nodes/ in a /Swarm/ cluster; ~manager~ and ~worker~.
The *first* node used to initialize the cluster is the /leader/ node which is
also a ~manager~ node.
***** Leader
For the ~leader~ node, our tasks are going to be initializing the cluster.
Before we do so, let's create our /quick and dirty/ *Ansible* ~inventory~ file.
#+begin_src yaml
---
all:
hosts:
children:
leader:
hosts:
node001:
ansible_host: 192.168.0.100
ansible_user: user
ansible_port: 22
ansible_become: yes
ansible_become_method: sudo
manager:
worker:
hosts:
node002:
ansible_host: 192.168.0.101
ansible_user: user
ansible_port: 22
ansible_become: yes
ansible_become_method: sudo
#+end_src
#+BEGIN_EXPORT html
<div class="admonition warning">
<p class="admonition-title">warning</p>
#+END_EXPORT
This isn't meant to be deployed in *production* in a /professional/ setting. It
goes without saying, the ~leader~ is static, not highly available and prone to
failure. The ~manager~ and ~worker~ node tasks are, also, dependent on the
successful run of the initialization task on the ~leader~.
#+BEGIN_EXPORT html
</div>
#+END_EXPORT
Now that we've taken care of categorizing the nodes and writing the /Ansible/
~inventory~, let's initialize a /Swarm/ cluster.
#+begin_src yaml
---
- name: Init a new swarm cluster
community.docker.docker_swarm:
state: present
advertise_addr: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}"
register: clustering_swarm_cluster
#+end_src
#+BEGIN_EXPORT html
<div class="admonition note">
<p class="admonition-title">Note</p>
#+END_EXPORT
We use ~hostvars[inventory_hostname]['ansible_default_ipv4']['address']~ which
returns the IP address of the node itself. This is the IP adress used to advertise.
#+BEGIN_EXPORT html
</div>
#+END_EXPORT
#+BEGIN_EXPORT html
<div class="admonition note">
<p class="admonition-title">Note</p>
#+END_EXPORT
We use ~register~ to save the returned response from the cluster initialization
into a new variable we called ~clustering_swarm_cluster~. This will come handy later.
#+BEGIN_EXPORT html
</div>
#+END_EXPORT
This should take care of initializing a new /Swarm/ cluster.
You can verify if /Swarm/ is running.
#+begin_src shell
$ docker system info 2>&1 | grep Swarm
Swarm: active
#+end_src
***** Manager
If you have a larger number of nodes, you might require more than one ~manager~
node. To join more /managers/ to the cluster, we can use the power of /Ansible/ again.
#+begin_src yaml
---
- name: Add manager node to Swarm cluster
community.docker.docker_swarm:
state: join
advertise_addr: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}"
join_token: "{{ hostvars[groups['leader'][0]]['clustering_swarm_cluster']['swarm_facts']['JoinTokens']['Manager'] }}"
remote_addrs: [ "{{ hostvars[groups['leader'][0]]['ansible_default_ipv4']['address'] }}:2377" ]
#+end_src
#+BEGIN_EXPORT html
<div class="admonition note">
<p class="admonition-title">Note</p>
#+END_EXPORT
We access the token we saved earlier on the ~leader~ to join a ~manager~ to the cluster using ~hostvars[groups['leader'][0]]['clustering_swarm_cluster']['swarm_facts']['JoinTokens']['Manager']~.
#+BEGIN_EXPORT html
</div>
#+END_EXPORT
#+BEGIN_EXPORT html
<div class="admonition note">
<p class="admonition-title">Note</p>
#+END_EXPORT
If we can get a hostvar from a different node, we can also get the IP of such
node with ~hostvars[groups['leader'][0]]['ansible_default_ipv4']['address']~.
#+BEGIN_EXPORT html
</div>
#+END_EXPORT
Now that we've taken care of the ~manager~ code, let's work on the ~worker~ nodes.
***** Worker
Just as easily as we created the /task/ to *join* a ~manager~ node to the cluster,
we do the same for the ~worker~.
#+begin_src yaml
---
- name: Add worker node to Swarm cluster
community.docker.docker_swarm:
state: join
advertise_addr: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}"
join_token: "{{ hostvars[groups['leader'][0]]['clustering_swarm_cluster']['swarm_facts']['JoinTokens']['Worker'] }}"
remote_addrs: [ "{{ hostvars[groups['leader'][0]]['ansible_default_ipv4']['address'] }}:2377" ]
#+end_src
#+BEGIN_EXPORT html
<div class="admonition note">
<p class="admonition-title">Note</p>
#+END_EXPORT
Déjà vu when it comes to the ~join_token~, except that we use the ~worker~ token instead.
#+BEGIN_EXPORT html
</div>
#+END_EXPORT
The /glue code/ you're looking for that does the magic is this.
#+begin_src yaml
---
- name: Bootstrap Swarm depedencies
include_tasks: common.yml
- name: Bootstrap leader node
include_tasks: leader.yml
when: inventory_hostname in groups['leader']
- name: Bootstrap manager node
include_tasks: manager.yml
when: inventory_hostname in groups['manager']
- name: Bootstrap worker node
include_tasks: worker.yml
when: inventory_hostname in groups['worker']
#+end_src
Each of the tasks described above should be in its own file, as shown in the
/glue code/, and they will *only* run on the group they are meant to run on.
Following these tasks, I ended up with the cluster below.
#+begin_src shell
# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
h4scu4nry2r9p129rsdt88ae2 * node001 Ready Active Leader 20.10.17
uyn43a9tsdn2n435untva9pae node002 Ready Active 20.10.17
#+end_src
There, we see both nodes and they both seem to be in a ~Ready~ state.
**** Conclusion
If you're /outside/ a professional setting and you find yourself needing to run a
container orchestration platform, some platforms might be overkill. /Docker
Swarm/ has great community support in /Ansible/ making the management of small
clusters on low resource devices extremly easy. It comes with the added bonus of
having built-in /service discovery/ and /networking/. Give it a try, you might
be pleasently surprised like I was.
** K3s :@k3s: ** K3s :@k3s:
*** DONE Building k3s on a Pi :arm:kubernetes: *** DONE Building k3s on a Pi :arm:kubernetes:
:PROPERTIES: :PROPERTIES:

View file

@ -0,0 +1,262 @@
+++
title = "Raspberry Pi, Container Orchestration and Swarm right at home"
author = ["Elia el Lazkani"]
date = 2022-08-24
lastmod = 2022-08-24
tags = ["docker", "linux", "arm", "swarm", "traefik", "raspberry-pi"]
categories = ["container"]
draft = false
+++
When I started looking into solutions for my home container orchestration, I
wanted a solution that runs on my 2 Raspberry Pis. These beasts have 4 virtual
CPUs and a whoping 1GB of memory each. In other words, not a lot of resources to
go around. What can I run on these? I wonder!
<!--more-->
## Consideration {#consideration}
If we look at the state of _container orchestration_ today, we see that
_Kubernetes_ domates the space. _Kubernetes_ is awesome, but will it run on my
Pis ? I doubt it.
Fret not ! There are other, _more lightweight_, solutions out there. Let's
discuss them briefly.
### K3s {#k3s}
I have experience with _K3s_. I even wrote a blog [post]({{< relref "building-k3s-on-a-pi" >}}) on it. Unfortunately, I
found that _K3s_ uses almost half of the memory resources of the Pis to run.
That's too much overhead lost.
### MicroK8s {#microk8s}
_MicroK8s_ is a Canonical project. It has similarities to _K3s_ in the way of
easy deployment and lightweight focus. The end result is also extremly similar
to _K3s_ in resource usage.
### Nomad {#nomad}
_Nomad_ is a _HashiCorp_ product and just all their other products, it is very
well designed, very robust and extremly versatile. Running it on the Pis was a
breeze, it barely used any resources.
It shoulds great so far, doesn't it ? Well, sort of. The deployment and
configuration of _Nomad_ is a bit tricky and requires a bit of moving
components. Those can be automated with _Ansible_ eventually. Aside that,
_Nomad_ requires extra configuration to install and enable CNI and service
discovery.
Finally, it has a steep learning curve to deploy containers in the cluster and
you have HCL to deal with.
### Swarm {#swarm}
I was surprised to find that not only _Docker Swarm_ is still alive, it also
became a mode which comes preshipped with _docker_ since a few years ago.
I also found out that _Swarm_ has great _Ansible_ integration, for both
initializing and creating the cluster and deploying _stacks_ and _services_ into
it. After all, if you are already familiar with _docker-compose_, you'll feel
right at home.
## Setting up a Swarm cluster {#setting-up-a-swarm-cluster}
I set up to deploy my _Swarm Cluster_ and manage it using _Ansible_. I didn't
want to do the work again in the future and I wanted to go the IaC
(_Infrastructure as Code_) route, as should you.
At this stage, I have to take a few assumptions. I assume that you already have
at least 2 machines with a Linux Distribution installed on them. I, also, assume
that _docker_ is already installed and running on both machines. Finally, all
the dependencies required to run _Ansible_ on both hosts (`python3-docker` and
`python3-jsondiff` on _Ubuntu_).
There are **two** types of _nodes_ in a _Swarm_ cluster; `manager` and `worker`.
The **first** node used to initialize the cluster is the _leader_ node which is
also a `manager` node.
### Leader {#leader}
For the `leader` node, our tasks are going to be initializing the cluster.
Before we do so, let's create our _quick and dirty_ **Ansible** `inventory` file.
```yaml
---
all:
hosts:
children:
leader:
hosts:
node001:
ansible_host: 192.168.0.100
ansible_user: user
ansible_port: 22
ansible_become: yes
ansible_become_method: sudo
manager:
worker:
hosts:
node002:
ansible_host: 192.168.0.101
ansible_user: user
ansible_port: 22
ansible_become: yes
ansible_become_method: sudo
```
<div class="admonition warning">
<p class="admonition-title">warning</p>
This isn't meant to be deployed in **production** in a _professional_ setting. It
goes without saying, the `leader` is static, not highly available and prone to
failure. The `manager` and `worker` node tasks are, also, dependent on the
successful run of the initialization task on the `leader`.
</div>
Now that we've taken care of categorizing the nodes and writing the _Ansible_
`inventory`, let's initialize a _Swarm_ cluster.
```yaml
---
- name: Init a new swarm cluster
community.docker.docker_swarm:
state: present
advertise_addr: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}"
register: clustering_swarm_cluster
```
<div class="admonition note">
<p class="admonition-title">Note</p>
We use `hostvars[inventory_hostname]['ansible_default_ipv4']['address']` which
returns the IP address of the node itself. This is the IP adress used to advertise.
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
We use `register` to save the returned response from the cluster initialization
into a new variable we called `clustering_swarm_cluster`. This will come handy later.
</div>
This should take care of initializing a new _Swarm_ cluster.
You can verify if _Swarm_ is running.
```shell
$ docker system info 2>&1 | grep Swarm
Swarm: active
```
### Manager {#manager}
If you have a larger number of nodes, you might require more than one `manager`
node. To join more _managers_ to the cluster, we can use the power of _Ansible_ again.
```yaml
---
- name: Add manager node to Swarm cluster
community.docker.docker_swarm:
state: join
advertise_addr: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}"
join_token: "{{ hostvars[groups['leader'][0]]['clustering_swarm_cluster']['swarm_facts']['JoinTokens']['Manager'] }}"
remote_addrs: [ "{{ hostvars[groups['leader'][0]]['ansible_default_ipv4']['address'] }}:2377" ]
```
<div class="admonition note">
<p class="admonition-title">Note</p>
We access the token we saved earlier on the `leader` to join a `manager` to the cluster using `hostvars[groups['leader'][0]]['clustering_swarm_cluster']['swarm_facts']['JoinTokens']['Manager']`.
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
If we can get a hostvar from a different node, we can also get the IP of such
node with `hostvars[groups['leader'][0]]['ansible_default_ipv4']['address']`.
</div>
Now that we've taken care of the `manager` code, let's work on the `worker` nodes.
### Worker {#worker}
Just as easily as we created the _task_ to **join** a `manager` node to the cluster,
we do the same for the `worker`.
```yaml
---
- name: Add worker node to Swarm cluster
community.docker.docker_swarm:
state: join
advertise_addr: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}"
join_token: "{{ hostvars[groups['leader'][0]]['clustering_swarm_cluster']['swarm_facts']['JoinTokens']['Worker'] }}"
remote_addrs: [ "{{ hostvars[groups['leader'][0]]['ansible_default_ipv4']['address'] }}:2377" ]
```
<div class="admonition note">
<p class="admonition-title">Note</p>
Déjà vu when it comes to the `join_token`, except that we use the `worker` token instead.
</div>
The _glue code_ you're looking for that does the magic is this.
```yaml
---
- name: Bootstrap Swarm depedencies
include_tasks: common.yml
- name: Bootstrap leader node
include_tasks: leader.yml
when: inventory_hostname in groups['leader']
- name: Bootstrap manager node
include_tasks: manager.yml
when: inventory_hostname in groups['manager']
- name: Bootstrap worker node
include_tasks: worker.yml
when: inventory_hostname in groups['worker']
```
Each of the tasks described above should be in its own file, as shown in the
_glue code_, and they will **only** run on the group they are meant to run on.
Following these tasks, I ended up with the cluster below.
```shell
# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
h4scu4nry2r9p129rsdt88ae2 * node001 Ready Active Leader 20.10.17
uyn43a9tsdn2n435untva9pae node002 Ready Active 20.10.17
```
There, we see both nodes and they both seem to be in a `Ready` state.
## Conclusion {#conclusion}
If you're _outside_ a professional setting and you find yourself needing to run a
container orchestration platform, some platforms might be overkill. _Docker
Swarm_ has great community support in _Ansible_ making the management of small
clusters on low resource devices extremly easy. It comes with the added bonus of
having built-in _service discovery_ and _networking_. Give it a try, you might
be pleasently surprised like I was.