blog.lazkani.io/content/posts/building-k3s-on-a-pi.md

+++
title = "Building k3s on a Pi"
author = ["Elia el Lazkani"]
date = 2020-08-09T21:00:00+02:00
lastmod = 2021-06-28T00:00:45+02:00
tags = ["arm", "kubernetes"]
categories = ["k3s"]
draft = false
+++

I have had a **Pi** laying around used for a simple task for a while now.
A few days ago, I was browsing the web, learning more about privacy, when I stumbled upon [AdGuard Home](https://adguard.com/en/welcome.html).

I have been using it as my internal DNS on top of the security and privacy layers I add to my machine.
Its benefits can be argued but it is a DNS after all and I wanted to see what else it can do for me.
Anyway, I digress. I searched to see if I could find a container for **AdGuard Home** and I did.

At this point, I started thinking about what I could do to make the [Pi](https://www.raspberrypi.org/) more useful.

That's when [k3s](https://k3s.io/) came into the picture.

<!--more-->


## Pre-requisites {#pre-requisites}

As this is not a **Pi** tutorial, I am going to be assuming that you have a _Raspberry Pi_ with **Raspberry Pi OS** _Buster_ installed on it.
The assumption does not mean you cannot install any other OS on the Pi and run this setup.
It only means that I have tested this on _Buster_ and that your milage will vary.


## Prepare the Pi {#prepare-the-pi}

Now that you have _Buster_ already installed, let's go ahead and [fix](https://rancher.com/docs/k3s/latest/en/advanced/#enabling-legacy-iptables-on-raspbian-buster) a small default configuration issue with it.

**K3s** uses `iptables` to route things around correctly. _Buster_ uses `nftables` by default, let's switch it to `iptables`.

```text
 $ sudo iptables -F
 $ sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
 $ sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
 $ sudo reboot
```

At this point, your _Pi_ should reboot. Your **OS** is configured for the next step.


## Pre-install Configuration {#pre-install-configuration}

After testing **k3s** a few times, I found out that by _default_ it will deploy a few extra services like [Traefik](https://docs.traefik.io/).

Unfortunately, just like anything the _default_ configuration is just that. It's plain and not very useful from the start. You will need to tweak it.

This step could be done either _post_ or _pre_ deploy. Figuring out the _pre-deploy_ is a bit more involving but a bit more fun as well.

The first thing you need to know is that the normal behavior of **k3s** is to deploy anything found in `/var/lib/rancher/k3s/server/manifests/`.
So a good first step is, of course, to proceed with creating that.

```text
 $ mkdir -p /var/lib/rancher/k3s/server/manifests/
```

The other thing to know is that **k3s** can deploy _Helm Charts_.
It will create the _manifests_ it will deploy by default, before beginning the setup, in the manifest path I mentioned.
If you would like to see what it deployed and how, visit that path after **k3s** runs.
I did, and I took their configuration of **Traefik** which I was unhappy with its _defaults_.

My next step was securing the _defaults_ as much as possible and I found out that **Traefik** can do [basic authentication](https://docs.traefik.io/v2.0/middlewares/basicauth/).
As a starting point, that's great. Let's create the credentials.

```text
 $ htpasswd -c ./auth myUser
```

That was easy so far. Let's turn up the notch and create the manifest for **k3s**.

Create `traefik.yaml` in `/var/lib/rancher/k3s/server/manifests/` with the following content.

```yaml
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: traefik
  namespace: kube-system
spec:
  chart: https://%{KUBERNETES_API}%/static/charts/traefik-1.81.0.tgz
  valuesContent: |-
    rbac:
      enabled: true
    ssl:
      enabled: true
    dashboard:
      enabled: true
      domain: traefik-ui.example.com
      auth:
        basic:
          myUser: $ars3$4A5tdstr$trSDDa4467Tsa54sTs.
    metrics:
      prometheus:
        enabled: false
    kubernetes:
      ingressEndpoint:
        useDefaultPublishedService: true
    image: "rancher/library-traefik"
    tolerations:
      - key: "CriticalAddonsOnly"
        operator: "Exists"
      - key: "node-role.kubernetes.io/master"
        operator: "Exists"
        effect: "NoSchedule"
```

It's a **Pi**, I don't need prometheus so I disabled it.
I also enabled the dashboard and added the credentials we created in the previous step.

Now, the _Helm Chart_ will deploy an ingress and expose the dashboard for you on the value of `domain`.

<div class="admonition note">
<p class="admonition-title">Note</p>

I figured out the values to set in `valuesContent` by reading the _Helm Chart_

</div>


## K3s {#k3s}

If everything is in place, you are ready to proceed.
You can install **k3s**, now, but before I get to that step, I will say a few things about **k3s**.

**K3s** has a smaller feature set than **k8s**, hence the smaller footprint.
Read the documentation to see if you need any of the missing features.
The second thing to mention is that **k3s** is a one binary deploy that uses **containerd**.
That's why we will use the script installation method as it adds the necessary **systemd** configuration for us.
It is a nice gesture.

Let's do that, shall we ?

```text
 $ curl -sfL https://get.k3s.io | sh -s - --no-deploy traefik
```

<div class="admonition note">
<p class="admonition-title">Note</p>

We need to make sure that **k3s** does not deploy its own **traefik** but ours.
Make sure to add `--no-deploy traefik` to our deployment command.

</div>

Point `traefik.example.com` to your **Pi** `IP` in `/etc/hosts` on your machine.

```text
 traefik.example.com  192.168.0.5
```

When the installation command is done, you should be able to visit [http://traefik.example.com/](http://traefik.example.com/)

You can get the _kubeconfig_ from the _Raspberry Pi_, you can find it in `/etc/rancher/k3s/k3s.yaml`. You will need to change the `server` **IP**.


## Conclusion {#conclusion}

If you've made it so far, you should have a **k3s** cluster running on a single _Raspberry Pi_.
The next steps you might want to look into is disable the _metrics_ server and use the resources for other things.