+++ title = "Local Kubernetes Cluster on KVM" author = ["Elia el Lazkani"] date = 2019-02-17 lastmod = 2019-06-21 tags = ["rancher", "rancheros", "kvm", "libvirt"] categories = ["kubernetes"] draft = false +++ I wanted to explore _kubernetes_ even more for myself and for this blog. I've worked on pieces of this at work but not the totality of the work which I would like to understand for myself. I wanted, also to explore new tools and ways to leverage the power of _kubernetes_. So far, I have been using _minikube_ to do the deployments but there is an inherit restriction that comes with using a single bundled node. Sure, it is easy to get it up and running but at some point I had to use `nodePort` to go around the IP restriction. This is a restriction that you will have in an actual _kubernetes_ cluster but I will show you later how to go around it. For now, let's just get a local cluster up and running. ## Objective {#objective} I needed a local _kubernetes_ cluster using all open source tools and easy to deploy. So I went with using _KVM_ as the hypervisor layer and installed `virt-manager` for shallow management. As an OS, I wanted something light and made for _kubernetes_. As I already know of Rancher (being an easy way to deploy _kubernetes_ and they have done a great job so far since the launch of their Rancer 2.0) I decided to try _RancherOS_. So let's see how all that works together. ## Requirements {#requirements} Let's start by thinking about what we actually need. Rancher, the dashboard they offer is going to need a VM by itself and they [recommend](https://rancher.com/docs/rancher/v2.x/en/quick-start-guide/deployment/quickstart-vagrant/) _4GB of RAM_. I only have _16GB of RAM_ on my machine so I'll have to do the math to see how much I can afford to give this _dashboard_ and _manager_. By looking at the _RancherOS_ hardware [requirements](https://rancher.com/docs/os/v1.x/en/), I can tell that by giving a each node _2GB_ of RAM I should be able to host a _3 node cluster_ and with _2_ more for the _dashboard_ that puts me right on _8GB of RAM_. So we need to create _4 VMs_ with _2GB of RAM_ each. ## Installing RancherOS {#installing-rancheros} Once all 4 nodes have been created, when you boot into the _RancherOS_ [ISO](https://rancher.com/docs/os/v1.x/en/installation/running-rancheros/workstation/boot-from-iso/) do the following.

Note

Because I was using _libvirt_, I was able to do `virsh console ` and run these commands.
## Virsh Console {#virsh-console} If you are running these VMs on _libvirt_, then you can console into the box and run `vi`. ```text # virsh list Id Name State ------------------------- 21 kube01 running 22 kube02 running 23 kube03 running 24 rancher running # virsh console rancher ``` ## Configuration {#configuration} If you read the _RancherOS_ [documentation](https://rancher.com/docs/os/v1.x/en/), you'll find out that you can configure the _OS_ with a `YAML` configuration file so let's do that. ```text $ vi cloud-config.yml ``` And that file should hold. ```yaml --- hostname: rancher.kube.loco ssh_authorized_keys: - ssh-rsa AAA... rancher: network: interfaces: eth0: address: 192.168.122.5/24 dhcp: false gateway: 192.168.122.1 mtu: 1500 ``` Make sure that your **public** _ssh key_ is replaced in the example before and if you have a different network configuration for your VMs, change the network configuration here. After you save that file, install the _OS_. ```text $ sudo ros install -c cloud-config.yml -d /dev/sda ``` Do the same for the rest of the servers and their names and IPs should be as follows (if you are following this tutorial): ```text 192.168.122.5 rancher.kube.loco 192.168.122.10 kube01.kube.loco 192.168.122.11 kube02.kube.loco 192.168.122.12 kube03.kube.loco ``` ## Post Installation Configuration {#post-installation-configuration} After _RancherOS_ has been installed, one will need to configure `/etc/hosts` and it should look like the following if one is working off of the _Rancher_ box. ```text $ sudo vi /etc/hosts ``` ```text 127.0.0.1 rancher.kube.loco 192.168.122.5 rancher.kube.loco 192.168.122.10 kube01.kube.loco 192.168.122.11 kube02.kube.loco 192.168.122.12 kube03.kube.loco ``` Do the same on the rest of the servers while changing the `127.0.0.1` hostname to the host of the server. ## Installing Rancher {#installing-rancher} At this point, I have to stress a few facts: - This is not the Rancher recommended way to deploy _kubernetes_. - The recommended way is of course [RKE](https://rancher.com/docs/rke/v0.1.x/en/). - This is for testing, so I did not take into consideration backup of anything. - There are ways to backup Rancher configuration by mounting storage from the `rancher` docker container. If those points are understood, let's go ahead and deploy Rancher. First, `$ ssh rancher@192.168.122.5` then: ```text [rancher@rancher ~]$ docker run -d --restart=unless-stopped -p 80:80 -p 443:443 rancher/rancher ``` Give it a few minutes for the container to come up and the application as well. Meanwhile configure your `/etc/hosts` file on your machine. ```text 192.168.122.5 rancher.kube.loco ``` Now that all that is out of the way, you can login to and set your `admin` password and the `url` for Rancher. ## Deploying Kubernetes {#deploying-kubernetes} Now that everything is ready, let's deploy _kubernetes_ the easy way. At this point you should be greeted with a page that looks like the following. {{< figure src="/ox-hugo/01-add-cluster.png" caption="Figure 1: Add Cluster Page" target="_blank" link="/ox-hugo/01-add-cluster.png" >}} Click on the **Add Cluser** {{< figure src="/ox-hugo/02-custom-cluster.png" caption="Figure 2: Custom Cluster Page" target="_blank" link="/ox-hugo/02-custom-cluster.png" >}} Make sure you choose **Custom** as a _provider_. Then fill in the **Cluser Name** in our case we'll call it **kube**. {{< figure src="/ox-hugo/03-calico-networkProvider.png" caption="Figure 3: Network Provider: Calico (Optional)" target="_blank" link="/ox-hugo/03-calico-networkProvider.png" >}} Optionally, you can choose your **Network Providor**, in my case I chose **Calico**. Then I clicked on **show advanced** at the bottom right corner then expanded the _newly shown tab_ **Advanced Cluster Options**. {{< figure src="/ox-hugo/04-nginx-ingressDisabled.png" caption="Figure 4: Nginx Ingress Disabled" target="_blank" link="/ox-hugo/04-nginx-ingressDisabled.png" >}} We will disable the **Nginx Ingress** and the **Pod Security Policy Support** for the time being. This will become more apparent why in the future, hopefully. Then hit **Next**. {{< figure src="/ox-hugo/05-customize-nodes.png" caption="Figure 5: Customize Nodes" target="_blank" link="/ox-hugo/05-customize-nodes.png" >}} Make sure that you select all **3 Node Roles**. Set the **Public Address** and the **Node Name** to the first node and then copy the command and paste it on the _first_ node. Do the same for _all the rest_. Once the first docker image gets downloaded and ran you should see a message pop at the bottom. {{< figure src="/ox-hugo/06-registered-nodes.png" caption="Figure 6: Registered Nodes" target="_blank" link="/ox-hugo/06-registered-nodes.png" >}}

warning

Do **NOT** click _done_ until you see all _3 nodes registered_.
## Finalizing {#finalizing} Now that you have _3 registered nodes_, click **Done** and go grab yourself a cup of coffee. Maybe take a long walk, this will take time. Or if you are curious like me, you'd be looking at the logs, checking the containers in a quad pane `tmux` session. After a long time has passed, our story ends with a refresh and a welcome with this page. {{< figure src="/ox-hugo/07-kubernetes-cluster.png" caption="Figure 7: Kubernetes Cluster" target="_blank" link="/ox-hugo/07-kubernetes-cluster.png" >}} Welcome to your Kubernetes Cluster. ## Conclusion {#conclusion} At this point, you can check that all the nodes are healthy and you got yourself a kubernetes cluster. In future blog posts we will explore an avenue to deploy _multiple ingress controllers_ on the same cluster on the same `port: 80` by giving them each an IP external to the cluster. But for now, you got yourself a kubernetes cluster to play with. Enjoy.