198 lines
8.4 KiB
Markdown
198 lines
8.4 KiB
Markdown
+++
|
|
title = "Local Kubernetes Cluster on KVM"
|
|
author = ["Elia el Lazkani"]
|
|
date = 2019-02-17
|
|
lastmod = 2019-06-21
|
|
tags = ["rancher", "rancheros", "kvm", "libvirt"]
|
|
categories = ["kubernetes"]
|
|
draft = false
|
|
+++
|
|
|
|
I wanted to explore _kubernetes_ even more for myself and for this blog. I've worked on pieces of this at work but not the totality of the work which I would like to understand for myself. I wanted, also to explore new tools and ways to leverage the power of _kubernetes_.
|
|
|
|
So far, I have been using _minikube_ to do the deployments but there is an inherit restriction that comes with using a single bundled node. Sure, it is easy to get it up and running but at some point I had to use `nodePort` to go around the IP restriction. This is a restriction that you will have in an actual _kubernetes_ cluster but I will show you later how to go around it. For now, let's just get a local cluster up and running.
|
|
|
|
<!--more-->
|
|
|
|
|
|
## Objective {#objective}
|
|
|
|
I needed a local _kubernetes_ cluster using all open source tools and easy to deploy. So I went with using _KVM_ as the hypervisor layer and installed `virt-manager` for shallow management. As an OS, I wanted something light and made for _kubernetes_. As I already know of Rancher (being an easy way to deploy _kubernetes_ and they have done a great job so far since the launch of their Rancer 2.0) I decided to try _RancherOS_. So let's see how all that works together.
|
|
|
|
|
|
## Requirements {#requirements}
|
|
|
|
Let's start by thinking about what we actually need. Rancher, the dashboard they offer is going to need a VM by itself and they [recommend](https://rancher.com/docs/rancher/v2.x/en/quick-start-guide/deployment/quickstart-vagrant/) _4GB of RAM_. I only have _16GB of RAM_ on my machine so I'll have to do the math to see how much I can afford to give this _dashboard_ and _manager_. By looking at the _RancherOS_ hardware [requirements](https://rancher.com/docs/os/v1.x/en/), I can tell that by giving a each node _2GB_ of RAM I should be able to host a _3 node cluster_ and with _2_ more for the _dashboard_ that puts me right on _8GB of RAM_. So we need to create _4 VMs_ with _2GB of RAM_ each.
|
|
|
|
|
|
## Installing RancherOS {#installing-rancheros}
|
|
|
|
Once all 4 nodes have been created, when you boot into the _RancherOS_ [ISO](https://rancher.com/docs/os/v1.x/en/installation/running-rancheros/workstation/boot-from-iso/) do the following.
|
|
|
|
<div class="admonition note">
|
|
<p class="admonition-title">Note</p>
|
|
|
|
Because I was using _libvirt_, I was able to do `virsh console <vm>` and run these commands.
|
|
|
|
</div>
|
|
|
|
|
|
## Virsh Console {#virsh-console}
|
|
|
|
If you are running these VMs on _libvirt_, then you can console into the box and run `vi`.
|
|
|
|
```text
|
|
# virsh list
|
|
Id Name State
|
|
-------------------------
|
|
21 kube01 running
|
|
22 kube02 running
|
|
23 kube03 running
|
|
24 rancher running
|
|
|
|
# virsh console rancher
|
|
```
|
|
|
|
|
|
## Configuration {#configuration}
|
|
|
|
If you read the _RancherOS_ [documentation](https://rancher.com/docs/os/v1.x/en/), you'll find out that you can configure the _OS_ with a `YAML` configuration file so let's do that.
|
|
|
|
```text
|
|
$ vi cloud-config.yml
|
|
```
|
|
|
|
And that file should hold.
|
|
|
|
```yaml
|
|
---
|
|
hostname: rancher.kube.loco
|
|
ssh_authorized_keys:
|
|
- ssh-rsa AAA...
|
|
rancher:
|
|
network:
|
|
interfaces:
|
|
eth0:
|
|
address: 192.168.122.5/24
|
|
dhcp: false
|
|
gateway: 192.168.122.1
|
|
mtu: 1500
|
|
```
|
|
|
|
Make sure that your **public** _ssh key_ is replaced in the example before and if you have a different network configuration for your VMs, change the network configuration here.
|
|
|
|
After you save that file, install the _OS_.
|
|
|
|
```text
|
|
$ sudo ros install -c cloud-config.yml -d /dev/sda
|
|
```
|
|
|
|
Do the same for the rest of the servers and their names and IPs should be as follows (if you are following this tutorial):
|
|
|
|
```text
|
|
192.168.122.5 rancher.kube.loco
|
|
192.168.122.10 kube01.kube.loco
|
|
192.168.122.11 kube02.kube.loco
|
|
192.168.122.12 kube03.kube.loco
|
|
```
|
|
|
|
|
|
## Post Installation Configuration {#post-installation-configuration}
|
|
|
|
After _RancherOS_ has been installed, one will need to configure `/etc/hosts` and it should look like the following if one is working off of the _Rancher_ box.
|
|
|
|
```text
|
|
$ sudo vi /etc/hosts
|
|
```
|
|
|
|
```text
|
|
127.0.0.1 rancher.kube.loco
|
|
192.168.122.5 rancher.kube.loco
|
|
192.168.122.10 kube01.kube.loco
|
|
192.168.122.11 kube02.kube.loco
|
|
192.168.122.12 kube03.kube.loco
|
|
```
|
|
|
|
Do the same on the rest of the servers while changing the `127.0.0.1` hostname to the host of the server.
|
|
|
|
|
|
## Installing Rancher {#installing-rancher}
|
|
|
|
At this point, I have to stress a few facts:
|
|
|
|
- This is not the Rancher recommended way to deploy _kubernetes_.
|
|
- The recommended way is of course [RKE](https://rancher.com/docs/rke/v0.1.x/en/).
|
|
- This is for testing, so I did not take into consideration backup of anything.
|
|
- There are ways to backup Rancher configuration by mounting storage from the `rancher` docker container.
|
|
|
|
If those points are understood, let's go ahead and deploy Rancher.
|
|
First, `$ ssh rancher@192.168.122.5` then:
|
|
|
|
```text
|
|
[rancher@rancher ~]$ docker run -d --restart=unless-stopped -p 80:80 -p 443:443 rancher/rancher
|
|
```
|
|
|
|
Give it a few minutes for the container to come up and the application as well. Meanwhile configure your `/etc/hosts` file on your machine.
|
|
|
|
```text
|
|
192.168.122.5 rancher.kube.loco
|
|
```
|
|
|
|
Now that all that is out of the way, you can login to <https://rancher.kube.loco> and set your `admin` password and the `url` for Rancher.
|
|
|
|
|
|
## Deploying Kubernetes {#deploying-kubernetes}
|
|
|
|
Now that everything is ready, let's deploy _kubernetes_ the easy way.
|
|
|
|
At this point you should be greeted with a page that looks like the
|
|
following.
|
|
|
|
{{< figure src="/ox-hugo/01-add-cluster.png" caption="Figure 1: Add Cluster Page" target="_blank" link="/ox-hugo/01-add-cluster.png" >}}
|
|
|
|
Click on the **Add Cluser**
|
|
|
|
{{< figure src="/ox-hugo/02-custom-cluster.png" caption="Figure 2: Custom Cluster Page" target="_blank" link="/ox-hugo/02-custom-cluster.png" >}}
|
|
|
|
Make sure you choose **Custom** as a _provider_. Then fill in the **Cluser Name** in our case we'll call it **kube**.
|
|
|
|
{{< figure src="/ox-hugo/03-calico-networkProvider.png" caption="Figure 3: Network Provider: Calico (Optional)" target="_blank" link="/ox-hugo/03-calico-networkProvider.png" >}}
|
|
|
|
Optionally, you can choose your **Network Providor**, in my case I chose **Calico**. Then I clicked on **show advanced** at the bottom right corner then expanded the _newly shown tab_ **Advanced Cluster Options**.
|
|
|
|
{{< figure src="/ox-hugo/04-nginx-ingressDisabled.png" caption="Figure 4: Nginx Ingress Disabled" target="_blank" link="/ox-hugo/04-nginx-ingressDisabled.png" >}}
|
|
|
|
We will disable the **Nginx Ingress** and the **Pod Security Policy Support** for the time being. This will become more apparent why in the future, hopefully. Then hit **Next**.
|
|
|
|
{{< figure src="/ox-hugo/05-customize-nodes.png" caption="Figure 5: Customize Nodes" target="_blank" link="/ox-hugo/05-customize-nodes.png" >}}
|
|
|
|
Make sure that you select all **3 Node Roles**. Set the **Public Address** and the **Node Name** to the first node and then copy the command and paste it on the _first_ node.
|
|
|
|
Do the same for _all the rest_. Once the first docker image gets downloaded and ran you should see a message pop at the bottom.
|
|
|
|
{{< figure src="/ox-hugo/06-registered-nodes.png" caption="Figure 6: Registered Nodes" target="_blank" link="/ox-hugo/06-registered-nodes.png" >}}
|
|
|
|
<div class="admonition warning">
|
|
<p class="admonition-title">warning</p>
|
|
|
|
Do **NOT** click _done_ until you see all _3 nodes registered_.
|
|
|
|
</div>
|
|
|
|
|
|
## Finalizing {#finalizing}
|
|
|
|
Now that you have _3 registered nodes_, click **Done** and go grab yourself a cup of coffee. Maybe take a long walk, this will take time. Or if you are curious like me, you'd be looking at the logs, checking the containers in a quad pane `tmux` session.
|
|
|
|
After a long time has passed, our story ends with a refresh and a welcome with this page.
|
|
|
|
{{< figure src="/ox-hugo/07-kubernetes-cluster.png" caption="Figure 7: Kubernetes Cluster" target="_blank" link="/ox-hugo/07-kubernetes-cluster.png" >}}
|
|
|
|
Welcome to your Kubernetes Cluster.
|
|
|
|
|
|
## Conclusion {#conclusion}
|
|
|
|
At this point, you can check that all the nodes are healthy and you got yourself a kubernetes cluster. In future blog posts we will explore an avenue to deploy _multiple ingress controllers_ on the same cluster on the same `port: 80` by giving them each an IP external to the cluster.
|
|
|
|
But for now, you got yourself a kubernetes cluster to play with. Enjoy.
|