.. title: Your First Minikube Helm Deployment
.. date: 2019-02-10
.. updated: 2019-06-21
.. status: published
.. tags: minikube, kubernetes, ingress, helm, prometheus, grafana,
.. category: kubernetes
.. slug: your-first-minikube-helm-deployment
.. authors: Elia el Lazkani
.. description: Deploying your first minikube helm charts.
.. type: text


In the last post, we have configured a basic *minikube* cluster. In this post we will deploy a few items we will need in a cluster and maybe in the future, experiment with it a bit.

.. TEASER_END

Prerequisite
============

During this post and probably during future posts, we will be using *helm* to deploy to our *minikube* cluster. Some offered by the helm team, others by the community and maybe our own. We need to install ``helm`` on our machine. It should be as easy as downloading the binary but if you can find it in your package manager go that route.

Deploying Tiller
================

Before we can start with the deployments using ``helm``, we need to deploy *tiller*. It's a service that manages communications with the client and deployments.

.. code:: text

   $ helm init --history-max=10
   Creating ~/.helm
   Creating ~/.helm/repository
   Creating ~/.helm/repository/cache
   Creating ~/.helm/repository/local
   Creating ~/.helm/plugins
   Creating ~/.helm/starters
   Creating ~/.helm/cache/archive
   Creating ~/.helm/repository/repositories.yaml
   Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
   Adding local repo with URL: http://127.0.0.1:8879/charts
   $HELM_HOME has been configured at ~/.helm.

   Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.

   Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
   To prevent this, run ``helm init`` with the --tiller-tls-verify flag.
   For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation

*Tiller* is deployed, give it a few minutes for the pods to come up.

Deploy Prometheus
=================

We often need to monitor multiple aspects of the cluster easily. Sometimes maybe even write our applications to (let's say) publish metrics to prometheus. And I said 'let's say' because technically we offer an endpoint that a prometheus exporter will consume regularly and publish to the prometheus server. Anyway, let's deploy prometheus.

.. code:: text

   $ helm install stable/prometheus-operator --name prometheus-operator --namespace kube-prometheus
   NAME:   prometheus-operator
   LAST DEPLOYED: Sat Feb  9 18:09:43 2019
   NAMESPACE: kube-prometheus
   STATUS: DEPLOYED

   RESOURCES:
   ==> v1/Secret
   NAME                                           TYPE    DATA  AGE
   prometheus-operator-grafana                    Opaque  3     4s
   alertmanager-prometheus-operator-alertmanager  Opaque  1     4s

   ==> v1beta1/ClusterRole
   NAME                                              AGE
   prometheus-operator-kube-state-metrics            3s
   psp-prometheus-operator-kube-state-metrics        3s
   psp-prometheus-operator-prometheus-node-exporter  3s

   ==> v1/Service
   NAME                                          TYPE       CLUSTER-IP      EXTERNAL-IP  PORT(S)    AGE
   prometheus-operator-grafana                   ClusterIP  10.107.125.114         80/TCP     3s
   prometheus-operator-kube-state-metrics        ClusterIP  10.99.250.30           8080/TCP   3s
   prometheus-operator-prometheus-node-exporter  ClusterIP  10.111.99.199          9100/TCP   3s
   prometheus-operator-alertmanager              ClusterIP  10.96.49.73            9093/TCP   3s
   prometheus-operator-coredns                   ClusterIP  None                   9153/TCP   3s
   prometheus-operator-kube-controller-manager   ClusterIP  None                   10252/TCP  3s
   prometheus-operator-kube-etcd                 ClusterIP  None                   4001/TCP   3s
   prometheus-operator-kube-scheduler            ClusterIP  None                   10251/TCP  3s
   prometheus-operator-operator                  ClusterIP  10.101.253.101         8080/TCP   3s
   prometheus-operator-prometheus                ClusterIP  10.107.117.120         9090/TCP   3s

   ==> v1beta1/DaemonSet
   NAME                                          DESIRED  CURRENT  READY  UP-TO-DATE  AVAILABLE  NODE SELECTOR  AGE
   prometheus-operator-prometheus-node-exporter  1        1        0      1           0                   3s

   ==> v1/Deployment
   NAME                          DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
   prometheus-operator-operator  1        1        1           0          3s

   ==> v1/ServiceMonitor
   NAME                                         AGE
   prometheus-operator-alertmanager             2s
   prometheus-operator-coredns                  2s
   prometheus-operator-apiserver                2s
   prometheus-operator-kube-controller-manager  2s
   prometheus-operator-kube-etcd                2s
   prometheus-operator-kube-scheduler           2s
   prometheus-operator-kube-state-metrics       2s
   prometheus-operator-kubelet                  2s
   prometheus-operator-node-exporter            2s
   prometheus-operator-operator                 2s
   prometheus-operator-prometheus               2s

   ==> v1/Pod(related)
   NAME                                                     READY  STATUS             RESTARTS  AGE
   prometheus-operator-prometheus-node-exporter-fntpx       0/1    ContainerCreating  0         3s
   prometheus-operator-grafana-8559d7df44-vrm8d             0/3    ContainerCreating  0         2s
   prometheus-operator-kube-state-metrics-7769f5bd54-6znvh  0/1    ContainerCreating  0         2s
   prometheus-operator-operator-7967865bf5-cbd6r            0/1    ContainerCreating  0         2s

   ==> v1beta1/PodSecurityPolicy
   NAME                                          PRIV   CAPS      SELINUX           RUNASUSER  FSGROUP    SUPGROUP  READONLYROOTFS  VOLUMES
   prometheus-operator-grafana                   false  RunAsAny  RunAsAny          RunAsAny   RunAsAny   false     configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim
   prometheus-operator-kube-state-metrics        false  RunAsAny  MustRunAsNonRoot  MustRunAs  MustRunAs  false     secret
   prometheus-operator-prometheus-node-exporter  false  RunAsAny  RunAsAny          MustRunAs  MustRunAs  false     configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim,hostPath
   prometheus-operator-alertmanager              false  RunAsAny  RunAsAny          MustRunAs  MustRunAs  false     configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim
   prometheus-operator-operator                  false  RunAsAny  RunAsAny          MustRunAs  MustRunAs  false     configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim
   prometheus-operator-prometheus                false  RunAsAny  RunAsAny          MustRunAs  MustRunAs  false     configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim

   ==> v1/ConfigMap
   NAME                                           DATA  AGE
   prometheus-operator-grafana-config-dashboards  1     4s
   prometheus-operator-grafana                    1     4s
   prometheus-operator-grafana-datasource         1     4s
   prometheus-operator-etcd                       1     4s
   prometheus-operator-grafana-coredns-k8s        1     4s
   prometheus-operator-k8s-cluster-rsrc-use       1     4s
   prometheus-operator-k8s-node-rsrc-use          1     4s
   prometheus-operator-k8s-resources-cluster      1     4s
   prometheus-operator-k8s-resources-namespace    1     4s
   prometheus-operator-k8s-resources-pod          1     4s
   prometheus-operator-nodes                      1     4s
   prometheus-operator-persistentvolumesusage     1     4s
   prometheus-operator-pods                       1     4s
   prometheus-operator-statefulset                1     4s

   ==> v1/ClusterRoleBinding
   NAME                                            AGE
   prometheus-operator-grafana-clusterrolebinding  3s
   prometheus-operator-alertmanager                3s
   prometheus-operator-operator                    3s
   prometheus-operator-operator-psp                3s
   prometheus-operator-prometheus                  3s
   prometheus-operator-prometheus-psp              3s

   ==> v1beta1/Role
   NAME                         AGE
   prometheus-operator-grafana  3s

   ==> v1beta1/Deployment
   NAME                                    DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
   prometheus-operator-kube-state-metrics  1        1        1           0          3s

   ==> v1/Alertmanager
   NAME                              AGE
   prometheus-operator-alertmanager  3s

   ==> v1/ServiceAccount
   NAME                                          SECRETS  AGE
   prometheus-operator-grafana                   1        4s
   prometheus-operator-kube-state-metrics        1        4s
   prometheus-operator-prometheus-node-exporter  1        4s
   prometheus-operator-alertmanager              1        4s
   prometheus-operator-operator                  1        4s
   prometheus-operator-prometheus                1        4s

   ==> v1/ClusterRole
   NAME                                     AGE
   prometheus-operator-grafana-clusterrole  4s
   prometheus-operator-alertmanager         3s
   prometheus-operator-operator             3s
   prometheus-operator-operator-psp         3s
   prometheus-operator-prometheus           3s
   prometheus-operator-prometheus-psp       3s

   ==> v1/Role
   NAME                                   AGE
   prometheus-operator-prometheus-config  3s
   prometheus-operator-prometheus         2s
   prometheus-operator-prometheus         2s

   ==> v1beta1/RoleBinding
   NAME                         AGE
   prometheus-operator-grafana  3s

   ==> v1beta2/Deployment
   NAME                         DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
   prometheus-operator-grafana  1        1        1           0          3s

   ==> v1/Prometheus
   NAME                            AGE
   prometheus-operator-prometheus  2s

   ==> v1beta1/ClusterRoleBinding
   NAME                                              AGE
   prometheus-operator-kube-state-metrics            3s
   psp-prometheus-operator-kube-state-metrics        3s
   psp-prometheus-operator-prometheus-node-exporter  3s

   ==> v1/RoleBinding
   NAME                                   AGE
   prometheus-operator-prometheus-config  3s
   prometheus-operator-prometheus         2s
   prometheus-operator-prometheus         2s

   ==> v1/PrometheusRule
   NAME                                                      AGE
   prometheus-operator-alertmanager.rules                    2s
   prometheus-operator-etcd                                  2s
   prometheus-operator-general.rules                         2s
   prometheus-operator-k8s.rules                             2s
   prometheus-operator-kube-apiserver.rules                  2s
   prometheus-operator-kube-prometheus-node-alerting.rules   2s
   prometheus-operator-kube-prometheus-node-recording.rules  2s
   prometheus-operator-kube-scheduler.rules                  2s
   prometheus-operator-kubernetes-absent                     2s
   prometheus-operator-kubernetes-apps                       2s
   prometheus-operator-kubernetes-resources                  2s
   prometheus-operator-kubernetes-storage                    2s
   prometheus-operator-kubernetes-system                     2s
   prometheus-operator-node.rules                            2s
   prometheus-operator-prometheus-operator                   2s
   prometheus-operator-prometheus.rules                      2s


  NOTES:
  The Prometheus Operator has been installed. Check its status by running:
    kubectl --namespace kube-prometheus get pods -l "release=prometheus-operator"

  Visit https://github.com/coreos/prometheus-operator for instructions on how
  to create & configure Alertmanager and Prometheus instances using the Operator.

At this point, prometheus has been deployed to the cluster. Give it a few minutes for all the pods to come up. Let's keep on working to get access to the rest of the consoles offered by the prometheus deployment.

Prometheus Console
==================

Let's write an ingress configuration to expose the prometheus console. First off we need to list all the service deployed for prometheus.

.. code:: text

   $ kubectl get service prometheus-operator-prometheus -o yaml -n kube-prometheus
   apiVersion: v1
   kind: Service
   metadata:
     creationTimestamp: "2019-02-09T23:09:55Z"
     labels:
       app: prometheus-operator-prometheus
       chart: prometheus-operator-2.1.6
       heritage: Tiller
       release: prometheus-operator
     name: prometheus-operator-prometheus
     namespace: kube-prometheus
     resourceVersion: "10996"
     selfLink: /api/v1/namespaces/kube-prometheus/services/prometheus-operator-prometheus
     uid: d038d6fa-2cbf-11e9-b74f-48ea5bb87c0b
   spec:
     clusterIP: 10.107.117.120
     ports:
     - name: web
       port: 9090
       protocol: TCP
       targetPort: web
     selector:
       app: prometheus
       prometheus: prometheus-operator-prometheus
     sessionAffinity: None
     type: ClusterIP
   status:
     loadBalancer: {}

As we can see from the service above, its name is ``prometheus-operator-prometheus`` and it's listening on port ``9090``. So let's write the ingress configuration for it.

.. code:: yaml

   ---
   apiVersion: extensions/v1beta1
   kind: Ingress
   metadata:
     name: prometheus-dashboard
     namespace: kube-prometheus
     annotations:
       nginx.ingress.kubernetes.io/rewrite-target: /
   spec:
     rules:
     - host: prometheus.kube.local
       http:
         paths:
         - path: /
           backend:
             serviceName: prometheus-operator-prometheus
             servicePort: 9090

Save the file as ``kube-prometheus-ingress.yaml`` or some such and deploy.

.. code:: text

   $ kubectl apply -f kube-prometheus-ingress.yaml
   ingress.extensions/prometheus-dashboard created

And then add the service host to our ``/etc/hosts``.

.. code:: text

   192.168.39.78   prometheus.kube.local

Now you can access http://prometheus.kube.local from your browser.

Grafana Console
===============

Much like what we did with the prometheus console previously, we need to do the same to the grafana dashboard.

First step, let's check the service.

.. code:: text

   $ kubectl get service prometheus-operator-grafana -o yaml -n kube-prometheus
   apiVersion: v1
   kind: Service
   metadata:
     creationTimestamp: "2019-02-09T23:09:55Z"
     labels:
       app: grafana
       chart: grafana-1.25.0
       heritage: Tiller
       release: prometheus-operator
     name: prometheus-operator-grafana
     namespace: kube-prometheus
     resourceVersion: "10973"
     selfLink: /api/v1/namespaces/kube-prometheus/services/prometheus-operator-grafana
     uid: cffe169b-2cbf-11e9-b74f-48ea5bb87c0b
   spec:
     clusterIP: 10.107.125.114
     ports:
     - name: service
       port: 80
       protocol: TCP
       targetPort: 3000
     selector:
       app: grafana
       release: prometheus-operator
     sessionAffinity: None
     type: ClusterIP
   status:
     loadBalancer: {}

We get ``prometheus-operator-grafana`` and port ``80``. Next is the ingress configuration.

.. code:: yaml

   ---
   apiVersion: extensions/v1beta1
   kind: Ingress
   metadata:
     name: prometheus-grafana
     namespace: kube-prometheus
     annotations:
       nginx.ingress.kubernetes.io/rewrite-target: /
   spec:
     rules:
     - host: grafana.kube.local
       http:
         paths:
         - path: /
           backend:
             serviceName: prometheus-operator-grafana
             servicePort: 80

Then we deploy.

.. code:: text

  kubectl apply -f kube-grafana-ingress.yaml
  ingress.extensions/prometheus-grafana created

And let's not forget ``/etc/hosts``.

.. code:: text

   192.168.39.78   grafana.kube.local

And the grafana dashboard should appear if you visit http://grafana.kube.local.
