+++ title = "Your First Minikube Helm Deployment" author = ["Elia el Lazkani"] date = 2019-02-10 lastmod = 2019-06-21 tags = ["minikube", "ingress", "helm", "prometheus", "grafana"] categories = ["kubernetes"] draft = false +++ In the last post, we have configured a basic _minikube_ cluster. In this post we will deploy a few items we will need in a cluster and maybe in the future, experiment with it a bit. ## Prerequisite {#prerequisite} During this post and probably during future posts, we will be using _helm_ to deploy to our _minikube_ cluster. Some offered by the helm team, others by the community and maybe our own. We need to install `helm` on our machine. It should be as easy as downloading the binary but if you can find it in your package manager go that route. ## Deploying Tiller {#deploying-tiller} Before we can start with the deployments using `helm`, we need to deploy _tiller_. It's a service that manages communications with the client and deployments. ```text $ helm init --history-max=10 Creating ~/.helm Creating ~/.helm/repository Creating ~/.helm/repository/cache Creating ~/.helm/repository/local Creating ~/.helm/plugins Creating ~/.helm/starters Creating ~/.helm/cache/archive Creating ~/.helm/repository/repositories.yaml Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com Adding local repo with URL: http://127.0.0.1:8879/charts $HELM_HOME has been configured at ~/.helm. Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster. Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy. To prevent this, run ``helm init`` with the --tiller-tls-verify flag. For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation ``` _Tiller_ is deployed, give it a few minutes for the pods to come up. ## Deploy Prometheus {#deploy-prometheus} We often need to monitor multiple aspects of the cluster easily. Sometimes maybe even write our applications to (let's say) publish metrics to prometheus. And I said 'let's say' because technically we offer an endpoint that a prometheus exporter will consume regularly and publish to the prometheus server. Anyway, let's deploy prometheus. ```text $ helm install stable/prometheus-operator --name prometheus-operator --namespace kube-prometheus NAME: prometheus-operator LAST DEPLOYED: Sat Feb 9 18:09:43 2019 NAMESPACE: kube-prometheus STATUS: DEPLOYED RESOURCES: ==> v1/Secret NAME TYPE DATA AGE prometheus-operator-grafana Opaque 3 4s alertmanager-prometheus-operator-alertmanager Opaque 1 4s ==> v1beta1/ClusterRole NAME AGE prometheus-operator-kube-state-metrics 3s psp-prometheus-operator-kube-state-metrics 3s psp-prometheus-operator-prometheus-node-exporter 3s ==> v1/Service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE prometheus-operator-grafana ClusterIP 10.107.125.114 80/TCP 3s prometheus-operator-kube-state-metrics ClusterIP 10.99.250.30 8080/TCP 3s prometheus-operator-prometheus-node-exporter ClusterIP 10.111.99.199 9100/TCP 3s prometheus-operator-alertmanager ClusterIP 10.96.49.73 9093/TCP 3s prometheus-operator-coredns ClusterIP None 9153/TCP 3s prometheus-operator-kube-controller-manager ClusterIP None 10252/TCP 3s prometheus-operator-kube-etcd ClusterIP None 4001/TCP 3s prometheus-operator-kube-scheduler ClusterIP None 10251/TCP 3s prometheus-operator-operator ClusterIP 10.101.253.101 8080/TCP 3s prometheus-operator-prometheus ClusterIP 10.107.117.120 9090/TCP 3s ==> v1beta1/DaemonSet NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE prometheus-operator-prometheus-node-exporter 1 1 0 1 0 3s ==> v1/Deployment NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE prometheus-operator-operator 1 1 1 0 3s ==> v1/ServiceMonitor NAME AGE prometheus-operator-alertmanager 2s prometheus-operator-coredns 2s prometheus-operator-apiserver 2s prometheus-operator-kube-controller-manager 2s prometheus-operator-kube-etcd 2s prometheus-operator-kube-scheduler 2s prometheus-operator-kube-state-metrics 2s prometheus-operator-kubelet 2s prometheus-operator-node-exporter 2s prometheus-operator-operator 2s prometheus-operator-prometheus 2s ==> v1/Pod(related) NAME READY STATUS RESTARTS AGE prometheus-operator-prometheus-node-exporter-fntpx 0/1 ContainerCreating 0 3s prometheus-operator-grafana-8559d7df44-vrm8d 0/3 ContainerCreating 0 2s prometheus-operator-kube-state-metrics-7769f5bd54-6znvh 0/1 ContainerCreating 0 2s prometheus-operator-operator-7967865bf5-cbd6r 0/1 ContainerCreating 0 2s ==> v1beta1/PodSecurityPolicy NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES prometheus-operator-grafana false RunAsAny RunAsAny RunAsAny RunAsAny false configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim prometheus-operator-kube-state-metrics false RunAsAny MustRunAsNonRoot MustRunAs MustRunAs false secret prometheus-operator-prometheus-node-exporter false RunAsAny RunAsAny MustRunAs MustRunAs false configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim,hostPath prometheus-operator-alertmanager false RunAsAny RunAsAny MustRunAs MustRunAs false configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim prometheus-operator-operator false RunAsAny RunAsAny MustRunAs MustRunAs false configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim prometheus-operator-prometheus false RunAsAny RunAsAny MustRunAs MustRunAs false configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim ==> v1/ConfigMap NAME DATA AGE prometheus-operator-grafana-config-dashboards 1 4s prometheus-operator-grafana 1 4s prometheus-operator-grafana-datasource 1 4s prometheus-operator-etcd 1 4s prometheus-operator-grafana-coredns-k8s 1 4s prometheus-operator-k8s-cluster-rsrc-use 1 4s prometheus-operator-k8s-node-rsrc-use 1 4s prometheus-operator-k8s-resources-cluster 1 4s prometheus-operator-k8s-resources-namespace 1 4s prometheus-operator-k8s-resources-pod 1 4s prometheus-operator-nodes 1 4s prometheus-operator-persistentvolumesusage 1 4s prometheus-operator-pods 1 4s prometheus-operator-statefulset 1 4s ==> v1/ClusterRoleBinding NAME AGE prometheus-operator-grafana-clusterrolebinding 3s prometheus-operator-alertmanager 3s prometheus-operator-operator 3s prometheus-operator-operator-psp 3s prometheus-operator-prometheus 3s prometheus-operator-prometheus-psp 3s ==> v1beta1/Role NAME AGE prometheus-operator-grafana 3s ==> v1beta1/Deployment NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE prometheus-operator-kube-state-metrics 1 1 1 0 3s ==> v1/Alertmanager NAME AGE prometheus-operator-alertmanager 3s ==> v1/ServiceAccount NAME SECRETS AGE prometheus-operator-grafana 1 4s prometheus-operator-kube-state-metrics 1 4s prometheus-operator-prometheus-node-exporter 1 4s prometheus-operator-alertmanager 1 4s prometheus-operator-operator 1 4s prometheus-operator-prometheus 1 4s ==> v1/ClusterRole NAME AGE prometheus-operator-grafana-clusterrole 4s prometheus-operator-alertmanager 3s prometheus-operator-operator 3s prometheus-operator-operator-psp 3s prometheus-operator-prometheus 3s prometheus-operator-prometheus-psp 3s ==> v1/Role NAME AGE prometheus-operator-prometheus-config 3s prometheus-operator-prometheus 2s prometheus-operator-prometheus 2s ==> v1beta1/RoleBinding NAME AGE prometheus-operator-grafana 3s ==> v1beta2/Deployment NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE prometheus-operator-grafana 1 1 1 0 3s ==> v1/Prometheus NAME AGE prometheus-operator-prometheus 2s ==> v1beta1/ClusterRoleBinding NAME AGE prometheus-operator-kube-state-metrics 3s psp-prometheus-operator-kube-state-metrics 3s psp-prometheus-operator-prometheus-node-exporter 3s ==> v1/RoleBinding NAME AGE prometheus-operator-prometheus-config 3s prometheus-operator-prometheus 2s prometheus-operator-prometheus 2s ==> v1/PrometheusRule NAME AGE prometheus-operator-alertmanager.rules 2s prometheus-operator-etcd 2s prometheus-operator-general.rules 2s prometheus-operator-k8s.rules 2s prometheus-operator-kube-apiserver.rules 2s prometheus-operator-kube-prometheus-node-alerting.rules 2s prometheus-operator-kube-prometheus-node-recording.rules 2s prometheus-operator-kube-scheduler.rules 2s prometheus-operator-kubernetes-absent 2s prometheus-operator-kubernetes-apps 2s prometheus-operator-kubernetes-resources 2s prometheus-operator-kubernetes-storage 2s prometheus-operator-kubernetes-system 2s prometheus-operator-node.rules 2s prometheus-operator-prometheus-operator 2s prometheus-operator-prometheus.rules 2s NOTES: The Prometheus Operator has been installed. Check its status by running: kubectl --namespace kube-prometheus get pods -l "release=prometheus-operator" Visit [[https://github.com/coreos/prometheus-operator]] for instructions on how to create & configure Alertmanager and Prometheus instances using the Operator. ``` At this point, prometheus has been deployed to the cluster. Give it a few minutes for all the pods to come up. Let's keep on working to get access to the rest of the consoles offered by the prometheus deployment. ## Prometheus Console {#prometheus-console} Let's write an ingress configuration to expose the prometheus console. First off we need to list all the service deployed for prometheus. ```text $ kubectl get service prometheus-operator-prometheus -o yaml -n kube-prometheus apiVersion: v1 kind: Service metadata: creationTimestamp: "2019-02-09T23:09:55Z" labels: app: prometheus-operator-prometheus chart: prometheus-operator-2.1.6 heritage: Tiller release: prometheus-operator name: prometheus-operator-prometheus namespace: kube-prometheus resourceVersion: "10996" selfLink: /api/v1/namespaces/kube-prometheus/services/prometheus-operator-prometheus uid: d038d6fa-2cbf-11e9-b74f-48ea5bb87c0b spec: clusterIP: 10.107.117.120 ports: - name: web port: 9090 protocol: TCP targetPort: web selector: app: prometheus prometheus: prometheus-operator-prometheus sessionAffinity: None type: ClusterIP status: loadBalancer: {} ``` As we can see from the service above, its name is `prometheus-operator-prometheus` and it's listening on port `9090`. So let's write the ingress configuration for it. ```yaml --- apiVersion: extensions/v1beta1 kind: Ingress metadata: name: prometheus-dashboard namespace: kube-prometheus annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: rules: - host: prometheus.kube.local http: paths: - path: / backend: serviceName: prometheus-operator-prometheus servicePort: 9090 ``` Save the file as `kube-prometheus-ingress.yaml` or some such and deploy. ```text $ kubectl apply -f kube-prometheus-ingress.yaml ingress.extensions/prometheus-dashboard created ``` And then add the service host to our `/etc/hosts`. ```text 192.168.39.78 prometheus.kube.local ``` Now you can access from your browser. ## Grafana Console {#grafana-console} Much like what we did with the prometheus console previously, we need to do the same to the grafana dashboard. First step, let's check the service. ```text $ kubectl get service prometheus-operator-grafana -o yaml -n kube-prometheus ``` Gives you the following output. ```yaml --- apiVersion: v1 kind: Service metadata: creationTimestamp: "2019-02-09T23:09:55Z" labels: app: grafana chart: grafana-1.25.0 heritage: Tiller release: prometheus-operator name: prometheus-operator-grafana namespace: kube-prometheus resourceVersion: "10973" selfLink: /api/v1/namespaces/kube-prometheus/services/prometheus-operator-grafana uid: cffe169b-2cbf-11e9-b74f-48ea5bb87c0b spec: clusterIP: 10.107.125.114 ports: - name: service port: 80 protocol: TCP targetPort: 3000 selector: app: grafana release: prometheus-operator sessionAffinity: None type: ClusterIP status: loadBalancer: {} ``` We get `prometheus-operator-grafana` and port `80`. Next is the ingress configuration. ```yaml --- apiVersion: extensions/v1beta1 kind: Ingress metadata: name: prometheus-grafana namespace: kube-prometheus annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: rules: - host: grafana.kube.local http: paths: - path: / backend: serviceName: prometheus-operator-grafana servicePort: 80 ``` Then we deploy. ```text $ kubectl apply -f kube-grafana-ingress.yaml $ ingress.extensions/prometheus-grafana created ``` And let's not forget `/etc/hosts`. ```text 192.168.39.78 grafana.kube.local ``` And the grafana dashboard should appear if you visit .