.. title: Your First Minikube Helm Deployment .. date: 2019-02-10 .. updated: 2019-06-21 .. status: published .. tags: minikube, kubernetes, ingress, helm, prometheus, grafana, .. category: kubernetes .. slug: your-first-minikube-helm-deployment .. authors: Elia El Lazkani .. description: Deploying your first minikube helm charts. .. type: text In the last post, we have configured a basic *minikube* cluster. In this post we will deploy a few items we will need in a cluster and maybe in the future, experiment with it a bit. .. TEASER_END Prerequisite ============ During this post and probably during future posts, we will be using *helm* to deploy to our *minikube* cluster. Some offered by the helm team, others by the community and maybe our own. We need to install ``helm`` on our machine. It should be as easy as downloading the binary but if you can find it in your package manager go that route. Deploying Tiller ================ Before we can start with the deployments using ``helm``, we need to deploy *tiller*. It's a service that manages communications with the client and deployments. .. code:: text $ helm init --history-max=10 Creating ~/.helm Creating ~/.helm/repository Creating ~/.helm/repository/cache Creating ~/.helm/repository/local Creating ~/.helm/plugins Creating ~/.helm/starters Creating ~/.helm/cache/archive Creating ~/.helm/repository/repositories.yaml Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com Adding local repo with URL: http://127.0.0.1:8879/charts $HELM_HOME has been configured at ~/.helm. Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster. Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy. To prevent this, run ``helm init`` with the --tiller-tls-verify flag. For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation *Tiller* is deployed, give it a few minutes for the pods to come up. Deploy Prometheus ================= We often need to monitor multiple aspects of the cluster easily. Sometimes maybe even write our applications to (let's say) publish metrics to prometheus. And I said 'let's say' because technically we offer an endpoint that a prometheus exporter will consume regularly and publish to the prometheus server. Anyway, let's deploy prometheus. .. code:: text $ helm install stable/prometheus-operator --name prometheus-operator --namespace kube-prometheus NAME: prometheus-operator LAST DEPLOYED: Sat Feb 9 18:09:43 2019 NAMESPACE: kube-prometheus STATUS: DEPLOYED RESOURCES: ==> v1/Secret NAME TYPE DATA AGE prometheus-operator-grafana Opaque 3 4s alertmanager-prometheus-operator-alertmanager Opaque 1 4s ==> v1beta1/ClusterRole NAME AGE prometheus-operator-kube-state-metrics 3s psp-prometheus-operator-kube-state-metrics 3s psp-prometheus-operator-prometheus-node-exporter 3s ==> v1/Service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE prometheus-operator-grafana ClusterIP 10.107.125.114 80/TCP 3s prometheus-operator-kube-state-metrics ClusterIP 10.99.250.30 8080/TCP 3s prometheus-operator-prometheus-node-exporter ClusterIP 10.111.99.199 9100/TCP 3s prometheus-operator-alertmanager ClusterIP 10.96.49.73 9093/TCP 3s prometheus-operator-coredns ClusterIP None 9153/TCP 3s prometheus-operator-kube-controller-manager ClusterIP None 10252/TCP 3s prometheus-operator-kube-etcd ClusterIP None 4001/TCP 3s prometheus-operator-kube-scheduler ClusterIP None 10251/TCP 3s prometheus-operator-operator ClusterIP 10.101.253.101 8080/TCP 3s prometheus-operator-prometheus ClusterIP 10.107.117.120 9090/TCP 3s ==> v1beta1/DaemonSet NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE prometheus-operator-prometheus-node-exporter 1 1 0 1 0 3s ==> v1/Deployment NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE prometheus-operator-operator 1 1 1 0 3s ==> v1/ServiceMonitor NAME AGE prometheus-operator-alertmanager 2s prometheus-operator-coredns 2s prometheus-operator-apiserver 2s prometheus-operator-kube-controller-manager 2s prometheus-operator-kube-etcd 2s prometheus-operator-kube-scheduler 2s prometheus-operator-kube-state-metrics 2s prometheus-operator-kubelet 2s prometheus-operator-node-exporter 2s prometheus-operator-operator 2s prometheus-operator-prometheus 2s ==> v1/Pod(related) NAME READY STATUS RESTARTS AGE prometheus-operator-prometheus-node-exporter-fntpx 0/1 ContainerCreating 0 3s prometheus-operator-grafana-8559d7df44-vrm8d 0/3 ContainerCreating 0 2s prometheus-operator-kube-state-metrics-7769f5bd54-6znvh 0/1 ContainerCreating 0 2s prometheus-operator-operator-7967865bf5-cbd6r 0/1 ContainerCreating 0 2s ==> v1beta1/PodSecurityPolicy NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES prometheus-operator-grafana false RunAsAny RunAsAny RunAsAny RunAsAny false configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim prometheus-operator-kube-state-metrics false RunAsAny MustRunAsNonRoot MustRunAs MustRunAs false secret prometheus-operator-prometheus-node-exporter false RunAsAny RunAsAny MustRunAs MustRunAs false configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim,hostPath prometheus-operator-alertmanager false RunAsAny RunAsAny MustRunAs MustRunAs false configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim prometheus-operator-operator false RunAsAny RunAsAny MustRunAs MustRunAs false configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim prometheus-operator-prometheus false RunAsAny RunAsAny MustRunAs MustRunAs false configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim ==> v1/ConfigMap NAME DATA AGE prometheus-operator-grafana-config-dashboards 1 4s prometheus-operator-grafana 1 4s prometheus-operator-grafana-datasource 1 4s prometheus-operator-etcd 1 4s prometheus-operator-grafana-coredns-k8s 1 4s prometheus-operator-k8s-cluster-rsrc-use 1 4s prometheus-operator-k8s-node-rsrc-use 1 4s prometheus-operator-k8s-resources-cluster 1 4s prometheus-operator-k8s-resources-namespace 1 4s prometheus-operator-k8s-resources-pod 1 4s prometheus-operator-nodes 1 4s prometheus-operator-persistentvolumesusage 1 4s prometheus-operator-pods 1 4s prometheus-operator-statefulset 1 4s ==> v1/ClusterRoleBinding NAME AGE prometheus-operator-grafana-clusterrolebinding 3s prometheus-operator-alertmanager 3s prometheus-operator-operator 3s prometheus-operator-operator-psp 3s prometheus-operator-prometheus 3s prometheus-operator-prometheus-psp 3s ==> v1beta1/Role NAME AGE prometheus-operator-grafana 3s ==> v1beta1/Deployment NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE prometheus-operator-kube-state-metrics 1 1 1 0 3s ==> v1/Alertmanager NAME AGE prometheus-operator-alertmanager 3s ==> v1/ServiceAccount NAME SECRETS AGE prometheus-operator-grafana 1 4s prometheus-operator-kube-state-metrics 1 4s prometheus-operator-prometheus-node-exporter 1 4s prometheus-operator-alertmanager 1 4s prometheus-operator-operator 1 4s prometheus-operator-prometheus 1 4s ==> v1/ClusterRole NAME AGE prometheus-operator-grafana-clusterrole 4s prometheus-operator-alertmanager 3s prometheus-operator-operator 3s prometheus-operator-operator-psp 3s prometheus-operator-prometheus 3s prometheus-operator-prometheus-psp 3s ==> v1/Role NAME AGE prometheus-operator-prometheus-config 3s prometheus-operator-prometheus 2s prometheus-operator-prometheus 2s ==> v1beta1/RoleBinding NAME AGE prometheus-operator-grafana 3s ==> v1beta2/Deployment NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE prometheus-operator-grafana 1 1 1 0 3s ==> v1/Prometheus NAME AGE prometheus-operator-prometheus 2s ==> v1beta1/ClusterRoleBinding NAME AGE prometheus-operator-kube-state-metrics 3s psp-prometheus-operator-kube-state-metrics 3s psp-prometheus-operator-prometheus-node-exporter 3s ==> v1/RoleBinding NAME AGE prometheus-operator-prometheus-config 3s prometheus-operator-prometheus 2s prometheus-operator-prometheus 2s ==> v1/PrometheusRule NAME AGE prometheus-operator-alertmanager.rules 2s prometheus-operator-etcd 2s prometheus-operator-general.rules 2s prometheus-operator-k8s.rules 2s prometheus-operator-kube-apiserver.rules 2s prometheus-operator-kube-prometheus-node-alerting.rules 2s prometheus-operator-kube-prometheus-node-recording.rules 2s prometheus-operator-kube-scheduler.rules 2s prometheus-operator-kubernetes-absent 2s prometheus-operator-kubernetes-apps 2s prometheus-operator-kubernetes-resources 2s prometheus-operator-kubernetes-storage 2s prometheus-operator-kubernetes-system 2s prometheus-operator-node.rules 2s prometheus-operator-prometheus-operator 2s prometheus-operator-prometheus.rules 2s NOTES: The Prometheus Operator has been installed. Check its status by running: kubectl --namespace kube-prometheus get pods -l "release=prometheus-operator" Visit https://github.com/coreos/prometheus-operator for instructions on how to create & configure Alertmanager and Prometheus instances using the Operator. At this point, prometheus has been deployed to the cluster. Give it a few minutes for all the pods to come up. Let's keep on working to get access to the rest of the consoles offered by the prometheus deployment. Prometheus Console ================== Let's write an ingress configuration to expose the prometheus console. First off we need to list all the service deployed for prometheus. .. code:: text $ kubectl get service prometheus-operator-prometheus -o yaml -n kube-prometheus apiVersion: v1 kind: Service metadata: creationTimestamp: "2019-02-09T23:09:55Z" labels: app: prometheus-operator-prometheus chart: prometheus-operator-2.1.6 heritage: Tiller release: prometheus-operator name: prometheus-operator-prometheus namespace: kube-prometheus resourceVersion: "10996" selfLink: /api/v1/namespaces/kube-prometheus/services/prometheus-operator-prometheus uid: d038d6fa-2cbf-11e9-b74f-48ea5bb87c0b spec: clusterIP: 10.107.117.120 ports: - name: web port: 9090 protocol: TCP targetPort: web selector: app: prometheus prometheus: prometheus-operator-prometheus sessionAffinity: None type: ClusterIP status: loadBalancer: {} As we can see from the service above, its name is ``prometheus-operator-prometheus`` and it's listening on port ``9090``. So let's write the ingress configuration for it. .. code:: yaml --- apiVersion: extensions/v1beta1 kind: Ingress metadata: name: prometheus-dashboard namespace: kube-prometheus annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: rules: - host: prometheus.kube.local http: paths: - path: / backend: serviceName: prometheus-operator-prometheus servicePort: 9090 Save the file as ``kube-prometheus-ingress.yaml`` or some such and deploy. .. code:: text $ kubectl apply -f kube-prometheus-ingress.yaml ingress.extensions/prometheus-dashboard created And then add the service host to our ``/etc/hosts``. .. code:: text 192.168.39.78 prometheus.kube.local Now you can access http://prometheus.kube.local from your browser. Grafana Console =============== Much like what we did with the prometheus console previously, we need to do the same to the grafana dashboard. First step, let's check the service. .. code:: text $ kubectl get service prometheus-operator-grafana -o yaml -n kube-prometheus apiVersion: v1 kind: Service metadata: creationTimestamp: "2019-02-09T23:09:55Z" labels: app: grafana chart: grafana-1.25.0 heritage: Tiller release: prometheus-operator name: prometheus-operator-grafana namespace: kube-prometheus resourceVersion: "10973" selfLink: /api/v1/namespaces/kube-prometheus/services/prometheus-operator-grafana uid: cffe169b-2cbf-11e9-b74f-48ea5bb87c0b spec: clusterIP: 10.107.125.114 ports: - name: service port: 80 protocol: TCP targetPort: 3000 selector: app: grafana release: prometheus-operator sessionAffinity: None type: ClusterIP status: loadBalancer: {} We get ``prometheus-operator-grafana`` and port ``80``. Next is the ingress configuration. .. code:: yaml --- apiVersion: extensions/v1beta1 kind: Ingress metadata: name: prometheus-grafana namespace: kube-prometheus annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: rules: - host: grafana.kube.local http: paths: - path: / backend: serviceName: prometheus-operator-grafana servicePort: 80 Then we deploy. .. code:: text kubectl apply -f kube-grafana-ingress.yaml ingress.extensions/prometheus-grafana created And let's not forget ``/etc/hosts``. .. code:: text 192.168.39.78 grafana.kube.local And the grafana dashboard should appear if you visit http://grafana.kube.local.