Deploying Grafana And Prometheus Application On Kubernetes With Helm …

Applications On Kubernetes cluster

Helm in Kubernetes

Hello All….!!

Aim :

How to Create Helm Chart for Prometheus and Grafana monitoring tools from scratch and Launch into AWS on Kubernetes Multi Node Cluster.

Before Going Further we have to first understand the basic and important information and concepts about the helm and helm chart.

Helm :

Helm is a tool that streamlines installing and managing Kubernetes applications. Helm helps us to install software, Automatically install software dependencies, Upgrade software, Configure software deployments, Fetch software packages from repositories.

Helm Chart :

Helm Charts are simply Kubernetes YAML manifests combined into a single package that can be advertised to your Kubernetes clusters. Once packaged, installing a Helm Chart into your cluster is as easy as running a single helm install, which really simplifies the deployment of containerized applications.

Pre-Requisite :

AWS Account

Kubernetes Cluster Setup On AWS

Docker Hub Account

For setup kubernetes cluster on AWS refer below article in that I explain all the things from basic and for doing setup I use automated tool Ansible. So you get extra knowledge of Ansible.

Create Multi Node Kubernetes Cluster using Ansible(Automation) Roles inside EC2 Instance Of AWS.

Now We are going to deploy Grafana and Prometheus application on top of Kuberenetes . So let’s see more about it..

Grafana :

Grafana is a multi-platform open source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources. Grafana is an open source solution for running data analytics, pulling up metrics that make sense of the massive amount of data & to monitor our apps with the help of cool customizable dashboards.

Prometheus :

Prometheus is a free software application used for event monitoring and alerting. It records real-time metrics in a time series database built using a HTTP pull model, with flexible queries and real-time alerting.Using Prometheus, you can monitor application metrics like throughput (TPS) and response times of the Kafka load generator (Kafka producer), Kafka consumer, and Cassandra client. Node exporter can be used for monitoring of host hardware and kernel metrics.

Now, We are going to write a Dockerfile for creating a image of Grafana and Prometheus.

DockerFile mean’s :

Dockerfile is a simple text file that consists of instructions to build Docker images. Mentioned below is the syntax of a Dockerfile to creating Grafana Docker image.

So create a file as Dockerfile and Ensure D should be capital in Dockerfile.

vim Dockerfile

and write the below code inside Dockerfile.

FROM centos:7
RUN yum install wget -y
RUN wget https://dl.grafana.com/oss/release/grafana-7.0.3-1.x86_64.rpm
RUN yum install grafana-7.0.3-1.x86_64.rpm -y
WORKDIR /usr/share/grafana
CMD /usr/sbin/grafana-server start && /usr/sbin/grafana-server enable && /bin/bash

After writing the code, build the docker image with the help of following command.

docker build -t username/imagename:version 

eg. docker build -t pritee55/grafana:v1

Now login to your DockerHub Account. But if you haven’t DockerHub Account then first of all create it then move ahead with below command.

docker login

After it, just push your image to DockerHub so that you can use it in the future.

docker push username/imagename:version

After running above command, the output should be similar to the following:

Now , Same Process you can do for Prometheus Dockerfile and push it into the dockerhub.

But if you don’t want to create this container image then you can simply pull my pre-created image from the Dockerhub with help of docker pull command.

docker pull pritee55/grafana:v1
docker pull pritee55/prometheus

We will use this image in the upcoming steps when we will create deployment.yaml file in the Helm chart. But before it, let's know about how to install Helm.

Here, I am Created Cluster on a EC2-linux instance of AWS . So I have to install helm software related and supported to my cluster. So in my Case I use below command to install helm…

wget https://get.helm.sh/helm-v3.5.2-linux-amd64.tar.gz

So this is an tar file we have to untar this for that use below command..

tar -xvzf helm-v3.5.2-linux-amd64.tar.gz

After this, copy linux-amd64/helm file in the /usr/bin/

cp linux-amd64/helm /usr/bin/

Also, Ensure that Helm is installed or not with the help of the helm version and the output should be similar to the following:

Now if you have any issue related with helm installment or you want the suitable software to your cluster then refer below official website of helm…

Installing Helm

Now , Let’s create a helm chart …

let’s create a new Helm Chart from the scratch. Helm created a bunch of files for you that are usually important for a production-ready service in Kubernetes. To concentrate on the most important parts, we can remove a lot of the created files. Let’s go through the only required files for this example.

Create a Deployment and Expose the services for Prometheus and Grafana Using Dry Run from thier Docker Images using below given commands..

Dry run is not a concept of Kubernetes k8s. This is an expression used when practicing a demonstration or process before the actual performance. Dry run mode allows you to run commands without side effects to test the actual command you want to run.

Create a Helm Chart for Grafana. So here we first create helm chart for grafana after that we create a chart for Prometheus.

mkdir grafana-app
cd grafana-app

But here we need a project file that is called Chart.yaml and contains all the metadata information.
🔸 So, create this file inside the grafana-app folder. Also, C should be capital in the Chart.yaml.

vim Chart.yaml

Now , we have to write some code in that Chart.yml file as a metadata. So below is a code you should write in it..

apiVersion: v2
name: Grafana
description: A Helm chart for Kubernetes
type: application
version: 0.1.0
appVersion: 1.16.0

Now create one folder inside grafana-app and that is templates and go inside that folder using cd command.

mkdir templates
cd templates

Inside that we create on yaml file for deployment. So run dry run command which is given below.

kubectl create deployment grafana-app --image=pritee55/grafana:v1 --dry-run -o yaml > deployment.yaml

The output should be like this …

Now , go inside the Grafana-app directory and install the helm chart for grafana.

cd 
helm install grafana-app ./grafana-chart

Here grafana-app is the name of Helm Chart and ./grafana-chart is the path of the chart.

After running above command, the output should be similar to the following:

Now again go inside grafana-app/templates and use the below command to create a code of the service.yaml file.

cd grafana-app/templates/
kubectl expose deployment grafana-app --port=3000 --type=NodePort --dry-run -o yaml > service.yaml

After it, run the below command for exposing the grafana-app pod.

kubectl apply -f service.yaml

To ensure your pod is working well with the below commands.

kubectl get pods
kubectl get deployment
kubectl get svc

After running these commands, the output should be similar to the following:

You can check the list of the helm you have using the helm list command. Now we can check the pods in which slaves are running in the Kubernetes cluster.

Hopefully, Everything is working well till yet. So let’s check that Grafana is working fine or not. For this, you have to take the public_ip_of_instance and port no. of your pod.

Where 65.2.132.12 is the public IP of my instance that contains all the setup which I have done till yet and 30821 is the port no. of grafana pod which you can see in the above screenshot after running kubectl get svc command. So browse it.

After browsing it, login page will pop up. So login with by-default username and password admin.

After logging you will get the page for changing the username and password, So you can if you want.

Now Grafana is ready to use.

Now are going to do same steps for Prometheus very quickly. Create a Helm Chart for Prometheus.

mkdir prometheus-app
cd prometheus-app

But here we need a project file that is called Chart.yaml and contains all the metadata information.
🔸 So, create this file inside the prometheus-app folder. Also, C should be capital in the Chart.yaml.

vim Chart.yaml

Now create one folder inside prometheus-app and that is templates and go inside that folder using cd command.

mkdir templates
cd templates

Inside that we create on yaml file for deployment. So run dry run command which is given below.

kubectl create deployment prometheus-app --image=pritee55/grafana:v1 --dry-run -o yaml > deployment.yaml

The output should be like this …

Now , go inside the prometheus-app directory and install the helm chart for Prometheus.

cd 
helm install prometheus-app ./prometheus-chart

After running above command, the output should be similar to the following:

Now again go inside prometheus-app/templates and use the below command to create a code of the service.yaml file.

cd grafana-app/templates/
kubectl expose deployment prometheus-app --port=3000 --type=NodePort --dry-run -o yaml > service.yaml

After it, run the below command for exposing the prometheus-app pod.

kubectl apply -f service.yaml

To ensure your pod is working well with the below commands.

kubectl get pods
kubectl get deployment
kubectl get svc

After running these commands, the output should be similar to the following:

You can check the list of the helm you have using the helm list command. Now we can check the pods in which slaves are running in the Kubernetes cluster.

Hopefully, Everything is working well till yet. So let’s check that Prometheus is working fine or not. For this, you have to take the public_ip_of_instance and port no. of your pod.

Where 13.234.111.8 is the public IP of my instance that contains all the setup which I have done till yet and 31125 is the port no. of prometheus pod which you can see in the above screenshot after running kubectl get svc command. So browse it.

Now Prometheus is ready to use. So let’s enjoy ☺️

Packing resources inside the Helm Chart.

So helm chart ready inside the grafana-chart/ directory and prometheus-chart/ directory, but we can’t publish it as it is. Firstly, we have to create a package for this helm charts.

🔸 Create one directory named charts. Make sure this directory should be inside grafana-chart/ directory and prometheus-chart/.

mkdir charts/

Now, run the following command to packages the chart and store it inside the charts/ directory.

helm package /root/grafana-chart -d charts/
helm package /root/prometheus-chart -d charts/

💠 you will get output something like this:

For Grafana

For Prometheus :

Creating an index.yaml file.

For every Helm repository, we must require an index.yaml file. The index.yaml file contains the information about the chart that is present inside the current repository/directory.

🔸 For generating index.yaml file inside charts/ directory, run following command.

helm repo index charts/

💠 You can see an index.yaml file generated with the details of the chart.

For Grafana :

For Prometheus :

Hosting the chart with GitHub pages

Now everything is fine in our helm chart we can publish this to ArtifactHub so that we can use it in the future when we require it.

But before we have to host this chart anywhere. So that we can publish it to ArtifactHub. So I am going to host it with the GitHub page. but before doing this, We have to push the chart to Github. So let’s follow the following steps.

🔸 Install git and config cluster with your GitHub account.

sudo apt-get install git -y
git config --global user.name 'username'
git config --global user.email 'usermail@gmail.com'

After installing git go inside grafana-app and prometheus-appdirectory and then initialize it with the help of below command.

git init

🔸 Now add, commit then push this directory to GitHub.

git add .
git commit -m "give any msg according to you"
git branch -M main
git remote add origin URL
git push -u origin main  

After pushing it, go to the Github repository and click on setting then GitHub Pages.

To activate the GitHub page, you have to select branch first which you want to activate then save it.

For Grafana :

For Prometheus :

Publishing the Helm chart to ArtifactHub

Artifact Hub includes a fine-grained authorization mechanism that allows organizations to define what actions can be performed by their members. It is based on customizable authorization policies that are enforced by the Open Policy Agent. So go to your ArtifactHub Account and login to it.

🔸 Now click on profile icon > control Panel> Add repository.

Make sure your repository’s url should be like https://username.github.io/repository_name/charts/ when you're adding repository in ArtifactHub. Otherwise, it can create some issues related to url.

For Grafana :

For Prometheus :

After giving the required information to your Helm repository, click on Add. if you had provided the right information then it will create a Helm repository.

So, we had successfully published our Helm chart to the ArtifactHub.

The power of a great templating engine and the possibility of executing releases, upgrades, and rollbacks makes Helm great. On top of that comes the publicly available Helm Chart Hub that contains thousands of production-ready templates. This makes Helm a must-have tool in your toolbox if you work with Kubernetes on a bigger scale!

I hope this article help you …….!!

Thanks for reading and if you have any suggestion related with this article then command on this and if you want to connect with me then below is my LinkedIn Profile Link…… :)

Pritee Dharme - Ansible (RH294) - ARTH - The School of Technologies | LinkedIn

Thanks Once Again …….See you in next article till then Be Safe …….