In this article, we will look at how to monitor Node.js applications using Graphite and StatsD, and plot the visualizations on a Grafana dashboard. Node.js is a popular framework for creating microservices. Its asynchronous nature allows for high scalability and low latency especially for I/O bound tasks. However, it is important to have a proper monitoring setup for any application which is running in a production environment.
MetricFire is a Hosted Graphite, StatsD and Grafana platform for monitoring time-series metrics. You can use these open source software directly in the MetricFire web app, with no set up or hassle when it comes to scaling, updates, or plugins. You should sign up for the MetricFire free trial here and start monitoring your Node.js now.
Graphite is an Open Source software which can store multiple types of metrics such as counters, timers and gauges. It primarily consists of three components -
StatsD is a network daemon which listens for metrics from applications and delivers them to Graphite. Hence, StatsD acts as a middle man between the Graphite and the application.
One question that normally arises in the minds of users is: When Graphite itself can listen for metrics, why do we need StatsD? Well, the answer is aggregation. StatsD can do the aggregation of your metrics and then it sends the aggregated metric over to Graphite, saving Graphite from unnecessary increased load.
When using MetricFire, we actually have a custom agent designed for avoiding the hassle of a typical StatsD and collectd stack. Check out the MetricFire agent here.
For our use case, we will be using Grafana to plot the visualizations from the data stored inside Graphite. This is because Grafana is an advanced tool for visualization compared to Graphite Web.
We will not go into the details of Graphite or Grafana in this article. If you like you can go over our existing articles on Graphite such as Graphite and Grafana, Monitoring with Graphite: Installation and Setup and Monitoring with Graphite: Architecture and Concepts.
We will use docker to run Graphite, Grafana and StatsD on our system. Here is the simple docker-compose file which will do the job for us:
<p>CODE: https://gist.github.com/denshirenji/90805a1722b4117ff0f1778c2f05c05a.js </p>
We will also verify that Grafana is up and running by going to http://localhost:3000. The default credentials are admin/admin.
Now, we will add Graphite as the data source in Grafana. Browse to the data source section of Grafana and add Graphite as the data source, as shown below:
We will create a very simple Node.js Express application to demonstrate the capturing of metrics in Graphite and StatsD.
Express is the most popular Node.js framework to create microservices. We’ll use a library open sourced by Uber called express-statsD to capture the response times and status code metrics and send it to StatsD.
First, let’s set up the package.json file, which is a manifest file that describes the dependencies of our node application. The sample package.json to be used is given below:
<p>CODE: https://gist.github.com/denshirenji/96494075fec6cf6d7cad00adef351d32.js </p>
Now we will write the code for our application which creates two endpoints, hooks up the middleware to send the information to StatsD, and responds with either success or failure:
<p> CODE: https://gist.github.com/denshirenji/0ed80e9a62d35b609ffac695b3b81665.js </p>
Save both the files as package.json and app.js inside a directory and run the application by executing “npm install” and then “node app.js”. Once the application is running, browse to endpoints http://localhost:8000/ and http://localhost:8000/products to execute the endpoints.
Let’s head over to Grafana and create the visualizations for the metrics currently being emitted through our application. Create a new Dashboard in Grafana and add a new graph visualization. We will choose the data source as Graphite and construct the query to represent the response time of our endpoints. In the end, the configuration should look like below:
The above dashboard represents Latency Per Route. In a similar way, we can plot many interesting metrics such as Average Latency, No. of 5xx Errors and Request Rate.
If we add those visualizations just mentioned, we can create a dashboard that looks like this for our node.js application:
You can also put up alerts on these visualizations if they exceed some critical threshold value. Check out Grafana Dashboards from Basic to Advanced to learn how to set up Grafana alerts, and build custom dashboards.
In this article, we saw how easy it is to set up the Node.js monitoring dashboard with help of Graphite, StatsD and Grafana.
MetricFire also provides production ready Hosted Graphite and Hosted Grafana along with many other benefits so that users can focus on building their applications, and let MetricFire take care of managing the scalability and storage of their metrics.
Sign up for the MetricFire free trial to set up Graphite and build Grafana dashboards now. You can also book a demo and talk to the MetricFire team about how you can best set up your monitoring stack.