Grafana is a popular tool for doing monitoring and analytics. It allows us to build dashboards to visualize, query, analyze data, and to set up alert notifications for certain conditions. To start from the beginning, check out our article on Grafana Dashboards, and our article called Getting Started with Grafana. This article will go in-depth on the most popular Grafana data sources and how to use them.
Get started by signing up for the MetricFire free trial, where we have a Hosted Grafana Service that you can use without any installation or configuration. This free trial will let you follow along with this article by using Grafana directly in your browser.
Grafana works with data, which must be stored in a database before it can be accessed by Grafana. There are several different kinds of databases. Additionally, some other systems can be used to store data, even though their main purpose is not focused on data storage.
A Grafana data source is any place from which Grafana can pull data. In other words, you don’t need to load data into Grafana to analyze it. Instead, you should create a connection between Grafana and the data source. Grafana will then work with the connected data source to analyze the data stored there and derive some insights and perform monitoring.
To add a data source in Grafana, you should hover over the gear icon at the top right (Also called the Configuration menu) and then click on the Data Sources button:
In the Data Sources section of the Configuration menu, you can press the Add data source button. In the screenshot below you can see that we’ve already added data sources such as Hosted Graphite and Metricfire-Prometheus. If you have a fresh local installation of Grafana, you will not see any data sources here until you add them. In general, this is the place where you can view all of your connected data sources.
When you click the Add data sources button, you will see the list of officially supported data sources available for connection:
Select the preferred data source and click on it. The next step will be to specify the required parameters (URL, authorization details, names, etc.):
After you finish filling in the required parameters, you need to press the Save & Test button:
When Grafana establishes the connection, you will see the corresponding message. Then, you can go to the Dashboards section and start exploring the data from the connected data source.
Let’s now explore and briefly describe the data sources that are available “out of the box” in Grafana. We can split them into a few key groups:
Let’s describe some popular community-driven data sources.
Cloudera Manager is the tool for Apache Hadoop administration. It also provides functionality for monitoring and reporting. Using this plugin you can connect the Cloudera Manager to Grafana and build dashboards using data collected from the Hadoop cluster.
Consul is the solution for connecting cloud services and securing them. This helps move from static to dynamic architecture. Consul collects metrics that reflect the health and performance of the connected services, which you can monitor in Grafana.
Apache Druid is a real-time analytics database. With the community-supported plugin, you can access the data stored in Apache Druid from Grafana.
Google BigQuery is a well-known cloud data warehouse that allows reliable storage of data and efficient analytics.
This plugin could be used to access information about events scheduled with Google Calendar. For each event, there is the start and end time, that’s how the transformation to the time-series view occurs.
IBM Cloud Application Performance Management enables application monitoring. With this plugin, you can collect and visualize metrics from IBM APM in Grafana.
Oracle Cloud Infrastructure
This plugin allows running queries against the Oracle Cloud Infrastructure Monitoring tool.
ClickHouse is a popular open-source column-oriented database often used for storing information about events in web analytics, gaming software, etc. Events (for example, user clicks) usually have date-time fields, which makes it possible to transform to time-series data and analyze it in Grafana.
This plugin is developed for fetching data from Quandl API - a financial data provider. Financial data is usually time-series data, so it can be used easily in Grafana.
DarkSky API provides you with historical information about the weather conditions and future forecasts. This plugin allows you to monitor the weather in Grafana, which could be very important for many applications and use cases.
The list above is incomplete. There are many other community-driven data source plugins. In addition, developers constantly create new data sources and make them available for the community. So, if you need a specific data source, it is always a good idea to check the page for community-supported data source plugins.
As you remember, when you click on the Add data source button, you will be presented with the list of officially supported data sources. If you scroll the page down, you will see the "Find more data source plugins on grafana.com" link:
If you follow this link, you will see the page with the available plugins (you can see the selection menu on the left hand side, showing the plugin type as data source):
The officially supported plugins are denoted as “by Grafana Labs”, while community-supported plugins show the names of their developers:
If you click on any of the data sources provided here, you will enter the page with the details about the plugin and the installation information:
After you install the plugin, the data source should appear in the list of the available data sources in your Grafana UI.
You can find more detailed instructions about plugin installation here.
If you cannot find the needed data source among the officially supported or community-provided ones, you can create your own data source. The idea is that a custom plugin can be created for any database that uses HTTP protocol for communication with clients. The plugin should transform data from the database into time-series data for Grafana to be able to correctly represent it in the dashboard's visualizations.
To develop a plugin for the new data source, three components are required:
You can read more about the creation of custom data sources in this documentation.
This article explored Grafana data sources and explained how to work with them. We described a range of popular officially supported data sources, as well as community-driven plugins. Finally, we introduced the process of installing third-party data sources and creating your own data sources.
If you want to use Grafana for your monitoring or data analysis needs, don’t forget that we have a 14-day free trial for Grafana as a Service. If you have any questions, please contact us or write to firstname.lastname@example.org.