MQTT and Grafana Integration

Motivation

With the introduction of MQTT logging the visualization of the statistics is fostered by usage of Grafana. The full chain from MQTT log to Grafana is provided with this extension. This extension provides a visualization of data in graphical format, which is desired for some use cases, e.g. long term analysis, several instances of the Linux RealTime Communication Testbench running, and showcases. The combination MQTT/Grafana is chosen as it is the most recognized and used application for data visualization, but requires additional services for the visualization. Those services are Telegraf and InfluxDB. Telegraf performs the transfer of the MQTT data into InfluxDB insertions. InfluxDB itself is the data provider for Grafana. The configuration for those services are provided allowing the direct start with data visualization with Grafana. A docker-composer file that allows that can be generated via a bash script provided.

Enabling the build

MQTT support is integrated in the build system as an option to enable it add the flag:

cmake -DCMAKE_BUILD_TYPE=Release -DWITH_MQTT=TRUE ..

Configuration parameters

The parameters which affect the MQTT integration are presented in the table below:

Features & configure options

Option	Description
StatsCollectionIntervalNS	Interval in which a summary of the data is generated, typically 1s, i.e. 1000000000Ns
LogMqtt	Enable and disable MQTT logging
LogMqttThreadPriority	Thread priority for the logging thread, usually low < 7
LogMqttThreadCpu	Thread CPU affinity
LogMqttBrokerIP	IP address where MQTT broker is running, default value is 127.0.0.1
LogMqttBrokerPort	Port used by the MQTT Broker, default value is 1883
LogMqttKeepAliveSecs	Keep alive time for the connection, default value 60s
LogMqttMeasurementName	Used to distinguish measurements coming from different machines. Default value is reference.

Docker Composer

The first step is to create the docker-compose.yml file. As there is an uncertainty about the licensing on those docker-composer files and not much to be copyrighted on those, an indirect approach to deliver the file was taken. In the docker/mqtt-composer directory the file create_docker_compose.sh is provided. It clones an upstream docker-compose project, applies the local changes from docker_composer.diff and places the resulting docker-compose.yml into the directory. Afterwards, set_permissions.sh prepares the data directories. Run both scripts like this:

cd docker/mqtt-composer
./create_docker_compose.sh
./set_permissions.sh

Creating the services

The generated docker-compose.yml makes the integration into Grafana simple. A simplistic way to have the Grafana running is it can be used, is to instantiate the docker-compose in the machine where reference is running, by doing that all parameters concerning the MQTT broker are default. To instantiate the docker-composer you need docker and docker-compose installed. Start the services by issuing the commands below from the rtc-testbench directory.

cd docker/mqtt-composer
docker-compose --compatibility up -d

At this point all services needed to have the data available in Grafana is done. The most important part is now how to access the data. The verification of data availability can be done in two steps: first at InfluxDB and second via Grafana.

Influx Access

Used database for the provided configuration is “testbench” and is defined in the telegraf.conf file. The measurement name is the “LogMqttMeasurementName” provided in the configuration. The Timestamp provided is the time taken in the application side when the last frame in the recording period “StatsCollectionIntervalNS” is received, and the traffic class name is one of the columns of the measurement. This way one can investigate the data base by providing the following code when running influx:

use testbench
SELECT * from ${LogMqttMeasurementName}

where ${LogMqttMeasurementName} is a placeholder for the real name used.

Grafana Access

Grafana can be accessed by the IP of the machine that the docker-compose was started and port defined on the docker-compose.yml file. In Grafana a new data source for InfluxDB can be created and queries added to create dashboards.

Grafana Password

Grafana password has to be reset for the first usage. In some cases Grafana might also consider the password as expired refusing to allow the admin to login, e.g. extreme clock changes. To do so execute the “change_grafana_password.sh” script, with the containers running, providing the new password as parameter as below:

cd docker/mqtt-composer
./change_grafana_password.sh $NEW_PASSWORD

Grafana Example

The example provided for Grafana has the time display of the 3 Profinet real-time traffic classes, TSN-High, TSN-Low and RTC. Dials are provided to show the current maximum visualized within the visible time range. Colored bars are provided for representing the number of errors and outliers found.