In the past few years, Telegraf has become a go-to open source standard for collecting metrics for various stacks, sensors and systems. It boasts over 200 available plugins for collecting on a wide variety of system events. These events it sends to either an InfluxDB or Prometheus server for storage and visualization.
But all systems are not created equal. What happens when your metrics flow outstrips the computing capacity of your InfluxDB or Prometheus server?
Vertical scaling can work, up to a point. Horizontal scaling in InfluxDB requires an enterprise license, while Prometheus doesn’t even have a viable option.
So when you have hundreds, thousands or possibly tens of thousands Telegraf agents producing metrics, what do you do? Have no fear, M3 to the rescue!
What is M3?
M3 is an open source time series database. Originally developed by Uber, the goal of M3 is to provide easy horizontal scalability, central storage for all metrics and compatibility with several standard industry interfaces for producing and consuming metrics. In particular, M3 has 100% Prometheus Query Language (PromQL) compatibility and InfluxDB Line Protocol format.
With its already proven track record of storing 10’s of billions of time series, M3 is a perfect replacement for an InfluxDB or Prometheus backend, and it allows us to retain our current Telegraf agents.
How do we do this? Who offers M3 in the cloud?
Aiven for M3
Aiven is one of the first cloud providers to offer M3 as a service. Not only do we provide M3 as a service for our customers, we also use it internally to monitor all of our customer VM’s and services (Aiven for Apache Kafka, Aiven for PostgreSQL, etc.) in one single central M3 service.
We deploy the Telegraf agent on all of our managed VM’s to monitor dozens of server and process metrics including CPU utilization, RAM Utilization, Network IO, and disk utilization to name a few.
Sounds sweet? Head over to our Developer Portal for the details, but in general terms, here’s what you can do:
- Create a new Aiven for M3 service in the Aiven Console or CLI to collect metrics.
- Install Telegraf Agent as an endpoint for the metrics.
- Integrate Telegraf with M3 so it can actually get those metrics.
- Create an Aiven for Grafana service to visualize the metrics.
- Enjoy the fruits of your labor!
Yep, that’s all. Granted that it’s a few clicks and keypresses away, but it’s not rocket science by any means.
Not using Aiven services yet? Sign up now for your free trial at https://console.aiven.io/signup!
Feb 22, 2021
Aiven for M3: The first managed open-source database
The new era of time series has arrived with the first managed open-source database of its kind, Aiven for M3. Find out everything you need to know to be a part of it.
Feb 24, 2021
How to choose a time series database: 6+3 considerations
With real-time data on the rise, time series databases are more and more in demand and on offer. Read this article to find out how you can choose between them.
Aug 12, 2021
Metrics and graphs with M3 and Grafana
Find out how to use M3 to monitor your systems, and Grafana to make the results easier to understand.
Subscribe to the Aiven newsletter
All things open source, plus our product updates and news in a monthly newsletter.