We've gotten a lot of great feedback on our service logs and we'd like to share some recent game-changing improvements that we've made to them, namely to our console logs and a recently enabled Aiven Elasticsearch service integration.
Specifically, we’ve made improvements to the following:
- Integration of all Aiven services with Elasticsearch
- Console log UI, visibility, and accessibility
In this post, we’ll look at the improvements we made to the Aiven console logs and the Elasticsearch logs service integration in greater detail. Let’s start with the biggest improvement—the Elasticsearch logs service integration.
Push your service logs to Aiven Elasticsearch
You can now send service logs from their Aiven Kafka, PostgreSQL, Elasticsearch, Cassandra, Redis, InluxDB, and Grafana services to an Aiven Elasticsearch instance by enabling the Elasticsearch logs integration.[click to enlarge]
By using this integration, you will have unlimited retention for your logs and the ability to take advantage of Aiven Elasticsearch’s powerful search features to access relevant data from among them.
Even better, if you are also using Aiven Grafana, the latest release allows you to configure alerts based upon their Elasticsearch queries, giving you even greater command over your log management.
Elasticsearch logs integration can be quickly created from the “Enhance my logs” button within the logs tab or the “Manage” button within the Service integrations row of each service panel. For the specific steps, check out Getting started with Elasticsearch log integration.
Aiven console service logs updates
Previously, our console logging allowed you to only see the last 100 entries, which didn’t allow for a deep dive into your log history. This proved even more frustrating for users with high volume services.
For instance, performing any kind of historical search would be impossible with a service such as Kafka because the logs could change by the second.
Now, we've provided support for our web console (as well as our API and CLI) so that users can access the last two days of logs at will, making it easy to remain current with the latest service logs.
For our web console users, we’ve made improvements to the logs UI to enhance readability as well as allow users to freely scroll the last two days of their service logs.[click to enlarge]
Now, let’s take a look at some simple commands that our CLI users can access their logs with. Aiven tooling now supports tailing and fetching the last few days of service logs.
$ avn service logs myservice -f
$ avn service logs myservice -f -S desc
Whether you choose to continue using our console logs or take advantage of the Elasticsearch logs integration, you will have greater visibility into and flexibility over your service logs.
We’re also looking to add 3rd-party integrations with popular logging platforms, and should have a few by year-end. So, please continue giving us feedback because that’s how we prioritize our additions.