You can now view Apache Kafka messages live in the Aiven Console

Viewing Kafka messages live from the Aiven Console provides a host of benefits. Find out what they are and how to set it up in this post.

28 April 2020
John HamminkDeveloper Advocate at Aiven

Apache Kafka users need a quick and easy way to view the state of their message queues and see if recent messages are being ingested correctly. Previously, you’d have needed external tools, monitoring systems, or APIs to view messages residing on a specific Kafka topic.

Now you can view Kafka messages live in the Aiven Console.

Viewing the message queues dynamically in the console allows Aiven for Apache Kafka users to get a feel for their data, as well as troubleshoot their event streams, content and schema.

The message-viewing feature comes with the ability to sort messages by location, partition, offset, timeout and maximum size. This adds simplicity and provides a degree of granularity useful for troubleshooting complex message queues.

How to use Aiven Console to produce and view messages

Messages written to Aiven for Kafka topics appear under the Topics section of your Kafka instance in the Console. Let’s generate a few messages to show how the feature works.

Viewing messages from the console

NOTE: You’ll need an Aiven for Kafka Business-4 plan or better to see Kafka messages in Aiven Console. And you’ll need to enable Kafka REST API to use the feature, as follows:

Viewing kafka messages from ui step 1

In the “Topics” tab on Aiven Console, create a couple of topics, for example jsontest and other_topic:

Viewing kafka messages from ui step 2

Now, let’s try producing and viewing messages directly from the jsontest topic from within Aiven Console. First, click Show messaged icon icon in the jsontest topic row — this will expand the message view window.

Next, press the produce message dialogue button button to invoke the Produce Message dialog. Enter a valid JSON message and value, and hit Produce message execution button. Once you are looking at the message view again, just hit Fetch messages button to update the list.

Here’s how that looks altogether:

Viewing kafka messages from ui step 3

Of course, the messages would also appear here if you’d piped them in from a producer, or used, for example curl to generate new messages to your topic. (You’d simply press the Fetch messages button button to refresh the list.)

Wrapping up

The ability to produce and fetch messages from within the Aiven Console makes it a lot easier for Kafka users to test their clusters, event streams, and topic configurations. These troubleshooting tools remove some of the guesswork and extra tooling needed for users to get a feel for their data as it moves through their messaging queue.

Aiven for Kafka is a work in progress, so we’ll always be adding new features. Remember subscribe to our changelog and blog RSS feeds, or follow us on Linkedin and Twitter to stay up-to-date.

kafka

Start your free 30 day trial

Test the whole platform for 30 days with no ifs, ands, or buts.

Aiven logo

Let‘s connect

Apache Kafka, Apache Kafka Connect, Apache Kafka MirrorMaker 2, Apache Cassandra, Elasticsearch, PostgreSQL, MySQL, Redis, InfluxDB, Grafana are trademarks and property of their respective owners. All product and service names used in this website are for identification purposes only and do not imply endorsement.