Aiven's latest open source connector allows you stream Kafka data to GCS, read this post to learn more about it.
Kyle Buzzell
|RSS FeedHead of Growth Marketing at Aiven
Subscribe for the latest news and insights on open source, Aiven offerings, and more.
We’ve recently created and released the Aiven GCS Kafka Sink Connector. This connector will allow you to move data from your Kafka clusters to Google Cloud Storage. Even better, it is open source.
By making it open source, we ensure that you can use our connector regardless of whether you host and manage Kafka yourself or choose a service provider to do so.
You can learn how to configure and use it here. If you would like to make contributions to the project and request features, visit our project here. But before you go, let’s look at Kafka connectors in a little more detail.
Simply put, Kafka needs connectors to live up to its full potential. These connectors allow you to couple Kafka with other systems so that you can easily do things such as streaming Change Data Capture.
They come in two forms, source and sink. The former connects a source of data to Kafka while the latter does the opposite. For example, the GCS sink connector for sending Kafka data to Google Cloud Storage.
But there’s an issue: many of these connectors have either been released as proprietary or undergone license changes that have restricted their use.
That is part of the reason why we’ve begun to create our own connectors, to support our customers’ needs. However, there’s an additional philosophical reason—to continue supporting the open source community.
Our first two open source Kafka connectors are sinks to Amazon S3 and GCS. We’ll continue to improve on our existing connectors and extend the list. Remember, we consider your feedback when deciding what to offer.
So make sure you let us know what Kafka connectors you’re interested in and if there is enough demand, we’ll make sure to include it.
In the meantime, sign up to our changelog and blog RSS feeds to stay up-to-date with the latest Aiven news, and don’t forget to follow us on Twitter or LinkedIn. If you’re not an Aiven customer, get informed about Aiven for Apache Kafka® and test our platform 30 days for free.
