Skip to content
Log in
Book a demo
Start for free
Search Developer Center
Use cases
Technologies
Developer Center
Guides to powerful and flexible tools for real-time data processing, analysis, and visualisation. Simplify your workflows and visit the Aiven Developer Center
Featured
What is Apache Kafka®?
Have you ever been confused by all this talk about kafkas and streaming? Get the basics in this article full of information and resources.
Terraform Cookbook (Apache Kafka)
This is a collection of Terraform Cookbook for Apache Kafka
Aiven’s favorite Apache Kafka® connectors
Getting started with Aiven for Apache Kafka®? Check out this list of our most used Kafka connectors!
Stream data from Apache Kafka® to ClickHouse® for real-time analytics
Learn how to create a ClickHouse sink connector for Apache Kafka® with Aiven platform
Synthetic Data for AI with Aiven and ShadowTraffic
Generating artificial datasets that closely resemble real data without compromising sensitive information
Developing memory-rich AI systems with Valkey™, OpenSearch® and RAG
Adding long term memory to agent communication with OpenSearch®, Amazon Bedrock, RAG and LangChain
Real-time AI Agent communication with Valkey™ and an LLM
Integrating Valkey® pub/sub with Amazon Bedrock for AI agent communication
Building a real-time AI pipeline for data analysis with Apache Flink® and OpenAI
Make sense of social media data with Apache Kafka® and Apache Flink®
Enriching product reviews with Apache Kafka® and Amazon SageMaker
Ever wondered how to enrich your real-time streaming data with machine learning-driven analysis?
Move from batch to streaming with Apache Kafka® and Apache Flink®
Find out how Apache Kafka® and Apache Flink® allow you to move from batch processing to streaming, but keep using SQL in the data pipeline.
Create your own data stream for Apache Kafka® with Python and Faker
How can you test an empty data pipeline? Read on to discover how to create pretend streaming data using Python and Faker.
Serverless event driven architecture with AWS Lambda functions and Apache Kafka®
Build serverless Event Driven Architectures (EDA) by combining Apache Kafka® with AWS Lambda functions. Learn how to trigger Lambda functions based on events flowing in an Apache Kafka topic
Configure Apache Kafka® with topics and an HTTP sink connector using Terraform
Use an HTTP sink connector to feed data into Apache Kafka® and deploy it using Terraform. A part of Aiven's Terraform Cookbook.
Aiven for Apache Kafka® as a source for Aiven for ClickHouse®
Deploy an Aiven for Apache Kafka® node and integrate it as a data source with Aiven for ClickHouse® using Terraform. A part of Aiven's Terraform Cookbook.
Debezium source connector - PostgreSQL® to Apache Kafka®
Use Debezium as a source connector to integrate PostgreSQL® and Apache Kafka® and use Terraform to deploy. A part of Aiven's Terraform Cookbook.
Apache Kafka® with MongoDB source connector
Use MongoDB as a source for Apache Kafka® topics and deploy the integration with both using Terraform. A part of Aiven's Terraform Cookbook.
Cross-cluster replication with Apache Kafka® MirrorMaker 2
Replicate data from one Apache Kafka® cluster to another using MirrorMaker 2. Useful when migrating Apache Kafka® services. A part of Aiven's Terraform Cookbook.
Apache Kafka® with Karapace Schema Registry
Manage Apache Kafka® with the open source Karapace schema registry, and deploy both using Terraform. A part of Aiven's Terraform Cookbook.
Apache Kafka® with custom configurations
Deploy an Apache Kafka® node to Aiven for Apache Kafka® with custom configurations defined in Terraform. A part of Aiven's Terraform Cookbook.
Apache Kafka® as source and sink with Apache Flink® to process streaming data
Learn how to use Apache Kafka® as a source and sink to process streaming data, and how to deploy that with Terraform. A part of Aiven's Terraform Cookbook.
Connect Apache Kafka® to OpenSearch® with Terraform
Use Terraform to connect Apache Kafka® to OpenSearch®. A part of Aiven's Terraform Cookbook series.
Connect to Apache Kafka® with Franz
Franz is a desktop client for managing Apache Kafka® deployments. Learn how to connect to an Aiven for Apache Kafka® service using Franz
Social search in real time: Exploring Mastodon data with Apache Kafka® and OpenSearch®
Mastodon → Apache Kafka® → OpenSearch® → knowledge
First steps with the Apache Kafka® Java client library
Learn how to produce and consume records with Apache Kafka Java client library
Webhook data from Apache Kafka®️ to the world
Let your data flow with the Apache Kafka®️ HTTP sink connector
Stream Mastodon data to Apache Kafka® using NodeJS and TypeScript
If you want to analyze Mastodon posts, getting them into Apache Kafka® is a sensible first step. Read on to find out how to do this with Typescript and NodeJS.
(Re)Create Apache Kafka® topics with Terraform
Terraform can orchestrate your infrastructure the same way every time, giving you confidence in your consistent platform. Idempotency for the win.
Ways to balance your data across Apache Kafka® partitions
When it comes to making a performant Apache Kafka® cluster, partitioning is crucial. Read this article to learn which tools you can use to partition your data.
Remove naughty words from your data using DataCater
We all know we shouldn't use naughty words. Learn how to remove them from your streaming data using DataCater.
Connecting Apache Kafka® and Aiven for ClickHouse®
Learn how to perform analytics with fast response time over huge volumes of data without the need to downsample data.
Database migration with Apache Kafka® and Apache Kafka® Connect
Find out how to use Apache Kafka® to migrate across database technologies while keeping the target continually in sync with the source.
Teach yourself Apache Kafka® and Python with a Jupyter Notebook
Learn the basics of Apache Kafka® Producers and Consumers through building an interactive notebook in Python.
Apache Flink® SQL client on Docker
Pure SQL pipelines using Apache Flink®? Learn how to set them up with a local Docker-based platform for Flink, including an SQL client!
Using Kafka Connect JDBC Source: a PostgreSQL® example
Find out how to use Apache Kafka® Connect to update an old app-to-db design to use up-to-date tech tools without disrupting the original solution.
