Kafka MCP: Manage Apache Kafka From Your AI Assistant

You're building with Claude or Cursor, and you need to know what's actually happening on your Kafka cluster. Your AI assistant knows Apache Kafka® in the abstract, but not your topics, your retention, or that a consumer group has been slipping since this morning.

So you leave the editor and go digging through logs, a CLI, and a few dashboards, correlating by hand to answer questions like:

• Which consumer group is lagging the most right now, and why?
• Which topics are seeing sudden backlog or throughput drops?
• Has any consumer stopped committing offsets?
• Is this broker health, partition imbalance, or a slow consumer?
• Which configs (retention, partitions, batch size) are hurting performance?
• Did a recent deployment change how you consume from Kafka?

The Aiven MCP (EA) turns each of those into a sentence you type where you already work. It gives your assistant a direct line to the Kafka service in your Aiven project: the same topics and offsets the console shows, reachable from your editor.

What's a Kafka MCP, and why should you care?

Kafka MCP is a Model Context Protocol server that gives an AI assistant read (and, on approval, write) access to a running Apache Kafka cluster: topics, consumer group offsets, connectors, and messages, not just Kafka knowledge from training data. Instead of your assistant guessing at APIs or hallucinating CLI flags, it gets a structured set of actions it can take against your real cluster. See the Apache Kafka MCP server for a full overview.

Connect in under a minute

On Claude Code, one command does it:

Loading code...

Every other client (Cursor, VS Code, Claude Desktop, Gemini CLI) uses the same server. Add it to your MCP config:

Loading code...

Aiven MCP docs cover local, read-only and scoped setup. The MCP server is open source: browse the code at github.com/aiven-open/mcp-aiven.

Step 1: See what you actually have

Ask your agent:

Loading code...

One call, and it reads back the topic config:

Loading code...

168 hours is Kafka's default, applied to any topic created without an explicit retention setting. It's worth noticing, because it's exactly the kind of thing that's easy to create by accident and forget about.

Step 2: Look at the data, not just the metadata

Config tells you how a topic is set up. It doesn't tell you what's actually in it.

Loading code...

The assistant consumes through Kafka's REST interface and hands back the decoded records:

Loading code...

Two prompts in, you know the topic exists, how it's tuned, and what a real record looks like. This is the point where you'd normally reach for a console consumer or a throwaway script, decide whether you want JSON or binary output, and figure out which offset to start reading from. Here it's a sentence. And if the topic is schema-encoded (Avro, Protobuf, JSON Schema), the assistant handles the decoding too, so you're reading the actual field values instead of a base64 blob.

Lag is a subtraction your AI assistant does for you.

Step 3: Find the lag

Is anything falling behind? That's the question that sends people to the console more than any other.

Consumer lag is the gap between the newest offset on a partition and the last offset a consumer group has committed. The Aiven Kafka MCP's topic view returns both numbers for every partition, so lag is a subtraction your assistant does for you.

Loading code...

Loading code...

That's the single-topic case. It gets more useful when you don't know which topic is the problem. A Kafka service with a dozen topics and a handful of consumer groups doesn't have one screen that ranks "which group is furthest behind, and where." You'd normally check each topic in turn and keep a running tally yourself. Asking the assistant to do that sweep and report back the worst offenders turns a multi-tab investigation into one answer, because it's making the same per-topic call it would make anyway, just across everything instead of one thing at a time.

Current lag is the number you can measure right now. Where it's heading is the number that actually matters for deciding whether to act. Aiven for Apache Kafka runs a consumer lag predictor on the platform that forecasts lag trend over time from the same underlying data, so a group that's quietly falling behind shows up before it's urgent instead of after.

Step 4: Fix it where you are

Reading is half the job. Once you've found the problem, the same connection can change the setting, with your sign-off.

Loading code...

The topic's retention_ms moves from Kafka's default_config (7 days, unset) to an explicit topic_config value (30 days). That source field is a good way to audit a cluster in general: any topic still showing default_config for retention is one nobody has actually configured since it was created.

Loading code...

Same mechanism: your assistant proposes the exact change and waits for your approval before writing anything. Partition count is worth treating with a bit more care than retention, since it's a one-way door: you can raise it, but Kafka won't let you lower it again without recreating the topic. It matters here because partitions are the unit of parallelism for a consumer group: a topic with one partition caps you at one consumer actively reading it, no matter how many instances of your app you run. If lag keeps building because a single consumer can't keep up with produce rate, more partitions (and more consumer instances to match) is usually the actual fix, not just a batch-size tweak.

Every change here (retention, partitions, anything else) goes through the same review-then-approve step. Nothing gets written to a running cluster because the assistant decided to; it proposes, you confirm.

Nothing gets written to a running cluster because the assistant decided to; it proposes, you confirm.

Step 5: Stream a database into it

Kafka doesn't watch your database on its own. Something has to notice a row changed and turn that into a message. That's what change data capture (CDC) is for: instead of your application code writing to Postgres and remembering to also publish an event, a connector watches the database's replication stream and emits a Kafka message for every insert, update, and delete, automatically and in order. Debezium is the standard open-source connector for this, and it's what turns "a row changed in Postgres" into "a topic anyone downstream can subscribe to," with no application code changes required.

Wiring one up is normally its own small project: install Kafka Connect, get the plugin onto it, figure out the database connection config, get the credentials into the right place, and create the destination topic. Through the MCP, it's one instruction. And because both the Kafka service and the PostgreSQL service can live in the same Aiven project, the MCP resolves the source database's credentials automatically, instead of you copying a password between tabs.

Loading code...

Your assistant assembles the connector configuration (connector class, database host and credentials pulled from orders-pg, the table to watch, the topic prefix), shows it to you, and creates it once you approve. From that point, every insert into orders shows up as a Kafka message without anything else changing:

Loading code...

"op": "c" marks an insert, captured and delivered. before/after show the full row state, which is what makes CDC useful for more than just notifications: a consumer reading this topic can reconstruct exactly what changed, not just that something did.

Built for safe agentic workflows

Giving an AI assistant write access to a running cluster only makes sense if you stay in control of what it can touch. The Aiven MCP is built for that:

• Read-only mode. Switch it on and every write disappears. The assistant can inspect topics, consumer groups, and connector status, but it can't create, delete, or produce, regardless of what you ask it. For a first look at a production cluster, turn this on explicitly.

• Scoped tools. Only debugging Kafka today? Scope the connection to the services you care about and keep the assistant's context focused. Less noise, sharper answers.

• Your permissions, enforced. The assistant can only do what your own Aiven account is allowed to do. It inherits your access, it doesn't widen it.

• Hosted or local. Connect to the Aiven-hosted MCP and authorize in the browser with no token to manage, or run it locally with your own API token.

• Docs built in. The MCP can search Aiven's documentation in context with aiven_docs_search, so the assistant looks up the right answer instead of guessing.

Comprehensive, not complex

The MCP that reads your consumer lag is the same one that queries the PostgreSQL feeding your topics and wires the connector between them. One platform, one connection, one approval flow. Your assistant isn't a Kafka expert with no memory of your setup, it's something closer to a teammate who knows the stream, the database behind it, and, via Aiven Apps, the service running on top of both.

That platform runs on the cloud you already use: AWS, Google Cloud, Azure, and more. No new query language, no lock-in traded for convenience. The tools you'd reach for stay the tools you know; the assistant just reaches them faster.

One platform, one connection, one approval flow.

What's next

Today the Aiven MCP (EA) speaks PostgreSQL®, Apache Kafka®, and Aiven Apps (LA). OpenSearch®, ClickHouse®, and deeper day-to-day workflows are on the way, so your agent's reach grows alongside your platform.

Your data platform is a prompt away. Get building.