Skip to main content

Configure rack awareness in Aiven for Apache Kafka® MirrorMaker 2

Configure rack awareness in Aiven for Apache Kafka® MirrorMaker 2 to reduce cross-availability zone (AZ) network traffic by directing MirrorMaker to read from local follower replicas instead of remote partition leaders.

About rack awareness in MirrorMaker 2​

Rack awareness in MirrorMaker 2 requires follower fetching to be enabled on the source Aiven for Apache Kafka® service and on the replication flow.

When follower fetching is enabled for a replication flow, MirrorMaker automatically sets a rack value based on the availability zone where the MirrorMaker node runs and prefers reading from in-sync follower replicas in the same availability zone. This can reduce cross-availability zone network traffic and associated costs.

If the source Kafka cluster does not support follower fetching or uses different rack identifiers, Kafka ignores the rack value and MirrorMaker reads from partition leaders.

Follower fetching is enabled by default for new replication flows. If follower fetching is disabled for a replication flow, MirrorMaker reads only from partition leaders for that flow and rack awareness has no effect.

For details on follower fetching, see Follower fetching in Aiven for Apache Kafka®.

When to use rack awareness​

Use rack awareness when:

  • MirrorMaker 2 and the source Kafka service run across multiple availability zones.
  • Cross-AZ network costs or latency are a concern.
  • The source Kafka service is hosted on Aiven.

Rack awareness does not provide benefits when the Kafka cluster runs in a single availability zone.

Prerequisites​

  • A running Aiven for Apache Kafka® MirrorMaker 2 service.
  • Follower fetching enabled on the source Aiven for Apache Kafka® service.

Enable or disable rack awareness for a replication flow​

Rack awareness is controlled by follower fetching and can be enabled or disabled per replication flow.

When enabled, MirrorMaker assigns a rack value based on the availability zone where the node runs and prefers reading from in-sync follower replicas in the same availability zone. When disabled, MirrorMaker reads from partition leaders for that replication flow.

  1. In the Aiven Console, open the MirrorMaker 2 service.
  2. Click Replication flows.
  3. Create a replication flow or edit an existing one.
  4. Set Follower fetching enabled to on or off.
  5. Click Create or Save.

Related pages