This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.

We are excited to announce the general availability of Apache Kafka Connect, Kafka Compaction and Mirror Maker2 compatibility of Azure Event Hubs.

Why use Event Hubs to run Kafka workloads?   

Azure Event Hubs takes real-time data streaming to the next level with its powerful platform as a service (PaaS) offering. It's a multi-protocol event ingestion service that natively supports Apache Kafka, AMQP and HTTPS.

With Azure Event Hubs you can:

• Effortlessly migrate your Kafka workload to the cloud without any code modifications. 
• Benefit from simplified operations through a fully managed platform as a service (PaaS) which eliminates the need for the management of Kafka clusters.
• Experience superior performance and reliability. 
• Significantly reduced costs compared to other managed Kafka offerings.

As an Azure service, Event Hubs leverages the power of Azure across multiple dimensions, including robust security measures, compliance adherence, and high availability. Additionally, it easily integrates with other Azure services, allowing you to harness the full potential of the Azure ecosystem for your data streaming needs.

Kafka Connect support GA

Azure Event Hubs Kafka Connect support GA enables users to seamlessly stream data between Event hubs and any other system that supports Kafka Connect. You can use any Kafka source or sink connectors which is compatible with Apache Kafka Connect APIs with Event Hubs.

By leveraging Azure Event Hubs and Kafka Connect, you can establish reliable and scalable data pipelines for efficient data transfer within your ecosystem.

To learn more about Kafka Connect support in Azure Event Hubs see Integrate Azure Event Hubs with Apache Kafka Connect

Log Compaction/Kafka Compaction supports GA

Compacting the event logs is a common requirement when you stream a recurring series of updatable events. As compacted event hubs/Kafka topics only keep the latest events, users don't need to worry about the growth of the event storage. Therefore, log compaction is commonly used in scenarios such as Change Data Capture (CDC), maintaining event in tables for stream processing applications and event caching.

You can enable log compaction at each event hub/Kafka topic level and ingest events to a compacted article from any supported protocol.  The way the feature works is that when compaction is enabled on an event hub, a compaction job is initiated. The compaction job scans the configured event hub and only keeps the latest event of a given event key.

With log compaction GA, Event Hubs now support all production workloads related to compacted event hubs/Kafka topics.

To learn more about log compaction support in Event Hubs, see Log compaction in Azure Event Hubs.

MirrorMaker2 support GA for migrate existing Kafka clusters to Event Hubs

Apache Kafka MirrorMaker 2.0 (MM2) is designed to make it easier to mirror or replicate topics from one Kafka cluster to another. MirrorMaker 2 uses the Kafka Connect framework to simplify configuration and scaling.

As Azure Event Hubs is compatible with Apache Kafka protocol, you can use Mirror Maker 2 to replicate data between an existing Kafka cluster and an Event Hubs namespace.

Mirror Maker 2 dynamically detects changes to topics and ensures source and target topic properties are synchronized, including offsets and partitions. It can be used to replicate data bi-directionally between Kafka cluster and Event Hubs namespace.

See Use Apache Kafka MirrorMaker 2 with Azure Event Hubs to learn more about using MirrorMaker2 with Azure Event Hubs.

Next Steps

With the introduction of general availability support for Kafka Connect, Kafka Compaction, and MirrorMaker2, we take a significant leap forward in achieving complete Kafka compatibility within Event Hubs. This significant advancement brings us closer to seamless integration and comprehensive functionality, enabling a seamless Kafka experience within the Event Hubs ecosystem.

Explore more about Azure Event Hubs’ Kafka capabilities at Apache Kafka Azure Cloud with Event Hubs.