The Snowflake sink connector for Aiven for Apache Kafka® has been upgraded to version 1.9.3. This version brings several new features, important configuration changes, and improved error handling.
New features
- Support for n:1 topics:table (SNOW-514303): The connector now supports mapping multiple Kafka topics to a single Snowflake table.
- Schema mapping (SNOW-625820): The connector has enhanced support for schema mapping, allowing you to map Kafka topic schemas to Snowflake table schemas, providing flexibility and control over how your data is structured in Snowflake.
- Auto table creation (SNOW-628677): This new feature enables the automatic creation of Snowflake tables to store Kafka data, reducing manual configuration requirements.
Configuration changes
- Exactly once for streaming ingest (SNOW-536429): Configuration support has been introduced for exactly-once semantics during streaming data ingestion, guaranteeing data consistency and integrity.
Improvements in error handling
- Offset token retries (SNOW-543664 and SNOW-451197): The handling of failed offset tokens has been improved, including incorporating automatic retries and offset replays for heightened reliability.
- Handling of failed insert rows and delivery to Dead-Letter Queue (DLQ) (SNOW-530369): Improved failure handling for row insertion has been implemented along with the new functionality to deliver failed records to a DLQ.
- Implementation of ErrantRecordReporter (SNOW-530369): Kafka records from the sink connector can be transferred to the DLQ for further analysis and reprocessing.
In addition to the above, there have also been updates to dependencies and multiple bug fixes to improve the overall performance and stability of the Snowflake Kafka Connector. For a comprehensive overview of all the changes, see the changelog on GitHub. To learn how to use this connector, see Configure Snowflake for a sink connector.
