Aiven now supports sstableloader, an open source utility which allows you to easily migrate all of the data from your external Apache Cassandra instances into a managed Aiven for Apache Cassandra cluster. Sstableloader allows this seamless migration by bulk-loading data from remote SSTables into an Aiven for Apache Cassandra cluster via the target nodes’ internode ports.
How sstableloader works
An SStable is a persistent, immutable file format where your Apache Cassandra data is stored. It is created when fast memory stores called memtables are flushed to disk. Sstableloader works by first connecting to the new Cassandra cluster to determine which nodes of the target cluster should receive which rows, then uploading the SSTable data to the target nodes.
This data upload - called streaming - is implemented as a command line client using the same Java code the Cassandra service uses internally to replicate data between cluster nodes, connecting to the Cassandra internode port to achieve a more efficient way to transfer large amounts of data.
Clients cannot normally connect to this internode port, and Aiven requires sstableloader to connect with SSL through client certificates to secure access. As a result, large clusters for which inserting the source cluster's data row by row with CQL would take unfeasibly long can now be migrated.
With sstableloader, you now have the industry standard for bulk data loading into Apache Cassandra which makes migrating your full data pipeline to Aiven that much simpler.
You can find the complete procedure here.
Jun 17, 2022
Announcing Dynamic Disk Sizing
Book additional storage by yourself in all Aiven services. No need to switch plans!
Dec 16, 2021
Announcing Apache Cassandra® 4.0
Aiven now supports Apache Cassandra® 4.0. This is the first major version upgrade since Apache Cassandra® 3.0 was released in 2015.
Subscribe to the Aiven newsletter
All things open source, plus our product updates and news in a monthly newsletter.