End to end managed development pipelines with Aiven and Qovery

Hey there! It's Romaric from Qovery, and I'm excited to bring you this article that I co-wrote with Sébastien Blanc from Aiven. In this article, we'll show you how Aiven for PostgreSQL® and Qovery work together to streamline your app development and deployment on AWS. But before we dive into the details, let's take a closer look at Aiven and Qovery separately.

Aiven x Qovery

Aiven and Qovery are two powerful tools that, when used together, provide a great solution for modern application development. Aiven offers a simple and fast way to spin up open source data infrastructures, like PostgreSQL®, Redis® and Apache Kafka®, while Qovery [provides a platform ] for developers to easily deploy, manage and scale their applications.

When it comes to developing modern applications, it's important to have a database that is fast, reliable, and easy to manage. This is where Aiven for PostgreSQL® comes in. Aiven provides a cloud-based database service that is fully managed and can be set up and configured in a matter of minutes. With Aiven, you get access to a fully managed PostgreSQL database designed to scale with your needs, all while ensuring high availability, data safety, and data security.

Qovery, on the other hand, provides a modern platform for developers to deploy and manage their applications. With Qovery, developers can easily create multiple environments that allow them to test and preview their applications before deployment to production. Qovery also makes it easy to integrate external services, like Aiven for PostgreSQL, into your application.

Together, Aiven and Qovery provide a powerful solution for modern application development. Developers can easily spin up an Aiven for PostgreSQL instance and integrate it with their Qovery application, all while taking advantage of Qovery's powerful platform for managing and deploying their application. With Aiven and Qovery, developers can focus on building great applications without worrying about the underlying infrastructure.

Now that we've covered the benefits of Aiven and Qovery let's take a closer look at how they work together to make your app development process even more efficient.

How Aiven and Qovery work together

To demonstrate the integration between Aiven and Qovery, we'll use a sample application written in Java. The application is a simple to-do list, and we'll deploy it on AWS with Qovery and Aiven for PostgreSQL® as the backend database.

We'll assume that you already have some familiarity with Qovery and Aiven, but if you need any help getting started, be sure to check out our Qovery and Aiven documentation.

To get started, you’ll need both an Aiven and a Qovery account. If you don’t have an Aiven account, sign up for a free trial to follow along. Qovery has a great free plan you can sign up for too.

To integrate Aiven with Qovery, we'll use Qovery's Lifecycle Job feature. A Lifecycle Job is a script that is run at specific points in your environment’s lifecycle, such as before it is deployed or after it is shut down. With Lifecycle Jobs, we can create and configure our Aiven Postgres instance automatically and ensure that it is available for use when our application starts.

In the diagram above, we have three main components: Qovery, Aiven, and our AWS account. It's important to note that Qovery deploys containerized apps on your AWS account rather than on a separate hosting platform (read how to create an AWS account). Meanwhile, Aiven uses its own AWS account to deploy and manage the PostgreSQL instance they provide to you.

In the Qovery environment, we define two services: our Aiven for PostgreSQL instance with a Terraform script and our to-do app with a Dockerfile. Both of these services come from GitHub repositories, and Qovery can build and execute them in the correct order. The Aiven for PostgreSQL service is deployed first, and then the to-do app service.

It looks like this from the Qovery web interface:

This is what our Terraform script looks like:

#main.tf

terraform {
  required_providers {
    aiven = {
      source  = "aiven/aiven"
      version = "~> 3.9.0"
    }
  }

  backend "s3" {}
}

provider "aiven" {
  api_token = var.aiven_api_token
}

resource "aiven_pg" "pg_instance" {
  project                 = var.aiven_project_name
  cloud_name              = "aws-us-east-2"
  plan                    = "startup-4"
  # make the service name unique by adding a suffix based on the first digit of the Qovery environment ID
  service_name            = "pg-${split("-", var.qovery_environment_id)[0]}"
  maintenance_window_dow  = "monday"
  maintenance_window_time = "10:00:00"
}

resource "aiven_pg_database" "pg_database" {
  database_name = var.aiven_database_name
  project       = var.aiven_project_name
  service_name  = aiven_pg.pg_instance.service_name
} 
Copy to clipboard

You can see it’s a vanilla Terraform script with the Aiven Provider module that it’s quite nice and straightforward to use.

At this point, you might wonder how to connect our to-do app to Aiven for PostgreSQL. If you are familiar with Terraform, you know that some variables are only known post-execution of our manifest. Qovery supports Terraform output values out of the box.

Here are our outputs:

output "postgres_host" {
  description = "The Aiven PG connection host"
  value       = aiven_pg.pg_instance.service_host
}

output "postgres_port" {
  description = "The Aiven PG connection port"
  value       = tostring(aiven_pg.pg_instance.service_port)
}

output "postgres_user" {
  description = "The Aiven PG username"
  value       = aiven_pg.pg_instance.service_username
}

output "postgres_db_name" {
  description = "The Aiven PG database name"
  value       = aiven_pg_database.pg_database.database_name
}

output "postgres_password" {
  description = "The Aiven PG password"
  value       = aiven_pg.pg_instance.service_password
  sensitive   = true
}
Copy to clipboard

So Qovery can inject those values as environment variables to our to-do application when known.

No magic! It simply works and it’s awesome.

If we look at our Aiven web console, we can see that our PostgreSQL database is also running properly.

Multiple Environments with Aiven and Qovery

With Qovery, you can easily create multiple environments and avoid the issues that arise with a shared staging environment. In traditional development environments, developers may have to share a single staging environment, which can lead to "staging starvation." If the environment is broken or unavailable, it affects all developers, slowing down the entire development process.

Qovery solves this problem by providing an efficient way to create multiple on-demand staging environments. Each developer can have their own staging environment, which is a replica of the production environment. This enables developers to test their code in an isolated and safe environment.

The power of Qovery doesn't stop there. When cloning an environment, all services included in the environment are duplicated. This means that your Aiven for PostgreSQL instance is also duplicated, and all the data from the original instance is available in the cloned instance. This ensures that your cloned environment has the same data as the original environment, making it easy to test your changes without affecting the production environment.
What about the data? You can clone the data if needed. Aiven provides everything you need to make it a breeze.

Conclusion

In this article, we've explored the benefits of using Aiven in combination with Qovery to accelerate development workflows and facilitate the creation of preview environments. By leveraging Qovery's lifecycle jobs and multiple environments, developers can enjoy a streamlined experience that speeds up development cycles and eliminates the traditional staging starvation problem.

Furthermore, the ease of integrating Aiven for PostgreSQL with Qovery showcases the power of combining best-in-class solutions for creating a truly robust and efficient development environment. And as we've seen, this combination is incredibly flexible and can be adapted to suit the unique needs of individual teams.

We hope this article has provided a useful overview of how Aiven and Qovery can work together. We're excited to continue collaborating with Sebastien and the Aiven team to explore other use cases, such as integrating Apache Kafka.

Stay tuned for our next collaboration!

Further reading

• The GitHub repository for this article
• The GitHub repository for the TODO app
• How to automagically open pull requests with GitHub Actions