PostgreSQL 12 performance on in GCP and AWS [Benchmark]

In our latest benchmark, we look at Aiven for PostgreSQL version 12 plan performance between Amazon Web Service and Google Cloud Platform in terms of transactions per second and cost.

15 April 2020
John HamminkDeveloper Advocate at Aiven

At Aiven, we’ve run different PostgreSQL cloud performance benchmarks since 2016. In this latest post, we’ll consider four different benchmarks that compare write performance for PostgreSQL 12.1 between Amazon Web Services (AWS) and Google Cloud Platform (GCP).

We not only measured the performance in terms of speed and cost variations across the two clouds, but also examined the effects of adding CPUs while adjusting RAM and storage upward to write performance.

We generated load using pgbench, which we ran on another VM in the same cloud. The load generator commands for each plan were as follows:

$ pgbench --client=<startup_plan> --jobs=2 --report-latencies --time=3600 --log --aggregate-interval=1 keepalives='1' keepalives_count='12' keepalives_idle='180' keepalives_interval='10' sslmode='require'

In this case the --client switch was the variable adjusted geometrically to correspond to the startup plan under test. Possible values are 2, 4, 8, 16, for each of the four startup plans, respectively. Additionally:

  • The load generator had only 2 cores and 2GB RAM when tests ran Aiven startup-4 or startup-8 plans,
  • The load generator had 4 cores and 4GB RAM for the other plans.

As in a production PG setup, the local SSDs running PG used LUKS full-disk encryption with WAL archiving enabled. The PG instances under test were initialized as follows (shown here without connection parameters):

$ pgbench --initialize --scale=125 --quiet keepalives='1' keepalives_count='12' keepalives_idle='180' keepalives_interval='10' sslmode='require'

Here, the --scale parameter was adjusted to accommodate each plan; in this case, the DB size was roughly double the size of RAM, with the possible values of 125, 500, 2000, and 4000.

Thus, our four benchmarks measured and compared performance of PostgreSQL 12.1 running on Amazon Web Services (AWS) and Google Cloud Platform in each of the following four Aiven managed PostgreSQL plan configurations:

Aiven Plans for PG 12 Benchmark

To minimize the effects of variable latency on our benchmark, we used these comparable regions for each provider, respectively:

AWS regions:

  • eu-west-1
  • eu-west-2
  • us-east-1
  • us-west-1

GCP regions:

  • europe-west1
  • europe-west2
  • us-east4
  • us-west1

Cost calculation comparison

Finally, we concluded the benchmark by calculating the average mTPS/$ per PG plan.

Transactions Per Second: Aiven Startup-4 plans

Aiven startup-4 plan tests consisted of the following configurations:

Load generator: 2 cores and 2GB RAM

System under test:

  • 1 dedicated VM
  • 1 core
  • 4 GB RAM
  • 80 GB storage

Average Transactions per Second on Aiven Startup-4 plans

On Aiven for PostgreSQL 12.1 startup-4 plans, the difference was 483 to 413 respectively, with AWS leading by 16.76 percent.

Transactions Per Second: Aiven Startup-8 plans

Aiven startup-8 plan tests consisted of the following configurations:

Load generator:

  • one VM deployed in the same region as the DB
  • 2 cores
  • 2 GB RAM

System under test:

  • 1 dedicated VM
  • 2 cores
  • 8 GB RAM
  • 175 GB storage

Average Transactions per Second on Aiven for PostgreSQL Startup-8 plans

On this, as on all plans tested subsequently, GCP was the leader: with 940 TPS to AWS’s 645 TPS (a 31.34% difference).

Transactions Per Second: Aiven Startup-16 plans

Aiven startup-16 plan tests consisted of the following configurations:

Load generator:

  • one VM deployed on the same region as the DB
  • 4 cores
  • 4 GB RAM

System under test:

  • 1 dedicated VM
  • 4 cores
  • 15 GB RAM
  • 350 GB storage

Average Transactions per Second on Aiven for PostgreSQL Startup-16 plans

Here, the performance gap between GCP and AWS widened to 45.72% (GCP’s 1881 vs. AWS’s 1021 TPS).

Transactions Per Second: Aiven Startup-32 plans

Aiven startup-32 plan tests consisted of the following configurations:

Load generator:

  • one VM deployed on the same region as the DB
  • 4 cores
  • 4 GB RAM

System under test:

  • 1 dedicated VM
  • 8 cores
  • 30 GB RAM
  • 700 GB storage

Average Transactions per Second on Aiven for PostgreSQL Startup-32 plans

Finally, on Aiven’s largest startup plan, the performance gap between the clouds narrowed slightly to 42.73% (with GCP at 3802 and AWS at 2177 TPS).

mTPS/$ average per service version

Performance value was calculated as an average of millions of transactions per second per dollar or (mTPS/$) for each cloud provider. We found that, overall, the performance/cost difference between cloud providers quite substantially favored Google, although as before with the exception only of Startup-4 plans:

Table for transactions per second per dollar per postgreSQL plan.

Conclusions

When averaging all the data across regions and plans for either cloud provider, overall the better cost performance boils down to better performance generally on GCP. Google was hands down the better performer, with 1742 TPS vs. AWS’s 1081 TPS (a difference of 61.09%) for all Aiven plans on PG 12.1

Mean Transactions per Second Average on all Aiven PostgreSQL plans

Of course, many factors affect PostgreSQL performance, including latency differences between the different regions, which we used but didn’t differentiate for in this benchmark. There is also a difference to consider with variably-sized workloads.

We always recommend that standalone PostgreSQL users identify performance bottlenecks and tune their database configuration, workload or hardware configuration to optimize. However, in our managed PostgreSQL services, PostgreSQL system parameters come preconfigured for typical workloads, and you can try out different hardware configurations by choosing different Aiven for PostgreSQL plans.

We’ll be publishing more benchmarks like these moving forward, so stay tuned. In the meantime, check out our no commitment, 30-day trial, or find out more on the product page!

Many thanks go out to Felipe Morato for his hard work on these benchmarks!

Blog CTA PostgreSQL webinar

benchmarkspostgresql

Start your free 30 day trial

Test the whole platform for 30 days with no ifs, ands, or buts.

Aiven logo

Let‘s connect

Apache Kafka, Apache Kafka Connect, Apache Kafka MirrorMaker 2, Apache Cassandra, Elasticsearch, PostgreSQL, MySQL, Redis, InfluxDB, Grafana are trademarks and property of their respective owners. All product and service names used in this website are for identification purposes only and do not imply endorsement.