We look at Aiven for PostgreSQL version 12 plan performance between Amazon Web Service and Google Cloud Platform in terms of transactions per second and cost.
John Hammink
|RSS FeedDeveloper Advocate at Aiven
Subscribe for the latest news and insights on open source, Aiven offerings, and more.
Check out the updated article for 2021: Aiven for PostgreSQL 13 performance on GCP, AWS and Azure
At Aiven, we’ve run different PostgreSQL cloud performance benchmarks since 2016. In this latest post, we’ll consider four different benchmarks that compare write performance for PostgreSQL 12.1 between Amazon Web Services (AWS) and Google Cloud Platform (GCP).
We not only measured the performance in terms of speed and cost variations across the two clouds, but also examined the effects of adding CPUs while adjusting RAM and storage upward to write performance.
We generated load using pgbench, which we ran on another VM in the same cloud. The load generator commands for each plan were as follows:
``$$ pgbench --client=<startup_plan> --jobs=2 --report-latencies --time=3600 --log --aggregate-interval=1 keepalives='1' keepalives_count='12' keepalives_idle='180' keepalives_interval='10' sslmode='require'```
In this case the --client switch was the variable adjusted geometrically to correspond to the startup plan under test. Possible values are 2, 4, 8, 16, for each of the four startup plans, respectively. Additionally:
As in a production PG setup, the local SSDs running PG used LUKS full-disk encryption with WAL archiving enabled. The PG instances under test were initialized as follows (shown here without connection parameters):
``$$ pgbench --initialize --scale=125 --quiet keepalives='1' keepalives_count='12' keepalives_idle='180' keepalives_interval='10' sslmode='require'```
Here, the --scale parameter was adjusted to accommodate each plan; in this case, the DB size was roughly double the size of RAM, with the possible values of 125, 500, 2000, and 4000.
Thus, our four benchmarks measured and compared performance of PostgreSQL 12.1 running on Amazon Web Services (AWS) and Google Cloud Platform in each of the following four Aiven managed PostgreSQL plan configurations:
To minimize the effects of variable latency on our benchmark, we used these comparable regions for each provider, respectively:
Finally, we concluded the benchmark by calculating the average mTPS/$ per PG plan.
Aiven startup-4 plan tests consisted of the following configurations:
Load generator: 2 cores and 2GB RAM
System under test:
On Aiven for PostgreSQL 12.1 startup-4 plans, the difference was 483 to 413 respectively, with AWS leading by 16.76 percent.
Aiven startup-8 plan tests consisted of the following configurations:
Load generator:
System under test:
On this, as on all plans tested subsequently, GCP was the leader: with 940 TPS to AWS’s 645 TPS (a 31.34% difference).
Aiven startup-16 plan tests consisted of the following configurations:
Load generator:
System under test:
Here, the performance gap between GCP and AWS widened to 45.72% (GCP’s 1881 vs. AWS’s 1021 TPS).
Aiven startup-32 plan tests consisted of the following configurations:
Load generator:
System under test:
Finally, on Aiven’s largest startup plan, the performance gap between the clouds narrowed slightly to 42.73% (with GCP at 3802 and AWS at 2177 TPS).
Performance value was calculated as an average of millions of transactions per second per dollar or (mTPS/$) for each cloud provider. We found that, overall, the performance/cost difference between cloud providers quite substantially favored Google, although as before with the exception only of Startup-4 plans:
When averaging all the data across regions and plans for either cloud provider, overall the better cost performance boils down to better performance generally on GCP.
Google was hands down the better performer, with 1742 TPS vs. AWS’s 1081 TPS (a difference of 61.09%) for all Aiven plans on PG 12.1
Of course, many factors affect PostgreSQL performance, including latency differences between the different regions, which we used but didn’t differentiate for in this benchmark. There is also a difference to consider with variably-sized workloads.
We always recommend that standalone PostgreSQL users identify performance bottlenecks and tune their database configuration, workload or hardware configuration to optimize. However, in our managed PostgreSQL services, PostgreSQL system parameters come preconfigured for typical workloads, and you can try out different hardware configurations by choosing different Aiven for PostgreSQL plans.
We’ll be publishing more benchmarks like these moving forward, so stay tuned. In the meantime, check out our no commitment, 30-day trial, or find out more on the product page!
Many thanks go out to Felipe Morato for his hard work on these benchmarks!
