Scale disk storage automatically for your Aiven for MySQL® service
Automatically increase the disk storage of your Aiven for MySQL® service when it's running out of space, instead of resizing it manually.
Use the Aiven Autoscaler to automatically increase the storage capacity of a service disk when it's running out of space. Disk autoscaler only increases storage, it doesn't scale storage down.
Why use disk autoscaling
- Cost efficiency: Start with a regular-sized disk and let Aiven scale it up only when needed, without the risk of running out of disk space.
- Resiliency: Avoid a service becoming non-functional because it ran out of disk space, including during unexpected spikes in demand.
How it works
-
You create an autoscaler integration endpoint in your project, setting the maximum total disk size to allow.
-
You enable an autoscaler integration for your service using that endpoint.
-
Aiven monitors the disk space usage of your service.
-
When disk usage reaches the threshold for your service type, Aiven increases the available storage by at least 10%, using the current used space as a baseline.
noteThe exact increase depends on the service type and cloud provider. Some providers enforce a minimum increase of 10 GB.
Autoscale thresholds per service typeThe threshold that triggers disk autoscaling is a percentage of the available disk storage capacity:
- Aiven for OpenSearch®: 75% of the available disk storage capacity
- All other supported service types: 85% of the available disk storage capacity
-
The disk increase is recorded in the project event log, and you receive a notification about the added disk space.
Limitations and considerations
- Maximum storage: The maximum storage that the autoscaler can allocate for your service is limited by both the maximum disk size set on the autoscaler endpoint and the maximum disk storage supported for your service plan.
- Timing: Autoscaling takes a moment to complete. In the meantime, the service disk might fill up and the service might enter read-only mode until autoscaling finishes, unless the autoscaler's disk capacity limit is reached.
- Maintenance updates: Autoscaling works only on fully running services and can't happen during a maintenance update.
- Manual changes: Changing disk space manually can delay an autoscaling event.
- Terraform: Don't manage disk space with the Aiven Terraform Provider on a service that uses the autoscaler, to avoid conflicts between the two.
- Performance: Disk added through autoscaling is slower than the original disk until the next maintenance update applies. This might affect I/O-intensive workloads.
Prerequisites
- An Aiven organization, project, and service that's up and running
- The operator role for the organization, project, and service
- Dynamic disk sizing support on your service plan and cloud region
- One of the following to manage the autoscaler:
Enable disk autoscaling
To enable disk autoscaling, create an autoscaler integration endpoint, then enable an autoscaler integration on your service using that endpoint.
- Console
- API
- CLI
- Terraform
Create an autoscaler endpoint:
- Log in to Aiven Console, and go to your organization and project.
- On the left sidebar, click Integration endpoints.
- Click Aiven Autoscaler > Add new endpoint.
- Set the endpoint name and the maximum total disk storage in GB, and click Add endpoint.
Enable the autoscaler on a service:
- On the left sidebar, click Services, and open your service.
- On the left sidebar, click Integrations.
- In Endpoint integrations, click Aiven Autoscaler.
- Click the endpoint you created, and click Enable.
-
Call ServiceIntegrationEndpointCreate to create an autoscaler integration endpoint on your project:
curl --request POST \--url https://api.aiven.io/v1/project/PROJECT_NAME/integration_endpoint \--header "Authorization: Bearer TOKEN" \--header "Content-Type: application/json" \--data '{"endpoint_name": "ENDPOINT_NAME","endpoint_type": "autoscaler","user_config": {"autoscaling": [{"type": "autoscale_disk","cap_gb": 300}]}}' -
Call ServiceIntegrationCreate to enable the autoscaler integration on your service, using the endpoint ID from the previous response:
curl --request POST \--url https://api.aiven.io/v1/project/PROJECT_NAME/integration \--header "Authorization: Bearer TOKEN" \--header "Content-Type: application/json" \--data '{"dest_endpoint_id": "ENDPOINT_ID","integration_type": "autoscaler","source_project": "PROJECT_NAME","source_service": "SERVICE_NAME"}'
-
Create an autoscaler integration endpoint using avn service integration-endpoint-create:
avn service integration-endpoint-create \--project PROJECT_NAME \--endpoint-name ENDPOINT_NAME \--endpoint-type autoscaler \--user-config-json '{"autoscaling": [{"type": "autoscale_disk", "cap_gb": 300}]}' -
Find the ID of the new endpoint:
avn service integration-endpoint-list --project PROJECT_NAME -
Enable the autoscaler integration on your service, using the endpoint ID from the previous step:
avn service integration-create \--dest-service SERVICE_NAME \--integration-type autoscaler \--source-endpoint-id ENDPOINT_ID
Use the
aiven_service_integration_endpoint
and
aiven_service_integration
resources:
resource "aiven_service_integration_endpoint" "autoscaler_endpoint" {
project = "PROJECT_NAME"
endpoint_name = "ENDPOINT_NAME"
endpoint_type = "autoscaler"
autoscaler_user_config {
autoscaling {
type = "autoscale_disk"
cap_gb = 300
}
}
}
resource "aiven_service_integration" "autoscaler_integration" {
project = "PROJECT_NAME"
integration_type = "autoscaler"
source_service_name = "SERVICE_NAME"
destination_endpoint_id = aiven_service_integration_endpoint.autoscaler_endpoint.id
}
See the disk autoscaler guide for more details.
Change the maximum disk space for autoscaling
After you enable disk autoscaling, you can update the maximum total disk size at any time.
- Console
- API
- CLI
- Terraform
- On the left sidebar, click Integration endpoints.
- Click Aiven Autoscaler.
- Find your endpoint, click Actions, and click the option to edit it.
- Set a new maximum disk storage value, and save your changes.
Call
ServiceIntegrationEndpointUpdate
with the new cap_gb value:
curl --request PUT \
--url https://api.aiven.io/v1/project/PROJECT_NAME/integration_endpoint/ENDPOINT_ID \
--header "Authorization: Bearer TOKEN" \
--header "Content-Type: application/json" \
--data '{
"user_config": {
"autoscaling": [
{
"type": "autoscale_disk",
"cap_gb": 500
}
]
}
}'
avn service integration-endpoint-update ENDPOINT_ID \
--user-config-json '{"autoscaling": [{"type": "autoscale_disk", "cap_gb": 500}]}'
Update the cap_gb value in the autoscaling block of your
aiven_service_integration_endpoint
resource, then apply the change.
Turn off disk autoscaling
To turn off disk autoscaling, remove the autoscaler integration from your service. You can also delete the integration endpoint if you no longer need it.
- Console
- API
- CLI
Disconnect the service from the autoscaler:
- On the left sidebar, click Services, and open your service.
- On the left sidebar, click Integrations.
- In Endpoint integrations, find Aiven Autoscaler, click Actions, and click the option to disconnect it.
Delete the autoscaler endpoint, if you no longer need it:
- On the left sidebar, click Integration endpoints.
- Click Aiven Autoscaler.
- Find your endpoint, and delete it.
-
Call ServiceIntegrationDelete to remove the autoscaler integration from your service:
curl --request DELETE \--url https://api.aiven.io/v1/project/PROJECT_NAME/integration/INTEGRATION_ID \--header "Authorization: Bearer TOKEN" -
Call ServiceIntegrationEndpointDelete to delete the autoscaler integration endpoint, if you no longer need it:
curl --request DELETE \--url https://api.aiven.io/v1/project/PROJECT_NAME/integration_endpoint/ENDPOINT_ID \--header "Authorization: Bearer TOKEN"
-
Find the ID of the integration to remove:
avn service integration-list SERVICE_NAME -
Remove the autoscaler integration from your service:
avn service integration-delete INTEGRATION_ID -
Find the ID of the integration endpoint to delete, if you no longer need it:
avn service integration-endpoint-list --project PROJECT_NAME -
Delete the autoscaler integration endpoint:
avn service integration-endpoint-delete ENDPOINT_ID
Related pages