Create and manage snapshots in Aiven for OpenSearch® Limited availability
Create, list, retrieve, or delete snapshots in your Aiven for OpenSearch custom repositories.
Prerequisites
- Aiven Console
- OpenSearch API
- Aiven API
- Custom repositories enabled as a limited availability feature
- Running Aiven for OpenSearch service
- Access to the Aiven Console
- Access to a supported object storage service (AWS S3, GCS, or Azure)
- Credentials for the selected storage provider
- Configured custom repository
- Custom repositories enabled as a limited availability feature
- Maintenance updates applied for your service
- Security management enabled for your service
- Snapshot permissions and snapshot repository permissions configured
- Custom repositories enabled as a limited availability feature
- Running Aiven for OpenSearch service
- Aiven API and authentication token
- Access to a supported object storage service (AWS S3, GCS, or Azure)
- Credentials for the selected storage provider
- Configured custom repository
Automatic snapshot scheduling is not supported. You must create, list, and delete snapshots manually.
Limitations
- Aiven Console
- OpenSearch API
- Aiven API
Supported storage services are:
- Amazon S3
- Google Cloud Storage (GCS)
- Microsoft Azure Blob Storage
- Supported storage services are:
- Amazon S3
- Google Cloud Storage (GCS)
- Microsoft Azure Blob Storage
- The following operations are not supported via native OpenSearch API:
- Restore from snapshot has a couple of security-related restrictions.
- Create a snapshot
and
delete a snapshot
are not supported for snapshots in Aiven-managed repositories (prefixed with
aiven_repo
).
Supported storage services are:
- Amazon S3
- Google Cloud Storage (GCS)
- Microsoft Azure Blob Storage
Create a snapshot
Create a snapshot in a custom repository.
- Aiven Console
- OpenSearch API
- Aiven API
- Log in to the Aiven Console, go to your project, and open your service's page.
- Click Snapshots in the sidebar.
- On the Snapshots page, do one of the following:
- Click Create snapshot.
- Find your custom repository and click Actions > Create snapshot.
- In the Create snapshot window:
- Select a destination repository.
- Enter a snapshot name.
- Specify which indices to include.
- Optionally, enable the following:
- Ignore unavailable indices
- Include global state
- Partial snapshot
- Click Create.
Your snapshot is being created. Monitor its status until it shows Success.
Use the Create Snapshot native OpenSearch API endpoint.
curl -s -X POST \
--url "https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/{repository_name}/{snapshot_name}/_restore" \
--header "Authorization: Bearer $TOKEN" \
--header "Content-Type: application/json" \
-d '{"indices": "test*"}'
Example response:
{
"accepted": true
}
Restore from snapshots
Refrain from actions such as updating firewalls, changing index settings, or modifying security configurations during the restore process as it can cause restore failures.
- Aiven Console
- OpenSearch API
- Aiven API
-
Log in to the Aiven Console, go to your project, and open your service's page.
-
Click Snapshots in the sidebar.
-
On the Snapshots page, find your custom repository and click to expand the list of snapshots inside.
-
Find the snapshot to restore and click Actions > Restore to this service.
-
In the Restore snapshot window:
-
In the Indices field, enter the indices to include in the snapshot, separated by commas. These indices will be closed automatically before the restore begins.
Existing data may be overwritten as a result of the restoration process. To prevent this, use advanced configuration to add suffixes to the included indices.
-
Toggle Enable advanced configuration to set the following:
- Rename pattern: a regular expression that matches the original index name from the snapshot
- Rename replacement: a string that replaces the matched part of the index name
- Ignore unavailable indices
- Include aliases
-
Click Continue > Close indices.
This triggers the closing of the selected indices. Wait for Indices closed to be displayed, and click Continue.
-
Check the box labeled I understand the effects of this action.
warningThe restoration process you're about to start cannot be interrupted. Refrain from changes during this process: Updating firewalls, index settings, or security configuration during restore may cause failures.
-
Click Start restore.
This triggers the restoration process. It may take time, and it's length depends on the snapshot size.
-
Shut down the Restore snapshot window by clicking Close either during the restore process or when it completes.
tipIf you close the Restore snapshot window before the restore process is complete, you can preview its status on the Snapshots page.
-
To restore data from a snapshot, use the Restore Snapshot native OpenSearch API endpoint.
curl -s -X POST \
"https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/{repository_name}/{snapshot_name}/_restore" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"indices": "test-index-*",
"include_aliases": true,
"ignore_unavailable": false,
"rename_pattern": "index_(.+)",
"rename_replacement": "restored_index_$1"
}'
Example response:
{
"accepted": true
}
List snapshots in progress
Preview snapshots that are still being created in a repository.
- Aiven Console
- Aiven API
- Log in to the Aiven Console, go to your project, and open your service's page.
- Click Snapshots in the sidebar.
- On the Snapshots page, find your custom repository and click to expand the list of snapshots inside.
- In the column with snapshot status information, find in progress values.
curl -s --url "https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/aws-repo/_status" \
--header "Authorization: Bearer $TOKEN"
--header "Content-Type: application/json"
Example response:
{
"snapshots": [
{
"repository": "aws-repo",
"snapshot": "second-snapshot",
"state": "SUCCESS",
"shards_stats": {
"done": 1,
"failed": 0,
"total": 1
},
"uuid": "osmCbdF-RMyyUKpWD-4bJA"
}
]
}
List snapshots in a repository
Preview all snapshots, including completed and failed ones.
- Aiven Console
- OpenSearch API
- Aiven API
- Log in to the Aiven Console, go to your project, and open your service's page.
- Click Snapshots in the sidebar.
- On the Snapshots page, find your custom repository and click to expand the list of snapshots inside.
Use native OpenSearch API endpoint GET /_snapshot/REPOSITORY_NAME/_all
, replacing the
REPOSITORY_NAME
with the actual name of your repository.
Example response:
{
"snapshots" : [
{
"snapshot" : "opensearch-123qaz456wsx789edc123q-infrequent",
"uuid" : "-abCabCabC-abCabCabCab",
"version_id" : 123456789,
"version" : "N.NN.N",
"remote_store_index_shallow_copy" : false,
"indices" : [
".plugins-ml-config",
".opensearch-sap-log-types-config",
".kibana_N",
".opensearch-observability"
],
"data_streams" : [ ],
"include_global_state" : true,
"state" : "SUCCESS",
"start_time" : "YYY-MM-DDTHH:MM:SS.619Z",
"start_time_in_millis" : 1234567891234,
"end_time" : "YYY-MM-DDTHH:MM:SS.624Z",
"end_time_in_millis" : 1234567891234,
"duration_in_millis" : 1234,
"failures" : [ ],
"shards" : {
"total" : 4,
"failed" : 0,
"successful" : 4
}
},
{
"snapshot" : "opensearch-123qaz456wsx789edc123q-frequent",
"uuid" : "-abCabCabC-abCabCabCab",
"version_id" : 123456789,
"version" : "N.NN.N",
"remote_store_index_shallow_copy" : false,
"indices" : [
".plugins-ml-config",
".opensearch-sap-log-types-config",
".kibana_N",
".opensearch-observability"
],
"data_streams" : [ ],
"include_global_state" : true,
"state" : "SUCCESS",
"start_time" : "YYY-MM-DDTHH:MM:SS.219Z",
"start_time_in_millis" : 12345678912345,
"end_time" : "YYY-MM-DDTHH:MM:SS.220Z",
"end_time_in_millis" : 1234567891234,
"duration_in_millis" : 1234,
"failures" : [ ],
"shards" : {
"total" : 4,
"failed" : 0,
"successful" : 4
}
},
{
"snapshot" : "opensearch-123qaz456wsx789edc123q-frequent",
"uuid" : "-abCabCabC-abCabCabCabQ",
"version_id" : 123456789,
"version" : "N.NN.N",
"remote_store_index_shallow_copy" : false,
"indices" : [
".plugins-ml-config",
".opensearch-sap-log-types-config",
".kibana_N",
".opensearch-observability"
],
"data_streams" : [ ],
"include_global_state" : true,
"state" : "SUCCESS",
"start_time" : "YYY-MM-DDTHH:MM:SS.088Z",
"start_time_in_millis" : 12345678912345,
"end_time" : "YYY-MM-DDTHH:MM:SS.890Z",
"end_time_in_millis" : 1234567891234,
"duration_in_millis" : 1234,
"failures" : [ ],
"shards" : {
"total" : 4,
"failed" : 0,
"successful" : 4
}
}
]
}
curl -s --url "https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/aws-repo/_all" \
--header "Authorization: Bearer $TOKEN"
--header "Content-Type: application/json"
Example response:
{
"snapshots": [
{
"snapshot": "first-snapshot",
"state": "SUCCESS",
"indices": ["test"],
"uuid": "7cdWedW7RC6FMSktlZTCDw"
},
{
"snapshot": "second-snapshot",
"state": "SUCCESS",
"indices": ["test"],
"uuid": "osmCbdF-RMyyUKpWD-4bJA"
}
]
}
View snapshot details
Get details of a specific snapshot.
- Aiven Console
- OpenSearch API
- Aiven API
- Log in to the Aiven Console, go to your project, and open your service's page.
- Click Snapshots in the sidebar.
- On the Snapshots page, find your custom repository, click to expand the list of snapshots inside, find a snapshot to be previewed, and click Actions > View snapshot details.
Use the native OpenSearch API endpoints: Get Snapshot or Get Snapshot Status.
curl -s --url "https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/aws-repo/first-snapshot" \
--header "Authorization: Bearer $TOKEN"
--header "Content-Type: application/json"
Example response:
{
"snapshots": [
{
"snapshot": "first-snapshot",
"state": "SUCCESS",
"indices": ["test"],
"uuid": "7cdWedW7RC6FMSktlZTCDw"
}
]
}
Delete a snapshot
Delete a snapshot from a repository.
- Aiven Console
- OpenSearch API
- Aiven API
- Log in to the Aiven Console, go to your project, and open your service's page.
- Click Snapshots in the sidebar.
- On the Snapshots page, find your custom repository, click to expand the list of snapshots inside, find a snapshot to be deleted, click Actions > Delete snapshot > Delete.
Use the Delete Snapshot native OpenSearch API endpoint.
curl -s -X DELETE \
--url "https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/aws-repo/first-snapshot" \
--header "Authorization: Bearer $TOKEN"
--header "Content-Type: application/json"
Example response:
{
"acknowledged": true
}
Error handling
The Aiven API returns OpenSearch errors as they are.
Exceptions:
- 502: OpenSearch did not respond.
- 409: The service is not powered on or does not support this feature.
Related pages