Migrate a public service to a Virtual Private Cloud (VPC)
When operating your Aiven service over the public Internet, you might consider enhancing security or connectivity by transitioning to a Virtual Private Cloud (VPC, also known as VNet). This move allows you to restrict access and establish a more controlled environment between your VPC and Aiven.
Aiven simplifies the migration process, allowing for seamless one-click transitions between different regions and cloud providers, including the migration from the public Internet to a VPC. However, migrating your service can disrupt connectivity due to the service URI's change in resolution from public IP addresses to IP addresses within a VPC.
To maintain uninterrupted access throughout this transition, Aiven provides a public access advanced configuration This ensures that your service remains accessible over both the public Internet and within the VPC during migration.
To ensure uninterrupted access to your service during the migration phase, conduct a few simple tests to verify connectivity.
Migrate a public service to a VPN
- VPC creation and peering: Create a VPC and establish peering.
- Network peering validation: Test the network peering with a non-critical service.
- Enable public access: Activate public access for all services to be migrated.
- Application configuration update: Modify your application to use public access hostnames.
- Service migration: Start the migration of the service into the VPC.
- Peering connections validation: Confirm the functionality of peering connections with private hostnames and ports.
- Switch to private access hostnames: Change application configuration to use private access hostnames.
- Disable public access: Turn off public access for all services.
Steps 3, 4, 6, 7, 8 are optional but highly recommended. Following these steps ensures that networking configuration and firewall rules are set up correctly.
Initial setup
Before you begin, make sure you have created your VPC and that peering is active. Automating this process with Terraform is an option. For guidance, refer to setting up VPC peering on the Aiven platform.
To illustrate the process, we will use the google-us-east1
VPC as a
reference. Ensure both your VPC and its peering connection are in an
Active
state.
Testing the connection
To confirm network peering, test the connection with a non-critical service within your VPC. This involves creating a small service in an existing VPC to check the network connectivity.
Deploy your service into a VPC. Then, from a host within your VPC, ensure you can resolve the DNS hostname and connect to the service port. The following commands are typically used on Ubuntu 20.04 LTS and should be applicable for most Linux distributions:
nslookup {host}
nc -vz {host} {port}
Enable public access
Enable the public_access.{service type}
configuration for all services
you are migrating. This setting is found in the Advanced
Configuration section of the Aiven
Console on your service's Overview
page. For example, for Aiven for Apache Kafka®, the configuration would
be public_access.kafka
. This action creates a new hostname and port,
which remain publicly accessible even after the service is moved into
the VPC.
See the new hostname and port under Connection Information by selecting the Public Access Route. Make sure you can connect to each new host/port combination.
Configure and redeploy application
It's recommended to configure your applications to use the public access route during the migration. This ensures continued access to the services as they transition to a private network. In dual access mode, test all connections and ports before switching over to the private hostname(s) and IP addresses.
Migrate an Aiven service to your VPC
You can migrate your Aiven services into a VPC using the Aiven Console .
- Log in to the Aiven Console select your project and select the service.
- On the service page, select Service settings from the sidebar.
- In the Cloud and network section, click Actions > Change cloud or region.
- In the Migrate service to another cloud window, select the
VPC tab and choose the appropriate region for your project's
dedicated VPC. The
Public Internet
tag is displayed. - Start the migration process.
- Once the migration is complete, ensure the
Project VPC
tag appears on the service's page. This tag indicates the migration was successful.
In Aiven Console, use the Cloud and VPC >
Migrate cloud section on the service's Overview page to
migrate your Aiven services into a VPC. Note the Public Internet
tag.
Ensure that you select the region from the VPC
tab. This is a
dedicated VPC for your project.
Ensure that you see the Project VPC
tag after migration. You can
monitor the migration status on the service's page in Aiven
Console.
Testing the service connections
Post-migration, use the nslookup
command to see private IP addresses.
Ensure you can connect to the private hostnames and ports, verifying
that firewall rules and routing are functioning correctly.
Configure and redeploy your applications
After the migration, reconfigure your applications to use the private hostname.
Cleanup by disabling public access
To finalize the migration, disable the public_access.{service type}
configuration for all your services. This option is located in the
Advanced configuration section of the Aiven Console on your
service's Overview page. Disabling it removes the public-
prefixed hostname and port.
Conclusion
You can migrate services from public to VPC with minimal downtime, ensuring safe and tested connections throughout the process. Always equip your client applications with failure and retry logic to adapt to changes in servers and IP addresses. While this is typically straightforward for clustered services like Apache Kafka® and OpenSearch®, additional configurations might be necessary for services like PostgreSQL® and Caching.