Get things done with the Aiven CLI

There's more than one way to use Aiven. Read on to learn how to use the Aiven command line tool to do common tasks.

The Aiven command line

GUIs (Graphical User Interfaces) like the Aiven web
console are a very approachable way to interact with systems,
especially since they can guide you towards the actions you might want to do.
When it comes to repeatability and automation however, these interfaces have limitations.

Luckily, almost every action available in the web console is also possible
using the open source avn CLI tool,
aiven-client. This tool is also heavily used
within Aiven itself, which means that it is well supported and gains new
service management features regularly.

In this article I will lead you through some common operations, using a
PosgreSQL® service as an example.

Install `avn`

Following the instructions at Aiven CLI: Getting
started, install the Aiven
CLI tool with pip:

pip install aiven-client

A fun way to check that the command is working is with this special feature:

avn crab

This will give you a nice piece of ASCII art.

Get help with the `avn` command

You can get help from avn in a few different ways.

To get general help on how to use the command, including a summary of the switches
the command accepts, and a list of the top-level subcommands:

avn --help

You can also get help on specific commands by adding --help to the end of
the command line. For example to get some help with how to use the avn user login command:

avn user login --help

You can get a (long) list of all the avn commands and a brief note of what
they each do by using:

avn help

This can be useful if you're trying to "guess" what command to use.

For instance, here's a part of the output relating to PostgreSQL®,
showing us the commands to retrieve the service connection information
as a string, as a URI, or in the format that psql needs:

service connection-info pg string             PostgreSQL connection string
service connection-info pg uri                PostgreSQL service URI
service connection-info psql                  psql command string

Get an account

If you've already got an Aiven account, then you're ready to go.

Otherwise, you can sign up at the Aiven console - either for a free plan or for a free trial.

Log in to Aiven with `avn`

Most avn commands require authentication.
At the command line, this is normally done by logging in with an
authentication token.

Using a token rather than username and password is considered good practice.
Tokens can be limited to a specific time period, and they can also
be revoked, removing access only for those using that token, without affecting
other use cases. Resetting a password would have a much wider impact.

By default, logging in to Aiven with a password is not enabled. See Enable Aiven password if you really need it.

Get an authentication token, as described at Create an authentication
token,
copy it, and log in using the following command. You'll need to replace
YOUR-EMAIL-ADDRESS with the email address you've registered with Aiven.

avn user login YOUR-EMAIL-ADDRESS --token

This will prompt you to paste in your token, and then report that it has
saved your credentials to a JSON file on your filesystem.

An alternative to logging in is to specify the authentication token with each command. This approach can be useful in a script, where you would set an environment variable (for instance, AVN_AUTH_TOKEN) to the token, and then write each command as avn --auth-token $AVN_AUTH_TOKEN ...

Choose a project

Aiven uses "projects" to organise which services you can access. You can get a
list of the projects that you have access to, their default clouds, and some
information on payments with a command like this:

avn project list

For instance, as a Developer Educator at Aiven, I see output something like the following:

PROJECT_NAME   DEFAULT_CLOUD        CREDIT_CARD
=============  ===================  ===========
dev-advocates  google-europe-west3  N/A

Once you know which project you're going to be using, you can select it, replacing
PROJECT-NAME with the name of the actual project:

avn project switch PROJECT-NAME

This makes that the default project for future commands.

Commands will generally also take a --project switch if you want to use a project without selecting it.

Find out about a project

To find out more about a project:

avn project details

This will typically show the default cloud for the project, its billing
currency, and maybe details of the VAT ID being used, the billing address and
credit card data (if appropriate).

PROJECT_NAME   DEFAULT_CLOUD     BILLING_CURRENCY  VAT_ID
=============  ================  ================  ==========
dev-advocates  aws-eu-west-1     EUR               FIxxxxxxxx
    credit_card     = N/A
    billing_address = "Some address, somewhere"
    country_code    = FI

Get JSON data

Commands that return data also take a --json switch, which causes them to
return their data in JSON. This often includes more information than is
provided by the standard text output. For instance altering our earlier command to include the switch:

avn project details --json

This version gives a lot more detail, including account name and id, available credits and
current estimated balance.

Sometimes the data produced with --json can be overwhelming, especially if
you're only looking for one particular field. In that case, it can be useful
to pipe the JSON output into the jq tool,
and use a query to select just the information you want.

For instance, to find the name of the default cloud for the current project, do:

avn project details --json | jq '.[] | .default_cloud'

The command returns just the name of the cloud.

Find out what services are available

You can find out what services (PostgreSQL®, Apache Kafka®, OpenSearch®, etc.)
are available for the current project with this command:

avn service types

This will will list the services and the abbreviation used for each service
type.

For instance, the entry for PostgreSQL looks like this:

pg      PostgreSQL - Object-Relational Database Management System

Since PostgreSQL is such an excellent database, we're going to create a pg service and show off some more avn commands.

Find a cloud

In order to create a new service, you need to decide what region you want it
to run in, and what capabilities you need. To find out what clouds and regions
are available:

avn cloud list

This will show the available clouds and information about them - specifically,
a description of the cloud, its name, its location (as latitude and longitude)
and its region.

The output will be lines looking something like:

Europe, Belgium - Google Cloud: Belgium     google-europe-west1     50.45    3.82     europe

Using the --json switch and jq can help if you want to get more
details of just part of the information. For instance, to find the information
for all the services in europe, try this:

avn cloud list --json | jq '.[] | select(.geo_region=="europe")'

You should see a list of entries like the one below:

{
  "cloud_description": "Europe, Finland - Google Cloud: Finland",
  "cloud_name": "google-europe-north1",
  "geo_latitude": 60.5693,
  "geo_longitude": 27.1878,
  "geo_region": "europe"
}

To get just that particular entry, you could use:

avn cloud list --json | jq '.[] | select(.cloud_name=="google-europe-north1")'

Find a service plan

Aiven uses "service plans" to define the number of servers and what kind of
memory, CPU, and disk resources are allocated to your service.

To get a list of the available service plans for PostgreSQL in the cloud
google-europe-north1, including the plan name, its cost and what resources
it provides, use this command:

avn service plans --service-type pg --cloud google-europe-north1

Here's part of the output I get:

PostgreSQL - Object-Relational Database Management System Plans

    pg:hobbyist                    $0.026/h  Hobbyist (1 CPU, 2 GB RAM, 8 GB disk)
    pg:startup-4                   $0.103/h  Startup-4 (1 CPU, 4 GB RAM, 80-240 GB disk)
    pg:startup-8                   $0.205/h  Startup-8 (2 CPU, 8 GB RAM, 175-525 GB disk)

It's worth trying different clouds to see how pricing differs between them.

The price quoted is all inclusive (although some services also allow the purchase of extra storage space).

Create a PostgreSQL^® service

For this first service creation, create a PostgreSQL service on the Finnish
Google cloud at the Hobbyist level.

It's important to give the service a unique name - I like to put my own
name into it, but you should use your own! Beware that the name can't be
changed after you've created the service.

avn service create YOUR-NAME-demo-pg --service-type pg \
    --cloud google-europe-north1 --plan hobbyist

The command will return immediately. Meanwhile, Aiven will start creating the service, which will take a little while to build.

If your target cloud includes VPCs (virtual private clouds) then it is safest to be explicit whether you want to create the new service in the VPC or not. See avn vpc list if you're not sure if your project has VPCs, and avn service create for the VPC related switches, specifically --no-project-vpc and --project-vpc-id PROJECT_VPC_ID

If you want to wait for the service to start, you can use the wait command for the service:

avn service wait YOUR-NAME-demo-pg

This will report the progress of the newly-created service as it comes up.

INFO	Service 'YOUR-NAME-demo-pg' state is now 'REBUILDING'
INFO	Waiting for services to start
...
INFO	Waiting for services to start
INFO	Service 'YOUR-NAME-demo-pg' state is now 'RUNNING'
INFO	Service(s) RUNNING: YOUR-NAME-demo-pg

The command will return when the service is RUNNING (the ... means there might be multiple "Waiting" messages).
The wait command is especially useful in a script, when you have another command to run when things are ready.

You can explicitly check the state of the service, as well as other
information:

avn service get YOUR-NAME-demo-pg

This returns service information like this:

SERVICE_NAME       SERVICE_TYPE  STATE       CLOUD_NAME            PLAN      CREATE_TIME           UPDATE_TIME           NOTIFICATIONS
=================  ============  ==========  ====================  ========  ====================  ====================  =============
YOUR-NAME-demo-pg  pg            REBUILDING  google-europe-north1  hobbyist  2022-05-25T12:18:02Z  2022-05-25T12:18:43Z

If you just want the current state of the service, use --json and jq as we did earlier to
get the field you want from the output:

avn service get YOUR-NAME-demo-pg --json | jq .state

Interact with the PostgreSQL service

Once the PostgreSQL service is running, we can use the command line to
interact with it.

To find out what databases are present:

avn service database-list YOUR-NAME-demo-pg

This will just show the default database since we just created a new service:

defaultdb

We can create a new database using avn:

avn service database-create YOUR-NAME-demo-pg --dbname database2

If you list the databases again, the new one will appear:

database2
defaultdb

Since this is a PostgreSQL database, we could get the connection details from avn
and use them to connect to the
database:

avn service get YOUR-NAME-demo-pg --json | jq .service_uri_params

But it's probably easier to use the built-in helper:

avn service cli YOUR-NAME-demo-pg

This will run the appropriate CLI for the service - for our PostgreSQL service, that means psql.

Finally for this section, I like to know the backup status of services I'm
using. I can get a list of when backups were made and what region they're
stored in with:

avn service backup-list YOUR-NAME-demo-pg

Backup handling differs from service to service and from plan to plan. See Backups at Aiven for an overview, and PostgreSQL® backups if you just want the PostgreSQL specifics.

Change plan, cloud and region

We created our PostgreSQL service using the Hobbyist plan. That's fine for
trying out a service, but it's definitely not suitable for production
use.

The avn service update command allows us to change the details of our service.

To update to a Startup-8 plan running on AWS in Ireland (aws-eu-west-1):

avn service update YOUR-NAME-demo-pg --plan startup-8 --cloud aws-eu-west-1

This will take a little while, because it has to start a new service with a
copy of the original data. How long it takes will depend on the amount of
data, and how busy the systems are. As before, you can use avn service get
or avn service wait to tell when it has finished. See the documentation for
Migrate
services and
Scale your
service. (Aiven takes care that there is no service interruption during a change like
this).

Once the service is RUNNING again, check the new plan and location:

avn service get YOUR-NAME-demo-pg

Keep things tidy

If you're not going to be using a service for a while, then powering it off
will stop you being charged for it:

avn service update YOUR-NAME-demo-pg --power-off

The command won't return until the service is powered off.

Some services that have "event-like" state, such as Apache Kafka® and Apache Flink®, will lose that state when they are powered off.

You can power the service back on again by using the update command once more:

avn service update YOUR-NAME-demo-pg --power-on

Remember it will take a while before the service returns to the RUNNING state.

If you no longer need the service, then delete it:

avn service terminate YOUR-NAME-demo-pg

Since this will delete the service and all associated data, it prompts you to
type in the service name to confirm the operation.