How to automate Terraform deployments

Maintaining cloud infrastructure of any substantial size can often require you to perform the same actions or processes over and over. While some might enjoy the routine, there is however another way. Automating repetitive and reactionary tasks is the perfect solution to saving time and guaranteeing results and what better way to manage this than using Terraform!

In this guide, we’ll look into what you need to consider when automating infrastructure deployments using Terraform. If you haven’t yet set up Terraform with the UpCloud module on your computer, head over to our starters guide first to get going.

Creating a new configuration

Automating tasks like Terraform deployments is a great way, for example, to create reactive scalability for a load-balanced web site. If you expect to be deploying similar setups on multiple occasions, it’ll be worth templating the configuration from the start.

First, make a new directory for the template configuration.

mkdir -p ~/terraform/template && cd ~/terraform/template

Next, let’s create the template itself. Open a new file in your text editor, for example, using nano as below.

nano template.tf

Then add a following basic config as a starting point. The example here already uses variables for most fields, we’ll fill these in later.

provider "upcloud" {
  # export UPCLOUD_USERNAME="Username for Upcloud API user"
  # export UPCLOUD_PASSWORD="Password for Upcloud API user"
}

resource "upcloud_server" "template" {
  hostname = var.hostname
  zone     = var.zones[var.zone]
  plan     = var.plans[var.plan]

  network_interface {
    type = "public"
  }
  template {
    size    = lookup(var.storages, var.plans[var.plan])
    storage = var.templates[var.template]
  }
  
  login {
    user = "root"
    keys = [
      var.public_key,
    ]
  }
  connection {
    host        = self.network_interface[0].ip_address
    type        = "ssh"
    user        = "root"
    private_key = file(var.private_key_path)
  }
}

Make sure to include your SSH keys, the public key in the login section, and the private key file path in the connection. These are needed for automation in order to avoid password authentication.

With a simple Terraform template created, you should initialize the working directory so that we can make a test deployment further down the line.

Initializing a new Terraform directory generally does not need input but it accepts the following parameter for non-attended use.

terraform init -input=false

Initializing provider plugins...

Terraform has been successfully initialized!

Now that Terraform has been initialized with the required plugin, continue below on how to use variables to automate Terraform deployments.

Customising deployment with variables

Terraform variables are very useful for reusing the same configuration file with different outcomes. With Terraform, you can include variables straight from the command line or add them in the configuration files. If you’re not entirely up to speed on Terraform variables and how to use them, learn more at the guide to Terraform variables.

Below is an example of setting a variable on the command line during planning.

terraform plan -var="variable=value"

The above is handy for setting a couple of variables but not in large numbers. To set multiple variables at the time, it is more convenient to specify their values in a variable definitions file. Terraform automatically loads any filename ending in either .tfvars or .tfvars.json within the same working directory but the variable file can also be included at the command line with -var-file parameter.

terraform plan -var-file="variables.tfvars"

Now to give Terraform some variables to work with, let’s create the two files as shown below.

touch config-options.tf default.tfvars

Then add the following variables to the config-options.tf file.

variable "private_key_path" {
  type = string
  default = "/home/user/.ssh/terraform_rsa"
}

variable "public_key" {
  type = string
  default = "ssh-rsa terraform_public_key"
}

variable plans {
  type = map
  default = {
    "5USD"  = "1xCPU-1GB"
    "10USD" = "1xCPU-2GB"
    "20USD" = "2xCPU-4GB"
  }
}

variable storages {
  type = map
  default = {
    "1xCPU-1GB" = "25"
    "1xCPU-2GB" = "50"
    "2xCPU-4GB" = "80"
  }
}

variable zones {
  type = map
  default = {
    "amsterdam" = "nl-ams1"
    "london"    = "uk-lon1"
    "frankfurt" = "de-fra1"
    "helsinki1" = "fi-hel1"
    "helsinki2" = "fi-hel2"
    "chicago"   = "us-chi1"
    "singapore" = "sg-sin1"
  }
}

variable templates {
  type = map
  default = {
    "ubuntu" = "01000000-0000-4000-8000-000030080200"
    "centos" = "01000000-0000-4000-8000-000050010300"
    "debian" = "01000000-0000-4000-8000-000020040100"
  }
}

variable hostname { default = "template" }
variable plan { default = "5USD" }
variable zone { default = "amsterdam" }
variable template { default = "debian" }

Afterwards, add the example values in the example.tfvars file.

"hostname" = "example"
"plan"     = "5USD"
"zone"     = "frankfurt"
"template" = "ubuntu"

Once you’ve set up the variable files, test the template by running the plan command as below.

terraform plan -var-file=example.tfvars

Setting up variables this way allows you to configure your Terraform deployment quickly and easily without needing to edit the files.

The above examples for setting variables can be used together in any combination. If the same variable is assigned multiple values, Terraform uses the last input, overriding any previous values.

Terraform loads variables in a predefined order. Later sources take precedence over earlier ones.

• Environment variables
• The terraform.tfvars file, if present.
• The terraform.tfvars.json file, if present.
• Any *.auto.tfvars or *.auto.tfvars.json files, processed in lexical order of their filenames.
• Any -var and -var-file options on the command line, in the order they are provided.

Next up, let’s have a look at how to test the template deployment.

Plan, apply, destroy

Planning and deploying Terraform configurations is pretty simple and only requires a couple of extra steps to automate the process.

Continuing our examples above where we set up a template plan with configurable variables. The following plan command would then be used to test out our configuration.

terraform plan -input=false -var-file=example.tfvars -var=hostname=test -out=tfplan

This command includes a number of command-line parameters:

• -input=false disables Terraform from asking to input variables if not directly set
• -var-file=defaults.tfvars includes your variables file
• -var=hostname=test set a hostname for the new server
• -out=tfplan saves the plan to a local file called tfplan for applying

Next, granted that everything in the plan looks correct, we can deploy the new cloud server with the Terraform’s apply command.

Since we already saved the plan at the previous step, the output file can now be used to apply the configuration. Using a saved plan we can be sure of the actions Terraform is going to take before committing the changes.

terraform apply tfplan -auto-approve

The only additional parameter comes with the need to skip the manual approval check by adding -auto-approve to automatically answer yes.

Lastly, when you no longer need a particular cloud resource, remove the server either using the Terraform destroy command or planning for destroying.

terraform destroy -auto-approve

terraform plan -destroy -out=tfdestroy
terraform apply tfdestroy -auto-approve

That’s just how simple Terraform commands are to automate!

Note that the destroy command will delete all resources configured in that Terraform directory. If you want to remove only a part of your cloud resources defined by the same Terraform deployment, remove the resource from the configuration and use the apply command to update your infrastructure. Terraform will then figure out the differences to the live deployment and apply the necessary changes.

Automating Terraform

We’ve now got the required basics down and have everything we need to automate a deployment using Terraform.

One of the advantages of cloud servers is how quickly they can be deployed and deleted. If you only need to run something for a short amount of time, deploying a new server is not just a clean starting point but also cost-effective.

Let’s test this in practice by scripting an automated benchmark for the cloud server.

Open your favourite text editor, for example, nano with the command below.

nano ~/benchmark.sh

Then add the following to the script.

#!/bin/bash

# Create a new Terraform configuration from the template
mkdir ~/terraform/geekbench && cd ~/terraform/geekbench

# Copy only the plan and variable files
cp ~/terraform/template/*.tf ~/terraform/geekbench/
cp ~/terraform/template/*.tfvars ~/terraform/geekbench/

# Renaming the resource
sed -i 's/"template"/"geekbench"/g' ~/terraform/geekbench/template.tf

# Opening the last set of brackets in the resource
sed -i '$ s/^}$//g' ~/terraform/geekbench/template.tf

# Adding a remote execution script to the configuration
cat <> ~/terraform/geekbench/template.tf
  provisioner "remote-exec" {
    inline = [
      "curl http://cdn.geekbench.com/Geekbench-4.4.1-Linux.tar.gz -o ~/geekbench.tar.gz",
      "tar -zxf ~/geekbench.tar.gz",
      "~/Geekbench-4.4.1-Linux/geekbench4 | grep 'https://browser.geekbench.com'",
    ]
  }
}
EOT

# Initialise the configuration
terraform init -input=false
# Plan and deploy
terraform plan -input=false -var=hostname=geekbench -var=plan=5USD -out=tfplan
terraform apply tfplan

# Terraform will run the above script to benchmark the server
# Find your results in the Terraform output

# Once finished, destroy the server
terraform plan -destroy -out=tfdestroy
terraform apply tfdestroy

# Cleaning up
rm -r ~/terraform/geekbench

When done, save the file and exit the editor.

Make the script executable

sudo chmod +x ~/benchmark.sh

Then test the benchmark script.

~/benchmark.sh

You can follow the deployment as Terraform outputs information about various parts of the process. Once deployed, Terraform will download and run the Geekbench script. The benchmarking might take a couple of minutes, you’ll see an output similar to the one below with the URL to your results.

upcloud_server.geekbench: Still creating... (5m50s elapsed)
upcloud_server.geekbench (remote-exec):   https://browser.geekbench.com/v4/cpu/14430944
upcloud_server.geekbench (remote-exec):   https://browser.geekbench.com/v4/cpu/14430944/claim?key=432214
upcloud_server.geekbench: Creation complete after 5m57s (ID: 00249d8a-89e1-4c29-833c-93e8ffd433ae)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Afterwards, Terraform will destroy the server and clean up the configuration files without ever touching the original template.

If you want to try benchmarking servers with more resources, simply edit the planning command of your script. For example, select the 1xCPU 2GB RAM plan like shown in the example underneath.

terraform plan -input=false -var=hostname=geekbench -var=plan=10USD -out=tfplan

Then run the script again as before.

Conclusions

Automating deployments with Terraform is a great way of creating reliable and easy to customise configurations for many use cases. The example in this guide is but a scratch on the surface of what is possible with Terraform automation. For ideas on other uses, have a look at the Terraform documentation about different types of provisioners.