{"id":2056,"date":"2021-06-18T15:10:35","date_gmt":"2021-06-18T12:10:35","guid":{"rendered":"https:\/\/upcloud.com\/global\/us\/resources\/tutorials\/upcloud-object-storage-terraform\/"},"modified":"2021-06-18T15:10:35","modified_gmt":"2021-06-18T12:10:35","slug":"upcloud-object-storage-terraform","status":"publish","type":"tutorial","link":"https:\/\/upcloud.com\/global\/resources\/tutorials\/upcloud-object-storage-terraform\/","title":{"rendered":"How to deploy UpCloud Object Storage using Terraform"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/upcloud.com\/global\/blog\/upcloud-verified-terraform-provider\/\" target=\"_blank\" rel=\"noopener\">Terraform<\/a> is a simple yet powerful open-source infrastructure management tool developed by HashiCorp. Terraform allows you to safely and predictably manage your cloud infrastructure from Cloud Servers to <a href=\"https:\/\/upcloud.com\/global\/products\/object-storage\/\" target=\"_blank\" rel=\"noopener\">Object Storage<\/a> by codifying APIs into declarative configuration files.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img decoding=\"async\" src=\"https:\/\/upcloud.com\/media\/terraform_logo.png\" alt=\"Terraform logo\" class=\"wp-image-6158\" \/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">Building your cloud setup around Terraform is a great way to configure something once and be able to deploy it again and again with no effort at all. In this guide, we will show how to deploy UpCloud Object Storage using Terraform.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/signup.upcloud.com\/\">Test Terraform on UpCloud!<\/a><\/div>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Setting up new deployment<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Each Terraform deployment lives in its own directory. To get started, create a new directory for your Terraform project and move into the new directory.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">mkdir -p ~\/terraform\/object-storage &amp;&amp; cd ~\/terraform\/object-storage<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Deploying onto UpCloud using Terraform works using our <a href=\"https:\/\/upcloud.com\/global\/resources\/tools\/upcloud-terraform-provider\">verified provider module<\/a>. Using the UpCloud Terraform provider is as simple as declaring the required providers in a configuration file. A standard name for this file is <tt>version.tf<\/tt>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Create the file with the below command then open it for editing with your preferred text editor.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">touch version.tf<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Then, add the following to the file.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">terraform {\n  required_providers {\n    upcloud = {\n      source = \"UpCloudLtd\/upcloud\"\n    }\n  }\n}\n\nprovider \"upcloud\" {\n  # Your UpCloud credentials are read from the environment variables\n  # export UPCLOUD_USERNAME=\"Username for Upcloud API user\"\n  # export UPCLOUD_PASSWORD=\"Password for Upcloud API user\"\n  # Optional configuration settings can be depclared here\n}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Afterwards, save the file and exit the editor. With the required module declaration in place, you can continue with the next step.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Next, initialise the new Terraform configuration directory by running the command underneath.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">terraform init<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">You should then see an output similar to the example below.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">Initializing the backend...\n\nInitializing provider plugins...\n- Finding latest version of upcloudltd\/upcloud...\n- Installing upcloudltd\/upcloud v2.1.0...\n- Installed upcloudltd\/upcloud v2.1.0 (self-signed, key ID 60B4E1988F222907)\n\nPartner and community providers are signed by their developers.\nIf you'd like to know more about provider signing, you can read about it here:\nhttps:\/\/www.terraform.io\/docs\/cli\/plugins\/signing.html\n\nTerraform has created a lock file .terraform.lock.hcl to record the provider\nselections it made above. Include this file in your version control repository\nso that Terraform can guarantee to make the same selections by default when\nyou run \"terraform init\" in the future.\n\n<span style=\"color: #33da7a;\"><strong>Terraform has been successfully initialized!<\/strong><\/span>\n\n<span style=\"color: #26a269;\">You may now begin working with Terraform. Try running \"terraform plan\" to see\nany changes that are required for your infrastructure. All Terraform commands\nshould now work.\n\nIf you ever set or change modules or backend configuration for Terraform,\nrerun this command to reinitialize your working directory. If you forget, other\ncommands will detect it and remind you to do so if necessary.\n<\/span><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Also, as shown in the provider section of the <tt>version.tf<\/tt> file, to allow Terraform to access your UpCloud account, you need to set an account name and password in your environmental variables. If you haven\u2019t done this already, check out our <a href=\"https:\/\/upcloud.com\/global\/resources\/tutorials\/get-started-terraform\">Terraform intro tutorial<\/a> to set your API credentials before continuing with the rest of this tutorial.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Your Terraform configuration directory should then be ready. Continue in the next section to create your Object Storage configuration.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Configuring Object Storage<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">With Terraform, you can easily create Object Storage instances by simply declaring the resource in your Terraform config files. For this guide, we\u2019ll configure an Object Storage instance with the starter storage size and some examples for buckets.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Create a new Terraform configuration file, e.g. <tt>object-storage.tf<\/tt> with the following command.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">touch object-storage.tf<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Then open the file for edit and add the following configuration to the file. Remember to set your access and secret keys highlighted in red.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">resource \"upcloud_object_storage\" \"my_object_storage\" {\n  # Allocate 250GB for this Object Storage\n  size  = 250\n\n  # Instance name which forms part of the URL used to access the storage\n  name = \"storage-name\"\n\n  # Location in which the Object Storage is deployed\n  zone  = \"nl-ams1\"\n\n  # The access and secret keys used to access the Object Storage instance\n  access_key = \"<span style=\"color: #ff0000;\">awodhouafsefrfhj<\/span>\"\n  secret_key = \"<span style=\"color: #ff0000;\">3piafn895a9fjpi5<\/span>\"\n  \n  description = \"catalogue\"\n\n  # Create a bucket called \"images\"\n  bucket {\n    name = \"images\"\n  }\n\n  # Create a bucket called \"products\"\n  bucket {\n    name = \"products\"\n  }\n}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The configuration options include the following:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Storage size in GB \u2013 can only be increased<\/li>\n\n\n\n<li>The name of the storage \u2013 change will cause a replacement<\/li>\n\n\n\n<li>Location the Object Storage is deployed to \u2013 change will cause replacement<\/li>\n\n\n\n<li>Access and secret keys for authentication<\/li>\n\n\n\n<li>Description of the storage<\/li>\n\n\n\n<li>Optional bucket definitions<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Once your Object Storage configuration is all set, run the usual check and deploy commands.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">terraform plan<\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\">terraform apply<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">You\u2019ll then see an output similar to the example below.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:\n  <span style=\"color: #26a269;\">+<\/span> create\n\nTerraform will perform the following actions:\n\n  <strong># upcloud_object_storage.my_object_storage<\/strong> will be created\n  <span style=\"color: #26a269;\">+<\/span> resource \"upcloud_object_storage\" \"my_object_storage\" {\n      <span style=\"color: #26a269;\">+<\/span> access_key  = \"awodhouafsefrfhj\"\n      <span style=\"color: #26a269;\">+<\/span> created     = (known after apply)\n      <span style=\"color: #26a269;\">+<\/span> description = \"catalogue\"\n      <span style=\"color: #26a269;\">+<\/span> id          = (known after apply)\n      <span style=\"color: #26a269;\">+<\/span> name        = \"object-storage-terraform\"\n      <span style=\"color: #26a269;\">+<\/span> secret_key  = \"3piafn895a9fjpi5\"\n      <span style=\"color: #26a269;\">+<\/span> size        = 250\n      <span style=\"color: #26a269;\">+<\/span> state       = (known after apply)\n      <span style=\"color: #26a269;\">+<\/span> url         = (known after apply)\n      <span style=\"color: #26a269;\">+<\/span> used_space  = (known after apply)\n      <span style=\"color: #26a269;\">+<\/span> zone        = \"uk-lon1\"\n\n      <span style=\"color: #26a269;\">+<\/span> bucket {\n          <span style=\"color: #26a269;\">+<\/span> name = \"images\"\n        }\n      <span style=\"color: #26a269;\">+<\/span> bucket {\n          <span style=\"color: #26a269;\">+<\/span> name = \"products\"\n        }\n    }\n\n<strong>Plan:<\/strong> 1 to add, 0 to change, 0 to destroy.\n<strong>upcloud_object_storage.my_object_storage: Creating...\nupcloud_object_storage.my_object_storage: Still creating... [10s elapsed]\nupcloud_object_storage.my_object_storage: Still creating... [20s elapsed]\nupcloud_object_storage.my_object_storage: Creation complete after 25s [id=06c73581-a9db-4e8b-ad53-4be1a437054a]<\/strong>\n\n<span style=\"color: #33da7a;\"><strong>Apply complete! Resources: 1 added, 0 changed, 0 destroyed.<\/strong><\/span><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Once deployed, your Object Storage is ready to use!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Making changes to Object Storage<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Terraform also allows you to easily modify any Object Storage deployment by just making changes to the configuration file. Terraform will figure out the differences between the existing and desired state and then deploy the changes to the live services.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Note that not all configuration options can be changed without replacing the Object Storage instance. For example, storage size can only be increased to prevent data loss. Furthermore, changing the Object Storage name or location will require resources to be replaced.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"color: #ff0000;\">Warning<\/span>: operations that force replacement will cause all data in the Object Storage to be deleted.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In the example configuration below, we\u2019ve increased the Object Storage size from 250 GB to 500 GB, updated the access and secret keys, changed the description and removed one of the buckets.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">resource \"upcloud_object_storage\" \"my_object_storage\" {\n  # Increasing storage size\n  size  = <span style=\"color: #ff0000;\">500<\/span>\n\n  # The access and secret keys as well as the description can be updated\n  access_key = \"<span style=\"color: #ff0000;\">iejdyugafsefsfhw<\/span>\"\n  secret_key = \"<span style=\"color: #ff0000;\">wad1e2evgh3olhym<\/span>\"\n  description = \"hosting\"\n\n  # (unchanged attributes hidden)\n  # Buckets can be added and deleted\n  bucket {\n    name = \"images\"\n  }\n}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">When you have made your changes to the configuration, use the plan and apply commands to update your Object Storage.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">terraform plan<\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\">terraform apply<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">You\u2019ll then see an output similar to the example below.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><strong>upcloud_object_storage.my_object_storage: Refreshing state... [id=06a6930f-d5ed-4e65-bdb6-28f73d562f12]<\/strong>\n\nTerraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:\n  ~ update in-place\n\nTerraform will perform the following actions:\n\n  <strong># upcloud_object_storage.my_object_storage<\/strong> will be updated in-place\n  ~ resource \"upcloud_object_storage\" \"my_object_storage\" {\n      ~ access_key  = \"awodhouafsefrfhj\" -&gt; \"iejdyugafsefsfhw\"\n      ~ description = \"catalogue\" -&gt; \"hosting\"\n        id          = \"06a6930f-d5ed-4e65-bdb6-28f73d562f12\"\n      ~ name        = \"obj-terraform-raiou\" -&gt; \"storage-name\"\n      ~ secret_key  = \"3piafn895a9fjpi5\" -&gt; \"wad1e2evgh3olhym\"\n      ~ size        = 250 -&gt; 500\n        # (5 unchanged attributes hidden)\n\n      - bucket {\n          - name = \"products\" -&gt; null\n        }\n        # (1 unchanged block hidden)\n    }\n\n<strong>Plan:<\/strong> 0 to add, 1 to change, 0 to destroy.\n<strong>upcloud_object_storage.my_object_storage: Modifying... [id=06a6930f-d5ed-4e65-bdb6-28f73d562f12]\nupcloud_object_storage.my_object_storage: Modifications complete after 4s [id=06a6930f-d5ed-4e65-bdb6-28f73d562f12]<\/strong>\n\n<span style=\"color: #33da7a;\"><strong>Apply complete! Resources: 0 added, 1 changed, 0 destroyed.<\/strong><\/span><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Afterwards, your Object Storage is up and running again with the new changes in place.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Connecting to Object Storage using S3cmd<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">When you\u2019ve deployed your Object Storage using Terraform, you\u2019ll next probably want to connect to it. S3cmd is a simple yet powerful command-line tool for managing S3-compatible object storage and works great with UpCloud Object Storage.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To make use of S3cmd, you\u2019ll need to install and configure it for your Object Storage instance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">You\u2019ll need to install Python, download and install pip, and then install S3cmd using pip.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"># Ubuntu and Debian\nsudo apt-get install python3 python3-distutils -y\nwget https:\/\/bootstrap.pypa.io\/get-pip.py -O ~\/get-pip.py\nsudo python3 ~\/get-pip.py\nsudo pip install s3cmd<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Next, get the required connection details from the Terraform state file using the following command.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">terraform show<\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"># upcloud_object_storage.my_object_storage:\nresource \"upcloud_object_storage\" \"my_object_storage\" {\n    <span style=\"color: #26a269;\">access_key  = \"iejdyugafsefsfhw\"<\/span>\n    created     = \"2021-06-14T15:10:31Z\"\n    description = \"hosting\"\n    id          = \"068ea905-7120-4927-b9ef-acf34243aa2e\"\n    name        = \"storage-name\"\n    <span style=\"color: #26a269;\">secret_key  = \"wad1e2evgh3olhym\"<\/span>\n    size        = 500\n    state       = \"started\"\n    <span style=\"color: #26a269;\">url         = \"https:\/\/storage-name.nl-ams1.upcloudobjects.com\/\"<\/span>\n    used_space  = 0\n    zone        = \"nl-ams1\"\n\n    bucket {\n        name = \"images\"\n    }\n}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Make note of your Object Storage\u2019s <tt>access_key<\/tt>,&nbsp;<tt>sercret_key<\/tt>, and the endpoint <tt>url<\/tt>&nbsp;which are needed to be able to connect using an S3 client.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Next, use the command below to run the S3cmd configuration script.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">s3cmd --configure<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Then enter the required details in the order they are asked. Set your keys and endpoint URL as shown in the output of the <tt>terraform show<\/tt> command. Also, set a password for file transfer encryption. Other fields can be skipped by simply pressing <em>Enter<\/em>&nbsp;key to use the default value.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Then at the end, confirm to save settings.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">Save settings? [y\/N] y\nConfiguration saved to '\/home\/user\/.s3cfg'<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">If you created a bucket in your Terraform configuration, you can confirm it by listing the buckets in your Object Storage.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">s3cmd ls<\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\">2021-06-16 16:34  s3:\/\/images<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">A response showing your buckets confirms you\u2019ve configured S3cmd correctly and are all set to start uploading files.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you want to learn more about S3cmd, check out our getting started tutorial. It includes example commands on how to get started using your Object Storage straight from your command line.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/upcloud.com\/global\/resources\/tutorials\/s3cmd-object-storage\">How to use S3cmd to manage your Object Storage<\/a><\/p>\n<\/blockquote>\n","protected":false},"author":3,"featured_media":16930,"comment_status":"open","ping_status":"closed","template":"","community-category":[268,232,235],"class_list":["post-2056","tutorial","type-tutorial","status-publish","has-post-thumbnail","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/tutorial\/2056","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/tutorial"}],"about":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/types\/tutorial"}],"author":[{"embeddable":true,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/comments?post=2056"}],"version-history":[{"count":0,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/tutorial\/2056\/revisions"}],"wp:attachment":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/media?parent=2056"}],"wp:term":[{"taxonomy":"community-category","embeddable":true,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/community-category?post=2056"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}