How to deploy Managed Object Storage using Terraform
In this guide, we’ll go through the steps needed to deploy an UpCloud Managed Object Storage service using Terraform.
This guide assumes you are familiar with Terraform and have already set up your UpCloud credentials in your environment variables. Please refer to our getting started with Terraform guide if you need help with the initial setup and configuration.
Set up a working directory
Start by creating a working directory for your Terraform project and move into it.
mkdir -p ~/terraform
cd ~/terraformSet up the UpCloud Terraform configuration block
Within your working directory, create a configuration block to define the required UpCloud settings:
nano versions.tfterraform {
required_providers {
upcloud = {
source = "UpCloudLtd/upcloud"
version = "~> 5.0"
}
}
}Note: To configure bucket policies, CORS rules, or lifecycle rules, see Configure bucket policies, CORS, and lifecycle rules at the end of this guide.
Save the file and exit the editor.
Initialize Terraform in your working directory:
terraform initThis will download the required UpCloud provider and set up your working directory.
Write the Managed Object Storage configuration
Next, write the configuration for the Managed Object Storage by declaring the resources in a Terraform configuration file.
nano managed-object-storage.tfThe configuration will define:
- A Managed Object Storage service
- Buckets within that storage
- A user with access permissions
- Access credentials for the user
- Output blocks for:
- Public endpoint
- Any buckets created
- User credentials
# Define your bucket names
variable "bucket_names" {
type = set(string)
description = "Names of buckets to be created"
default = [
"<BUCKET-NAME-1>", # Replace with your first bucket name
"<BUCKET-NAME-2>", # Replace with your second bucket name
"<BUCKET-NAME-3>" # Replace with your third bucket name
]
}
# Create the managed object storage service
resource "upcloud_managed_object_storage" "terraform_storage" {
name = "<STORAGE-NAME>"
region = "<REGION>"
configured_status = "started"
# Public access
network {
family = "IPv4"
name = "<PUBLIC-NETWORK-NAME>"
type = "public"
}
# Private access (optional)
network {
family = "IPv4"
name = "<PRIVATE-NETWORK-NAME>"
type = "private"
uuid = "<PRIVATE-NETWORK-UUID>" # The private network must already exist
}
# Optional label
labels = {
managed-by = "terraform"
}
}
# Create buckets within the storage service
resource "upcloud_managed_object_storage_bucket" "terraform_bucket" {
for_each = var.bucket_names
service_uuid = upcloud_managed_object_storage.terraform_storage.id
name = each.value
}
# Create a user for the storage service
resource "upcloud_managed_object_storage_user" "user" {
service_uuid = upcloud_managed_object_storage.terraform_storage.id
username = "<USERNAME>"
}
# Create access credentials for the storage user
resource "upcloud_managed_object_storage_user_access_key" "user_key" {
service_uuid = upcloud_managed_object_storage.terraform_storage.id
username = upcloud_managed_object_storage_user.user.username
status = "Active"
}
# Attach access policy to the storage user
resource "upcloud_managed_object_storage_user_policy" "user_policy" {
username = upcloud_managed_object_storage_user.user.username
service_uuid = upcloud_managed_object_storage.terraform_storage.id
name = "<POLICY>"
}
# Output the public access endpoints
output "public_endpoints" {
value = {
endpoint = upcloud_managed_object_storage.terraform_storage.endpoint
}
}
# Output all buckets created
output "all_buckets" {
value = {
for k, v in upcloud_managed_object_storage_bucket.terraform_bucket : k => {
name = v.name
id = v.id
}
}
}
# Output the user access credentials (hidden by default)
output "access_credentials" {
sensitive = true
value = {
access_key_id = upcloud_managed_object_storage_user_access_key.user_key.access_key_id
secret_access_key = upcloud_managed_object_storage_user_access_key.user_key.secret_access_key
}
}Before saving the configuration above, replace the placeholders with your own values:
<STORAGE-NAME>- Name for your Managed Object Storage service<PUBLIC-NETWORK-NAME>- Name for your public network<BUCKET-NAME-1>to<BUCKET-NAME-3>- Names for your buckets<USERNAME>- Username for your user
For the <REGION>, choose one of:
apac-1europe-1europe-2europe-3europe-4us-1
For more details about region locations and their SDN accessibility, see our availability page.
For <POLICY>, choose from one of:
ECSS3FullAccess- Full access to all bucketsECSS3ReadOnlyAccess- Read-only access to all bucketsIAMFullAccess- Full IAM managementIAMReadOnlyAccess- Read-only IAM accessECSDenyAll- Denies all access
You can also create custom policies with specific permissions - see the Creating and attaching a custom user policy section later in this guide.
For private network access (optional):
<PRIVATE-NETWORK-NAME>- Name for your private network<PRIVATE-NETWORK-UUID>- UUID of your existing private network. It must be in the same availability zone as the Managed Object Storage region. To deploy with a new private network instead, see the section of the guide titled Deploying with a new private network.
To deploy without private network access, remove the entire "Private access" network block from the configuration.
(Optional) Deploying with a new private network
In the previous section we saw how to deploy a managed object storage attached to an existing private network.
To deploy a new Managed Object Storage service with a new (non existing) private network, first create a new router and new network, then attach the network to your new object storage service. Here's the modified configuration for that:
nano managed-object-storage.tf# Create router for the private network
resource "upcloud_router" "storage_router" {
name = "<ROUTER-NAME>"
}
# Create private network
resource "upcloud_network" "storage_network" {
name = "<PRIVATE-NETWORK-NAME>"
zone = "<ZONE>"
ip_network {
address = "<NETWORK-ADDRESS>"
dhcp = true
family = "IPv4"
}
router = upcloud_router.storage_router.id
}
# Create the managed object storage service
resource "upcloud_managed_object_storage" "terraform_storage" {
name = "<STORAGE-NAME>"
region = "<REGION>"
configured_status = "started"
# Public access
network {
family = "IPv4"
name = "<PUBLIC-NETWORK-NAME>"
type = "public"
}
# Private access
network {
family = "IPv4"
name = "<PRIVATE-NETWORK-NAME>"
type = "private"
uuid = upcloud_network.storage_network.id
}
}
# Add the same bucket, user, policy, and output configurations as shown in the previous sectionReplace the placeholders with your own values:
<ROUTER-NAME>- Name for your new router<PRIVATE-NETWORK-NAME>- Name for your private network<NETWORK-ADDRESS>- Private network address range (e.g., "172.16.2.0/24")
For <ZONE>, use one of the zones that can access your Object Storage region. The availability page lists the zones for each region.
The rest of the placeholders remain the same as in the previous configuration.
Plan and apply the infrastructure
Run Terraform plan to preview the changes:
terraform planYou'll see an output showing what will be deployed. If you're happy with it, run the apply command:
terraform applyType yes when asked to confirm the deployment. This will begin deploying the resources described in the configuration file on your UpCloud account.
The process may take a while, but when it completes, Terraform will display:
- The storage endpoint
- Access credentials for your user (marked as sensitive)
You can view the sensitive credentials by running the following output command:
terraform output access_credentialsTo see your storage endpoint, run:
terraform output public_endpointsOnce you have your storage service running and credentials ready, you might want to test it out. Check out our guide on Connecting to an UpCloud Object Storage instance using S3cmd to learn how to upload and manage files in your new storage service using the S3cmd command-line tool.
Modifying the infrastructure
Add additional buckets
To add more buckets, simply add new bucket names to the bucket_names variable in your configuration:
# Define your bucket names
variable "bucket_names" {
type = set(string)
description = "Names of buckets to be created"
default = [
"<BUCKET-NAME-1>", # Replace with your first bucket name
"<BUCKET-NAME-2>", # Replace with your second bucket name
"<BUCKET-NAME-3>", # Replace with your third bucket name
"<BUCKET-NAME-4>" # Add more bucket names as needed
]
}Add more users
Add new users and their access credentials:
# Create a second user
resource "upcloud_managed_object_storage_user" "second_user" {
service_uuid = upcloud_managed_object_storage.terraform_storage.id
username = "<USERNAME-2>"
}
# Create access credentials for the second user
resource "upcloud_managed_object_storage_user_access_key" "second_user_key" {
service_uuid = upcloud_managed_object_storage.terraform_storage.id
username = upcloud_managed_object_storage_user.second_user.username
status = "Active"
}
# Attach access policy to the second user
resource "upcloud_managed_object_storage_user_policy" "second_user_policy" {
username = upcloud_managed_object_storage_user.second_user.username
service_uuid = upcloud_managed_object_storage.terraform_storage.id
name = "<POLICY>"
}After adding these resources, run terraform plan followed by terraform apply to deploy the additional resources.
Creating and attaching a custom user policy
To use a custom policy with specific permissions instead of the predefined ones like ECSS3FullAccess, add the following custom policy to your existing configuration:
# Create a custom policy
resource "upcloud_managed_object_storage_policy" "custom_policy" {
name = "<CUSTOM-POLICY-NAME>"
description = "<CUSTOM-POLICY-DESCRIPTION>"
service_uuid = upcloud_managed_object_storage.terraform_storage.id
document = urlencode(file("<CUSTOM-POLICY-DOCUMENT>")) # path to json file
}Then, update your policy attachment block to use the custom policy:
# Attach access policy to the storage user
resource "upcloud_managed_object_storage_user_policy" "user_policy" {
username = upcloud_managed_object_storage_user.user.username
service_uuid = upcloud_managed_object_storage.terraform_storage.id
name = "<CUSTOM-POLICY-NAME>"
}Replace these placeholders:
<CUSTOM-POLICY-NAME>- Name for your custom policy<CUSTOM-POLICY-DESCRIPTION>- Brief description of what the custom policy does<CUSTOM-POLICY-DOCUMENT>- The path to your custom JSON policy document
Remove resources
To remove resources, delete their configuration blocks from the file and run terraform plan followed by terraform apply. Terraform will identify the resources to be destroyed and remove them when you confirm the apply.
For example, to remove a bucket, modify the bucket_names variable by removing or commenting out the bucket name:
# Define your bucket names
variable "bucket_names" {
type = set(string)
description = "Names of buckets to be created"
default = [
"<BUCKET-NAME-1>",
"<BUCKET-NAME-2>",
#"<BUCKET-NAME-3>" # REMOVE OR COMMENT THIS LINE TO DELETE THIS BUCKET
]
}To remove a user, you'll need to delete all three associated resource blocks: the user configuration, access key, and policy attachment. For example:
# Delete all these blocks to remove the user
resource "upcloud_managed_object_storage_user" "user" {
service_uuid = upcloud_managed_object_storage.terraform_storage.id
username = "<USERNAME>"
}
resource "upcloud_managed_object_storage_user_access_key" "user_key" {
service_uuid = upcloud_managed_object_storage.terraform_storage.id
username = upcloud_managed_object_storage_user.user.username
status = "Active"
}
resource "upcloud_managed_object_storage_user_policy" "user_policy" {
username = upcloud_managed_object_storage_user.user.username
service_uuid = upcloud_managed_object_storage.terraform_storage.id
name = "<POLICY>"
}Note: The upcloud_managed_object_storage_bucket resource uses the UpCloud API to manage Managed Object Storage buckets. The main difference to S3 API is that the buckets can be deleted even when the bucket contains objects.
The plan will show which resources will be destroyed. Review it carefully before confirming the apply.
If you want to remove all resources and completely tear down the infrastructure, you can use:
terraform destroyThis command will remove all resources defined in your Terraform configuration. Use it with caution as it will delete everything, including all buckets, users, and the storage service itself. Terraform will show you a plan of what will be destroyed and ask for confirmation before proceeding.
Configure bucket policies, CORS, and lifecycle rules
The upcloud provider manages the Managed Object Storage service and its buckets, users, and access keys. To configure the settings of a bucket, such as its bucket policy, CORS rules, and lifecycle rules, use the objsto provider. UpCloud develops the objsto provider for managing S3-compatible object storage through the S3 API.
By using both providers (upcloud and objsto) together, you get complete control over your object storage setup.
Each bucket setting has its own resource in the objsto provider:
| Bucket setting | Resource |
|---|---|
| Bucket policy | objsto_bucket_policy |
| CORS rules | objsto_bucket_cors_configuration |
| Lifecycle rules | objsto_bucket_lifecycle_configuration |
| Versioning | objsto_bucket_versioning |
Only bucket policies use a JSON policy document. CORS and lifecycle rules are defined in their own resources, as shown in the sections below.
Update the provider configuration
First, update your versions.tf file to include both providers:
terraform {
required_providers {
upcloud = {
source = "UpCloudLtd/upcloud"
version = "~> 5.0"
}
objsto = {
source = "UpCloudLtd/objsto"
version = "~> 0.4"
}
}
}Configure the objsto provider
Create a new file for the bucket settings:
nano bucket-settings.tfAdd the objsto provider configuration:
provider "objsto" {
endpoint = "https://${[for e in upcloud_managed_object_storage.terraform_storage.endpoint : e.domain_name if e.type == "public"][0]}"
region = upcloud_managed_object_storage.terraform_storage.region
access_key = upcloud_managed_object_storage_user_access_key.user_key.access_key_id
secret_key = upcloud_managed_object_storage_user_access_key.user_key.secret_access_key
}The provider takes the public endpoint, region, and access key from the resources you created earlier in this guide, so you don't need to copy credentials into the file.
Then add the resources for the bucket settings you need to the same file. In each resource, replace <BUCKET-NAME> with the name of one of the buckets in your bucket_names variable.
Add a bucket policy
A bucket policy controls who can access a bucket and its contents. For example, this policy allows public read access to a bucket. Create a file named public-read-policy.json:
{
"Version": "2012-10-17",
"Statement": [
{
"Principal": {
"AWS": ["*"]
},
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::<BUCKET-NAME>"
]
},
{
"Principal": {
"AWS": ["*"]
},
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::<BUCKET-NAME>/*"
]
}
]
}Then attach the policy to the bucket in bucket-settings.tf:
resource "objsto_bucket_policy" "public_read" {
bucket = upcloud_managed_object_storage_bucket.terraform_bucket["<BUCKET-NAME>"].name
policy = file("public-read-policy.json")
}Add CORS rules
Cross-Origin Resource Sharing (CORS) rules let web applications on other domains access the bucket from a browser. This example allows requests from any origin:
resource "objsto_bucket_cors_configuration" "cors" {
bucket = upcloud_managed_object_storage_bucket.terraform_bucket["<BUCKET-NAME>"].name
cors_rule {
allowed_headers = ["*"]
allowed_methods = ["GET", "HEAD", "DELETE", "PUT", "POST"]
allowed_origins = ["*"]
expose_headers = ["x-amz-server-side-encryption"]
max_age_seconds = 3000
}
}A bucket can have only one CORS configuration. To add more rules, add more cors_rule blocks to the same resource. To allow only your own site, replace "*" in allowed_origins with its address, for example "https://www.example.com".
Add lifecycle rules
Lifecycle rules delete objects automatically after a set time. This example turns on versioning for the bucket and adds two rules:
- Expire old versions deletes previous versions of objects 7 days after they're replaced. This rule only has an effect on buckets with versioning turned on.
- Expire completed objects deletes objects tagged
status=completed30 days after they're created.
resource "objsto_bucket_versioning" "versioning" {
bucket = upcloud_managed_object_storage_bucket.terraform_bucket["<BUCKET-NAME>"].name
versioning_configuration {
status = "Enabled"
}
}
resource "objsto_bucket_lifecycle_configuration" "lifecycle" {
bucket = upcloud_managed_object_storage_bucket.terraform_bucket["<BUCKET-NAME>"].name
rule {
id = "Expire old versions"
status = "Enabled"
filter {
prefix = ""
}
noncurrent_version_expiration {
noncurrent_days = 7
}
}
rule {
id = "Expire completed objects"
status = "Enabled"
filter {
tag = {
key = "status"
value = "completed"
}
}
expiration {
days = 30
}
}
}If you don't use versioning, leave out the objsto_bucket_versioning resource and the "Expire old versions" rule. A bucket can have only one lifecycle configuration, so add all its rules to the same resource. For more about versioning and lifecycle rules, see How to enable and manage S3 object versioning and How to configure lifecycle policies.
Apply the configuration
After updating the files, initialize Terraform to install the objsto provider, then apply the changes:
terraform init
terraform plan
terraform applyStoring Terraform state in Managed Object Storage
Once you have a Managed Object Storage instance up and running, you can also use it as a remote backend for your Terraform state file. This is especially useful for shared configurations and CI/CD pipelines, where many operators and machines work on the same state. See How to use Managed Object Storage as an S3 state backend for Terraform for the full configuration.