{"id":2083,"date":"2021-01-06T07:26:03","date_gmt":"2021-01-06T05:26:03","guid":{"rendered":"https:\/\/upcloud.com\/global\/us\/resources\/tutorials\/mount-object-storage-cloud-server-s3fs-fuse\/"},"modified":"2021-01-06T07:26:03","modified_gmt":"2021-01-06T05:26:03","slug":"mount-object-storage-cloud-server-s3fs-fuse","status":"publish","type":"tutorial","link":"https:\/\/upcloud.com\/global\/resources\/tutorials\/mount-object-storage-cloud-server-s3fs-fuse\/","title":{"rendered":"How to mount Object Storage on Cloud Server using s3fs-fuse"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">UpCloud Object Storage offers an easy-to-use file manager straight from the control panel. There are also a number of S3-compliant third-party file manager clients that provide a graphical user interface for accessing your Object Storage. However, using a GUI isn\u2019t always an option, for example when accessing Object Storage files from a headless Linux Cloud Server. This is where s3fs-fuse comes in.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/github.com\/s3fs-fuse\/s3fs-fuse\" target=\"_blank\" rel=\"noopener\">s3fs-fuse<\/a> is a popular open-source command-line client for managing object storage files quickly and easily. It is frequently updated and has a large community of contributors on GitHub. In this guide, we will show you how to mount an UpCloud Object Storage bucket on your Linux Cloud Server and access the files as if they were stored locally on the server.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/signup.upcloud.com\/\">Test hosting on UpCloud!<\/a><\/div>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Installing and configuring s3fs<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">To get started, you\u2019ll need to have an existing Object Storage bucket. If you do not have one yet, we have a guide describing <a href=\"https:\/\/upcloud.com\/global\/resources\/tutorials\/object-storage\">how to get started with UpCloud Object Storage<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">After logging into your server, the first thing you will need to do is install s3fs using one of the commands below depending on your OS:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"># Ubuntu and Debian\nsudo apt install s3fs\n\n# CentOS\nsudo yum install epel-release\nsudo yum install s3fs-fuse<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Once the installation is complete, you\u2019ll next need to create a global credential file to store the S3 Access and Secret keys. These would have been presented to you when you created the Object Storage.<br><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-19364\" src=\"https:\/\/upcloud.com\/media\/s3-object-storage-keys.png\" alt=\"S3 Object Storage keys\" width=\"640\" height=\"555\" \/><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you did not save the keys at the time when you created the Object Storage, you can regenerate them by clicking the <em>Settings<\/em> button at your Object Storage details.<br><img loading=\"lazy\" decoding=\"async\" width=\"640\" height=\"153\" class=\"alignnone size-large wp-image-19366\" src=\"https:\/\/upcloud.com\/media\/s3-object-storage-settings-1024x245.png\" alt=\"S3 Object Storage Settings\" \/><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Then scrolling down to the bottom of the Settings page where you\u2019ll find the <em>Regenerate<\/em> button. After new Access and Secret keys have been generated, download the key file and store it somewhere safe.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/upcloud.com\/media\/s3-object-storage-regenerate-keys-1024x378.png\" alt=\"S3 Object Storage access key settings\" class=\"wp-image-19367\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Next, on your Cloud Server, enter the following command to generate the global credential file. Be sure to replace <span style=\"color: #ff0000;\">ACCESS_KEY<\/span> and <span style=\"color: #ff0000;\">SECRET_KEY<\/span> with the actual keys for your Object Storage:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">echo \"<span style=\"color: #ff0000;\">ACCESS_KEY<\/span>:<span style=\"color: #ff0000;\">SECRET_KEY<\/span>\" | sudo tee \/etc\/passwd-s3fs\n<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Then use <tt>chmod<\/tt> to set the necessary permissions to secure the file. 600 ensures that only the root will be able to read and write to the file. If this step is skipped, you will be unable to mount the Object Storage bucket:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">sudo chmod 600 \/etc\/passwd-s3fs\n<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">With the global credential file in place, the next step is to choose a mount point. This is the directory on your server where the Object Storage bucket will be mounted. It can be any empty directory on your server, but for the purpose of this guide, we will be creating a new directory specifically for this.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">sudo mkdir \/mnt\/my-object-storage<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Any files will then be made available under the directory <tt>\/mnt\/my-object-storage\/<\/tt>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Mounting your Object Storage bucket<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">We\u2019re now ready to mount the bucket using the format below. Please note that this is not the actual command that you need to execute on your server. You must first replace the parts highlighted in <span style=\"color: #ff0000;\">red<\/span> with your Object Storage details:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">sudo s3fs <span style=\"color: #ff0000;\">{bucketname}<\/span> <span style=\"color: #ff0000;\">{\/mountpoint\/dir\/}<\/span> -o passwd_file=\/etc\/passwd-s3fs -o allow_other -o url=https:\/\/<span style=\"color: #ff0000;\">{private-network-endpoint}<\/span>\n<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong><span style=\"color: #ff0000;\">{bucketname}<\/span><\/strong> is the name of the bucket that you wish to mount.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong><span style=\"color: #ff0000;\">{\/mountpoint\/dir\/}<\/span><\/strong> is the empty directory on your server where you plan to mount the bucket (it must already exist).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\/etc\/passwd-s3fs<\/strong> is the location of the global credential file that you created earlier. If you created it elsewhere you will need to specify the file location here.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>-o allow_other<\/strong> allows non-root users to access the mount. Otherwise, only the root user will have access to the mounted bucket.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>-o url<\/strong> specifies the private network endpoint for the Object Storage. This can be found by clicking the <em>S3 API access<\/em> link. Don\u2019t forget to prefix the private network endpoint with <strong>https:\/\/<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/upcloud.com\/media\/s3-api-access-2-1024x596.png\" alt=\"S3 API access\" class=\"wp-image-19408\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The private network endpoint allows access to Object Storage via the utility network. This avoids the use of your transfer quota for internal queries since all utility network traffic is free of charge. However, note that Cloud Servers can only access the internal Object Storage endpoints located within the same data centre.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Using all of the information above, the actual command to mount an Object Storage bucket would look something like this:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">sudo s3fs my-bucket \/mnt\/my-object-storage -o passwd_file=\/etc\/passwd-s3fs -o allow_other -o url=https:\/\/my-object-storage-internal.sg-sin1.upcloudobjects.com\n<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">You can now navigate to the mount directory and create a dummy text file to confirm that the mount was successful.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">touch \/mnt\/my-object-storage\/testing.txt\n<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">If all went well, you should be able to see the dummy text file in your UpCloud Control Panel under the mounted Object Storage bucked.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/upcloud.com\/media\/s3-object-storage-bucket-files-1024x430.png\" alt=\"S3 Object Storage bucket files\" class=\"wp-image-19386\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Likewise, any files uploaded to the bucket via the Object Storage page in the control panel will appear in the mount point inside your server.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/upcloud.com\/media\/s3-file-upload-from-hub-1024x457.png\" alt=\"File added to S3 bucket from control panel\" class=\"wp-image-19390\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">If you then check the directory on your Cloud Server, you should see both files as they appear in your Object Storage.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/upcloud.com\/media\/Screenshot-2021-01-06-at-6.16.51-PM-1024x272.png\" alt=\"List of files on S3 mount point\" class=\"wp-image-19391\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Mounting your Object Storage bucket automatically on boot<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">To detach the Object Storage from your Cloud Server, unmount the bucket by using the umount command like below:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">umount \/mnt\/my-object-storage\n<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">You can confirm that the bucket has been unmounted by navigating back to the mount directory and verifying that it is now empty.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Unmounting also happens every time the server is restarted. After every reboot, you will need to mount the bucket again before being able to access it via the mount point.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">However, it is possible to configure your server to mount the bucket automatically at boot. You can do so by adding the s3fs mount command to your <tt>\/etc\/fstab<\/tt> file. For the command used earlier, the line in fstab would look like this:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">s3fs#my-bucket \/mnt\/my-object-storage fuse _netdev,allow_other,passwd_file=\/etc\/passwd-s3fs,url=https:\/\/my-object-storage-internal.sg-sin1.upcloudobjects.com\/ 0 0\n<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">If you then reboot the server to test, you should see the Object Storage get mounted automatically.<\/p>\n","protected":false},"author":23,"featured_media":16932,"comment_status":"open","ping_status":"closed","template":"","community-category":[223,268,232],"class_list":["post-2083","tutorial","type-tutorial","status-publish","has-post-thumbnail","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/tutorial\/2083","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/tutorial"}],"about":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/types\/tutorial"}],"author":[{"embeddable":true,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/users\/23"}],"replies":[{"embeddable":true,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/comments?post=2083"}],"version-history":[{"count":0,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/tutorial\/2083\/revisions"}],"wp:attachment":[{"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/media?parent=2083"}],"wp:term":[{"taxonomy":"community-category","embeddable":true,"href":"https:\/\/upcloud.com\/global\/wp-json\/wp\/v2\/community-category?post=2083"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}