How to get started with Libcloud compute nodes

Apache Libcloud is a Python programming language library that unifies supported cloud provider APIs under a single interface. The Libcloud compute driver for UpCloud allows you to deploy and manage cloud servers using simple functions. Follow the instructions below to install the Libcloud library and check the usage examples to get started.

Enable API permissions

Before you begin, allowing Libcloud to connect to your UpCloud account requires you to enable the API permissions.

We recommend creating a new workspace member account just for the API communication. Log in to your UpCloud account and go to the People tab. You can check out the instructions on how to do this in our getting started guide for the API.

Note that you should restrict the API account from your other UpCloud services. Take care handling your credentials when programming automation.

Installing prerequisites

To begin developing with Libcloud, you will need to have a Python programming environment configured on your computer. You can find instructions to install Python for your operating system in their beginner’s guide.

Installing Libcloud

Libcloud is available for most Linux distributions with the package managers or directly from the Libcloud downloads page on other operating systems as well.

However, the easiest and most universal way to install Libcloud is using the Python Package Index. It is the official third-party software repository for the Python programming language and contains almost everything you could ever need.

In most cases, the command line client pip should have been installed together with the Python environment. If it was not, follow the install instructions in their documentation.

With pip installed, getting the Libcloud libraries is a simple task. Use the following command to install the apache-libcloud package.

pip install apache-libcloud

If you already had a version of Libcloud installed, you can upgrade it with the next command.

pip install --upgrade apache-libcloud

To get the most up-to-date packages, you will need to use the development version. The Libcloud driver for UpCloud will initially be only available on the development channel until it gets included in the stable release.

Install the Libcloud development version with the command underneath.

pip install -e git+https://git-wip-us.apache.org/repos/asf/libcloud.git@trunk#egg=apache-libcloud

With the libraries installed, you are then ready to start programming. Continue below with examples of how to create and manage nodes on UpCloud using the Libcloud API.

Instantiating a new driver

To issue commands to the UpCloud services, you first need to instantiate a driver with your credentials. Begin by creating a file called libctest.py with the following content.

#!/usr/bin/env python
import os
import libcloud
from pprint import pprint

cls = libcloud.get_driver(libcloud.DriverType.COMPUTE,libcloud.DriverType.COMPUTE.UPCLOUD)
driver = cls(os.environ['UpCloud_API_username'], os.environ['UpCloud_API_password'])

pprint(driver.list_locations())

Next, assign your credentials to environmental variables named UpCloud_API_username and UpCloud_API_password. For example on Linux use the following commands.

export UpCloud_API_username='username'
export UpCloud_API_password='password'

These can also be stored permanently in your .bashrc or .zshrc profiles avoiding having to rerun the export commands manually on every new terminal.

Set the test program as executable with the next command.

chmod +x libctest.py

Then run the short program on the terminal to test that it connects to the UpCloud API correctly using the command below.

./libctest.py

[<NodeLocation: id=de-fra1, name=Frankfurt #1, country=DE, driver=Upcloud>,
 <NodeLocation: id=fi-hel1, name=Helsinki #1, country=FI, driver=Upcloud>,
 <NodeLocation: id=fi-hel2, name=Helsinki #2, country=FI, driver=Upcloud>,
 <NodeLocation: id=nl-ams1, name=Amsterdam #1, country=NL, driver=Upcloud>,
 <NodeLocation: id=sg-sin1, name=Singapore #1, country=SG, driver=Upcloud>,
 <NodeLocation: id=uk-lon1, name=London #1, country=UK, driver=Upcloud>,
 <NodeLocation: id=us-chi1, name=Chicago #1, country=US, driver=Upcloud>]

If you see an output like the above, your driver is working!

Creating a new server

The basic process of creating a new server requires the following minimum parameters: system image, configuration size, data centre location, and a name for the server. Below are some examples of how to select the parameters for the deployment call.

# Get the Ubuntu 16.04 public template
for image in driver.list_images():
    if image.name.startswith('Ubuntu') 
    and image.name.count('16.04') 
    and image.id.endswith('0'):
        break

# Select the node size from the preconfigured instances
for size in driver.list_sizes():
    if size.name == '1xCPU-1GB':
        break

# Pick the zone by the city name
for location in driver.list_locations():
    if location.name.startswith('London'):
        break

node = driver.create_node(
    image=image,
    size=size,
    location=location,
    name='Libcloud Example',
    ex_hostname='libcloud.example.com'
    )

pprint(node.state)

Simply add the above code to the end of your libctest.py test file. Then run the program as before to deploy the first server.

Rebooting a server

Issuing a reboot command requires you to first get the right node object. Afterwards, you have the following two options to restart the node.

Asking the driver to reboot the node.

node = [node for node in driver.list_nodes() if node.name == 'Libcloud Example'][0]
driver.reboot_node(node)

Alternatively, you can command the node object directly.

node = [node for node in driver.list_nodes() if node.name == 'Libcloud Example'][0]
node.reboot()

The functions will return True once the server has been rebooted successfully. If something went wrong, you will get False instead.

Destroying a server

Similarly to the reboot commands, a node can be destroyed using either of the two methods below. The process will shut down the target node and remove the server without deleting the storage device.

Select the node you wish to destroy and ask the driver to remove it.

node = [node for node in driver.list_nodes() if node.name == 'Libcloud Example'][0]
driver.destroy_node(node)

Or tell the node to destroy itself.

node = [node for node in driver.list_nodes() if node.name == 'Libcloud Example'][0]
node.destroy()

The functions will return True once the server has been successfully deleted. In case the server could not be removed, you will get False as a response.

Note that after destroying a server, the state of the node object will no longer be up to date. You should always refresh the list of nodes after creating or destroying nodes.

Including SSH keys at deployment

SSH keys are a safe and convenient way to authenticate when logging in to your cloud servers. You can include the public half of your SSH key pair when deploying a new node by reading it from the key file into the auth parameter.

By default, the public key is added to the root user account. If you wish to define a custom username, you can add it to the ex_username parameter. The new username is created at deployment and given sudo privileges.

from libcloud.compute.base import NodeAuthSSHKey

with open('/path/to/public_ssh_key', 'r') as public:
    public_ssh_key = public.read().replace('n', '')

node = driver.create_node(
    image=image,
    size=size,
    location=location,
    name='Libcloud Example',
    ex_hostname='libcloud.example.com',
    auth=NodeAuthSSHKey(public_ssh_key),
    ex_username='username'
    )

Defining a username already at deployment allows you to skip the hassle of setting up sudo access manually.

Using init scripts at deployment

Libcloud also supports running custom scripts at server deployment. The scripts can contain any normal Linux commands and tasks but will only work with Linux system templates.

Using an init script will require a private SSH key that corresponds to your public SSH key. The Libcloud driver needs to connect to the new server over SSH to run the script. Include the file path to your private SSH key in the ssh_key parameter as shown in the example below.

You also have the option to run the init script under a specific user account by defining the ssh_username parameter.

Note that the init script and deployment time SSH details are only available in the deploy_node() function.

from libcloud.compute.deployment import ScriptDeployment
from libcloud.compute.base import NodeAuthSSHKey

# Create an init script that is run during deployment
initscript = ScriptDeployment('echo "Deployed with Libcloud" > ~/libcloud.out')

# Include SSH keys that are used to authenticate on the server
private_ssh_key_file = '/path/to/private_ssh_key'
with open('/path/to/public_ssh_key', 'r') as public:
    public_ssh_key = public.read().replace('n', '')

node = driver.deploy_node(
    ssh_key=private_ssh_key_file,
    ssh_username='root',
    deploy=initscript,
    image=image,
    size=size,
    location=location,
    name='Libcloud Example',
    ex_hostname='libcloud.example.com',
    auth=NodeAuthSSHKey(public_ssh_key)
    )

Once deployed, the process will attempt to connect to the server and run the init script.

In case the SSH connection fails, the function will raise an HTTP exception.