Updated on 24.5.2023

How to benchmark Cloud Servers

Cloud service providers often define host tiers by the allocated resources, but the differences in the underlying hardware, architecture and performance tuning can result in varying capabilities even between similar configurations.

The easiest way to measure the real differences between servers is to run a set of tests, i.e. a benchmark, to create simple-to-read values for comparisons.

Preparations to benchmark cloud servers

Benchmarking cloud servers help in quantifying the real performance behind the specifications, but a significant part of getting comparable results is to eliminate as many of the variables that could otherwise affect the benchmarks. For example:

  • Use the same operating system on each server.
  • Update software to their latest versions.
  • Check that there are no resource-hungry processes in the background.

Having the servers adequately set up, you are ready to start off with the first cloud server benchmarks. Continue below with instructions on how to rate the CPU and RAM performance.

Geekbench 3

Geekbench 3 is one of the most popular cross-platform processor benchmarking tools. It offers a standardized scoring system that separates single-core and multicore results for better comparison.

Running Geekbench on your server is quite a straightforward process, but you will need to install a couple of dependencies beforehand. The free version is 32-bit only; therefore, your 64-bit servers will need additional runtime libraries.

Ubuntu

sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install libc6:i386 libstdc++6:i386

Debian

sudo dpkg --add-architecture i386
sudo aptitude update
sudo aptitude install libc6:i386 libstdc++6:i386

CentOS

sudo yum install wget glibc.i686 libstdc++ libstdc++.i686

With the prerequisites fulfilled, go ahead and download the benchmark package. On a good network, the download is almost instantaneous, but should not take long in any case.

wget http://cdn.primatelabs.com/Geekbench-3.4.1-Linux.tar.gz ~/

Afterwards, unpack the files and switch to the newly created folder. The following command will do just that.

tar -zxvf ~/Geekbench-3.4.1-Linux.tar.gz && cd ~/dist/Geekbench-3.4.1-Linux/

Finally, you can run the benchmark with the command below.

./geekbench

The Geekbench 3 performs a series of tests to measure the CPU performance with integer and floating point tasks as well as collects memory bandwidth data to rate the system memory. Once complete, you will see an output similar to the example underneath.

Open the first link in your web browser to view the results. Alternatively, if you register on the Geekbench website, you can use the second link to claim the results to your profile. Claiming the results allows you to save and compare the scores later.

Upload succeeded. Visit the following link and view your results online:

 http://browser.primatelabs.com/geekbench3/6088030

Visit the following link and add this result to your profile:

 http://browser.primatelabs.com/geekbench3/claim/6088030?key=

The total score is scaled directly one-to-one to the CPU performance; double the score means double the performance. The scoring system is calibrated against a baseline score of 2500, which is the result for an Intel Core i5-2520M @ 2.50 GHz.

fio

A big part of the overall performance of a cloud server comes from the storage device’s read and write speeds. A good option to use for this purpose is fio, which is an I/O benchmarking and stress testing tool available on a multitude of platforms. Install fio using the commands applicable to your system below.

Debian and Ubuntu

sudo apt-get install fio

CentOS

sudo yum install wget libaio libibverbs librdmacm librbd1-devel
wget https://kojipkgs.fedoraproject.org/packages/fio/2.2.8/2.el7/x86_64/fio-2.2.8-2.el7.x86_64.rpm ~/
sudo rpm -iv ~/fio-2.2.8-2.el7.x86_64.rpm

IOPS is a measuring unit of input/output operations per second and is a commonly used performance metric for storage devices. The usual way to test IOPS is to perform random read and write operations, with fio this can be done using the three examples below.

Random read/write performance
fio --name=randrw --ioengine=libaio --direct=1 --bs=4k --iodepth=64 --size=4G --rw=randrw --rwmixread=75 --gtod_reduce=1
Random read performance
fio --name=randread --ioengine=libaio --direct=1 --bs=4k --iodepth=64 --size=4G --rw=randread --gtod_reduce=1
Random write performance
fio --name=randwrite --ioengine=libaio --direct=1 --bs=4k --iodepth=64 --size=4G --rw=randwrite --gtod_reduce=1

As with the Geekbench scores, the higher the IOPS, the faster the storage. For comparison, a standard 7,200 rpm SATA drive HDD would have a score of 75-100 IOPS.

IOPing

Another way to benchmark storage performance is to measure the latency on individual requests. IOPing is a simple tool that does just that. It runs I/O requests to the storage device to benchmark the time to reply. The results display disk latency in the same way the ping test measures network latency.

Debian and Ubuntu
wget https://launchpad.net/ubuntu/+archive/primary/+files/ioping_0.9-2_amd64.deb ~/
sudo dpkg -i ~/ioping_0.9-2_amd64.deb
CentOS
wget https://kojipkgs.fedoraproject.org/packages/ioping/0.9/1.el7/x86_64/ioping-0.9-1.el7.x86_64.rpm ~/
sudo rpm -iv ~/ioping-0.9-1.el7.x86_64.rpm

Then run the test with the command below.

ioping -c 10 .

The output will show something similar to the example underneath. The time shows the I/O latency measured in microseconds.

4 KiB from . (ext4 /dev/vda1): request=1 time=78 us
4 KiB from . (ext4 /dev/vda1): request=2 time=175 us
4 KiB from . (ext4 /dev/vda1): request=3 time=167 us
4 KiB from . (ext4 /dev/vda1): request=4 time=174 us
4 KiB from . (ext4 /dev/vda1): request=5 time=168 us
4 KiB from . (ext4 /dev/vda1): request=6 time=185 us
4 KiB from . (ext4 /dev/vda1): request=7 time=179 us
4 KiB from . (ext4 /dev/vda1): request=8 time=178 us
4 KiB from . (ext4 /dev/vda1): request=9 time=177 us
4 KiB from . (ext4 /dev/vda1): request=10 time=195 us
--- . (ext4 /dev/vda1) ioping statistics ---
10 requests completed in 9.00 s, 5.97 k iops, 23.3 MiB/s
min/avg/max/mdev = 78 us / 167 us / 195 us / 30 us

The latency number measures the delay in response, which means the lower the delay, the better the performance.

Conclusions

Cloud server benchmarking tools are a very convenient way to compare performance between cloud service providers. They are readily available and straightforward to use regardless of the operating systems available. Performance-demanding web applications might see a noticeable difference in the quality of the service when running on genuinely capable hardware, but even if you are not worried about numbers you should still know what you are getting for your money.

If you wish to read more about benchmarking between cloud server providers, check out our Cloud Benchmark at Slush blog post for how DigitalOcean and AWS EC2 fared against UpCloud. Also, if you are still unsure about cloud server performance, go ahead and find out more about our Cloud Server vs. VPS vs. Dedicated Server comparison.

Janne Ruostemaa

Editor-in-Chief

  1. Mike van Eckendonk

    Great article. Need to update it a bit because on Centos it’s outdated and now Geekbech 5 is out.

  2. Janne Ruostemaa

    Hi Mike, thanks for the comment. The tutorial is indeed a little dated, we’ll make sure to get it updated to include the latest versions.

  3. What’s the best way to benchmark the internal private network? Traceroute?

  4. Janne Ruostemaa

    Hi Peter, thanks for the question. The right answer will depend on what is your intended use for the network. Traceroute or ping should show very low latency between cloud servers when using the private network. Also important to remember, private networks are unmetered.

  5. John Bonachon

    try iperf

  6. what is the best benchmarking tool for cloud service provider like AWS

  7. Janne Ruostemaa

    Hi there, thanks for the question. Geekbench is generally a good starting point being very user friendly. If you want something more advanced, have a look at sysbench.

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top