Tutorials How to install secure MQTT broker on Ubuntu

How to install secure MQTT broker on Ubuntu

MQTT stands for MQ Telemetry Transport. It is a publish/subscribe, extremely simple and lightweight messaging protocol, designed for constrained devices and low-bandwidth, high-latency or unreliable networks. The MQTT protocol defines two types of network entities: a message broker and a number of clients. An MQTT broker is a server that receives all messages from the clients and then routes the messages to the appropriate destination clients. An MQTT client is any device (from a microcontroller up to a full-fledged server) that runs an MQTT library and connects to an MQTT broker over a network.

In this tutorial, you will learn how to install, configure and secure an MQTT broker. For this task, we will use a popular message broker Mosquitto. In addition, you will learn how to use Certbot to automatically acquire Let’s Encrypt SSL/TLS certificate for your server. We will show you how to install and configure a simple Node.js web server for monitoring MQTT messages remotely from a web browser. We will also install a MongoDB for storing MQTT messages in the database.

Prerequisites

Domain name

You will need a valid domain name pointed to your server IP address. If you do not have one, you can purchase it from many domain name seller, e.g. Namecheap, GoDaddy, Domain.com, or any other that you prefer. If you do not know how to point a domain name to an IP address, check this guide to domain name systems or refer to an instruction from your domain name seller. For the purpose of this tutorial, we will use mqtt.example.com domain name as an example, replace it with your domain where asked.

Key pair

You will need to have a key pair to be able to use SSH keys login. If you do not have one, you will need to generate it.

On Linux and macOS open a terminal window. At the shell prompt, type the following command:

ssh-keygen -t rsa

The ssh-keygen program will prompt you for the location of the key file. You can use the default one or specify your own. Another option is to specify a passphrase to protect your key material. Note the location to which your public and private keys were saved because they will be required later.

On Windows, download and install PuTTY from the official website. Go to Start > All Programs > PuTTY > PuTTYgen and start the application. Click the Generate button and follow the instructions. Once the key generation is finished, you will be presented with the results. Click Save Private Key to save the private key as a file. Repeat the process for the public key, or simply copy the public key from PuTTY’s text area into your clipboard and save it as a text file. Note the location to which your public and private keys were saved because they will be required later.

Step 1 - Deploying a Cloud Server

First things first, if you are not registered on the UpCloud yet, begin by getting signed up. Take a moment to create an account after which you can easily deploy your own cloud servers.

Deploy a new cloud instance, where the first Simple Plan of 1 CPU core, 1 GB memory and 25 GB storage is sufficient. Of course, if you are planning to put your Server on some heavy tasks, use another Simple plan or the Flexible one. Select an availability zone of your choice and the Ubuntu Server 18.04 LTS (Bionic Beaver) from the Public Templates. You can find in-depth instructions on all configuration options in a guide for how to deploy a server.

Step 2 - Initial Server Configuration

In this step, you will find out how to configure your Ubuntu Server for increasing security and usability. This will give you a solid foundation for subsequent actions. You will learn about SSH Keys login, creating a new user with administrative privileges and basic firewall settings. If you already did this during deployment phase with Initialization Scripts, or manually after the deployment phase, you should skip this step.

If you are not already connected to your server, go ahead and log in as the root user using the following command:

ssh [email protected]_server_ip_address

After successfully logging in, we will create a new user called donald and grant him administrative privileges. You can name your user whatever you want.

useradd --create-home --shell "/bin/bash" --groups sudo donald

Create a hidden folder to your user account home directory on your cloud server with the following command:

mkdir -p /home/donald/.ssh

Insert public key from your key pair on your local machine to authorized_keys in the previously created hidden folder.

echo "your_public_key" >> /home/donald/.ssh/authorized_keys

Adjust SSH configuration ownership and permissions:

chmod 0700 /home/donald/.ssh
chmod 0600 /home/donald/.ssh/authorized_keys
chown -R donald:donald /home/donald/.ssh

Disable root to log in using SSH:

sed -i 's/^PermitRootLogin.*/PermitRootLogin no/g' /etc/ssh/sshd_config

Disable login with password:

sed -i 's/^PasswordAuthentication.*/PasswordAuthentication no/g' /etc/ssh/sshd_config

Restart the SSH service to apply the changes by using the command below:

systemctl restart sshd

You can configure the firewall using UpCloud Firewall. But, in this tutorial, we will set up a basic firewall by using UFW application. By doing this we will make sure to allow connections only to certain services. First, you need to install UFW with the next command:

apt install ufw

Sometimes, the application can already have registered profiles for UFW, so it can be managed with those profile names. You can check which applications have these profiles by inserting the following command:

ufw app list

You should at least get an OpenSSH as an answer since we are going to allow it in our firewall with:

ufw allow OpenSSH

Now we just need to enable ufw with the next command:

ufw enable

All other connections, that we explicitly did not set in allow list, are blocked. You can check the current status of your firewall at any time by typing:

ufw status

At this point, you have a solid foundation for your server. You should log out from your root user and login with your private key to your newly created user.

For consideration

Make sure to regularly check for updates on your server. Begin by updating the package list:

apt update

Next, upgrade installed packages to their latest available versions:

apt upgrade

Once the updates have finished, you can perform additional upgrades that involve changing dependencies, adding or removing new packages as necessary, with the following command:

apt dist-upgrade

This will take care of a set of upgrades which may have been held back by regular upgrade command.

Step 3 - Setup Certbot to acquire Let's Encrypt TLS Certificate

Let’s Encrypt is a nonprofit Certificate Authority providing free TLS certificate for your site. In this section, you will learn how to set up Certbot to automatically acquire SSL certificates. You need to install Certbot, but to be sure that you get the latest version, first, add Certbot’s repository:

sudo add-apt-repository ppa:certbot/certbot

Next, you need to update the package list with the newest repository:

sudo apt update

Install Certbot with the following command:

sudo apt install certbot

Certbot needs an open port 80 or 443 to acquire the TLS certificate, and since we are blocking all ports (except SSH) with firewall, you need to open one of these two. We are going to use port 80:

sudo ufw allow 80

Now, we can run our Certbot. Use the next command and follow onscreen instruction:

sudo certbot certonly --standalone --preferred-challenges http -d mqtt.example.com

You’ll need to complete the following selection:

  1. On the first installation on any specific host, you will need to enter a contact email.
  2. Next, go through the Let’s Encrypt Terms of Service and select Agree if you accept the terms and wish to use the service.
  3. Lastly, select whether you want to share your email address with the Electronic Frontier Foundation, a founding partner of the Let’s Encrypt project and the non-profit organization that develops Certbot.

That is it! You should see the congratulation message and also a path where your certificates are stored. Please remember this path because you will need for subsequent actions. Anyhow, it should be in /etc/letsencrypt/live/mqtt.example.com folder (replace mqtt.example.com with your domain). You can list your certificates with:

sudo ls /etc/letsencrypt/live/mqtt.example.com

This certificate is only valid for 90 days, but Certbot adds a script to cron.d that runs twice a day and automatically renews any certificate that is within 30 days of expiration. Later on, we are going to cover renewals and adding some extra commands to renew config file.

Step 4 - Install and configure Mosquitto MQTT broker

Eclipse Mosquitto is an open-source (EPL/EDL licensed) message broker that implements the MQTT protocol versions 5.0, 3.1.1 and 3.1. Mosquitto is lightweight and is suitable for use on all devices from low power single board computers to full servers. To install the latest version of Mosquitto you will firstly need to add Mosquitto’s repository:

sudo apt-add-repository ppa:mosquitto-dev/mosquitto-ppa

Next, you need to update the packages list with the newest repository:

sudo apt update

Then, install Mosquitto with the following command:

sudo apt install mosquitto mosquitto-clients

By doing this, you have successfully installed Mosquitto MQTT broker. You can use it out-of-the-box as installed, but we do not recommend that. We suggest you configure your server for some additional security. We will add a new Mosquitto user secured with a password using the following command:

sudo mosquitto_passwd -c /etc/mosquitto/passwd mqttdonald

Open up a new configuration file named custom.conf in /etc/mosquitto/conf.d/ folder:

sudo vi /etc/mosquitto/conf.d/custom.conf

Copy the following commands and paste them in the custom.conf file. Replace the mqtt.example.com with your domain on each certificate and key file line.

allow_anonymous false
password_file /etc/mosquitto/passwd
listener 1883 localhost
listener 8883
certfile /etc/letsencrypt/live/mqtt.example.com/cert.pem
cafile /etc/letsencrypt/live/mqtt.example.com/chain.pem
keyfile /etc/letsencrypt/live/mqtt.example.com/privkey.pem
listener 8083
protocol websockets
certfile /etc/letsencrypt/live/mqtt.example.com/cert.pem
cafile /etc/letsencrypt/live/mqtt.example.com/chain.pem
keyfile /etc/letsencrypt/live/mqtt.example.com/privkey.pem

Save the file and exit by typing the :wq command. Next, the Mosquitto broker needs to be restarted so the configuration can take place.

sudo systemctl restart mosquitto

After this, add new rules to firewall to match the .conf file:

sudo ufw allow 8883
sudo ufw allow 8083

With this configuration file, we told our MQTT broker that anonymous users will not be tolerated. We have specified the path to a file where passwords are being stored. We have configured three listeners. First is on port 1883 which is unencrypted and only allowed to be used in the localhost environment. It is mostly intended for testing purposes. The second listener is on port 8883, which is encrypted with TLS certificate. The third listener is on port 8083 which is encrypted with TLS certificate as well, but it is intended for use over WebSocket protocol.

For the testing purposes, log in to your server in a second terminal to have two command line available at the same time.

In the first terminal, run the following command to subscribe to some topics (e.g. “mqtt_topic_name”):

mosquitto_sub -h localhost -t mqtt_topic_name -u "mqttdonald" -P "password"

Then, in the second terminal, run the following command to publish the message to the previously mentioned topic:

mosquitto_pub -h mqtt.example.com -t mqtt_topic_name -m "Hello MQTT World" -p 8883 --capath /etc/ssl/certs/ -u "mqttdonald" -P "password"

You should receive the following message: “Hello MQTT World” in the first terminal.

Do not forget to close the second terminal and exit from mosquitto_sub command in the first terminal with CTRL+C.

To test your MQTT broker via WebSocket you can use some popular online services like Eclipse Paho, HiveMQ, MQTTLens, or some other that you prefer.

Step 5 - Install Node.js

Node.js is an open-source, cross-platform, JavaScript runtime environment that executes JavaScript code outside of a web browser. Node.js lets developers use JavaScript to write command-line tools and for server-side scripting to produce dynamic web page content before the page is sent to the user’s web browser.

As with the other software we’ve installed so far, we first need to add Node.js repository to get the latest version. But, this time procedure is somewhat different. We are going to add the repository by using the following command, which will execute a script from the URL:

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -

Now we can install Node.js:

sudo apt install nodejs

To be sure that npm (which is installed with Node.js) is going to work properly you need to install additional packages that have some general usage like compilers, libraries and some other utilities:

sudo apt install build-essential

We are later going to cover the setup of a basic webpage with Node.js.

Step 6 - Install MongoDB

MongoDB is a general-purpose, cross-platform document-based database program. Classified as a NoSQL database program, MongoDB uses JSON-like documents with schema and it is built for modern application developers and for the cloud era. To install the latest version you need to import the MongoDB public GPG Key from the URL below and manually add the repository:

wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | sudo apt-key add -

Create a list file for MongoDB on your version of Ubuntu:

echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.2.list

Update package list with the newest repository:

sudo apt update

Install the latest stable version:

sudo apt install mongodb-org

Next, we need to start the service manually with the following command:

sudo systemctl start mongod

Then, enable the MongoDB service to have it start at system boot:

sudo systemctl enable mongod

That is it! We are not going to expose mongo service outside from our server, so we can use MongoDB as it is – with the default configuration.

Step 7 - Setup Simple Website

In this step, we will show you how to set up a basic website. We will keep things simple and set up everything manually. First, we are going to create our main work folder:

mkdir ~/webserver

Then we are going to navigate to our newly created folder:

cd ~/webserver

Now we can install the required npm packages:

npm install express ejs mqtt socket.io moment jquery mongoose

In our Simple Website, we will have two pages – Main and History. Since we are using EJS template engine and will need one more additional folder for these files:

mkdir views

EJS is a simple templating language that lets you generate HTML pages. Our first page is for the Main view, where we are going to make real-time MQTT dataflow. Open a new file called main in the folder views:

vi views/main.ejs

Copy the following EJS/HTML code and paste it to your newly created file:

<!DOCTYPE html>
<html>
  <head>
    <title>MQTT Client - Mainpage</title>
    <script type="text/javascript" src="/lib/socket.io.js"></script>
    <script type="text/javascript" src="/lib/jquery.min.js"></script>
    <script type="text/javascript">
      var socket = io();
      socket.on('mqtt', function(msg) {
      $('#tbl').prepend('<tr><td>' + msg.datetime + '</td><td>' + msg.topic + '</td><td>' + msg.message + '</td></tr>');
      });
    </script>
    <style>
      table, th, td {
        border: 1px solid black;
      }
      table {
        width: 100%;
      }
      .col1 {
        min-width: 150px;
        text-align: left;
      }
      .col2 {
        width: 40%;
        text-align: left;
      }
      .col3 {
        width: 40%;
        text-align: left;
      }
    </style>
  </head>
  <body>
    <div>
      <a href="/">MAIN</a> <a href="/history">HISTORY</a>
    </div>
    <div>
      <table>
        <thead>
          <tr>
            <th class="col1">Datetime</th>
            <th class="col2">Topic</th>
            <th class="col3">Payload</th>
          </tr>
        </thead>
        <tbody id="tbl"></tbody>
      </table>
    </div>
  </body>
</html>

Then save the file and exit the editor.

Next, again in the views folder, open new a file called history.ejs:

vi views/history.ejs

Paste the following code, which will create a web page where the recorded data from the database will be shown:

<!DOCTYPE html>
<html>
  <head>
    <title>MQTT Client - History</title>
    <style>
      table, th, td {
        border: 1px solid black;
      }
      table {
        width: 100%;
      }
      .col1 {
        min-width: 150px;
        text-align: left;
      }
      .col2 {
        width: 40%;
        text-align: left;
      }
      .col3 {
        width: 40%;
        text-align: left;
      }
    </style>
  </head>
  <body>
    <div>
      <a href="/">MAIN</a> <a href="/history">HISTORY</a>
    </div>
    <div>
      <table>
        <thead>
          <tr>
            <th class="col1">Datetime</th>
            <th class="col2">Topic</th>
            <th class="col3">Payload</th>
          </tr>
        </thead>
        <tbody id="tbl">
          <% history.forEach(function (data) { %>
          <tr>
            <td><%= data.datetime %></td>
            <td><%= data.topic %></td>
            <td><%= data.payload %></td>
          </tr>
          <% }) %>
        </tbody>
      </table>
    </div>
  </body>
</html>

Lastly, we have the most important file where the main JavaScript code for our Node.js web server is going to go. Open a new file called server.js in the main work folder:

vi ~/webserver/server.js

With this code, we are implementing MQTT client and storing received data in the MongoDB database. Paste the following code.

Remember to replace the relevant parts with your own data including your domain name and MQTT username and password.

#!/usr/bin/env node

/** server.js */

// Dependencies
const fs = require('fs');
const http = require('http');
const https = require('https');
const express = require('express');
const path = require('path');
const mqtt = require('mqtt');
const moment = require('moment');
const mongoose = require("mongoose");

//Certificate
const privateKey = fs.readFileSync('/etc/letsencrypt/live/mqtt.example.com/privkey.pem', 'utf8');
const certificate = fs.readFileSync('/etc/letsencrypt/live/mqtt.example.com/cert.pem', 'utf8');
const ca = fs.readFileSync('/etc/letsencrypt/live/mqtt.example.com/chain.pem', 'utf8');

const credentials = {
  key: privateKey,
  cert: certificate,
  ca: ca
};

//Connection to MongoDB
mongoose.Promise = global.Promise;
mongoose.connect("mongodb://localhost:27017/mqtt", { useNewUrlParser: true, useUnifiedTopology: true } );

//MongoDB MQTT Schema
var mqttSchema = new mongoose.Schema({
  datetime: {
    type: String,
    default: () => moment().format("YYYY-MM-DD HH:mm:ss")
  },
  topic: String,
  payload: String
});
var MqttData = mongoose.model("mqttData", mqttSchema);

//Connection to MQTT
const client = mqtt.connect('mqtts://mqtt.example.com', {
  port: 8883,
  username: 'mqttdonald',
  password: '##########'
});

//Creating Express App
const app = express();

//Starting both http & https servers
const httpServer = http.createServer(app);
const httpsServer = https.createServer(credentials, app);

//Connection to Socket.io
const io = require('socket.io')(httpsServer);

//Handling new user
io.on('connection', function(socket){
  console.log('A new user has been connected');
});

//Subscribing to # topic on connection
client.on('connect', function () {
  client.subscribe('#', function (err) {});
});

//On received MQTT message
client.on('message', function (topic, message) {
  //Emit event to socket
  io.emit("mqtt", { datetime: moment().format("YYYY-MM-DD HH:mm:ss"), topic: topic, message: message.toString()});

  //Saving received data to MongoDB
  var mongomqttdata = new MqttData({
    topic: topic,
    payload: message.toString()
  });
  mongomqttdata.save();
});

//Use EJS view engine
app.set('view engine', 'ejs');

//Expose socket.io-client and jquery to clients in browser
app.use('/lib', express.static(path.join(__dirname, 'node_modules/socket.io-client/dist/')));
app.use('/lib', express.static(path.join(__dirname, 'node_modules/jquery/dist')));

//If request is via https execute next, else redirect to https
app.use((req, res, next) => {
  if (req.secure) {
    next();
  } else {
    res.redirect('https://' + req.headers.host + req.url);
  }
});

//Render History page
app.use('/history', (req, res) => {
  MqttData.find({}, function(err, data) {
    res.render('history', {
      history: data
    });
  });
});

//Render Main page
app.use('/', (req, res) => {
  res.render('main');
});

//Open https listener
httpsServer.listen(443, () => {
  console.log('HTTPS Server running on port 443');
});

//Open http listener
httpServer.listen(80, () => {
  console.log('HTTP Server running on port 80');
});

Afterwards, save the file and exit the editor.

We are then done with the coding of our webpage, now we just need to open port 443 in the firewall (port 80 is already opened):

sudo ufw allow 443

You can now test your node application with:

sudo node server.js

Open your domain on a web browser, publish something to your MQTT broker and that message should appear on the web page. Open history page and you should be able to see previously input messages.

However, our job is still not finished with the web page since we need to make it start and run with the system.

First, we will make our server.js file executable:

sudo chmod +x ~/webserver/server.js

Then, we need to create a new file that will actually be a service file in our system folder:

sudo vi /etc/systemd/system/webpage.service

Copy and paste the following code in your file:

[Unit]
Description=Node.js HTTPS Server

[Service]
PIDFile=/tmp/webpage-99.pid
User=root
Group=root
Restart=always
KillSignal=SIGQUIT
WorkingDirectory=/home/donald/webserver/
ExecStart=/home/donald/webserver/server.js

[Install]
WantedBy=multi-user.target

To ensure that our server.js file is working properly as a service, make sure that you are using Unix-style newlines.

This can be done by opening the server.js file using the vi text editor:

sudo vi server.js

Edit and save the file with the following commands:

:se ff=unix
:wq

Then, enable the service and start it

sudo systemctl enable webpage.service
sudo systemctl start webpage.service

To check that everything is working properly you can run:

sudo systemctl status webpage.service

That is it! You are now done with your web page.

Step 8 - Adding renew hooks to Certbot

All that is left now is to fix our automatic SSL renew hook. Since the port 80 is already occupied with Node.js server we need to temporarily open it. After successful certificate renewal, we need to restart involved services, Mosquitto and the webpage. Open the Certbot renew hook configuration file. Note that you need to open the configuration file specific to your domain name.

sudo vi /etc/letsencrypt/renewal/mqtt.example.com.conf

Next, append the following commands to the end of the file:

renew_hook = systemctl restart mosquitto.service ; systemctl restart webpage.service
post_hook = systemctl start webpage.service
pre_hook = systemctl stop webpage.service

Then save your file and run following command to test your new renew conf file:

sudo certbot renew --dry-run

Certbot will then make a dry-run to attempt to renew the SSL certificates without actually making any changes.

Once finished, if there are no errors, you’ve succeeded and everything is set!

Conclusion

We hope that this tutorial helped you learn about the MQTT broker and additional applications that expand its capabilities and make it more secure. We kept things simple, and it is up to you to adapt it for some real usage. Feel free to post a comment with your impressions or suggestions. Thank you for your attention!

Leave a Reply

Your email address will not be published. Required fields are marked *

Locations

Helsinki (HQ)

In the capital city of Finland, you will find our headquarters, and our first data centre. This is where we handle most of our development and innovation.

London

London was our second office to open, and a important step in introducing UpCloud to the world. Here our amazing staff can help you with both sales and support, in addition to host tons of interesting meetups.

Singapore

Singapore was our 3rd office to be opened, and enjoys one of most engaged and fastest growing user bases we have ever seen.

Seattle

Seattle is our 4th and latest office to be opened, and our way to reach out across the pond to our many users in the Americas.