Building a Secure MQTT Communication System: Lessons from a UNIX 511 Final Project

As part of my UNIX 511 course, I had the opportunity to dive into a challenging yet rewarding project: building a secure MQTT-based communication system. This project about understanding the architecture of message queues, understanding TLS certificates, and broker authentication and authorization. In this article, I’ll walk you through the journey, the challenges I faced, and the lessons I learned.

Understanding the Problem

The core goal of the project was to create a secure messaging system where multiple components could communicate safely:

Mosquitto broker as the central hub, routing messages securely between EV simulators (publishers) and the subscriber daemon over TLS-enabled channels

• Broker — the central server that routes messages.
• Publishers — in this case, EV simulators that generate messages.
• Subscriber — a daemon (evlogd_mqtt.cpp) that receives and logs messages.

Messages were sent on topics such as ev/EV1/charge and ev/EV2/charge, and security was enforced using TLS certificates, Access Control Lists (ACLs), and authentication mechanisms.

Setting Up the Mosquitto Broker

Configuring Mosquitto was one of the most challenging parts of the assignment. I had to understand:

• How Mosquitto loads configuration snippets from conf.d.
• The difference between system-wide installations and assignment-specific setups.
• Authentication vs. authorization.
• Why different ports mattered (1883 for unencrypted MQTT vs. 8883 for TLS).

Initially, the concepts were overwhelming, but reading the documentation carefully helped me configure the broker correctly. After setup, I was able to establish secure, authorized connections and verify that the broker was handling encrypted communication.

Key Broker Configurations

Some of the critical settings I applied in mosquitto/conf.d/evlogd included:

• require_certificate true — only clients with valid certificates could connect.
• use_identity_as_username true — linked client certificates to ACL usernames.
• acl_file — defined topic-level access rules.
• tls_version tlsv1.2 — enforced secure TLS connections.

I initially experimented with password files but ultimately relied solely on certificates for authentication.

Authentication and Authorization Challenges

Understanding Mosquitto’s authentication logic was eye-opening:

• allow_anonymous true — anyone could connect without credentials.
• allow_anonymous false — clients must authenticate.
• use_identity_as_username true — username extracted from the certificate CN.

Authorization was separate from authentication. Even with a valid certificate, a client couldn’t access topics unless explicitly allowed in the ACL file. ACLs defined exactly what each user could read or write, for example:

user evlogd-test
 topic readwrite ev/+/charge
 topic read ev/+/ack

These rules ensure that even a valid client can only interact with specific topics.

Certificates and TLS Setup

Setting up TLS required generating multiple certificates:

1. CA certificate (ca.crt) — trusted by all components.
2. Server certificate (server.crt + server.key) — used by the broker.
3. Client certificates — for each EV simulator and subscriber daemon.

Some key lessons from this process:

• Paths matter: I switched from relative to absolute paths to avoid connection failures.
• Permissions are critical: Mosquitto needs read access to certificates but only write access when generating them.
• Certificate consistency: All components must reference the same CA certificate to establish trust.

I also learned the importance of verifying certificates using OpenSSL, checking fingerprints, expiration dates, and key pairings. Once configured, TLS 1.2 ensured that all communication was encrypted and authenticated.

Configuring the Subscriber Daemon

The subscriber daemon (evlogd_mqtt.cpp) needed to handle incoming messages correctly. Some key steps:

• Updated all certificate paths to absolute locations.
• Ensured callbacks (on_connect, on_message) were registered before initiating the TLS handshake.
• Verified that the username extracted from the certificate matched the ACL rules.
• Kept the main loop running to continuously process messages.

After these adjustments, messages from the EV simulator were successfully logged in real time.