Making IoT Devices Carrier Agnostic

Like I talked about in my previous blog post, Optimizing IoT Devices For Low Quality Cellular Regions, our IoT devices are communicating with our server in an extremely efficient manner however our infrastructure is carrier locked. While fine for a short term solution, long term this hurts our flexibility, coverage, and pricing.

What is the Carrier doing

To start figuring out how to make our devices carrier agnostic, we first need to start by looking at what our current carrier does. Breaking our communication protocol down into steps as follows:

1. Node sends UDP packet to carrier
2. Carrier packages UDP data into MQTTS message
3. Carrier sends MQTTS message to our server for processing
4. Server sends MQTTS response to Carrier via broker
5. Carrier receives MQTTS message and converts it to a UDP packet
6. Carrier sends UDP packet to Node

We can see that our carrier is doing two main operations as a part of its router.

1. Convert message to and from MQTT format and forwards back and forth between Node and Server
2. Encrypts data before it leaves the cell network

In order to bypass the carrier altogether we will need to create an in-house solution for both of these steps.

Carrier Agnostic Communications

To start implementing step one we need to figure out the most generic way to send data across a network to maximize compatibility. Lucky for us, we are already using the most generic form of communication possible. Because we are using UDP to send data to the carrier, and most of the modern communication protocols utilize UDP as a transport layer in some fashion we can be confident that all carriers / networks will support this operation.

If we were to just bypass the carriers routing and send UDP messages directly to our server we would have achieved step one. And its essentially as simple as it sounds, make a UDP server using the standard library included in most languages and we can (almost) achieve full communication.

The last caveat is the carrier NAT. Generally as a part of a carriers network security they do not allow inbound messages unless a port has already been opened. If you were on your home network you could open a port on your firewall via port-forwarding to bypass this issue all together. However, on the carrier’s network we have to find a different way to open ports.

While there are many ways to achieve this, such as a “NAT Punch Through”, the simplest way that works for us, is to ensure that the server responds on the same port that it received data from. Which looks as follows:

Server listens on 8000

Node : 5000 → Server : 8000

Server : 8000 → Node : 5000

In this way, the NAT opens an external port on its firewall that forwards back to our Node device on port 5000. As long as the server captures the external port used for the initial message our server can send a message back.

While this simplex design deviates from our original duplex design (different ports for sending and receiving), and the node device always has to initiate communication, if properly managed server side this solution is just as fast as our previous setup.

End to End Encryption

With the UDP server setup, we have bypassed the carriers first step which was the transformation to MQTT, however we are now sending raw UDP packets across the network which is a big security risk. So how do we go about re-encrypting our data? We would need to start with some symmetric key algorithm, for which we can use AES since its a standard. AES is well supported so it should be no problem to add to both our server and our IoT devices.

However, this only solves one part of the problem, we still need to share our keys with the server. The standard way to do this would be to generate a new random AES key for each packet, then transfer those keys with an asymmetric key algorithm like RSA. However, we chose to go against this route for a couple of important reasons.

1. The asymmetric key algorithm used in TLS is what caused our connectivity problems discussed in my last blog post
2. Common asymmetric key algorithms are not quantum-resistant and thus not future proof, we would need to implement a newer algorithm which would add to the complexity and work required

For these reasons we opted to leverage our production process and add a key exchange step to our flashing / registration process. Baked into the bootloader is an AES key generation and reporting command which is used as follows:

1. Flash Node device with Bootloader
2. Use AES command to get AES key (as well as other UID data such as ICCID and IMEI)
3. Computer flashes main program onto Node device and registers the device on our server, passing along the AES key

This essentially acts as a one time key-exchange system, and is fully secure from remote attackers. The only way to attack our devices’ communications with this system would be to physically attack the hardware to retrieve the key, which would only expose a single device and can be easily mitigated with physical attack preventions

Results

After implementing these new systems we have made our devices fully carrier agnostic (minus small firmware changes like APN). This has allowed us to customize carriers for different use cases, expanding our coverage and reducing network costs for end users. In this process we have also added full end to end encryption which was lacking on our old setup as data wasn't encrypted between the node and the cellular network, only after leaving the network. As well as future proofed our data against quantum decryption and save now, decrypt later attacks.

Furthermore, we only increased our entire packet size by ~20 bytes for things link the AES Initialization Vector (IV) and packet size.