Optimizing IoT Devices For Low Quality Cellular Regions
Having a robust datalink is essential for the modern day IoT device. While consumer devices often exist where cellular connectivity is strong, this is often the opposite for industrial IoT devices, which in many cases need to operate in poor quality cellular areas, such as the rural parts of major US Trucking routes, or abroad in countries with lacking cellular infrastructure.
The Problem
When starting work on the packet communication protocols for our Node IoT devices, we originally wanted to use something designed for IoT, well tested, and with lots of recourses. We settled on using an MQTT broker, specifically the MQTTS variant apart of AWS, and sending data as a JSON for easy parsing and debugging. And while this system proved effective while testing, we found that when testing along the major US Interstates we were dropping packets. After doing some research and data analysis, we found that the likely culprit was a loss of network connection before the MQTT connection could be established and the data sent.
Optimizing Data
As a first attempt at fixing this, we decided to optimize our packet size by converting our JSON data to fixed binary streams according to a custom encoding / decoding protocol. This essentially took the form of a byte stream where labels were removed and numbers were reduced to their raw bytes.
For example a simple temperature packet may look as follows
hex: 54 23 00 00 00 41 43 33 33
decoded: T 35 12.2
This packet decoded gives us a measurement of 12.2C @35s, where ‘Tʼ serves as both a start of packet indicator but also indicates the type of packet. For a relatively large packet we were able to reduce our packet size 10x. However, although this was a step in the right direction it did not fully solve our problem, after pushing out the fix and doing more testing we realized that a lot of connections were dropped after a failure to handle the TLS handshake required by MQTTS, often this process required multiple transfers of 1kb+ certificate files which at this point far exceeded our 60 byte packet size. The relatively large file transfer requirements often exceeded the limited network bandwidth and limited time that our devices were in cellular range during transport.
Carrier Workaround
This is when we decided to find a 3rd party workaround for offloading the TLS handshake and stumbled upon our IoT providers routing options. Here we were able to specify that our carrier should intercept all packets of a given format and handle the handshake steps for us. This meant that the handshake was now always performed on a stable network and just the raw packet data was sent over the poor quality cellular network.
At this point we opted to further optimize our data transfer by reducing our protocol to raw UDP packets, this worked fine as our carrier still intercepted and encrypted them while also removing any now-unused MQTT overhead. This also allowed us to utilize our unique data requirements to pack data into the server ACK for further data savings.
Results
- Decreased Packet Loss: As a result of both our packet optimizations and handshake offloading we were able to decrease our packet loss from 95% along rural areas to only about 5% which was easily mitigated with the implementation of transmission retries and data storage.
- Data Rate Savings: Since we reduced our base packet size by 10x as well as offloaded the ~4kb TLS handshake to a no-extra-charge service provided by our carrier we saw significant savings in our data transfer cost, both in cellular fees and AWS fees.
While we can say that the solutions discussed were overall very successful, there is still one major downside to its implementation. Our devices were now locked into a carrier due to our infrastructures dependence on a carrier specific feature that may differ from carrier to carrier, or simply not exist. This is a problem that we recently solved after a need to switch to a different Cellular IoT provider, which can be read about in our blog post Making IoT Devices Carrier Agnostic.