In the age of small data, embedded engineers have new challenges

February 18, 2016

In the age of small data, embedded engineers have new challenges

Dealing with limited system resources was once one of the main challenges facing embedded engineers. By using a smaller microcontroller with less memo...

Dealing with limited system resources was once one of the main challenges facing embedded engineers. By using a smaller microcontroller with less memory, developers could lower manufacturing costs. However, that meant they needed to put their effort into minimizing memory and processor use, which required tough software decisions, typically centered on the cost, efficiency, and function of the application.

The game has changed. Powerful processors with large RAM and flash arrays are now available at commodity prices. Networking is ubiquitous, and while the function of an application is important, the higher risks and value in the Internet of Things (IoT) world are quickly shifting from the application itself to the data the “thing” stores and communicates. While much is discussed about cloud computing and big data, it has become critical for embedded engineers to develop a strategy to securely and reliably manage this small – yet highly valuable – data.

Of course, the security risks of IoT data being exposed or hacked can’t be denied, and the embedded industry still has work to do to meet appropriate software development and testing standards to help address that. But what if flash data isn’t guaranteed to be stored reliably? What if the file system becomes corrupt or the flash wears before the end of the planned life? The system simply might be too unreliable and expensive to maintain, something that might not be known until the system fails because security and reliability risks can’t be detected by testing. Such risks of data loss and exposure aren’t addressed by careful development or software testing alone.

Engineering must require collaboration at a system level, and this extends to hardware. When choosing flash storage, for example, well-known risks include the limited storage lifespan and the fact that cells will wear out eventually. However, there are other, less-known risks – some perpetrated by imprecise claims. For instance, a file system may be labeled as “safe,” but unless it explicitly defines behavior required by the application, the driver, and the media, it’s a meaningless claim.

Many suppliers specify flash (SD cards) as industrial grade or similar. But this often only refers to operating temperature and doesn’t guarantee any additional data integrity, which means that these devices shouldn’t be used to store critical data. Developers should also have a strategy for devices with more complex issues such as wear leveling, bad block management, error correction, and similar flash complexities.

To an embedded engineer, an IoT device looks conceptually similar to other embedded applications. It has an embedded processor running I/O functions to collect information and control an application, flash memory to store subscriber and usage data, and a communications interface.

The core engineering challenges presented by embedded data safety are fundamentally the same as those posed by functional safety. These problems can be overcome by appropriate knowledge, risk assessment, and the use of well-understood system-level design and process practices. The risk assessment of the device and its components must ensure that its data – which has real value – is stored in a fail-safe manner and protected from unauthorized access.

This can be tackled at the development or industry level, but the proliferation of networked IoT devices in the next few years will create a huge number of data-centered vulnerabilities. Basic questions developers must ask to assess data risks include:

• Who can potentially read or modify stored data?

• What will be the consequences of data exposure, loss, or corruption?

• Can anyone gain control of the application?

• Who can communicate with the device?

• Will data be stored reliably, even under power loss or reset?

• Is data storage persistent and for how long?

• Can the code or application be modified?

The purpose of this type of assessment is to extend the objective of application quality from “Will it operate correctly?” to “Is the communication and storage of data appropriately managed?” In this new age, reliable storage and secure communications of small data is one of the most critical issues facing embedded developers. It’s too important to be left to the lowest level of implementation.

David Brook is director of sales and marketing at HCC Embedded. David began his career in 1992 as a staff engineer with Motorola, specializing in embedded systems and real-time software. A veteran of international business, he earned an MBA in 2000, and has launched and managed successful technology businesses in locations such as Munich, Bristol, Boston, and Edinburgh. He is a graduate of the University of the West of Scotland.

HCC Embedded

[email protected]



David Brook, HCC Embedded