Techniques to Improve UX for Success in Embedded Markets

April 16, 2021

Story

Creating a meaningful and relevant user experience (UX) for IoT devices is much harder than building a SaaS application, website, or smartphone app. Your design must account for software and hardware features, and run on platforms with less resources and tighter power constraints than the average cloud or web app.

Bringing forces together to release an embedded GUI application that beats consumers’ expectations is a matter of understanding these requirements. As we see with our customers, let’s address the four biggest challenges and solutions embedded UX designers and developers have in delivering successful products to market.

Understanding how to maximize graphics capabilities

While everyone wants a smartphone-like experience on their devices, it can be more challenging to create them on non-mobile platforms such as microcontrollers (MCU) and microprocessors (MPU). Graphics typically consume the most resources and planning your pipeline carefully helps to create a sophisticated experience that doesn’t glitch or stutter at runtime. Think about these things when developing your graphics:

1. Image format and loading mechanism can have a big impact on runtime performance — encoded formats such as PNG and BMP require resources to decode while raw file formats are typically pushed directly to the display with less overhead (this webinar explains how to choose the best options, including when to use flash versus RAM memory).

2. Animations can improve explainability, but they may be confusing to the user if not designed well. Having the ability to create, test, and edit animation sequences within development tools before target deployment can help streamline the process.

3. Consider using hardware acceleration, where available, to offload the CPU from performing high-running graphics tasks.

4. Similarly, there are a few software rendering engines, either open source or provided by the hardware manufacturer (such as ChromeART, G2D, OpenGL, and Vulkan), that optimize graphics performance.

Fitting UX into device size and form factor

Whether it’s a large smart home display or a tiny wearable on your wrist, your UX is always limited by the device’s format and screen size. It’s easy to trade greater usability and sophistication for reduced form factor. There’s a world of difference between designing controls for an e-commerce site versus building GUIs that fit onto a 40 mm-wide smartwatch touchscreen.

Your project may also support one or more product lines, where you’re developing for different platforms with different features:

“The proliferation of special-purpose devices leads to experiences that can take place on multiple devices — or what is referred to as multi-device experiences. Creating these experiences requires device-agnostic design, which asks designers to think holistically in terms of a user journey.” (Trends in UX Design: the IoT Perspective, Design World)

There is a direct relationship between UX and screen characteristics, and with different form factors come these considerations for embedded GUI developers:

1. Simplify and declutter screen content to reduce the task load on users and make scaling to different devices easier.

2. There may be less room for text, so consider replacing strings with icons or images that clearly communicate the same function.

3. Use simple navigation gestures, buttons, and infinitely scrolling menus to help users access information fast (and maintain this design language across your product line to make interactions easier for users).

Minimizing power consumption is critical

“We all dislike the chore of charging our devices every day, so manufacturers must build products that will last longer — a week or months in some cases,” said marketing manager of wearables & personal devices of NXP Semiconductors, Eduardo Montanez.

Longer battery life is the hallmark of any IoT product pitch, and the device’s screen is typically the biggest power consumer. The most obvious strategy for saving power is to use sleep mode whenever possible, waking the screen only when there’s a user-facing event, but there are other factors to consider:

1. For LCD displays, dimming the backlight when inactive or in low light can save a lot of power.

2. For OLED or AMOLED screens, a GUI dimming layer or muted color palette design draws less power.

3. Graphics rendering has a large impact on power draw — if you’re doing both 2D and 3D graphics, running a hybrid rendering model (such as with Crank Storyboard, allowing you to switch between G2D acceleration for 2D and OpenGL ES for 3D) consumes less power than running 3D acceleration throughout the application lifecycle.

Planning and tracking memory management

Understanding and accounting for the different memory requirements of your embedded graphics application is key to performance and user experience. Not only do application slowdowns, freezes, and crashes negatively impact experience, they are extremely difficult to isolate and debug.

Your choice of system-level memory (heap, stack, and static code), and hardware-level storage (flash and RAM) impacts application performance and thus, user experience.

1. Embedded RAM is faster than embedded flash memory, so a general rule of thumb is to store dynamic content in RAM (such as variables and framebuffers) and more persistent data (such as code) in flash.

2. Using solid shapes, fills, and polygons for graphics uses less memory than loading image files, however they do impact runtime performance and should be carefully considered.

3. Don’t forget about the framebuffer. Offscreen pixel storage is just as important as onscreen and must be included in your memory planning, especially when screen transitions and animations are being composited.

4. If you know how much space your variables and stack are going to require, they can be pre-allocated in advance to determine the most efficient data typing and packing strategy.

Above all else, make sure you profile and understand how memory is being used at all stages of the application’s runtime:

“By not tracking where memory is going they can quickly run out of code space or run-time issues related to the heap being fragmented.” (7 Tips for Memory Management, Jacob Beningo)

Conclusion

As we see more embedded GUI teams move into IoT devices, we also see unique challenges. Developing these products successfully requires the expertise described here and strong collaboration between UX designers and developers to align expectations and needs.