The voice of machine learning starts and ends with humans

By Mark Benson

Head of Engineering

Samsung SmartThings

March 02, 2017

The voice of machine learning starts and ends with humans

The Internet of Things (IoT) represents new opportunities for manufacturers to capitalize on the value of data for their business. One of those opportunities is through leveraging an approach...

The Internet of Things (IoT) represents new opportunities for manufacturers to capitalize on the value of data for their business. One of those opportunities is through leveraging an approach called machine learning, which is a branch of artificial intelligence that enables machines (or virtual representations of machines in the cloud) to learn new behaviors based on their external environments, internal health, and changing inputs. However, in order for machine learning to work, humans must be able to grok the context of how the machine data is collected, aggregated, and consumed.

Machine learning algorithms are nothing if humans do not understand how they work or the business results they aim to achieve. For example, a building automation system could benefit from a machine learning algorithm enabling it to learn about optimal operating conditions based on external weather conditions, occupancy rates, and seasonal fluctuations, and then use those learnings to optimize the operation of the HVAC system to reduce energy use and improve the comfort of occupants. This type of machine learning takes time to develop and mature, and also requires smart humans to understand how the building automation system works, the nature and frequency of the data being collected, and what types of insights can make it easier for people to do their jobs more effectively.

Finding the voice in machine learning

Analytics enable humans to give machines a voice – the ability for the machines to provide or derive insights based on data. Making machines more human with a voice in this way doesn’t happen overnight, but the road that leads there starts with basic connectivity and data collection. Once data is collected, further levels of analytics maturity may be achieved, including the ability to remotely describe a problem, diagnose the root cause of a problem, and predict a problem that may happen in the future. The following diagram shows the most common progression that manufacturers take when building a machine learning capability to give voice to their device fleet data.

[Figure 1 | The voice of machine learning]

As the maturity of a machine learning improves for a given device fleet, the level of human interaction decreases, the cost of collecting the data decreases, the rate of data collection can become optimized, and the resulting value improves significantly within the context of the following stages:

  1. Basic data collection. Machines are instrumented with sensors to collect measurement points of possible interest and to enable remote configuration and control of the asset.
  2. Descriptive analytics. Enough data is being collected to enable basic descriptive analytics, including ascertaining how a machine performs under load, or behaves just prior to a failure.
  3. Diagnostic analytics. If the number of sensors and the rate of data collection from the sensors is sufficient, remote diagnostics are possible to determine what things went wrong leading up to a failure condition and what the possible remedy to fix it might be. At this stage, it is possible to introduce machine learning algorithms that help a machine become better at diagnosing its own problems by looking at patterns from previous failures from peers in the same device fleet.
  4. Predictive analytics. With a statistically significant number of observed failures, it is possible for an analytics model to forecast problems before they occur. Machine learning algorithms may be used in this stage to continuously learn how to better predict future failures based on a class of observed behaviors from other machines of a similar type.
  5. Prescriptive analytics (cooperative). Cooperative prescriptive analytics provide recommended remedies to prevent a future machine failure. For instance, if a piece of rotary industrial equipment has faulty ball bearings and is exhibiting high vibration levels, a cooperative prescriptive analytics algorithm might suggest an accelerated maintenance schedule for the machine so that the ball bearings can be replaced and avoid unplanned downtime. In this way, the algorithm is cooperative because it still relies on a human to intervene and fix the machine.
  6. Prescriptive analytics (automatic). In some situations, automatic prescriptive analytics can be used to enable machines to not only predict what a future failure might look like, but to take autonomous active steps to self-diagnose the root cause of the problem and to apply a remedy without requiring immediate human intervention. Examples of this might include a self-calibrating feature, or machines that enter a lower-efficiency or lower-power mode to prolong a failure scenario, thereby lengthening the window of time for a long-term remedy to be applied.

Curative versus preventive actions

In Figure 1, steps 1-3 can enable what we might call curative actions. Curative actions are things which can be done after an event has occurred. For example, if a pump fails, descriptive and diagnostic analytics can tell an operator what went wrong and how it might be fixed. This information enables the operator to apply a curative action to resolve the problem and put the machine back into healthy operation.

By contrast, steps 4-6 can enable what we might call preventive actions, and are often the strategic goal of an analytics program for connected machines. Preventative actions are things which can be done to proactively forestall nefarious failures before they occur. For example, if a pump is vibrating, getting hot, or the flow rate is lower than it should be, predictive or prescriptive analytics can tell an operator ahead of time what preventive action should be done to resolve the problem at an off-peak time.


No matter what insights machine learning algorithms reveal, only humans can determine answers to essential questions such as what critical business problems the organization must solve. In this way, the entirety of machine learning, and more generally the financial success of a connected product offering, depends on human’s ability to understand how machines work, what sorts of data must be collected and analyzed, and how algorithmic results should be interpreted. The real winner of machine learning then is neither the machine alone, nor the human alone, but the two working together collaboratively.

Mark Benson is the Chief Technology Officer at Exosite. In this role, he navigates emerging IoT developments and leads the company’s technology. Over the course of his career, Benson has led highly-creative and multi-disciplined teams in the development of advanced embedded product designs for industrial, medical, aerospace, military and consumer industries. Additionally, he has authored the book, The Art of Software Thermal Management for Embedded Systems, and several technical papers and articles, and he is a regular speaker at industry conferences. Benson holds a bachelor’s degree in computer science from Bethel University in Arden Hills, Minnesota, and a master’s degree in software engineering from the University of Minnesota.

Accomplished C-level technology leader. Experienced in strategic, operational, and financial management of cross-functional teams in highly-competitive markets. Successful track record of defining and realizing pragmatic product strategies to meet the requirements of complex enterprises across multiple functions and verticals. Strong interpersonal and formal communication skills.

More from Mark