In the battle for the hearts and minds of mobile users, companies in the mobile ecosystem are judged by the quality of their services delivered over mobile data networks.

Billions of dollars are invested in building out networks and services capable of keeping up with the growing demand of our mobile society. To be sure that these costly and complicated investments are performing around the clock, leading companies have recognized the need for monitoring their mobile networks, applications, and services. But is the current system enough?

While most companies are familiar with inside-the-firewall passive monitoring, they aren’t seeing it from the end user’s point of view. Since the networks on which data travels are often very complex and intertwined, it becomes all the more important to see how things are working from the customers’ perspectives because the data can traverse multiple networks, third parties, and even competitive networks.

To get this critical information, mobile performance monitoring should include active testing, which provides real-time data on customers’ mobile content-delivery experiences at any given moment.

“The whole point is that this is an area of huge importance for service providers,” said Ari Banerjee, an analyst with The Yankee Group in Boston. “Service assurance, service management – at the end of the day, it’s all about how the network is performing. Especially for wireless providers, this is a huge differentiator. It’s critical that you know how your customers are experiencing your service.”

When it comes to mobile performance monitoring, there are two distinct yet complementary approaches designers need to know about: passive monitoring and active testing.

Active testing and passive monitoring comparison

Passive monitoring allows watching what is occurring on a live system without actually sending out data to replicate what customers are doing on the systems. Active testing goes further by actually sending out requests just like customers would and analyzing the round-trip performance.

Passive monitoring is performed on random services and customers on the network inside the corporate firewall to give snapshots of network performance, but because it is random, problems that are uncovered can’t be replicated. Because the monitoring is done within the firewall, it can’t give a true sense of what is going on where the customers are – outside the firewall.

Active testing such as Keynote’s Mobile Data Perspective strategy illustrated in Figure 1 allows not only finding performance problems, but also easily reproducing them so their causes can be determined and addressed. Only by seeing exactly what customers are experiencing will designers be able to determine the performance customers are getting in the real world. With active testing, an end-to-end view of what is happening between the server and end users is obtained; the testing data is collected outside the corporate firewall, where the customers are, so one can truly walk in their shoes.

Active testing gives deeper performance insights

While active performance monitoring strategies are important to the core mobile network, they may be even more so when it comes to mobile services, according to Charles King, principal analyst with Hayward, Calif.-based Pund-IT Inc.

“Mobile communications and computing are fundamentally changing the way that people exchange information and do business, so it’s critical for mobile service providers to make certain that their networks are performing as they should be,” King said, adding that service providers can reassure customers they’re getting what they pay for.

Although passive monitoring can be valuable, active testing is much more helpful and informative, King asserted. “In a real sense, active testing can allow users to keep their fingers on the pulse of their mobile networks, to be alerted when problems arise and to troubleshoot and resolve those issues before they negatively impact customers,” he said. “Active performance testing qualifies as one of the keys to enabling and ensuring customer satisfaction.”

A hybrid approach is critical

Jim Frey, an analyst with Enterprise Management Associates, Inc., said that the best monitoring approach is a combined one.

“In my view, you ideally should employ a hybrid approach, mixing both active and passive testing,” Frey said. “Active testing is essential because that will give you the ability to understand availability and responsiveness of your most important mobile applications, sites, and services, regardless of whether or not someone is currently using them. Passive testing is important because you can never fully anticipate every potential way in which those assets will be used, nor can active testing truly replicate a real human. It’s an approximation at best, even when it’s a good approximation. This hybrid approach pays great dividends in the broader Web services assurance world.”

In mobile uses, the devices themselves can determine what kinds of monitoring can be used, Frey said. Sometimes there is no way to passively monitor a mobile terminal, especially when incorporating smartphones and intelligent (but non-PC) mobile terminals. In those cases, the need for a strong active testing strategy becomes critical, Frey said.

All about experience

Passive monitoring is needed for troubleshooting performance issues, analyzing trends, and planning for future growth. Chances are passive monitoring is already deployed in the network and infrastructure using probes or through instrumenting the infrastructure to sniff traffic. However, passive monitoring solutions on their own have limitations in gathering the insight needed for optimal service delivery assurance because they do not extend to the end customer.

To capture the true end-user experience, any testing solution needs to extend beyond the core network and operations firewall, measuring the data transaction from the point of initiation to its final destination. The testing solution must have both an active and passive component, with active testing complementing existing passive network testing capabilities. Doing less paints an incomplete portrait of the mobile landscape and puts the customers’ experience at risk.

Nisheeth Mohan is the product manager for mobile test and measurement products at Keynote Systems, where he is responsible for the strategy and development of new mobile products. Prior to Keynote, Nisheeth held product management roles at Telephia, now Nielsen Mobile, where he built some of the industry’s first market research and audience measurement products for mobile media. He holds a BS in Engineering from the University of Mumbai, India, and an MS in Electrical Engineering from the University of Texas at Arlington.

Keynote Systems
650-403-2400
www.keynote.com