As industry focuses on mobile video, OpenSignal intros new metric

OpenSignal introduces new mobile video experience metric

Video traffic is taking up more and more capacity on mobile networks, and mobile network operators are grappling with how to handle the impacts on their capacity and operations. They want users to have a good experience watching video on their mobile devices, yet struggle with how to handle the enormous amounts of video traffic. Cisco has predicted that by 2020, mobile video will account for more than 75% of the global mobile data traffic.

As the industry is forced to reckon with the impact of video, testing company OpenSignal has launched a new metric focused on end-user experience of mobile video — which the company says not only provides new insights into the customer experience, but eventually might be able to give perspective on how well operators are balancing the video user experience with overall traffic management.

“We think video is massively important,” said Brendan Gill, CEO and co-founder of OpenSignal. “Every operator we speak to in the industry is focused on video right now. When it comes to mobile, it’s the majority of what people are doing on their mobile devices, the vast majority of the traffic out there, and it’s only growing and becoming a bigger issue over time.”

So, he said, the company developed what it’s calling the first independent measure of real-world, measured network experience when it comes to watching video on a mobile device. As OpenSignal typically does, it is a device-based measure from millions of devices. Gill said that OpenSignal uses video streaming from major providers such as YouTube and Akamai, and then measures a series of parameters related to the video viewing experience: loading time, buffering rates, picture quality; the resolution, the bit rate and the codec used, among other things. All of the factors are condensed into a single score based on an International Telecommunications Union methodology for assessing video user experience, to become a “concise summary of exactly what the video experience is for the typical user.”

“The device plays a big factor, and we think it’s really important to capture that,” Gill said. “Our tests run over the full spectrum of devices out there, and the variations in experience that you get as a result of your device is averaged into the overall metric,” he added. “And not just the devices — how you use your phone, how you hold it, the location you use it” — after all, he said, most mobile use is indoors now.

“All of this is the real-world experience that an individual has, on a real-world device, in a real-world situation,” he said. “We’re not doing simulation or inferences. We are directly observing what people see and laying analytics on top of this, to turn it into a report.”

Does video really have a need for network speed?

In terms of the report’s findings, OpenSignal said that users have the best mobile video experience in the Czech Republic (with a score of 68.52 out of 100) followed by Hungary (67.89) and Norway (67.41). The U.S. was near the bottom of the rankings, with a score of 46.84 out of 100. In this first global report, no country or carrier obtained an excellent rating — they maxed out at “very good” or “good.”

“We think there’s work to be done everywhere,” Gill said. 4G, he said, is a huge part of that and “provides a good experience for the most part, but there is still work to be done.”

“When you load a video with  quick loading time and you’re going to have a low chance of stalling, and decent picture quality, ultimately that’s what you want as an end consumer,” he said. “The other thing that we’ve seen is that there’s not necessarily a really clear-cut relationship between speed and video experience.” 

Perhaps surprisingly, OpenSignal found that “once a country passes the 15 Mbps threshold in average overall download speed, the raw power of connections has little bearing on streaming video quality.”

“What we tend to see near the low end, speed is important — to a certain level of speed,” Gill said. “But as you get toward the high end, that relationship gets very loose. … To have very fast video start-up, you need a responsive network. You need really low latency. That’s a more important factor when it comes to the load time. When it comes to buffering, you don’t need a super-fast network, but a network continuously offering you a decent speed — it’s more about the consistent speed or bandwidth the network can provide. The bottom line is, speed is important up to a point, but it’s really not the only factor. There are other ones that are really important,” Gill said.

So the networks that claim the fastest speeds, he said, don’t necessarily offer the same thing as a good network experience when it comes to things like watching video — which is reflected in OpenSignal’s rankings. It offers a reminder that mobile network user experience is far more complex than speed alone, despite speed often being used as shorthand for a good end-user experience.

“We see this as a massive step toward moving away from just speed as the sole measure of a network,” Gill said.

Does “differentiating” video traffic actually do what it promises?

Gill said that the video metric is particularly interesting for the U.S. mobile industry right now, when “all operators are getting into a level of traffic management applied to video in particular, and that really means its effect on the experience that consumers are getting.”

Mobile network operators’ traffic management policies have come under scrutiny of late, with app-based research identifying that users’ video traffic is, indeed, sometimes being “differentiated,” or throttled. All four U.S. carriers have been identified as using traffic differentiation for video; some of them use it as a marketing tool, framing speed limits for video as a unique offering (such as T-Mobile US’ Binge On) or a way to stretch out users’ high-speed data allowances under unlimited plans (AT&T’s Stream Saver).

“Today in the U.S., the majority of operators have some level of this going on — all of them having it to different degrees,” Gill said. “We think that’s important to take into account.”

After all, he said, the idea of offerings that limit video bandwidth is that video watchers are still getting an acceptably good experience on a very small screen, while maintaining good experience for users who are doing other things on the network. Gill said OpenSignal’s opinion is that throttling in and of itself is neither good nor bad, but a recognition of the balancing act that operators face between revenues and the capacity demands that video is placing on their networks.

“The equation is not balancing today, and operators need to find a way to do that,” he said. “The choices are only, earn more revenues from customers, or save money. I think it’s important to look at, and ultimately, that’s what we’re going to do. Can these traffic management policies be put in place without impacting the video experience of people, and what does it mean overall in the … availability of the network and the speed?” In other words, are operators actually offering a better experience for everyone, by limiting video?

The new video report, he said, “will hint at some of these, but answering that is really a question for the industry, and over time the report will be … able to add more color to that.”

Gill also added that there is a perception that mobile operators “are providing a commodity service, and what we get from one operator is the same as another, it doesn’t really matter … let’s pick the one that has the best price. … If an operator is to invest in providing a better experience — and that takes money — ultimately then, they’ve got to have a way of differentiating, and not be perceived as a commodity service.” Mobile video user experience can potentially be a key differentiator.

Gill also said that the perception of mobile being a commodity product is important in the context of merger talks in the U.S. between Sprint and T-Mobile US.

“It’s very easy to say, if operators are a commodity service, then you need more of them. Have less means less competition,” he said. “In reality, they’re not a commodity service and if T-Mobile and Sprint combined can provide a better network that provides more competition to AT&T and Verizon, then that’s a good thing overall. I think that’s something that’s often missed in terms of considering the mobile operators against each other.”