Spirent Communications released the latest results from its ongoing analysis of customer experience on LTE networks in the United Kingdom, including new data from a dozen cities. They ranked O2 as the top U.K. operator for customer experience.
This is the second round of the ongoing assessment of the LTE mobile networks of Vodafone, EE, O2 (which commissioned the ongoing study) and 3. They were tested in those 20 cities, using Spirent’s Datum user analytics solution and Apple’s iPhone 5C devices that Spirent purchased at retail stores. Rankings were made on the basis of more than 175,000 data tests that examined speeds for Web browsing and file uploads and downloads, along with data service reliability, according to Spirent.
The first round, conducted earlier this year, also ranked O2 first.
Coincidentally, British telecom regulator Ofcom also released results of a similar customer experience study this week with slightly different results. Ofcom examined both 3G and LTE mobile broadband networks in five British cities, and its findings gave EE the fastest average 4G download speed with O2 in second place, and had O2 tying for third place in Web-browsing speeds. Ofcom said that it relied on about 210,000 LTE and 3G tests for its data.
Jeff Atkins, director of service experience solutions at Spirent, explained that different methodologies account for the different results, but both are valid network snapshots from earlier this year and were in fact both conducted using Spirent’s Datum testing solution for assessing the mobile customer experience. Both Spirent and Ofcom tested mobile broadband indoors and outdoors. Atkins said that Spirent’s testing also included a variety of mobility circumstances: stationary tests, walk tests and drive tests. He also noted that the two approaches had different methodologies in regards to uploads and downloads of defined file sizes vs. burst testing with large files to assess the top throughput.
Both Spirent and Ofcom’s testing showed that there were, in most cases, only slim differences in performance across the four mobile networks, even if they had differing numbers as to what that performance was. Spirent’s numbers were significantly slower than the speeds found by Ofcom. Atkins said that Spirent’s methodology of including stationary and moving tests influenced this, as moving tests typically result in slower network speeds.
The findings included:
–Web browsing: Spirent pegged Web page upload speeds at 2.4 seconds for O2 and 3.2 seconds for the slowest network. Ofcom, meanwhile, found browsing speeds of less than 1 second for all operators, with 3 registering the fastest speed at 0.62 seconds and O2 and Vodafone tying for third place at 0.82 seconds.
–Download speeds: Ofcom put the average LTE download speed at 15.1 megabits per second, with EE registering the fastest speed of 18.4 Mbps, O2 above average at 15.6 Mbps and 3 the slowest at 10.7 Mbps. Spirent, meanwhile, said that O2 averaged 10.9 Mbps across 20 cities and the slowest LTE network in its testing chugged along at just 5.7 Mbps.
–Upload speeds: Ofcom found LTE upload speeds for EE and O2 at 14.7 Mbps and 13 Mbps respectively, with Vodafone at 11.4 Mbps and 3 at 11.1 Mbps. Spirent’s approach put average file upload speeds on the slowest LTE network at 3 Mbps and O2 in first place with 5.5 Mbps.
In addition, Spirent looked at what it called “reliability,” or the ability to connect to an operator’s LTE network; and Ofcom looked at latency, which Spirent considered included in its overall speed assessments.
Atkins added that although O2 commissioned Spirent to do its testing on an independent basis, it also requested that the testing be audited by the British Approvals Board for Telecommunications in order to verify that the results were accurate and unbiased. The audit included the review of methodology as well as observing a set of measurements being taken.
In general, Atkins said, the focus on customer experience testing is an emerging trend in both the U.K. and wherever LTE is being deployed.
“This is part of the big trend that we’re seeing,” said Atkins. “Where in the past, operators used to do very technical, engineering-type of measurements of their networks to figure out how well it was working … the big shift is around trying to really accurately understand exactly how the customer would experience the service.”
He added that carriers are interested in the customer experience data from both a marketing point of view and to make network improvements, and that regulators such as Ofcom seek to use it to help consumers better understand the services they are getting.
Spirent’s study started out with an initial round of measurements from May through July, and added new customer experience tests in August through September. The dozen cities that had new data in the second round included London, Birmingham, Brighton, Bristol, Cardiff, Coventry, Leeds, Leicester, Liverpool, Manchester, Plymouth and Wakefield, according to Spirent.
Ofcom’s study looked at 3G and LTE coverage by the four operators in Birmingham, Edinburgh (included in Spirent’s initial testing round), Glasgow, London and Manchester.
This story has been updated from an earlier version.
