Azimuth Systems compared the four national wireless operators’ network performance based on device/network interactions, and it has released its results in a case study.
The most pertinent information — which operator had which performance levels — was not released, but the findings nonetheless allow for some interesting guesses. The test was conducted with four commercial Samsung Galaxy S4 devices in a Boston suburb where all four operators have LTE coverage, according to Azimuth, and the devices were tested simultaneously by running a series of applications that included FTP uploads and downloads, video streaming from YouTube and Web browsing.
The testing examined KPIs from network Layers 1 through the application layer, Azimuth said, and relied on the company’s Analytics benchmarking solution to collect and analyze the data.
Azimuth said that “clear performance differences were observed, which led to insightful and in some cases counterintuitive results.” It highlighted the following:
- The fastest network was 20% to 50% faster than the others; was the most likely to offer best battery life for devices; and also had the most device-network interactions — the latter would normally be expected to slow down performance, but did not.
- The operator with the best app throughput also had the highest number of device/network interactions.
- One operator had devices spend 40% to 60% of their time on 3G rather than LTE, “potentially due to the network settings among other factors,” Azimuth concluded, and added that the “relatively high time spent on 3G had a direct impact on user experience.” The other three operators’ networks had the device connected via LTE for more than 90% of the time.
- One network had a “very good” rate for over-the-air data connections, “but couldn’t extend these gains to the subscriber,” Azimuth said, “resulting in low user perceived data rates.”
“Ever increasing network and device complexity has increased the impact of device-network interactions on the subscriber experience,” said Vivek Vadakkuppattu, director and head of Azimuth’s Analytics business unit. “Traditional testing has not yet caught up to this new reality and instead remains focused on device-specific or network-specific KPIs.”
The results from this case study, he added, “highlight that an understanding of device-network interactions can provide new insights into device and network performance, ultimately impacting end user experience.”
