New study compares performance across 5G devices and chipsets, and gives a glimpse of VoNR
Do you need the latest and most expensive 5G phone to get good 5G performance? Not according to a recent benchmarking study of 5G devices and chipsets, courtesy of Signals Research Group.
According to a preview of the report, which SRG said that it believes is the first independent study of its kind, SRG tested nine different smartphone models (four from Samsung, two from LG, and one each from Apple, Motorola and OnePlus), using 5G New Radio baseband modems from three different suppliers (Qualcomm, Samsung and MediaTek). The testing relied on T-Mobile US’ 2.5 GHz 5G Standalone network, both because of the global interest in using midband spectrum for 5G and because SRG anticipated 5G performance differences among the devices and chipsets would be more apparent across a 100-megahertz TDD radio channel. All of the devices except the Apple iPhone 12 supported 5G SA; SRG filtered results to ensure that all of the phones were receiving data on 2.5 GHz and attached to the same cell site for an apples-to-apples comparison, and also conducted tests on devices one at a time, and with multiple devices at once, while stationary or moving around.
SRG said that what it uncovered was a mix of the predictable and the surprising. First off, no single device or chipset stood out markedly from their peers; the differences in performances were more nuanced and were highly dependent on the radio frequency conditions and the specific test, SRG said. However, over the full span of the RF conditions that SRG tested, the Samsung Nokia 10 Plus—which came out more than two years ago and doesn’t have the latest-and-greatest modem—generally did the best. SRG also concluded that “list price is a poor indicator of [a smartphone’s] 5G NR performance. “We found a $380 smartphone performed just as well as a $1,200 smartphone over a wide range of RF conditions,” SRG said, although it added the caveat that more expensive 5G devices tend to have a more advanced processor, which does provide some benefits under ideal RF conditions and at very high data transfer speeds. “Obviously, other features and functions contribute to the price differences between the two smartphones, but better RF performance isn’t necessarily one of them,” the report said.
SRG said that while it’s difficult to attribution good or bad smartphone performance specifically to the underlying chipset, it did draw some system-level conclusions—primarily around the different strategies that Qualcomm, MediaTek and Samsung use to maximize performance that, based on SRG’s observations, ends up “in many cases … very comparable” once you get to the handset + chipset, system-level outcomes.
SRG conducted its stress-testing on T-Mobile US’ network in downtown Los Angeles and stationary testing in Buena Park, as well as some additional testing in and around Minneapolis, MN, using test solutions and analytics from Accuver Americas and Spirent Communications.
Intriguingly, the testing consultancy also turned up evidence of pre-launch Voice over New Radio on T-Mobile US’ network. Part of SRG’s stress-testing involved establishing a full-buffer downlink/uplink data transfer and then placing a call to the smartphone under test. In most cases, SRG reported, the smartphones switched to 5G NonStandalone mode to take the VoLTE call (“The impact on throughput was very interesting to say the least,” SRG noted) — but in a tantalizing preview of Voice over New Radio to come, the fact that SRG had inadvertently turned off some of the LTE bands on the S21 Ultra led to a working VoNR call on T-Mobile US’ network.
Read more from Signals Research Group here.
