With latency becoming an increasingly important part of enabling new services and monetizing 5G, just what does testing to assure low latency look like?
Rich McNally, senior director of mobile service strategy with Spirent Communications, discussed real-world observations and lessons on latency testing at the recent Mobile Edge Forum virtual event. Here are five key takeaways from his observations.
1. Testing generally falls into three types, at different points in the life cycle. McNally said that Spirent breaks out testing in 1) design and launch assessment, 2) continuous network monitoring and 3) customer experience management.
Service design and launch might include something like an edge zone launch, he said, and would include testing that includes security assessment and also answers business questions like, is the service ready for launch in the given city or market, and how does it compare, performance-wise, to the competition?
Continuous monitoring including typical service assurance and network change management testing, as well as service-level validation. But in changing networking, he said, Spirent is seeing that there is “an even more critical need for latency, segmentation and fault isolation, particularly due to the partnerships between operators and cloud providers, and the importance of quickly triaging where latency issues exist within hybrid networks.”
Additionally, he said, “Realizing that potential of edge services is going to require attention to making your enterprise customers happy. … It’s really important to be able to model mission-critical enterprise app traffic ahead of launch, but also [ensure] that there’s a way to address customer support and performance issues that come up along the way, and to base that on quantified controlled results, and not just anecdotal complaints.”
2. Different tests have more or less usefulness in an edge scenario—but new approaches are being developed as well. Ping roundtrip time has long been the most common option and still has its usefulness, McNally said–”But it really has its shortcomings,” he added. Given routing differences in traffic types, pings can miss the priority given to normal edge traffic because the ping itself is routed differently than application data. “It also doesn’t delineate between up and downlink latency,” McNally said. “This is important because different edge applications can have different dependencies on either direction. And if you’re not measuring it, you’re guessing, really.”
UDP (User Datagram Protocol) one-way measurements make up for some of these shortcomings, he added, since application data is routed via UDP and you can also get separate statistics on uplink and downlink latencies. McNally noted that packet sizes can be varied with UDP tests and emulation of different use cases and traffic profiles is possible. Using both ping roundtrip and UDP one-way measurements can cover a lot of bases for launch assessment, benchmarking, enterprise application emulation, and various service assurance use cases, he added.
But he also highlighted a new approach being worked on by the Broadband Forum, which Spirent has implemented, called DeltaQ or QED. It is UDP-based and separates uplink and downlink statistics, and “goes another step further by allowing you to see latency contributions from geographic distance, from serialization/routing, and from network loading,” he explained. QED also ties in the concept of a quantitative time agreement, or QTA, in order to realize SLA commitments with customers, McNally said.
3. In real-world testing, edge zones successfully provide lower latency. Spirent conducted tests on publicly available edge services in various part of the world on a regular basis, and recently tested four cities and two networks in North America and Japan, using commercially available handsets. Latency ranged from less than 10 milliseconds to the low 20 ms for the downlink, and less than 10 ms to the high 20-ms range in the uplink for 5G traffic only.
But consistency of latency was problematic—and that’s important to a stable low-latency service. Downlink latency showed, generally, more consistency in cities tested in the U.S., while uplink latency was more consistent in Japan; however, the specifics varied by city.
4. There are multiple factors that impact latency, and some of them are going to persist for awhile. These include handoffs between different access networks to serve target customers in various geographies and challenging radio frequency environments for MEC applications with high mobility. McNally said that one big lesson that Spirent has learned is that service providers have to plan ahead to avoid delays in testing that can happen due to network security triggers in both edge and cloud zones. “You have to be careful not to trigger denial of service sensors and things of that nature” where normal testing procedures could accidentally set off a service shutdown.
5. There is still a lot to learn when it comes to enabling edge services—but good testing practices will help answer questions and reveal areas to work on. McNally said that active testing, both end-to-end and within a specific networks, is “really critical to “capture issues before users do” and that testing capabilities need to be at-the-ready in order to quickly turn around solutions when issues arise.
McNally said that when it comes to latency, consistency is critical—and that in some cases, it may eventually overshadow throughput as the most important aspect of a service. With all these new edge services and applications that could come to be, there is no single latency standard that will suit everything. “When testing, it is important to model different data footprints of important applications. There’s not one test case that covers everything out there,” he said.
For more on-demand content from Mobile Edge Forum, go here.
