Editor’s Note: You can find Part 1 of this piece here.
Measurement Scenarios
When defining the measurement scenario and setup, it is important to decide what connection should be measured. The test offers high flexibility regarding the position of the responding server. For an interactivity measurement of a mobile network targeting ultra-low latencies in 5G, the server should be located as close to the network edge as possible. This is facilitated by the use of the two-way active measurement protocol (TWAMP), which is supported by selected infrastructure equipment. However, if the measurement target is a realistic emulation of a real- time eGaming experience, it is better to place the server in the cloud or in a private network where eGaming service providers are usually located.
To give full flexibility to customers and users, mobile network testing products should offer a lightweight Linux VM installation that can be installed on servers preferred by the user to reflect the targeted test situation; so that they can be installed anywhere in the cloud, private network, or even on another smartphone as long as it has a public IP address.
Interactivity Test Accuracy
To evaluate the measurement accuracy that can be achieved in an ideal network channel, a reference setup with extremely low latency was created. A smartphone with the interactivity test running on a Rohde & Schwarz QualiPoc handheld was connected directly via Ethernet (over a USB interface) to a Linux server where the session reflector was located. The whole setup was standalone and not connected to the Internet so that no background data could interfere with the test.
In this reference setup, more than 1,000 tests with the eGaming real-time pattern were conducted, and very low latencies, packet delay variation, and a high interactivity score were achieved. Specifically, an average two-way latency of 1.26 ms ± 0.01 ms was measured. The average interactivity score is 99.6 %. The analysis of outlier ratios shows that only 1 in 5,000 packets arrived more than 10 ms later than the median round-trip time (RTT). Overall, the
measurement accuracy is high enough to measure round-trip latencies of down to 1 ms and thus it is suitable for URLLC measurements.
Results of Real-Time eGaming Testing
Measurements conducted in good network conditions in 4G/LTE with a fast, well-connected in-country cloud server showed median round trip latencies of approximately 40 ms down to 25 ms. Under good network conditions, no packet is lost or delayed beyond the packet delay budget of 2 × 50 ms, and the channel QoS is therefore 100%. The interactivity score had values between 75% and 85% due to the rather long round-trip latency for real-time applications. However, even in this range, a fair to good real-time eGaming experience can be expected. Measurements conducted in good network conditions in 5G EN-DC mode with dual connectivity showed that the latency is typically only a few ms less than in 4G networks. In addition, the packet delay variation is also a few ms less compared to LTE. The interactivity score is thus higher and in the range of 85% to 90%, which indicates a good to very good real-time eGaming experience.
The reason for the rather small difference in round-trip latency between measurements in 4G and 5G EN-DC is that the underlying radio technology and core network are still the same. Therefore, there is no native benefit to be expected for latency measurements. It is expected that a clear benefit of the 5G technology will only be visible once the Standalone mode with the dedicated core network is available.
The interactivity test is especially valuable to detect short-term problems in the packet flow for highly interactive applications like eGaming; such problems include a temporal build-up of the latency, lost and highly delayed packets, and longer interruptions. These kinds of impairments are usually invisible to most types of application tests. However, using the eGaming real-time traffic pattern in certain scenarios highlights these issues. In one case, adding a second 4G carrier partway through the test leads to temporarily increased latencies beyond the allowed
maximum delay of 100 ms. Similar problems occur during handovers, and also when the 5G carrier is added. Overall, the median round-trip latency is still in a tolerable range, but high packet delay variation and packet loss results in an interactivity score of 0%. In these network conditions, it is not possible to play a real-time game. Carrier aggregation and aggressive cell changes maximize data throughput but at the expense of latency and real-time interactivity. This finding will have to be addressed by network providers as the importance of interactive applications is on the rise.
Conclusion
5G will usher in a new era of highly interactive services that will require ultra-high reliability and the ability for real-time interaction. For these applications, it is vital to measure and to rate the interactivity of a network and to efficiently identify bottlenecks throughout the network. The interactivity test combines several factors into a single test that can measure very short latencies with very high accuracy. The configuration is very flexible and scalable and can easily be extended to a large range of application types, including use cases such as eGaming.
