When you can continuously test network software, how does RF testing also need to change?
CI/CD is a tool of the IT world, of hyperscalers. Mobile network operators have worked for years to remake themselves in that image, to become software-centric in order to achieve the same scale and agility. But MNOs are MNOs precisely because they have something that no one else does: A Radio Access Network. And for all that it is invisible to the eye, RF is ultimately a physical interface.
“The cloud-scale companies like AWS, Azure, and Google Cloud, they live this world,” says David Woodcock, who handles product strategy for test orchestration company ACentury. “They’re used to frequent, small changes, multiple times a day. And the mobile network operators are going to need to be able to get to that as well, to take advantage of the agility and the capability of cloud-native networking. But they’re handcuffed to the fact that they’ve got this giant physical asset of a wireless network that is going to need to be able to be tested at the same kind of speed.
“CI/CD is all about automating all the test suites – which is really easy when what you’re testing is virtual,” he continues. “But the radio layer is very much physical.” That often means manual test set-up in a lab that mirrors, as much as possible, the production network. “You can’t really do continuous integration and continuous deployment, with continuous testing in the middle, if you have to stop doing what you’re doing and re-align your lab to do the next test,” Woodcock says. “RF is different. I’ve seen it many times in my career where people from the wired IT industry assume that the radio link part of any network is just like an Ethernet cable, and it’s not. So you need to incorporate that layer into your end-to-end test – and right now, it’s the limiting factor for how fast you can test.”
He explains: “When [carriers are] doing new feature testing or regression testing, when Apple releases a new iOS update, or there’s new Android firmware, or Network Vendor A releases a quarterly patch to their infrastructure, they have hundreds or thousands of test cases they need to run in their labs to make sure all that stuff works, still. And that involves doing 100s of mobile originated phone calls and file downloads and video calls all while simulating roaming between all their different site designs and vendor combinations and to do one of those tests it sometimes takes them hours or days to set up the RF environment in their lab to do a test that takes 20 minutes.”
Because there is so much to test, he continues, he has seen firsthand that there can be time-to-market delays in, say, being able to sell a flagship device because the lab isn’t available. “It’s an ongoing challenge. It’s going to get worse, too,” Woodcock says. Not only does every “G” add new spectrum bands and increase the complexity of the RF environment (and add to test time), the emergence of Open RAN and the pushing of disaggregation into the RAN adds yet another layer that needs to be integrated. “The number of vendors involved is an order of magnitude higher, and who knows what their cadence is going to be for when they release their driver updates?” He foresees that operators will manage that complexity by limiting the number of vendor combinations they allow in their networks and demanding that those vendors work together ahead of time on integration. But, he says, “They’re going to need to build a testing infrastructure that allows them to play with those combinations and try to find ways that they break, and prevent that. And [do it] fast.” Because otherwise they lose the very speed and agility that they have been working to achieve by implementing software so deeply and broadly across the network.
Carriers, Woodcock says, “honestly have had their hands full with the transition to cloud-native and virtualized networks. … They were fairly comfortable with how they handled the RF part of their worlds. But … more and more now, they’re starting to think, this is now the limiting case in how we accelerate. So they need to rethink about how to speed up the RF portion of the end-to-end test.”
For more insights on CI/CD and continuous testing in telecom networks, check out this on-demand webinar and download the RCR Wireless News feature report The continuous challenge: Software, CI/CD and testing in the new network.
