Six teams share $750,000 in prizes in the second event of DARPA’s Spectrum Collaboration Challenge
LAUREL, MARYLAND–The moment came with the ding of a winning bell, cheers and a spotlight flashing around the auditorium at the Johns Hopkins Advanced Physics Lab, as the first team in the Defense Advanced Research Projects Agency’s Spectrum Collaboration Challenge watched its autonomous radios’ data push past a finish line projected onto a screen at the front of the auditorium.
That moment, according to DARPA program manager Paul Tilghman, was evidence that autonomous, artificial intelligence-powered radios working in a dynamic spectrum environment could outperform the status quo of human-assigned, static spectrum licenses that has dominated spectrum management for the past hundred years: exactly what DARPA’s SC2 challenge was designed to explore.
In short order, another five teams had also crossed the “payline,” as it was dubbed, to win their share of $750,000 in prize money for their achievement. This was the second preliminary event of DARPA’s three-year Spectrum Collaboration Challenge, which seeks to apply artificial intelligence to software-defined radios to create radios that can autonomously navigate complex, congested spectrum environments and work in concert with other radios to manage spectrum resources efficiently.
Fifteen teams participated in the event at Johns Hopkins APL, which had three rounds of testing that ultimately whittled the competition down to the six event winners — although all 15 are eligible to participate in the third year of the competition and its $2 million in prizes. Test scenarios included 10-node networks competing for sufficient spectrum to serve up voice and data communications for a military squad accompanied by drones, protect a ground-based satellite incumbent, provide wireless hot spot access in a crowded outdoor mall as usage shifted over the course of a day, enable emergency communications for a wildfire ground crew and air tankers, and deal with increasingly sophisticated communications jammers — the last one, a scenario that was a surprise to the participating teams. They had submitted their final code for their software-defined radio code several weeks ago and only saw the final data yesterday, accompanied by play-by-play commentary from Tilghman and Gnu Radio President Ben Hilburn at the APL.
DARPA created a remotely-accessible wireless emulation testbed specifically for this challenge: the Colosseum, which is housed at the Johns Hopkins APL not far from the auditorium where the final match results were dissected. Colosseum can emulate tens of thousands of potential interactions among hundreds of various types of wireless communications devices, with its 256 software-defined radio units. It generates 1.88 TB of data per scenario model, according to DARPA, and with about 400 matches played, the official testing took around five days, according to Tilghman. After the data was verified, DARPA cherry-picked particularly interesting matches to showcase at the final event with its data visualization tool and Tilghman and Hilburn to break things down. The test data had been recorded and was played back at about 4x during the presentation to speed things along — except for two matches where data was pulled directly from Colosseum, Tilghman said. But the teams hadn’t seen the final results, and none of them knew whether their radios would successfully navigate all of DARPA’s hurdles.
DARPA didn’t just highlight successes; the commentators also analyzed numerous matches that delved into failures of one type or another.
“DARPA is about success and failure, so we like to highlight both,” Tilghman said. He went on to compare the challenge scoring to a graduate school test, where an average grade is often 50%. “We’re not expecting a perfect run,” he said.
In the third and final round that determined the winners, the teams’ radios were first assigned 5 megahertz of spectrum in a match-up with an ensemble of four other teams’ radios, in a traditional “walls-up”, static spectrum assignment, as a baseline measure of performance. Then, as Tilghman explained, the “walls” came down and every team had to prove that it could beat its baseline performance in a free-for-all spectrum environment. Because the challenge centers on collaboration, each team’s evaluation was based not just on their own radios’ performance, but whether they ensured that the four other teams in the ensemble could also outperform their baseline. Cheers and groans echoed through the auditorium as the data played out, with each team getting five attempts to succeed. Six of them did, and the rest now get a chance to tweak their work and compete for smaller “half prizes” in the next few weeks as part of the continuing program.
Paul Tilghman (left) and Ben Hilburn discuss a wildfire communications scenario. (Image: Kelly Hill/RCR Wireless News)
Tilghman said that since the first event last year, the teams have broken new ground on what constitutes the state of the art for AI-powered radios. The results that were made public yesterday, he said, show both enormous progress from the first preliminary event last year as well as “glimmers of success” for 50 radios to successfully and dynamically share complex radio environments.
The final event of the challenge will be held at Mobile World Congress Americas 2019 in Los Angeles, which means packing up Colosseum and transporting it to California from Maryland. Tilghman has already been out to the Los Angeles Convention Center to scout the location and figure out how to get Colosseum there and running — which, he said, is comparable to operating a data center server farm on the show floor. There, the 15 teams from the preliminary event will get another chance at the $2 million in final prizes, in an event that is anticipated to ask even more of autonomous radios — and offer a glimpse of what is possible in dynamic spectrum management.
https://twitter.com/DARPA/status/1072939006206377985
https://twitter.com/DARPA/status/1072972298431406080
