AT&T’s AI strategy suggests the real breakthrough is not bigger language models, but smarter orchestration. As inference becomes a network workload, telcos are starting to hint at how enterprise AI will scale efficiently, economically, and everywhere.
Very quickly, because it’s Friday, and I am going camping in the New Forest with a boy and a dog. This blog by Andy Markus, chief data and AI officer at AT&T, has some good numbers in it: AT&T is processing 45 billion inference tokens every day, for the record, and also training its own AT&T-specific models on more than one trillion tokens – including 400 billion tokens to post-train its latest open-source OTel 2.0 model. Which is a brand new network workload – at that kind of scale, surely, versus a sand-pit software project. Which is probably the most interesting thing here: less the language model (although a telco-one is an achievement in itself, and bellwether for industrial AI take-up), and more this ‘AI gateway’ mechanism that chooses which model to use. Instead of throwing every query at the biggest and most expensive model, it routes jobs to the cheapest one – which can do them properly.
It even switches models halfway through a conversation if it makes sense. AT&T reckons it is cutting AI costs by up to 90 percent – so saving millions, apparently. That has to be the key shift. The top of the market, pulling in the cash, digging up the planet, is still mostly talking about bigger models. The penny-drop, literally, for telcos and every industry is to run AI economically at scale. The same conversation is happening, related, about GPUs and CPUs – which works, well enough, and not just which is best. Which is exactly how telecoms works, anyway. Networks are not generally engineered to be the best they can be – unless for private set-ups inside factories, perhaps; they are engineered to deliver deterministic performance as efficiently as possible. AI is starting to be treated in exactly the same way.
Also, if I have time, I would point you to Zayo’s latest bandwidth report, which says long-haul fiber demand doubled last year, and metro dark-fiber demand jumped by as much as 20-times (!!!) in the busiest AI markets as inference shifts closer to users. Which feels like the same story from the other end of the telco telescope. AT&T says AI has become another network workload; Zayo says networks themselves have to be built around AI workloads. Everyone is saying the same of course – per the comments from Ciena yesterday, and the assumptions made from AT&T’s and Nokia’s latest quarterly results, this week, in particular. But networks used to carry applications; they are now part of the AI system itself. Which sounds like a marketing line, but is probably right.
If AI is like electricity (yawn, and apologies; already, if we believe the utilisation rates at data centers, and the inference rates in Zayo’s report), then the network is not just about the wires in your walls, but the power station in behind, which they all snake back to somehow.
