Consumer Reports takes on tough assignment: smart phones

Under the banner, “Simpler and Slimmer,” Consumer Reports’ January 2007 issue weighed in on a handset category that has even the experts scratching their heads.
At CTIA I.T. in September, a panel of leading wireless industry analysts proffered a variety of multi-faceted explanations for how so-called smart phones might be defined. The wireless version of a familiar joke, repeated then, underscored the issue: ask two analysts to define “smart phone” and you’ll get three answers.
For those in the wireless industry, the Consumer Reports package of articles on mobile phones and carrier service was less a reality-check on industry’s efforts and more a window onto how a popular magazine might present the industry to the general public’s subset of careful consumers.
For example, the magazine defined the growing smart-phone market segment by what tasks consumers would likely do with their devices. “Advanced” smart phones offer “robust e-mail and editing capabilities,” while “basic” smart phones offer “nominal” e-mail capabilities, the magazine suggested-a simplified notion, to be sure, but one with some credibility with at least one analyst.
“This distinction makes reasonably good sense,” said Avi Greengart, handset analyst with Current Analysis. “But the T-Mobile Sidekick 3, for instance, is placed in the ‘basic’ category and the fact is, it has the best keyboard in the business.”
Of the dozens of models on the market, the magazine chose to look at eight, with what appeared to be representative models from Palm Inc., Research In Motion Ltd., Nokia Corp., Motorola Inc. and carrier-branded offerings from T-Mobile USA Inc.
CR recommended that potential buyers consider their need to read or edit documents and what sort of e-mail they’re likely to use, from simple to those with data-heavy attachments. The magazine suggested that CDMA networks would provide the best service for data-heavy e-mail attachments due to CDMA2000 1x EV-DO network speeds available at Verizon Wireless and Sprint Nextel Corp. Slower, EDGE-based GSM networks at T-Mobile USA and Cingular Wireless L.L.C. would better serve those less data-intensive applications, the magazine said, though it also pointed to GSM’s lower battery drain and worldwide ubiquity. Consumer Reports did not mention Cingular’s ongoing HSDPA buildout.
The magazine took a brief stab at explaining the pros and cons of the leading operating systems: Palm OS was “most versatile,” Windows Mobile was lauded for synchronization, BlackBerry OS scored on “simplest for e-mail,” while Symbian’s basic version is “difficult to use” and its advanced version is “hard to navigate.” Consumers were advised to assess keyboards and displays and the various combinations of voice and data plans available.
On evaluating devices, the magazine offered a table that purported to track performance based on voice quality, talk time, sensitivity (voice quality with a weak signal) and messaging. Among advanced smart phones, the magazine rated only four devices, with the best first: the Palm Treo 700p at Sprint Nextel and Verizon Wireless, the T-Mobile USA MDA, the Palm Treo 700 wx at Sprint Nextel and Verizon Wireless and the Nokia 9300 at Cingular. Among basic smart phones, the nod went to the BlackBerry Pearl at T-Mobile USA, the Nokia N80 for GSM carriers, the Moto Q at Verizon Wireless and the T-Mobile USA Sidekick 3.
So, how did Consumer Reports’ methodology and results resonate with a knowledgeable wireless insider?
Greengart pointed out that one could slice the smart-phone market by any one of several factors, including form factor, e-mail capabilities, screens or keyboards, and that Consumer Reports looked at eight of dozens of models on the market. He gave the magazine points for actually testing voice quality, but said that an annual report in an industry that “changes by the minute” was an exercise fraught with peril.
The analyst said that the relatively slow uptake of smart phones in the United States might well reflect the complexity of choices in the market segment, a complexity that Consumer Reports clearly wrestled with in its January issue. Consumers might find the report as confusing as the market itself, he said.