Big data gap: Only .5% of data is analyzed


There is an untapped big data gap. Only .5 % of data is currently analyzed by companies to help them make decisions. IDC’s latest Digital Universe Study sponsored by EMC showed that in 2012, 23% of the information in the digital universe (or 643 exabytes) would be useful if it was tagged and analyzed. However, only 3% of the potentially useful data is tagged and even less — only .5% — is analyzed.

The newly released survey shows that the vast majority of new data being generated is unstructured. As a consequence, companies know very little about this data unless it is somehow characterized or tagged—a practice that results in metadata, which is one of the fastest-growing sub-segments of the digital universe.

IDC estimates that by 2020, as much as 33% of the digital universe (more than 13,000 exabytes) will contain information that might be valuable if analyzed, compared with 25% today. This untapped value can be found in social media usage patterns, correlations in scientific data from discrete studies, medical information intersected with sociological data, faces in security footage, and so on.

The survey forecasts the digital universe will be generating 40 zettabytes by 2020. Last year’s forecast for 2020 was 35 ZB. This raises the question: how can companies protect this amount of data? The survey showed that the proportion that will need some sort of protection will increase from less than a third in 2010 to more than 40% in 2020.


Other survey highlights:

  • From 2005 to 2020, the digital universe will grow by a factor of 300 from 130 exabytes to 40,000 exabytes, or 40 trillion gigabytes (more than 5,200 gigabytes for every man, woman and child in 2020). From now until 2020, the digital universe will about double every two years.
  • The investment in spending on IT hardware, software, services, telecommunications and staff that could be considered the “infrastructure” of the digital universe and telecommunications will grow by 40% between 2012 and 2020. As a result, the investment per gigabyte during that same period will drop from $2 to 20 cents. Of course, investment in targeted areas like storage management, security, big data and cloud computing will grow considerably faster.
  • Between 2012 and 2020, emerging markets’ share of the expanding digital universe will grow from 36% to 62%.
  • By 2020, nearly 40% of the information in the digital universe will be “touched” by cloud computing providers—meaning that a byte will be stored or processed in a cloud somewhere in its journey from originator to disposal.

About Author

Editor, Americas
Roberta Prescott is responsible for Latin America reporting news and analysis, interviewing key stakeholders. Roberta has worked as an IT and telecommunication journalist since March 2005, when she started as a reporter with InformationWeek Brasil magazine and its website IT Web. In July 2006, Prescott was promoted to be the editor-in-chief, and, beyond the magazine and website, was in charge for all ICT products, such as IT events and CIO awards. In mid-2010, she was promoted to the position of executive editor, with responsibility for all the editorial products and content of IT Mídia. Prescott has worked as a journalist since 1998 and has three journalism prizes. In 2009, she won, along with InformationWeek Brasil team, the press prize 11th Prêmio Imprensa Embratel. In 2008, she won the 7th Unisys Journalism Prize and in 2006 was the editor-in-chief when InformationWeek Brasil won the 20th media award Prêmio Veículos de Comunicação. She graduated in Journalism by the Pontifícia Universidade Católica de Campinas, has done specialization in journalism at the Universidad de Navarra (Spain, 2003) and Master in Journalism at IICS – Universidad de Navarra (Brazil, 2010) and MBA – Executive Education at the Getulio Vargas Foundation.