When it comes to fully leveraging big data in government, there are just as many challenges — from the complexities of using different databases to the sheer volume of data — as there are opportunities. In fact, consulting firm Gartner Inc. reported that enterprise data growth rates now average 40 percent to 60 percent annually.
As reported in Washington Technology recently at the Big Data Technology Leadership Series 2013 event, presented by GTSI Corp. and FCW, Simon Szykman, CIO of the Commerce Department, outlined his top nine big data challenges:
-
Data acquisition
-
Storage
-
Processing
-
Data transport and dissemination
-
Data management and curation
-
Archiving
-
Security
-
Workforce with specialized skills
-
Cost of all of the above
Szykman also discussed how these challenges are linked to each other. For example, data acquisition – how we get data – leads into the next two challenges, data storage and processing.
As for data transportation and dissemination, the biggest challenge is getting the data from the place where it is analyzed to the people who need it. And, in most cases, real-time data is critical.
In the article, Szykman goes into more details on the other challenges, which of course includes the cost. However, when used properly, big data will be a main driver for enhanced efficiencies and improved performance, which will mitigate any cost issues and concerns.
Based on the discussions at the recent TLS event, it is obvious that big data will be a main driver for the government in 2013 and beyond. Of course, there are inherent challenges in managing massive volumes of data, but once we have collectively – industry and government – developed the right solutions for making this data actionable, in real-time, we will all benefit.