I talked about how big data discussions are starting to gain traction and build buzz across the federal government. Last week I had an opportunity to attend the FCW federal briefing on big data, which turned out to be a very good event. The packed room, consisting of government employees, decision makers, vendors, solution providers and academia had a lively discussion on some excellent topics. There were keynotes and panel discussions on big data by some very influential folks including Dr.Suzi Iacono (Sr. Science Advisor at NSF), Dr. George Strawn (Director at NCO and Co-Chair at NITRD), Dr. Chris Greer (Associate Director at NIST) and several others. This one day seminar was centered on the emerging challenges, trends, directions and solutions for big data and data analytics. The speakers provided a consistent understanding of the challenges and opportunities associated with big data, sharing some thought provoking nuggets of information.
Agency data is growing exponentially from a volume, velocity and variety perspective (both structured and unstructured data, including full motion video, emails, voice, social networks, sensor-enabled facilities, web and biometrics data). Agencies are also trying to figure out how they can best store, merge, manage and leverage this unbelievable amount of data and try to make it actionable. And they are doing so at the same time they are trying to address the challenges posed by security and privacy that comes with the merging of federal data from various sources.
In order to harness the power of big data, agencies need to create a data ecosystem. They need to collaborate with academia and industry to leverage innovation and technological developments. The systems have to be architected keeping in mind security, interoperability and open standards from conception. Federal agencies have to work through processes, policies and politics and break down data silos. Advances in hardware and software that have the capability to process multi-structured and multi-sourced data and new kinds of scalable storage including solid-state memory and parallel architectures are required to make Big Data a reality. Dr. George Strawn made a remark that he has never before seen a technology evolve so fast and go from a “bag of tricks” to working applications. Dr. Strawn made it clear that the focus in government has transitioned from open government to cloud computing to big data, and that if we do not take this seriously and start producing real results in the near term, the focus might again shift to something new like “cyber-physical systems” next year.
I truly believe big data and advanced analytics are valuable to the agency mission. I am happy to see the collaboration between the agencies, standards bodies and industry finally taking shape to tackle big data problems. I am honored to be serving on the TechAmerica Foundation big data commission as a commissioner and be able to play a key role in helping shape and provide guidance on how government should best leverage big data to address critical business issues and better deliver on the mission. It is now time to put on our thinking caps and start determining how to make the most of the opportunity presented to us by the challenge of big data. In the coming weeks and months, I will be blogging on many different aspects of big data. Let me know your thoughts. Follow me on Twitter at GTSI_Architect.