There are two initiatives that are top of mind for public sector agencies dealing with information overload: big data analytics and cloud computing. In my previous blogs, I have discussed how big data provides significant advantages for the public sector – from helping agencies fight crime to providing event-driven operational intelligence. This week, I would like to discuss how the cloud computing and big data analytics technologies are coming together.
Many government agencies are experiencing humongous growth in data; however, they are still reluctant to migrate their data over to cloud providers, owing to concerns regarding multi-tenancy, data security, software licensing issues and data integration. But the government’s ever increasing appetite for collecting and storing information is driving agencies to seriously consider the promised cost savings and benefits of cloud computing.
A suitable IT infrastructure is a key prerequisite for embarking on a successful Big Data strategy. Agencies looking to leverage big data solutions will have to evolve their IT infrastructures including massively scalable storage and network infrastructure designs, as well as considerations for data protection, data sharing, data reuse, ongoing analysis, compliance, security/privacy issues, data retention and availability.
Government agencies are now starting to explore the use of cloud computing to support their big data projects. So the question that comes to the mind is “What makes cloud computing a viable choice for hosting big data analytics solutions?” Big data environments require clusters of servers to support the tools that process large volumes, high velocity and different formats of data. cloud computing fits the bill really well since clouds are already deployed on pools of server, storage and networking resources and being elastic, they can scale up and down as needed to meet the expanding volume and variety of data.
In addition, the Obama Administration has made public data access a priority and desires to make government employees more productive by allowing them to access systems and files remotely. Cloud computing not only improves data access but also allows government agencies to easily put this invaluable resource into more people’s hands.
As key evidence of the increasing use of cloud computing for big data initiatives, the CIA recently agreed to a cloud computing contract with Amazon Web Services (AWS), worth up to $600M over 10 years to help improve the agency’s cloud storage and warehousing abilities. Just last week, IBM signed a cloud computing deal with the U.S. Department of the Interior worth up to $1 billion over the next 10 years with an intention to help the DOI move its technology into the cloud by providing cloud storage, Web & Application hosting.
OEM vendors like Cisco, VMware and IBM are bringing new innovative end-to-end cloud solutions to the market to effectively deal with growing volumes of data. For instance, IBM recently launched PureFlex System which is a fully integrated infrastructure system and caters to government agencies that would like to have ultimate control of their infrastructure pieces in their IT environment. From a cloud computing services view, the PureFlex System can be viewed as an infrastructure as a service (IaaS). IBM Flex System represents an entirely new generation of technology, with more performance and bandwidth, true integrated enterprise SAN storage, and far more capability to consolidate and virtualize than previous systems.
IBM offers SmartCloud on PureFlex System with high density x86 and Power Systems compute nodes. On System x, the IBM SmartCloud is delivered as a VMware vApp in a VMware ESXi virtual image. IBM Systems Director and VMControl plug-in to Systems Director helps with providing management, automation, and health monitoring of the cloud. IBM Systems Director Storage Control is able to provide an effective way to discover and collect inventory, and monitor health of select storage subsystems like the IBM Storwize V7000 storage system (used in PureFlex System) and Fibre Channel switches. The IBM Tivoli Provisioning Manager for Images provides excellent capabilities to capture and deploy physical or virtual systems, anywhere-to-anywhere server image conversions, and snapshot and restore of servers, independent of the underlying hardware.
Many government agencies are consolidating their existing data centers and moving toward shared IT solutions, with cloud computing the most prominent option. Not only are cloud-based data systems more efficient, but such systems also allow for faster, more precise analytics. Government agencies considering the cloud for data analytics projects, but concerned about security, can use service level agreements to arrange a cloud setup in which they do not need to worry about losing control of their data. Besides, the new converged system like IBM PureFlex, which integrates servers, storage, and networking under a single management framework, is the future of enterprise computing and a viable solution for analyzing massive amounts of data sets and workloads in a scalable manner in the cloud environment.
In coming months and years, I anticipate many Big data Projects converging with the private and public cloud environment. Are you seeing a similar trend take shape? Please share your comments and thoughts with me. Follow me on Twitter at GTSI_Architect.