Scaling Data Management to Meet the Cloud

The cloud may be the newest technology on the enterprise block, but in many ways it is following the same deployment pattern of previous hardware and software developments.

One habit that never seems to change is the need to get up and running with the latest offering and save pesky details like monitoring and management for later. Unfortunately, some top-tier firms that are already pushing workloads onto the cloud are finding that later is already here.

That’s one of the key findings in Network Instruments’ latest State of the Network Global Study, in which more than half of respondents reported having implemented cloud computing but admit they lack the kinds of tools needed to troubleshoot problems. In fact, more than a third of those surveyed said the need to identify and correct trouble spots has increased under virtual/cloud architectures, while some 85 percent said finding problem sources is one of the most significant obstacles in deploying or scaling up cloud services.

It is very possible, then, that cloud management could quickly become the primary function of the CIO at many top organizations, according to Novell’s Benjamin Grubin. As IT transforms from a provider of services to a utility model, the focus will be less on application development and user interface management and more on service-level management and resource allocation. For the CIO, that means a gradual lessening of strategic planning responsibilities and an increase in tasks like portal and protocol management. Of course, both of these skill sets will be in high demand during the transition phase when the cloud exists alongside traditional infrastructure.

As can be expected, there is already a plethora of tools ready to provide insight into the inner workings of the cloud. New firms like FireScope are seeking to blend internal and cloud-based infrastructure under one management scheme through a mixture of hardware, virtual appliances and on-demand services. FireScope relies on a message queue architecture that delivers metrics and management events across SaaS, Iaas, PaaS and hybrid architectures. The goal is to treat the entire framework as a single business service.

Management issues become even more pronounced as the cloud takes on HPC functions. Platform Computing has developed a number of approaches designed to smooth out the rough patches involved in unpredictable, ultra-heavy workload management. These include integrating the company’s dynamic host capability with hosted private cloud services like Amazon VPC, bypassing the host service altogether through a cloud-ready multi-cluster platform, and enabling a dynamic cluster extension that scales to internal and external resources. Each program is designed to provide a means to accommodate workload spikes quickly and easily based on data priority and required resources.

Since virtually no one has much experience managing large amounts of data on the cloud, it’s safe to say that the most of the solutions on the market today are rudimentary at best. As cloud architectures are deployed and then scaled upward, the relationship among data, resources and users is likely to become more complex.

The wisest course of action, then, would be to implement a management strategy now for base-level cloud operations — one that can improve and expand along with its environment. If you treat management as an afterthought, you might find yourself surrounded by a swirling cloud that can only be brought under control through great expense.