Accommodating Scale-out Architectures in the Cloud

The cloud is all about scale. As enterprises seek to cobble together the resources needed to handle the unrelenting increase in data loads, a ready supply of on-demand IT resources is a welcome development indeed.

Unfortunately, there is a dark cloud to this silver lining. The fact is, there are different ways to scale resources in the physical world, and there are different kinds of application architectures designed to take advantage of different kinds of scale – and these architectures may or may not lend themselves well to a given cloud environment.

The two primary types of scale are scale-up and scale-out. The former involves replacing current hardware and software architectures with larger, more powerful systems, while the latter involves the addition of more devices of equal or lesser capability. In the cloud, these distinctions are somewhat blurred from an actual hardware perspective, but they can have significant consequences for your application environment.

As Schooner Information Technology Chairman and CTO Dr. John Busch points out, the cloud is much more adept at handling applications designed for scale-up architectures. The multicore technologies and virtualization platforms at the base of most cloud services can easily scale up the number of VMs to handle rising workload volumes and then scale back down when the pressure eases. But scale-out applications like databases, cache services and key-value stores, particularly those dealing with requests from high numbers of Web servers, can quickly overload cloud platforms – or at least ramp up the number of VMs and other requirements to drive the initial cost/benefit ratio through the roof. Hybrid clouds can mitigate this problem somewhat, he says, but what is really needed is a new form of virtualization that integrates data-tiering software with Flash and multicore technology within a unified management regime.

We may be seeing the development of a rudimentary scale-out cloud architecture already. A company called Gluster recently introduced a new scale-out NAS platform designed specifically for virtual and cloud environments. The system consists of the Gluster Virtual Storage Appliance that can be integrated into a VMware Virtual Storage Appliance or the Amazon Machine Image (AMI) component of the Amazon Web Services platform. The architecture is designed to reduce the complexity of scale-out cloud storage operations and reduce the performance impact that dynamic scaling typically has on virtual workloads.

Enhanced cloud interoperability should do a world of good for scale-out cloud architectures as well. The new Bexar release of RackSpace’s OpenStack platform allows for the development and pre-installation of a wide variety of application environments followed by the duplication of those environments as needs dictate. As an open-source governance framework, OpenStack should be able to accommodate scale-out as well as scale-up architectures.

No matter what you do, however, the cloud will require a completely different approach to application development itself, says Sinclair Schuller, CEO and co-founder of SaaS specialist Apprenda. The rigid server architecture of the bricks-and-mortar data center is giving way to dynamic scale-up and/or scale-out environments. Apps that can’t deal with the high availability, highly fluid requirements of such shared architectures are destined to fail.

The point of all this is to build awareness of the fact that not all clouds are the same. While it is tempting to think that simply writing a check to a cloud provider is all it takes to accommodate increasingly large and complex data loads, the reality is that the cloud is more suitable for some types of data and applications than others – for the moment at least.

It doesn’t mean the cloud is a bust, but it does mean you need to do your homework before making a commitment.