In Search of S3: Read the Fine Print

For businesses using and creating applications for the Internet, it is becoming more difficult to ignore Amazon Simple Storage Service (S3). S3 is the massively scalable, cost-effective cloud storage solution developed specifically to house the huge influx of data created by organizations around the world. Amazon S3 commands twice the market share of all its closest competitors combined and is likely to be the storage platform of choice for on-premises hybrid or private cloud deployments for years to come. 

S3 has become the standard for cloud storage. Almost every application connects to S3 and most storage vendors have already announced that they connect to S3 or are working to do so. In addition to Amazon, there are a number of competing storage implementations that are S3-compliant, including Google Cloud Storage, Openstack Swift, Rackspace’s Cloud Files and Ceph. These services use the standard programming interface but have different underlying technologies and business models. 

The rise of S3, which Amazon describes as "cost-effective object storage," has also helped to drive the adoption of object storage. In addition to Amazon, household brands such as Facebook, Netflix, Dropbox and Twitter all use object storage. It’s also deployed by enterprises for applications that require massive amounts of unstructured data, including content media storage, bioinformatics, data analytics, private cloud, file distribution and sharing, and backup and archiving. 

With S3 and object storage so heavily interlinked, there are compelling reasons why companies seeking to implement a cloud strategy should take a closer look at the credentials of their storage supplier. 

Compatibility matters 

For organizations considering deploying an open hybrid cloud or moving data between S3 and the private cloud, they need to understand the level of compatibility a storage platform claims versus its compatibility in reality. With S3 quickly becoming the object storage standard, choosing the right storage platform for the hybrid or private cloud can save money and shave months off the deployment time. In essence, not all S3 compatibility is the same. 

Companies and developers implementing applications with the Amazon S3 API depend significantly on its compatibility to ensure their applications function and remain interoperable. But because no one enforces standards over S3 compatibility, choosing the right storage platform can be much more complex than providers would like customers to believe. 

When specifying your application management requirements, make sure the numbers add up. There are 51 total operations available through the S3 API (9 simple, 18 moderately complex and 24 advanced, a good reference to S3 API set is here: http://info.cloudian.com/S3API-wp-lp-0515.html). 

Compatibility is based on a storage platform’s ability to perform some, many, or all of them. This means that, in practice, storage platforms may claim S3 compatibility even though they only support the nine simple operations that allow only very basic manipulation of data through the API. And while other platforms might be able to provide a moderate level of compatibility, most of them are still unable to perform all of the 18 additional moderately complex operations.  

Needless to say, most storage platforms find it even harder, if not impossible, to provide the 51 total operations of the S3 API. This should be a concern for organizations and developers that want the assurance their applications are S3 compatible and will work seamlessly with their hybrid or private cloud. Only then can they have certainty that whatever strategy they adopt and whatever application they use will survive into the future. If organizations opt for a platform with only limited S3 compatibility, their storage will not support the commands required by future applications further down the line. 

SDS is ideally suited for private and hybrid clouds 

The storage platforms best suited to providing private or hybrid cloud are likely to be software-defined storage (SDS) solutions. Because SDS abstracts logical storage services and capabilities from the underlying physical storage systems, it is massively scalable, resilient, highly available and reduces complexity by allowing a single application to execute data services across multiple storage hardware platforms. Allowing the use of commodity hardware, instead of expensive proprietary equipment, saves costs by eliminating vendor lock-in and the expense of forklift upgrades. 

In effect, SDS allows organizations to mimic the Amazon S3 experience in their data centers and on their private and hybrid clouds. This makes it even more beneficial for enterprises to look for SDS-based object storage platforms that are truly S3 compatible. 

Which means that, when looking for S3 object storage, make sure you read the label.