WANdisco democratizes data lake migration to the cloud with the launch of LiveData Migrator

WANdisco announced the launch of LiveData Migrator, an automated, self-service solution that democratizes cloud data migration at any scale by enabling companies to easily start migrating Hadoop data from on-premises to Amazon Web Services (AWS) within minutes, even while the source data sets are under active use. Available as a free trial for up to five terabytes, businesses can migrate HDFS data without the expertise of engineers or other consultants – the program can be implemented immediately to enable companies’ digital transformations. LiveData Migrator works without any production system downtime or business disruption while ensuring the migration is complete and continuous and any ongoing data changes are replicated to the target cloud environment.

“Through 2022, more than 50% of data migration initiatives will exceed their budget and timeline – and potentially harm the business – because of flawed strategy and execution,” according to Gartner in a recent report. This is despite the fact that nearly 70% of enterprises have started moving data to the cloud as part of their digital transformations. 

Migrating large data volumes with traditional approaches, such as transfer devices or DistCp (distributed copy), requires disrupting the operation of on-premises applications and doesn’t cater to data that is modified or created during migration. Reconciliation at scale is costly and cannot guarantee a completely consistent data outcome. The overhead required to achieve non-disruptive, no-downtime big data migration is significant due to repeated scans, systems out of sync and manual intervention for anticipated failures and interruptions.

“Enterprises that want to move their data to the cloud but are concerned about the risks of doing so now have a powerful solution that’s self-service and extremely easy to use,” said WANdisco co-founder and CEO David Richards. “Regardless of company size or technical expertise, LiveData Migrator enables businesses to migrate their data risk-free to the cloud on a massive scale without any disruption to business operations. Data can be accommodated without any risk of data loss while modernizing data and applications to stay competitive.” 

GoDaddy is using LiveData Migrator to migrate 500 TB of HDFS data to Amazon Simple Storage Service (Amazon S3). GoDaddy will operate in a hybrid cloud mode and continue to update S3 data as changes are made to Hadoop content in the source, so that it is immediately available for cloud analytics using Amazon EMR. The first 70 TB was migrated within days with zero on-premises system downtime. 

“At GoDaddy, deep technical knowledge is in our DNA, and we often build applications in-house to support growth,” said Wayne Peacock Chief Data and Analytics Officer, GoDaddy. “In the use case of a Hadoop to Amazon S3 data migration and replication, we found WANdisco’s LiveData Migrator to be the optimal approach to deliver the best time to value, rather than running a more time-consuming and costly manual migration project internally. With LiveData Migrator, we have completed the first phase, giving us a quick win by fully automating the HDFS migration of 70 TB in five days, avoiding disruption of production processes and making the data immediately available in Amazon S3 for testing.”

WANdisco LiveData Migrator eliminates data migration risks with immediate, live and scalable migration. Administrators can easily and immediately move data lake content to the cloud while minimizing IT resource involvement with automated migration capabilities. No changes are required to applications, cluster or node configuration or operation, and there is no production system downtime or business disruption, while ensuring their data changes are migrated continuously and completely.

WANdisco is an Advanced Technology Partner in the AWS Partner Network (APN) and has now achieved AWS Migration Competency status with the release of LiveData Migrator. WANdisco was one of four APN Partners who collaborated with AWS to define the requirements to cover different use cases, including Hadoop, storage and database data migration, and mainframe data integration.

“We are delighted to have APN Partners like WANdisco that exhibit extensive knowledge and proven expertise in migration and replication technology,” said Sabina Joseph, Director of Americas ISVs, Amazon Web Services, Inc. “LiveData Migrator is designed to offer customers a platform that automates and safely accelerates large-scale on-premises Hadoop migrations to AWS without business disruption. Customers will benefit from optimizing costs and expanding analytics and insights capabilities leveraging industry leading services like Amazon S3, Amazon EMR and Amazon Athena.” 

LiveData Migrator gives enterprises the opportunity to free their systems from the capital constraints of acquiring storage hardware by leveraging Amazon  S3 and associated services. LiveData Migrator delivers fast time to value for migrating unstructured data into cloud storage to then take advantage of machine-learning (ML) powered cloud analytics such as Amazon EMR, Databricks or Snowflake to gain valuable business insights. LiveData Migrator also enables the transition to a hybrid architecture, where on-premises and cloud environments are kept consistent for active-active replication capabilities, and sets the foundation for a future multi-cloud architecture. 

LiveData Migrator Capabilities

Complete and Continuous Data MigrationMigrates any changes made to the source data sets, allowing applications to continue to modify the source system’s data without causing divergence between source and target.

Rapid Availability
Enables data to become available for use in the target environment as soon as it has been migrated, without having to wait for all data set migrations to complete.

Any ScaleMigrates any volume of data, from terabytes to exabytes, to cloud storage without needing to stop changes to data at source during migration

Hadoop & Object Storage Conversion
Migrates HDFS data to other Hadoop-compatible file systems and cloud storage, including the ongoing changes made to those data before, throughout and after migration.

Selective Migration
Allows selection of which data sets should be migrated and selectively excludes data from migration to specific clusters in the new environment.