Migration and Transfer
AWS DataSync
Online data transfer service that copies files or objects between on-premises storage (NFS, SMB, HDFS, or object storage), other clouds, and AWS services such as Amazon S3, Amazon EFS, and Amazon FSx. It automates the transfer with encryption and data integrity validation.
Exam phrases that point here
«migrate terabytes of files from an on-premises NAS over NFS or SMB to Amazon S3, Amazon EFS, or Amazon FSx over the network»
DataSync is the AWS online transfer service: an agent reads the on-premises storage over NFS or SMB and copies the data to S3, EFS, or FSx, with encryption, integrity verification, and bandwidth control.
When not: If the network cannot move the volume within the required time, a physical transfer fits: AWS Snowball Edge is no longer offered to new customers and AWS suggests AWS Data Transfer Terminal or partner solutions. If on-premises applications must keep accessing the files already moved, AWS Storage Gateway fits.
«periodically sync on-premises data with AWS until the cutover day»
DataSync tasks can be scheduled and, on each run, copy only what changed since the previous one; this keeps the destination up to date until the cutover.
When not: If what must be kept in sync is a database with ongoing changes, the tool is AWS DMS with CDC; if it is entire servers, AWS Transform MGN (formerly AWS Application Migration Service).
«copy data between Amazon EFS, Amazon FSx, and Amazon S3, even across Regions, without writing scripts»
Between AWS storage services in the same account, DataSync copies without an agent, within a Region or across Regions, with scheduling and verification.
When not: To continuously copy objects from one bucket to another, S3 Replication fits better; to apply an action to millions of objects, S3 Batch Operations.
«copy data from another cloud provider to Amazon S3»
DataSync supports storage from other clouds (for example, Google Cloud Storage or Azure) as a source and copies it to S3 on a schedule, with verification.
When not: If the requirement is for external users or partners to upload files over SFTP, FTPS, or AS2, the answer is AWS Transfer Family: DataSync does not offer a file server for third parties.
«verify that the copied data is identical to the source»
DataSync checks integrity during the copy and, optionally, compares the source and destination checksums at the end, in addition to producing reports for each run.
When not: If you use tools such as rsync or the S3 CLI, you have to build that verification by hand; if the shipment is physical, the verification depends on that process (for example, Snowball Edge produces a report with the checksums of each file), not on DataSync.
Often paired with
- Amazon S3
- Amazon EFS
- Amazon FSx