AWS Database Migration Service helps you migrate databases to and from AWS quickly and securely
The AWS Database Migration Service can migrate your data to and from most widely used commercial and open-source databases. Your source database will be fully operational during the migration which minimizes downtime on applications that rely on the database.
AWS Database Migration Service (AWS DMS) is a cloud service that makes it easy to migrate relational databases, data warehouses, NoSQL databases, and other types of data stores. You can use AWS DMS to migrate your data into the AWS Cloud, between on-premises instances (through an AWS Cloud setup), or between combinations of cloud and on-premises setups.
With AWS DMS, you can perform one-time migrations, and you can replicate ongoing changes to keep sources and targets in sync. If you want to change database engines, you can use the AWS Schema Conversion Tool (AWS SCT) to translate your database schema to the new platform. You then use AWS DMS to migrate the data. Because AWS DMS is a part of the AWS Cloud, you get the cost efficiency, speed to market, security, and flexibility that AWS services offer.
Benefits
Simple to use
AWS Database Migration Service is simple to use. You can begin a database migration with just a few clicks in the AWS Management Console. DMS manages all the complexities of the migration process including automatically replicating data changes that occur in the source database during the migration process. You can also use this service for continuous data replication with the same simplicity.
Minimal downtime
AWS Database Migration Service helps you migrate your databases to AWS with virtually no downtime. All data changes to the source database that occur during the migration are continuously replicated to the target, allowing the source database to be fully operational during the migration process.
Supports widely used databases
AWS Database Migration Service can migrate your data to and from most of the widely used commercial and open source databases. It supports homogeneous migrations such as Oracle to Oracle, as well as heterogeneous migrations between different database platforms, such as Oracle to Amazon Aurora. Migrations can be from on-premises databases to Amazon RDS or Amazon EC2, databases running on EC2 to RDS, or vice versa, as well as from one RDS database to another RDS database. It can also move data between SQL, NoSQL, and text based targets.
Low cost
AWS Database Migration Service is a low cost service. You only pay for the compute resources used during the migration process and any additional log storage. Conventional database migration methods are very expensive compared to this.
On-going replication
You can set up a DMS task for either one-time migration or on-going replication. An on-going replication task keeps your source and target databases in sync. Once set up, the on-going replication task will continuously apply source changes to the target with minimal latency.
Reliable
The AWS Database Migration Service is highly resilient and self–healing. It continually monitors source and target databases, network connectivity, and the replication instance. In case of interruption, it automatically restarts the process and continues the migration from where it was halted.
Use cases
Homogeneous Database Migrations
In homogeneous database migrations, the source and target database engines are the same or are compatible like Oracle to Amazon RDS for Oracle, MySQL to Amazon Aurora, MySQL to Amazon #RDS for #MySQL, or Microsoft SQL Server to Amazon RDS for SQL Server. Since the schema structure, data types, and database code are compatible between the source and target databases, this kind of migration is a one step process. You create a migration task with connections to the source and target databases, then start the migration with the click of a button. AWS Database Migration Service takes care of the rest.
The source database can be located in your own premises outside of AWS, running on an Amazon EC2 instance, or it can be an Amazon RDS database. The target can be a database in Amazon EC2 or Amazon RDS.
Heterogeneous Database Migrations
In heterogeneous database migrations the source and target databases engines are different, like in the case of Oracle to Amazon Aurora, Oracle to PostgreSQL, or Microsoft SQL Server to MySQL migrations. In this case, the schema structure, data types, and database code of source and target databases can be quite different, requiring a schema and code transformation before the data migration starts. That makes heterogeneous migrations a two step process. First use the AWS Schema Conversion Tool to convert the source schema and code to match that of the target database, and then use the AWS Database Migration Service to migrate data from the source database to the target database. All the required data type conversions will automatically be done by the AWS Database Migration Service during the migration.
The source database can be located in your own premises outside of AWS, running on an Amazon EC2 instance, or it can be an Amazon RDS database. The target can be a database in Amazon EC2 or Amazon RDS.
Development and Test
AWS Database Migration Service can be used to migrate data both into and out of the cloud for development purposes. There are two common scenarios. The first is to deploy development, test or staging systems on AWS, to take advantage of the cloud’s scalability and rapid provisioning. This way, developers and testers can use copies of real production data, and can copy updates back to the on-premises production system. The second scenario is when development systems are on-premises (often on personal laptops), and you migrate a current copy of an AWS Cloud production database to these on-premises systems either once or continuously. This avoids disruption to existing DevOps processes while ensuring the up-to-date representation of your production system.
Database Consolidation
You can use AWS Database Migration Service to consolidate multiple source databases into a single target database. This can be done for homogeneous and heterogeneous migrations, and you can use this feature with all supported database engines. The source databases can be located in your own premises outside of AWS, running on an Amazon EC2 instance, or it can be an Amazon RDS database.
The sources databases can also be spread across different locations. For example, one of the source databases can be in your own premises outside of AWS, while the second one in Amazon EC2, and the third one is an Amazon RDS database. The target can be a database in Amazon EC2 or Amazon RDS.
Continuous Data Replication
You can use AWS Database Migration Service to perform continuous data replication. Continuous data replication has a multitude of use cases including Disaster Recovery instance synchronization, geographic database distribution and Dev/Test environment synchronization. You can use DMS for both homogeneous and heterogeneous data replications for all supported database engines. The source or destination databases can be located in your own premises outside of AWS, running on an Amazon EC2 instance, or it can be an Amazon RDS database.
You can replicate data from a single database to one or more target databases or data from multiple source databases can be consolidated and replicated to one or more target databases.
The Steps to Create AWS DMS
We have to create two endpoints first such as source and target. An endpoint provides connection, data store type, and location information about your data store. While creating the endpoints DMS will allow us to select the database engine like MySql, Oracle etc. If the instances are AWS RDS, we can easily select them using the AWS console. Otherwise we have to give the real url of the database instances.
Next we have to create the DMS engine instance to use for our migration. There you have to select the appropriate instance class and DMS engine version. When you create an AWS DMS replication instance, AWS DMS creates it on an Amazon EC2 instance in a virtual private cloud (VPC) based on the Amazon VPC service. You use this replication instance to perform your database migration.
Finally you have to create the migration task by selecting the endpoints and DMS instance you have created earlier. An AWS Database Migration Service task is where all the work happens. You specify what tables (or views) and schemas to use for your migration and any special processing, such as logging requirements, control table data, and error handling. The main setting to remember here is Migration Type, currently AWS DMS supports 3 types namely Migrate Existing data, Existing data with replicate ongoing changes and replicate data changes only.
My Experience
Using DMS is a very reliable approach, that without writing any code the database migration is possible.
It was so fast and the ongoing data replication was also in real time synch.
Constant monitoring of migration and synch is possible through AWS Console and DMS diagnostic tools.
Validation possible, but found it as not reliable and time consuming. Also if failed, there is no way to restart validation for specific tables.
Extremely big Blobs have some issues, so not recommended.
AWS Support team was so helpful on resolving some issues we faced on non-prod environments
Points to take care
Do not forget to put back your indices.
Do not forget to put back your foreign keys.
Do not forget to re enable the auto increment columns.
Always enable AWS cloud-watch logs.
Properly set the full load tuning settings.
Look for LOBS in your db and take appropriate action.
Remember there are some pitfalls if we enable validation.
Always do trials in non-prod which help us to identify data and validation issues beforehand.
Yorumlar