This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .
Migrating a database from one cloud to another can be a time consuming and complex endeavor, but if you’ve prepared a comprehensive migration strategy, your chances of successful completion increase multi-fold. A successful large-scale database migration involves several key phases, including planning, preparation, performing a test migration, performing the final migration, validation, and then cutover.
This post covers various strategies for migrating AWS RDS for MySQL to Azure Database for MySQL, how to use them to maximize efficiency and cost savings, different migration considerations, the importance of proper planning and preparation, and potential pitfalls that can arise during the process. With these details, you’ll be able to make a more informed decision about the best approach to take when migrating your MySQL database from AWS to Azure.
Before you embark on the journey of migrating from Amazon RDS for MySQL to Azure Database for MySQL, there are several areas considerations to take into account.
The business operations team understands the criticality of the application and its corresponding databases, and the team plays a pivotal role in defining the acceptable downtime or near-zero downtime in some cases. The amount of downtime that you application can tolerate is an extremely critical factor to consider when determining the best migration approach to take and planning the overall migration to ensure that it occurs within the desired timeframe.
Version considerations and incompatibilities
Azure Database for MySQL - Flexible Server is designed to be wire protocol-compatible with MySQL 5.7 and 8.0, allowing MySQL client drivers to communicate with the database. As a result, most of the applications, drivers, and tools that are currently used with AWS RDS MySQL databases can also be used with Azure Database for MySQL - Flexible Server with little or no change. Also, be sure that the application and database are deployed in the same region. Azure Database for MySQL - Flexible Server is available in these regions.
Network and security
Ensure you have a secure and fast network connection between your AWS and Azure clouds. Depending on your speed, latency, reliability, service-level agreement (SLA), complexity, and cost requirements, you can use a site-to-site VPN gateway, Azure ExpressRoute, or AWS’s direct connect with cloud exchange to establish AWS to Azure infrastructure connectivity. For more details, see Connectivity to other cloud providers - Cloud Adoption Framework.
Database performance is key to application performance. Gathering performance benchmarks and evaluations on the source AWS RDS MySQL databases can assist in sizing the target Azure Database for MySQL instance. After provisioning the target instances, it’s recommended to simulate the peak load and conduct performance tests in lower environments to ensure that they meet the performance benchmarks and overall expectations.
The cost associated with migrating your databases is another key consideration, so you should have a plant in place to optimize your costs over time. The pricing calculator is a great tool for estimating the monthly cost of using Azure Database for MySQL- Flexible Server.
Since Azure Database for MySQL is suitable for running mission critical databases that require high uptime, business continuity, as well as the recovery time objective (RTO) and recovery point objective (RPO) for each application, is an important consideration. Business continuity capabilities protect your databases in the event of:
- Planned outages, such as minor version upgrades, new feature rollouts, scaling Compute up/down, Scaling Storage up.
- Unplanned outages, such as database server failures, Storage failures, or region failure.
In each case, functionality such as high availability, redundancy, and resiliency can help to prevent disruptions in your business operations. For more information, see Overview of business continuity.
After identifying the workloads to migrate, the migration process typically involves a series of phases including planning, preparation, performing a dry run, performing the actual migration, and finally validation and database cutover.
1 - Planning
As part of planning, discovery, and assessment, it is important to understand the AWS data estate including various interdependencies between your applications and databases, the server SKU, CPU usage, storage, database sizes, and extensions usage as some of the critical data to help with migrations.
Acceptable downtime will play a critical role in determining the migration approach (covered later in the blog).
A fallback plan is to switch back to the source database in case of any issues post migration. As a precautionary measure, it is advisable to have a fallback plan to mitigate potential mishaps or difficulties that may occur during migration especially for critical database clients. In my experience, I have seen customers have a pipeline for backward migration for a week to ensure that both the databases are in sync.
2 - Preparation
Ensure that a secure and fast network connectivity between your AWS and Azure is established. Ensure that appropriate privilege is provided on the source database e.g., “READ” & “SELECT” privilege for the ability to select objects, “SHOW VIEW” & “TRIGGER” for migrating triggers and views. When using a firewall appliance in front of your source database(s), you may need to add firewall rules to allow connections from Virtual Network for Azure Database Migration Service to access the source database(s) for migration.
Database capacity planning helps in estimating the size of the target database & determining the hardware configuration. It is important to perform database capacity planning anticipating future capacity for growth and application performance requirement. Provision target database by referring to Create an instance in Azure Database for MySQL.
Azure Database for MySQL - Single Server is on the retirement path. I strongly recommend for you to upgrade to Azure Database for MySQL - Flexible Server. Also, the MySQL version for the target database server must be greater than or equal to that of the source database server MySQL version. Open your Windows firewall to allow connections from Virtual Network for Azure Database Migration Service to access the source MySQL Server, which by default is TCP port 3306.
3 - Dry Run/Test migration
Prior to executing the production migration, it is recommended to perform multiple dry runs in the lower environment. This will help in identifying & mitigating potential issues, validating that the performance and throughput is sufficient, and time-to-migration is as per the downtime expectations of the business.
It is also highly recommended to do a performance testing in lower environment considering the peak load to ensure that application and database performance optimization and tuning is performed. I have observed that this critical part is ignored in many migrations leading to performance issues post migration.
4 - Final migration
Final migration is the migration of production databases from source to destination. Monitoring the database migration process is crucial to ensure its successful completion. Also make sure that all read replicas if any are ready.
It is advisable to intimate and brief the backend support team regarding the migration beforehand so that they can be on stand-by and help if needed.
5 - Validation and Cutover
Post migration, it is critical to ensure data completeness and consistency in the target databases. Once the migration is validated, cutover can be performed, and client connections can be redirected to target Azure database.
Before selecting the best tool to use for a specific migration, it’s important to determine whether to perform an offline migration or an online migration.
- With offline migration, the source system is offline during the migration process. Using this option ensures that no transactions occur in the source system so that the state of the data is exactly what is expected when restored in target Azure system.
- With online migration, however, data is migrated in near real time. As a result, this option works well when you require minimal downtime for users and need to allow source server updates, as the ongoing changes between are replicated between the source and target servers in addition to the initial dump and restore on the target.
Offline migrations are often the preferred choice, as they are simple and easy to execute. Online migration is the best choice for systems such as e-commerce order processing, payment systems, etc., that require minimal downtime.
Offline migration options
If you decide to perform an offline migration, there are several popular options for accomplishing the AWS RDS for MySQL to Azure Database for MySQL migration, as described in the following table.
mysqldump is a popular tool for backing up MySQL databases. Users can create a backup and export a database’s data and schema definitions from a MySQL server. Users can also import a database backup into MySQL server using the SOURCE SQL command (run from inside the database) or the MySQL command line.
Use the standard command line tool to export MySQL data and import it into Azure Database for MySQL. For more information, see Migrate using dump and restore - Azure Database for MySQL.
mydumper/myloader are open-source import/export tools for migrating MySQL databases larger than 1 TB to Azure Database for MySQL. Mydumper/myloader overcome performance issues by exploiting parallelism and utilizing the available IO bandwidth during both export and import process.
Some of their key benefits are:
For migration steps and best practices, see Migrate large databases to Azure Database for MySQL using mydumper/myloader and Best Practices for migrating large databases to Azure Database for MySQL.
Azure Database Migration Service (offline)
Azure Database Migration Service is a fully managed service designed to enable seamless migrations from multiple database sources to Azure data platforms with minimal downtime.
The MySQL Offline migration is supported only on the Premium DMS SKU. Migration from lower version MySQL servers (v5.6 and above) to higher versions is also supported.
Please refer below tutorial for offline MySql migration using DMS.
MySQL Workbench is a powerful unified visual tool for database architects, developers, and DBAs. MySQL Workbench also provides a Migration Wizard for an easy and convenient way to move your Amazon RDS for MySQL databases to Azure Database for MySQL. With the Migration Wizard, you can conveniently select which schemas and objects to migrate. It also allows you to view server logs to identify errors and bottlenecks in real time. As a result, you can edit and modify tables or database structures and objects during the migration process when an error is detected, and then resume migration without having to restart from scratch.
For more information, see Migrate Amazon RDS for MySQL to Azure Database for MySQL using MySQL Workbench.
Online migration options
If you determine to perform an online migration, there are also several options to choose from regarding the best tool/method to use, as described in the following table.
mydumper and myloader with data-in-replication
Use mydumper/myloader for initial data seeding, and then after the initial migration, use Data-in replication to synchronize delta data from an AWS MySQL server into the Azure Database for MySQL server.
Data-in replication is a technique that replicates data changes from the source server to the destination server based on the binary log file position method. In this scenario, the MySQL instance operating as the source (on which the database changes originate) writes updates and changes as “events” to the binary log.
Azure Database Migration Service (online) - (Preview)
Azure Database Migration Service is a fully managed service designed to enable seamless migrations from multiple database sources to Azure data platforms with minimal downtime (online migrations). This method is recommended if the database size is less than 1 TB.
Migration from lower version MySQL servers (v5.6 and above) to higher versions is also supported. A detailed blog of creating an Online migration from Amazon RDS for MySQL to Azure Database for MySQL using DMS will be coming soon.
3rd Party - Striim
The Striim platform is an enterprise-grade data integration and streaming solution for moving data in real-time to the cloud. Microsoft and Striim recently announced a strategic collaboration for database modernization in the Azure cloud.
This method is recommended if the database size is greater than 1 TB.
After reading this article, you should have a better understanding of the various methods for migrating from Amazon RDS for MySQL to Azure Database for MySQL and be able to make a more knowledgeable decision about the best tool to use to complete the process.
Look for another post in the near future that will provide detailed, step-by-step instructions for performing an online migration from AWS RDS MySQL to Azure Database for MySQL - Flexible Server using DMS.
If you have any feedback or questions about the information provided above, please leave a comment below or email us at AskAzureDBforMySQL@service.microsoft.com. Thank you!