Amazon Adopts Amazon Aurora for Inventory Database

Unfortunately, IMS was starting to outgrow its technical foundations. It used a pair of on-premises Oracle databases for each warehouse. Given the large maintenance burden of an unmanaged Oracle database, a large team of database engineers were dedicated to maintenance activities such as OS patching and upgrades. AFT was also challenged by complicated and inefficient hardware provisioning.

“Implementing and scaling hundreds of on-premises instances across the database fleet consumed all database team member resources for many weeks and was a relatively low value-add task for database administrators," says Brent Bigonger, senior database engineer at Amazon. “We knew we could increase efficiency and free up time if we used a managed database service in the AWS cloud.”

In May 2017, AFT decided to migrate the system to Amazon Aurora. A MySQL- and PostgreSQL-compatible database built for the cloud, Aurora provides the performance and availability of Oracle at approximately one-tenth the cost. Amazon Aurora is fully managed by Amazon Relational Database Service (Amazon RDS), which automates time-consuming administration tasks such as hardware provisioning, database setup, patching, and backups.

PostgreSQL was selected as the database because it could handle complex write transactions at speeds comparable to the Oracle system. “Our writes have a lot of components,” says Michael Wu, principal software development engineer at Amazon. “They include things like location, current state, and financial operations. The PostgreSQL-compatible edition of Amazon Aurora offers high performance for these scenarios.”

Once Amazon Aurora was selected, the team began planning the migration. AFT took a replatforming approach, preserving existing service architectures and database schemas to minimize complexity and disruption. Replatforming allowed more than 200 external services to migrate their dependencies with minor code changes.

Automation was essential to enabling such a large-scale migration. “Manual deployment simply wouldn’t work,” says Wu. “We wanted to migrate as quickly as possible by building mechanisms that would allow teams to manage the migration effectively.”

AWS Database Migration Service was used to migrate data from the Oracle databases to the PostgreSQL databases running on Amazon Aurora. Thanks to the modular nature of AWS, it took a team averaging three engineers only three months to complete the automation framework.

Because of the need for always-on operations in the fulfillment centers, the downtime window for the migration in each center was only two hours. Thanks to the automation built in the first stage, the team maintained a 100 percent migration success rate. “It’s always challenging to migrate a large system in a short time frame, but with AWS Database Migration Service, we were able to far exceed the initial expectations,” says Anthony Girolamo, senior software development manager at Amazon.

The elastic capacity of preconfigured database hosts on AWS eliminated much of the administrative overhead required to scale the system. The transformation has been significant. “In the Oracle world, a seemingly simple change such as scaling from a medium to a large database instance required provisioning hardware, standing up primary and standby databases, and managing failover during transitions, which could take a full day for each instance,” says Bigonger. “Not to mention the fact that we were using specialized hardware that had to be ordered months in advance. After migrating to Amazon Aurora, provisioning additional capacity is achieved through a few simple mouse clicks or API calls.” The scaling effort has been reduced by as much as 95 percent.

High availability is another key benefit of Amazon Aurora. “Previously, if we lost a piece of hardware, it would impact our ability to maintain redundancy for up to a day,” says Bigonger. “With Amazon Aurora, reprovisioning happens automatically in just minutes, ensuring data is always fully protected.”

With the performance of Amazon Aurora, the team is no longer limited by the input/output operations the database instance can handle. “During the process, we shed software ‘technical debt,’ making our software systems better and enabling greater agility,” says Wu. “Our engineers can kickstart projects by spinning up their own test instances whenever they need to. It took the complexity out of the process.”

Migrating IMS to Amazon Aurora resulted in a range of benefits—the most important of which is dependable fulfillment for customers. “Using Amazon Aurora, we can keep up with Amazon’s business growth,” says Girolamo. “That means customers get products faster—which is why we do what we do.”