Realtor.com Case Study

Realtor.com® is a trusted resource for home buyers, sellers, and dreamers, offering a comprehensive database of for-sale properties as well as the information, tools, and professional expertise to help people move confidently through every step of their home journey. As the official site of the National Association of REALTORS, realtor.com pioneered the world of digital real estate 20 years ago and today helps make all things home simple, efficient, and enjoyable. Realtor.com is operated by News Corp subsidiary Move, Inc.

Over the past several years, Realtor.com has experienced explosive growth. Industry data shows that in the first half of 2015 the company grew 38 percent year over year. “We’ve seen double-digit growth for the past 18 months. More people are visiting the site and staying longer,” says Alan Lewis, the company’s principal architect. “That represents growth not only in the number of people using the site, but also in the number and complexity of new features we’re developing.”

Supporting such growth, however, was becoming increasingly challenging for the company. “We ran our website platform in an on-premises data center,” says Lewis. “In order to scale that platform to keep up with our growth expectations, we knew we needed to get outside the constraints of the data center. For example, we didn’t want to keep investing in hardware and power and cooling costs. We also lacked the agility to quickly develop some important new features we wanted to create.”

Additionally, the company’s customers needed to change their ad campaigns in real time and not have to wait for days or weeks. This required realtor.com to evolve from batch processing to real-time processing.

Realtor.com also felt limited in terms of its ability to effectively support user traffic, which varied depending on the time of day or year. “Our traffic changes frequently, with peak hours in the evenings and lower traffic later at night,” Lewis says. “Also, the website has higher traffic on Sundays and Mondays, and during the home-buying season. It was difficult to try to estimate how much capacity we needed. If we guessed wrong, we were either not serving our customers’ needs or overpaying for unused capacity.”

The company’s desire for more scalability and flexibility led it to the cloud. “We realized the cloud was the technology we needed to help us continue our fast growth,” says Lewis. Move decided to use Amazon Web Services (AWS), which was already being used by the organization’s parent company, News Corp. “We had also acquired another company that had transferred its entire IT environment to AWS, and we saw how AWS enabled that company’s growth.”

Realtor.com started by using AWS to support the development of Turbo Listings, a new feature that customers use to purchase additional impressions for their real estate listings on realtor.com in real time. “This product gives agents and brokers a lot more exposure for their listings,” says Lewis. “It requires enormous scale, which is why we couldn’t have built it using our on-premises data center.”

Realtor.com uses AWS to track the ad impressions that customers purchase. Each ad impression is collected and streamed through Amazon Kinesis Data Streams. The ad impressions are then consumed by the company’s internal billing system, which uses Amazon Kinesis Firehose to aggregate and send the data to an Amazon Redshift data warehouse for analysis. “We use Amazon Redshift to capture each impression, and then we use the data repository for billing and auditing if necessary,” says Lewis.

Realtor.com also supports the Turbo Listings feature with an application that uses Amazon Elastic Compute Cloud (Amazon EC2) instances to run the business logic for the ad campaigns. Once all ad impressions are delivered, that ad campaign is automatically shut down. The application reads data from Amazon Kinesis Data Streams and uses AWS Lambda to process each event. “All the AWS services we use work together to manage the flow of data for Turbo Listings,” says Lewis. The company is also in the process of migrating its entire website to the AWS Cloud.

Realtor.com was assisted by AWS Professional Services during the building of Turbo Listings. “AWS Professional Services worked side by side with our developers to make sure we built the application quickly and with the highest possible quality,” says Lewis. “The AWS team helped us understand all the different options within AWS, and they recommended the approach of sending data from Amazon Firehose into Amazon Redshift.”

Realtor.com can easily scale its environment to support the growth of its Turbo Listings feature. “We are able to take advantage of the massive scale of the AWS Cloud to grow new features and expand our company as a whole,” Lewis says. “Our new Turbo Listings feature has thousands of campaigns sold, and we’re processing tens of millions of ad impressions a day. We know we can scale far higher than that as the product grows.” The company will also be able to continue its rapid website growth once it completes the migration to AWS. “Using AWS, we know we will have the capacity to support many new site visitors.”

The company’s customers can now launch campaigns and make changes to their campaigns in real time. Lewis adds, “With Amazon Kinesis and the other AWS analytics services, we can create real-time applications easily and quickly.”

Once its website migration is finished, the company will have the elasticity to automatically manage different traffic patterns. “We will be able to predict how much capacity we need without knowing any other details about a specific system, because of the automatic dynamic scalability in AWS,” says Lewis. “We can provision and use exactly the amount of compute we want and allow that to automatically shift to meet our needs. It will give us more flexibility and help us save money.”

Realtor.com also reduced development costs. “Developing Turbo Listings in our own data center would have been much more costly, because we would have to build out the underlying platform,” says Lewis. “Using Amazon Kinesis and the other AWS services, we can now build sophisticated real-time applications that wouldn’t be economically feasible on premises. That means we can build things we wouldn’t have been able to build before.”

Additionally, the company benefits from the ease of use of AWS services. “Amazon Kinesis Firehose worked without us having to do anything,” Lewis says. “There’s no administration or configuration required—it simply pipes in all the ad impression data we need to capture, which allows us to focus on building business applications and quickly creating data insights.”

Realtor.com is also better able to support its DevOps approach. “AWS has really helped us accelerate our move to a DevOps model,” says Lewis. “The AWS services are easier to understand than our data center stack was, and that puts our application developers on more common ground with our operations team. They’re not all looking through different lenses. They’re looking through a single lens—AWS.”

Realtor.com plans to complete its all-in AWS migration over the next several years. “We’re migrating everything to AWS, including our search functionality, geo-services, and business intelligence solution,” says Lewis. “AWS is the right technology to take us into the future.