BandLab Decreases Costs and Improves Availability Using Amazon S3

As a collaborative social music service, BandLab was experiencing a phenomenal rate of growth and needed to upgrade its storage and accessibility. With 15 million users producing more than half a billion audio files, plus tens of thousands of new users joining daily, BandLab sought to move beyond its existing cloud storage provider and find one that was scalable, reliable, and easy to use. Amazon Simple Storage Service (Amazon S3) turned out to be the solution BandLab needed, as it delivers scalable, secure, and reliable cloud storage, with a range of storage classes that helped the company optimize costs as it rocketed toward 26 million users by the end of 2020. 

BandLab promotes music creation in the cloud, enabling individuals to compose, mix, revise, sync, and collaborate through smartphones, tablets, and computers. A digital audio workshop hailed as a “recording studio in your pocket,” the full-featured BandLab app sees upward of 40,000 new users every day. Faced with this sustained growth, BandLab needed to find a place for the nearly 4 PB of data it had accrued and for its future storage needs. “Before, the app was running on another cloud storage provider,” says Nick den Engelsman, senior cloud systems engineer and architect for BandLab. “But as our user base grew, the previous solution was struggling with the amount of new audio being uploaded.” 

Den Engelsman was familiar with Amazon Web Services (AWS) and had been involved in AWS migrations for several years before he joined BandLab. At BandLab, he found that engineers were already experimenting using AWS and understood the benefits of migrating to AWS. The rationale for migrating was to enable scaling for growth and to establish a complete solution in which all functions could be done in one place.

“We made the decision in early 2017 that any audio-related stuff—transcoding, mixing, you name it—would be done on AWS,” says den Engelsman. “The main reason we wanted to move was because we expected the number of users would scale in the future, and as a result, so would the number of audio samples that users would upload and that we’d have to transcode. We knew that Amazon S3 would be able to handle it, as it has virtually unlimited scale.”

One of the biggest advantages of moving to Amazon S3 is that it can simultaneously store the hundreds of millions of audio files BandLab manages and run the website and apps that support the business. “All the static files are hosted within Amazon S3,” says den Engelsman. BandLab also uses Amazon CloudFront—a fast content delivery network service that securely delivers data, videos, applications, and APIs to customers globally with low latency and high transfer speeds within a developer-friendly environment—and Lambda@Edge, a feature of Amazon CloudFront that lets developers run code closer to application users, which improves performance and reduces latency. Den Engelsman explains, “We make use of Lambda@Edge and Amazon CloudFront to host the website, and the front-end team is using Angular,” an open-source web application framework. BandLab added logic within the Lambda@Edge functions to make them prerender webpages, using cookies to load users’ information as soon as they log in. “In combining Amazon S3 with the Lambda@Edge feature,” den Engelsman says, “we’re on a path where we don’t really have to think about scaling.” 

All told, BandLab stores about 500 million original files, each of which also has a transcoded version for use on mobile devices. BandLab sends different audio files to different storage classes of Amazon S3 based on how quickly users are likely to require access. Every audio sample that comes in initially remains in the Amazon S3 Standard storage class, which offers high durability, availability, and performance object storage for frequently accessed data. After 30 days, BandLab transitions audio files to Amazon S3 Intelligent-Tiering, the storage class designed for customers who want to optimize storage costs automatically when data access patterns change, without performance impact or operational overhead. “Aside from this, we have a backup account from which we replicate every incoming original audio file,” says den Engelsman. “We use Amazon S3 Replication to move them to another bucket, and there we transition them to Amazon S3 Glacier.” 

Being able to store audio files in different buckets with different levels of accessibility at different costs is a huge benefit for BandLab. Of the total projects, around one-third are considered “hot projects”—that is, projects being actively edited. The Amazon S3 storage classes enable BandLab to cut costs on storing the millions of projects that users don’t need immediate access to. “AWS undoubtedly was beneficial with cost efficiency,” says den Engelsman. “They showed us numbers and illustrated actual savings by storing samples in different storage classes. They demonstrated how by migrating to different storage classes, and possibly Amazon S3 Glacier Deep Archive, we can reduce the cost further.” The Amazon S3 Glacier Deep Archive storage class is the lowest-cost cloud storage and supports long-term retention and digital preservation for data that may be accessed once or twice in a year. 

During the COVID-19 pandemic, BandLab’s storage solution using Amazon S3 rose to the challenge when the service saw a sharp increase in users and content creation. The total number of users in December 2019 was 12 million. That number grew by 1 million per month from January to May 2020 and content creation neared 10 million tracks a month. By November 2020, it was up to 26 million users. BandLab was able to seamlessly scale to meet demand without sacrificing user experience.

 Moving to Amazon S3, which is designed for 99.999999999 percent durability, enabled BandLab to safely and cost-effectively store and provide easy access to the millions of audio files created by users. The reliability of Amazon S3 also means BandLab can focus on delivering additional functionality. The company envisions live shows, producing and selling original loops, fostering a vibrant music community, supporting artists through the sale of music and merchandise, and continuing to improve audio quality.   

Den Engelsman believes that all these ideas for the future of BandLab will be possible using AWS solutions. “I’m always amazed by all the features that still get released. Sometimes you haven’t even thought about it, and then AWS releases a new feature, and it’s exactly what you needed.” It’s not just the products that den Engelsman appreciates but that AWS focuses on the customer. “I find it amazing that we can just have these close relationships with the product teams, and we are actually able to speak with the people within product teams,” he says. “It’s really nice to see.”