Atlassian Reduces Bitbucket Response Times by up to 45% on AWS

In early 2020, Australian software company Atlassian saw an opportunity to help users of its code-management service Bitbucket by improving performance and removing friction and uncertainty from its network. Specifically, the company wanted to reduce latency, improve reliability in key regions, and reduce network-related support cases. The global footprint of Amazon Web Services (AWS) helped Atlassian move internet traffic efficiently without time-consuming, costly changes to the Bitbucket application.

Atlassian used AWS Global Accelerator, a networking service that routes user traffic through the global network infrastructure of AWS to improve internet user performance by up to 60 percent. Its use enabled Atlassian to bring Bitbucket’s edge closer to users, reducing the potential problems created by internet service provider (ISP) hops and improving application performance.

Atlassian’s project management applications help teams collaborate effectively. Bitbucket manages and stores source code and gives teams one place to plan projects, collaborate on code through peer review, and test and deploy. More than one million teams and tens of millions of developers worldwide use Bitbucket, and although its AWS data center is based in the US East (Northern Virginia) Region, over 50 percent of its users are based outside the US. 

Bitbucket’s previous network solution used single-homed Network Load Balancers with a small number of static internet protocol (IP) addresses to accommodate customers who use IP allow lists. Network Load Balancers route connections to targets within Amazon Virtual Private Cloud (Amazon VPC), based on IP data. Amazon VPC enables users to launch AWS resources in a logically isolated, user-defined virtual network, offering complete control over the virtual networking environment.

However, customers far from Bitbucket’s network edge experienced latency issues because traffic had to travel significant distances across the public internet to reach Bitbucket’s home base. Customers reported internet outages, blocked IP addresses, and slow service. “To get from a customer’s house to Bitbucket, there might be 5–10 ISPs in between,” says Ben McAlary, principal engineer for network services at Atlassian. “We were feeling pressure from the customer base to address Bitbucket’s performance issues.”

To address these issues, Atlassian decided to use AWS Global Accelerator. The service has more than 95 worldwide points of presence, so Bitbucket can ingest user traffic into the AWS network’s closest location, route it to the healthiest application endpoint, and then return a response to the optimal user endpoint. Bitbucket now avoids congestion and unreliable local ISPs, which in turn facilitates faster code collaboration for users and relieves pressure from Bitbucket’s team to resolve complex network problems outside the company’s control.

AWS Global Accelerator supports static anycast IP addresses using Atlassian’s own provider-independent IP address space. This enables Bitbucket customers to allow list the same IP addresses and connect to them from anywhere. For security reasons, many of Bitbucket’s large customers are strict about which IP addresses can have their source codes modified. “Many more customers than we expected use allow listing to permit only certain operations to happen with certain IP addresses,” McAlary says. “Without using a static IP on AWS Global Accelerator, that capability would have been a no go.”

Additionally, Bitbucket’s implementation of Git over the network protocol secure shell and, in particular, its IP-based access controls for code repositories and workspaces require its network to preserve and carry the customer’s source IP address. AWS Global Accelerator can fulfill this requirement, upholding Bitbucket’s support of access controls. “If we were losing the information about the source IP address, the allow list feature would break or we’d need to rewrite it,” says McAlary. “Implementing AWS Global Accelerator was one of the simplest infrastructure tasks we’ve ever done, and we did it without making any changes to the application.”

The major obstacle to migrating Bitbucket traffic to AWS Global Accelerator was the application’s use of nonstandard network ports. Atlassian needed to use AWS Global Accelerator to direct traffic to the port that the application expected. A port override function developed by the AWS team helped resolve this remaining issue quickly. “Throughout the process, the AWS Global Accelerator team was willing to support us and was attentive and responsive to any questions we had,” McAlary says.  

To migrate the Bitbucket domain from Network Load Balancers to AWS Global Accelerator, Atlassian used Amazon Route 53, a scalable cloud Domain Name System web service designed to give developers and businesses a reliable, cost-effective way to route users to internet applications. The solution enabled Atlassian to migrate customers by shifting traffic from Network Load Balancers to AWS Global Accelerator in 1 percent increments, regularly checking for user issues. Atlassian migrated 100 percent of its traffic to AWS Global Accelerator between mid-October and mid-November 2020, producing performance improvements that Atlassian could not have realized without building multiregion edge load balancing or decomposing Bitbucket. “We brought forward a capability in the application that would have otherwise taken Atlassian many years to build,” McAlary says.  

Performance improvements in the Bitbucket application impact both the secure shell–powered Git tool, which helps developers pull or push code quickly, and the front-end code collaboration space. Synthetic tests showed improvement in Bitbucket’s web application experience, suggesting that users feel a benefit during browser-based code review. 

AWS Global Accelerator improved response times by 8–45 percent globally, with a 20 percent improvement on the US West Coast and in the United Kingdom and Ireland. “We can occupy years of several engineers’ time to get a 5 percent improvement in application performance,” says McAlary. “To get a 20 percent improvement from two engineers’ work for a month is outstanding.” On average, users saw a 500 ms improvement, and certain users with slower response times saw an improvement of 800 ms. Some customers also experienced an improvement of 100 percent or more in download speed. 

Using AWS Global Accelerator, Atlassian moved Bitbucket’s network connection closer to users and improved performance, reliability, and function to enable faster code collaboration. The solution gave Atlassian greater control over the network experience, reducing the possibility of external ISP issues and increasing productivity for Atlassian and Bitbucket teams and users. “AWS went out of its way to help us and even built a feature for us,” says McAlary. “That’s pretty incredible.”