Due to its rapid growth Skyscanner had serious performance problems with the aggregation of price data. The ever-increasing volume of information did not allow the existing system to keep up and this caused delays in sending responses as well as data inconsistency. As a result of this, users were disappointed by not seeing actual prices on the website, which meant decreased revenue for the company.
The new system divided the single complex existing process into several smaller tasks, each individually scalable depending on demand, thus improving overall performance and eliminating the single point of failure problem of the previous system. The newly designed system also allows for easy definition of new data aggregations, allowing the company to present users with more relevant search results, including flights with multiple stops, flights available to any airport in a given country, flights available to ski resort areas, and many more.
Thanks to the implemented solutions, the database size shrank 10x and the execution time of new user queries was cut by half, thus significantly improving overall response time. Moreover, aggregations are now simple to manage and maintain, and the architecture is much easier to scale.