Skyscanner – website testing

Problem / Goal
From the beginning of our cooperation with Skyscanner in 2005, all Skyscanner website quality assurance tasks were in the hands of our team. At first, we performed simple verification and validation tasks, but as the functionality of the website grew, more sophisticated testing tasks appeared.
Solution
We used typical tools to store test descriptions and bug reporting e.g. Testlog or Fogbugz.

During our cooperation with Skyscanner we performed the following test types:

  • Usability testing

    Usability tests were conducted in the initial stage of nearly every project. They were performed with the participation of random people, third parties (eye-tracking) and inside the company.
    Specifications were analysed by quality assurance and localisation engineers.

    Many times the results of such tests had a great impact on the projects. Thanks to them the company could discover weak projects early enough, thus avoiding substantial costs of implementing projects destined to fail.

  • Functional testing

    During project implementation our QA engineers actively participated in the verification and validation of new functionality. Such tests were carried out either on the basis of documentation or discussions with project managers and developers. Tests were already performed using Agile methodology in 2011.

    For testing purposes usually a “black box” method was used, but many times low-level tests were performed. Database interfaces, buffering or database commits etc. were verified.

  • Tests automation and regression testing

    An increasing number of functions available to users necessitated the automation of as large a number of tests as possible. For this purpose, we used various tools, such as WebAii, Selenium, Fitness and C# and Python programming languages. About 60% of tests were automated.

    In 2011 we implemented an innovative system of performing automated testing (TES), which allowed for parallel test execution in virtual machines. Such a system allowed us to shorten the time of automated test execution from 12 h to about 2 h.

    In parallel to automation, documentation of regression testing (partially executed manually) was also maintained. The TES system was integrated with the code continuous integration system, which we also helped build.

  • Localisation testing

    A very important part of the project was its localisation into 24 languages. Once translations were received, a suite of localisation tests was conducted on the site. We also hired third parties to assess translation quality.

  • Performance testing

    In 2009/2010 our team was responsible for the deployment of the website code onto public servers. To maintain the highest quality we used the staged publication method. At the same time, during a release we used performance tests based on the WebLoad application and our own tools that monitored database performance.

  • A/B testing

    Some projects had many variations or configuration settings and required live verification. Our team participated in A/B test implementation using Google Web Optimizer and Google Analytics (private variables).

Results
The testing services offered by our company to Skyscanner ensured reliability and optimum operation throughout the period of cooperation. The quality assurance process proposed by OOC resulted in the customer being sure that the software is made according to specifications, while the validation we performed ensured that the software is also compatible with the user’s expectations.

TEAM

Wit Więch
Co-founder and CEO
Małgorzata Marek
Co-founder and QA Manager
Radosław Szuban
Infrastructure Engineer