Problem / Goal
During our cooperation with Skyscanner we performed the following test types:
Usability tests were conducted in the initial stage of nearly every project. They were performed with the participation of random people, third parties (eye-tracking) and inside the company.
Specifications were analysed by quality assurance and localisation engineers.
Many times the results of such tests had a great impact on the projects. Thanks to them the company could discover weak projects early enough, thus avoiding substantial costs of implementing projects destined to fail.
During project implementation our QA engineers actively participated in the verification and validation of new functionality. Such tests were carried out either on the basis of documentation or discussions with project managers and developers. Tests were already performed using Agile methodology in 2011.
For testing purposes usually a “black box” method was used, but many times low-level tests were performed. Database interfaces, buffering or database commits etc. were verified.
Tests automation and regression testing
An increasing number of functions available to users necessitated the automation of as large a number of tests as possible. For this purpose, we used various tools, such as WebAii, Selenium, Fitness and C# and Python programming languages. About 60% of tests were automated.
In 2011 we implemented an innovative system of performing automated testing (TES), which allowed for parallel test execution in virtual machines. Such a system allowed us to shorten the time of automated test execution from 12 h to about 2 h.
In parallel to automation, documentation of regression testing (partially executed manually) was also maintained. The TES system was integrated with the code continuous integration system, which we also helped build.
A very important part of the project was its localisation into 24 languages. Once translations were received, a suite of localisation tests was conducted on the site. We also hired third parties to assess translation quality.
In 2009/2010 our team was responsible for the deployment of the website code onto public servers. To maintain the highest quality we used the staged publication method. At the same time, during a release we used performance tests based on the WebLoad application and our own tools that monitored database performance.
Some projects had many variations or configuration settings and required live verification. Our team participated in A/B test implementation using Google Web Optimizer and Google Analytics (private variables).