Large data processing

Problem / Goal
Apator needed a scalable and efficient solution for processing large quantities of data. The solution needed to be robust and fault tolerant.
Solution
The solution involved the design and implementation of multiple subsystems operating in a layered model. A web-based panel with an advanced UI was created, allowing the user to gain an understanding of the system’s condition.

The system size and complexity required extensive testing, both manual and automated. The latter was accomplished through a large number of FitNesse tests that were run on demand during modifications of key components of the system. These tests included multiple scenarios for typical system operations and edge cases along with failures of specific components. Additionally, a solution for monitoring key components was designed and deployed.

Results
Close cooperation between Order Of Code and Apator teams resulted in the creation of a high-quality system which was able to process large quantities of data. System performance was several times higher than initially requested by the client.

Team

Wit Więch
Co-founder and CEO
Tomasz Marcinek
Co-founder and Software Architect
Kamila Marcinek
Database Developer
Agnieszka Niedziela
Junior Database Developer
Radosław Szuban
Infrastructure Engineer
Norbert Tu-Van
Software Developer
Radosław Wierzbicki
Software Developer