Qytera - Case study
Qytera software is a company dedicated to providing fast and effective quality assurance for their customers. Their area of expertise includes Test Management, Test Automation, Continuous Testing and Agile Testing. All of this centered around the latest web and mobile technologies.
As part of an incoming load test campaign, he was looking for a solution to easily scale his JMeter tests.
His challenge was that he had to generate a huge amount of load from various locations in the cloud
but the customer wasn't exactly sure of how much load had to be generated.
Initial situation¶
Qytera was tasked with testing the application for local elections in a federal state. The customer was not quite sure of the exact load to expect on their application. It is always difficult to anticipate how many concurrent users to expect on an application, but for elections it can get even more difficult since these do not occur on a regular basis. The test would also have to be conducted from several cloud locations to emulate different network conditions and latency.
From all these requirements, Moritz derived that a huge installation would be required in order to launch all the tests. It might have to be kept alive for a while since the lack of a clear goal meant the tests were going to be an iterative process. Creating an maintaining it all could notable increase the cost of the test campaign. This made a strong case for finding a tool to answer all these requirements.
Early work with OctoPerf¶
At first OctoPerf's user interface made it easy to design the scripts, meaning little time would have to be spent in that area. The automatic download of resources made this even easier, since it would automatically pick up new statics in case the application changes.
Other useful features were the CSV variable to feed a unique value to every user and the DNS override to bypass the gateway.
Baseline¶
Once these were in place, the first step was to find a proper baseline that would then be used to scale the test at higher loads. Finding the proper hit rate per concurrent virtual user can be a challenge and that's where the support team was able to help.
Once this step was out of the way Moritz was able to run his firsts tests and estimate the load handled by the application under test. At this stage it was very easy to assess what would be the cost of running a full blown test campaign, and this was achieved at a minimal cost since all OctoPerf's licenses include every feature.
Load tests¶
The rest of the test campaign was an iterative process, with various load levels. The only change required to increase the load was a simple configuration change in OctoPerf and our auto-scaling would do the rest. And with as many different cloud locations as needed.
The reliability of OctoPerf results allowed Moritz to quickly compare configuration changes and along the way more than 4 bottlenecks were identified and fixed:
- Misconfiguration in a certificate,
- Incorrect network route,
- Misconfiguration of network cards,
- Insufficient bandwidth on some of the network components.
Conclusion¶
The combination of OctoPerf's product and support with Moritz's work and knowledge made this load testing campaign a success. This collaboration allowed everyone to focus on their area of expertise which is key to bring the most efficient outcome for everyone.
We know we will continue to collaborate in the future and we would like to take this opportunity to thank Moritz and Qytera Software for trusting us with this project.