Hidden Benefits of Performance Testing
If you are a performance testing specialist or a QA Manager or Programme Manager or anyone involved in the production of quality software then you understand why performance testing is required and its benefits in ensuring your products meet your Quality Criteria for release into production.
The costs of delivering performance testing are easily worth the investment as software that performs not only ensures you and your company have a reputation for delivering well performing software but business users will, in my opinion, overlook and embrace small functional workarounds if the software performs well.
As an investment it is worth the cost of building robust and reusable performance tests for the purposes of performance regression testing and that in itself will justify the cost of keeping them current and executable against the latest versions of code which is a maintenance activity that needs to continue as your product changes and evolves.
Some organisations leave their performance tests purely for the purpose of performance testing but there are other uses for these performance tests and by using them for these additional uses you can save time and money on the building and maintaining alternative tests.
We are going to look at some of the additional uses of performance tests in this post that will further justify the building of a complex performance testing suite.
Why do this¶
Before looking into the alternative uses it is worth justifying the reasons to do so.
The additional benefits that can be gained from re-using your performance test are:
- You have already built the tests and the investment has been made,
- You are reusing something you are already maintaining,
- These additional uses will assist other aspects of the QA and development effort,
- It will ensure you deliver a better product,
- It encourages other members of the team to execute and maintain the performance tests encouraging knowledge sharing amongst the teams.
What are these benefits¶
The benefits your performance test can bring are numerous:
- Ability to create data in large quantities for the purposes of testing,
- Measure of system availability in the form of a sanity or smoke test,
- System reconciliation activities,
- Hardening and race condition tests,
- Test production monitoring tooling.
Benefits in detail¶
Let's have a look at these in more detail as their benefit is not always obvious and it is important that they are an additional benefit rather than more work for the performance testing team, they must be an activity that the tests and the testing framework can accomplish with little or no additional effort.
Data Creation¶
It is likely that your performance test will as part of their execution cycles create data, whether this is users, policies, claims, accounts, anything really; the list is endless, and this data can be extremely useful for other members of your QA team.
If functional tests are automated they may create data from scratch to test against, but they may not and if the functional testing effort is manual then data creation can be an overhead that slows down the QA effort.
If you are following best practise in terms of modularisation and parameterisation of performance scripts, then changing the environment against which they are run against should be trivial and you can easily schedule overnight tests to generate data for your QA colleagues.
System Availability¶
As with the data creation a modular, parameterised approach would make the execution of you tests against any environment simple. After each deployment a single successful iteration of each test would be enough to satisfy the programme that a build was stable would be really useful.
The process of the test running after an evening deployment with a pass or fail status when the teams arrive in the morning would be extremely useful.
System Reconciliation¶
Many projects will require a data migration from a legacy system and once the migration exercise is complete, whether as part of testing or in the production environment when it is done as part of go-live there is a requirement to ensure the data is accurate.
Many performance testing scripts check data volumes in a database or check the quality of data in the database as part of performance testing, an example being where you are performance testing a message queue and you want to time how long the queue takes to process a set volume of messages.
Where you have tests that test this behaviour then checking table volumes from one database to another is easy to do and checking data quality just as easy.
Hardening Tests¶
We are talking about race condition tests or component collision tests here where you may want to simulate out of sequence messages arriving at the same time or in quick succession to ensure the components handle idempotency.
Performance testing by its very nature relies on the ability to simulate load from many different technologies and protocols and the ability to simulate these conditions is certainly something that the performance testing team should be involved in.
Production Monitoring Tooling¶
A new application or system being developed will, as part of the process of putting it into production, require some tooling to ensure critical thresholds are not exceeded.
These production monitoring tools do sometimes work on thresholds being exceeded and their configuration can be difficult to judge. Execution of peak volumes of load and concurrency can help with the configuration of these tools in terms of expected CPU, memory etc usage at seasonal peaks.
Conclusion¶
It is not about giving the performance testing teams more work its just about re-purposing scripts. If you have scripts that cover these additional activities, then use them for the benefit of the overall QA effort.
Several of the examples may require a little bit of additional work but certainly not significant.
There may be many more examples you can think of that are relevant to your organisation and I hope that this post encourages you to use the performance testing suite you have carefully built to help other members of the QA, or Development, effort as ultimately you are all working towards the same goal.