If you recall part one of this blog post, we were going to use ChatGPT in parallel with how we would work to cover these aspects of performance testing.
We left the first part of this blog post at the point at which we had compared Requirements Gathering and Risk Assessment, we will pick this post up by looking at Script Creation before concluding with Results Analysis.
If you believe many articles online you would believe that automation in testing will soon be defined, managed and executed by Artificial Intelligence (AI). AI is embedded in many organisations technology landscape and to think that this model will change is shortsighted. AI is here to stay undoubtedly in one form or another, but should it be responsible for the automated testing of your applications under test ?
Clearly the level of involvement that AI has in your automation is up to you, but to depend on it exclusively will reduce its effectiveness and make it little more than a series of tests that provide little benefit when it comes to truly understanding how your application behaves. This post will look at performance testing and the automation that surrounds this discipline but many of the observations are also true for functional automation.
The many articles that support the wholesale use of AI in automation fail to see the bigger picture: most organizations are complex and any technology change is difficult and involves many applications and many stakeholders. So, while simply getting AI to build you a performance test suite in a language of your choice might be possible in some organizations for the vast majority, this is not a viable option.
Contrary to what this introduction might imply, the use of AI in performance testing is advantageous when used sensibly, this post will look to discuss this in more detail.
The problem with test data is that it can become stale very quickly. This is either through its use from testing or from the fact that it is naturally aging in the test environments.
This is not just an issue for performance testing, although the volumes of data sometimes required for performance testing do make it harder. This also affects functional testing as well as batch testing and business acceptance testing amongst others.
Now we have previously written posts on how after completion of performance testing you leave data created by the execution of your tests which may be of use to other members of your test or development community. And we have also discussed how you can use existing data in your performance testing environment in your tests in the most effective manner.
But in both cases, this is during or after your performance testing takes place, for performance test to be executed you need quality data in your test environments. This is also true for functional, batch and user acceptance testing, really its true for any type of activity that wants to use data in the test environments.
In this post we are going to look at how you can spot trends in your performance test results and use this trend analysis to help shape the way you address performance testing. Performance testing can generate a large volume of performance test data and using this data to define your future performance testing coverage, scope and volumes can really add benefit to your non-functional testing process.
To use data to support performance testing you are going to need to store it in a way that makes it accessible and comparable. The ideal solution and one we will discuss in this post is to store this data in a database. This post will look at the positive benefits of storing lots of performance data, especially data that spans a significant period of time, and we will look at ways of using this data to define performance test coverage and identify areas of performance concern.
In this blog post we are going to look at using JMeter to support business testing in production. This is a slightly different topic to the one discussed in this post on testing in production. The one in the link above is around running your performance testing in production for reasons that are discussed in that post. This post is going to focus on how you can leverage your performance testing tools to support this activity, as discussed above we will focus on JMeter in some of the examples. But any load testing tool, or even functional testing tool, can provide the same benefits.