Skip to content

2015

Performance testing and objectives

There's no denying the importance of a performance test campaign in the quality assurance process. But I have lost count of customers requesting load test with no specific objective (or just "improve the performance"). Running a test campaign with no objective in mind is quite risky. You might spend a lot of efforts on useless improvements whereas the critical pain points remain.

Endless tuning

Plan

I worked on performance campaigns with no objective defined a couple of times. Usually, this implies running tests until you run out of time/budget and optimizing whatever you can. It does not mean these campaigns were not successful, but they probably cost a lot more money than they should have.

What makes a good performance tester

You have probably read countless articles about what a good tester is. Of course this is a tricky question and there is no perfect answer. Performance testing requires a mix of technical, organisational and communication skills.

I have seen good technical testers fail to deliver the right message even if they identified the source of all issues.

On the other hand, I have seen performance testers with low technical skills perform very well thanks to their communication.

There is no perfect mix of expertise I would recommend but instead, let's see what the testing challenges are.

What to do with my results

Since I started my career as a tester, I'm often impressed by how disregarded communication in load and performance testing is. However, you can be as skilled as you want as a technical tester, if you can't explain your analysis and convince decision makers to take actions, most of your work was in vain. In fact, I've seen colleagues who were not interested in the technical aspects of the job get better customer satisfaction. The good thing is, communication is just a skill amongst others that you can train.

Today I would like to share everything I learned about presenting results.

Once upon a time in my load test campaign

Once upon a time

In my opinion, you should always come up with a short story describing your load test campaign. Rest assured, it does not have to include fairies and princesses, the truth will do just fine. It allows everyone to understand quickly what happened during the campaign. And it's pretty obvious that a good story always sells better.

Which environment to use?

We can all agree that running a realistic load test is important to assess server performance correctly. One of the aspects of such a test is the testing environment. It must be selected carefully based on your requirements.

The sooner the better

Since I began my career as a tester I've been told it is better to test early to prevent costly defects. But sadly I must admit I ran a lot more tests in production than in an early Q&A environment. This usually happens because load testing is disregarded and only conducted at the last moment. However, testing early comes with a lot benefits even if it is also quite different than a typical load test.

What is the plan?

If you are a tester, chances are you already know what a test plan is. If not, the short definition would be : "A specification document of your load test campaign".

This document is very close to the functional test plan in its structure because both share some common ground. The question I'm often asking myself when preparing a load test is if this document should be used at all.

Joker-plan

Don't get me wrong, the thinking behind it is mandatory. Otherwise your test campaign might not be very effective. But what about the actual document a tester usually produces? For years, I have been a tester working for different customers every week or every two weeks. I have to admit that I don't rely on a test plan half of the time. Even if you don't switch context so often, I believe there is some common ground depending on the project length and complexity.