Skip to content

Load-Testing

Cloud testing vs Internal testing

The rise of cloud platforms lead to a major evolution for load and performance testing. Since generating load requires a powerful infrastructure for a short period of time, one might even argue this is the perfect use case. But since we started OctoPerf we had a lot of feedback from users wanting to test from their own machines for a lot of good reasons.

Web application does not mean web-site

First of all, not all web applications are available on the internet. In particular test environments, which are usually mostly meant to be used internally. As testing the application before its deployment is important, the load tests are often done on such environments.

Also, large companies have a wide range of intranet applications that can't be accessed from outside their network. Of course they need tests and in that case testing from the cloud does not make much sense.

Should I test my external providers

If you ever recorded a load testing scenario on a website you probably noticed that the list of external links can be scary: Server List

Plus these links might occur several times, making your user profile difficult to read. Because of this even a simple task like separating the different steps into distinct transactions can become tedious: Lots of requests

Every time I work on such tests the question is always the same: "Do we need to simulate these requests?"

What makes a realistic load test

Performance testing covers a wide range of different tests. The benefits can vary whether you test a single URL or a complete user journey through the application. It might seem obvious, but to add the most value to your load tests you should make them as realistic as reasonably possible.

What is the point of making a test realistic?

Quick and unrealistic load tests often prove useless or even counterproductive. It is often the best way to be overwhelmed with unreliable data.

overwhelmed

A realistic load test will allow you to:

  • Stress all layers of your application. This way you can assess the performance of all your servers and detect bottlenecks.
  • Pinpoint real bottlenecks. For example a simple load test based on a single URL might point to limitations on your front servers when actually the real users won't experience it because the backend is the first to fail.
  • Avoid false positives. If you fail to launch realistic tests and encounter a lot of errors, you might stress the CPU of your front servers to handle these errors. The same goes for the network if you don't simulate the load from the right location.

Preparations and prerequisites

During my past years as a tester, I often wondered how to get a higher customer satisfaction. I could go great lengths describing the required skill set and give examples (actually I already did that over here). When actually the truth is much simpler.

It's not rocket science, it's all about preparations and prerequisites.

Prepare to be prepared

Prepared

Over the years, I have worked with a lot of testers. We more or less shared the same views on how to do our jobs and even when all was not going according to the plan.

Performance testing and objectives

There's no denying the importance of a performance test campaign in the quality assurance process. But I have lost count of customers requesting load test with no specific objective (or just "improve the performance"). Running a test campaign with no objective in mind is quite risky. You might spend a lot of efforts on useless improvements whereas the critical pain points remain.

Endless tuning

Plan

I worked on performance campaigns with no objective defined a couple of times. Usually, this implies running tests until you run out of time/budget and optimizing whatever you can. It does not mean these campaigns were not successful, but they probably cost a lot more money than they should have.