--- title: Performance, Load and Stress Testing Explained url: https://blog.octoperf.com/performance-load-and-stress-testing-explained/ date: '2018-04-09' authors: - jloisel categories: - Load-Testing tags: - Performance Testing - Load Testing - Scalability description: It's not easy to know the key differences between stress testing, load testing, soak and performance testing. When to run each kind of test? Never get stuck during a job interview again. --- # Performance, Load and Stress Testing Explained This is a typical questions [QA Engineers](https://www.quora.com/What-does-a-QA-engineer-do) and [Devops](https://www.quora.com/What-is-DevOps) may be asked during an interview. And **we know how confusing these tests can be**. That's why we've written this article to **explain once and for all** the key differences between all types of **performance testing**. As stated on [Software Performance Testing](https://en.wikipedia.org/wiki/Software_performance_testing#Load_testing), there are many different ways to test software including: - [Load Testing](https://en.wikipedia.org/wiki/Load_testing), - [Stress Testing](https://en.wikipedia.org/wiki/Stress_testing) (also called **Torture Testing** sometimes), - [Soak Testing](https://en.wikipedia.org/wiki/Soak_testing): testing over long period of time, - **Spike Testing**: simulate huge spike of load on a very short period of time, - [Scalability Testing](https://en.wikipedia.org/wiki/Scalability_testing): test how the applications scales horizontally or vertically. Each type of testing has its own set of rules and goals. Let's explore which type of testing you should use, when and the results you get. Let's start our **visual tour**! ## Performance Testing ### Introduction All the type of testing mentioned above are in fact **Performance Testing**. It's the general category in which each sub-type of testing falls. Typically, performance tests are conducted to validate: - **Software changes**: that newly introduced changes into the tested software don't break it, - **Continuous Performance Validation**: validate the performance of the system, not only [correctness](https://en.wikipedia.org/wiki/Correctness). As Wikipedia explains it so well: > In software engineering, performance testing is in general, a testing practice performed to determine how a system performs in terms of responsiveness, [scalability](https://en.wikipedia.org/wiki/Scalability) and [stability](https://en.wiktionary.org/wiki/reliability) under a particular workload. ### Testing Methodology This typically involves the following steps: - **Recording a simulation** of the steps to perform on the target application, - **Running** those simulation **at large scale** with thousands of concurrent users, - **Analyze Test Results** to find [bottlenecks](https://en.wiktionary.org/wiki/bottleneck), and other performance degradations. I know **how impatient you are** so let's see them in greater details! ### Load Testing ![Load Testing](https://blog.octoperf.com/img/blog/performance-stress-load-testing-explained/load-testing.png "Rampup Load Test with peak load maintained for a few minutes or hours") This is the simplest form of performance testing: - **Duration doesn't import much**: Although a minimum duration is required for meaningfull results, the goal is mainly to reach a target load and check the system status, - **Rampup Load**: usually, load is gradually increased until reaching a peak load which is maintained afterwards, - **Monitoring**: server-side components are usually monitored during the load test to pinpoint any backend issue. The goal of load tests is to **validate that the system can handle a given load**. As it's general purpose, load testing is suitable for pretty much any simulation whose results should indicate how well the system behaves under a given load. Those tests are generally runs as part of a [CI/CD cycle](https://en.wikipedia.org/wiki/Continuous_integration) although in that context they might be simplified a bit for greater maintainability. ### Soak Testing ![Soak Testing](https://blog.octoperf.com/img/blog/performance-stress-load-testing-explained/soak-testing.png "Gradual Load Increase with long running peak load") This kind of test has the following characteristics: - **Long-running**: involves testing a system with a typical production load, over a continuous availability period, to validate system behavior under production use, - **Production Load**: simulate load which is as close to load seen in production as possible, - **May be shorter duration than in production**: simulated load can be accelerated in order to test more quickly, - **Reach up to peak load during test**: max simulated load should reach production peak loads, and not just average load. This kind of test is also known as **endurance testing**. It involves testing the system for an extended period of time with load near to peak admissible loads. The system should maintain the performance over a long period of time. The goal is to determine **how the system behaves under sustained load**. Whenever you need to determine the absolute number of users a system can handle. **Soak Tests** should be run regularly to check the system is maintaining a good endurance even after code updates. ### Spike Testing ![Spike Testing](https://blog.octoperf.com/img/blog/performance-stress-load-testing-explained/spike-testing.png "Dramatically increase load for a short period of time, aka a Spike") This kind of test has the following characteristics: - **Short-running**: test is ran very quickly (usually a few minutes), - **Spike Load**: load increases dramatically and very quickly to a huge surge. The goal is to test the **system resistance** in the event of a sudden spike of concurrent users. **Spike Tests** are suitable to prevent issues during short-term floods of users. Typically, this happens during events like the *Black Friday* on online shops. It can be during a TV ad campaign too, bringing a flood of users when the ad is run. ### Stress Testing ![Stress Testing](https://blog.octoperf.com/img/blog/performance-stress-load-testing-explained/stress-testing.png "Increasing Load during the whole test to find the break-point") Typically, stress tests involve: - **Always Increasing Load**: load is increasing during the whole test until the servers being tested fall apart, - **Stepping load** : it is recommended to increase load with steps so that you can maintain a stable level of load for some time before going to the next one. This will make analysis of server behavoir much easier for each step, - **Various Durations**: test duration only depends on the rate of load increase you want to apply on the system, - **Goes well above expectations**: load can increase well beyond what the system has been designed for. The goal is to **check robustness** of a system in case of **extreme load** (greater than system capabilities). This may be followed by a light load test to check how much time the system needs to recover from the surge. ## Recap Table As you can see, the differences can be pretty much resumed in the following table: | **Test Type** | **Duration** | **Simulated Load** | **Usage** | |-----------|----------|----------------|-------| | **Load Testing** | Variable | Slowly increasing, then maintain | As part of CI/CD cycles | | **Soak Testing** | Long | Mostly maintain 90% peak load | Test system's resistance to sustained load | | **Spike Testing** | Short | Surge increases with Spikes | Test how the system handles huge load on Ponctual Events | | **Stress Testing** | Long | Always Increasing | Find system's max load; test system resilience | The **shape of the load curve** really determines what kind of test you are running. We hope this article will help you land the [Performance Engineer job](https://www.indeed.com/q-Performance-Engineer-jobs.html) of your dreams!