---
title: Soak Test – A Practical Example
url: https://blog.octoperf.com/soak-test--a-practical-example/
date: '2019-02-21'
authors:
- sjenkins
categories:
- JMeter
tags:
- Design
- Debug
- Scalability
description: A look at how we can build a complex soak test using BeanShell Samplers.
---

# Soak Test – A Practical Example

As performance testers [Soak Testing](https://en.wikipedia.org/wiki/Soak_testing "Soak Testing") is a test that is often overlooked.

We always think of [Peak Volume](https://en.wikipedia.org/wiki/Load_testing "Peak Volume") tests and we always like to push the system to its limits in the form of a [Scalability Test](https://en.wikipedia.org/wiki/Scalability_testing "Scalability Test") but often the [Soak Test](https://en.wikipedia.org/wiki/Soak_testing "Soak Test") is forgotten about.

It’s a great test and it tells you **many critical aspects** about your application under load, things that **no other test can** and is critical in determining if you application is fit for production.

**What is it?**

It’s a test that run for a **long period of time**, what is a long period of time .. well it depends upon your application, organisation, production requirements, appetite for testing….


## Soak Test Considerations

Lets consider the first 3; **application**, **organisation** and **production requirements**.

If you work in an organisation where the production systems are shut-down and re-started every night then your [Soak Test](https://en.wikipedia.org/wiki/Soak_testing "Soak Test") has to last no longer than 24 hours.

If however your application is re-started weekly as part of a maintenance cycle then your [Soak Test](https://en.wikipedia.org/wiki/Soak_testing "Soak Test") needs to last 7 days.

If you application is 24/7/365 then you are not going to test for a year, that is not practical.... My view is that **2 overnight cycles** are required at the minimum so effectively a **3 day test**.

### Why do we test

* To expose memory leaks
* To check for suitable garbage collection policies
* To check that connection pools are being recycled
* To make sure log cycling is appropriate
* To check file handlers are being recycled

etc...etc...etc...the list goes on… really anything that will not be picked up on by tests that only last a couple of hours.

### Building a test
Ok so we have discussed the reasons and benefits for this much undervalued test so lets think about how we test it and how we can leverage [JMeter](https://jmeter.apache.org/ "JMeter") to support this.

### Volumes
If you have built a [Peak Volume Test](https://en.wikipedia.org/wiki/Load_testing "Peak Volume Test") and you understand your seasonal transaction and concurrency peaks then a [Soak Test](https://en.wikipedia.org/wiki/Soak_testing "Soak Test") needs to be run at approximately 60% of these volumes to simulate volumes closer to your application averages.

## JMeter Test Plan
We recently posted a **Blog** about how to use properties in [JMeter](https://jmeter.apache.org/ "JMeter") to make [Flexible and Configurable Test Plans](https://blog.octoperf.com/jmeter-properties-configurable-test-plans/ "Flexible and Configurable Test Plans"), it is worth reading this and understanding the principles as we will use these principles to show how with the simple use of a [JSR223 Sampler](https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Sampler "JSR223 Sampler") we can easily and elegantly turn your existing tests into a soak test that is easy and flexible to control.

Let’s start with our test, it is a simple tests consisting of 4 [Dummy Samplers](https://jmeter-plugins.org/wiki/DummySampler/ "Dummy Samplers")

![Simple Test Plan](https://blog.octoperf.com/img/blog/soak-test-practical-example/simple-test.jpg "Simple Test Plan")

You will notice that we have replaced the **Thread Properties** and **Scheduler Configuration** sections with [Simplified Property Function](https://jmeter.apache.org/usermanual/functions.html#__P "Simplified Property Function"), the rationale for this is discussed in our **Blog Post** on [Flexible and Configurable Test Plans](https://blog.octoperf.com/jmeter-properties-configurable-test-plans/ "Flexible and Configurable Test Plans")

The samplers have a [Constant Throughput Timer](https://jmeter.apache.org/usermanual/component_reference.html#Constant_Throughput_Timer "Constant Throughput Timer") to manage the transaction rate.

We are using [Dummy Samplers](https://jmeter-plugins.org/wiki/DummySampler/ "Dummy Samplers") so we don’t [Soak Test](https://en.wikipedia.org/wiki/Soak_testing "Soak Test") a live web site for the purposes of this post
 
![Dummy Sampler](https://blog.octoperf.com/img/blog/soak-test-practical-example/dummy-sampler.jpg "Dummy Sampler")

The [Test Plan](https://jmeter.apache.org/usermanual/build-test-plan.html "Test Plan") is not that important, and it could be any test that you need to meet your [Non Functional Requirements](https://en.wikipedia.org/wiki/Non-functional_requirement "Non Functional Requirements")

OK now to show you how easy it is to make this into a simple configurable [Soak Test](https://en.wikipedia.org/wiki/Soak_testing "Soak Test"), firstly we need to add a [JSR223 Timer](https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Timer "JSR223 Timer") to our test
 
![JSR223 Timer](https://blog.octoperf.com/img/blog/soak-test-practical-example/jsr223-timer.jpg "JSR223 Timer")

This [JSR223 Timer](https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Timer "JSR223 Timer") simply tells all threads in the [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group") to sleep for a period of time, this needs to be a [Simplified Property Function](https://jmeter.apache.org/usermanual/functions.html#__P "Simplified Property Function") in order for the test to work and to help you understand how parameters can be abstracted from the test see our **Blog Post** [Flexible and Configurable Test Plans](https://blog.octoperf.com/jmeter-properties-configurable-test-plans/ "Flexible and Configurable Test Plans")

Initially we set the **threadSleep** [Simplified Property Function](https://jmeter.apache.org/usermanual/functions.html#__P "Simplified Property Function") in the  [JSR223 Timer](https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Timer "JSR223 Timer") above to be **0** as we do not want to pause the test and the **dRation** [Simplified Property Function](https://jmeter.apache.org/usermanual/functions.html#__P "Simplified Property Function") that controls the **Test Duration** to an arbitrary value. 

These will both be **overwritten by code** as the test executes and we will discuss this next

To demonstrate our parameter values the **[properties file](https://blog.octoperf.com/img/blog/soak-test-practical-example/soak.properties)** we will use looks like this:
 
![Properties File](https://blog.octoperf.com/img/blog/soak-test-practical-example/properties-file.jpg "Properties File")

We now have a simple [Test Plan](https://jmeter.apache.org/usermanual/build-test-plan.html "Test Plan") configured to run for **300** seconds at a rate of **600** transactions per minute, this does not seem like much of a [Soak Test!](https://en.wikipedia.org/wiki/Soak_testing "Soak Test!")

Now to turn this into a [Soak Test](https://en.wikipedia.org/wiki/Soak_testing "Soak Test") we simply add a separate [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group") and a [JSR223 Sampler](https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Sampler "JSR223 Sampler")... simple really, let’s do this now.

![Thread Group](https://blog.octoperf.com/img/blog/soak-test-practical-example/threadgroup.jpg "Thread Group")

You will notice that this [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group") contains a single Thread and uses the same [Simplified Property Function](https://jmeter.apache.org/usermanual/functions.html#__P "Simplified Property Function") value as the **dRation** [Simplified Property Function](https://jmeter.apache.org/usermanual/functions.html#__P "Simplified Property Function") in the **[octo-perf-soak](https://blog.octoperf.com/img/blog/soak-test-practical-example/octoperfsoak.jmx)** [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group") this ensures that both finish at the same time.

Now we add our [JSR223 Sampler](https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Sampler "JSR223 Sampler") that does all the calculations.
 
![JSR223 Sampler](https://blog.octoperf.com/img/blog/soak-test-practical-example/jsr223-sampler.jpg "JSR223 Sampler")

We will look at our script in a moment, but it is **important** to understand what we are trying to achieve.

We are going to simulate a application that is available between 08:00 and 23:00 and we want to run our test across 2 overnight cycles as per the **recommendation** earlier in this post

To do this we will

* Start our Soak Test at some point on Day 1
* Require that it executes until 23:00
* Re-start at 08:00 the next day where it executes until 23:00
* Pause until 08:00 the following day
* Completing at 23:00 on the third day

## Let’s visualise the test
 
![Test Representation](https://blog.octoperf.com/img/blog/soak-test-practical-example/test-representation.jpg "Test Representation")

Whilst simple, this is a good representation of a [Soak Test](https://en.wikipedia.org/wiki/Soak_testing "Soak Test"), let’s look at the script that manages the execution.

We will look at it in an organised way, but don’t worry **we’ll publish the fully annotated code at the end**

Firstly two helper methods to help us in our calculations, one to calculate **seconds to a particular time on the current day**

```java
public long currentDayMilliCount(int dayOfMonth, int hourOfDay) { 
	Calendar c = Calendar.getInstance();
	c.add(Calendar.DAY_OF_MONTH, dayOfMonth);
	c.set(Calendar.HOUR_OF_DAY, hourOfDay);
	c.set(Calendar.MINUTE, 0);
	c.set(Calendar.SECOND, 0);
	c.set(Calendar.MILLISECOND, 0);
	long howMany = (c.getTimeInMillis()-System.currentTimeMillis());
	return howMany;
}
```

And the second to **return the number of milliseconds for any number of hours duration**

```java
public long calculateMilliCount(int noHours) {
	return noHours * 3600 * 1000; 	
}
```

OK that’s the helper methods build, now let’s build the **main method** that will manage the execution flow, this is broken into small chunks to allow us to discuss in detail.
 
```java
public void manageSoakTestExecution(int noDaysExecution, int startTime, int dailyDuration) {
```

We create the method with three parameters

- no of days we want to execute the test for
- time we want to start daily
- duration of a daily test.

To start with we return the number of milliseconds until 23:00 on the first day of the test using one of our helper methods, remember the test could start at any time so **day ones calculations** are slightly different from the remaining days

```java
long milliToTestStopDayOne = currentDayMilliCount(0, 23);
```

We will show all these calculation times in our visualised test
 
![Day 1](https://blog.octoperf.com/img/blog/soak-test-practical-example/day1.jpg "Day 1")

We also need to determine how long until midnight on day one, so we can calculate the **sleep time** for the test on the first day

```java
long milliToMidnightDayOne = currentDayMilliCount(1, 0);
```
 
![Sleep time](https://blog.octoperf.com/img/blog/soak-test-practical-example/sleep-time.jpg "Sleep time")

We calculate the sleep time for our test from **midnight each day** until the desired start time

```java
long milliStartOfDay = calculateMilliCount(startTime);
```
 
![Day 2](https://blog.octoperf.com/img/blog/soak-test-practical-example/day2.jpg "Day 2")

We calculate the execution time for each full day of the test

```java
long milliOnDayExecutionTime = calculateMilliCount(dailyDuration);
```
 
![Sleep time 2](https://blog.octoperf.com/img/blog/soak-test-practical-example/sleep-time2.jpg "Sleep time 2")

We calculate the remaining sleep time for each full day of the test

```java
long milliEndOfDay = calculateMilliCount(24 - (startTime + dailyDuration));
```
 
![Remaining sleep time](https://blog.octoperf.com/img/blog/soak-test-practical-example/remaining-sleep-time.jpg "Remaining sleep time")

We calculate the duration of the whole day element of the test

```java
long milliAllDaysDuration = calculateMilliCount(24 * noDaysExecution);
```
 
![Remaining Days](https://blog.octoperf.com/img/blog/soak-test-practical-example/remaining-days.jpg "Remaining Days")

Now we have our basic calculations we can manage the test, we firstly set the duration of the [Soak Test](https://en.wikipedia.org/wiki/Soak_testing "Soak Test") by changing the **dRation** parameter. This is effectively the time to midnight on day one of the test plus the number of days of solid execution we wish to run for.

We do this by overwriting the dRation [Simplified Property Function](https://jmeter.apache.org/usermanual/functions.html#__P "Simplified Property Function") that manages the execution time

```java
long soakTestDuration = milliToMidnightDayOne + milliAllDaysDuration;
props.put("dRation", soakTestDuration.toString());
```
 
![Test duration](https://blog.octoperf.com/img/blog/soak-test-practical-example/test-duration.jpg "Test duration")

We now start our test with two [Thread Groups](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group") running with the **manage-test-duration** [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group") controlling the **[octo-perf-soak](https://blog.octoperf.com/img/blog/soak-test-practical-example/octoperfsoak.jmx)** [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group")

We now pause our **manage-test-duration** [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group"), until it is required to make changes to [Simplified Property Function](https://jmeter.apache.org/usermanual/functions.html#__P "Simplified Property Function") in the **[octo-perf-soak](https://blog.octoperf.com/img/blog/soak-test-practical-example/octoperfsoak.jmx)** [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group"), the next action will be to pause the test when we have been running for the time set in `milliToTestStopDayOne`

We therefore tell the **manage-test-duration** sampler to sleep until needed

```java
Thread.sleep(milliToTestStopDayOne);
```
 
![Milliseconds day 1](https://blog.octoperf.com/img/blog/soak-test-practical-example/milliseconds-day1.jpg "Milliseconds day 1")

We iterate our **noDaysExeuction** variable here using a for loop and follow the same execution pattern for each subsequent day. 

```java
for(i=1; i<=noDaysExecution; i++) {
```

Once we have waited for the end of the test on **Day 1** we need to pause our **[octo-perf-soak](https://blog.octoperf.com/img/blog/soak-test-practical-example/octoperfsoak.jmx)** [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group") until it needs to starts again which is effectively the remaining sleep time for the test for a day plus the sleep time for the next day when the test starts again

We overwrite the **threadSleep** variable in the manage-sleep-time  [JSR223 Timer](https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Timer "JSR223 Timer")

```java
props.put("threadSleep", milliEndOfDay + milliStartOfDay);
```

![Thread sleep](https://blog.octoperf.com/img/blog/soak-test-practical-example/thread-sleep.jpg "Thread sleep")

And we pause our **manage-test-duration** [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group") until we need to perform another action, this is effectively the time when the next day’s **[octo-perf-soak](https://blog.octoperf.com/img/blog/soak-test-practical-example/octoperfsoak.jmx)** [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group") iteration needs to stop

```java
Thread.sleep(milliEndOfDay + milliStartOfDay + milliOnDayExecutionTime);
}
}
```
 
**And that is it!!!!**

## Summary

All we are doing is using one [Thread Group](https://jmeter.apache.org/usermanual/test_plan.html#thread_group "Thread Group") to control another using [Simplified Property Function](https://jmeter.apache.org/usermanual/functions.html#__P "Simplified Property Function") values and **thread sleep time**

All that remains to be done is to call the method, with appropriate parameters to manage the [Soak Test](https://en.wikipedia.org/wiki/Soak_testing "Soak Test")

```java
manageSoakTestExecution(2, 8, 15);
```

### Full JSR223 Sampler extract

Full [JSR223 Sampler](https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Sampler "JSR223 Sampler") extract:

```java
/* Function to calculate seconds to a particular time on the current day */
public long currentDayMilliCount(int dayOfMonth, int hourOfDay) {
	/* Return the number of milliseconds from the current time to a particular hour of the day */ 
	Calendar c = Calendar.getInstance();
	c.add(Calendar.DAY_OF_MONTH, dayOfMonth);
	c.set(Calendar.HOUR_OF_DAY, hourOfDay);
	c.set(Calendar.MINUTE, 0);
	c.set(Calendar.SECOND, 0);
	c.set(Calendar.MILLISECOND, 0);
	long howMany = (c.getTimeInMillis()-System.currentTimeMillis());
	return howMany;	
}

public long calculateMilliCount(int noHours) {
	/* Return the number of milliseconds for any number of hours duration */
	return noHours * 3600 * 1000; 	
}

public void manageSoakTestExecution(int noDaysExecution, int startTime, int dailyDuration) {

	/* We will build an execution schedule for the Soak Test which will manage the test */
	/* Firstly we need to determine how long the test needs to run for on the current day */
	/* The test could be started at a any time of the day and in our example we want it to */
	/* stop before midnight and not start again before a certain time on the following day */
	/* Using our functions we calculate how much time between now and our desired end time */
	long milliToTestStopDayOne = currentDayMilliCount(0, 23);

	/* We also need to determine how long until midnight so we can calculate the sleep time */
	/* for the test on the first day, we use the same function with alternative parameters */
	long milliToMidnightDayOne = currentDayMilliCount(1, 0);

	/* We calculate the sleep time for our test from midnight each day until the desired start time */
	long milliStartOfDay = calculateMilliCount(startTime);

	/* We calculate the execution time for the test */
	long milliOnDayExecutionTime = calculateMilliCount(dailyDuration);

	/* We calculate the remaining sleep time for the test */
	long milliEndOfDay = calculateMilliCount(24 - (startTime + dailyDuration));

	/* We calculate the duration of the whole day element of the test */
	long milliAllDaysDuration = calculateMilliCount(24 * noDaysExecution);

	/* Now we have our basic calculations we can manage the test */
	/* We firstly set the duration of the soak test by changing the dRation parameter */
	/* The first thing we do is set the test duration for the whole test day */
	/* This is effectively the time to midnight on the start day of the test plus the number of days */
	/* of solid execution we wish to run for */
	
	/* We do this by overwriting the dRation property that manages the execution time */
	long soakTestDuration = milliToMidnightDayOne + milliAllDaysDuration;

	props.put("dRation", soakTestDuration.toString());

	/* We now pause our manage-test-duration sampler, until it is required to change further parameters*/
	/* The next action will be to pause the test when we have executed for the time set in the milliToTestStopDayOne variable */
	Thread.sleep(milliToTestStopDayOne);

	/* We iterate our noDaysExeuction variable here and follow the same pattern for each day */
	for(i=1; i<=noDaysExecution; i++) {

		/* Once we have waited for the end of the test we need to pause our */
		/* octo-perf-soak sampler until it starts again which */
		/* is effectively the remaining sleep time for the test for a day plus the sleep time for the next day when */
		/* the test starts again*/
		/* We overwrite the threadSleep variable in the manage-sleep-time sampler */
		props.put("threadSleep", milliEndOfDay + milliStartOfDay);
		
		/* And we pause our manage-test-duration sampler until we need to perform another action */
		/* This is effectively the time when the next days tests needs to stop */
		Thread.sleep(milliEndOfDay + milliStartOfDay + milliOnDayExecutionTime);
	}
}

	/* Call the function to manage the soak test */
	manageSoakTestExecution(2, 8, 15);
```