--- title: 'Gatling: Loops, Conditions and Pauses' url: https://blog.octoperf.com/gatling-loops-conditions-and-pauses/ date: '2020-05-07' authors: - gpereira categories: - Load-Testing tags: - Gatling - Load Testing - HTTP - Domain Specific Language description: How to improve a dynamic Virtual User by using loops, conditional statements and pauses? This articles give you tool to make your load tests even more realistic and meaningful. --- # Gatling: Loops, Conditions and Pauses This blog post is a guide to help you write Gatling scripts in order to load test web applications efficiently. It follows our second [Gatling Simulation scripts parameterization](https://blog.octoperf.com/gatling-simulation-scripts-parameterization/) article. We will continue to load test a fake e-commerce, and so we are going to improve our [Virtual User](https://blog.octoperf.com/gatling-getting-started-with-simulation-scripts/#gatling-terminology) to make it browse the store in a more humanly way. To do it we will cover several topics: * [Loops](#loops) to make it browse several articles of each category, * [Conditions](#conditions) to change its behavior depending on dynamic parameters, * [Pauses](#pauses) to simulate a real user think-time. **We start where the previous blog post ended, with a simulation script that uses a CSV feeder and a Regular Expression extractor to visit dynamic pages of the pet store: [Download Sample Script](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation0.scala).** ## Loops In computer science, [a loop is a control flow statement for specifying iteration](https://en.wikipedia.org/wiki/For_loop), which allows code to be executed repeatedly. Various keywords are used to specify this statement in Gatling Simulations: * [For Each](#for-each-loop) to iterate over the values of a Set, * [Repeat](#repeat-loop) to execute HTTP requests a given number of times, * [During](#during-loop) to repeatedly execute code for a certain duration, * [As well as several other loops](#other-loops). > __Warning:__ > > Gatling Simulations are written using the Scala programming language but use a dedicated DSL. > You must use specific DSL components like the `.forEach()` or `.doIfOrElse()` for loops and conditions instead of native `if` or`foreach` expressions. We saw in the previous blog post [how to extract values from a CSV File using a Gatling Feeder](https://blog.octoperf.com/gatling-simulation-scripts-parameterization/#csv-feeder-declaration). Now let's start with the **For Each loop to iterate over the values of this CSV Feeder**. ### For Each Loop Previously we loaded the CSV File as a Feeder: `val csvFeeder = csv("two/categories.csv").random`. It's some kind of iterator that puts the values one by one in the session when we use the `.feed` keyword. Now want to loop over the complete [categories.csv](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/categories.csv) file values. And the `.foreach` DSL component takes a Sequence in parameter, not a Feeder. [![Gatling CSV For Each Record](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/gatling-csv-foreach-record.png "Gatling CSV For Each Record")](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation1.scala) So we need to load the complete file records with the `readRecords` statement: ```scala val csvRecords = csv("three/categories.csv").readRecords ``` Then, for more clarity, we declare separate variables to store the requests to the _Category_ page and to the _Product_ page: ```scala val categoryRequest = http("Catalog ${categoryId}") .get("/actions/Catalog.action") .queryParam("viewCategory", "") .queryParam("categoryId", "${categoryId}") .check(regex("""productId=(.*)"""").findRandom.saveAs("productId")) val productRequest = http("Product ${productId}") .get("/actions/Catalog.action") .queryParam("viewProduct", "") .queryParam("productId", "${productId}") ``` We declared the _categoryRequest_ such that it needs a _categoryId_ value in the session: The `"${categoryId}"` syntax uses Expression Language to directly fetch the value from the session. **Let's see how we inject the category IDs:** ```scala val scn = scenario("PetStoreSimulation") .exec(http("Homepage").get("/actions/Catalog.action")) .foreach(csvRecords, "category") { exec(flattenMapIntoAttributes("${category}")) .exec(categoryRequest) .exec(productRequest) } ``` Here the `.foreach` statement takes the _csvRecords_ variable in parameter. This sequence is loaded only once when the test starts and stored in a variable. The second parameter is the name of the current value. As the value is stored in the Gatling session, you can load it with the syntax `session("category")` or more easily with Expression Language `"${category}"`. > __Note:__ > > You may also want to loop over a dynamic value. **Remember that everything that is dynamic in Gatling is stored in the Session.** > > In such case you would pass the key of the values Set where it is stored in the Session. > For instance, if a previous request execution has saved a Set of values with `.check(regex("""categoryId=(.*)"""").findAll.saveAs("categoryIds")`, you can iterate over it with the `foreach("categoryIds", "categoryId") {}` statement. The `readRecords` operator load the entire CSV file as a Sequence of [Maps](https://en.wikipedia.org/wiki/Map_(higher-order_function)) (one map per line). Our _categories.csv_ file only contains one column _categoryId_. So the generated maps only contain one entry with the key _categoryId_ and value changing from _DOGS_, _CATS_, etc. **Another dedicated Gatling keyword - `flattenMapIntoAttributes` - extracts this `categoryId` entry in the Gatling Session**, allowing us to use it directly within Expression Language thereafter: `"${categoryId}"`. The complete script [for this For Each DSL component is downloadable here](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation1.scala). ### Repeat Loop Now that we loop over the categories, it would be nice to iterate over the products. For the sake of this tutorial we will do it using the `.repeat` loop. Currently, only one product ID is extracted from the server response of the Category page. **To extract all the product IDs, we must configure the Regular Expression extractor with the `.findAll` option:** ```scala val categoryRequest = http("Catalog ${categoryId}") .get("/actions/Catalog.action") .queryParam("viewCategory", "") .queryParam("categoryId", "${categoryId}") .check(regex("""productId=(.*)"""").findAll.saveAs("productIds")) ``` We also changed the `.saveAs` statement to store the extracted value in the session "productId**s**" entry instead of "productId" since it is now a list of IDs. Let's have a look at the `.repeat` syntax: ```scala val scn = scenario("PetStoreSimulation") .exec(http("Homepage").get("/actions/Catalog.action")) .foreach(csvRecords, "category") { exec(flattenMapIntoAttributes("${category}")) .exec(categoryRequest) .repeat(session => session("productIds").as[List[Any]].size, "productIndex"){ exec(session => { val productIds = session("productIds").as[List[Any]] val productIndex = session("productIndex").as[Int] session.set("productId", productIds(productIndex)) }) .exec(productRequest) } } ``` The _repeat_ loop is the most simple one. It's like a `for` in Java: the first parameter is the number of iterations and the second one is the counter name (the value is automatically injected in the Session). **We can see in the sample code above that an `exec(session => {})` statement is used to:** * Get the product Ids list from the session `val productIds = session("productIds").as[List[Any]]`, * Get the _repeat_ loop _productIndex_ counter current value `val productIndex = session("productIndex").as[Int]`, * Put the current product ID in the session: `session.set("productId", productIds(productIndex))`. This statement only modifies the session. No HTTP request is sent here. But it is followed by the execution of the _productRequest_ that get the proper Product page using the productId. The complete script is available [here](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation2.scala). #### Using Expression Language Having to manipulate the Session is a bit cumbersome here. **There is a simpler way to do it using advanced Expression Language!** Simply remove the `exec(session => {})` statement from the repeat loop and update the productRequest to directly use the _productIndex_: ```scala val productRequest = http("Product ${productIds(productIndex)}") .get("/actions/Catalog.action") .queryParam("viewProduct", "") .queryParam("productId", "${productIds(productIndex)}") ``` The syntax `"${productIds(productIndex)}"` returns the element of the _productIds_ at the position _productIndex_ (starting from 0 like in any other programming language). The updated script is available [here](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation3.scala). There are many possibilities with EL: | Sample | Description | | --- | --- | | `"${productIds.size()}"` | Returns the number of elements in the productIds list. | | `"${productIds.random()}"` | Returns a random element of the productIds list. | | `"${productId.exists()}"` | Returns true if the _productId_ key is present in the session (Use `"${productId.isUndefined()}"` to check that it is NOT present). | | `"${productIds(2)}"` | Just like the example above but we can also use a static index. | | `"${category.categoryId}"` | If [Category is a map](#for-each-loop), return the value for the given _categoryId_ key | ### During Loop Until now we iterated over a sequence of values (ids or indexes alike). **The _During_ loop allows you to iterate for a specified amount of time.** The syntax is `during(duration, counterName, exitASAP) { exec(...) }`: * The first parameter _duration_ is expressed like `1500 milliseconds` or `5` for 5 seconds, * The second parameter _counterName_ is optional and is the name of the counter (Integer incremented by one for each iteration) that is injected in the Session, * The last parameter _exitASAP_ defaults to true meaning that the loop will exit as soon as the duration is reached, even if several actions are to be executed to complete the current iteration (that can be an issue if the execution of all requests in the loop is required by actions coming after the _during_ loop). Let's update our script to use such loop instead of the _repeat_ ([Complete Script](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation4.scala): ```scala val productRequest = http("Product ${productId}") .get("/actions/Catalog.action") .queryParam("viewProduct", "") .queryParam("productId", "${productId}") val scn = scenario("PetStoreSimulation") .exec(http("Homepage").get("/actions/Catalog.action")) .foreach(csvRecords, "category") { exec(flattenMapIntoAttributes("${category}")) .exec(categoryRequest) .during(100 milliseconds, "productCounter") { exec(session => { val productIds = session("productIds").as[List[Any]] val productCounter = session("productCounter").as[Integer] val productIndex = productCounter % productIds.size session.set("productId", productIds(productIndex)) }) .exec(productRequest) } } ``` Here we loop during 100 milliseconds and set the counter name to _productCounter_. The first _exec_ updates the Session by computing the current product ID using the counter and the modulo of the product IDs list size: **we will iterate over each product sequentially during 100 milliseconds**. > __Note:__ > > Our _during_ loop only executes for 100ms because no [pauses](#pauses) are configured on the scenario. > Using a longer loop duration would generate too many requests for an easy debugging of the script. > Once pauses or [pacing](#pacing) are added, the duration of the loop should be increased accordingly. **What if we want to go to a random product page on each iteration?** There are two solutions here. **The first solution** is to shuffle the _productIds_ list beforehand using a `transform`: ```scala val categoryRequest = http("Catalog ${categoryId}") .get("/actions/Catalog.action") .queryParam("viewCategory", "") .queryParam("categoryId", "${categoryId}") .check(regex("""productId=(.*)"""").findAll.transform(productIds => util.Random.shuffle(productIds)).saveAs("productIds")) ``` Here we update the _categoryRequest_ to apply `transform(productIds => util.Random.shuffle(productIds))` on the extracted list. The drawback of this solution is that the shuffling is only done once. So the Virtual User will loop over the same sequence. **The second best solution is to update the _productId_ computing part to use a random number generator** ([Download script here](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation5.scala): ```scala exec(session => { val productIds = session("productIds").as[List[Any]] val productIndex = util.Random.nextInt(productIds.size) session.set("productId", productIds(productIndex)) }) ``` `util.Random.nextInt(max)` returns a random Integer that is equal or superior to 0 and strictly inferior to _max_. ### Other Loops Several other loops are available in Gatling: * `asLongAs(condition, counterName, exitASAP) { exec(...) }` iterates over the loop **as long as** the condition is true (sort of equivalent of the _while_ loop but the condition can be evaluated for each contained request with the _exitASAP_ parameter), * `doWhile(condition, counterName) { exec(...) }` same as the _asLongAs_ but the condition is evaluated after the execution of all the contained requests, * `asLongAsDuring(condition, duration, counterName) { exec(...) }` mix of the _asLongAs_ and _during_ loops, * `doWhileDuring(condition, duration, counterName) { exec(...) }` mix of the _doWhile_ and _during_ loops, * `forever(counterName) { exec(...) }` iterates forever. ## Conditions In computer science, [a conditional statement is](https://en.wikipedia.org/wiki/Conditional_(computer_programming)) performs different actions depending on whether a specified boolean condition evaluates to true or false. Various keywords are used to specify this statement in Gatling Simulations: * [doIfOrElse](#if-or-else) to execute some actions when the condition is true and some other actions when it is false, * [doSwitchOrElse](#switch-statement) to switch the sub-chain execution based on a key equivalence evaluation, * [As well as several other conditions](#other-conditions). **For the purpose of this tutorial, we are going to simulate a different user behavior based on the category visited.** To do so we are going to create two execution chains. The first one simulates a really interested visitor that will look at each product of the current category. So we create a _foreach_ loop that sequentially make a request to each product: ```scala val sequentialProducts = foreach(session => session("productIds").as[List[Any]], "productId") { exec(productRequest) } ``` The second one simulates a less assiduous visitor that only checks a random product and leaves. It is done by extracting one random product Id from the list of _productIds_ present in the session before executing one single _productRequest_. ```scala val randomProduct = exec(session => { val productIds = session("productIds").as[List[Any]] val productIndex = util.Random.nextInt(productIds.size) session.set("productId", productIds(productIndex)) }) .exec(productRequest) ``` ### If Or Else **Let's start by simulating the behavior of someone looking for a dog.** He opens the Dogs category page and look at every pet available. Out of curiosity he will check one random pet from every other category. The corresponding script ([download here](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation6.scala)) is as follows: ```scala val scn = scenario("PetStoreSimulation") .exec(http("Homepage").get("/actions/Catalog.action")) .foreach(csvRecords, "category") { exec(flattenMapIntoAttributes("${category}")) .exec(categoryRequest) .doIfOrElse(session => session("categoryId").as[String].equals("DOGS")) { sequentialProducts } { randomProduct } } ``` The _doIfOrElse_ statement takes a function in parameter that must return a boolean: `(session: Session) => boolean`. **This function evaluates a condition using dynamic information from the session and returns true or false.** Here it checks if the _categoryId_ is equal to _DOGS_. If the value is true then the _sequentialProducts_ execution chain declared previously is executed. The _randomProduct_ one is executed otherwise. > __Note:__ > > The execution chain between the first pair of curly braces `{}` is executed when the condition is true. The one between the second pair is executed when its false. > It is not mandatory to declare the execution chains in dedicated variables. It is just cleaner IMHO. > For instance, you could have written `doIfOrElse(session => session("categoryId").as[String].equals("DOGS")) { exec(http("True request")...) } { exec(http("False request")...) }`. ### Switch Statement Now we simulate the behavior of a visitor that is looking for a pet for his children. **He is not decided between a dog or a cat.** So he browses all dogs and then all cats from the store. Out of curiosity he will check one random pet from every remaining category. Here is the corresponding script ([download here](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation7.scala)): ```scala val scn = scenario("PetStoreSimulation") .exec(http("Homepage").get("/actions/Catalog.action")) .foreach(csvRecords, "category") { exec(flattenMapIntoAttributes("${category}")) .exec(categoryRequest) .doSwitchOrElse("${categoryId}")( "DOGS" -> sequentialProducts, "CATS" -> sequentialProducts ) ( randomProduct ) } ``` **This time we use a _doSwitchOrElse_ statement.** It takes a string in parameter that is evaluated as the current Category ID thanks to Expression Language: `"${categoryId}"`. The first pair of parenthesis (**not curly braces here!**) contains the pairs of matching values/execution chains. The second pair of parenthesis contains the execution chain of actions that will be executed if none of the keys matched the current value. ### Other Conditions Several other conditional statements are available in Gatling DSL: * `doIf("${myCondition}") { exec(...) }` executes the chain of actions between curly braces only if the session's _myCondition_ entry is true (the condition can also be a function that takes the Session as a parameter and returns a boolean like we saw in the _doIfOrElse_ condition), * `doIfEquals("${actualValue}", "expectedValue") { exec(...) }` same as above but the session's _actualValue_ must be equal to the _expectedValue_ for the action chain to be executed (`doIfEqualsOrElse` is also available), * `doSwitch` just like the _doSwitchOrElse_ but nothing is executed if no key matching is found, * `randomSwitch` uses percentages and a randomly generated number to select the executed action chain (`randomSwitchOrElse`, `uniformRandomSwitch` and `roundRobinSwitch` also exist). ## Pauses Both conditions and loops helped us create a realistic load testing scenario. **But real users think before they click!** As we did not add any form of think-time when writing our scripts, executing them will simulate far too many request for a realistic load (given a fixed number of concurrent users). This chapter explains the various possibilities offered by Gatling to simulate pauses: * [How to configure static pauses on the scenario](#scenario-pauses-configuration), * [How to parameterize pauses on the scenario or globally](#pauses-parameterization), * [How to use pacing](#pacing). ### Scenario Pauses Configuration Let's start by updating our script to add a fixed _pause_ statement after each request ([Download Script](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation8.scala)): ```scala val scn = scenario("PetStoreSimulation") .exec(http("Homepage").get("/actions/Catalog.action")) .pause(5) .foreach(csvRecords, "category") { exec(flattenMapIntoAttributes("${category}")) .exec(categoryRequest) .pause(1500 milliseconds) .foreach(session => session("productIds").as[List[Any]], "productId") { exec(productRequest) .pause(1 seconds, 2 seconds) } } ``` The `.pause()` Gatling DSL component takes a duration in parameter: * `.pause(5)` waits for 5 seconds (default duration unit), * `.pause(1500 milliseconds)` waits for 1500ms, * `.pause(1 seconds, 2 seconds)` waits between one and two seconds (random ranged duration). That's perfect for simulating realistic users at runtime, but it's annoying to have to wait for the longer script executing when debugging ... ### Pauses Parameterization The idea is to give different parameters to our script (environment variables) when running/debugging it in order to configure the delays. If you run Gatling directly, this can by done by updating the *JAVA_OPTS* environment variable: `JAVA_OPTS="-DDELAY=500"`. #### Directly on the Scenario **The first option to parameterize think-times is to uses variables in directly in the `.pause` statements of the scenario.** But first we need to inject the environment variable into our script. Integer env variables can be retrieved with the following syntax: ```scala val delay = Integer.getInteger("DELAY", 500) val doubleDelay = 2*delay ``` `val delay = Integer.getInteger("DELAY", 500)` fetches the _DELAY_ environment variable and places its value in the _delay_ val. If the env variable is not defined, the 500 default value is used. `val doubleDelay = 2*delay` simply computes the double of this delay. Using these injected parameters in the script is pretty simple. **We just need to replace the static values by our created values _delay_ and _doubleDelay_:** ```scala val scn = scenario("PetStoreSimulation") .exec(http("Homepage").get("/actions/Catalog.action")) .pause(delay milliseconds) .foreach(csvRecords, "category") { exec(flattenMapIntoAttributes("${category}")) .exec(categoryRequest) .pause(delay milliseconds) .foreach(session => session("productIds").as[List[Any]], "productId") { exec(productRequest) .pause(delay milliseconds, doubleDelay milliseconds) } } ``` All pauses are now using durations in milliseconds. Passing 0 as the _DELAY_ environment variable will completely deactivate think times. Updating it allows us to configure the delays on demand. The complete script is [available here](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation9.scala). > __Note:__ > > If you need dynamic pauses, you can use a session function in parameter: `.pause(session => session("dynamicPause").as[Duration])`. #### Globally on the SetUp **The second option to parameterize think-times is to do it on the simulation `setUp`.** This time the _DELAY_ env variable is injected as a Long value, with the following syntax (don't forget the `.toLong` statement at the end, used to convert a Java Long into a Scala Long value): ```scala val delay = java.lang.Long.getLong("DELAY", 500L).toLong ``` **Using a Long is mandatory because the `.customPauses(session => delay)` DSL component requires a Long value:** ```scala setUp(scn.inject(atOnceUsers(1))) // .disablePauses .customPauses(session => delay) .protocols(httpProtocol) ``` It takes a function in parameter, with the following signature: `(session: Long) => Long`. This lets you return dynamic pauses depending on the Gatling session state. Passing 0 as the _DELAY_ environment variable will also completely deactivate think times here. A quicker option is to use the `.disablePauses` setting on the _setUp_. The complete script is [available here](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation10.scala). > __Note:__ > > To avoid [synchronicity issues during your load tests](https://api.octoperf.com/doc/analysis/tips/#synchronicity) it's a good idea to introduce randomness in your script pauses. > Gatling has dedicated configurations that can be put on the `setUp`: _exponentialPauses_, _normalPausesWithStdDevDuration_, _normalPausesWithPercentageDuration_, and _uniformPausesPlusOrMinusPercentage_. ### Pacing To complete this chapter about think-times we are about to talk about Pacing. **Pacing usually refers to the time between the iterations of your virtual users.** This is unlike the think-time (`.pause()`) which refers to the delay between individual actions. Pacing allows the load test to be even more realistic and simulate the time gap between two user sessions. In Gatling, the pacing is not configured at the iteration level but inside a loop, any loop. Let's update our script to use the dedicated keyword `pace(duration)` inside a 10 seconds _duration_ loop ([Download script](https://blog.octoperf.com/img/blog/gatling-loops-conditions-pauses-think-times/PetStoreSimulation11.scala)): ```scala val scn = scenario("PetStoreSimulation") .exec(http("Homepage").get("/actions/Catalog.action")) .pause(500 milliseconds) .foreach(csvRecords, "category") { exec(flattenMapIntoAttributes("${category}")) .exec(categoryRequest) .pause(500 milliseconds) .during(10 seconds) { pace(1 seconds) .exec(session => { val productIds = session("productIds").as[List[Any]] val productIndex = util.Random.nextInt(productIds.size) session.set("productId", productIds(productIndex)) }) .exec(productRequest) } } ``` You can see that there is no pause after the `.exec(productRequest)` statement in the script above. > _Notes:_ > > Use a _forever_ loop that encapsulate your scenario if you want to apply iteration pacing. > > The `pace()` DSL component can be configured with durations like the `.pause()` one: You can pass it a fixed number (default time unit is _seconds_), a duration like _100 milliseconds_, a range of durations, etc. ## Conclusion We have only scratched the surface of what Gatling is capable of. So keep posted as other blog posts are coming to help you master Gatling scripting language. **Feel free to share this guide if you found it useful!**