---
title: 'Gatling: Simulation Scripts Parameterization'
url: https://blog.octoperf.com/gatling-simulation-scripts-parameterization/
date: '2020-03-14'
authors:
- gpereira
categories:
- Load-Testing
tags:
- Gatling
- Load Testing
- HTTP
- CSV
- Regexp
- Domain Specific Language
- JSONPath
description: How to create realistic (and meaningful) load tests by parameterizing Gatling scripts using feeders and regular expression extractors.
---
# Gatling: Simulation Scripts Parameterization
This blog post is a tutorial for writing Gatling scripts to load test web applications. It follows our
first [getting started with Gatling simulation scripts](https://blog.octoperf.com/gatling-getting-started-with-simulation-scripts/) article.
The application under test is a fake e-commerce. We are going to create
a [Virtual User](https://blog.octoperf.com/gatling-getting-started-with-simulation-scripts/#gatling-terminology) that browses articles in this
shop. To create a dynamic load test we will cover several topics:
* [Feeders](#values-feeders) to inject values taken from a static file,
* [Regular Expression extractors](#variable-extractors) to extract value (s) from a server response and inject it in
subsequent requests,
* [Cookies](#cookies) management.
## Prerequisites
### Gatling Basics
Since this blog post is the second of a series, **you are expected to know
the [basics of writing a Gatling simulation](https://blog.octoperf.com/gatling-getting-started-with-simulation-scripts/)**, with:
* The structure of the simulation Class,
* The HTTP protocol configuration,
* Writing a very basic scenario to make a single GET request,
* Load injection profiles,
* Running Gatling.
### HTTP Overview
You also **need to know the base principles of
the [Hypertext Transfer Protocol](https://developer.mozilla.org/en-US/docs/Web/HTTP/Overview)**
and [how is built an HTTP request](https://blog.octoperf.com/gatling-getting-started-with-simulation-scripts/#http-overview).
#### HTTP Query Parameters
Let's focus on the [Query Parameters](#get-request-and-query-parameters)
(or [Query String](https://en.wikipedia.org/wiki/Query_string)) as they are going to be set using Gatling's DSL in this
guide.
A typical URL containing a query string is as follows:
> [https://petstore.octoperf.com/actions/Catalog.action?viewCategory=&categoryId=DOGS](https://petstore.octoperf.com/actions/Catalog.action?viewCategory=&categoryId=DOGS)
When a server receives a request for such a page, it may run a program to generate the HTTP response. The query string
follows the question mark `?` character and is passed to the underlying program. Here, query parameters are separated by
the ampersand `&` character:
* `viewCategory=`: is a simple key _viewCategory_ without value,
* `categoryId=DOGS`: is a key/value pair that defines the ID of the category to view.
**The returned HTML changes with the given parameters even though the path of the resource `/actions/Catalog.action`
does not.**
For example, open the
URL [https://petstore.octoperf.com/actions/Catalog.action?viewProduct=&productId=K9-PO-02](https://petstore.octoperf.com/actions/Catalog.action?viewProduct=&productId=K9-PO-02)
and you will see a different page rendered.
#### Stateless Protocol
**HTTP is a stateless protocol.** HTTP servers do not need to store information or status about each user for the
duration of multiple requests. Put another way, there is no link between two requests being sent on the same connection.
Left like that it would be problematic for many web applications such as our sample e-commerce. There must to be a way
to store information about a visitor while he browses the shop and add articles to his cart!
In fact there are several:
* [HTTP Cookies](https://en.wikipedia.org/wiki/HTTP_cookie): it's a piece of data sent from the server, stored on the
user's web browser and sent back with every subsequent request while the user is browsing,
* [Local storage](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage): it's a JavaScript object that
stores data with no expiration date (unlike the _sessionStorage_ that is cleared when the browser tab is closed),
* [XMLHttpRequests](https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest) and [the javascript
`fetch()` API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API): these JavaScript tools allow modern Single
Page web application to send requests to the server without reloading the page (states can then be stored in JS
objects).
The JPetStore uses a [JSESSIONID Cookie](#cookies) to store the user session identifier.
## GET Request and Query Parameters
Enough theory!
**Let's continue what was initiated in the first blog post and improve the created script to make it more realistic**
(download it
here). Indeed, this first version is only load testing a single page of
the [sample PetStore](https://petstore.octoperf.com/actions/Catalog.action):
This web page contains links to the categories of the PetStore (the different kinds of pet). In the HTML code of the
page, you can see that links are represented by the `link name` tag. The URL of the page changes
when the visitor clicks on one of those links.
For instance, if the visitor clicks on the _Fish_ category, the page is reloaded with the new
URL [https://petstore.octoperf.com/actions/Catalog.action?viewCategory=&categoryId=FISH](https://petstore.octoperf.com/actions/Catalog.action?viewCategory=&categoryId=FISH).
The query parameters are:
* `viewCategory=`: instructs the server to return the HTML for viewing a category,
* `categoryId=FISH`: tells the server what category to view.
If you click on a product, the query string will change for `?viewProduct=&productId=FI-SW-01` and if you click on an
item it will change for `?viewItem=&itemId=EST-1`. **So updating query parameters in our Gatling simulation allows us to
simulate the behavior of a visitor that browses articles in the shop.**
In a Gatling `.scala` Simulation script, you set the query parameters either by appending then manually at the end of
the request path:
```scala
http("Homepage").get("/Catalog.action?viewCategory=&categoryId=FISH")
```
Or a better solution is to use the `queryParam` operator:
```scala
http("Homepage").get("/Catalog.action")
.queryParam("viewCategory", "")
.queryParam("categoryId", "FISH")
```
The _viewCategory_ parameter has no value, but you still need to pass an empty string `""`.
> __Note:__
>
> Both keys and values of query parameters have to be [URL encoded](https://en.wikipedia.org/wiki/Percent-encoding).
> You can use this [online URL encoder](https://www.w3schools.com/tags/ref_urlencode.asp) to do it manually or use the
> _java.net.URLEncoder_:
> `.queryParam("key", java.net.URLEncoder.encode("value to encode", "UTF-8"))`.
## Values Feeders
But we are not going to copy/paste GET requests for every Category/Product/Item of the PetStore. **It's easier to
maintain a dynamic load testing script when the tested application is updated (for example with new products in the case
on an e-commerce).**
For the sake of this tutorial we are going to use a Feeder. For a real test, it would be better to extract the value
from the homepage HTML like we will do in the [next chapter](#variable-extractors). Usually, Feeders are mostly used to
generate values that cannot be extracted from server responses, such as user credentials.
In Gatling, a Feeder is an object that iterates over a list of values and **feeds** it to a scenario execution.
### CSV Feeder Declaration
There are many kinds of Feeders in Gatling, from a simple array to
a [JDBC](https://en.wikipedia.org/wiki/Java_Database_Connectivity) reader.
**We will focus on the [CSV](https://en.wikipedia.org/wiki/Comma-separated_values ) Feeder**, but feel free to write a
comment if you would like explanations on another Feeder in particular.
```
key1,key2,key3
record1 value1,record1 value2,record1 value3
record2 value1,record2 value2,record2 value3
record3 value1,record3 value2,record3 value3
```
A Comma-Separated Values file is a delimited text file that uses a comma `,` to separate values. Each line of the file
is a data record.
**For Gatling, the first line defines the name of each column.** Only subsequent lines are fed to the scenario. Let's
create a simpler CSV file named `categories.csv` that contains only on pet shop categories:
```
categoryId
BIRDS
FISH
DOGS
REPTILES
CATS
```
In Gatling, copy this file in
the `/user-files/resources/two/` folder.
In the simulation script, declaring a CSV Feeder is done using the `csv` keyword:
```scala
val csvFeeder = csv("two/categories.csv").random
```
Here the csv feeder is assigned to the csvFeeder variable `val csvFeeder = ` and will be read randomly `.random`. The
_random_ suffix defines the reading strategy. There are several strategies available:
* `.queue`: the default value if nothing is written after `.csv()`, it reads each line one after the other,
* `.random`: reads a random line when a value is generated,
* `.shuffle`: shuffles all lines then reads them one by one,
* `.circular`: reads lines one by one and restarts at the top of the file when the end is reached.
> __Warning:__
>
> If you use the `.queue` or `.shuffle` strategies and your CSV file has not enough values to feed every iteration of
> your scenario, Gatling will stop the simulation execution!
> __Note:__
>
> A Comma-Separated Values file uses a comma `,` to separate values.
> The same principle can be used with different separators, for example, a semi-colon `;` or a tab character `\t`.
>
> In Gatling scripts, specific Feeders are dedicated to each use case:
>
> * `val tsvFeeder = csv("categories.tsv")` for tabulations,
> * `val ssvFeeder = csv("categories.ssv")` for semicolons.
>
> You can even use a custom separator with the syntax `val feeder = separatedValues("categories.txt", '/')`.
### Feeder Usage
Time to use our feeder. **The feeder is added to the execution chain of the scenario with the `.feed` keyword.**
You can then use Gatling's Expression Language to inject values anywhere you want.
For instance, our CSV file contains the _categoryId_ column and is configured with the _random_ strategy. So you can
inject a random category with the `${categoryId}` string:
```scala
val scn = scenario("PetStoreSimulation")
.exec(http("Homepage").get("/actions/Catalog.action"))
.feed(csvFeeder)
.exec(http("Catalog ${categoryId}")
.get("/actions/Catalog.action")
.queryParam("viewCategory", "")
.queryParam("categoryId", "${categoryId}"))
```
The complete simulation script
is downloadable
here:
### Distributed Load and File Splitting
All resources and simulations are copied on each host before the execution. So every injector will use a duplicated
_categories.csv_ file to generate categories. You may want to **use a different set of values for each load injector**.
For instance with two injectors and our PetStore, we will try to have a different set of categories visited depending on
the injector:
* _BIRDS_ and _FISH_ for one injector,
* _DOGS_, _REPTILES_ and _CATS_ for the other.
#### Setup Environment Variables
Using environment variables allows Gatling to know what values should be used. The idea here is to inject an identifier
of the categories when running the load test, let's say *CATEGORIES_SET_ID*.
* **Key**: Defines the environment variable name,
* **Value**: Defines the environment variable value,
* **Scope**: The ID of a specific host or `Global` if the environment variable is used on every injector.
Here, the env variable *CATEGORIES_SET_ID* will have the value *categories1* on the *host-1* host and *categories2* on
the *host-2*. If you have more that two hosts you could easily add a row to define the Categories Set ID for it. For
instance, we could run the test on three hosts and add a line *CATEGORIES_SET_ID* | *categories2* | *host-3*.
Then, in the Gatling script you inject the environment variable with the following syntax:
```scala
val categoriesSetId = System.getProperty("CATEGORIES_SET_ID")
val csvFeeder = csv("two/" + categoriesSetId + ".csv").random
```
#### Create One File Per Injector
You need to manually split the _categories.csv_ file in two, with the following content.
categories1.csv:
```
categoryId
BIRDS
FISH
```
categories2.csv:
```
categoryId
DOGS
REPTILES
CATS
```
Then you can easily **load the appropriate file depending on the environment variable**:
```scala
val categoriesSetId = System.getProperty("CATEGORIES_SET_ID")
val csvFeeder = csv(categoriesSetId + ".csv").random
```
Let's run this simulation with 10 concurrent users on each injector and see how it goes (download the
script here).
> __Note:__
>
> **The same method can be used to have a different load injection strategy per host.**
>
> For example, setup the environment variable *CONCURRENT_USERS* and inject in with
> `val load = Integer.getInteger("CONCURRENT_USERS", 42)` (42 being the default value if the property is not set).
> Then use it in the injection policy `setUp(scn.inject(atOnceUsers(load))).protocols(httpProtocol)`.
## Variable Extractors
Let's go one level deeper into the PetShop e-commerce and have our virtual user open a random Product page. This time we
will **extract available products from the server response** instead of using a CSV Feeder.
There are [many variables extractors in Gatling](#other-extractors). In fact they are called _Checks_ using the Gatling
terminology. IMO, the most expandable is the Regular Expression Check. There are easier variable extractors for specific
needs, but you can use the _regexp_ no matter the type of content returned by the server: HTML, XML, JSON, etc.
### Regular Expressions
Before looking at the syntax of Gatling scripts, we must learn a bit
about [Regular Expressions](https://techterms.com/definition/regular_expression).
> A regular expression (or "regex") is a search pattern used for matching one or more characters within a string. It can
> match specific characters, wildcards, and ranges of characters.
I think that every load tester (and more generally every developer) should have at least a basic knowledge of Regexps
and be able to write even simple patterns. Gatling being written in Scala, it uses
the [Java patterns](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/regex/Pattern.html) format.
**You can find more information
about [Regular Expression on Oracle's tutorial](https://docs.oracle.com/javase/tutorial/essential/regex/).**
Even though the syntax is a bit different
in [JMeter, here are some example of Regex used in load testing scripts](https://blog.octoperf.com/jmeter-regex-extractor-how-to-extract-variables/#example-regular-expressions).
A few thing to remember for this tutorial:
* The string `.*` matches any characters,
* Parenthesis `()` are capturing groups,
* Capturing groups allows you to extract a part of the matched character string.
Finally, it's often quicker to copy/paste the server response in an online Regex tester to check that it works fine
instead of running a load tests. There are several tools available:
* [Regex 101](https://regex101.com/),
* [Regex Planet](https://www.regexplanet.com/advanced/java/index.html).
### Gatling's Check and Regexp
Gatling's `.check` keyword is used for two things:
* Checking that the server response matches expectations (.ie returns a 2XX HTTP status),
* Capturing some elements of the server response.
We will focus on the second usage here (A dedicated blog post will cover the first use case and assertions in general).
You have to extract the product identifier from the HTML.
The product ID is present in the `` tag, between the string `productId=` and `"`. The regular expression in
this case is **`productId=(.*)"`**.
**Let's copy the HTML body in an [online regexp testing tool](https://regex101.com/) to
try it out:**
As you can see all the product IDs are found.
**In Gatling's simulation script, use this regex as follows:**
```scala
.exec(http("Catalog ${categoryId}")
.get("/actions/Catalog.action")
.queryParam("viewCategory", "")
.queryParam("categoryId", "${categoryId}")
.check(regex("""productId=(.*)"""").findRandom.saveAs("productId")))
.exec(http("Product ${productId}")
.get("/actions/Catalog.action")
.queryParam("viewProduct", "")
.queryParam("productId", "${productId}"))
```
* `.check` is appended at the end of the GET request chain,
* `regex("""productId=(.*)"""")` escapes the regular expression String by placing it between the `"""` characters,
* `.findRandom` extract a random product ID (amongst other options you can use `.findAll` to extract a list of values or
`.count` to count the matching tokens),
* `.saveAs("productId")` stores the extracted value into Gatling Session,
* `.queryParam("productId", "${productId}"))` gets the _productId_ from the session using Expression Language and
injects it in a query parameter.
> __Gatling Sessions:__
>
> A Gatling Session is a memory space dedicated to a Virtual User instance/iteration.
> You can store values on the fly in this Map in order to create a dynamic load test.
You can download the complete
script [here](https://blog.octoperf.com/img/blog/gatling-simulation-script-parameterization/PetStoreSimulation3.scala).
That looks good! Debugging this script in shows us that three requests are executed (as long as the resources inferring
is commented out of course):
* The static Homepage,
* A random Category page using the [CSV Feeder](#values-feeders),
* A random Product page using our newly created Regexp Check.
**Want to go further? You can update the script to extract item IDs (from the HTML of the product pages) and make our
virtual user go to a random one.**
### Other Extractors
Many values extractors are available in Gatling, they take the place of the `regex` keyword in the simulation script.
Each one have a specific use case:
* `jsonPath()`: To [extract value (s) from a JSON](https://goessner.net/articles/JsonPath/) response,
* `css()`: To extract value (s) from an HTML body using [CSS selectors](https://lagarto.jodd.org/csselly/csselly),
* `xpath()`: To extract value (s) from an XML document
using [XPath 1.0 expressions](https://www.w3schools.com/xml/xpath_intro.asp).
## Cookies
We saw earlier in this blog post that cookies are used to store the user session
as [HTTP is a stateless protocol](#stateless-protocol).
Here is a simple test to view how Cookies are managed on the PetStore.
1. Open **a new Incognito Window** on your web browser,
2. Go to the PetStore: [https://petstore.octoperf.com/](https://petstore.octoperf.com/),
3. Open the developers Console (F12 on Chrome and FireFox) and head to the _Network_ tab,
4. Click on the [Enter the Store](https://petstore.octoperf.com/actions/Catalog.action) link in the web page,
5. In the Network tab of the console, open the _Catalog.action_ request,
6. You will see the HTTP **response** header
`set-cookie: JSESSIONID=BDF1B88FEEBD0AC4460CC1B6E0C83CAD; Path=/; HttpOnly`, it sets the JSESSIONID Cookie value,
7. Click on the [Fish Category](https://petstore.octoperf.com/actions/Catalog.action?viewCategory=&categoryId=FISH)
link,
8. In the Network tab of the console, open the **last** _Catalog.action_ request,
9. Its request URL should be `?viewCategory=&categoryId=FISH`,
10. In the HTTP **request** headers you will see `cookie: JSESSIONID=27AA1FFC6EB67A44DA79197426D2B141`,
That is how the _JSESSIONID_ Cookie set in the browser (with the `set-cookie` response header) and sent back to the
server (using the `cookie` request header). This allows the PetStore server to track what page is visited by whom (and
later on what item is added to whose cart).
**What about Gatling? Once again it behaves like a web browser and handles Cookies transparently**
(like [JMeter does](https://api.octoperf.com/doc/design/edit-virtual-user/action-types/jsr223-actions/jsr223-samples/#cookies)).
If you need some specific behavior it offers the following methods to manage the current Virtual User cookies:
* `exec(addCookie(Cookie("cookieName", "cookieValue")))`: Adds a Cookie,
* `exec(getCookieValue(CookieKey("cookieName"))).saveAs("myCookie")`: Saves the Cookie named _cookieName_ in Gatling's
session at _myCookie_,
* `exec(flushSessionCookies)`: Flushes Session cookies (like when a user closes its web browser),
* `exec(flushCookieJar)`: Flushes all cookies.
## Running the Load Test
Let's roll! It's time to run our simulation script with 100 concurrent users and see how it goes (The complete script
is [downloadable here](https://blog.octoperf.com/img/blog/gatling-simulation-script-parameterization/PetStoreSimulation4.scala):
```scala
setUp(scn.inject(constantConcurrentUsers(100) during(3 minutes))).protocols(httpProtocol)
```
Opening a Grafana report shows us how many request are sent for each page:
First of all, that's a lot of requests (more than 150K all pages combined) for only a few concurrent users. The issue
here is that we did not add any form of think-time when writing our script. We simulated users that would click on links
without even taking the time to read the page (not mentioning the omitted browser rendering time). **That is not
realistic at all!**
Also, the HomePage has 25K requests, 5 times more than the Category pages. This is also not realistic for an e-commerce:
visitors would probably more often browse several categories and products without having to go back to the home page.
**We must use loops to simulate this
path.** [Check out this blog post to learn more about loops and pauses.](https://blog.octoperf.com/gatling-loops-conditions-and-pauses/)