--- title: 'Gatling: Post requests and modular scripts' url: https://blog.octoperf.com/gatling-post-requests-and-modular-scripts/ date: '2020-08-18' authors: - gpereira categories: - Load-Testing tags: - Gatling - Load Testing - HTTP - Domain Specific Language description: How to send an HTTP POST request using Gatling and how to create modular scripts to reuse chunks of load testing virtual users? --- # Gatling: Post requests and modular scripts This article is the fourth part of a series of tutorials dedicated to Gatling Load Testing. We will focus on [POST requests](#post-requests) and [script modularization](#modular-script): > In the previous blog post we created [a realistic Virtual User that browses the store](https://blog.octoperf.com/gatling-loops-conditions-and-pauses/) without buying anything. > On the contrary, here we are going to **simulate the behavior of a user that connects to the web store**, searches for items, adds some to his cart and proceeds to the checkout. > Then we will combine both Virtual Users to simulate a diverse load on the PetStore. ## POST Requests Most actions to simulate a user that connects to the store are done via POST HTTP requests. **But what exactly is an HTTP POST request?** ### HTTP POST Definition We saw [what is an HTTP Request](https://blog.octoperf.com/gatling-simulation-scripts-parameterization/#http-overview) in a previous tutorial. **The [HTTP POST method](https://en.wikipedia.org/wiki/POST_(HTTP)) has the particularity to send data to the server.** A header field (`Content-Type`) in the POST request usually indicates the message body's Media Type. Web applications such as our PetStore typically send POST requests via the submission of an HTML form. The content type of the request varies depending on what is sent or how it is sent: * `application/x-www-form-urlencoded`: keys and values are encoded in key-value pairs separated by '&', with a '=' between the key and the value. All non-alphanumeric characters are percent encoded. * `multipart/form-data`: each value is sent as a body part, boundaries separating each part. This format allows to send binary data (i.e. file upload), while the `x-www-form-urlencoded` format prevents this because of the encoding. Modern web application like [SPAs](https://en.wikipedia.org/wiki/Single-page_application) made with **Angular, React or Vue send POST requests with an [XMLHttpRequest](https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest) instead of a form**. In this case the content type can be anything. For instance, it's usually `application/json` when sending a JSON object to the server. > __POST vs PUT:__ > > The difference between PUT and POST is idempotence: > > * Calling a PUT request once or several times successively has the same effect, > * Successive identical POST request may have side effects, like adding an item to a cart several times. ### Gatling POST Syntax Gatling's DSL addresses all the cases previously mentioned. It **automatically sets the Content-Type header for you** if you do not specify one: It will use `application/x-www-form-urlencoded` except if there are body parts, in which case it will set it to `multipart/form-data`. 1. To send **a URL encoded form**, use the syntax: `http("URL Ecnoded POST").post("request-url").formParam("key", "value")`, 2. To post **a multipart request**, for instance by uploading a file: `http("Multipart POST").post("request-url").formParam("key", "value").formUpload("key2", "fileToUpload.txt")`, 3. To simulate sending **a JSON body to a REST API**: `http("JSON Body").post("request-url").header("Content-Type", "application/json").body(RawFileBody("body.json")).asJson`. The file _body.json_ must be present in the project resources. You can replace _RawFileBody_ by `ElFileBody` if you need to inject session variables into the body. ### Posting a Login Form Let's try to apply this to our e-commerce and create a Login request. Le login process is divided in several steps. A first GET request fetches the [Login HTML Form](https://petstore.octoperf.com/actions/Account.action?signonForm=): ![PetStore Login Form](https://blog.octoperf.com/img/blog/gatling-post-forms-modular-scripts/petstore-login-form.png "PetStore Login Form") The user types his credentials and clicks on the _Login_ button. **A POST request (`application/x-www-form-urlencoded`) is send to the server.** It answers with a [302 HTTP response code](https://en.wikipedia.org/wiki/HTTP_302) and the header `location: /actions/Catalog.action` to redirect the user to the homepage. Here is a Chrome console view of the Login request: ![PetStore Login Request](https://blog.octoperf.com/img/blog/gatling-post-forms-modular-scripts/petstore-login-request.png "PetStore Login Request") To simulate the connexion of different users, we use a [CSV Feeder](https://blog.octoperf.com/gatling-simulation-scripts-parameterization/#values-feeders) ``` val credentials = csv("four/credentials.csv").random ``` It uses the file `credentials.csv`: ``` login,password user1,pass user2,pass user3,pass user4,pass user5,pass j2ee,j2ee ``` Then we create the first GET request to load the HTML form: ``` val signonFormRequest = http("Signon Form") .get("/actions/Account.action") .queryParam("signonForm", "") ``` And the POST request for the actual login: ``` val loginRequest = http("Login ${login}") .post("/actions/Account.action") .formParam("username", "${login}") .formParam("password", "${password}") .formParam("signon", "Login") .check(substring("Welcome ABC!").exists) ``` **Here we use the keyword `.post` combined with `.formParam` to send a URL encoded form with the same parameters that we saw in the Chrome console screenshot.** We also check if the message "Welcome ABC!" is present in the response. The scenario definition uses the feeder to inject the user credentials and chains the two requests: ``` val scn = scenario("PetStoreSimulation") .exec(signonFormRequest) .feed(credentials) .exec(loginRequest) ``` > The **complete script** for this chapter is available: > > * For download, [the simulation script here](https://blog.octoperf.com/img/blog/gatling-post-forms-modular-scripts/PetStoreSimulation1.scala) and [the Credentials CSV here](https://blog.octoperf.com/img/blog/gatling-post-forms-modular-scripts/credentials.csv), ### Search And Checkout **Let's complete this Virtual User by making it search for an item, add it in his cart and order it.** It's only simple GET requests with variable extraction followed by form POSTs. Check out the [blog post about Gatling Simulation scripts parameterization](https://blog.octoperf.com/gatling-simulation-scripts-parameterization/#variable-extractors) if you need help with this. Here is an extract of the script: ``` val searchRequest = http("Search ${term}") .post("/actions/Catalog.action") .formParam("keyword", "${term}") .formParam("searchProducts", "Search") .check(regex("""productId=([^"]*)""").findRandom.saveAs("productId")) val productRequest = http("Product ${productId}") .get("/actions/Catalog.action") .queryParam("viewProduct", "") .queryParam("productId", "${productId}") .check(regex("""itemId=([^"]*)""").findRandom.saveAs("itemId")) val addItemToCartRequest = http("Add item ${itemId} to cart") .get("/actions/Cart.action") .queryParam("addItemToCart", "") .queryParam("workingItemId", "${itemId}") val newOrderFromRequest = http("New Order Form") .get("/actions/Order.action") .queryParam("newOrderForm", "") val orderRequest = http("Order") .post("/actions/Order.action") .formParam("order.cardType", "Visa") .formParam("order.creditCard", "999 9999 9999 9999") .formParam("order.expiryDate", "12/03") .formParam("order.billToFirstName", "ABC") .formParam("order.billToLastName", "XYX") .formParam("order.billAddress1", "901 San Antonio Road") .formParam("order.billAddress2", "MS UCUP02-206") .formParam("order.billCity", "Palo Alto") .formParam("order.billState", "CA") .formParam("order.billZip", "94303") .formParam("order.billCountry", "USA") .formParam("newOrder", "Continue") val confirmOrder = http("Confirm Order") .get("/actions/Order.action") .queryParam("newOrder", "") .queryParam("confirmed", "true") ``` > The **complete script** for this chapter is available: > > * For download, [the simulation script here](https://blog.octoperf.com/img/blog/gatling-post-forms-modular-scripts/PetStoreSimulation2.scala) and [the Search CSV here](https://blog.octoperf.com/img/blog/gatling-post-forms-modular-scripts/search.csv), ## Modular Script So we have two virtual users, one from the previous tutorial that browses the shop, and one we just created. **Our goal is to start both VUs at the same time, in the same scenario.** How to do this without duplicating the scripts? Whether it is in [JMeter](https://blog.octoperf.com/modularisation-in-jmeter/) or Galting, [modular scripts](https://en.wikipedia.org/wiki/Modularity-driven_testing) is the solution: > The idea is to split our Virtual Users in smaller chunks, for example the login operation or the search. > Then we use these test parts in a bigger simulation. > A benefit, in addition to producing more readable scripts, is that **if the tested application changes you only have to update one script chunk**. ### From Simulation Class to Simple Object Let's start with the _Visitor_ behavior. We get the Simulation from the previous blog post and transform it into a requests and scenario _"module"_. It's straightforward since we already declared a `var` for each request and scenario. We just need to drop the `extends Simulation` and change the `class` into an `objet`: ``` package com.octoperf.tutorials.four import scala.concurrent.duration._ import io.gatling.core.Predef._ import io.gatling.http.Predef._ import io.gatling.jdbc.Predef._ object PetStoreSimulationVisitor { val csvRecords = csv("four/categories.csv").readRecords val categoryRequest = http("Catalog ${categoryId}") .get("/actions/Catalog.action") .queryParam("viewCategory", "") .queryParam("categoryId", "${categoryId}") .check(regex("""productId=(.*)"""").findAll.saveAs("productIds")) val productRequest = http("Product ${productId}") .get("/actions/Catalog.action") .queryParam("viewProduct", "") .queryParam("productId", "${productId}") val scn = scenario("PetStoreSimulationVisitor") .exec(http("Homepage").get("/actions/Catalog.action")) .pause(500 milliseconds) .foreach(csvRecords, "category") { exec(flattenMapIntoAttributes("${category}")) .exec(categoryRequest) .pause(500 milliseconds) .during(10 seconds) { pace(1 seconds) .exec(session => { val productIds = session("productIds").as[List[Any]] val productIndex = util.Random.nextInt(productIds.size) session.set("productId", productIds(productIndex)) }) .exec(productRequest) } } } ``` Then it's the same for the Buyer virtual user: ``` package com.octoperf.tutorials.four import scala.concurrent.duration._ import io.gatling.core.Predef._ import io.gatling.http.Predef._ import io.gatling.jdbc.Predef._ object PetStoreSimulationBuyer { val credentials = csv("four/credentials.csv").random val signonFormRequest = http("Signon Form") .get("/actions/Account.action") .queryParam("signonForm", "") val loginRequest = http("Login ${login}") .post("/actions/Account.action") .formParam("username", "${login}") .formParam("password", "${password}") .formParam("signon", "Login") .check(substring("Welcome ABC!").exists) val searchTerms = csv("four/search.csv").random val searchRequest = http("Search ${term}") .post("/actions/Catalog.action") .formParam("keyword", "${term}") .formParam("searchProducts", "Search") .check(regex("""productId=([^"]*)""").findRandom.saveAs("productId")) [...] val scn = scenario("PetStoreSimulationBuyer") .exec(signonFormRequest) .pause(500 milliseconds) .feed(credentials) .exec(loginRequest) .pause(500 milliseconds) .feed(searchTerms) .exec(searchRequest) .pause(500 milliseconds) .exec(productRequest) .pause(500 milliseconds) .exec(addItemToCartRequest) .pause(500 milliseconds) .exec(newOrderFromRequest) .pause(500 milliseconds) .exec(orderRequest) .pause(500 milliseconds) .exec(confirmOrder) } ``` We also added pauses on the scenario. Check this chapter about [load testing pauses and pacing](https://blog.octoperf.com/gatling-loops-conditions-and-pauses/#pauses) to know why. ### Combining Script in a Simulation The final step is to import these two objects (request and scenario modules) into a unique Simulation class: ``` package com.octoperf.tutorials.four import scala.concurrent.duration._ import io.gatling.core.Predef._ import io.gatling.http.Predef._ import io.gatling.jdbc.Predef._ import com.octoperf.tutorials.four.PetStoreSimulationBuyer._ import com.octoperf.tutorials.four.PetStoreSimulationVisitor._ class PetStoreSimulationModular extends Simulation { val httpProtocol = http .baseUrl("https://petstore.octoperf.com") .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8") .doNotTrackHeader("1") .acceptLanguageHeader("en-US,en;q=0.5") .acceptEncodingHeader("gzip, deflate") .userAgentHeader("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0") .inferHtmlResources(BlackList(), WhiteList("https://petstore.octoperf.com/.*")) setUp(PetStoreSimulationBuyer.scn.inject(constantConcurrentUsers(2) during(2 minutes)),PetStoreSimulationVisitor.scn.inject(constantConcurrentUsers(10) during(2 minutes))).protocols(httpProtocol) } ``` The modules are imported with the syntax `import com.octoperf.tutorials.four.PetStoreSimulationBuyer._` and `import com.octoperf.tutorials.four.PetStoreSimulationVisitor._`. We will simulate 2 concurrent buyers and 10 concurrent visitors during 2 minutes. This is done by setting up the load test using their respective scenario `setUp(PetStoreSimulationBuyer.scn.inject(constantConcurrentUsers(2) during(2 minutes)),PetStoreSimulationVisitor.scn.inject(constantConcurrentUsers(10) during(2 minutes))).protocols(httpProtocol)`. > The **complete scripts** for this chapter are available: > > * For download: > * [the Buyer script here](https://blog.octoperf.com/img/blog/gatling-post-forms-modular-scripts/PetStoreSimulationBuyer.scala), > * [the Visitor script here](https://blog.octoperf.com/img/blog/gatling-post-forms-modular-scripts/PetStoreSimulationVisitor.scala), > * [the categories.csv file used in the Visitor VU here](https://blog.octoperf.com/img/blog/gatling-post-forms-modular-scripts/categories.csv), > * [the modular Simulation script here](https://blog.octoperf.com/img/blog/gatling-post-forms-modular-scripts/PetStoreSimulationModular.scala), Launching this test on two load injectors shows us that both virtual users are executed. ![Gatling Modular Script Userload](https://blog.octoperf.com/img/blog/gatling-post-forms-modular-scripts/gatling-modular-scipt-user-load.png "Gatling Modular Script Userload") **We only scratched the surface of what could be done in terms of modularization, for example:** * An object _PetStoreFeeders_ could be created with all feeders used to load test the JPetStore, * Another one could hold the set of requests used to login and logout, * A third one could contain the requests to search and browse random items, * etc. In the examples above we only declared HTTP request variables (`val searchRequest = http("Search ${term}") ...`). You can also declare variables that contain the successive execution of several requests: ``` val login = exec(signonFormRequest) .pause(500 milliseconds) .feed(credentials) .exec(loginRequest) .pause(500 milliseconds) ``` And use it in a scenario: `val scn = scenario("Buyer").exec(login, search, checkout, logouut)`.