Gatling: Post requests and modular scripts

This article is the fourth part of a series of tutorials dedicated to Gatling Load Testing.

We will focus on POST requests and script modularization:

In the previous blog post we created a realistic Virtual User that browses the store without buying anything. On the contrary, here we are going to simulate the behavior of a user that connects to the web store, searches for items, adds some to his cart and proceeds to the checkout. Then we will combine both Virtual Users to simulate a diverse load on the PetStore.

POST Requests

Most actions to simulate a user that connects to the store are done via POST HTTP requests. But what exactly is an HTTP POST request?

OctoPerf is JMeter on steroids!

Schedule a Demo

HTTP POST Definition

We saw what is an HTTP Request in a previous tutorial. The HTTP POST method has the particularity to send data to the server.

A header field (Content-Type) in the POST request usually indicates the message body's Media Type. Web applications such as our PetStore typically send POST requests via the submission of an HTML form. The content type of the request varies depending on what is sent or how it is sent:

• application/x-www-form-urlencoded: keys and values are encoded in key-value pairs separated by '&', with a '=' between the key and the value. All non-alphanumeric characters are percent encoded.
• multipart/form-data: each value is sent as a body part, boundaries separating each part. This format allows to send binary data (i.e. file upload), while the x-www-form-urlencoded format prevents this because of the encoding.

Modern web application like SPAs made with Angular, React or Vue send POST requests with an XMLHttpRequest instead of a form. In this case the content type can be anything. For instance, it's usually application/json when sending a JSON object to the server.

POST vs PUT:

The difference between PUT and POST is idempotence:

• Calling a PUT request once or several times successively has the same effect,
• Successive identical POST request may have side effects, like adding an item to a cart several times.

Gatling POST Syntax

Gatling's DSL addresses all the cases previously mentioned. It automatically sets the Content-Type header for you if you do not specify one: It will use application/x-www-form-urlencoded except if there are body parts, in which case it will set it to multipart/form-data.

1. To send a URL encoded form, use the syntax: http("URL Ecnoded POST").post("request-url").formParam("key", "value"),
2. To post a multipart request, for instance by uploading a file: http("Multipart POST").post("request-url").formParam("key", "value").formUpload("key2", "fileToUpload.txt"),
3. To simulate sending a JSON body to a REST API: http("JSON Body").post("request-url").header("Content-Type", "application/json").body(RawFileBody("body.json")).asJson.

The file body.json must be present in the project resources. You can replace RawFileBody by ElFileBody if you need to inject session variables into the body.

Posting a Login Form

Let's try to apply this to our e-commerce and create a Login request.

Le login process is divided in several steps. A first GET request fetches the Login HTML Form:

The user types his credentials and clicks on the Login button. A POST request (application/x-www-form-urlencoded) is send to the server. It answers with a 302 HTTP response code and the header location: /actions/Catalog.action to redirect the user to the homepage. Here is a Chrome console view of the Login request:

To simulate the connexion of different users, we use a CSV Feeder

val credentials = csv("four/credentials.csv").random

It uses the file credentials.csv:

login,password
user1,pass
user2,pass
user3,pass
user4,pass
user5,pass
j2ee,j2ee

Then we create the first GET request to load the HTML form:

val signonFormRequest = http("Signon Form")
  .get("/actions/Account.action")
  .queryParam("signonForm", "")

And the POST request for the actual login:

val loginRequest = http("Login ${login}")
  .post("/actions/Account.action")
  .formParam("username", "${login}")
  .formParam("password", "${password}")
  .formParam("signon", "Login")
  .check(substring("Welcome ABC!").exists)

Here we use the keyword .post combined with .formParam to send a URL encoded form with the same parameters that we saw in the Chrome console screenshot. We also check if the message "Welcome ABC!" is present in the response.

The scenario definition uses the feeder to inject the user credentials and chains the two requests:

val scn = scenario("PetStoreSimulation")
  .exec(signonFormRequest)
  .feed(credentials)
  .exec(loginRequest)

The complete script for this chapter is available:

• For download, the simulation script here and the Credentials CSV here,

Search And Checkout

Let's complete this Virtual User by making it search for an item, add it in his cart and order it.

It's only simple GET requests with variable extraction followed by form POSTs. Check out the blog post about Gatling Simulation scripts parameterization if you need help with this.

Here is an extract of the script:

  val searchRequest = http("Search ${term}")
      .post("/actions/Catalog.action")
      .formParam("keyword", "${term}")
      .formParam("searchProducts", "Search")
      .check(regex("""productId=([^"]*)""").findRandom.saveAs("productId"))

  val productRequest = http("Product ${productId}")
      .get("/actions/Catalog.action")
      .queryParam("viewProduct", "")
      .queryParam("productId", "${productId}")
      .check(regex("""itemId=([^"]*)""").findRandom.saveAs("itemId"))

  val addItemToCartRequest = http("Add item ${itemId} to cart")
      .get("/actions/Cart.action")
      .queryParam("addItemToCart", "")
      .queryParam("workingItemId", "${itemId}")

  val newOrderFromRequest = http("New Order Form")
      .get("/actions/Order.action")
      .queryParam("newOrderForm", "")

  val orderRequest = http("Order")
      .post("/actions/Order.action")
      .formParam("order.cardType", "Visa")
      .formParam("order.creditCard", "999 9999 9999 9999")
      .formParam("order.expiryDate", "12/03")
      .formParam("order.billToFirstName", "ABC")
      .formParam("order.billToLastName", "XYX")
      .formParam("order.billAddress1", "901 San Antonio Road")
      .formParam("order.billAddress2", "MS UCUP02-206")
      .formParam("order.billCity", "Palo Alto")
      .formParam("order.billState", "CA")
      .formParam("order.billZip", "94303")
      .formParam("order.billCountry", "USA")
      .formParam("newOrder", "Continue")

  val confirmOrder = http("Confirm Order")
      .get("/actions/Order.action")
      .queryParam("newOrder", "")
      .queryParam("confirmed", "true")

The complete script for this chapter is available:

• For download, the simulation script here and the Search CSV here,

Modular Script

So we have two virtual users, one from the previous tutorial that browses the shop, and one we just created. Our goal is to start both VUs at the same time, in the same scenario. How to do this without duplicating the scripts?

Whether it is in JMeter or Galting, modular scripts is the solution:

The idea is to split our Virtual Users in smaller chunks, for example the login operation or the search. Then we use these test parts in a bigger simulation. A benefit, in addition to producing more readable scripts, is that if the tested application changes you only have to update one script chunk.

From Simulation Class to Simple Object

Let's start with the Visitor behavior. We get the Simulation from the previous blog post and transform it into a requests and scenario "module". It's straightforward since we already declared a var for each request and scenario. We just need to drop the extends Simulation and change the class into an objet:

package com.octoperf.tutorials.four

import scala.concurrent.duration._

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._

object PetStoreSimulationVisitor {

    val csvRecords = csv("four/categories.csv").readRecords

    val categoryRequest = http("Catalog ${categoryId}")
            .get("/actions/Catalog.action")
            .queryParam("viewCategory", "")
            .queryParam("categoryId", "${categoryId}")
            .check(regex("""productId=(.*)"""").findAll.saveAs("productIds"))

    val productRequest = http("Product ${productId}")
          .get("/actions/Catalog.action")
          .queryParam("viewProduct", "")
          .queryParam("productId", "${productId}")

    val scn = scenario("PetStoreSimulationVisitor")
        .exec(http("Homepage").get("/actions/Catalog.action"))
        .pause(500 milliseconds)
        .foreach(csvRecords, "category") {
          exec(flattenMapIntoAttributes("${category}"))
          .exec(categoryRequest)
          .pause(500 milliseconds)
          .during(10 seconds) {
            pace(1 seconds)
            .exec(session => {
              val productIds = session("productIds").as[List[Any]]
              val productIndex = util.Random.nextInt(productIds.size)
              session.set("productId", productIds(productIndex))
            })
            .exec(productRequest)
          }
        }
}

Then it's the same for the Buyer virtual user:

package com.octoperf.tutorials.four

import scala.concurrent.duration._

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._

object PetStoreSimulationBuyer {

  val credentials = csv("four/credentials.csv").random

  val signonFormRequest = http("Signon Form")
      .get("/actions/Account.action")
      .queryParam("signonForm", "")

  val loginRequest = http("Login ${login}")
      .post("/actions/Account.action")
      .formParam("username", "${login}")
      .formParam("password", "${password}")
      .formParam("signon", "Login")
      .check(substring("Welcome ABC!").exists)


  val searchTerms = csv("four/search.csv").random

  val searchRequest = http("Search ${term}")
      .post("/actions/Catalog.action")
      .formParam("keyword", "${term}")
      .formParam("searchProducts", "Search")
      .check(regex("""productId=([^"]*)""").findRandom.saveAs("productId"))

  [...]

  val scn = scenario("PetStoreSimulationBuyer")
      .exec(signonFormRequest)
      .pause(500 milliseconds)
      .feed(credentials)
      .exec(loginRequest)
      .pause(500 milliseconds)
      .feed(searchTerms)
      .exec(searchRequest)
      .pause(500 milliseconds)
      .exec(productRequest)
      .pause(500 milliseconds)
      .exec(addItemToCartRequest)
      .pause(500 milliseconds)
      .exec(newOrderFromRequest)
      .pause(500 milliseconds)
      .exec(orderRequest)
      .pause(500 milliseconds)
      .exec(confirmOrder)
}

We also added pauses on the scenario. Check this chapter about load testing pauses and pacing to know why.

Combining Script in a Simulation

The final step is to import these two objects (request and scenario modules) into a unique Simulation class:

package com.octoperf.tutorials.four

import scala.concurrent.duration._

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._

import com.octoperf.tutorials.four.PetStoreSimulationBuyer._
import com.octoperf.tutorials.four.PetStoreSimulationVisitor._

class PetStoreSimulationModular extends Simulation {

    val httpProtocol = http
        .baseUrl("https://petstore.octoperf.com")
        .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
        .doNotTrackHeader("1")
        .acceptLanguageHeader("en-US,en;q=0.5")
        .acceptEncodingHeader("gzip, deflate")
        .userAgentHeader("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0")
        .inferHtmlResources(BlackList(), WhiteList("https://petstore.octoperf.com/.*"))


    setUp(PetStoreSimulationBuyer.scn.inject(constantConcurrentUsers(2) during(2 minutes)),PetStoreSimulationVisitor.scn.inject(constantConcurrentUsers(10) during(2 minutes))).protocols(httpProtocol)
}

The modules are imported with the syntax import com.octoperf.tutorials.four.PetStoreSimulationBuyer._ and import com.octoperf.tutorials.four.PetStoreSimulationVisitor._.

We will simulate 2 concurrent buyers and 10 concurrent visitors during 2 minutes. This is done by setting up the load test using their respective scenario setUp(PetStoreSimulationBuyer.scn.inject(constantConcurrentUsers(2) during(2 minutes)),PetStoreSimulationVisitor.scn.inject(constantConcurrentUsers(10) during(2 minutes))).protocols(httpProtocol).

The complete scripts for this chapter are available:

• For download:
• the Buyer script here,
• the Visitor script here,
• the categories.csv file used in the Visitor VU here,
• the modular Simulation script here,

Launching this test on two load injectors shows us that both virtual users are executed.

We only scratched the surface of what could be done in terms of modularization, for example:

• An object PetStoreFeeders could be created with all feeders used to load test the JPetStore,
• Another one could hold the set of requests used to login and logout,
• A third one could contain the requests to search and browse random items,
• etc.

In the examples above we only declared HTTP request variables (val searchRequest = http("Search ${term}") ...). You can also declare variables that contain the successive execution of several requests:

val login = exec(signonFormRequest)
  .pause(500 milliseconds)
  .feed(credentials)
  .exec(loginRequest)
  .pause(500 milliseconds)

And use it in a scenario: val scn = scenario("Buyer").exec(login, search, checkout, logouut).

Want to become a super load tester?

Request a Demo