--- title: How to extract data from Json response using JMeter url: https://blog.octoperf.com/how-to-extract-data-from-json-response-using-jmeter/ date: '2017-03-09' authors: - jloisel categories: - JMeter tags: - Design - JSON - JSR223 - Regexp - Dummy - JSONPath - Plugins description: Learn by examples how to use JMeter Json Path Extractor, JSR223 and Beanshell scripts to work with Json server responses. a lot of real-world examples with detailed JMeter screenshots and explanations. --- # How to extract data from Json response using JMeter If you're here, it's probably because you need to **extract variables** from a [Json](https://en.wikipedia.org/wiki/JSON) response using [JMeter](https://jmeter.apache.org/). Good news! You're on the **definitive guide** to master **JMeter Json Extractor**. Complementary to our [Rest API Testing Guide](https://blog.octoperf.com/rest-api-testing-with-jmeter-step-by-step-guide/), you'll learn **everything you need** to master [Json Path Expressions](https://goessner.net/articles/JsonPath/). Let's go! And don't panic, there is **nothing difficult** down there. ## Json Format To get a better understanding of **what Json is**, here is an example Json document: ``` json { "store": { "book": [ { "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95 }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99 } ], "bicycle": { "color": "red", "price": 19.95 } }, "expensive": 10 } ``` [Json](https://en.wikipedia.org/wiki/JSON) is an extremely simple data format which has taken over [XML](https://en.wikipedia.org/wiki/XML) a few years ago. You probably ask yourself: **why do I need to learn Json?** An increasing number of [REST APIs](https://en.wikipedia.org/wiki/Representational_state_transfer) and servers are using Json as their primary data exchange format. At OctoPerf, we are heavily using Json to exchange data between our [AngularJS](https://angularjs.org/) frontend client and our [Spring Boot](https://spring.io) backend. Want to know the best part? Since, JMeter 3.0, it's far easier to extract data from Json responses using the Json variable extractor. In other words, **Json extractors are natively available**. ## JMeter JsonPath Plugin ![JMeter Json Plugin](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/json-plugin.png) JMeter JsonPath Extractor Plugin can be [downloaded and installed](https://jmeter-plugins.org/wiki/JSONPathExtractor/) from jmeter-plugins website. As of JMeter 3.0 and above, **Json plugin is optional**. **Installing JMeter JsonPath Plugin** - Download [plugins-manager.jar](https://jmeter-plugins.org/install/) and put it into `JMETER_HOME/lib/ext` directory, - Restart JMeter, - Click on `Options > Plugins Manager` in the top menu, - Select `Available Plugins` tab, - Select `Json Plugins` and click on *Apply Changes and Restart JMeter*. The **JMeter Json Plugin** should be available in right click menu `Add > Post Processors > Json Path Extractor`. By the way, we encourage you to read our [JMeter Plugins Installation Guide](https://blog.octoperf.com/jmeter-plugins-installation-guide/) for more details about JMeter Plugins. Are you lazy? Because I am. Let's **use the native JsonPath Extractor** instead! ## JMeter Json Path Extractor JMeter's [Json Post Processor](https://jmeter.apache.org/usermanual/component_reference.html#JSON_Extractor) uses [Json Way](https://github.com/jayway/JsonPath), a Java Json Path API, to perform JSon path extraction on server responses. ![JMeter Json Post-Processor](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/json-post-processor.png) **The Json Path extractor should be placed under an HTTP Sampler.** It has several possible settings, hence the most relevant are: - **Variables Names**: semi-colon separate variable names, - **JSON Path Expressions**: expressions to extract content from the json response, - **Match Numbers**: `-1` for all, `0` for a random one, `n` for the nth one, - **Compute concatenation var**: create a variable like `${foo_ALL}` containing the concatenation of all extracted values, - And **Default values**: in the case the expression doesn't apply to the json document being processed. Awesome! But **how do I get started?** ## Example Json Paths Here are some example Json Path expressions that can be used to extract data from the Json document exposed above: | JsonPath (click link to try)| Result | | :------- | :----- | | $.store.book[*].author | The authors of all books | | $..author | All authors | | $.store.* | All things, both books and bicycles | | $.store..price | The price of everything | | $..book[0,1] | The first two books | | $..book[:2] | All books from index 0 (inclusive) until index 2 (exclusive) | | $..book[2:] | Book number two from tail | | $..book[?(@.isbn)] | All books with an ISBN number | | $.store.book[?(@.price < 10)] | All books in store cheaper than 10 | | $..book[?(@.price <= $['expensive'])] | All books in store that are not "expensive" | | $..book[?(@.author =~ /.*REES/i)] | All books matching regex (ignore case) | | $..* | Give me every thing | $..book.length() | The number of books | As you can see, it's easy and flexible to query specific information from a Json document and put them into variables. Let's explore some of the examples above with JMeter. Guess what? We're going to try them out. ## Real-life JMeter Examples ![JMeter Json Extractor Sample JMX](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/jmeter-json-extractor-sample.png) Our [sample JMX](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/json-extractor.jmx) shows how both the JMeter Json Extractor and [Plugin JsonPath Extractor](https://jmeter-plugins.org/wiki/JSONPathExtractor/) work. Before JMeter 3.0, the plugin was required to perform JsonPath extractions. As of JMeter 3.0, there is an integrated support for Json Extractions. Ready for some action? Let's go! ### Arrays Extraction ![JMeter Json Extractor Arrays](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/jmeter-json-extractor-arrays.png "Extracting all authors from the store") Extracting Arrays makes possible to extract multiple values from a single Json document at once. For example, we could extract all the authors from the book store: - **Variable Name**: `authors` resulting in variable `${authors}`, - **JSONPath Expression**: `$..author`, selects all authors from any depth. You will get the following variables: ``` authors_1=Nigel Rees authors_2=Evelyn Waugh authors_3=Herman Melville authors_4=J. R. R. Tolkien authors_ALL=Nigel Rees,Evelyn Waugh,Herman Melville,J. R. R. Tolkien (if Compute concatenation checked) authors_matchNr=4 ``` ![JMeter Json Extractor Arrays](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/jmeter-json-extractor-arrays-result.png "We got all the authors of all the books!") ### Conditional Extraction ![JMeter Json Selective Extraction](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/extract-book-titles.png "Extracting Book Titles selectively") Suppose now that we want to extract the title of the books whose price is less than or equal to 10: - **Variable Name**: `titles` resulting in `${titles}` variable, - **Match Number**: `-1`, - **JSONPath Expression**: `$.store.book[?(@.price<= 10)].title` (book titles with price <= 10). You will get the following variables: ``` titles_1=Sayings of the Century titles_2=Moby Dick titles_matchNr=2 ``` ![JMeter Selective Json Extraction](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/selective-extract-titles.png "Title of books priced below 10.") ### Multiple Extraction ![JMeter Json Multiple Extraction](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/extract-title-and-author.png "Extracting Both Book Author and Title") Suppose now that we want to extract multiple Json fields at the same time. For example, we would like to query all author and titles: - **Variable Name**: `multiple`, - **Match Number**: `-1`, - **JSONPath Expression**: `$..['author','title']`. You will get the following variables: ``` multiple_1={"title":"Sayings of the Century","author":"Nigel Rees"} multiple_2={"title":"Sword of Honour","author":"Evelyn Waugh"} multiple_3={"title":"Moby Dick","author":"Herman Melville"} multiple_4={"title":"The Lord of the Rings","author":"J. R. R. Tolkien"} multiple_matchNr**=4 ``` ![JMeter Json Multiple Extraction Result](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/multiple-json-extraction-result.png "Extracting Both Book Author and Title") And that's the results being displayed within JMeter UI. ### Concatenation Extraction Sometimes, you want to extract and concatenate all results into a single string. In this Example, I took [HTTPBin headers](https://httpbin.io/headers) json endpoint. This can be achieved using the `Compute Concatenation var (suffi _ALL)` option. ![JMeter Json Concatenation Extraction Result](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/jmeter-json-extractor-compute-all-response.png "Json response received when calling HTTPBin from JMeter") The endpoint returns a json containing the headers sent by the client. You should see something like: ```json { "headers": { "Connection": "close", "Host": "httpbin.org", "User-Agent": "Apache-HttpClient/4.5.5 (Java/1.8.0_161)" } } ``` Now let's modify the Json extractor and enable the option to concatenate results: - **Name of created variables**: `foo` (resulting in `${foo}`), - **Json Path Expression**: `$.headers.*`, - **Match Nr**: `-1` which means extracting all occurences, - **Compute concatenation var**: checked. Now let's see the result. ![JMeter Json Concatenation Extraction Result](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/jmeter-json-extractor-compute-all.png "Extracting A concatenation of all results") Finally, let's use a [Debug Sampler](https://jmeter.apache.org/usermanual/component_reference.html#Debug_Sampler) to view the variable being extracted (named `foo` here). ![JMeter Json Concatenation Extraction Result](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/jmeter-json-extractor-compute-all-result.png "Debugging the result") The result should be something like: ``` foo_1=close foo_2=httpbin.org foo_3=Apache-HttpClient/4.5.5 (Java/1.8.0_161) foo_ALL=close,httpbin.org,Apache-HttpClient/4.5.5 (Java/1.8.0_161) foo_matchNr=3 ``` Is that useful? I'm not really sure. But, whenever you have this specific need, JMeter does it for you easily. ## Using Response Assertion That's a pretty common question: **How to validate a variable extracted from a json using an assertion?** Said otherwise, you want to make sure the extracted variable is correct. Let's see how to do this. ![JMeter Json Dummy Sampler](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/jmeter-plugin-dummy-sampler.png "JMeter Dummy Sampler Plugin") For this example, we're going to use the [Dummy Sampler plugin](https://jmeter-plugins.org/wiki/DummySampler/). Why? Because it allows to generate a sample result with any json we want. Let's use the following Json: ```json { "firstname": "John", "lastname": "Smith" } ``` We're simply going to configure the dummy sampler to send this Json as response. ![JMeter Json Dummy Sampler](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/jmeter-plugin-dummy-sampler-config.png "JMeter Dummy Sampler Plugin Config") Then, we create a Json extractor to extract the `firstname` json field. ![JMeter Json Dummy Sampler](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/dummy-sampler-json-extractor.png "JMeter Json Extractor") Now let's configure a [Response Assertion](https://jmeter.apache.org/usermanual/component_reference.html#Response_Assertion). > The response assertion must be located **after** the json extractor in order to work. ![JMeter Response Assertion Variable](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/response-assertion-variable.png "Validating variable value using Response Assertion") The **Response Assertions** is configured as following: - **Apply To**: JMeter variable name to use, - **Field to Test**: Text response, - **Pattern Matching Rules**: Equals To, - **Patterns to Test**: `John` in our example. It's time to execute the thread group and see the results. ![JMeter Response Assertion Success](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/response-assertion-success.png "The assertion is passed successfully") If we replace the assertion patterns to test by `titi`. ![JMeter Response Assertion Success](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/response-assertion-failure.png "The assertion is failing") Great! Now you know how to extract a variable from a json response and validate the variable value using a **Response Assertion**. ## 3 Common Mistakes Now, you're probably wondering: **what can possibly go wrong?** The **3 common mistakes** that should be avoided are: - Don't define multiple variables within a single Json Path extractor: the script may become hard to understand / maintain, - Don't write expressions susceptible to work only on specific json responses, try to stick to the general case, - The simpler the solution, the better will be the script maintainability. ## Good to know Work-Arounds Depending on the case, you may use alternate techniques to extract content from a server response. ### Regular Expression Extractor Suppose you have a very simple Json document with the following content and you want all first names: ``` json { "name":"Simpsons family", "members":[ {"firstName":"Homer", "lastName":"Simpson"}, {"firstName":"Marge", "lastName":"Simpson"}, {"firstName":"Bart", "lastName":"Simpson"} ] } ``` In this case, **the regular expressions extractor may fit well** because it's very simple to write a regular expression. ![JMeter Regexp Post-Processor](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/regexp-extractor.png) We have defined the following settings: - **Reference Name**: `firstname_RegEx`, - **Regular expression**: `"firstName":"(.+?)"`, - **Template:** `$1$`, - **Match Nr**: `3`, (we want *Bart*) - **Default value**: `D'oh!`. ### JSR223 with External Library By using the Minimal Json Library, and adding it to JMeter you can do the job of extracting json data from a server response too. **Configuring JMeter With an external Lib** - Download the [Minimal Json Library latest Release](https://github.com/ralfstx/minimal-json/releases), - Put it in `/lib/ext`, - Restart JMeter. Now create a [JSR223 Post processor](https://jmeter.apache.org/usermanual/component_reference.html#JSR223_PostProcessor) under the Http Sampler whose server response is a Json document. Select **Java** language and inspire from the following script: ``` java import com.eclipsesource.json.JsonObject; String jsonString = prev.getResponseDataAsString(); JsonArray members = Json.parse(jsonString).asObject().get("members").asArray(); vars.put("firstName",String.valueOf(members.get(2).getString("firstName",""))); ``` The code above extract the *firstName* of the third family member and puts it in a variable. ### JSR223 with Groovy [JSR223 PostProcessor](https://jmeter.apache.org/usermanual/component_reference.html#JSR223_PostProcessor) has Groovy language support which has built-in JSON support so you won't have to add any .jars. Example code: ``` java import groovy.json.JsonSlurper def jsonSlurper = new JsonSlurper(); def response = jsonSlurper.parseText(prev.getResponseDataAsString()); vars.put("firstName", response.members[2].firstName.toString()); ``` The `${firstname}` can then later be reused as needed. ### BeanShell Json Extractor Although the same result can be achieved using the [BeanShell post processor](https://jmeter.apache.org/usermanual/component_reference.html#BeanShell_PostProcessor), we don't recommend to do so for performance reason. JSR223 post processors should be used in favor of BeanShell post processors. **JSR223 with Groovy is several magnitudes faster than BeanShell**. ![BeanShell Regexp Post-Processor](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/beanshell-json-extractor.png) Configuration is **very similar to JSR223**. Here we have the final variable `${firstname_BSH}`. ### JMeter Plugins (Json Path Extractor) **Since JMeter 3.0**, [JMeter Json Extractor Plugin](https://jmeter-plugins.org/wiki/JSONPathExtractor/) should be abandoned in favor of the built in Json Path extractor. This plugin is still **useful if you are using JMeter** `<= 2.13`. ![Json Path JMeter Plugin](https://blog.octoperf.com/img/blog/how-to-json-path-extractor-jmeter/json-path-jmeter-plugin.png) ### Use-cases Json extractors are particularly useful in the following cases: - [Json REST Apis](https://blog.octoperf.com/rest-api-testing-with-jmeter-step-by-step-guide/): more and more rest apis are based on Json, - [OAuth authentication](https://oauth.net/articles/authentication/) mechanisms, which uses Json to send and receive access and refresh tokens, - Single Page Web Apps ([React](https://reactjs.org/) or [AngularJS](https://angularjs.org) are mostly seen) which communicate with JSon REST backends. ## Final Words This huge Json tutorial comes to an end! But, wait? It's not finished yet! You may be interested in **checking our other guides** on: - [XPath Extractor](https://blog.octoperf.com/jmeter-xpath-extractor-guide/): extract content from XML responses (like [SOAP](https://en.wikipedia.org/wiki/SOAP), - [CSS Jquery Extractor](https://blog.octoperf.com/jmeter-css-jquery-extractor/): use *css selectors* to extract content from [HTML](https://www.w3schools.com/html/) responses, - And the well known [Regex Extractor](https://blog.octoperf.com/jmeter-regex-extractor-how-to-extract-variables/): use [Regular expressions](https://www.regular-expressions.info/) to extract part of responses. There are several ways to extract data from Json document using JMeter. Our favorite one is the built-in Json Extractor. And, best of all, [Json extractors](https://api.octoperf.com/doc/design/edit-virtual-user/action-types/post-processors/jsonpath-variable-extractor/) are **fully supported by OctoPerf**! By the way, If you want to dig further, here are some interesting **JMeter books**.