JMeter 3.1 Key Improvements
JMeter 3.1 is now available.
JMeter 3.1 is a fix release with some moderate improvements. Let's dive into the JMeter 3.1 version changes and decrypt them for you. Good news we already integrated JMeter 3.1 to OctoPerf!
New Features¶
In this section, we are going to review the new features introduced in JMeter 3.1.
Improved HTML Reporting¶
Basically, some new graphs have been added to the HTML report. The HTML report drastically improved the easiness to analyze test results when using JMeter, but it still lacks real-time test results reporting. For people who like advanced statistics, this is a must have.
Sent Bytes¶
It's a new metric available in test reports. It reflects the number of bytes sent to the remote server when executing a request. This is pretty interesting since JMeter was only reporting received bytes until now.
We already expose bytes sent per second at load generator level by gathering network interface metrics. It will be much easier to pinpoint how much data is sent by each request by using the sent bytes metric.
Improvements¶
timer.factor property¶
timer.factor property allows to define a float factor which multiplies the pause times computed by advanced Timers like Gaussian or Poisson. While this is handy to quickly hack how timers affect the execution flow, you can quickly forget you have put this property in your JMeter configuration. Your colleague working on the same script may not experience the same response time because of different timer settings.
In OctoPerf, we introduced the ability to override all think times when editing a scenario profile.
CSS Parser Caching¶
The CSS Parser runs when enabling Retrieve All Embedded Resources option on an HTTP sampler. Parsing CSS stylesheets can be CPU heavy, this is why caching has been introduced to compute CSS resources (like fonts or images) only once per stylesheet. Caching is basically a trade-off between CPU and memory usage. By decreasing CPU usage, memory usage is increased.
Caching is one of the most difficult concept in programming. It involves complex concepts like:
- Cache eviction policy: when to evict entries from the cache?
- and concurrency: how to properly handle concurrent cache computations?
Unfortunately, the CSS Parser cache introduced in JMeter 3.1 suffers from a race condition which may cause the same CSS stylesheet to be computed multiple times. We opened a the bug 59934 and made a pull request a few months ago to fix this.
We suggest to disable this cache by default as it can easily lead to Java Out Of Memory errors. Also, increased memory pressure leads to increased Garbage Collection which itself leads to higher CPU usage. Increasing JMeter's heap usage may help but avoid using more than 50% of total RAM.
JSR223 Groovy as Default¶
Groovy is a programming language which runs on the JVM. It's best suited for quickly scratching small scripts without all the burden which comes with Java. JSR223 sampler is a scripting sampler which named the JSR223 action in OctoPerf.
Groovy scripting using JSR223 is great because it's way faster than BeanShell samplers. We strongly recommend using JSR223 samplers when you need custom scripts in your JMX. Also, BeanShell is apparently deprecated in JMeter 3.1.
Responses larger than 2GB¶
You are affected by bug 63039 if you simulated downloads of files larger than 2 GigaBytes. Two properties have been introduced to customize the behavior:
- httpsampler.max_bytes_to_store_per_request: Maximum number of bytes to keep in memory per request,
- httpsampler.max_buffer_size: Buffer size when reading response sent by the server.
Basically, it just means that simulating large downloads now works fine.
HTTPS with CPS fixed¶
The biggest regression introduced in JMeter 3.0 has been fixed in 3.1: HTTPS with SNI (Server Name Indication) requests to a server are failing when bandwidth limitation is enabled. For example, you can simulate ADSL bandwidth by editing jmeter.properties and putting:
httpclient.socket.http.cps=1024000
httpclient.socket.https.cps=1024000
GET requests with Body¶
It's not possible to send GET requests having a body. This is especially useful when testing ElasticSearch, because search queries are GET requests with a body.
Bug Fixes¶
Many fixes have been done in this new JMeter version. The complete list is available here.