New features and improvements
This blog post aims to resume all the cool new features and improvements we've made to OctoPerf since the beginning of the year.
Design¶
Improvements and features in this section are related to the online virtual user design exclusive to OctoPerf.
Enable / Disable actions¶
This is a must-have to quickly rework any existing virtual user. Any action can now be disabled to prevent it from being executed when running the virtual user. Actions no longer need to be removed from the virtual user to be taken out of the execution. This feature is especially useful to debug virtual users by running repeated validations.
Advanced Search¶
We know that handling dynamic parameters can sometimes be especially challenging. The best performance testers deserve the best tools to work with. We have vastly improved the ways you can search for values inside various aspects of the virtual user.
It's now possible to search for a token in very specific locations like query parameter names, post parameter values or even recorded responses. Be sure to find where this nasty dynamic parameter comes from!
Virtual User Validation Zone¶
Previously, all the virtual user validations were executed in our main datacenter in Ireland by default. Now, it's possible to select the provider and region where to run the virtual user validation. This is especially useful when your virtual user only runs on an on-premise location.
Please note that launching a virtual user validation in some regions make take some time as we spin up the required machine.
Validation JMeter Logs¶
Now, when running a virtual user validation to check if your virtual user is running properly, the JMeter logs can be viewed and downloaded at the end of the execution. This is useful to debug JSR223 scripts which may log information. It's also convenient to see if JMeter ran without any issue during the validation before scaling massively on our cloud platform.
This feature is also available on all our test results. All the JMeter logs are stored on our servers at the end of a test for each JMeter engine being run during a load test.
Advanced scripts¶
We're proud to introduce the JSR223 scripting actions. This actions makes Java and Javascript scripting a first class citizen in our load tests. Everything is possible with custom scripts! From logging messages to generating random data, nothing is impossible with a script. It uses the JSR223 JMeter sampler behind the scene.
With advanced scripting, you can easily tackle even the most complex load testing issues at ease.
JMeter JMX Import¶
We've vastly improved our JMeter importing capabilities. Now, user defined variables and test wide constants are also imported. Various fixes have been made to support the most exotic JMeter JMX configurations. Beanshell scripts are automatically converted to JSR223 scripts for enhanced performances.
Charles Proxy Support¶
Some of you are using the amazing Charles Proxy tool to record your scenarios. We've extended our HAR import feature to fully support importing HAR files generates by Charles Proxy.
JMX Export¶
At OctoPerf, we believe that you should never be locked with any paid software. We'd rather have you stay by providing an amazingly useful software rather than locking you in with proprietary technologies. We've added the possibility to export your virtual user in JMX format, anytime, for free. This way, you can keep a JMeter compliant copy of your virtual user design. Simply click on the Download JMX button when editing your virtual user.
Project File Explorer¶
With an increasing number of csv files and post file parameters to manage, it became difficult to see which files where actually stored on our servers. The Project file explorer makes it easy to list, add or delete existing files to your test.
Those files are automatically sent to all JMeter load generators when launching a test, making it insanely easy to scale test with file resources.
VU Duplication and copy¶
Sometimes, you need virtual users with just slightly different behavior and don't want to record it again. Copy / paste feature is for you! You can now copy and paste virtual user actions.
You can also copy-paste entire virtual users (for example to backup an original record, or to test out scenario variations).
As a bonus, when copying and pasting actions, recorded requests and responses are copied too.
Sanity check¶
To prevent issues before running a load test, we've incorporated a feature which checks the sanity of your project. It performs various checks like verifying if a CSV variable has effectively an existing file linked to it.
Runtime¶
We have added a few advanced settings to our scenarios to satisfy our advanced users.
Load-generator startup¶
We have drastically improved our load generators spin-up time in the last few months. Tests will be launched way faster.
UserLoad: Apply all¶
The apply all button located in the top right corner on the userload allows to apply the same userload settings to all other userloads within the scenario. This avoids to have to redo the same settings several times in the case you have very similar userloads.
UserLoad: Global Thinktime¶
Setting each thinktime inside the virtual user manually can be a time consuming task. Sometimes, you just want to override all thinktimes with a given value and be set. This feature has been exactly designed for this purpose.
By clicking on Duration tab inside the virtual user profile, there is a new section called Think time. By enabling the override, all thinktimes within the associated virtual user will be replaced by the thinktime defined here:
-
Constant Thinktime: replaces all thinktimes with a constant one,
-
Random thinktime: replaces all thinktimes with a variable thinktime whose value is within a given range,
-
Throughput thinktime: replaces all thinktimes by trying to stick to a given throughput in hits per unit of time.
This allows to simulate users with fine-tuned thinktime settings and unmatched realism.
UserLoad: DNS Settings¶
Websites today often involve CDN (Content Delivery Network) today to improve their loading speed. Load testing websites with CDNs can be tedious due to nasty dns issues. When load testing a CDN without proper DNS configuration, most of the virtual users will just load test the same server, which is unrealistic. We'll cover the whole thing in a dedicated blog post.
It's now possible to clear DNS cache on each virtual user iteration to ensure new domain IP resolution on each round. You can even specify your own DNS servers if you would like to. With these advanced settings, it becomes easy to realistically load test websites powered by CDNs. Once again, we've made our best to make things easy to understand and to configure.
Max Redirections¶
Previously, the maximum number of HTTP redirections was limited to 5. We've increased this number to 20 to support a broader number of websites.
Memory settings¶
Caution, this is a highly advanced setting. The memory allocated per virtual user on each load generator can be tuned here (16Mb max). This setting allows to override our optimized memory settings in the case your scenario may need some tuning due to heavy pages. We recommend to leave this settings as is. We have already spent time tuning JMeter's memory settings to give you the best possible performances.
Bench Results¶
JMeter Logs¶
JMeter logs can now be downloaded at the end of a test. All JMeter engines logs are kept in a central place which makes it easy to download them all. Those logs are ingested by our Jenkins CI plugin to generate performance test trends.
Behind the scene¶
Those are the improvements we've made you may haven't noticed.
Backend Upgrades¶
We're constantly upgrading many of the open-source apis and apps to ensure you get the latest benefits of those updates. We recently upgraded our Elasticsearch cluster to the latest version which should result in a small performance boost. We also keep track of the latest available Rancher version and upgrade each time a new stable release is available. This is one of the greatest benefits for our customers: we take care of the upgrades.
Frontend Upgrades¶
As for the backend, we keep an eye on what's new in this field and constantly upgrade the APIs we use. It's a very time consuming task on the frontend side as the technologies on this side are moving especially fast. Good news is that we take care of that too :)