OctoPerf v12.4 - Integrate with Postman, Microsoft Teams, Grafana and Dynatrace
The focus of OctoPerf 12.4 is on integrations, first in regards to creating test scripts through our new Postman import released a few weeks ago. Through postman we also open OctoPerf to swagger and open API imports.
Microsoft Teams is also on the menu in terms of alerting before/after the tests, but the main dish is the backend listeners. Through these you can have our load generators send metrics to your own database in real time during the test. And by database I also mean several APM tools, like dynatrace, datadog and others.
On top of all this we've upgraded the scheduler to be able to chain several load tests one after another, upgraded to JMeter 5.4.1 and a couple of other features detailed below.
Improvements¶
API testing with OctoPerf¶
API testing has always been possible in OctoPerf but to enable faster test design we've worked on a Postman integration.
The way it works is that you can import an existing postman collection into OctoPerf and we will create all the associated requests automatically for you:
And the cherry on top is that Postman allows for a lot of other formats that you then transfer in OctoPerf, like Open API definitions. This way you can convert even more API definitions into OctoPerf scripts.
Integration with external database¶
Our new integration with backend listeners allows OctoPerf load generators to also send metrics to a third party while the test is running. You can use custom implementations such as the datadog JMeter plugin or our native integration for influxDB/Grafana, Graphite or Dynatrace:
This allows you to have your test metrics both in OctoPerf and your tool of choice, thus enhancing the options available in terms of analysis and integration in your tech stack.
Be aware that large tests can use an important number of load generators (usually around 1 for each thousand users) and each one of them will be sending metrics, this can generate stress on your database or trigger rate limitations.
Include controller¶
OctoPerf already supports module controllers and test fragments. We've now added support for the include controller as well.
The challenge for us was that each thread group is imported in a different virtual user and executed in a distinct docker container. This allows a great flexibility when running tests but in this situation having a JMX referencing another one through an include controller would not work since it would be in a different machine. To circumvent this we've designed a way to import all your JMX at once as a zip file.
The idea is to upload a single zip file with all your JMX inside, using the proper path and subfolders. That way we can parse the main JMX and create an inventory of all the include controllers and link them to the main one at runtime, visually the result looks like this:
Schedule multiple tests¶
The test scheduler is the perfect way to automate your tests, but it was only possible to schedule one test at a time. Now with the new scheduler you can add several tasks to a single schedule. That way we will execute them for you automatically, one after another as defined in the Tasks section:
The scheduler will proceed to execute the tasks from top to bottom, launching several tests in a row one after another. With this you do not have to worry about overlapping tests since we will wait for them to finish properly before proceeding to the next one.
Combined with our new external reporting you can use this new scheduler to execute several unit tests on a regular basis and display all the metrics in your preferred analysis tool like Grafana.
Microsoft teams notifications¶
Using the incoming webhook for Microsoft Teams, you can receive notifications whenever a test is launched, finished, successful, etc... It is similar to our slack or mail notifications:
JMeter 5.4.1¶
OctoPerf has been running on JMeter 5.2.1 for quite a while, the reason for this was to allow some time for critical plugins to update and make sure the groovy migration would not have a negative impact on existing scripts. We balanced this against the amount of new features in JMeter 5.3 and 5.4 and decided to wait before updating.
Now that all these subjects are taken care of, we've updated to the latest version again. If you have issues with Groovy scripts, you can check our updated templates to find working alternatives. The most common impact is around date/time functions but if you stick to the Java SimpleDateFormat & Calendar you should be fine.
Generic CSV¶
Another addition is support for all types of CSV variables. Previously it was only possible to import a CSV variable as a global variable that will be placed outside of the thread group. The reason for this is that it makes it possible for us to split the CSV file between load generators automatically.
But the drawback is that such variable will only move to next line at the end of the iteration. When you want to loop on a CSV inside an iteration, you need to use a local CSV variable placed under a sampler or controller. To handle this we now import these local CSV as a generic CSV variable:
Documentation¶
Since our last release we've created/updated some important documentation pages:
JMeter JMX import¶
Our page on importing from JMeter has been updated with more side by side examples. The goal is to quickly highlight the small differences between OctoPerf and pure JMeter so that you can adapt your JMX scripts when required.
https://api.octoperf.com/doc/design/create-virtual-user/import-jmeter-jmx/
As far as we know these differences are very minor and they are required to provide all our features on top of JMeter, but if you experience a difference that is frustrating to you, make sure to report it through our live chat or support@octoperf.com.
Enterprise edition HTTPS install¶
It can be challenging to understand all the implications of switching your own OctoPerf server from HTTP to HTTPS. To make this process easier we've created a step by step process with examples you can follow:
https://api.octoperf.com/doc/enterprise-edition/tutorials/https-setup/
Enterprise edition data persistence¶
Another key step when running your own Enterprise edition server is to store your test results and scripts in a secure folder. Since docker volumes can be confusing we've created a guide:
https://api.octoperf.com/doc/enterprise-edition/persistence/
Full changelog¶
For the complete list of fixed bugs, please refer to OctoPerf 12.4 Release Notes.