BivwAk! - Case study
BivwAk! is the transformation hub for BNP Paribas. Its goal is to bring new services and skillsets at the crossroad of all BNP Paribas entities. It's in this context that Dino learned about OctoPerf.
Dino is very passionate about testing because it's a constantly evolving space.
With new methodologies and tools emerging all the time.
We met Dino at the french testing event JFTL. After a short discussion it was clear that OctoPerf would be a very good fit at BivwAk!
BivwAk! requirements¶
To understand why OctoPerf fit so well into BivwAk!, we need a closer look at their use case. BivwAk! needs tools that can fit in many different contexts. The testers will be working part time on load testing and for several different projects. That makes scheduling very difficult since projects can start all at once on a very short notice.
That means Dino evaluated several other tools and ruled them out because:
- The installation procedure was complex with a lot of manual actions,
- The tool maintenance would consume a lot of the tester time,
- They lacked the flexibility to be used concurrently by any number of users.
The tool selection¶
To sum up, the performance testing tool had to be:
- Available to several teams at the same time,
- Accessible in a web browser, or with little to no installation,
- Compatible with the existing CI environment,
- Cost-efficient in order to increase the testing capacity.
Other aspects were also considered, and we will get to them later, but this is at the core of what BivwAk! must be able to provide to all the teams working there.
For each of these requirements, OctoPerf had the perfect answer:
- Our licenses allow for any number of logins. There is no limit on concurrent tests. And load generators are automatically started from the cloud,
- Everything happens in a browser, even the recording. That means no installation required,
- OctoPerf is a UI built on top of a REST API. This allows to integrate in any context,
- Our pricing uses a single parameter: the max concurrent users. Everything else is unlimited. No need to buy test credits or split the license.
Feedback¶
After a quick proof of concept, several testers started using OctoPerf on real projects. In the next sections we've listed what they liked the most when using OctoPerf.
User interface¶
The user interface helped on-board everyone very fast. They all found it very user-friendly. This is especially important because each one of them had to deal with several test projects at once. And when you switch context often it is very helpful to have a quick vision over what was created before:
JMeter-based¶
The fact that OctoPerf is based on JMeter is also important.
First it is always reassuring when you need to extend your tests scripts with plugins.
But it also allows you to revert to open source at any time, this way you do not feel stuck with a particular product. and on OctoPerf's side, it pushes us to do a good job with the product if we don't want to lose all our users to JMeter.
Integrations¶
OctoPerf comes with many useful integrations, in particular BivwAk! uses our Dynatrace integration to gather all metrics in one place. Other integrations with CI tools or monitoring tools can also help in other contexts.
Support chat¶
But the most important is the reactivity when it comes to support. Since through the chat we answer in 5-10 minutes.
Conclusion¶
In the end BivwAk! was able to conduct around 10 load testing projects in the last 6 months. With sometimes 3 to 4 projects at the same time.
The testers were beginners; but easily on-boarded thanks to OctoPerf. Through our automation and ease of use they were operational in a matter of hours.
According to Dino the key selling points were :
- The SaaS version of OctoPerf: this way there is no infrastructure to manage. Load generators start on demand whatever the amount of concurrent tests.
- The reactivity of OctoPerf's support team.
And of course this is only the beginning of a very successful collaboration!