Why We Chose JMeter
When we started to think how to build OctoPerf, about one year ago, we had several options to create our stress testing platform:
- Write our own HTTP engine from the ground,
- Use command line tools like Curl,
- Reuse an existing open-source HTTP engine, but which one? (Gatling Tool, JMeter, Grinder...)
It took us several months to choose which one was the right option. When you are building a startup, the biggest constraint is obviously Time. It is the limitation factor.
But, you must spend the right time on your product to bring value to the customers.
Writing Our own Engine from scratch¶
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
Writing your own solution is something really enjoyable. Every developer likes to create APIs, write code. We love to show others what we have built, and how powerful it is.
But it comes at the expense of the time it takes, as well as the experience required to build such a complex application. Most HTTP Load testing tools are distributed and multi-threaded. It even more difficult and time-consuming to develop such applications.
This solution seemed really expensive in term of Time. Too Expensive. We could not afford spending two years or more just to develop the engine.
Driving Command-line tools¶
Command-line tools are the most straightforward way to send HTTP requests. As simple as it is, an integration with Java may be tedious. It would have been easy to send requests, but quite harder to get performance metrics using such tools.
As we are Java engineers, we soon shifted towards Java and Scala tools such as JMeter or Gatling.
Open-source HTTP Engines (JMeter, Gatling)¶
There are many open-source performance testing solutions on the market such as JMeter, the Gatling Tool or the Grinder.
Gatling Tool¶
Gatling tool is pretty new (a few years old - compared to JMeter) and its engine, based on Akka, is highly optimized. To sum up, this tool can simulate very high load of virtual users with fewer threads than other tools. Basically, most load testing tools are using one Thread per Virtual User instance, Gatling is not.
The major drawback of this tool is that it is based on Scala language. We spent some time studying the Scala language, but concluded that it would take too much time to get sufficient skills for being able to interface with Gatling.
Stéphane Landelle has made a great work on this tool. Scala is a functional language with some really nice improvements over Java, but the learning curve is stepper.
On the other hand, JMeter is written in pure Java, and comes with enjoyable plugin features.
JMeter¶
JMeter is the lead industry standard on the load testing market. JMeter is developed in Java since 1998. We're pretty confident by using JMeter as the HTTP engine for our load testing tool, there are years of development and experience behind it.
JMeter has a plugin architecture which allows to extend its features very easily. Everything has been thought for extension and improvement. The codebase is not perfect, but thanks to many contributors, and a large users community it's stable and well tested.
This open-source tool also supports many protocols, such as HTTP/HTTPS, FTP, JDBC, LDAP and more. If need be, we could quickly integrate them in our load testing platform.
What OctoPerf brings on top of JMeter¶
Given that JMeter is a free open-source tool, with great features, why the heck would you use OctoPerf?
We think there is a real added value to JMeter by upgrading it with:
- Modern Web Interface. Have a browser? You're ready,
- Accessibility: have your projects online, no IT backup management,
- No Installation Required: use any computer to access your data,
- Cloud Stress Test Infrastructure: no more time spent messing up with machines, we take care of it.
To create OctoPerf, JMeter is the right decision. Why would we reinvent the wheel?