Apache JMeter is old
"JMeter is old technology", I hear this a lot.
"Let’s use this tool or that tool instead of JMeter as it’s the latest", I hear this a lot.
"We need a lightweight tool without the GUI interface to write our tests as that will make us more agile", I hear this a lot.
It’s all nonsense it really is, there seems to always be a call for using the most modern technology for all forms of testing whether its Performance Testing or Automated Functional Testing and the result of this is that the people writing the tests spend to long learning the new tools and not enough time building something that will ensure the software being developed is the best it can be.
I am conscious that the above statement is a bit or a generalisation and there are some exceptions but for many organisations this is true.
The aim of all these tools is the same thing, to support your QA journey through to the delivery of a robust, well performing piece of software, the tools you use to accomplish this are not important if you have a robust modular framework in place, and you have a sensible approach to data management and reporting.
We are going to look at in this post the reason why JMeter is so good at what it does and why just because it is considered old technology by some still makes it the right choice for many testing professionals.
Exceptions¶
As mentioned in the introduction there are exceptions where some of the more lightweight tools can be used to build tests if the application you are testing against does not require complex data set up or the use of message queues or databases.
Reality¶
In many cases robust performance testing requires more than simple tests and in a world of microservices where there is interaction between multiple services data management and the use of many technologies in your testing is common and therefore a tool that supports many protocols is required.
With very few exceptions, and none that are Open Source, can tests that cover all these terminologies be built in any other single performance toolset.
Lightweight execution is important for many aspects for performance testing, especially when running from within Jenkins or Travis for example, and JMeter can be run from the command line and supports many switches to allow for complex execution scenarios.
There are also a growing number of technologies that encapsulate JMeter and offer a lightweight interface, many provide very comprehensive reporting approaches and are well documented online.
Get the job done¶
It’s all about the application testing, people forget this.
It does not matter what tool you use it matters how you use it.
JMeter offers a way to build very complex tests and supports many protocols, there are many libraries that are available online to enhance the vanilla version of JMeter giving it the functionality to build any scenario you can possibly imagine.
Ultimately are you making your software product better with your performance testing, if the user experience is going to improve as part of your efforts then the choice of tool is academic.
We touched upon this in the introduction, but it’s worth raising again, many organisations spend far too much time trialling and running proof of concepts against tools just for the sake or using the latest and if they do decide to use a new tool sometimes the migration activity takes a significant amount of time that could be better spend improving your framework or your data management techniques or looking at training or knowledge sharing.
Clearly if you have no performance testing capability, and you are starting from a position of building a performance testing framework then you need to explore new tools, but I would urge you to seriously consider JMeter regardless of its age as it is stable, scalable and there is a huge amount of online support and knowledge.
Online Support¶
The volume of forums, knowledge bases, example projects and shared libraries is significant, and I would feel confident in saying that there are few other non-commercial tools with the volume of online support.
I have found over the many years I have been using JMeter that I have always found an answer to even the most complex JMeter problem by using the wealth of knowledge that exists in the online community, even if it your first time using JMeter there is so much freely available knowledge about that even the most inexperienced user can be building tests in a very short space of time.
As long as that online presence is there, and it is growing, then JMeter will continue to thrive.
The Apache Foundation still release several stable releases a year and therefore that is an indication of its continued presence in the market and commitment by the Apache Foundation to make sure JMeter is supported and meets the demands of current technologies.
JMeter GUI¶
The move towards Dev in Test would make you think that every organisation had embedded QA roles in the development teams and whilst I agree that this is the way forward and is an exciting way to work it is not the reality for many organisations.
Many organisations have separate testing teams and in some cases the performance testing is done by a member of the QA team that has little experience in performance testing and in these cases the aspects of JMeter that many find the most antiquated are the most useful.
The JMeter GUI does come in for criticism in the way it looks, but I think that if you are a relative novice to performance testing than having a UI regardless of what it looks like is extremely useful. It gives you the ability to visualise the structure of your tests in a way that is difficult for the beginner when faced with a performance testing tools that is purely written in code.
You get a feel for how you can build complexity and manage conditional business flows in a visual way that can significantly speed up the time taken to build these complicate test flows.
Some other things it does well¶
This is not a comprehensive list by any stretch of the imagination but from a personal perspective the way JMeter makes performance testing these technologies easy is what makes it a real contender for any performance testing team.
AMQ and Kafka.¶
Message queues are a huge part of current technology architecture and to test these in JMeter is simple, and the samplers that exist to test message queues are easy to use and intuitive.
The ability to embed a message queue test into a more complex scenario is very powerful and difficult to do in other performance testing tools.
The way the message queues samplers can manage a complex load profile is good and as discussed before the ability to encompass this into a test with other protocols is extremely powerful.
Database technologies¶
Other tools can query databases as that is a fundamental principle of performance testing, but as with the message queue samplers adding database samplers to a complex test scenario is a very powerful combination.
The simple addition of a JSR223 sampler can give you all the control you need over the response to a database query, and it is these combinations of samplers that make JMeter so powerful.
Plugins¶
There is a Blog Post regarding Jmeter plugins that can be found here which discusses the rich set of features that have been written to add to the vanilla build of JMeter and is worth reading to gain more insight into the additional features that make JMeter so good at what it does.
Conclusion¶
Just because a testing tool is mature does not make it the wrong tool to use, the common thread running through this post is the fact that it is how you use the tool that is important rather than always focusing on the latest tool.
Using a tool just because it’s the latest is the wrong mindset for a QA function, if the latest tool gives you all you need in terms of support for your technology stack and you have resources available who understand the language and behaviours of the tool then that is clearly your best option but changing your toolset because it is considered outdated is not a good idea.
As stated in the Introduction, it is all about delivering benefit and making the product you are testing fit for purpose from a performance perspective and therefore how you accomplish this is not important.
While JMeter is considered old technology, it is still being developed and enhanced in line with technology changes.
So, if you hear somebody suggesting that you move all your performance testing to a new toolset, and you feel that this will cause you a considerable amount of work or think it will make you unproductive then challenge this as sometimes mature, even old testing technology is the best fit for your organisation and the product, that performs in line with expected requirements, that you deliver to a production environment is all that really matters regardless of the tools used to test it.