Playwright brings real-browser load testing to OctoPerf SaaS & On-Premise versions, offering client-side realism that protocol testing alone cannot capture.
Browser-based VUs execute actual user interactions and handle dynamic parameters automatically, reducing scripting effort.
Protocol-based VUs remain far more efficient, generating heavy loads at minimal resource cost while providing deep server-side insights.
A side-by-side campaign shows Playwright offers richer realism but consumes dramatically more CPU and delivers slower journeys.
The most effective approach is hybrid: protocol VUs for load, Playwright VUs for end-user experience under stress.
This balance keeps costs and resource usage under control while still exposing client-side performance issues.
This is Chapter 5 of our journey of non-functional testing with Otto Perf. This is the last chapter in Otto’s journey of performance testing a new application for OctoCar.
In Chapter 4 Otto was looking at ways to check the impact of his performance tests on the infrastructure that the application was running on.
He found out that the application had been instrumented using Dynatrace and he could use this to analyse all aspects of the application architecture.
He also discovered that with the use of a custom JMeter header in his test scripts he could easily identify the transactions that were generated as part of his performance testing.
The ability to create custom metrics based on his tests and to produce dashboards meant that Otto was in a very good place when it came to monitoring.
The introduction of application monitoring and a subsequent increase in server capacity uncovered an issue with Otto’s reporting process.
He had missed the fact that the application was regressing over time but because the transactions being measured were still withing their agreed non-functional requirement tolerances he did not notice this.
He did some work to ensure that he was tracking trends across his results to ensure that transaction regression would be picked up in the future. This trend analysis also offered the ability. To run tests at different times of the day and under alternative load profiles and determine the differences.
In this Chapter we will see development finish and Otto will discover that the end of a programme is not the end of performance testing and learn about the benefits that the creation of a regression test can bring.
Otto will also find alternative uses for his performance tests outside of their primary purpose of generating load and concurrency and will start to understand that the assets you create when building performance tests can provide benefit to many other IT and Non-IT related activities.
In Chapter 3 Otto started to use build and deployment technologies in the form of GIT and Jenkins to support his performance testing.
He learnt about the concepts of Continuous Integration and Continuous Deployment and spent some time integrating his performance tests into a push to production pipeline.
Otto was able to schedule his performance testing using Jenkins pipelines and he spent some time building a solution that analyse the test results for him so that he was able to spend more time building performance tests. This analysis solution also helped him determine if the performance tests had passed in the fully automated push to production pipelines.
Otto also discovered that being integrated within the development teams provided more benefits that just being able to develop tests and subsequently execute them earlier in the development lifecycle. He discovered that several of the development practises such as code reviews and pair programming could equally apply to test development and this was something that Quality Engineering teams had not, in his experience, regularly done.
In this Chapter we will follow Otto as he starts to explore the benefits that technologies that instrument your application can provide to performance testing. Otto will also dive deeper into trend analysis of his performance test results and look to see what the performance results data is telling him.
Otto was extremely busy in Chapter 2 where he had built the strategy, he would adopt for the performance testing and had considered the load profiles he would use to generate load against the application.
Otto also made a start in Chapter 2 on how he would aggregate and report on the tests he was executing and how he could compare tests results from previous tests execution cycles. He found out that his performance testing would influence how the application infrastructure would be sized in the production environment.
Otto also started writing tests and discovered that approaching these in a modular fashion can lead to significant time saving when the application changes, which is does when performance testing early in the delivery lifecycle.
In this chapter we will follow Otto as he discovers how he can execute his tests using his GIT repository and Jenkins. And how he can ensure that performance testing is integrated into the Continuous Delivery / Continuous Integration (CI/CD) framework that the development teams are working in.
Otto will also discover what push to prod pipelines are and how performance testing can be included in this approach to production deployment.
Otto had previously got to the point where he had defined a set of non-functional requirements and risk assessed these with the programme team. Otto had also had the opportunity to be involved in the design process which had allowed him to encourage the architects, and development leads to consider application performance as part of their architectural design principles.
In this chapter we will follow Otto’s progress as he starts to consider how he is going to write his performance tests and how he will incrementally write tests in line with the development activity. Otto will also consider how he can make sense of the performance tests results he will gather and how he will ensure that the infrastructure is fit for purpose.