We have spoken in a previous blog post about documentation about how we believe that if you are building performance tests that support either Continuous Integration or Continuous Delivery then having to produce performance testing documentation before and after each test does not fit with this methodology.
This got us to thinking that if we truly want to use an agile framework and continually push code to production then we need to also consider how we monitor and report on our performance test results produced from our testing tools.
If you have an agile testing process, then you will have functional tests built into this deployment process that run in each environment and if you have a mature robust process then this process deploys to production with no manual intervention if all the functional tests have passed along with any deployment checks.
Now in reality we need to have a performance testing stage in this process as well as each release needs to go through a performance test, whether that is a full suite of peak volume, soak test, scalability tests scenarios or whether that is a regression tests of core functionality.
And this is the bit that we are going to discuss in this blog post because if your organisation wants to push code directly to production then you do not have the luxury of running your performance test, analysing the results and writing up a report to be signed off.