Modern software performance testing practices have been changing and developing at an unprecedented speed. We are also witnessing a growing demand for much faster cycle times, including a consensus for performance testing to “shift left.” This “shifting” describes that if you want to increase test coverage with no slowing down, then you must “shift” earlier in the delivery cycle of software.
In order to understand why this change had to occur in the first place, we need to go back to the “Waterfall” era of performance testing. The good thing about the Waterfall era was that we had powerful performance testing tools at our disposal. However, the serious downside was that these tools were too complex to use. So, what was the solution? Well, these constraints actually gave birth to the CoE (Center of Excellence).
The CoE took care of the following performance-testing related tasks:
- Control of the testing infrastructure;
- Execute and design performance-related tests;
- Perform detail analysis;
- Report the performance results back
This in turn led to the short release cycles of DevOps and Agile. Due in part from the realization that the “old ways” where you perform a test and then wait for a testing window was slow and inefficient. The fundamental aspects of Agile and DevOps development relieved bottlenecks in building and testing.
Yet, there was one huge problem left to be solved. We needed to overcome the centralized bottleneck of performance testing process. This solution was found in numerous open-source tools, which were decentralized by many contributions. The great thing about the open-source tools is that we get frequent updates from developers from all corners of the world. In return, these developers get the feedback from the community that’s using these tools. It’s a two-way street and a win-win scenario for all participants. The performance testing process keeps improving because the code itself has been constantly improving and changing.
Today, Center of Excellence CoE 1.0 has transformed itself into Center of Enablement CoE 2.0. Instead of a centralized bottleneck, we have ended up with the democratized performance testing process. CoE 2.0 is perceived as the genuine provider of democratized frameworks, API keys, logins, and automation interfaces.
When open-source tools used for performance testing are combined with commercial platforms, what we get in return are improvements that allow us to compress test cycle times and run all the tests immediately with no need to wait in a queue. Test and speed coverage can then increase ten, a hundred or even a thousand times compared with old practices.