A Taxonomy of Bad Practices in Software Performance Testing: Insights from Gray Literature and Practitioner Validation

Abstract

Performance testing is essential to ensure that software systems can sustain realistic workloads and meet expected service levels under load. While the research community has extensively studied performance testing methodologies, workload modeling, and tool support, little is known, and no consolidated knowledge exists, about how performance tests are actually implemented in practice and about the recurring mistakes that may silently compromise the reliability of performance evaluations.

This paper presents the first systematic investigation of bad practices in software performance testing. We focus on the recurring mistakes, misconceptions, and test smells that arise during the design, implementation, and execution of performance tests. To this end, we design a two-phase mixed-methods study that includes a gray literature review of 415 practitioner-oriented sources and a practitioner study comprising a survey and follow-up interviews with experienced performance engineers. The study yields an empirically grounded taxonomy of 29 bad practices and reveals how practitioners perceive their severity, prevalence, and impact on real-world performance evaluations. These findings offer an empirically grounded foundation for improving workload modeling and strengthening the overall quality and trustworthiness of performance testing processes.