Smelly Bad Practices in Performance System Tests: Detection, Prevalence, and Lifetime

Abstract

Performance testing is essential to ensure that software systems sustain realistic workloads and meet expected service levels under load. While prior research has extensively investigated performance testing methodologies, workload modeling, and tool support, there is limited consolidated knowledge regarding recurring bad practices that may silently compromise the reliability of performance evaluations. %how performance tests are implemented in practice and which recurring mistakes may silently compromise the reliability of performance evaluations. This paper provides the first catalog of bad practices in performance test code, derived through a gray literature review of practitioner sources. Building on this catalog, we introduce PT-LINTR, an automated tool that detects instances of these bad practices in Apache JMeter test artifacts. We apply PT-LINTR to the E2EGit dataset, analyzing 244 open-source test artifacts to investigate the prevalence, evolution, and lifespan of performance test bad practices, as well as the reasons behind their introduction, removal, and worsening over time. The study identified ten bad practices and found that they are widespread, often introduced near major releases during periods of high developer workload.

Data and Code

A complete replication package is available on .