The impact of Test-First programming on branch coverage and mutation score indicator of unit tests: An experiment

• Published in: Information and software technology Vol. 52; no. 2; pp. 169 - 184
• Main Author: Madeyski, Lech
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.02.2010; Elsevier Science Ltd
• Subjects: Computer programming; Computer programs; Dependent variables; Deviation; Empirical study; Fault diagnosis; Indicators; Multivariate analysis; Mutation; Mutations; Programming; Software development; Students; Studies; Systems development; Test-driven development; Test-First programming; Unit tests; Unit tests; Test-First programming; Test-driven development; Empirical study
• ISSN: 0950-5849; 1873-6025
• DOI: 10.1016/j.infsof.2009.08.007

Test-First programming is regarded as one of the software development practices that can make unit tests to be more rigorous, thorough and effective in fault detection. Code coverage measures can be useful as indicators of the thoroughness of unit test suites, while mutation testing turned out to be effective at finding faults. This paper presents an experiment in which Test-First vs. Test-Last programming practices are examined with regard to branch coverage and mutation score indicator of unit tests. Student subjects were randomly assigned to Test-First and Test-Last groups. In order to further reduce pre-existing differences among subjects, and to get a more sensitive measure of our experimental effect, multivariate analysis of covariance was performed. Multivariate tests results indicate that there is no statistically significant difference between Test-First and Test-Last practices on the combined dependent variables, i.e. branch coverage and mutation score indicator, ( F ( 2 , 9 ) = . 52 , p > . 05 ), even if we control for the pre-test results, the subjects’ experience, and when the subjects who showed deviations from the assigned programming technique are excluded from the analysis. According to the preliminary results presented in this paper, the benefits of the Test-First practice in this specific context can be considered minor. It is probably the first-ever experimental evaluation of the impact of Test-First programming on mutation score indicator of unit tests and further experimentation is needed to establish evidence.