A Similarity-Based Approach to Identify and Manipulate Coincidental Correct Test Cases for Fault Localization

• Published in: Journal of Information Science and Engineering Vol. 40; no. 6; pp. 1297 - 1320
• Main Authors: Estesnaei, Mohammad Mahdi, Araban, Saeed, Harati, Ahad
• Format: Journal Article
• Language: English
• Published: Taipei 社團法人中華民國計算語言學學會 01.11.2024; Institute of Information Science, Academia Sinica
• Subjects: Computing costs; Euclidean geometry; Fault detection; Fault location; Identification methods; Localization; Similarity; software debugging; similarity measures; spectrum-based fault localization; weighted manipulation strategy; coincidental correct test cases
• ISSN: 1016-2364
• DOI: 10.6688/JISE.202411_40(6).0009

Spectrum-based fault localization (SBFL) is one of the most popular fault localization techniques that uses coverage information and test results to calculate a suspicious score for every program statement. The effectiveness of SBFL suffers from the occurrences of coincidental correctness, which occurs when a fault is executed but no failure is detected. Identifying coincidental correct (CC) test cases can be modeled as a classification problem. Except in exceptional cases, proven identification of CC tests is not possible, so instead of using 0/1 results, we propose a similarity-based approach to identify CC test cases. A strategy is suggested to manipulate CC test cases for SBFL. In the first step, a low-cost computational method is proposed to identify CC test cases based on the similarity of the passed executions to the failed ones. Then, we proposed new similarity measures based on the original ones (such as Jaccard similarity and Euclidean distance) and presented a method to identify proven CC. Finally, a weighted CC test case manipulation strategy is proposed to mitigate the negative impact of CC test cases in SBFL. We evaluated the proposed method by conducting extensive experiments on 443 faulty versions of 13 popular subject programs, containing artificial and real faults. The results show that the proposed method can improve the accuracy of SBFL techniques with a very low computational cost.