Multi‐objective optimization‐based and fault localization‐oriented test case generation for novice programs - Liu, Yong; Yang, Zezhong; Fan, Luxi; Wu, Yonghao; Chen, Xiang; Zhou, Xiaotang (Journal of software : evolution and process, 2024)
Summary Online judgment (OJ) systems are capable of evaluating program results by automatically executing test cases, significantly improving the efficiency of traditional guidance approaches. Moreover, existing studies attempt to assist novices through automated fault localization techniques to provide feedback to novices, which can help them quickly find the location of faulty statements. Among them, spectrum‐based fault localization (SBFL) techniques have been widely used for their lightweight and efficiency, which only requires coverage information and test results of test cases to conduct fault localization. However, manually constructing high‐quality test cases for a large number of OJ questions is tough work to complete. To solve this problem, we propose the novice program‐oriented Multi‐Objective Optimization‐Based Fault Localization‐Oriented Test Case Generation (MFTCG) for automatically generating test inputs. Specifically, we use multi‐objective optimization algorithms to evolve the test case in terms of both fault localization and faulty code detection capability. We conduct experiments with 8911 programs from the well‐known public OJ platform AtCoder. The results show that our proposed approach MFTCG can achieve the best fault localization performance compared with existing automated test case generation approaches in most cases and can achieve the similar faulty code detection capability compared to manually designed test cases. We propose a fault localization‐oriented test data generation approach multi‐objective optimization‐based fault localization‐oriented test case generation (MFTCG), which utilizes the multi‐objective optimization‐based algorithm to generate test data. To evaluate the effectiveness of our approach, we collect a novice program data set for faulty localization from programming website AtCoder, which contains 8911 faulty programs and 14,392 faults. The experimental results show that in most cases, test cases generated by our approach can achieve the best fault localization performance compared with the test cases generated by other approaches.