Scenario‐Driven Metamorphic Testing for Autonomous Driving Simulators

• Published in: Software testing, verification & reliability Vol. 34; no. 7
• Main Authors: Zhang, Yifan, Towey, Dave, Pike, Matthew, Cheng Han, Jia, Quan Zhou, Zhi, Yin, Chenghao, Wang, Qian, Xie, Chen
• Format: Journal Article
• Language: English
• Published: Chichester Wiley Subscription Services, Inc 01.11.2024
• Subjects: automated testing; autonomous driving system (ADS); Complex systems; metamorphic exploration (ME); metamorphic relation input pattern (MRIP); metamorphic relation pattern (MRP); metamorphic testing (MT); oracle problem; Passenger safety; simulation; Simulators
• ISSN: 0960-0833; 1099-1689
• DOI: 10.1002/stvr.1892

ABSTRACT The proliferation of driver‐assistance features in vehicles has resulted in a growing interest among the public in fully autonomous driving systems (ADSs). However, the integration of software and hardware in these complex systems presents significant testing challenges, particularly with respect to ensuring passenger safety. To address these challenges, simulation has emerged as a crucial step in the testing of ADSs. This paper presents a solution to the challenges faced in testing ADSs, with a focus on the validation of ADS simulators. The proposed approach involves using simulations and metamorphic testing (MT) to generate multiple concrete metamorphic relations (MRs) for testing ADS simulators. In order to accomplish this goal, we introduce three metamorphic relation patterns (MRPs). Each MRP is accompanied by a metamorphic relation input pattern (MRIP) that aids in generating detailed MRs. These MRs are designed to identify potential issues within the ADS simulator. To simplify the testing process and facilitate MT for testers, a self‐evolving scenario‐testing framework is also presented. The framework allows testers to improve test cases and MRs iteratively until issues detected are confirmed. The benefits and limitations of the framework are demonstrated using an industry case study. Overall, this study offers a practical solution to the challenges in testing ADSs and provides useful insights into improving testing efficiency for researchers and practitioners in the field. This study introduces a novel approach for testing autonomous driving systems (ADSs) using metamorphic testing (MT) and simulation. It presents three metamorphic relation patterns (MRPs) and three metamorphic relation input patterns (MRIPs) with a scenario‐driven MT framework, demonstrating MT's effectiveness in testing ADS simulators through an industry case study.