Automated Scenario Generation and Evaluation Strategy for Automatic Driving System

• Published in: 2020 7th International Conference on Information Science and Control Engineering (ICISCE) pp. 1722 - 1733
• Main Authors: Guo, Peng, Gao, Feng
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.12.2020
• Subjects: Automatic driving systems; automatic test and evaluation; Automatic testing; Bayes methods; Clustering algorithms; Clustering methods; combinational testing; Complexity theory; model-in-the-loop test; Partitioning algorithms; Three-dimensional displays
• DOI: 10.1109/ICISCE50968.2020.00340

In this paper, we propose a new automated generation algorithm of test scenario for automatic driving systems, which is called Combinatorial Testing Based on Complexity (CTBC). Moreover, a new automatic evaluation strategy is proposed, which can conduct the test automatically and efficiently. With the defined concept of scenario complexity, we use CTBC algorithm to generate a more compact and effective set of test scenarios and find the optimal parameter considering both effectiveness and testing cost by the Bayesian optimization algorithm. Then, a hierarchical clustering method is used to partition the generated scenarios into clusters with high similarity. Finally, a two-way evaluation considering both the availability of test scenarios and the performance of the system under test is carried out on the automatic testing system. The effectiveness of the proposed strategy is validated by applying it to a Traffic Jam Pilot system on a hardware-in-the-loop test platform. The results show that the bigger the complexity of scenario is, the easier it is to reveal system defects. Furthermore, the proposed algorithm can shift the overall complexity distribution of the test scenarios significantly to a higher place while ensuring the coverage.