Stress testing of task deadlines: A constraint programming approach

• Published in: Proceedings - International Symposium on Software Reliability Engineering pp. 158 - 167
• Main Authors: Di Alesio, Stefano, Nejati, Shiva, Briand, Lionel, Gotlieb, Arnaud
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.11.2013
• Subjects: Computational modeling; Constraint Programming; Genetic algorithms; Programming; Real-time systems; Stress; Stress Testing; Testing; Timing
• ISSN: 1071-9458
• DOI: 10.1109/ISSRE.2013.6698915

Safety-critical Real Time Embedded Systems (RT-ESs) are usually subject to strict timing and performance requirements that must be satisfied for the system to be deemed safe. In this paper, we use effective search strategies whose goal is finding worst case scenarios with respect to deadline misses. Such scenarios can in turn be used to test the target RTES and ensure that it satisfies its timing requirements even under worst case conditions. Specifically, we develop an approach based on Constraint Programming (CP) to automate the generation of test cases that reveal, or are likely to, task deadline misses. We evaluate it through a comparison with a state-of-the-art approach based on Genetic Algorithms (GA). In particular, we compare CP and GA in five case studies for efficiency, effectiveness, and scalability. Our experimental results show that, on the largest and more complex case studies, CP performs significantly better than GA. Furthermore, CP offers some advantages over GA, such as it guarantees a complete search when there is sufficient time, and, being deterministic, it doesn't rely on parameters that potentially have a significant effect on the search and therefore need to be tuned. Hence, we conclude that our results are encouraging and suggest this is an advantageous approach for stress testing of RTESs with respect to timing constraints.