Metamorphic testing 20 years later a hands-on introduction

• Published in: 2018 IEEE/ACM 40th International Conference on Software Engineering: Companion (ICSE-Companion) pp. 538 - 539
• Main Authors: Segura, Sergio, Zhou, Zhi Quan
• Format: Conference Proceeding
• Language: English
• Published: New York, NY, USA ACM 27.05.2018
• Series: ACM Conferences
• Subjects: Computer bugs; Google; metamorphic testing; Software and its engineering > Software creation and management > Software verification and validation > Software defect analysis > Software testing and debugging; Software engineering; Software quality; Software testing; tutorial; metamorphic testing; tutorial
• ISBN: 145035663X; 9781450356633
• ISSN: 2574-1934
• DOI: 10.1145/3183440.3183468

Two of the key challenges in software testing are the automated generation of test cases, and the identification of failures by checking test outputs. Both challenges are effectively addressed by metamorphic testing (MT), a software testing technique where failures are not revealed by checking an individual concrete output, but by checking the relations among the inputs and outputs of multiple executions of the software under test. Two decades after its introduction, MT is becoming a fully-fledged testing paradigm with successful applications in multiple domains including, among others, big data engineering, simulation and modeling, compilers, machine learning programs, autonomous cars and drones, and cybersecurity. This technical briefing will provide an introduction to MT from a double perspective. First, we will present the technique and the results of a novel survey outlining its main trends and lessons learned. Then, we will go deeper and present some of the successful applications of the technique, as well as challenges and opportunities on the topic. The briefing will be complemented with practical exercises on testing real web applications and APIs.