JavaScript: The (Un)Covered Parts

• Published in: ICST 2017 : proceedings : 10th IEEE International Conference on Software Testing, Verification and Validation : 13-17 March, Tokyo, Japan pp. 230 - 240
• Main Authors: Fard, Amin Milani, Mesbah, Ali
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.03.2017
• Subjects: Browsers; code coverage; empirical study; JavaScript applications; Libraries; Measurement; Production; test quality; Testing; Writing
• DOI: 10.1109/ICST.2017.28

Testing JavaScript code is important. JavaScript has grown to be among the most popular programming languages and it is extensively used to create web applications both on the client and server. We present the first empirical study of JavaScript tests to characterize their prevalence, quality metrics (e.g. code coverage), and shortcomings. We perform our study across a representative corpus of 373 JavaScript projects, with over 5.4 million lines of JavaScript code. Our results show that 22% of the studied subjects do not have test code. About 40% of projects with JavaScript at client-side do not have a test, while this is only about 3% for the purely server-side JavaScript projects. Also tests for server-side code have high quality (in terms of code coverage, test code ratio, test commit ratio, and average number of assertions per test), while tests for client-side code have moderate to low quality. In general, tests written in Mocha, Tape, Tap, and Nodeunit frameworks have high quality and those written without using any framework have low quality. We scrutinize the (un)covered parts of the code under test to find out root causes for the uncovered code. Our results show that JavaScript tests lack proper coverage for event-dependent callbacks (36%), asynchronous callbacks (53%), and DOM-related code (63%). We believe that it is worthwhile for the developer and research community to focus on testing techniques and tools to achieve better coverage for difficult to cover JavaScript code.