WatchMan project—A Python CASE framework for High Energy Physics data analysis in the LHC era

• Published in: Journal of computational science Vol. 4; no. 5; pp. 325 - 333
• Main Authors: Bianchi, Riccardo Maria, Brunelière, Renaud
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.09.2013
• Subjects: Computer aided software engineering; Data analysis; High Energy Physics; Large Hadron Collider; LHC; Physics framework; Python; High Energy Physics; Data analysis; Computer aided software engineering; Physics framework; Large Hadron Collider; LHC; Python
• ISSN: 1877-7503; 1877-7511
• DOI: 10.1016/j.jocs.2012.04.005

► Modern High Energy Physics experiments rely on very complex software frameworks. ► Writing data analysis code for those experiments is difficult and error prone. ► Many analyses have to be written to try to find out new physics from LHC data. ► We built a code generator framework to obtain reliable validated analysis code, ready to be run on data. ► Custom interfaces to different experiments and data formats can be added in a modular way. The world's largest particle collider LHC is taking data at CERN, in Geneva, providing a huge amount of data to be looked at, of the order of several Petabytes per year. Nowadays, Data Analysis in High Energy Physics (HEP) means handling billions of experimental data in custom software frameworks. Physicists have to access and select data interacting with the experiment using dedicated tools; they also have to apply filter functions and analysis algorithms to test hypotheses about the physics underlain. Modern HEP experiments rely on complex software frameworks, hence writing the analysis code is not always an easy task, and the learning curve is usually quite steep. Moreover each hypothesis requires a dedicated analysis, in order to have a better control on it and to be able to validate the results among different groups of researchers. And the writing of so many analyses can be error prone and time consuming. In order to ease the writing of such data analysis code, we built a software-generator: the idea is that the user inserts the settings of the physics analyses, and the final analysis code is automatically and dynamically generated, ready to be run on data. Python helped us to build such a package. Its high-level and dynamic nature, together with its flexibility and prototyping speed are the key features which made our choice. So we conceived and developed WatchMan, a Python CASE (Computer-Aided Software Engineering) framework to automatically generate reliable, easy to maintain and easy to validate HEP data analysis code.