Giving pandas ROOT to chew on: experiences with the XENON1T Dark Matter experiment

• Published in: Journal of physics. Conference series Vol. 898; no. 4; pp. 42003 - 42010
• Main Authors: Remenska, D, Tunnell, C, Aalbers, J, Verhoeven, S, Maassen, J, Templon, J
• Format: Journal Article
• Language: English
• Published: Bristol IOP Publishing 01.10.2017
• Subjects: Algorithms; Cloud computing; Cost analysis; Dark matter; Data analysis; Data processing; Libraries; Particle physics; Physics; Signal processing; Software; Xenon
• ISSN: 1742-6588; 1742-6596
• DOI: 10.1088/1742-6596/898/4/042003

In preparation for the XENON1T Dark Matter data acquisition, we have prototyped and implemented a new computing model. The XENON signal and data processing software is developed fully in Python 3, and makes extensive use of generic scientific data analysis libraries, such as the SciPy stack. A certain tension between modern "Big Data" solutions and existing HEP frameworks is typically experienced in smaller particle physics experiments. ROOT is still the "standard" data format in our field, defined by large experiments (ATLAS, CMS). To ease the transition, our computing model caters to both analysis paradigms, leaving the choice of using ROOT-specific C++ libraries, or alternatively, Python and its data analytics tools, as a front-end choice of developing physics algorithms. We present our path on harmonizing these two ecosystems, which allowed us to use off-the-shelf software libraries (e.g., NumPy, SciPy, scikit-learn, matplotlib) and lower the cost of development and maintenance. To analyse the data, our software allows researchers to easily create "mini-trees"; small, tabular ROOT structures for Python analysis, which can be read directly into pandas DataFrame structures. One of our goals was making ROOT available as a cross-platform binary for an easy installation from the Anaconda Cloud (without going through the "dependency hell"). In addition to helping us discover dark matter interactions, lowering this barrier helps shift the particle physics toward non-domain-specific code.