PYRO-NN: Python Reconstruction Operators in Neural Networks

• Published in: arXiv.org
• Main Authors: Syben, Christopher, Michen, Markus, Stimpel, Bernhard, Seitz, Stephan, Ploner, Stefan, Maier, Andreas K
• Format: Paper Journal Article
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 30.04.2019
• Subjects: Algorithms; Computed tomography; Computer Science - Computer Vision and Pattern Recognition; Image reconstruction; Iterative methods; Machine learning; Medical imaging; Neural networks; Operators; Projectors; Source code
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.1904.13342

Purpose: Recently, several attempts were conducted to transfer deep learning to medical image reconstruction. An increasingly number of publications follow the concept of embedding the CT reconstruction as a known operator into a neural network. However, most of the approaches presented lack an efficient CT reconstruction framework fully integrated into deep learning environments. As a result, many approaches are forced to use workarounds for mathematically unambiguously solvable problems. Methods: PYRO-NN is a generalized framework to embed known operators into the prevalent deep learning framework Tensorflow. The current status includes state-of-the-art parallel-, fan- and cone-beam projectors and back-projectors accelerated with CUDA provided as Tensorflow layers. On top, the framework provides a high level Python API to conduct FBP and iterative reconstruction experiments with data from real CT systems. Results: The framework provides all necessary algorithms and tools to design end-to-end neural network pipelines with integrated CT reconstruction algorithms. The high level Python API allows a simple use of the layers as known from Tensorflow. To demonstrate the capabilities of the layers, the framework comes with three baseline experiments showing a cone-beam short scan FDK reconstruction, a CT reconstruction filter learning setup, and a TV regularized iterative reconstruction. All algorithms and tools are referenced to a scientific publication and are compared to existing non deep learning reconstruction frameworks. The framework is available as open-source software at \url{https://github.com/csyben/PYRO-NN}. Conclusions: PYRO-NN comes with the prevalent deep learning framework Tensorflow and allows to setup end-to-end trainable neural networks in the medical image reconstruction context. We believe that the framework will be a step towards reproducible research