Does non-COVID-19 lung lesion help? investigating transferability in COVID-19 CT image segmentation

• Published in: Computer methods and programs in biomedicine Vol. 202; p. 106004
• Main Authors: Wang, Yixin, Zhang, Yao, Liu, Yang, Tian, Jiang, Zhong, Cheng, Shi, Zhongchao, Zhang, Yang, He, Zhiqiang
• Format: Journal Article
• Language: English
• Published: Ireland Elsevier B.V 01.04.2021
• Subjects: Algorithms; COVID-19; CT image; Databases, Factual; Humans; Image Processing, Computer-Assisted; Lung - diagnostic imaging; Lung - physiopathology; SARS-CoV-2; Segmentation; Tomography, X-Ray Computed; Transfer learning; COVID-19; CT image; Segmentation; Transfer learning
• ISSN: 0169-2607; 1872-7565; 1872-7565
• DOI: 10.1016/j.cmpb.2021.106004

•Investigating the transferability in COVID-19 CT segmentation.•Incorporating transferred lung lesion features from non-COVID19 datasets to gain significant improvement.•Evaluating four transfer learning methods on a publicly available COVID-19 CT dataset and three public non-COVID19 datasets.•Proposing a novel transfer learning strategy for COVID-19 dynamic feature selection and aggregation.•Transferability of COVID-19 model is crucial and helpful for doctors to make further assessment and quantifi cation. Background and Objective: Coronavirus disease 2019 (COVID-19) is a highly contagious virus spreading all around the world. Deep learning has been adopted as an effective technique to aid COVID-19 detection and segmentation from computed tomography (CT) images. The major challenge lies in the inadequate public COVID-19 datasets. Recently, transfer learning has become a widely used technique that leverages the knowledge gained while solving one problem and applying it to a different but related problem. However, it remains unclear whether various non-COVID19 lung lesions could contribute to segmenting COVID-19 infection areas and how to better conduct this transfer procedure. This paper provides a way to understand the transferability of non-COVID19 lung lesions and a better strategy to train a robust deep learning model for COVID-19 infection segmentation. Methods: Based on a publicly available COVID-19 CT dataset and three public non-COVID19 datasets, we evaluate four transfer learning methods using 3D U-Net as a standard encoder-decoder method. i) We introduce the multi-task learning method to get a multi-lesion pre-trained model for COVID-19 infection. ii) We propose and compare four transfer learning strategies with various performance gains and training time costs. Our proposed Hybrid-encoder Learning strategy introduces a Dedicated-encoder and an Adapted-encoder to extract COVID-19 infection features and general lung lesion features, respectively. An attention-based Selective Fusion unit is designed for dynamic feature selection and aggregation. Results: Experiments show that trained with limited data, proposed Hybrid-encoder strategy based on multi-lesion pre-trained model achieves a mean DSC, NSD, Sensitivity, F1-score, Accuracy and MCC of 0.704, 0.735, 0.682, 0.707, 0.994 and 0.716, respectively, with better genetalization and lower over-fitting risks for segmenting COVID-19 infection. Conclusions: The results reveal the benefits of transferring knowledge from non-COVID19 lung lesions, and learning from multiple lung lesion datasets can extract more general features, leading to accurate and robust pre-trained models. We further show the capability of the encoder to learn feature representations of lung lesions, which improves segmentation accuracy and facilitates training convergence. In addition, our proposed Hybrid-encoder learning method incorporates transferred lung lesion features from non-COVID19 datasets effectively and achieves significant improvement. These findings promote new insights into transfer learning for COVID-19 CT image segmentation, which can also be further generalized to other medical tasks.