Fine-Tuning and training of densenet for histopathology image representation using TCGA diagnostic slides

• Published in: Medical image analysis Vol. 70; p. 102032
• Main Authors: Riasatian, Abtin, Babaie, Morteza, Maleki, Danial, Kalra, Shivam, Valipour, Mojtaba, Hemati, Sobhan, Zaveri, Manit, Safarpoor, Amir, Shafiei, Sobhan, Afshari, Mehdi, Rasoolijaberi, Maral, Sikaroudi, Milad, Adnan, Mohd, Shah, Sultaan, Choi, Charles, Damaskinos, Savvas, Campbell, Clinton JV, Diamandis, Phedias, Pantanowitz, Liron, Kashani, Hany, Ghodsi, Ali, Tizhoosh, H.R.
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01.05.2021; Elsevier BV
• Subjects: Artificial neural networks; Cancer; Colorectal cancer; Colorectal carcinoma; Computer applications; Deep features; Deep learning; Endometrial cancer; Endometrium; Feature extraction; Genomes; Histopathology; Human pathology; Image acquisition; Image analysis; Image classification; Image processing; Image representation; Image search; Medical diagnosis; Medical imaging; Neural networks; Paraffin; Paraffins; Pathology; Performance evaluation; Representations; TCGA; Topology; Transfer learning; Deep learning; Image search; Histopathology; Transfer learning; Image representation; Deep features; TCGA; Image classification
• ISSN: 1361-8415; 1361-8423; 1361-8431; 1361-8423
• DOI: 10.1016/j.media.2021.102032

•Digital Pathology can benefit from computerized search methods.•More discriminative image features result in more accurate search and classification.•Training a deep network by a variety of tumor types will provide better image features.•Using image patches at 20X magnification for training will help network to focus on cell nuclei distributions and shapes.•Fine-tuned and trained DenseNet with 7 million weights to create a new network, KimiaNet, is customized for computational pathology.•KimiaNet can be used as “feature extractor” for image analysis.•An algorithm is proposed to employ unlabeled whole slide images for KimiaNet.•KimiaNet shows excellent results using three public histopathology datasets, among other an average 44% accuracy increase for 12 tumor types. [Display omitted] Feature vectors provided by pre-trained deep artificial neural networks have become a dominant source for image representation in recent literature. Their contribution to the performance of image analysis can be improved through fine-tuning. As an ultimate solution, one might even train a deep network from scratch with the domain-relevant images, a highly desirable option which is generally impeded in pathology by lack of labeled images and the computational expense. In this study, we propose a new network, namely KimiaNet, that employs the topology of the DenseNet with four dense blocks, fine-tuned and trained with histopathology images in different configurations. We used more than 240,000 image patches with 1000×1000 pixels acquired at 20× magnification through our proposed “high-cellularity mosaic” approach to enable the usage of weak labels of 7126 whole slide images of formalin-fixed paraffin-embedded human pathology samples publicly available through The Cancer Genome Atlas (TCGA) repository. We tested KimiaNet using three public datasets, namely TCGA, endometrial cancer images, and colorectal cancer images by evaluating the performance of search and classification when corresponding features of different networks are used for image representation. As well, we designed and trained multiple convolutional batch-normalized ReLU (CBR) networks. The results show that KimiaNet provides superior results compared to the original DenseNet and smaller CBR networks when used as feature extractor to represent histopathology images.