Interpretable pap-smear image retrieval for cervical cancer detection with rotation invariance mask generation deep hashing

• Published in: Computers in biology and medicine Vol. 154; p. 106574
• Main Authors: Özbay, Erdal, Özbay, Feyza Altunbey
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.03.2023; Elsevier Limited
• Subjects: Accuracy; Algorithms; Artificial intelligence; Artificial neural networks; Cancer; Cellular biology; Cervical cancer; Cervix Uteri; CNN; Datasets; Deep hashing; Deep learning; Developing countries; Female; Human papillomavirus; Humans; Image coding; Image Processing, Computer-Assisted - methods; Image retrieval; Industrialized nations; Infections; Internal Medicine; Invariance; LDCs; Machine learning; Medical imaging; Medical personnel; Medical screening; Methods; Modules; Mortality; Neural coding; Neural networks; Other; Pap smear; Papanicolaou Test - methods; Rotation; Semantics; Spatial attention; Uterine Cervical Neoplasms - diagnostic imaging; Wavelet transforms; Womens health; CNN; Image retrieval; Deep hashing; Spatial attention; Cervical cancer
• ISSN: 0010-4825; 1879-0534; 1879-0534
• DOI: 10.1016/j.compbiomed.2023.106574

Cervical cancer is a common disease in women, affecting their lives negatively and often resulting in death. Pap-smear tests are preferred by doctors as the primary tool in the early diagnosis and treatment of the disease. Physicians can be facilitated in the detection of five different categories of cervical cancer and similar cellular disease cases with the Pap-smear image retrieval technology. In this study, an algorithm for retrieval of cervical cancer images using hash coding with a Convolutional Neural Network (CNN) has been implemented. A sensitive deep hashing method combining interpretable mask generation and rotation invariance is proposed for cervical cancer detection. The distinctive features of cervical cancer cells with complex morphological features are focused on with the proposed hybrid dilated convolution spatial attention module and insignificant features are eliminated. Moreover, the loss function of Cauchy rotation invariance in terms of cervical cancer cell target is presented. In this way, the differences in the input samples are revealed, allowing the CNN to learn from different angles and achieve certain rotation invariance. The versatility and performance of the proposed method, as well as the efficiency of the loss function, have been tested on the SIPaKMeD and Mendeley LBC datasets consisting of cervical cancer images. In the experimental results obtained, it is shown that the proposed spatial attention module and rotational invariance deep hashing network generate high performance in cervical cancer image retrieval problems. •An HDC-SAM approach has been proposed to overcome cervical cancer detection problem.•Mask-generated CRI loss added to HDC-SAM focus on the nucleus and cytoplasm region.•A coarse-to-fine retrieval technique is applied instead of the traversal retrieval.•Experimental results have been tested on SIPaKMeD and Mendeley LBC datasets.•Our study has achieved higher retrieval performance in cervical cell image datasets.