Multi-channel content based image retrieval method for skin diseases using similarity network fusion and deep community analysis

• Published in: Biomedical signal processing and control Vol. 78; p. 103893
• Main Authors: Wang, Yuheng, Fariah Haq, Nandinee, Cai, Jiayue, Kalia, Sunil, Lui, Harvey, Jane Wang, Z., Lee, Tim K.
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.09.2022
• Subjects: Deep community; Graph analysis; Image retrieval; Multi-channel; Skin cancer; Multi-channel; Graph analysis; Image retrieval; Deep community; Skin cancer
• ISSN: 1746-8094; 1746-8108
• DOI: 10.1016/j.bspc.2022.103893

•A new method for skin lesion image retrieval using multiple input channels is proposed.•Similarity network fusion solves the problem of dimensional imbalance among inputs.•Community-based search technique helps find strongly connected subjects within similarity networks.•The technique is applicable to complex clinical scenarios and produces state-of-the-art performance. Content-based image retrieval (CBIR) could be an efficient diagnostic tool. Physicians could consult a CBIR system before making a diagnosis for a clinical case by retrieving a set of images with similar appearance and pathological diagnosis from a data archive. With access to various imaging modalities, physicians may want to match more than one image modality and non-image information. How to make full use of this diverse information is an important research question. In this paper, we propose a CBIR framework for skin diseases that incorporates multi-sourced information including dermoscopic images, clinical images, and meta information. The proposed framework fuses the multi-sourced features in mutual similarity level; thus, solving severe dimensional bias problems for image and non-image information. We then utilize a graph-based community analysis on similarity networks where similar images are strongly connected and help retrieve similar images with improved performance. Evaluations were carried out using two well-known skin datasets EDRA and ISIC 2019. The carefully designed framework demonstrates a substantial improvement in finding similar cases for different skin diseases with an average precision of 0.836, which is the state-of-the-art performance for retrieving skin disease types. In addition, the proposed framework is also applicable to scenarios with a single typed feature with improved performance. By integrating multi-sourced information from the same patient, the proposed CBIR system could be potentially used in complex clinical scenarios with a trustable performance benefitting from both abundant information and advanced community search technique.