autoSMIM: Automatic Superpixel-Based Masked Image Modeling for Skin Lesion Segmentation

• Published in: IEEE transactions on medical imaging Vol. 42; no. 12; pp. 3501 - 3511
• Main Authors: Wang, Zhonghua, Lyu, Junyan, Tang, Xiaoying
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.12.2023
• Subjects: Algorithms; Bayes Theorem; Data models; Dermoscopy - methods; Feature extraction; Humans; Image Processing, Computer-Assisted - methods; Image segmentation; Lesions; masked image modeling; self-supervised learning; Semantics; Skin; Skin Diseases - diagnostic imaging; Skin lesion segmentation; Skin Neoplasms; superpixel; Task analysis
• ISSN: 0278-0062; 1558-254X; 1558-254X
• DOI: 10.1109/TMI.2023.3290700

Skin lesion segmentation from dermoscopic images plays a vital role in early diagnoses and prognoses of various skin diseases. However, it is a challenging task due to the large variability of skin lesions and their blurry boundaries. Moreover, most existing skin lesion datasets are designed for disease classification, with relatively fewer segmentation labels having been provided. To address these issues, we propose a novel automatic superpixel-based masked image modeling method, named autoSMIM, in a self-supervised setting for skin lesion segmentation. It explores implicit image features from abundant unlabeled dermoscopic images. autoSMIM begins with restoring an input image with randomly masked superpixels. The policy of generating and masking superpixels is then updated via a novel proxy task through Bayesian Optimization. The optimal policy is subsequently used for training a new masked image modeling model. Finally, we finetune such a model on the downstream skin lesion segmentation task. Extensive experiments are conducted on three skin lesion segmentation datasets, including ISIC 2016, ISIC 2017, and ISIC 2018. Ablation studies demonstrate the effectiveness of superpixel-based masked image modeling and establish the adaptability of autoSMIM. Comparisons with state-of-the-art methods show the superiority of our proposed autoSMIM. The source code is available at https://github.com/Wzhjerry/autoSMIM