Generalized Zero-Shot Chest X-Ray Diagnosis Through Trait-Guided Multi-View Semantic Embedding With Self-Training

• Published in: IEEE transactions on medical imaging Vol. 40; no. 10; pp. 2642 - 2655
• Main Authors: Paul, Angshuman, Shen, Thomas C., Lee, Sungwon, Balachandar, Niranjan, Peng, Yifan, Lu, Zhiyong, Summers, Ronald M.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Annotations; Chest; Computed tomography; Datasets; Deep learning; Diagnosis; Diseases; Embedding; Learning algorithms; Machine Learning; Medical diagnosis; Medical imaging; Multi-view; Phenotype; Radiographs; Radiography; Radiology; self-training; Semantics; Training; Visualization; x-ray; X-ray imaging; X-Rays; zero-shot
• ISSN: 0278-0062; 1558-254X; 1558-254X
• DOI: 10.1109/TMI.2021.3054817

Zero-shot learning (ZSL) is one of the most promising avenues of annotation-efficient machine learning. In the era of deep learning, ZSL techniques have achieved unprecedented success. However, the developments of ZSL methods have taken place mostly for natural images. ZSL for medical images has remained largely unexplored. We design a novel strategy for generalized zero-shot diagnosis of chest radiographs. In doing so, we leverage the potential of multi-view semantic embedding, a useful yet less-explored direction for ZSL. Our design also incorporates a self-training phase to tackle the problem of noisy labels alongside improving the performance for classes not seen during training. Through rigorous experiments, we show that our model trained on one dataset can produce consistent performance across test datasets from different sources including those with very different quality. Comparisons with a number of state-of-the-art techniques show the superiority of the proposed method for generalized zero-shot chest x-ray diagnosis.