Transformer-based Automated Segmentation of the Median Nerve in Ultrasound Videos of Wrist-to-Elbow Region

• Published in: IEEE transactions on ultrasonics, ferroelectrics, and frequency control Vol. 71; no. 1; p. 1
• Main Authors: Gujarati, Karan R., Bathala, Lokesh, Venkatesh, Vaddadi, Mathew, Raji Susan, Yalavarthy, Phaneendra K.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Annotations; Automation; Cross-sectional Area; Elbow; Elbow (anatomy); Electric Power Supplies; Entrapment; Forearm; Humans; Image Processing, Computer-Assisted; Image segmentation; Matching; Median (statistics); Median nerve; Median Nerve - diagnostic imaging; Median Nerve Segmentation; Segmentation; Spatial data; Task analysis; Transformers; Ultrasonic imaging; Ultrasonography; Ultrasound Video; Video; Videos; Vision Transformer; Visualization; Wrist; Wrist - diagnostic imaging
• ISSN: 0885-3010; 1525-8955; 1525-8955
• DOI: 10.1109/TUFFC.2023.3330539

Segmenting the median nerve is essential for identifying nerve entrapment syndromes, guiding surgical planning and interventions, and furthering understanding of nerve anatomy. This study aims to develop an automated tool that can assist clinicians in localizing and segmenting the median nerve from the wrist, mid-forearm, and elbow in ultrasound videos. This is the first fully automated single deep-learning model for accurate segmentation of the median nerve from the wrist to the elbow in ultrasound videos, along with the computation of the cross-sectional area of the nerve. The visual transformer architecture, which was originally proposed to detect and classify 41 classes in YouTube videos, was modified to predict the median nerve in every frame of ultrasound videos. This is achieved by modifying the bounding box sequence matching block of the visual transformer. The median nerve segmentation is a binary class prediction, and the entire bipartite matching sequence is eliminated, enabling a direct comparison of the prediction with expert annotation in a frame-by-frame fashion. Model training, validation, and testing were performed on a dataset comprising ultrasound videos collected from 100 subjects, which were partitioned into 80, 10, and 10 subjects, respectively. The proposed model was compared with U-Net, U-Net++, Siam U-Net, Attention U-Net, LSTM U-Net, and Trans U-Net. The proposed transfomer-based model effectively leveraged the temporal and spatial information present in ultrasound video frames and efficiently segmented the median nerve with an average dice similarity coefficient of approximately 94% at the wrist and 84% in the entire forearm region.