Endoscopic Vision Augmentation Using Multiscale Bilateral-Weighted Retinex for Robotic Surgery

• Published in: IEEE transactions on medical imaging Vol. 38; no. 12; pp. 2863 - 2874
• Main Authors: Luo, Xiongbiao, Zeng, Hui-Qing, Wan, Ying, Zhang, Xiao-Bin, Du, Yan-Ping, Peters, Terry M.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.12.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: computer-assisted interventions; Endoscopes; endoscopic vision; Endoscopy; Evaluation; Illumination; illumination variation; Image color analysis; Image enhancement; Image restoration; Injury prevention; Laparoscopes; Lighting; Maintenance; minimally invasive surgery; Multiscale analysis; retinex; Retinex (algorithm); Robotic surgery; Robots; Sharpness; Surgery; Vision; Visualization
• ISSN: 0278-0062; 1558-254X; 1558-254X
• DOI: 10.1109/TMI.2019.2916101

Endoscopic vision plays a significant role in minimally invasive surgical procedures. The visibility and maintenance of such direct in situ vision is paramount not only for safety by preventing inadvertent injury but also to improve precision and reduce operating time. Unfortunately, the endoscopic vision is unavoidably degraded due to the illumination variations during surgery. This paper aims to restore or augment such degraded visualization and quantitatively evaluate it during robotic surgery. A multiscale bilateral-weighted retinex method is proposed to remove non-uniform and highly directional illumination and enhance surgical vision, while an objective no-reference image visibility assessment method is defined in terms of sharpness, naturalness, and contrast, to quantitatively and objectively evaluate the endoscopic visualization on surgical video sequences. The methods were validated on surgical data, with the experimental results showing that our method outperforms existent retinex approaches. In particular, the combined visibility was improved from 0.81 to 1.06, while three surgeons generally agreed that the results were restored with much better visibility.