OneSLAM to map them all: a generalized approach to SLAM for monocular endoscopic imaging based on tracking any point

• Published in: International journal for computer assisted radiology and surgery Vol. 19; no. 7; pp. 1259 - 1266
• Main Authors: Teufel, Timo, Shu, Hongchao, Soberanis-Mukul, Roger D., Mangulabnan, Jan Emily, Sahu, Manish, Vedula, S. Swaroop, Ishii, Masaru, Hager, Gregory, Taylor, Russell H., Unberath, Mathias
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.07.2024; Springer Nature B.V
• Subjects: Algorithms; Bundle adjustment; Cameras; Computer Imaging; Computer Science; Deep learning; Endoscopy; Health Informatics; Image reconstruction; Imaging; Localization; Medicine; Medicine & Public Health; Navigation systems; Original Article; Pattern Recognition and Graphics; Radiology; Sensors; Simultaneous localization and mapping; Sinuses; Surgery; Tracking; Vision; Computer vision; Monocular SLAM; Arthroscopy; Tracking any point; Image-based navigation; Endoscopy; 3D motion estimation
• ISSN: 1861-6429; 1861-6410; 1861-6429
• DOI: 10.1007/s11548-024-03171-6

Purpose Monocular SLAM algorithms are the key enabling technology for image-based surgical navigation systems for endoscopic procedures. Due to the visual feature scarcity and unique lighting conditions encountered in endoscopy, classical SLAM approaches perform inconsistently. Many of the recent approaches to endoscopic SLAM rely on deep learning models. They show promising results when optimized on singular domains such as arthroscopy, sinus endoscopy, colonoscopy or laparoscopy, but are limited by an inability to generalize to different domains without retraining. Methods To address this generality issue, we propose OneSLAM a monocular SLAM algorithm for surgical endoscopy that works out of the box for several endoscopic domains, including sinus endoscopy, colonoscopy, arthroscopy and laparoscopy. Our pipeline builds upon robust tracking any point (TAP) foundation models to reliably track sparse correspondences across multiple frames and runs local bundle adjustment to jointly optimize camera poses and a sparse 3D reconstruction of the anatomy. Results We compare the performance of our method against three strong baselines previously proposed for monocular SLAM in endoscopy and general scenes. OneSLAM presents better or comparable performance over existing approaches targeted to that specific data in all four tested domains, generalizing across domains without the need for retraining. Conclusion OneSLAM benefits from the convincing performance of TAP foundation models but generalizes to endoscopic sequences of different anatomies all while demonstrating better or comparable performance over domain-specific SLAM approaches. Future research on global loop closure will investigate how to reliably detect loops in endoscopic scenes to reduce accumulated drift and enhance long-term navigation capabilities.