RNIN-VIO: Robust Neural Inertial Navigation Aided Visual-Inertial Odometry in Challenging Scenes

• Published in: 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) pp. 275 - 283
• Main Authors: Chen, Danpeng, Wang, Nan, Xu, Runsen, Xie, Weijian, Bao, Hujun, Zhang, Guofeng
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2021
• Subjects: AR-6DoF; Deep learning; Dynamics; Fuses; Heuristic algorithms; IMU; Inertial navigation; INS; Motion Tracking; Neural networks; SLAM; VIO; Visualization
• DOI: 10.1109/ISMAR52148.2021.00043

In this work, we propose a tightly-coupled EKF framework for visual-inertial odometry with NIN (Neural Inertial Navigation) aided. Traditional VIO systems are fragile in challenging scenes with weak or confusing visual information, such as weak/repeated texture, dynamic environment, fast camera motion with serious motion blur, etc. It is extremely difficult for a vision-based algorithm to handle these problems. So we firstly design a robust deep learning based inertial network (called RNIN), using only IMU measurements as input. RNIN is significantly more robust in challenging scenes than traditional VIO systems. In order to take full advantage of vision-based algorithms in AR/VR areas, we further develop a multi-sensor fusion system RNIN-VIO, which tightly couples the visual, IMU and NIN measurements. Our system performs robustly in extremely challenging conditions, with high precision both in trajectories and AR effects. The experimental results of evaluation on dataset evaluation and online AR demo demonstrate the superiority of the proposed system in robustness and accuracy.