Robust 3D Self-Portraits in Seconds

• Published in: Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 1341 - 1350
• Main Authors: Li, Zhe, Yu, Tao, Pan, Chuanyu, Zheng, Zerong, Liu, Yebin
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2020
• Subjects: Bundle adjustment; Deformable models; Image reconstruction; Robustness; Shape; Strain
• ISSN: 1063-6919
• DOI: 10.1109/CVPR42600.2020.00142

In this paper, we propose an efficient method for robust 3D self-portraits using a single RGBD camera. Benefiting from the proposed PIFusion and lightweight bundle adjustment algorithm, our method can generate detailed 3D self-portraits in seconds and shows the ability to handle subjects wearing extremely loose clothes. To achieve highly efficient and robust reconstruction, we propose PIFusion, which combines learning-based 3D recovery with volumetric non-rigid fusion to generate accurate sparse partial scans of the subject. Moreover, a non-rigid volumetric deformation method is proposed to continuously refine the learned shape prior. Finally, a lightweight bundle adjustment algorithm is proposed to guarantee that all the partial scans can not only ``loop'' with each other but also remain consistent with the selected live key observations. The results and experiments show that the proposed method achieves more robust and efficient 3D self-portraits compared with state-of-the-art methods.