GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh

We introduce GoMAvatar, a novel approach for real-time, memory-efficient, high-quality animatable human modeling. GoMAvatar takes as input a single monocular video to create a digital avatar capable of re-articulation in new poses and real-time rendering from novel view-points, while seamlessly inte...

Full description

Saved in:
Bibliographic Details
Published inProceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 2059 - 2069
Main Authors Wen, Jing, Zhao, Xiaoming, Ren, Zhongzheng, Schwing, Alexander G., Wang, Shenlong
Format Conference Proceeding
LanguageEnglish
Published IEEE 16.06.2024
Subjects
Computational modeling
Deformable models
Pipelines
Rendering (computer graphics)
Solid modeling
Three-dimensional displays
Video on demand
Online AccessGet full text
ISSN1063-6919
DOI10.1109/CVPR52733.2024.00201

Cover

More Information
Summary:We introduce GoMAvatar, a novel approach for real-time, memory-efficient, high-quality animatable human modeling. GoMAvatar takes as input a single monocular video to create a digital avatar capable of re-articulation in new poses and real-time rendering from novel view-points, while seamlessly integrating with rasterization-based graphics pipelines. Central to our method is the Gaussians-on-Mesh (GoM) representation, a hybrid 3D model combining rendering quality and speed of Gaussian splatting with geometry modeling and compatibility of deformable meshes. We assess GoMAvatar on ZJU-MoCap, PeopleSnapshot, and various YouTube videos. GoMAvatar matches or surpasses current monocular human modeling algorithms in rendering quality and significantly outperforms them in computational efficiency (43 FPS) while being memory-efficient (3.63 MB per subject).
ISSN:1063-6919
DOI:10.1109/CVPR52733.2024.00201