Method of Geometry Reconstruction from a Set of RGB Images Using Differentiable Rendering and Visual Hull

• Published in: Programming and computer software Vol. 51; no. 3; pp. 150 - 159
• Main Authors: Lysykh, A. I., Zhdanov, D. D., Sorokin, M. I.
• Format: Journal Article
• Language: English
• Published: Moscow Pleiades Publishing 01.06.2025; Springer Nature B.V
• Subjects: Accuracy; Algorithms; Approximation; Artificial Intelligence; Color imagery; Complexity; Computer Science; Contour matching; Cultural heritage; Geometry; Machine learning; Neural networks; Operating Systems; Optimization; Ordinary differential equations; Parameter modification; Reconstruction; Rendering; Software Engineering; Software Engineering/Programming and Operating Systems; Topology
• ISSN: 0361-7688; 1608-3261
• DOI: 10.1134/S0361768825700033

The use of differentiable rendering methods is an up-to-date solution to the problem of geometry reconstruction from a set of RGB images without using expensive equipment. The disadvantage of this class of methods is the possible distortions of the geometry that arise during optimization and high computational complexity. Modern differentiable rendering methods calculate and use two types of gradients: silhouette gradients and normal gradients. Most distortions arising in geometry optimization are caused by modifications of parameters associated with silhouette gradients. The paper considers the possibility of increasing the efficiency of geometry reconstruction methods based on the use of differentiable rendering by dividing the reconstruction process into two stages: initialization and optimization. The first stage of reconstruction involves the creation of a visual Hull of the reconstructed object. This stage allows one to automate the process of selecting the original geometry and start the next stage under two conditions: the silhouettes of the object have already been reconstructed from all observation points and the topologies of the reconstructed and true objects are equivalent. The second stage comprises a geometry optimization cycle based on the fulfillment of the above conditions. This cycle consists of four steps: image rendering, loss function calculation, gradient calculation, and geometry optimization. Satisfying the condition of matching the contours of the original and reference geometry eliminates the need to use silhouette gradients. This solution significantly reduces the number of errors that occur during optimization, as well as reduces the computational complexity of the method by eliminating the calculation of the loss function, gradient calculation, and optimization of parameters associated with the silhouettes of objects. The testing and analysis of the results showed an increase in the accuracy of geometry reconstruction with a decrease in mesh resolution and a decrease in the total running time of the method in comparison with similar methods, as well as an up to two-fold increase in the speed of optimization steps.