Single-view-based high-fidelity three-dimensional reconstruction of leaves

• Published in: Computers and electronics in agriculture Vol. 227; p. 109682
• Main Authors: Wang, Longfei, Yang, Le, Xu, Huiying, Zhu, Xinzhong, Cabrel, Wouladje, Mumanikidzwa, Golden Tendekai, Liu, Xinyu, Jiang, Weijian, Chen, Hao, Jiang, Wenhang
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.12.2024
• Subjects: 3D visualization; agricultural sciences; agriculture; cameras; data collection; Diffusion model; electronics; Gaussian rendering; leaves; phenotype; potatoes; prediction; Single-view reconstruction; texture; Gaussian rendering; Single-view reconstruction; 3D visualization; Diffusion model
• ISSN: 0168-1699
• DOI: 10.1016/j.compag.2024.109682

•Integrated Solution Innovation: Combining explicit point cloud tech with implicit 3D Gaussian rendering to enhance leaf structure fidelity and address background blur.•UV Texture Optimization: Specialized refinement for thin structures, improving geometry and texture quality.•Extensive Validation and Performance Enhancement: Proven effective across various crops and thin objects, enhancing texture details and reconstruction fidelity. In modern agricultural science research, high-fidelity three-dimensional (3D) leaf models are crucial for crop growth analysis. However, reconstructing the complex morphology and texture of leaves from a single viewpoint under varying natural lighting conditions poses a significant challenge. To address the issues associated with this challenge, this paper presents a diffusion model-based method for single-view leaf reconstruction using potato leaves as the experimental subject. In the camera prediction process, the combination of an explicit point cloud generation technique and an implicit 3D Gaussian rendering technique enables the accurate prediction of camera parameters and the effective capture of leaf phenotypic features. In the synthesis of the 3D model of the leaf, a strategy for optimizing the coarse model UV texture is designed with the objective of achieving spatial consistency of texture details. Furthermore, the model was successfully applied to the reconstruction of other crop leaves and lamellar structural objects, and innovatively constructed a leaf reconstruction model with disease characteristics, aiming to provide a reference for the early 3D detection of crop diseases, as well as a reference for the 3D reconstruction and visualization of other lamellar objects. The results demonstrate that the method is effective in reconstructing the morphological structure and texture details of leaves, as well as thin sheet-like structured objects, achieving fast and high-fidelity single-view reconstruction.