Interpretable procedural material graph generation via diffusion models from reference images Interpretable procedural material graph generation via diffusion models from reference images

• Published in: The Visual computer Vol. 41; no. 13; pp. 11195 - 11205
• Main Authors: Lv, Xiaoyu, Wu, Zizhao, Xu, Jiamin, Gu, Xiaoling, Zeng, Ming, Xu, Weiwei
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.10.2025; Springer Nature B.V
• Subjects: Artificial Intelligence; Computer Graphics; Computer Science; Datasets; Diffusion models; Graphs; Image Processing and Computer Vision; Methods; Neural networks; Nodes; Optimization techniques; Real time; Semantics; Procedural material; Diffusion models; Texture generation; Procedural models
• ISSN: 0178-2789; 1432-2315
• DOI: 10.1007/s00371-025-04096-0

Procedural materials, generated through algorithmic processes, offer advantages such as resolution independence, editability, and real-time rendering capabilities. Despite these merits, constructing procedural material graphs remains a labor-intensive task. Recent advancements in generative neural networks, particularly diffusion models, have shown promise in automating this process. However, existing methods often struggle with issues related to generation quality, generalization, and interpretability. In this work, we introduce a novel approach for the interpretable generation of procedural material graphs from reference images using diffusion model. Our approach predicts individual nodes in reverse order, leveraging the generative capabilities of diffusion models to achieve significant improvements in generation quality, generalization, and interpretability. Specifically, we employ a two-stage framework: an adapter-based diffusion model predicts procedural nodes, forming an auxiliary graph, which is then refined using a DiffMat-based node parameter optimization method. To validate the effectiveness of our approach, we construct a fine-grained procedural material graph dataset containing extensive data and information defined at the node level. Our code and datasets are available at: https://github.com/InterS23/IPMGG .