Research on Virtual Try-on via Style Transformation and Local Rendering

• Published in: Taiyuan li gong da xue xue bao = Journal of Taiyuan University of Technology Vol. 52; no. 1; pp. 98 - 104
• Main Authors: Jun XU, Yuanyuan PU, Dan XU, Zhengpeng ZHAO, Wenhua QIAN, Hao WU, Qiuxia YANG
• Format: Journal Article
• Language: Chinese; English
• Published: Editorial Office of Journal of Taiyuan University of Technology 01.01.2021
• Subjects: image fidelity; local rendering; semantic segmentation; style transformation; virtual try-on
• ISSN: 1007-9432
• DOI: 10.16355/j.cnki.issn1007-9432tyut.2021.01.013

At present, people mainly rely on traditional try-on to observe the effect of clothes, and the research on how to make clothe virtual try-on on the image of characters through the computer has gradually become a hot topic. The results of the existing methods of virtual fitting show that the details of characters are lost and the features of non-fitting areas are changed, which affect the appearance of clothes and the judgment of tryers on clothes. Aiming at the above problems, a try on method combining style transformation and local rendering was proposed. First of all, in order to ensure the accuracy of try-on, a strategy of pixel-level semantic segmentation of human image was proposed to find the location of clothing area, and at the same time provide the implementation conditions for local rendering; Second, in order to realize try-on between different clothing styles, a learnable style transformation module was constructed; Again, in order to make the clothing better fit the body shape of the tryer, a learnable clothing deformation module was proposed; Finally, in order to retain the original details of the characters and maintain the appearance of the fitting image, a local rendering strategy was proposed, which only renders the clothing area to ensure that the details of non-fitting area will not be lost. The experimental results show that the method has a significant effect in maintaining details such as characters’ arms, hands, and other aspects. In addition to fitting area, the feature information of non-fitting area is better preserved, and the image is more realistic and of fidelity after fitting. This method was applied to the try-on task for the first time, which solves the problem that try-on cannot be done correctly when the pose is complex, so that it can meet the requirements of multi-pose try-on in daily scenarios, and also provide new research ideas for virtual try-on field.