Hierarchical Forgery Classifier on Multi-Modality Face Forgery Clues

• Published in: IEEE transactions on multimedia Vol. 26; pp. 2894 - 2905
• Main Authors: Liu, Decheng, Zheng, Zeyang, Peng, Chunlei, Wang, Yukai, Wang, Nannan, Gao, Xinbo
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Classifiers; Face forgery detection; Face recognition; Faces; Feature extraction; Forgery; frequency domain feature; Frequency-domain analysis; hierarchical classifier; Hydrofluorocarbons; Image quality; multi-modality face; Near infrared radiation; Representations; Social security; Task analysis; Videos
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2023.3304913

Face forgery detection plays an important role in personal privacy and social security. With the development of adversarial generative models, high-quality forgery images become more and more indistinguishable from real to humans. Existing methods always regard as forgery detection task as the common binary or multi-label classification, and ignore exploring diverse multi-modality forgery image types, e.g. visible light spectrum and near-infrared scenarios. In this article, we propose a novel H ierarchical F orgery C lassifier for M ulti-modality F ace F orgery D etection (HFC-MFFD) , which could effectively learn robust patches-based hybrid domain representation to enhance forgery authentication in multiple modality scenarios. The local hybrid domain representation is designed to explore strong discriminative forgery clues both in the image and frequency domain with the intra-attention mechanism. Furthermore, the specific hierarchical face forgery classifier is designed through the authenticity feedback strategy to integrate diverse discriminative clues. Experimental results on representative multi-modality face forgery datasets demonstrate the superior performance of the proposed HFC-MFFD compared with state-of-the-art algorithms.