Toward Label-Efficient Emotion and Sentiment Analysis

• Published in: Proceedings of the IEEE Vol. 111; no. 10; pp. 1159 - 1197
• Main Authors: Zhao, Sicheng, Hong, Xiaopeng, Yang, Jufeng, Zhao, Yanyan, Ding, Guiguang
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.10.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Affective computing; Annotations; artificial emotional intelligence (AEI); Artificial intelligence; Cognition & reasoning; Complexity theory; Data mining; Deep learning; emotion and sentiment analysis (ESA); Emotion recognition; Emotions; Human factors; label-efficient learning; Labeling; Labels; Sentiment analysis; Social factors; Speech recognition; Taxonomy; Training
• ISSN: 0018-9219; 1558-2256
• DOI: 10.1109/JPROC.2023.3309299

Emotion and sentiment play a central role in various human activities, such as perception, decision-making, social interaction, and logical reasoning. Developing artificial emotional intelligence (AEI) for machines is becoming a bottleneck in human-computer interaction. The first step of AEI is to recognize the emotion and sentiment that are conveyed in different affective signals. Traditional supervised emotion and sentiment analysis (ESA) methods, especially deep learning-based ones, usually require large-scale labeled training data. However, due to the essential subjectivity, complexity, uncertainty and ambiguity, and subtlety, collecting such annotations is expensive, time-consuming, and difficult in practice. In this article, we introduce label-efficient ESA from the computational perspective. First, we present a hierarchical taxonomy for label-efficient learning based on the availability of sample labels, emotion categories, and data domains during training. Second, for each of the seven paradigms, i.e., unsupervised, semisupervised, weakly supervised, low-shot, incremental, domain-adaptive, and domain-generalizable ESA, we give the definition, summarize existing methods, and present our views on the quantitative and qualitative comparison. Finally, we provide several promising real-world applications, followed by unsolved challenges and potential future directions.