Review of Eye Tracking Metrics Involved in Emotional and Cognitive Processes

• Published in: IEEE reviews in biomedical engineering Vol. 16; pp. 260 - 277
• Main Authors: Skaramagkas, Vasileios, Giannakakis, Giorgos, Ktistakis, Emmanouil, Manousos, Dimitris, Karatzanis, Ioannis, Tachos, Nikolaos, Tripoliti, Evanthia, Marias, Kostas, Fotiadis, Dimitrios I., Tsiknakis, Manolis
• Format: Journal Article
• Language: English
• Published: United States IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Arousal; Artificial Intelligence; Attention; blinks; Cognition; Cognition & reasoning; Cognitive ability; Cognitive processes; cognitive workload; cognitive workload datasets; Computer applications; emotional arousal; emotional arousal datasets; Emotions; Eye; Eye movements; Eye tracking; Eye-Tracking Technology; fixations; gaze; Gaze tracking; Humans; Machine learning; Measurement; Physiology; Pupil; Pupil size; Pupils; Recognition; saccades; smooth pursuit; stress; Task analysis; Tracking; visual attention; Visual perception; Visualization; Workload
• ISSN: 1937-3333; 1941-1189; 1941-1189
• DOI: 10.1109/RBME.2021.3066072

Eye behaviour provides valuable information revealing one's higher cognitive functions and state of affect. Although eye tracking is gaining ground in the research community, it is not yet a popular approach for the detection of emotional and cognitive states. In this paper, we present a review of eye and pupil tracking related metrics (such as gaze, fixations, saccades, blinks, pupil size variation, etc.) utilized towards the detection of emotional and cognitive processes, focusing on visual attention, emotional arousal and cognitive workload. Besides, we investigate their involvement as well as the computational recognition methods employed for the reliable emotional and cognitive assessment. The publicly available datasets employed in relevant research efforts were collected and their specifications and other pertinent details are described. The multimodal approaches which combine eye-tracking features with other modalities (e.g. biosignals), along with artificial intelligence and machine learning techniques were also surveyed in terms of their recognition/classification accuracy. The limitations, current open research problems and prospective future research directions were discussed for the usage of eye-tracking as the primary sensor modality. This study aims to comprehensively present the most robust and significant eye/pupil metrics based on available literature towards the development of a robust emotional or cognitive computational model.