Attentive pointing in natural scenes correlates with other measures of attention

• Published in: Vision research (Oxford) Vol. 135; pp. 54 - 64
• Main Authors: Jeck, Daniel M., Qin, Michael, Egeth, Howard, Niebur, Ernst
• Format: Journal Article
• Language: English
• Published: England Elsevier Ltd 01.06.2017
• Subjects: Adolescent; Adult; Attention; Attention - physiology; Eye Movements - physiology; Female; Fixation, Ocular - physiology; Fixations; Humans; Interest points; Male; Middle Aged; Natural scenes; Pattern Recognition, Visual - physiology; Pointing; Probability; Probability density estimation; Probability distributions; Salience; Saliency map; Tapping; Visual Perception - physiology; Young Adult; Pointing; Probability density estimation; Probability distributions; Fixations; Tapping; Salience; Attention; Interest points; Natural scenes; Saliency map
• ISSN: 0042-6989; 1878-5646; 1878-5646
• DOI: 10.1016/j.visres.2017.04.001

Finger pointing is a natural human behavior frequently used to draw attention to specific parts of sensory input. Since this pointing behavior is likely preceded and/or accompanied by the deployment of attention by the pointing person, we hypothesize that pointing can be used as a natural means of providing self-reports of attention and, in the case of visual input, visual salience. We here introduce a new method for assessing attentional choice by asking subjects to point to and tap the first place they look at on an image appearing on an electronic tablet screen. Our findings show that the tap data are well-correlated with other measures of attention, including eye fixations and selections of interesting image points, as well as with predictions of a saliency map model. We also develop an analysis method for comparing attentional maps (including fixations, reported points of interest, finger pointing, and computed salience) that takes into account the error in estimating those maps from a finite number of data points. This analysis strengthens our original findings by showing that the measured correlation between attentional maps drawn from identical underlying processes is systematically underestimated. The underestimation is strongest when the number of samples is small but it is always present. Our analysis method is not limited to data from attentional paradigms but, instead, it is broadly applicable to measures of similarity made between counts of multinomial data or probability distributions.