Test–Retest Reliability and the Role of Task Instructions when Measuring Listening Effort Using a Verbal Response Time Paradigm

• Published in: Seminars in hearing Vol. 44; no. 2; pp. 140 - 154
• Main Authors: Gustafson, Samantha J., Ortiz, Crystal, Nelson, Loren
• Format: Journal Article
• Language: English
• Published: 333 Seventh Avenue, 18th Floor, New York, NY 10001, USA Thieme Medical Publishers, Inc 01.05.2023; Thieme Medical Publishers Inc
• Series: Listening Effort Methodologies: Challenges and Future Directions
• Subjects: Listening; Listening comprehension; Reaction time; Review; Review Article; Test validity and reliability; verbal response time; listening effort; test–retest reliability; task instructions
• ISSN: 0734-0451; 1098-8955
• DOI: 10.1055/s-0043-1767667

Abstract Listening amidst competing noise taxes one's limited cognitive resources, leading to increased listening effort. As such, there is interest in incorporating a reliable test of listening effort into the clinical test battery. One clinically promising method for measuring listening effort is verbal response time (VRT) because it can be obtained using already-established clinical tasks. In order for widespread implementation of the VRT paradigm, a better understanding of the psychometric properties is needed. The purpose of this work was to improve the understanding of the reliability and sensitivity of the VRT listening task. Using within-subject study designs, we completed a pilot study to evaluate the test–retest reliability (Study 1) and the effects of task instructions and listening condition (Study 2). Preliminary results show that the VRT paradigm enjoys good to excellent test–retest reliability and that neither task instructions nor listening condition meaningfully influence VRT once measurement error is accounted for. Future studies should account for measurement error when considering statistically significant versus meaningful effects of experimental parameters when using listening effort tasks.