Automated prediction and analysis of job interview performance: The role of what you say and how you say it

• Published in: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) Vol. 1; pp. 1 - 6
• Main Authors: Naim, Iftekhar, Tanveer, M. Iftekhar, Gildea, Daniel, Hoque, Mohammed Ehsan
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.05.2015
• Subjects: Analytical models; Correlation; Facial features; Feature extraction; Interviews; Predictive models; Videos
• DOI: 10.1109/FG.2015.7163127

Ever wondered why you have been rejected from a job despite being a qualified candidate? What went wrong? In this paper, we provide a computational framework to quantify human behavior in the context of job interviews. We build a model by analyzing 138 recorded interview videos (total duration of 10.5 hours) of 69 internship-seeking students from Massachusetts Institute of Technology (MIT) as they spoke with professional career counselors. Our automated analysis includes facial expressions (e.g., smiles, head gestures), language (e.g., word counts, topic modeling), and prosodic information (e.g., pitch, intonation, pauses) of the interviewees. We derive the ground truth labels by averaging over the ratings of 9 independent judges. Our framework automatically predicts the ratings for interview traits such as excitement, friendliness, and engagement with correlation coefficients of 0.73 or higher, and quantifies the relative importance of prosody, language, and facial expressions. According to our framework, it is recommended to speak more fluently, use less filler words, speak as "we" (vs. "I"), use more unique words, and smile more.