An Acoustical and Lexical Machine-Learning Pipeline to Identify Connectional Silences

• Published in: Journal of palliative medicine Vol. 26; no. 12; p. 1627
• Main Authors: Matt, Jeremy E, Rizzo, Donna M, Javed, Ali, Eppstein, Margaret J, Manukyan, Viktoria, Gramling, Cailin, Dewoolkar, Advik Mandar, Gramling, Robert
• Format: Journal Article
• Language: English
• Published: United States 01.12.2023
• Subjects: Algorithms; Cohort Studies; Communication; Humans; Machine Learning; Natural Language Processing; machine-learning; silence; human connection; conversation analysis; artificial intelligence
• ISSN: 1557-7740
• DOI: 10.1089/jpm.2023.0087

Developing scalable methods for conversation analytics is essential for health care communication science and quality improvement. To assess the feasibility of automating the identification of a conversational feature, which is associated with important patient outcomes. Using audio recordings from the Palliative Care Communication Research Initiative cohort study, we develop and test an automated measurement pipeline comprising three machine-learning (ML) tools-a random forest algorithm and a custom convolutional neural network that operate in parallel on audio recordings, and subsequently a natural language processing algorithm that uses brief excerpts of automated speech-to-text transcripts. Our ML pipeline identified with an overall sensitivity of 84% and specificity of 92%. For and subtypes, we observed sensitivities of 68% and 67%, and specificities of 95% and 97%, respectively. These findings support the capacity for coordinated and complementary ML methods to fully automate the identification of in natural hospital-based clinical conversations.