Silent Speech Recognition as an Alternative Communication Device for Persons With Laryngectomy

• Published in: IEEE/ACM transactions on audio, speech, and language processing Vol. 25; no. 12; pp. 2386 - 2398
• Main Authors: Meltzner, Geoffrey S., Heaton, James T., Deng, Yunbin, De Luca, Gianluca, Roy, Serge H., Kline, Joshua C.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.12.2017
• Subjects: Alaryngeal speech; assistive technology; augmentative and alternative communication; Automatic speech recognition; Biology; Electromyography; EMG; Larynx; Speech recognition; Speech synthesis; subvocal speech recognition; Automatic Speech Recognition; Assistive technology; electromyography; Alaryngeal Speech; Augmentative and Alternative Communication; Subvocal Speech Recognition; EMG
• ISSN: 2329-9290; 2329-9304
• DOI: 10.1109/TASLP.2017.2740000

Each year thousands of individuals require surgical removal of the larynx (voice box) due to trauma or disease, and thereby require an alternative voice source or assistive device to verbally communicate. Although natural voice is lost after laryngectomy, most muscles controlling speech articulation remain intact. Surface electromyographic (sEMG) activity of speech musculature can be recorded from the neck and face, and used for automatic speech recognition to provide speech-to-text or synthesized speech as an alternative means of communication. This is true even when speech is mouthed or spoken in a silent (subvocal) manner, making it an appropriate communication platform after laryngectomy. In this study, eight individuals at least 6 months after total laryngectomy were recorded using eight sEMG sensors on their face (4) and neck (4) while reading phrases constructed from a 2500-word vocabulary. A unique set of phrases were used for training phoneme-based recognition models for each of the 39 commonly used phonemes in English, and the remaining phrases were used for testing word recognition of the models based on phoneme identification from running speech. Word error rates were on average 10.3% for the full eight-sensor set (averaging 9.5% for the top four participants), and 13.6% when reducing the sensor set to four locations per individual (n = 7). This study provides a compelling proof-of-concept for sEMG-based alaryngeal speech recognition, with the strong potential to further improve recognition performance.