Audiovisual Spatial-Audio Analysis by Means of Sound Localization and Imaging: A Multimedia Healthcare Framework in Abdominal Sound Mapping

• Published in: IEEE transactions on multimedia Vol. 18; no. 10; pp. 1969 - 1976
• Main Author: Dimoulas, Charalampos A.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.10.2016; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Abdominal sounds (AS); audiovisual content management; Biomedical monitoring; Localization; Medical diagnostic imaging; Medical services; Monitoring; Multimedia; Multimedia communication; multimedia-based healthcare; multimodal monitoring; Sound; sound imaging; sound localization; spatial audio analysis; Visualization
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2016.2594148

This paper presents the novel sound-field localization and visualization techniques for audiovisual spatial audio analysis, content description, and management. The method focuses on topographic analysis and mapping of gastro-intestinal motility (GIM) through multichannel recording of abdominal sounds (AS). Related research has attempted to study GIM physiology patterns and diagnose specific abnormalities or diseases. In this context, a new AS source localization method is implemented and evaluated. Sound imaging and spatiotemporal mapping utilities are deployed next by means of sound level distribution images. Novel audio description and management is introduced for the analysis automation of prolonged AS recordings, facilitating smart content browsing with audiovisual summarization and highlighting. The proposed modalities can be utilized in multimedia healthcare applications, where multimodal monitoring of GIM and other psychophysiological parameters can be combined for the study of human digestive patterns and their relation to other factors (i.e., subjects' medical history, nutrition, medication, psychological state, and others). The adopted methodology of spatial audio analysis introduces a generic framework that can be efficiently applied to various sound localization and imaging tasks, which are encountered both in bioacoustics and in contemporary multimedia involving multichannel/3D audio.