Conveying climate data through immersive sonification and interactive plant art in Unnatural Nature

• Published in: Personal and ubiquitous computing Vol. 28; no. 5; pp. 745 - 761
• Main Authors: Zelada, Eloysa, Çamcı, Anıl
• Format: Journal Article
• Language: English
• Published: London Springer London 01.10.2024; Springer Nature B.V
• Subjects: Audio data; Carbon dioxide; Carbon dioxide concentration; Climate change; Computer Science; Conveying; Design analysis; Ice sheets; Inertial sensing devices; Interactive systems; Mobile Computing; Original Article; Personal Computing; Public participation; Sensors; User Interfaces and Human Computer Interaction; Evaluation; Climate change; Sonification; Spatial audio; Interaction
• ISSN: 1617-4909; 1617-4917
• DOI: 10.1007/s00779-024-01807-7

This paper presents the design, implementation, and evaluation of a sound installation titled “Unnatural Nature,” which explores climate change through data sonification and interactivity. The installation comprises two parts: (1) immersive sonification of climate data on global temperature, CO 2 concentration, Arctic and Antarctic ice sheet mass, and sea level variation, and (2) interactive engagement with physical plants using a custom sensor system. In the first part, the sonification employs a diverse range of sounds that model elements of a natural soundscape presented through higher-order Ambisonics to establish an immersive experience for the audience. In the second part, the sensor system embedded into live plants detects touch and inertial interactions that are mapped to audio and visual events. In this way, the audience is invited to reflect on our interconnectedness with nature and our active involvement in its transformation, as evidenced by the data presented in the first part of the installation. In this paper, we first discuss the implementation of our sonification, describing its technical and aesthetic development. We then offer details of our sensor system, discussing the interactive and audiovisual design of the second part of our installation. Finally, we present the results of a user study conducted to understand participants’ interpretation of and engagement with the sonification and evaluate its effectiveness in conveying climate data. We also offer insights from the first public presentation of the installation, highlighting the visitors’ reflections on their experiences. By merging data sonification, interactive plant art, and immersive audio, “Unnatural Nature” aims to foster environmental awareness, evoke emotional engagement with the subject matter, and encourage participation in climate change discourse.