Swarm modulation: An algorithm for real-time spectral transformation

• Published in: 2015 IEEE Games Entertainment Media Conference (GEM) pp. 1 - 8
• Main Authors: Janzen, Ryan, Mann, Steve
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2015
• Subjects: Acoustics; Auditory system; Frequency control; Frequency modulation; Oscillators; Real-time systems
• DOI: 10.1109/GEM.2015.7377214

A novel class of modulation is introduced, for frequency transformation and spectral synthesis in real-time. Swarm modulation has potential applications to enhance human hearing with extended frequency ranges, in medical diagnostics for electrocardiogram (ECG), electroencephalogram (EEG) and other medical signals, for RADAR analysis, for user-interface sonification, and for sound synthesis or non-synthetic sound transformation. Swarm modulation is a new way to transform signals, and is demonstrated for transforming subsonic and ultrasonic sound into the audible range of human hearing. Swarm modulation is based on the principle of phase-incoherent frequency-roaming oscillation. Features in the frequency-time plane are reconstructed via a time-varying process, controllable with instantaneous zero-latency reaction time to new information. Swarm modulation allows prioritization of salient output spectral features for efficient processing, and overcomes cyclic beating patterns when Fourier and wavelet-based methods are applied in a stationary manner. Swarm modulation can flexibly re-map sound when a user expressively touches physical matter creating vibration. By detecting subsonic, sonic and ultrasonic vibrations, we can add to materials a rich acoustic user-feedback that can be adjusted to sound like a bell, xylophone, dull piece of wood, or a variety of other objects, in real-time. By dynamically controlling the output sound spectrum depending on the input spectrum, simultaneously with a continuous and low-latency temporal response, the system imitates the physicality of touching a real object. Applied in control panels and expressive control surfaces, swarm modulation can create realistic sonic feedback, for human head-up operation of controls in critical applications.