A Basic Tutorial on Novelty and Activation Functions for Music Signal Processing

• Published in: Transactions of the International Society for Music Information Retrieval Vol. 7; no. 1; pp. 179 - 194
• Main Authors: Müller, Meinard, Chiu, Ching-Yu
• Format: Journal Article
• Language: English
• Published: Ubiquity Press 19.09.2024
• Subjects: activation function; audio; beat; downbeat; music; novelty function; onset; structure
• ISSN: 2514-3298; 2514-3298
• DOI: 10.5334/tismir.202

In Music Information Retrieval (MIR), a general goal is to recognize times of novelty within music recordings. This includes estimating structural boundaries through the detection of changes in harmony, tempo, or instrumentation and identifying onsets of note and sound events by capturing changes in the music signal’s energy or spectral content. These tasks leverage novelty functions, which are one-dimensional, time-dependent functions characterized by sharp local maxima that indicate significant musical and acoustical changes. From a given music recording, novelty functions can be derived using a variety of methods, ranging from traditional signal-processing techniques to modern data-driven approaches, where they are often termed “activation functions.” In this tutorial, we explore the concept of novelty functions and some of their essential properties. We discuss methods to enhance these functions and improve their distinctive peak-like structures. These improvements are crucial for simplifying the identification of specific musical events using post-processing methods, from basic peak picking to more sophisticated approaches like periodicity analysis. We also assess novelty functions through commonly used metrics such as precision, recall, and F-measure but with an emphasis on error tolerance. Aimed at Bachelor’s degree and beginning Master’s degree students with basic knowledge of signal processing and mathematics, this tutorial uses illustrative figures to clarify key concepts, thereby broadening its accessibility to a wider MIR audience and enriching their comprehension of this significant subject. Furthermore, Jupyter notebooks, including Python source code for the core algorithms and audio examples that allow for reproducing the tutorial’s figures, are provided at https://github.com/groupmm/edu_novfct.