In this workshop we will quickly cover how to load sounds, use beat detection, and then create visualisations in p5js.
P5.js is a JavaScript library that simplifies creative coding in the browser, providing easy-to-use functions for creating interactive graphics and animations. It enables artists, designers, and beginners to express their ideas through code, making visual arts and creative projects accessible to a wider audience.
Programs in P5js are often called sketches.
Click the 
symbol on the right hand side to view the code for the sketch below.
In the IDE you can write code in the text editor, and run it by pressing play. The preview on the right will then display your sketch.
The code above loads in a recording of the Mile End canal, and uses mouse click to play it back. Try running the sketch and clicking on the big red square in the preview to hear the recording.
The FFT (Fast Fourier Transform) is an algorithm used to convert a time-domain signal into its frequency-domain representation. In music informatics, the FFT is important because it allows us to analyze and understand the different frequency components present in music. By analyzing the frequency spectrum, we can extract valuable information about the distribution and intensity of frequencies, aiding tasks such as audio classification, chord recognition, and audio effects processing.
Understanding the FFT (Fast Fourier Transform) is essential for research in AI and music. It enables the analysis of frequency components in audio, aiding tasks like music classification, genre recognition, and content-based recommendation. The FFT facilitates feature extraction, allowing AI models to understand the spectral and timbral characteristics of music. It is also valuable for music generation, composition, and audio effects processing. By leveraging the FFT in AI research, we can develop innovative applications, improve music understanding and creation, and push the boundaries of computational musicology.
The sketch above uses an FFT to split the audio in bass/middle/and treble, visualising the output as some bars. Follow the comments numbered 1. to 5. to make sense of the code.
Try replacing the sound with bee.mp3. Note where the frequency bands are most sensitive to changes (e.g. moving microphone, not bee noises).
Beat tracking is the automatic detection and estimation of the rhythmic structure in music. Beat tracking can provide insight into music genres, aids in music recommendation and synchronization, enhances music production and remixing, and serves as a feature for music analysis and retrieval. It enables a deeper understanding of music, improves recommendations and playlists, facilitates synchronization in multimedia applications, and enhances music production and MIR algorithms.
The example below uses a peak follower to track "beats" from our nature recording, following the peak energy every X number of frames. It uses this to then modify the width of an ellipse.
Explore the code below...
In a second tab go to "https://editor.p5js.org/".
We'll explore how to utilize the Fast Fourier Transform (FFT) and beat tracking to create a fun animation based on your sound walk recordings!
To load in your own audio (or animations) follow the three steps on the image below. 1) click to expand the left panel, 2) press the dropdown next to sketch.js, and 3) click upload file.
For inspiration below is a sketch combining the ideas above. It uses beat tracking to animate the wings of a bird flapping. Note that the beat detection algorithm operates on a frequency range more tuned to bird noises.
Some more helpful resources can be found below: