Declan Kolakowski

Proportional Visualisations of Music

Added on by Declan Kolakowski.

A long neglected facet of musical analysis is the use of "music as data"; the use statistical, scientific, economic and graphical data analysis to represent musical formalisms in a data specific way and thus enhance one's understanding of composition and music.

My first foray into this has been creating a proportional note visualiser, which you can try out here if you wish, (hopefully it will available to use with user upload files soonish). Currently it is loaded up with three different pieces of music: Bach, Cello Suite in G Major; Paganini, Caprice No.1 and Schoenberg, Mässige from Drei Klavierstücke.

Typical out example. (Bach Prelude)

Typical out example. (Bach Prelude)

The program works by taking a .mid file and enumerating cumulatively the "noteOn" events into an 87 place array - 87 being the number of MIDI events that signify pitches. These are then sorted from largest to smallest and drawn as a set of concentric circles with the radius of each circle directly proportional to the frequency of each pitch that appears in the piece. A user can then zoom in and out of this "pitch space", as if they were viewing a map, to get a sense of the sound area that a piece occupies.

The proportionality of the pieces is also absolute. For example; if you view the Paganini and then the Schoenberg you will notice that the Schoenberg appears much smaller initially, this is because the actual mass of notes is less. There really are a lot fewer pitches in the Schoenberg. This means it also relatively easy to derive a set of proportional relationships between pieces of music in cross analysis.

In what other ways does a visual representation of music data aid analysis? In the Bach we can see that one single pitch (I'm not being specific here as it is a problem I want to address below) occurs a great many more times than any other pitch, making up 17% of the piece's pitch content in total. We can also infer, by the huge proportional gap between the next most frequently played pitch, that this piece of Bach remains in closely related key centres throughout and thus get a simple overview of the music's structure. Now if we look at the Schoenberg:

This yields some interesting insights in comparison to the Bach, and to Schoenberg's method as a whole. Immediately the proportional relationship between pitches is much more even. With a roughly predictable pattern of relations spreading outwards. We might infer that the tonal centre in Schoenberg's work are much less clear (obviously if you have any knowledge of Schoenberg's music) BUT we also see a clear hierarchy of cumulative pitch content i.e. the relationship between the most numerous and least numerous pitches is quite wide. Considering that these works were meant to showcase serialism - which implies an even-ness of tonal content - we can see immediately from looking at a diagrammatic representation of the piece that this is not true.

Bear in mind. That the pitches in these diagrams are discrete; an A4 is enumerated separately from A3. Whereas in serialist composition pitches are viewed as continuous and therefore a B3 is interchangeable with a B4 when calculating contributions to a tone row. However, even with this observation taken into account there are gaps in the percentage contribution that pitches have. Using a graph like this we can quickly calculate that F# appears almost 9.25% of the time which is well above the 8.3% (100/12) of the piece it should make up to create an even tonal execution of the serialist technique. 

Problems: one glaring issue with this software is that MIDI files are absolute in how they represent data. Music is a fluid phenomena; thus a big problem arises when encountering tuning. In the Bach, my program says that the most numerous pitch is A4, which is completely wrong, but it comes out with this result because the MIDI file I used had adjusted the pitch content for Baroque tuning systems. So, even though it's a piece of music "In G" the most numerous pitch comes out as an A.

Expansion: There are several areas in which a program like this could be expanded. One interesting expansion would be mapping color to rhythmic content. Because RGB color values represent a three dimensional vector that has a distinctive visual representation in almost all computer programs, one could easily map this multiple dimensions of musical content to this; rhythm is just one interesting execution of this. For example; it would be very interesting to see how frequency of a pitch correlates with relative speed. I would expect that a more numerous pitch would generally have a higher average note speed. By potentially building in that functionality I can confirm or deny that theory very easily without having to go through the painstaking task of counting notes in a score. Furthermore, it is a solution that scales; with a fairly simple script I could set up this program to analyse hundreds of scores an hour and come out with the average data for an entire composer's work, or a group of composers, or even a genre of music.

In essence, representing musical material like this allows you to understand musical form and content free of its temporal and specific forms. The temporal nature of music means that analysing it in a structurally holistic manner requires holding the entire piece in your head, a feat which means musicians, more often than not, simply refer to a musical score, which is as much an abstraction of musical content as one of these graphs. These diagrams simply allow for that information to be displayed in a more efficient manner

As a qualitative judgement, it is irrelevant whether Schoenberg kept to the serialist technique when composing, in the end music is a form of entertainment that often sacrifices its abstract formalistic constraints in the name of being artistically satisfying. But, the fact that an observation like that can be reached so easily through music visualisation techniques is, I think, a gap in musical software and analysis that we are not addressing.