Goldsmiths - University of London

ISMS seminars 2007/08

Bruno Gingras

Improved score-performance matching using both structural and temporal information from MIDI recordings (20/11/07)

In order to study score-based music performance, one has to determine the corresponding score note for every performance note, a process called score-performance matching. Since a typical performance may contain thousands of notes, researchers have developed algorithms that automate this procedure. Such algorithms are called matchers. Automated matching is a complex problem due to the use of expressive timing by performers and the presence of notes that are unspecified in the score, such as performance errors and ornaments. Automated matchers typically use performance data extracted from MIDI recordings. In the last two decades, several scholars, such as Puckette & Lippe (1992), Large (1993), and Heijink (1996), have developed such matchers. For the most part, these algorithms use structural information, such as pitch and chronological succession, but do not use timing information. As a result, most matchers cannot deal satisfactorily with ornamented performances or performances that exhibit extreme variations in tempo. In an attempt to solve these issues, the author developed a matcher that relies both on structural information and on a temporal representation of the performance, which is obtained by sequentially tracking local tempo changes on a note-by-note basis and mapping performance events to the corresponding score events. This allows the matcher to generate an accurate match even for heavily ornamented performances. Furthermore, this matcher can identify and categorize all common types of errors and ornaments. Most existing algorithms are designed to find a solution that maximizes the number of matched performance notes, regardless of the perceptual relevance of such an approach. In order to increase the music-theoretical and perceptual validity of its output, the proposed matcher instead favours solutions that preserve the structural and temporal coherence of the individual voices. A comparison with human-made score-performance matches realized by the author (a music theorist) on a corpus of 80 MIDI recordings of organ performances, which were used as ground truth data for this purpose, shows a near-perfect agreement between the solutions found by the matcher and the human matches. Finally, in contrast to existing matchers which focus on piano performance, this matcher is designed to accommodate multi-channel MIDI recordings of performances from keyboard instruments with multiple manuals, such as organ or harpsichord, and could thus potentially be used to study recordings of ensemble performances of MIDI instruments, thereby providing a valuable tool for music performance research, as well as a significant improvement over previous algorithms.

Stephan Baumann

In Search of the Goosebump Factor - A Blueprint for Emotional Music Recommenders (15/11/07)

Music Information Retrieval (MIR) as an interdisciplinary research discipline achieved impressive progress over the last decade. Pandora, Last.fm or MyStrands are successful commercial webservices offering previously unavailable convenience for customers. Although such systems compute personalised recommendations based on relevance feedback on top of content-based, expert-based or community metadata, the embedding of emotional context is still a challenge. In my talk I will sketch a blueprint towards an architecture of an emotional music recommender in order to solve the abovementioned problem. The approach is in its infancy but we have already the core ingredients developed. Lifestream aggregation from Web2.0 platforms and the analysis of blog postings will be aligned with the analysis of song lyrics. Furthermore we propose an open Web2.0 platform in order to collect personal descriptions of "goosebump sensations" when listening to music. This collection will be available to researchers in the field to serve as a common ground for training emotional classifiers.

Ruth Dockwray

Stereophony has significantly impacted the way songs are produced and experienced, enabling the creation of a performance that exists exclusively on the record – this virtual performance can be conceptualised in terms of the ‘sound-box’ (Moore 1992), a four-dimensional virtual space. Within this, sounds can be located through: lateral placement within the stereo field; foreground and background placement due to volume; height according to sound vibration frequency; and time. The period from the mid 1960 to the early 1970s saw the gradual establishment of a ‘normative’ position for sound-sources within the sound-box, which we term the ‘diagonal mix’. In this talk I shall discuss the results of an AHRC-funded project to investigate the establishment of this normative mix. Using 3D rendering software, I will illustrate the diversity of sonic placement in tracks across the years (1966-72) and different styles (psychedelic rock, hard rock and easy listening). The main sound-box configurations, including the disparate mixes evident in the mid1960s and the normative mix will be presented by means of a ‘taxonomy of mixes’. The findings will in turn be related to established theoretical production norms, as in David Gibson’s Art of Mixing (2005) and various reasons for the adoption of the normative mix will be addressed.

Diana Young

Investigations of Bowed String Performance Through Measurement of Violin Bowing Technique (20/09/07)

Virtuosic bowed string performance in many ways exemplifies the incredible potential of human physical performance and expression. A great deal is known about the physics of the violin family and those factors responsible for its sound capabilities. However, there remains much to be discovered about the intricacies of how players control these instruments in order to achieve their characteristic range and nuance of sound. Today, technology offers the ability to study this player control under realistic, unimpeded conditions to lead to greater understanding of these performance skills. Presented here is a method for investigation of bowed string performance that uses a playable hardware measurement system to capture the gestures of right hand violin bowing technique. This measurement system was optimized to be small, lightweight, and portable and was installed on a carbon fiber violin bow and an electric violin to enable study of realistic violin performances. Included in the system are inertial and force sensors, and an electric field position sensor. In order to maximize the applicability of the gesture data provided by this system to related fields of interest, all of the sensors were calibrated in SI units. The gesture data captured by these sensors are recorded together with the audio data from the violin as they are produced by violinists in typical playing scenarios. To explore the potential of the bowing measurement system created, a study of standard bowing techniques, such as detache, martele, and spiccato, was conducted with expert violinist participants. Gesture data from these trials were evaluated and input to a classifier to examine physical distinctions between bowing techniques, as well as between players. Results from this analysis, and their implications on this methodology will be presented. In addition to this examination of bowing techniques, applications of the measurement system for study of bowed string acoustics and digital music instrument performance, with focus on virtual instruments created from physical models, will be discussed.