What Music Information Retrieval (MIR) is

Every time Shazam recognises a song in three seconds, or a player separates the vocal from the instrumental, or my own work turns a recording into a score, the same thing is happening underneath: Music Information Retrieval (MIR). It is not a tool or a product: it is the research field that makes all of that possible.

What MIR is

Music Information Retrieval is the field of computer science that studies how to extract, organise, and retrieve information from musical signals — whether recorded audio or symbolic score. The underlying question is always the same: given a piece of music, what can be known about it automatically?

Under that question sit very different tasks:

A clarification worth making, because in Spain the acronym causes confusion: the MIR I mean here is not the medical MIR (Spain’s residency programme for junior doctors, Médico Interno Residente). It is a discipline of computer science, with its own research community — the International Society for Music Information Retrieval (ISMIR), founded in 2000 — and its own annual conference, which is the reference event of the field.

Where MIDI, MusicXML, and AMT fit within MIR

If you have already read the earlier articles on this blog about MIDI, MusicXML, or automatic transcription, you probably treated them as separate pieces. They are not: they are part of the same field, with different roles.

MIDI and MusicXML are output formats: when an MIR system solves the transcription task, the result — notes, timings, pitches — has to be represented somehow, readable by machine and by person. MIDI does it with playback or sequencing in mind; MusicXML, with an editable printed score in mind. Neither of the two does MIR by itself: they are the vocabulary in which one of its results gets expressed.

AMT (automatic music transcription) is, within MIR, the specific task that interests me most: converting audio into symbolic representation. It is just one of the tasks in the table above, though it is the one at the centre of my doctoral research.

And artificial intelligence — which I already wrote about in an earlier article — is, today, the dominant technique for solving most of these tasks: neural networks trained on large corpora of annotated audio. But MIR predates the rise of those techniques — ISMIR has been running since 2000 — and not all of its tasks depend on deep learning: tempo recognition, for instance, is solved well with classic signal-processing methods, with no trained model needed.

My research within MIR

My doctoral work focuses on polyphonic automatic music transcription: pieces where several notes — and often several voices — sound at once, the hard case within AMT. I work on piano, taking current state-of-the-art systems as a starting point — Basic Pitch, Onsets & Frames, MT3 among them — and my focus is understanding where and why those systems fail note by note, not just repeating their final accuracy figure. A model that gets 90% of notes right can be failing consistently on the same kind of passage — dense chords, very short notes, sustained pedal — and that distinction matters more than the overall number.

From there I explore representations of the audio signal that are alternatives to the ones those models use, looking at whether a different way of looking at the audio — before it enters the model — helps correct any of those systematic failures. It is ongoing research work: some of the hypotheses I test are confirmed, others are discarded, and both are valid outcomes in the way I document this work in the open.

References

The references this article draws on, and where to read further:

• Müller, M. (2015). Fundamentals of Music Processing: Audio, Analysis, Algorithms, Applications. Springer.
• International Society for Music Information Retrieval. ismir.net — the field’s annual conference and Transactions of the International Society for Music Information Retrieval (TISMIR), its open-access journal.