Python for music: librosa, music21, and pretty_midi

Python is the reference language in music computing, but it isn’t a single library: it’s an ecosystem of specialised pieces that don’t overlap. When someone asks me “which library do I use to work with music in Python?”, the short answer is always another question: are you looking at recorded sound, a written score, or a MIDI file? Each answer leads to a different library — librosa, music21, or pretty_midi — and this guide is the map of all three: what each one does, when to use each, and how they combine in a real workflow.

The map: three libraries, three planes of music

Before the code, it’s worth fixing the underlying idea. Music can be looked at on at least three different planes, and each library lives on one:

They aren’t alternatives to one another: they’re complementary. A typical automatic music transcription workflow starts from audio (librosa to explore it), moves through a note-by-note event representation (pretty_midi to handle the result in MIDI), and can end up as a readable score (music21 to export it to MusicXML).

pip install librosa music21 pretty_midi

librosa: audio as signal

librosa works on the waveform: the audio as captured by a microphone, before knowing what notes it contains. It loads a file, computes spectrograms, extracts spectral features (MFCCs, chroma), and detects onsets and tempo.

import librosa

y, sr = librosa.load("extracto.mp3")
tempo, _ = librosa.beat.beat_track(y=y, sr=sr)

print(f"Frecuencia de muestreo: {sr} Hz")
print(f"Tempo estimado: {tempo:.1f} BPM")

librosa.load returns the waveform as a numpy array (y) and its sample rate (sr); from there, the rest of the library — spectrograms, MFCCs, beat detection — operates on those two values. I already dedicated a full tutorial to this library, with the step by step of computing a real spectrogram from a bagpipe recording: How to compute a spectrogram with librosa. Here it’s enough to place it on the map: it’s the entry point when the starting point is audio, not score or MIDI.

music21: the score as symbol

music21 is MIT’s Python library for computational musicology. It doesn’t look at the audio signal: it looks at the score, on its symbolic plane — notes, measures, key, fingering, dynamics. It reads and writes MusicXML, MIDI, and other notation formats, which makes it the usual tool for musical corpus analysis and computational music theory in code.

from music21 import corpus, key

# Corpus de ejemplo que trae music21 (un coral de Bach)
sinfonia = corpus.parse("bach/bwv66.6")

# Estimación de tonalidad a partir del análisis armónico
tonalidad = sinfonia.analyze("key")
print(f"Tonalidad estimada: {tonalidad}")

for nota in sinfonia.flatten().notes[:5]:
    print(nota.nameWithOctave, nota.quarterLength)

corpus.parse loads one of the sample scores shipped with music21 — Bach chorales among them; analyze("key") applies a key-estimation algorithm to the piece’s notes; flatten().notes walks through the notes one by one with their pitch (nameWithOctave) and duration (quarterLength). None of this touches audio: all the analysis happens on the already-encoded score.

That ability to read and write MusicXML is what I use to compare, with real data, what notation information survives exporting the same musical phrase to different formats: MIDI vs MusicXML in Python: what survives and what doesn’t.

pretty_midi: MIDI as manageable objects

pretty_midi closes the map: it works with MIDI files at the program level. It loads a .mid file and exposes its notes, instruments, control changes, and tempo changes as easy-to-walk Python objects, instead of raw bytes from the protocol.

import pretty_midi

pm = pretty_midi.PrettyMIDI("ejemplo.mid")

for instrumento in pm.instruments:
    print(f"Instrumento: {instrumento.program}")
    for nota in instrumento.notes[:5]:
        print(
            f"  pitch={nota.pitch} "
            f"start={nota.start:.2f} end={nota.end:.2f} "
            f"velocity={nota.velocity}"
        )

Each note carries its MIDI pitch (pitch, 0-127), its start and end in seconds — not in measures or note values, as in music21 — and its intensity (velocity). That’s the underlying difference with music21: pretty_midi describes a timed performance, not a score with measures and fingering. It’s the usual tool for generating the output MIDI in automatic music transcription systems and for evaluating a transcription by comparing note-by-note events against a reference.

When to use each one

The starting question decides the library:

• Do I want to analyse or transform recorded audio? librosa: spectrograms, MFCCs, onset and tempo detection, feature extraction.
• Do I want to work with the score — key, measures, fingering — or export to MusicXML? music21: harmonic analysis, notation generation, computational musicology.
• Do I want to read, generate, or edit a MIDI file at the note, instrument, or tempo level? pretty_midi: the abstraction layer over the MIDI protocol that makes it manageable to work with in code.

In practice you don’t pick just one: in the same research notebook it’s common to use librosa to explore the input audio, pretty_midi to handle the output of a transcription model, and music21 to turn that output into a readable score. All three work well with Jupyter, the usual environment for this kind of exploratory work: load, transform, and see the result in the same document without setting up a separate program.

Where this fits in MIR

The three libraries — and Python itself as the de facto language — are working tools within the field of Music Information Retrieval (MIR): the study of how to extract, organise, and retrieve information from musical signals, whether in audio or symbolic form. librosa covers the signal side; music21 and pretty_midi, the symbolic side — score and MIDI, respectively. None of them does MIR on its own: they’re the vocabulary in which its tasks are implemented, from chord recognition to automatic music transcription.

References

The references this article draws on, and where to read further:

• McFee, B., Raffel, C., Liang, D., Ellis, D. P. W., McVicar, M., Battenberg, E., & Nieto, O. (2015). librosa: Audio and Music Signal Analysis in Python. Proceedings of the 14th Python in Science Conference, 18–24.
• Cuthbert, M. S., & Ariza, C. (2010). music21: A Toolkit for Computer-Aided Musicology and Symbolic Music Data. Proceedings of the 11th International Society for Music Information Retrieval Conference (ISMIR).
• Official music21 documentation.
• Raffel, C., & Ellis, D. P. W. pretty_midi: A Python Library for Handling MIDI Data — official repository and documentation.
• Official librosa documentation.