Lunar Boom Learning
Chapter 1
Chapter 1
Turning Music Into Data
From vibrating air to the representations AI music models can analyze and generate
The book begins with representation. Before a model can learn from music, sound must be measured, organized, compressed, or translated into a numerical form. This chapter follows that path from physical vibration to waveform samples, spectrograms, symbolic notes, learned vectors, and discrete audio tokens.
What you’ll learn
- Trace sound from a physical vibration through a microphone to numerical audio samples.
- Explain amplitude, frequency, sample rate, bit depth, mono, and stereo in plain language.
- Distinguish raw waveform audio from time-frequency and symbolic music representations.
- Compare what waveforms, spectrograms, and MIDI preserve, omit, and make easier for a model to process.
- Explain how audio embeddings map clips into vector spaces where selected forms of similarity can be measured.
- Interpret cosine similarity, nearest-neighbor retrieval, clustering, and audio-text alignment as uses of embedding spaces.
- Describe the encoder, quantizer, and decoder stages of a neural audio codec.
- Explain how discrete audio tokens make autoregressive music generation possible while introducing bitrate, fidelity, and representation trade-offs.
Questions to carry with you
How does a physical sound become numbers stored in a computer?
What does sample rate measure, and how is it different from bit depth?
What information is directly available in a waveform?
Why can the same music be represented as a waveform, spectrogram, or MIDI sequence?
What does an embedding preserve, and what might it discard?
Why does similarity depend on the model and data used to create the embedding space?
How does a neural audio codec turn continuous audio into discrete tokens?
What is gained and lost when music is compressed into a lower-rate representation?
Sections
Section 1.1 · 11 min
What Is Sound?
See how vibrations become sound waves, how microphones turn those waves into signals, and how sample rate, bit depth, and channels shape a digital recording.
Section 1.2 · 12 min
Three Ways to Represent Music
Compare waveform audio, spectrograms, and MIDI, and learn why each representation reveals a different part of music.
Section 1.3 · 10 min
Audio Embeddings
Learn how models turn audio into vectors, compare sounds inside an embedding space, and connect music with written descriptions.
Section 1.4 · 12 min
Audio Tokens and Neural Codecs
Learn how neural codecs compress audio, how quantization creates discrete token streams, and why music models often generate tokens instead of individual waveform samples.
Chapter recap
Chapter takeaway
Chapter 1 establishes the representational foundation for the rest of the book. It begins with sound as a physical pressure wave, shows how recording systems store that signal as samples, compares waveform, spectrogram, and MIDI representations, and then introduces learned embeddings and discrete codec tokens used by modern AI music systems.
A computer does not receive music as a human concept. It receives numbers organized according to a representation.
Digital audio stores repeated measurements of a signal across one or more channels.
Sample rate controls how often the signal is measured, while bit depth controls the number of values available for each measurement.
Waveforms preserve detailed signal timing, spectrograms expose time-frequency structure, and MIDI represents symbolic musical events rather than recorded sound.
No representation is universally best. Each preserves selected information and makes different tasks easier or harder.
Audio embeddings compress clips into vectors whose geometry reflects the objectives and data used to train the encoder.
Embedding similarity is useful for retrieval and conditioning, but it does not mean that every musical property is preserved.
Neural audio codecs compress waveforms through an encoder, quantizer, and decoder.
Discrete codec tokens allow language-model-style generation of audio while introducing trade-offs among token rate, bitrate, fidelity, sequence length, and compute.
The choice of representation shapes the training examples that can be assembled in Chapter 2.
Chapter 2, Building the Musical Training Dataset, uses these representations to define training examples, collect and clean audio, attach metadata and captions, balance the dataset, and document licensing, consent, and provenance.
Check your understanding
Chapter 1 Assessment
Review the path from physical sound to digital samples, representation choices, learned embedding spaces, and discrete neural audio tokens.