Sources and citations

Office for Human Research Protections · United States Department of Health and Human Services · regulation

Official United States regulations for covered human-subject research. Applicability depends on funding, institution, jurisdiction, and study characteristics.

Junmin Gong, Sean Zhao, Sen Wang, Shengyuan Xu, and Jing Guo · arXiv · 2025 · paper

Primary research on a compressed diffusion and transformer music foundation model with multi-minute generation, temporal repainting, lyric editing, remixing, and specialized adaptation tasks.

Azalea Gui, Hannes Gamper, Sebastian Braun, and Dimitra Emmanouilidou · IEEE ICASSP · 2024 · paper

Music-specific analysis of FAD sensitivity to sample count, embedding choice, reference-set quality, and perceptual correlation.

Azalea Gui, Hannes Gamper, Sebastian Braun, and Dimitra Emmanouilidou · IEEE ICASSP · 2024 · paper

Music-specific study of FAD sample-size bias, audio-embedding choice, reference-set quality, and perceptual correlation.

NIST · National Institute of Standards and Technology · government-guidance

Official guidance for deciding whether an AI system achieves its intended purpose and whether development or deployment should proceed.

NIST · National Institute of Standards and Technology · government-guidance

Official guidance on post-deployment monitoring, external feedback, performance degradation, unexpected behavior, incidents, and risk response.

NIST · National Institute of Standards and Technology · government-guidance

Official guidance for documenting intended purpose, users, deployment context, expectations, and impacts before selecting evaluation criteria.

NIST · National Institute of Standards and Technology · government-guidance

Official guidance stating that evaluation approaches and metrics depend on purpose, audience, needs, and context, and should test whether a system is fit for purpose.

NIST · National Institute of Standards and Technology · government-guidance

Official guidance emphasising that evaluation methods and metrics should be selected according to purpose, audience, context, and intended use.

NIST · National Institute of Standards and Technology · government-guidance

Official guidance on selecting evaluation methods according to purpose, audience, context, intended use, and decision requirements.

ITU Radiocommunication Sector · International Telecommunication Union · 2023 · standard

Official recommendation specifying algorithms for programme loudness and true-peak measurement.

AWS · Amazon Web Services · documentation

Official explanation of message invisibility during processing and redelivery when processing is not acknowledged before the timeout.

Lyor Cohen and Toni Reid · YouTube · 2023 · official-product-announcement

Official announcement of Dream Track, developed with participating artists who chose to collaborate and initially offered to selected creators.

Avery Li-Chun Wang · International Society for Music Information Retrieval · 2003 · paper

Foundational description of a scalable spectral-landmark audio-fingerprinting system for identifying recordings from short, noisy, or compressed excerpts.

Liwei Lin, Gus Xia, Yixiao Zhang, and Junyan Jiang · arXiv · 2024 · paper

Primary research adapting MusicGen for inpainting, score-conditioned arrangement, and track-conditioned refinement.

NIST · National Institute of Standards and Technology · 2024 · government-report

Official cross-sector guidance for managing generative-AI risks, including governance, pre-deployment testing, content provenance, monitoring, and incident disclosure.

NIST · National Institute of Standards and Technology · 2024 · government-guidance

Official cross-sector guidance for identifying, measuring, managing, and governing generative-AI risks across development and deployment.

AWS · Amazon Web Services · documentation

Official example of queued asynchronous model serving for long processing times and large payloads, including scale-to-zero support.

Meta AI · Meta AudioCraft · documentation

Official MusicGen generation API combining the required model components and exposing text and melody generation methods.

Felix Kreuk et al. · International Conference on Learning Representations · 2023 · paper

Original AudioGen paper using objective and subjective evaluation for text-conditioned audio generation.

Haohe Liu et al. · International Conference on Machine Learning · 2023 · paper

Original AudioLDM paper using CLAP representations and objective and subjective text-to-audio evaluation.

Deja Kamil, Sanchez Ariadna, Roth Julian, and Cotescu Marius · arXiv · 2022 · preprint

Studies automatic speaker-similarity prediction using speaker embeddings and comparison with human perceptual ratings.

AWS · Amazon Web Services · documentation

Official guidance for autoscaling queued inference endpoints, including scaling to zero.

Florian Grötschla, Ahmet Solak, Luca A. Lanzendörfer, and Roger Wattenhofer · arXiv · 2025 · paper

Large human-preference benchmark using generated songs from multiple systems, illustrating the difficulty of aligning automated music metrics with human judgement.

C2PA · Coalition for Content Provenance and Authenticity · 2026 · technical-guidance

Official explanation of Content Credentials, provenance, asset bindings, ingredients, AI disclosure, durable credentials, trust, privacy, and the limits of provenance assertions.

C2PA · Coalition for Content Provenance and Authenticity · 2026 · technical-standard

Current technical specification for cryptographically bound Content Credentials, including audio actions, ingredients, AI models, datasets, prompts, seeds, and generative-system asset types.

C2PA · Coalition for Content Provenance and Authenticity · 2026 · technical-standard

Current technical specification for cryptographically bound Content Credentials, asset actions, ingredients, audit logs, and AI and ML asset types.

C2PA · Coalition for Content Provenance and Authenticity · 2026 · technical-standard

Current technical specification for Content Credentials, asset actions, ingredients, AI and ML asset types, and provenance assertions.

FinOps Foundation · FinOps Foundation · industry-guidance

Official FinOps guidance connecting cloud costs to product units, engineering decisions, pricing, and business outcomes.

Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang · IEEE ICASSP · 2023 · paper

Original CLAP paper describing contrastive audio and language encoders trained to produce a joint embedding space.

Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang · IEEE ICASSP · 2023 · paper

Original CLAP paper describing contrastive audio and text encoders trained to produce a shared multimodal representation.

FinOps Foundation · FinOps Foundation · industry-guidance

Official FinOps guidance for assigning cloud costs to users, products, teams, projects, and other accountable groupings.

Kubernetes contributors · Kubernetes · documentation

Official example in which parallel workers claim units of work from a task queue.

Flavio Ribeiro, Dinei Florêncio, Cha Zhang, and Mike Seltzer · IEEE ICASSP · 2011 · paper

Original crowdsourced subjective-audio study proposing methods for identifying inaccurate responses and obtaining repeatable quality judgements outside a laboratory.

Timnit Gebru et al. · Communications of the ACM · 2021 · paper

Original proposal for dataset documentation covering motivation, composition, collection, preprocessing, uses, distribution, maintenance, and limitations.

Timnit Gebru et al. · Communications of the ACM · 2021 · paper

Original proposal for documenting dataset motivation, composition, collection, processing, uses, maintenance, and limitations.

AWS · Amazon Web Services · documentation

Official guidance for time-limited upload and download access without exposing general storage credentials.

Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck · Interspeech · 2020 · paper

Primary paper introducing a widely used speaker-embedding architecture for automatic speaker verification.

Meta AI · Meta AudioCraft · documentation

Official AudioCraft documentation for training and evaluating EnCodec.

Nicholas Carlini et al. · USENIX Security Symposium · 2021 · paper

Foundational demonstration that model querying can recover identifiable training examples. Included for the distinction between memorisation and extractability.

Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi · Interspeech · 2019 · paper

Original Fréchet Audio Distance paper comparing distribution-level audio embeddings and human perception of audio distortions.

Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi · Interspeech · 2019 · paper

Original Fréchet Audio Distance paper introducing distributional comparison of audio embeddings and validating it against artificial distortions and human perception.

ITU Radiocommunication Sector · International Telecommunication Union · 2019 · standard

Official recommendation covering general sound-quality assessment, grading scales, comparison tests, random presentation order, listening conditions, statistical analysis, and reporting.

C2PA · Coalition for Content Provenance and Authenticity · 2026 · technical-guidance

Official guidance for applying Content Credentials to datasets, models, software, and media used during training and inference.

C2PA · Coalition for Content Provenance and Authenticity · 2026 · technical-guidance

Official guidance for applying Content Credentials to AI datasets, software, models, fine-tuning, inference, outputs, versioning, and release.

ITU Radiocommunication Sector · International Telecommunication Union · standard

Official guidance for selecting a subjective audio-assessment method according to the test purpose and expected impairment.

Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi · Transactions on Machine Learning Research · 2023 · paper

Original EnCodec paper combining reconstruction objectives, perceptual losses, ablations, and MUSHRA listening tests for music and other audio domains.

Yuxin Wen et al. · arXiv · 2025 · preprint

Evaluates recent audio-watermarking systems against a broad set of removal attacks and highlights the need to test claimed robustness under realistic transformations.

PyTorch contributors · PyTorch · documentation

Official documentation for disabling gradient-related tracking and selected autograd overhead during inference.

Office for Human Research Protections · United States Department of Health and Human Services · government-guidance

Official guidance on legally effective and documented informed consent for research covered by HHS requirements.

AWS · Amazon Web Services · api-documentation

Official API showing that asynchronous requests are enqueued and return result-location information rather than the inference result.

ITU Radiocommunication Sector · International Telecommunication Union · 2015 · standard

Official recommendation covering rigorous experimental design, listening conditions, subject selection, and reporting for critical audio assessment.

ITU Radiocommunication Sector · International Telecommunication Union · 1997 · standard

Official guide explaining that subjective audio-assessment methods should be selected according to the purpose and scale of the expected impairments.

ITU Radiocommunication Sector · International Telecommunication Union · 2015 · standard

Official recommendation defining the MUSHRA method and controlled subjective assessment principles for intermediate audio quality.

Kubernetes contributors · Kubernetes · documentation

Official documentation for run-to-completion workloads, retries, parallelism, completion tracking, and failure policies.

Or Tal, Alon Ziv, Itai Gat, Felix Kreuk, and Yossi Adi · IEEE ICASSP · 2024 · paper

Primary JASCO paper combining text with temporally aligned chord, melody, drum, and full-mix controls.

librosa contributors · librosa · documentation

Official documentation for chroma, spectral, mel, tonal, onset, rhythm, and related audio features.

librosa contributors · librosa · documentation

Official documentation for dynamic-programming beat tracking using onset strength and tempo estimation.

Lyria Team et al. · arXiv · 2025 · paper

Primary research introducing continuous real-time music streams with synchronized control and describing Magenta RealTime and Lyria RealTime.

Zach Evans, Julian D. Parker, CJ Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons · arXiv · 2024 · paper

Primary research reporting latent-diffusion music generation over long temporal contexts, with generated tracks of up to four minutes and forty-five seconds.

Google DeepMind · Google DeepMind · official-model-documentation

Official documentation describing Lyria 3, multimodal image conditioning, technical controls, and generation of cohesive tracks up to three minutes.

Google DeepMind · Google DeepMind · official-model-documentation

Official documentation describing real-time interactive music creation, control, and performance.

AWS · Amazon Web Services · documentation

Official documentation for transitioning objects to lower-cost classes and expiring objects through lifecycle rules.

Tianze Lu et al. · arXiv · 2025 · preprint

Emerging research on melody-aware music similarity under transformations including note splitting, arpeggiation, track dropout, and re-instrumentation. Included as a developing method rather than an established universal standard.

Yizhi Li et al. · International Conference on Learning Representations · 2024 · paper

Original MERT paper describing a music-specialised self-supervised representation evaluated across fourteen music-understanding tasks.

ITU Radiocommunication Sector · International Telecommunication Union · 2023 · standard

Official recommendation defining the MUSHRA method with multiple stimuli, hidden reference, anchors, controlled presentation, and subjective quality analysis.

ITU Radiocommunication Sector · International Telecommunication Union · 2015 · standard

Official recommendation for rigorous subjective evaluation of small audio impairments, including listening conditions, subject selection, training, test design, analysis, and reporting.

Colin Raffel et al. · mir_eval contributors · documentation

Official documentation for standardised evaluation functions used in music-information-retrieval research.

Colin Raffel et al. · International Society for Music Information Retrieval · 2014 · paper

Primary paper describing transparent implementations for beat, chord, melody, onset, pattern, segmentation, and other music-information-retrieval metrics.

Colin Raffel et al. · International Society for Music Information Retrieval · 2014 · paper

Primary source for transparent evaluation implementations covering melody, transcription, segmentation, rhythm, and other music-information-retrieval tasks.

mir_eval contributors · mir_eval · documentation

Official documentation for evaluating voicing and pitch contours in melody extraction.

MLflow contributors · MLflow · documentation

Official documentation for registered model versions, aliases, tags, source runs, and deployment-oriented version management.

MLflow contributors · MLflow · documentation

Official guidance on model signatures, environment capture, registry aliases, request validation, metadata, and serving.

Margaret Mitchell et al. · ACM Conference on Fairness, Accountability, and Transparency · 2019 · paper

Original proposal for model cards documenting intended uses, evaluation procedures, performance characteristics, limitations, and ethical considerations.

Margaret Mitchell et al. · ACM Conference on Fairness, Accountability, and Transparency · 2019 · paper

Original proposal for structured model documentation covering intended use, evaluation, limitations, and relevant context.

AWS · Amazon Web Services · documentation

Official comparison of real-time, asynchronous, serverless, and batch inference patterns.

NVIDIA · NVIDIA · documentation

Official guidance on loading, unloading, adding, and removing model versions while serving traffic.

NVIDIA · NVIDIA · documentation

Official model-repository layout and version-policy documentation.

Qingqing Huang et al. · International Society for Music Information Retrieval · 2022 · paper

Original MuLan paper linking music audio with unconstrained natural-language descriptions for retrieval and zero-shot music understanding.

Qingqing Huang et al. · International Society for Music Information Retrieval · 2022 · paper

Original MuLan paper linking music audio with unconstrained natural-language descriptions for retrieval and zero-shot applications.

Google DeepMind · Google DeepMind · 2025 · official-product-announcement

Official description of Lyria 2, Lyria RealTime, musician collaboration, continuous control, and SynthID watermarking.

Meta AI · Meta AudioCraft · documentation

Official overview of MusicGen models, EnCodec tokenization, pretrained checkpoints, and inference use.

Simon Rouard, Robin San Roman, Yossi Adi, and Axel Roebel · IEEE ICASSP · 2025 · paper

Primary research on parallel bass, drum, and other-stem generation, editing, and iterative source-conditioned composition.

Andrea Agostinelli et al. · Google Research · 2023 · paper

Original MusicLM paper reporting separate evaluation of audio quality and text adherence and introducing MusicCaps.

Andrea Agostinelli et al. · Google Research · 2023 · dataset-paper

Primary source for MusicCaps, described as approximately 5,500 music-text pairs with rich expert-written descriptions.

Andrea Agostinelli et al. · Google Research · 2023 · paper

Original MusicLM paper using automated and human measures of audio quality and text adherence and introducing the MusicCaps benchmark.

Andrea Agostinelli et al. · Google Research · 2023 · dataset-paper

Primary source for MusicCaps, a benchmark of approximately 5,500 music-text pairs with detailed human-written descriptions.

Andrea Agostinelli et al. · Google Research · 2023 · paper

Original MusicLM paper describing a pairwise human test with two ten-second clips, a caption, graded A/B preference responses, and instructions focused specifically on text adherence.

Andrea Agostinelli et al. · Google Research · 2023 · paper

Original MusicLM paper containing a training-data memorisation analysis based on known training examples, prefixes, and continuation similarity.

Geoffrey Cideron et al. · Proceedings of Machine Learning Research · 2024 · paper

Peer-reviewed ICML paper describing 300,000 pairwise music preferences and showing that text adherence and audio quality explain only part of human musical preference.

Geoffrey Cideron et al. · Proceedings of Machine Learning Research · 2024 · paper

Peer-reviewed research using reward models and 300,000 pairwise preferences to align a MusicLM-based generator.

OpenTelemetry contributors · OpenTelemetry · documentation

Official introduction to observability and instrumentation for understanding distributed systems.

OWASP contributors · OWASP · security-guidance

Community security guidance covering prompt injection, unsafe downstream output handling, excessive permissions, sensitive information, and related application risks.

Robin San Roman et al. · International Conference on Machine Learning · 2024 · paper

Introduces AudioSeal, a neural audio watermark with localised detection designed for identifying AI-generated speech regions.

MLflow contributors · MLflow · documentation

Official documentation for immutable prompt versions, aliases, comparison, reuse, A/B testing, and rollback.

Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, and Chiyuan Zhang · International Conference on Learning Representations · 2023 · paper

Primary research connecting discoverable memorisation with model capacity, duplicated examples, and prompting context. MusicLM and MusicGen adapt related methodology to music.

NVIDIA · NVIDIA · documentation

Official guidance on batching requests with variable input shapes and the padding trade-off.

Ralph Allan Bradley and Milton E. Terry · Biometrika · 1952 · paper

Foundational paper introducing the Bradley-Terry method for estimating rankings from pairwise comparison outcomes.

Baisen Wang, Chenxi Bao, and Qisong Han · arXiv · 2026 · preprint

Recent preprint exploring streaming latent generation, consistency distillation, and dynamic control for low-latency interaction. Included as emerging research rather than settled evidence.

NIST · National Institute of Standards and Technology · 2024 · government-report

Official secure-development profile for producers and acquirers of AI models and AI systems.

OpenTelemetry contributors · OpenTelemetry · documentation

Official definitions of traces, metrics, logs, baggage, and profiles.

Jade Copet et al. · NeurIPS · 2023 · paper

Original MusicGen paper combining automated evaluation, human ratings of audio quality and text relevance, ablations, and memorisation testing.

Jade Copet et al. · NeurIPS · 2023 · paper

Original MusicGen paper combining text-alignment, audio-quality, human, diversity, ablation, and memorisation evaluations.

Jade Copet et al. · NeurIPS · 2023 · paper

Original MusicGen paper reporting separate human ratings for overall perceptual quality and text relevance, crowdsourced response filtering, several ratings per sample, confidence intervals, and loudness normalisation.

Jade Copet et al. · NeurIPS · 2023 · paper

Original MusicGen paper describing token-based memorisation analysis, training-example continuations, similarity thresholds, and human and automated evaluation.

Research contributors · arXiv · 2024 · preprint

Investigates representations for singer identity and whether speech-trained speaker-recognition models transfer adequately to singing voice.

SAG-AFTRA and signatory record companies · SAG-AFTRA · 2024 · collective-bargaining-agreement

Contractual provisions for covered sound recordings, including definitions, consent requirements, compensation, and review of digital voice replicas. Scope is specific to the agreement.

Universal Music Group · Universal Music Group · 2024 · official-industry-announcement

Official announcement describing artist-specific vocal models trained on artists' own voice data, exclusive creative use, ownership control, and artistic approval.

Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi · IEEE/ACM Transactions on Audio, Speech, and Language Processing · 2021 · paper

Original SoundStream paper describing objective and subjective evaluation of an end-to-end neural codec across speech, music, and general audio.

Google DeepMind · Google DeepMind · official-technical-documentation

Official description of watermarking and detection across AI-generated media, including audio generated through Lyria.

National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research · United States Department of Health and Human Services · 1979 · government-guidance

Official ethical framework for research involving human participants, including respect for persons, beneficence, justice, and informed consent.

Information Commissioner's Office · Information Commissioner's Office · regulatory-guidance

Official data-protection guidance concerning personal-data processing for research, legal bases, safeguards, transparency, and participant rights.

Roser Batlle-Roca, Wei-Hsiang Liao, Xavier Serra, Yuki Mitsufuji, and Emilia Gómez · International Society for Music Information Retrieval · 2024 · paper

Music-specific study comparing raw-audio similarity metrics and proposing the MiRA workflow for replication assessment.

Roser Batlle-Roca, Wei-Hsiang Liao, Xavier Serra, Yuki Mitsufuji, and Emilia Gómez · International Society for Music Information Retrieval · 2024 · paper

Music-specific study evaluating raw-audio similarity metrics and proposing a model-independent replication-assessment workflow.

Roser Batlle-Roca, Wei-Hsiang Liao, Xavier Serra, Yuki Mitsufuji, and Emilia Gómez · International Society for Music Information Retrieval · 2024 · paper

Introduces MiRA, a model-independent music-replication assessment workflow using several raw-audio similarity metrics and controlled replication experiments.

OpenTelemetry contributors · OpenTelemetry · documentation

Official explanation of end-to-end request traces and spans across services.

NVIDIA · NVIDIA · documentation

Official documentation for dynamic batching, queue delays, priorities, timeouts, and throughput-oriented request combination.

Craig Greenberg et al. · National Institute of Standards and Technology · 2020 · government-paper

Official overview of systematic speaker-recognition and speaker-verification evaluation.

Universal Music Group · Universal Music Group · 2025 · official-industry-announcement

Official announcement describing a planned creation platform trained on authorized and licensed music, with fingerprinting, filtering, and protected platform controls.

AWS · Amazon Web Services · documentation

Official guidance for isolating messages that repeatedly fail processing.

Ruibin Yuan et al. · arXiv · 2025 · paper

Primary research on long-form lyrics-to-song generation using track-decoupled prediction and structural progressive conditioning.

PyTorch contributors · PyTorch · documentation

Official explanation of forward propagation, backward propagation, automatic differentiation, computational graphs, and stored parameter gradients.

PyTorch contributors · PyTorch · documentation

Official explanation of trainable parameters, gradient calculation, and optimisation foundations.

Diederik P. Kingma and Jimmy Ba · International Conference on Learning Representations · 2015 · paper

Original paper introducing Adam and its adaptive first-order moment estimates.

Google researchers and engineers · Google for Developers · documentation

Official guidance on profiling input pipelines, offline preprocessing, evaluation intervals, checkpoint selection, and experiment management.

PyTorch contributors · PyTorch · documentation

Official recipe for reducing synchronous training interruption while writing distributed checkpoints.

Simon Rouard et al. · arXiv · 2024 · paper

Explores textual inversion using a pretrained text-to-music model and a jointly trained style-conditioning music model with quantised audio features.

Fang-Duo Tsai et al. · International Society for Music Information Retrieval · 2024 · paper

Introduces attention-based audio adapters for a pretrained AudioLDM2 model, with 22 million trainable parameters for timbre transfer, genre transfer, and accompaniment generation.

Meta AI · Meta AudioCraft · documentation

Official API documentation for AudioCraft training and inference components including MusicGen, AudioGen, and EnCodec.

Meta AI · Meta AudioCraft · documentation

Official documentation for text, waveform, joint-embedding, classifier-free guidance, and attribute-dropout conditioning.

Meta AI · Meta AudioCraft · documentation

Official API for MusicGen language modelling, sampling parameters, classifier-free guidance, condition dropout, and codebook generation.

Meta AI · Meta AudioCraft · documentation

Official model interface for inference, text conditioning, melody conditioning, continuation, and generation parameters.

Meta AI · Meta AudioCraft · documentation

Official documentation stating that AudioCraft provides training and inference code for MusicGen, AudioGen, and EnCodec.

Meta AI · Meta AudioCraft · documentation

Official documentation for AudioCraft's PyTorch training pipelines, Flashy, Dora experiment management, environment configuration, grids, clusters, and experiment workflows.

PyTorch contributors · PyTorch · documentation

Official technical documentation for reverse automatic differentiation and gradient computation.

PyTorch contributors · PyTorch · documentation

Official recipe for autocasting and gradient scaling in mixed-precision training.

PyTorch contributors · PyTorch · documentation

Official explanation of autocasting, gradient scaling, non-finite gradient handling, and mixed-precision optimiser steps.

PyTorch contributors · PyTorch · documentation

Official examples covering autocasting, gradient scaling, gradient accumulation, clipping, and mixed-precision training.

PyTorch contributors · PyTorch · documentation

Official examples for mixed precision, gradient accumulation, unscaling, clipping, and multiple models or losses.

PyTorch contributors · PyTorch · documentation

Official examples for gradient accumulation, unscaling, clipping, and mixed-precision training.

Rich Harang and Hillary Sanders · arXiv · 2023 · paper

Compares retraining, fine-tuning, rehearsal, and regularisation approaches for updating models while retaining earlier performance.

scikit-learn contributors · scikit-learn · documentation

Official guidance on data leakage, inconsistent preprocessing, test separation, and controlling randomness.

scikit-learn contributors · scikit-learn · documentation

Official guidance on data leakage, inconsistent preprocessing, split discipline, and random-state handling.

scikit-learn contributors · scikit-learn · documentation

Official guidance distinguishing training, validation, and test data and warning that repeated hyperparameter decisions can leak information from the test set.

scikit-learn contributors · scikit-learn · documentation

Official documentation distinguishing training, validation, and test use and explaining how cross-validation estimates generalisation.

scikit-learn contributors · scikit-learn · documentation

Official guidance for separating training, validation, and test use.

PyTorch contributors · PyTorch · documentation

Official API documentation describing cross-entropy between input logits and targets.

PyTorch contributors · PyTorch · documentation

Official documentation for CUDA behaviour, the caching allocator, allocated memory, reserved memory, and memory-management functions.

PyTorch contributors · PyTorch · documentation

Official tutorial covering data-loader workers, pinned memory, prefetching, persistent workers, and data-pipeline throughput.

PyTorch contributors · PyTorch · documentation

Official documentation for datasets, batching, shuffling, sampling, and iterable data loading.

PyTorch contributors · PyTorch · documentation

Official tutorial for diagnosing distributed hangs with stack traces and collective-operation flight-recorder data.

Ilya Loshchilov and Frank Hutter · International Conference on Learning Representations · 2019 · paper

Original paper introducing decoupled weight decay for adaptive optimisers, commonly known as AdamW.

Katherine Lee et al. · Association for Computational Linguistics · 2022 · paper

Peer-reviewed language-model study on exact and near duplicate removal, memorised output, train-test overlap, and model efficiency. Used as cross-domain dataset evidence.

Nikhil Kandpal, Eric Wallace, and Colin Raffel · Proceedings of Machine Learning Research · 2022 · paper

Peer-reviewed language-model study showing a strong relationship between duplicate count and regeneration frequency. Used here as cross-domain evidence, not as a music-specific result.

Google researchers and engineers · Google for Developers · documentation

Official practical guide to deep-learning experimentation, tuning, pipeline implementation, and systematic performance improvement.

Google researchers and engineers · Google for Developers · documentation

Official guidance on identifying optimisation instability using learning-rate behaviour and gradient norms, with possible responses including warm-up and clipping.

PyTorch contributors · PyTorch · documentation

Official API documentation for model replication and gradient synchronisation across data-parallel processes.

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov · Journal of Machine Learning Research · 2014 · paper

Original peer-reviewed paper introducing dropout as a neural-network regularisation method.

Meta AI · Meta AudioCraft · documentation

Official AudioCraft documentation for training and evaluating the neural audio codec used by MusicGen.

Atharva Mehta, Shivam Chauhan, and Monojit Choudhury · arXiv · 2025 · paper

Music-generation study comparing adapter architectures and scales for adapting MusicGen and Mustango to underrepresented music traditions.

Nicholas Carlini et al. · USENIX Security Symposium · 2023 · paper

Peer-reviewed image-diffusion study demonstrating training-data extraction through generation and filtering. Included as cross-domain evidence about extraction, not as direct evidence about music models.

Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré · NeurIPS · 2022 · paper

Introduces tiled exact attention that reduces memory reads and writes between GPU memory levels.

Dimitrios Bralios et al. · IEEE ICASSP · 2024 · paper

Peer-reviewed audio study examining memorisation across training-set sizes, duplicated AudioCaps clips, and retrieval metrics for detecting replicated audio.

PyTorch contributors · PyTorch · documentation

Official documentation for organising process groups and multi-dimensional distributed parallelism.

PyTorch contributors · PyTorch · documentation

Official tutorial for saving and restoring state partitioned across distributed workers and changing trainer counts.

PyTorch contributors · PyTorch · documentation

Official tutorial for multi-process and multi-machine data-parallel training.

PyTorch contributors · PyTorch · documentation

Official documentation describing the memory burden of model parameters, gradients, and optimiser states and how these can be sharded for large-scale training.

PyTorch contributors · PyTorch · documentation

Official tutorial explaining sharding of parameters, gradients, and optimiser states across workers.

NVIDIA · NVIDIA · documentation

Official explanation of GPU execution structure, memory hierarchy, and common performance limitations.

scikit-learn contributors · scikit-learn · documentation

Official cross-validation method for domain-specific groups that should not overlap across folds.

scikit-learn contributors · scikit-learn · documentation

Official grouped splitting tool that assigns unique groups rather than individual samples across partitions.

PyTorch contributors · PyTorch · documentation

Official tutorial illustrating memory used by parameters, activations, gradients, optimiser states, and optimiser intermediates during training.

Or Tal et al. · International Society for Music Information Retrieval · 2024 · paper

Introduces JASCO for global text conditioning together with local symbolic and audio controls using a new flow-matching architecture and conditioning method.

PyTorch contributors · PyTorch · documentation

Official tutorial for combining tensor parallelism with fully sharded training for large Transformer models.

Hugging Face contributors · Hugging Face · documentation

Official conceptual guidance for LoRA matrices, target modules, and parameter-efficient adaptation.

Edward J. Hu et al. · International Conference on Learning Representations · 2022 · paper

Original LoRA paper describing frozen pretrained weights with trainable low-rank matrix updates.

Meta AI · Meta AudioCraft · documentation

Official documentation showing checkpoint-based fine-tuning and warning that configurations and changed layers must remain compatible.

Paulius Micikevicius et al. · International Conference on Learning Representations · 2018 · paper

Foundational paper describing lower-precision training, higher-precision weight updates, and loss scaling.

PyTorch contributors · PyTorch · documentation

Official tutorial for analysing peak GPU memory, allocation categories, activation checkpointing savings, and memory imbalance.

Cheng-Zhi Anna Huang et al. · Google Brain and Magenta · 2018 · paper

Original paper describing relative attention for long symbolic music sequences, expressive piano generation, motif continuation, and comparisons with recurrent approaches.

Google Magenta · Google Magenta · 2018 · website

Official project page explaining the event representation and the contrast between recurrent hidden-state compression and Transformer access to earlier events.

Meta AI · Meta AudioCraft · documentation

Official overview of the MusicGen model, codec-token representation, and AudioCraft implementation.

Meta AI · Meta AudioCraft · documentation

Official overview of MusicGen architecture, pretrained checkpoints, generation modes, and codec-token configuration.

Meta AI · Meta AudioCraft · documentation

Official documentation describing MusicGen's 32 kHz EnCodec tokenizer, four codebooks at 50 Hz, generation modes, and model configuration.

Meta AI · Meta AudioCraft · documentation

Official documentation for MusicGen architecture, EnCodec token streams, conditioning, and generation.

Meta AI · Meta AudioCraft · documentation

Official implementation showing MusicGen batch preparation, tokenisation, masking, per-codebook cross-entropy, backpropagation, gradient scaling, clipping, optimiser updates, scheduling, and metric logging.

Meta AI · Meta AudioCraft · documentation

Official implementation documentation for MusicGen training, validation, generation, model summaries, metrics, and optimisation.

Meta AI · Meta AudioCraft · documentation

Official implementation showing metadata and audio checks, condition preparation, codec encoding, padding masks, per-codebook loss, backward propagation, gradient scaling, clipping, optimiser updates, finite-loss checks, generation, and audio decoding.

NVIDIA · NVIDIA · documentation

Official guidance on GPU execution, batch sizing, memory-limited operations, and deep-learning performance.

PyTorch contributors · PyTorch · documentation

Official tutorial covering loss functions, optimisers, zeroing gradients, backward propagation, optimiser steps, and training and validation loops.

James Kirkpatrick et al. · Proceedings of the National Academy of Sciences · 2017 · paper

Foundational study of catastrophic forgetting during sequential neural-network training and a regularisation method for protecting important earlier parameters.

Hugging Face contributors · Hugging Face · documentation

Official PEFT documentation covering efficient adaptation without updating all base-model parameters.

Yiwei Ding and Alexander Lerch · arXiv · 2024 · paper

Music-specific comparison of adapter-based, prompt-based, and reparameterisation-based transfer methods against probing, fine-tuning, and small models trained from scratch.

Neil Houlsby et al. · International Conference on Machine Learning · 2019 · paper

Original paper introducing compact task-specific adapter modules while keeping the original network fixed.

Ian Simon and Sageev Oore · Google Magenta · 2017 · website

Official Magenta explanation of an LSTM-based recurrent model using event-based MIDI to generate polyphonic piano performances with expressive timing and dynamics.

PyTorch contributors · PyTorch · documentation

Official guidance for improving performance across data loading, memory, precision, and model execution.

PyTorch contributors · PyTorch · documentation

Official performance guidance noting that anomaly detection, profilers, and gradcheck are debugging tools rather than normal training settings.

Colin Raffel and contributors · Colin Raffel · documentation

Official documentation for parsing, inspecting, modifying, synthesising, and extracting information from MIDI files.

PyTorch contributors · PyTorch · documentation

Official guidance on random seeds, deterministic behaviour, DataLoader worker seeding, and the limits of reproducibility across releases and platforms.

PyTorch contributors · PyTorch · documentation

Official guidance on seeds, deterministic algorithms, data-loader randomness, and limits to exact reproducibility across releases and platforms.

PyTorch contributors · PyTorch · documentation

Official guidance on random seeds, deterministic behaviour, data-loader randomness, and reproducibility limitations.

PyTorch contributors · PyTorch · documentation

Official guidance for saving model state and general training checkpoints.

Matthew Inkawhich and PyTorch contributors · PyTorch · documentation

Official guidance for saving weights, general checkpoints, optimiser state, and restoring models.

PyTorch contributors · PyTorch · documentation

Official guidance for preserving model, optimiser, scheduler, and training state.

Jade Copet et al. · Meta AI · 2023 · paper

Original MusicGen paper describing a Transformer language model trained over several streams of compressed music tokens.

Jade Copet et al. · Meta AI · 2023 · paper

Original MusicGen paper describing a music language model trained over compressed EnCodec token streams.

Jade Copet et al. · NeurIPS · 2023 · paper

Original MusicGen paper. Its memorisation experiment tests exact and 80 percent partial matches in the first codec stream for five-second continuations from 20,000 sampled training examples.

Jade Copet et al. · NeurIPS · 2023 · paper

Original MusicGen paper describing a single-stage Transformer over multiple streams of compressed music tokens.

Jade Copet et al. · NeurIPS · 2023 · paper

Original MusicGen paper describing conditional autoregressive modelling over several streams of compressed music tokens.

Colin Raffel · Colin Raffel · dataset

Official project page describing 176,581 unique MIDI files, including 45,129 matched and aligned with Million Song Dataset entries.

Curtis Hawthorne et al. · Google Magenta · dataset

Official dataset page describing roughly 200 hours of aligned piano audio and MIDI, expressive performance information, metadata, predefined splits, and licence terms.

PyTorch contributors · PyTorch · documentation

Official automatic-differentiation documentation relevant to gradients and backward-pass state.

PyTorch contributors · PyTorch · documentation

Official automatic-differentiation documentation, including anomaly detection and gradient-checking tools.

PyTorch contributors · PyTorch · documentation

Official API for identifying finite and non-finite tensor elements.

PyTorch contributors · PyTorch · documentation

Official API documentation explaining that dropout randomly zeros activations during training and behaves differently during evaluation.

PyTorch contributors · PyTorch · documentation

Official module API containing forward and backward hook mechanisms for inspecting model computations.

PyTorch contributors · PyTorch · documentation

Official documentation for calculating and clipping the total norm of parameter gradients.

PyTorch contributors · PyTorch · documentation

Official API for calculating and clipping the total gradient norm.

PyTorch contributors · PyTorch · documentation

Official documentation for optimisation algorithms and learning-rate schedulers.

PyTorch contributors · PyTorch · documentation

Official profiler documentation for tracing CPU and CUDA operations, shapes, stacks, and memory.

PyTorch contributors · PyTorch · documentation

Official profiler for collecting performance and memory information during training and inference.

PyTorch contributors · PyTorch · documentation

Official API for registering callbacks executed when tensor gradients are calculated.

PyTorch contributors · PyTorch · documentation

Official documentation explaining activation checkpointing as a compute-for-memory trade-off.

PyTorch contributors · PyTorch · documentation

Official dataset, sampler, batching, worker, prefetch, and persistent-worker documentation.

PyTorch contributors · PyTorch · documentation

Official documentation for datasets, batching, sampling, workers, prefetching, and data loading.

Roser Batlle-Roca, Wei-Hsiang Liao, Xavier Serra, Yuki Mitsufuji, and Emilia Gómez · International Society for Music Information Retrieval · 2024 · paper

Peer-reviewed music-specific work proposing the MiRA assessment approach and evaluating several raw-audio similarity metrics in controlled replication tests.

Sasank Chilamkurthy and PyTorch contributors · PyTorch · documentation

Official PyTorch tutorial contrasting full fine-tuning with freezing a pretrained network and training only a final layer.

scikit-learn contributors · scikit-learn · documentation

Official example illustrating underfitting, suitable complexity, overfitting, and cross-validated error.

PyTorch contributors · PyTorch · documentation

Official guidance for recording and visualising CUDA memory snapshots and investigating allocation patterns.

scikit-learn contributors · scikit-learn · documentation

Official documentation explaining training and validation curves, underfitting, overfitting, and how additional data can affect generalisation.

PyTorch contributors · PyTorch · documentation

Official tutorial for recording and visualising training metrics, images, model graphs, and embeddings.

PyTorch contributors · PyTorch · documentation

Official tutorial distinguishing training and evaluation modes and calculating validation loss.

Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He · SC20 · 2020 · paper

Foundational work on eliminating replicated parameter, gradient, and optimiser-state memory in distributed training.

PyTorch contributors · PyTorch · documentation

Official explanation of gradient accumulation and the need to clear gradients between ordinary training steps.

Creative Commons · Creative Commons · documentation

Official overview of Creative Commons licence elements and the permissions and conditions attached to standard CC licences.

Google Research · Google Research · 2017 · website

Official announcement describing AudioSet as more than two million ten-second excerpts labelled with hundreds of sound-event categories.

Romain Hennequin, Jimena Royo-Letelier, and Manuel Moussallam · arXiv · 2018 · paper

Research showing how audio-based genre embeddings can identify duplicate labels, translate between tag systems, and recover genre relationships.

Caroline Chen and Moto Hira · PyTorch · documentation

Official tutorial on bandlimited audio resampling, filter parameters, roll-off, aliasing, and efficient reuse of resampling kernels.

Jort F. Gemmeke et al. · Google Research · 2017 · paper

Original paper describing a large-scale dataset of manually labelled audio events organised through an ontology.

Jort F. Gemmeke et al. · Google Research · 2017 · paper

Original paper describing a manually annotated, multi-label audio-event dataset organised through a structured ontology.

Jort F. Gemmeke et al. · Google Research · 2017 · paper

Original paper introducing AudioSet, its sound-event ontology, and its large multi-label collection with highly uneven class frequencies.

Ahmet Solak, Florian Grötschla, Luca A. Lanzendörfer, and Roger Wattenhofer · arXiv · 2025 · paper

Introduces the globally balanced GlobalDISCO evaluation dataset and reports disparities across regions, languages, and mainstream versus geographically niche genres.

Nadine Kroher, Helena Cuesta, and Aggelos Pikrakis · arXiv · 2023 · paper

Early experiment using MusicGen to create genre-conditioned synthetic training data for a music information retrieval classifier.

Lukas Lalinsky and contributors · AcoustID · documentation

Official repository describing Chromaprint as a compact audio fingerprint system designed for near-identical full-file identification and duplicate audio detection.

U.S. Copyright Office · U.S. Copyright Office · 2025 · documentation

Official report describing the legal and policy debate around copyrighted works in generative AI training and the fact-specific, jurisdiction-sensitive nature of the analysis.

U.S. Copyright Office · U.S. Copyright Office · 2020 · documentation

Official circular explaining that musical compositions and sound recordings are separate works and identifying the different material and authors associated with each.

Creative Commons · Creative Commons · 2025 · documentation

Official FAQ explaining that CC licences do not automatically clear third-party publicity, privacy, personality, trademark, patent, or other rights outside their scope.

MLCommons Croissant Working Group · MLCommons · 2026 · documentation

Machine-learning dataset metadata standard supporting licence fields and PROV-O relationships at dataset, resource, record, field, and value level.

Gabriel Meseguer-Brocal, Alice Cohen-Hadria, and Geoffroy Peeters · arXiv · 2019 · paper

Original paper introducing 5,358 audio tracks with time-aligned lyrics and vocal melody notes at four levels of granularity.

Benno Weck and Xavier Serra · Music Technology Group, Universitat Pompeu Fabra · 2023 · paper

Audio-specific study finding duplicates across SoundDesc train and evaluation data, showing overly optimistic retrieval performance, and proposing corrected splits.

R. Channing Moore, Daniel P. W. Ellis, Eduardo Fonseca, Shawn Hershey, Aren Jansen, and Manoj Plakal · arXiv · 2023 · paper

Audio-specific study showing that balancing AudioSet improved one public evaluation score while hurting a separate evaluation set and did not clearly improve rare-class performance.

Timnit Gebru et al. · Communications of the ACM · 2021 · paper

Foundational framework proposing documentation of dataset motivation, composition, collection, processing, uses, distribution, and maintenance.

Timnit Gebru et al. · Communications of the ACM · 2021 · paper

Foundational dataset-documentation framework covering motivation, composition, collection, processing, distribution, maintenance, uses, and limitations.

Curtis Hawthorne et al. · Google Magenta · 2018 · paper

Original paper introducing paired piano audio and MIDI with fine alignment for transcription, symbolic composition, and waveform synthesis.

Andre Holzapfel et al. · Transactions of the International Society for Music Information Retrieval · 2018 · paper

Music-specific discussion of cultural, social, data, and algorithmic biases in music information retrieval research and technology.

FFmpeg Project · FFmpeg · documentation

Official documentation for inspecting media containers and streams, selecting audio streams, checking usable stream configuration, and producing structured output.

Prafulla Dhariwal et al. · OpenAI · 2020 · paper

Original Jukebox paper describing music generation conditioned on artist, genre, and unaligned lyrics.

Elena V. Epure, Anis Khlif, and Romain Hennequin · arXiv · 2019 · paper

Research addressing diversity and subjectivity across genre tag systems and methods for translating annotations between taxonomies.

librosa development team · librosa · documentation

Official documentation for identifying non-silent intervals in mono or multichannel audio.

librosa development team · librosa · documentation

Official documentation for trimming leading and trailing silence using a threshold below a reference level.

librosa development team · librosa · documentation

Official documentation for converting a time series from an original sample rate to a target sample rate.

SeungHeon Doh and contributors · KAIST Music and Audio Computing Lab · 2023 · documentation

Official project repository documenting tag-to-caption generation and audio-to-caption training.

SeungHeon Doh, Keunwoo Choi, Jongpil Lee, and Juhan Nam · International Society for Music Information Retrieval · 2023 · paper

Original paper generating large-scale captions from music tags and evaluating caption quality through correct attributes and incorrect attributes.

Atharva Mehta, Shivam Chauhan, and Monojit Choudhury · arXiv · 2024 · paper

Large review of more than one million dataset hours and over 200 papers examining geographic and cultural representation in music-generation research.

Music Technology Group · Music Technology Group, Universitat Pompeu Fabra · dataset

Official annotation documentation describing raw labels from three annotators and a clean subset containing agreed labels while excluding unmatched and instrumental answers where relevant.

Atharva Mehta, Shivam Chauhan, Amirbek Djanibekov, Atharva Kulkarni, Gus Xia, and Monojit Choudhury · arXiv · 2025 · paper

Music-specific study measuring genre and cultural underrepresentation in generation datasets and testing parameter-efficient adaptation for Hindustani classical and Turkish makam music.

Google · Google · 2023 · dataset

Official dataset card describing 5,521 records and fields including source video ID, segment timing, AudioSet labels, aspect lists, and musician-written captions.

Google · Google · 2023 · dataset

Official dataset card describing 5,521 ten-second music examples with AudioSet labels, aspect lists, and free-text captions written by musicians.

Andrea Agostinelli et al. · Google Research · 2023 · paper

Original MusicLM paper introducing text-conditioned music generation and the MusicCaps dataset of about 5.5 thousand music-text pairs.

Andrea Agostinelli et al. · Google Research · 2023 · paper

Original MusicLM paper describing MusicCaps, expert-written captions, aspect lists, text conditioning, and caption adherence evaluation.

Sabina Tomkins et al. · arXiv · 2024 · paper

Argues that claims about dataset diversity need explicit definitions, dimensions, and measurements rather than broad unsupported descriptions.

W3C Provenance Working Group · World Wide Web Consortium · 2013 · documentation

W3C Recommendation providing classes and relationships for representing and exchanging provenance information across systems.

ITU Radiocommunication Sector · International Telecommunication Union · 2023 · documentation

Official recommendation defining objective programme loudness and true-peak measurement algorithms.

European Parliament and Council of the European Union · European Union · 2016 · documentation

Official EU regulation defining consent and, where processing relies on consent, requiring that it be demonstrable and withdrawable under the stated conditions.

Jade Copet et al. · Meta AI · 2023 · paper

Original MusicGen paper describing a music generation model conditioned on text descriptions or melodic features.

Jade Copet et al. · Meta AI · 2023 · paper

Original MusicGen paper describing full-length music standardised to 32 kHz, mono downmixing in the main setup, and random 30-second training crops.

Jade Copet et al. · Meta AI · 2023 · paper

Original MusicGen paper describing music generation conditioned on textual descriptions or melodic features.

Zach Evans et al. · Stability AI · 2024 · paper

Original paper describing an open text-to-audio model producing stereo audio at 44.1 kHz.

Stability AI · Stability AI · documentation

Official dataset documentation describing loading and resampling, optional LUFS normalisation, silence handling, padding and cropping, padding masks, and channel conversion.

Google Magenta · Google Magenta · 2018 · dataset

Official dataset page describing paired audio and MIDI, key velocities, pedal positions, descriptive metadata, and alignment of about 3 milliseconds.

Google Magenta · Google Magenta · 2018 · dataset

Official dataset page stating that MAESTRO is available under the Creative Commons Attribution Non-Commercial Share-Alike 4.0 licence.

Dmitry Bogdanov et al. · Music Technology Group, Universitat Pompeu Fabra · 2019 · dataset

Official dataset page describing more than 55,000 tracks and 195 uploader-provided genre, instrument, and mood or theme tags.

Dmitry Bogdanov et al. · Music Technology Group, Universitat Pompeu Fabra · 2019 · dataset

Official dataset page describing more than 55,000 tracks, 195 genre, instrument, and mood or theme tags, uploader-provided labels, published splits, and tag distributions.

Christopher J. C. Burges, John C. Platt, and Soumya Jana · Microsoft Research · 2005 · paper

Research describing audio fingerprinting for detecting duplicate clips even when files differ in compression quality or duration.

Creative Commons · Creative Commons · 2025 · documentation

Official guidance explaining that copyright treatment of AI training varies and discussing conservative handling of attribution, share-alike, noncommercial, and no-derivatives licence elements.

The MIDI Association · The MIDI Association · documentation

Official overview of note on, note off, velocity, program change, pitch bend, aftertouch, and control change messages.

The MIDI Association · The MIDI Association · documentation

Official explanation that Standard MIDI Files store timed instructions and events rather than recorded sound, with playback produced by a connected sound engine.

Jort F. Gemmeke et al. · Google Research · 2017 · paper

Original paper introducing the AudioSet ontology and large-scale human-labelled dataset used for audio event recognition.

Zalan Borsos et al. · Google Research · 2022 · paper

Original AudioLM paper describing audio generation as language modelling over discrete audio tokens and combining semantic and acoustic token types.

Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang · Microsoft Research · 2022 · paper

Original CLAP paper describing separate audio and text encoders trained contrastively in a shared multimodal embedding space.

Apple Developer Documentation · 2017 · documentation

Official overview of audio data formats, including sample rate, bits per channel, and channels per frame.

Adobe · 2021 · documentation

Official documentation covering microphones, digital sampling, sample rate, bit depth, amplitude values, channels, and audio file size.

Adobe · 2021 · documentation

Official documentation explaining waveform and spectral displays, their axes, amplitude peaks, frequency components, colour intensity, and the time-frequency resolution tradeoff.

Curtis Hawthorne et al. · Google Magenta · 2019 · paper

Original MAESTRO paper presenting aligned audio and note events for transcription, symbolic composition, and waveform synthesis.

Meta Research · Meta Research · documentation

Official repository documenting available 24 kHz and 48 kHz EnCodec models and supported bitrate settings.

Jesse Engel et al. · Google Magenta · 2019 · paper

Original paper describing neural audio synthesis in the spectral domain using log magnitudes and instantaneous frequencies.

Alexandre Defossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi · Meta AI · 2022 · paper

Original EnCodec paper describing a real-time encoder-decoder codec with a quantized latent space and evaluations across speech, music, mono, and stereo audio.

Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar · Descript · 2023 · paper

Original Descript Audio Codec paper describing high-fidelity universal audio compression into discrete codes using improved residual vector quantization and reconstruction training.

National Institute on Deafness and Other Communication Disorders · 2022 · website

Official explanation of how sound waves produce vibrations that are converted into electrical signals in the auditory system.

Qingqing Huang et al. · Google Research · 2022 · paper

Original MuLan paper describing a two-tower joint music and text embedding model for retrieval, tagging, transfer learning, and zero-shot tasks.

Cheng-Zhi Anna Huang et al. · Google Brain · 2018 · paper

Original paper describing a Transformer model for long symbolic piano performances, continuations, and melody-conditioned accompaniment.

Google Magenta · Google Magenta · 2018 · website

Official project overview and generated examples for Music Transformer.

Meta AI · Meta AudioCraft · documentation

Official implementation documentation describing MusicGen's 32 kHz EnCodec tokenizer and model configuration.

Meta AI · Meta AudioCraft · documentation

Official implementation documentation describing MusicGen's 32 kHz EnCodec tokenizer, four codebooks, and 50 Hz token rate.

Andrea Agostinelli et al. · Google Research · 2023 · paper

Original MusicLM paper explaining how MuLan music embeddings were used during training and MuLan text embeddings were used as conditioning during generation.

SciPy · documentation

Official SciPy documentation describing a spectrogram as a view of changing frequency content over time and explaining its segmented Fourier transform parameters.

Jade Copet et al. · Meta AI · 2023 · paper

Original MusicGen paper describing compressed discrete music representations, common music sample rates, and mono and stereo generation.

Jade Copet et al. · Meta AI · 2023 · paper

Original MusicGen paper describing one Transformer language model operating over several streams of compressed discrete music tokens.

TensorFlow · 2024 · documentation

Official TensorFlow Hub tutorial describing YAMNet, its AudioSet classes, audio input requirements, and its score, embedding, and spectrogram outputs.

Neil Zeghidour et al. · Google Research · 2021 · paper

Original paper describing an end-to-end encoder, residual vector quantizer, and decoder for speech, music, and general audio, including variable bitrate operation.

Google Magenta · Google Magenta · 2018 · dataset

Official dataset page describing about 200 hours of paired piano audio and MIDI aligned to about 3 milliseconds, including velocity and pedal data.

NASA · website

Official overview of sound as a longitudinal wave and the relationship of amplitude and frequency to loudness and pitch.

TensorFlow · 2024 · documentation

Official TensorFlow tutorial describing YAMNet's 1,024-dimensional embeddings and their use as pretrained features for a smaller audio classifier.

Leland McInnes, John Healy, and James Melville · arXiv · 2018 · paper

Original UMAP paper describing a scalable dimensionality reduction method used to project high-dimensional data for visualisation.

Microsoft Learn · 2021 · documentation

Official format documentation defining channel count, samples per second, and bits per sample for waveform audio.

Aaron van den Oord et al. · Google DeepMind · 2016 · paper

Original paper introducing an autoregressive model that generates raw audio waveforms one sample at a time and includes experiments on music.

Simon Rouard, Yossi Adi, Jade Copet, Axel Roebel, and Alexandre Défossez · arXiv · 2024 · paper

Introduces audio-based style conditioning, joint text and audio control, and separate guidance over the two condition types.

Haohe Liu et al. · ICML · 2023 · paper

Introduces text-to-audio generation through latent diffusion, an audio autoencoder, and aligned CLAP audio and text embeddings.

Haohe Liu et al. · ICML · 2023 · paper

Describes text-guided latent audio diffusion using aligned audio and text embeddings.

Google Research · Google Research · 2022 · website

Official project page with speech and piano continuation examples and the AudioLM abstract.

Google Research · Google Research · 2022 · website

Official project page containing speech and piano audio-continuation examples.

Zalan Borsos et al. · Google Research · 2022 · paper

Original paper mapping audio to discrete tokens and treating audio generation as language modelling, including piano continuations from audio prompts.

Zalan Borsos et al. · Google Research · 2022 · paper

Original paper describing hierarchical autoregressive language models over semantic and neural codec tokens for long-term audio consistency and acoustic quality.

Zalan Borsos et al. · Google Research · 2022 · paper

Original paper describing three-stage hierarchical generation with semantic, coarse acoustic, and fine acoustic tokens.

Jonathan Ho and Tim Salimans · arXiv · 2022 · paper

Introduces guidance based on combined conditional and unconditional diffusion predictions without requiring a separate classifier.

Jonathan Ho and Tim Salimans · arXiv · 2022 · paper

Introduces guidance based on combined conditional and unconditional predictions.

PyTorch contributors · PyTorch · documentation

Official documentation for computing cross-entropy loss between model logits and target classes.

Jiaming Song, Chenlin Meng, and Stefano Ermon · ICLR · 2021 · paper

Introduces non-Markovian diffusion sampling processes that use the same training objective while supporting faster generation with fewer sampling steps.

Jonathan Ho, Ajay Jain, and Pieter Abbeel · NeurIPS · 2020 · paper

Foundational paper describing a Gaussian forward corruption process, learned reverse denoising process, and noise-prediction training objective.

Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro · ICLR · 2021 · paper

Original paper describing conditional and unconditional waveform generation by converting white noise into structured audio through diffusion.

Zach Evans, CJ Carr, Josiah Taylor, Scott H. Hawley, and Jordi Pons · arXiv · 2024 · paper

Describes Stable Audio's text and timing conditioning for long-form, variable-length stereo generation.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer · CVPR · 2022 · paper

Foundational latent diffusion paper explaining diffusion in the compressed representation of a pretrained autoencoder to reduce computational requirements.

Alexander Quinn Nichol and Prafulla Dhariwal · ICML · 2021 · paper

Introduces improvements including learned reverse variances and an alternative noise schedule, with substantially fewer sampling passes in tested settings.

Meta AI · Meta AudioCraft · 2024 · documentation

Official AudioCraft documentation for temporally controlled generation using text, symbolic, and audio conditions.

Or Tal, Alon Ziv, Itai Gat, Felix Kreuk, and Yossi Adi · arXiv · 2024 · paper

Introduces JASCO, combining global text with time-aligned chord, melody, drum, and full-mix controls.

Prafulla Dhariwal et al. · OpenAI · 2020 · paper

Original paper describing a multi-scale VQ-VAE and autoregressive Transformer priors for generating music with singing over multiple minutes.

Prafulla Dhariwal et al. · OpenAI · 2020 · paper

Original paper describing multi-scale VQ-VAE compression, a top-level autoregressive prior, and middle and bottom upsampling models.

Prafulla Dhariwal et al. · OpenAI · 2020 · paper

Describes artist, genre, and unaligned lyric conditioning for raw-audio song generation.

Prafulla Dhariwal et al. · OpenAI · 2020 · paper

Describes a multi-scale VQ-VAE and autoregressive Transformer priors for raw-audio music generation with reported coherence over multiple minutes.

Zach Evans, Julian D. Parker, CJ Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons · arXiv · 2024 · paper

Describes a timing-conditioned diffusion Transformer operating on a highly downsampled continuous audio latent for long-form text-to-music generation.

Zach Evans, Julian D. Parker, CJ Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons · arXiv · 2024 · paper

Describes a diffusion Transformer operating at a latent rate of 21.5 positions per second, trained on long temporal contexts and generating music up to 4 minutes and 45 seconds.

Yi Zou et al. · ICASSP · 2022 · paper

Introduces a graph representation of bar-level melody relationships and separates structure generation from structure-conditioned melody generation.

Flavio Schneider, Ojasv Kamal, Zhijing Jin, and Bernhard Schölkopf · arXiv · 2023 · paper

Describes a two-stage system using a diffusion autoencoder and text-conditioned latent diffusion for long stereo music generation.

Peiling Lu et al. · arXiv · 2023 · paper

Introduces text-to-symbolic-music generation using explicit musical attributes as an intermediate control representation.

Botao Yu et al. · NeurIPS · 2022 · paper

Introduces fine attention to selected structure-related bars and coarse summaries for other bars, allowing longer symbolic sequences and improved repetition structure.

Cheng-Zhi Anna Huang et al. · Google Brain and Magenta · 2018 · paper

Original paper describing Transformer-based symbolic music generation with relative attention and long musical sequences.

Cheng-Zhi Anna Huang et al. · Google Brain and Magenta · 2018 · paper

Original paper describing autoregressive symbolic music generation with relative attention and minute-long piano event sequences.

Cheng-Zhi Anna Huang et al. · Google Brain and Magenta · 2018 · paper

Original paper describing relative attention for long symbolic sequences, minute-long piano generation, and motif-conditioned continuations.

Google Magenta · Google Magenta · 2018 · website

Official project page with generated piano examples and an accessible explanation of long-term musical structure.

Google Magenta · Google Magenta · 2018 · website

Official project page with generated continuations and examples of long-term piano structure.

Google Magenta · Google Magenta · 2018 · website

Official project page containing generated piano performances and an explanation of musical structure across several timescales.

Google · Google · 2023 · dataset

Official dataset card for music clips paired with musician-written captions, supporting the connection between Chapter 2 caption quality and text-conditioned generation.

Meta AI · Meta AudioCraft · 2024 · documentation

Official documentation for combining a short style-reference excerpt with textual conditioning.

Meta AI · Meta AudioCraft · documentation

Official implementation documentation describing autoregressive MusicGen token prediction and supported greedy, temperature, top-k, and top-p generation methods.

Meta AI · Meta AudioCraft · documentation

Official implementation documentation describing conditional and unconditional generation, audio continuation, temperature, top-k, and top-p sampling.

Meta AI · Meta AudioCraft · documentation

Official documentation for text generation, melody conditioning, and audio continuation in MusicGen.

Andrea Agostinelli et al. · Google Research · 2023 · paper

Original paper describing text-conditioned hierarchical sequence-to-sequence music generation using semantic and acoustic tokens.

Andrea Agostinelli et al. · Google Research · 2023 · paper

Original paper describing text-conditioned music generation and joint conditioning with hummed or whistled melodies.

Andrea Agostinelli et al. · Google Research · 2023 · paper

Describes hierarchical text-to-music generation at 24 kHz with reported consistency over several minutes.

Jan Melechovsky et al. · arXiv · 2023 · paper

Introduces a diffusion-based text-to-music system with control over chords, beats, tempo, key, and general text descriptions.

Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu · DeepMind · 2017 · paper

Original VQ-VAE paper introducing learned discrete latent representations paired with autoregressive priors.

Qingqing Huang et al. · arXiv · 2023 · paper

Introduces a cascade containing a text-conditioned diffusion generator and a diffusion cascader for high-fidelity 30-second music generation.

Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer · NeurIPS · 2015 · paper

Original paper describing the mismatch between training on true previous tokens and inference from model-generated tokens, and how errors can accumulate along a sequence.

Jade Copet et al. · Meta AI · 2023 · paper

Original MusicGen paper describing a Transformer language model over several streams of compressed music tokens, conditioned on text or melody.

Jade Copet et al. · Meta AI · 2023 · paper

Original MusicGen paper describing one autoregressive Transformer over several streams of compressed music tokens using token interleaving patterns.

Jade Copet et al. · Meta AI · 2023 · paper

Original MusicGen paper describing text-conditioned and melody-conditioned generation over compressed music tokens.

Jade Copet et al. · Meta AI · 2023 · paper

Describes autoregressive generation over several streams of compressed music tokens and provides context for the relationship between token rate and sequence length.

Zalan Borsos et al. · Google Research · 2023 · paper

Original paper replacing AudioLM's autoregressive acoustic stages with confidence-based parallel generation conditioned on semantic tokens.

Neil Zeghidour et al. · Google Research · 2021 · paper

Original paper describing a neural codec with residual vector quantisation used as the acoustic tokenizer in AudioLM and MusicLM.

Zach Evans, Julian D. Parker, CJ Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons · arXiv · 2024 · paper

Describes an open-weights latent diffusion text-to-audio model and evaluates its audio autoencoder separately from the full generative system.

Hugo Flores Garcia, Prem Seetharaman, Rithesh Kumar, and Bryan Pardo · ISMIR · 2023 · paper

Introduces masked acoustic-token generation supporting synthesis, continuation, inpainting, outpainting, compression, and looping with variation.

Interactive Audio Lab · Interactive Audio Lab · 2023 · website

Official project page with examples of VampNet continuation, inpainting, outpainting, and variation.

Aaron van den Oord et al. · Google DeepMind · 2016 · paper

Original paper describing fully probabilistic autoregressive prediction of raw audio samples conditioned on previous samples, with speech and music examples.

Aaron van den Oord et al. · Google DeepMind · 2016 · paper

Original paper describing probabilistic autoregressive waveform generation where each audio sample is conditioned on all previous samples.

Ziyu Wang, Lejun Min, and Gus Xia · arXiv · 2024 · paper

Introduces hierarchical symbolic representations for whole-song form, phrases, cadences, chords, accompaniment, and notes, generated through a top-down cascade.