MusicGen

Over Meta AI (FAIR)

Opgericht 2013 · Menlo Park, California, USA

Meta AI is the research division of Meta Platforms, originally founded as Facebook AI Research (FAIR) in 2013 by Yann LeCun. FAIR has produced many influential open-source releases including PyTorch, RoBERTa, Wav2Vec 2.0, the LLaMA family, and the audio research stack AudioCraft. MusicGen was published by FAIR's audio team in June 2023 (Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, Alexandre Defossez) as part of AudioCraft, a unified library that also includes EnCodec and AudioGen. AudioCraft was released under the permissive CC-BY-NC and MIT licences and remains one of the most popular reference implementations for controllable music generation in academia.

Architectuur

Single-stage Transformer language model over EnCodec audio tokens (text-and-melody-conditioned)

MusicGen is a single-stage auto-regressive Transformer that generates music by predicting tokens from a 32 kHz EnCodec residual vector quantiser at 50 Hz with four codebooks. The model interleaves the codebook streams using a delay pattern so that a single Transformer can model all four codebooks jointly, avoiding the cascaded models used in prior work such as MusicLM. Conditioning is provided by frozen T5 text encoders for text prompts and optionally by a chromagram-conditioned encoder for melody control, which lets users hum or upload a reference melody. The model was trained on 20,000 hours of licensed music (10K high-quality internal Meta tracks plus ShutterStock and Pond5 licensed catalogues), exclusively instrumental. Output is 32 kHz stereo or mono with a maximum length of 30 seconds in the public checkpoint; longer pieces are produced by sliding-window continuation. AudioCraft also ships training and fine-tuning scripts.

Parameters

300M (small), 1.5B (medium), 3.3B (large), 3.3B (melody)

Context

30 tokens

Mogelijkheden

• Text-to-music generation across many genres and moods
• Melody conditioning via chroma features (hum or upload a tune)
• Single-stage generation with no cascade of models (simpler than MusicLM)
• Up to 30-second clips at 32 kHz, sliding window for longer pieces
• Multiple sizes from 300M to 3.3B for laptops to data-centre GPUs
• Fully open weights with reproducible training recipe
• Best for: research on controllable music generation, indie composers, game soundtrack prototyping

Training & licentie

20,000 hours of licensed instrumental music: 10K hours from Meta's internal music dataset plus ShutterStock and Pond5 commercial libraries. All vocals were removed and the corpus is instrumental-only.

Licentie: Code under MIT; pretrained weights under CC-BY-NC 4.0 (non-commercial). Commercial use requires separate licensing.

Veiligheidstests: Limited; Meta describes risk of generating music similar to training data and recommends watermarking. No formal red-team report.