Instructions to use moonbox-dev/YuE2-3B-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use moonbox-dev/YuE2-3B-MLX with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download moonbox-dev/YuE2-3B-MLX --local-dir YuE2-3B-MLX
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
YuE2-3B MLX
MLX builds of YuE2-3B, made by Moonbox for Apple Silicon Macs. Each precision is a folder; keep a folder's files together. Only the parts listed below are converted: the rest of the pipeline comes from the original repository unless the section below says otherwise.
Files
The last column is how far each build is from the original PyTorch model on the same inputs (see the section below for what was measured).
| Precision | File | Size | sha256 | Checked against the original |
|---|---|---|---|---|
| 8bit | 8bit/LICENSE |
20.3 kB | 060985741d20 |
AR KL 0.0009 nats (p95 0.0019), top-1 agreement 97.2 %; audio 0.16 dB log-mel, MR-STFT 0.089, SNR 25.7 dB vs bf16 |
| 8bit | 8bit/THIRD_PARTY_NOTICES.md |
0.8 kB | 14d3fd9f6fee |
AR KL 0.0009 nats (p95 0.0019), top-1 agreement 97.2 %; audio 0.16 dB log-mel, MR-STFT 0.089, SNR 25.7 dB vs bf16 |
| 8bit | 8bit/ar.safetensors |
1.50 GB | 8fe39fdc9332 |
AR KL 0.0009 nats (p95 0.0019), top-1 agreement 97.2 %; audio 0.16 dB log-mel, MR-STFT 0.089, SNR 25.7 dB vs bf16 |
| 8bit | 8bit/config.json |
1.9 kB | a7b0daf7ce33 |
AR KL 0.0009 nats (p95 0.0019), top-1 agreement 97.2 %; audio 0.16 dB log-mel, MR-STFT 0.089, SNR 25.7 dB vs bf16 |
| 8bit | 8bit/mlx_model.py |
54.2 kB | 457dc446be16 |
AR KL 0.0009 nats (p95 0.0019), top-1 agreement 97.2 %; audio 0.16 dB log-mel, MR-STFT 0.089, SNR 25.7 dB vs bf16 |
| 8bit | 8bit/nar.safetensors |
1.51 GB | fc90aac152bb |
AR KL 0.0009 nats (p95 0.0019), top-1 agreement 97.2 %; audio 0.16 dB log-mel, MR-STFT 0.089, SNR 25.7 dB vs bf16 |
| 8bit | 8bit/qwen.tiktoken |
2.6 MB | b2b1b8dfb5cc |
AR KL 0.0009 nats (p95 0.0019), top-1 agreement 97.2 %; audio 0.16 dB log-mel, MR-STFT 0.089, SNR 25.7 dB vs bf16 |
| 8bit | 8bit/vae.safetensors |
265.4 MB | 2a677eaee050 |
AR KL 0.0009 nats (p95 0.0019), top-1 agreement 97.2 %; audio 0.16 dB log-mel, MR-STFT 0.089, SNR 25.7 dB vs bf16 |
| 8bit | 8bit/vocab.safetensors |
803.8 MB | e2072b33387e |
AR KL 0.0009 nats (p95 0.0019), top-1 agreement 97.2 %; audio 0.16 dB log-mel, MR-STFT 0.089, SNR 25.7 dB vs bf16 |
| 8bit | 8bit/yue2_generation_config.json |
0.5 kB | 203830cebde6 |
AR KL 0.0009 nats (p95 0.0019), top-1 agreement 97.2 %; audio 0.16 dB log-mel, MR-STFT 0.089, SNR 25.7 dB vs bf16 |
| 6bit | 6bit/LICENSE |
20.3 kB | 060985741d20 |
AR KL 0.0018 nats (p95 0.0034), top-1 agreement 96.0 %; audio 0.34 dB log-mel, MR-STFT 0.166, SNR 17.9 dB vs bf16 |
| 6bit | 6bit/THIRD_PARTY_NOTICES.md |
0.8 kB | 14d3fd9f6fee |
AR KL 0.0018 nats (p95 0.0034), top-1 agreement 96.0 %; audio 0.34 dB log-mel, MR-STFT 0.166, SNR 17.9 dB vs bf16 |
| 6bit | 6bit/ar.safetensors |
1.15 GB | 38df911aab6a |
AR KL 0.0018 nats (p95 0.0034), top-1 agreement 96.0 %; audio 0.34 dB log-mel, MR-STFT 0.166, SNR 17.9 dB vs bf16 |
| 6bit | 6bit/config.json |
1.9 kB | a7a2576e4874 |
AR KL 0.0018 nats (p95 0.0034), top-1 agreement 96.0 %; audio 0.34 dB log-mel, MR-STFT 0.166, SNR 17.9 dB vs bf16 |
| 6bit | 6bit/mlx_model.py |
54.2 kB | 457dc446be16 |
AR KL 0.0018 nats (p95 0.0034), top-1 agreement 96.0 %; audio 0.34 dB log-mel, MR-STFT 0.166, SNR 17.9 dB vs bf16 |
| 6bit | 6bit/nar.safetensors |
1.16 GB | dcd8ba3d93ae |
AR KL 0.0018 nats (p95 0.0034), top-1 agreement 96.0 %; audio 0.34 dB log-mel, MR-STFT 0.166, SNR 17.9 dB vs bf16 |
| 6bit | 6bit/qwen.tiktoken |
2.6 MB | b2b1b8dfb5cc |
AR KL 0.0018 nats (p95 0.0034), top-1 agreement 96.0 %; audio 0.34 dB log-mel, MR-STFT 0.166, SNR 17.9 dB vs bf16 |
| 6bit | 6bit/vae.safetensors |
265.4 MB | 2a677eaee050 |
AR KL 0.0018 nats (p95 0.0034), top-1 agreement 96.0 %; audio 0.34 dB log-mel, MR-STFT 0.166, SNR 17.9 dB vs bf16 |
| 6bit | 6bit/vocab.safetensors |
803.8 MB | e2072b33387e |
AR KL 0.0018 nats (p95 0.0034), top-1 agreement 96.0 %; audio 0.34 dB log-mel, MR-STFT 0.166, SNR 17.9 dB vs bf16 |
| 6bit | 6bit/yue2_generation_config.json |
0.5 kB | 203830cebde6 |
AR KL 0.0018 nats (p95 0.0034), top-1 agreement 96.0 %; audio 0.34 dB log-mel, MR-STFT 0.166, SNR 17.9 dB vs bf16 |
| 4bit | 4bit/LICENSE |
20.3 kB | 060985741d20 |
mixed 4 / 6 bits: AR KL 0.0074 nats (p95 0.0141), top-1 agreement 91.8 %; audio 0.78 dB log-mel, MR-STFT 0.319, SNR 14.2 dB vs bf16 (passes both marks; 4 bits throughout: KL 0.016, 87.4 %, 1.52 dB, 8.2 dB) |
| 4bit | 4bit/THIRD_PARTY_NOTICES.md |
0.8 kB | 14d3fd9f6fee |
mixed 4 / 6 bits: AR KL 0.0074 nats (p95 0.0141), top-1 agreement 91.8 %; audio 0.78 dB log-mel, MR-STFT 0.319, SNR 14.2 dB vs bf16 (passes both marks; 4 bits throughout: KL 0.016, 87.4 %, 1.52 dB, 8.2 dB) |
| 4bit | 4bit/ar.safetensors |
969.2 MB | df42e72eb900 |
mixed 4 / 6 bits: AR KL 0.0074 nats (p95 0.0141), top-1 agreement 91.8 %; audio 0.78 dB log-mel, MR-STFT 0.319, SNR 14.2 dB vs bf16 (passes both marks; 4 bits throughout: KL 0.016, 87.4 %, 1.52 dB, 8.2 dB) |
| 4bit | 4bit/config.json |
1.9 kB | 35766ac3ca19 |
mixed 4 / 6 bits: AR KL 0.0074 nats (p95 0.0141), top-1 agreement 91.8 %; audio 0.78 dB log-mel, MR-STFT 0.319, SNR 14.2 dB vs bf16 (passes both marks; 4 bits throughout: KL 0.016, 87.4 %, 1.52 dB, 8.2 dB) |
| 4bit | 4bit/mlx_model.py |
54.2 kB | 457dc446be16 |
mixed 4 / 6 bits: AR KL 0.0074 nats (p95 0.0141), top-1 agreement 91.8 %; audio 0.78 dB log-mel, MR-STFT 0.319, SNR 14.2 dB vs bf16 (passes both marks; 4 bits throughout: KL 0.016, 87.4 %, 1.52 dB, 8.2 dB) |
| 4bit | 4bit/nar.safetensors |
979.2 MB | f5ddfe2e9295 |
mixed 4 / 6 bits: AR KL 0.0074 nats (p95 0.0141), top-1 agreement 91.8 %; audio 0.78 dB log-mel, MR-STFT 0.319, SNR 14.2 dB vs bf16 (passes both marks; 4 bits throughout: KL 0.016, 87.4 %, 1.52 dB, 8.2 dB) |
| 4bit | 4bit/qwen.tiktoken |
2.6 MB | b2b1b8dfb5cc |
mixed 4 / 6 bits: AR KL 0.0074 nats (p95 0.0141), top-1 agreement 91.8 %; audio 0.78 dB log-mel, MR-STFT 0.319, SNR 14.2 dB vs bf16 (passes both marks; 4 bits throughout: KL 0.016, 87.4 %, 1.52 dB, 8.2 dB) |
| 4bit | 4bit/vae.safetensors |
265.4 MB | 2a677eaee050 |
mixed 4 / 6 bits: AR KL 0.0074 nats (p95 0.0141), top-1 agreement 91.8 %; audio 0.78 dB log-mel, MR-STFT 0.319, SNR 14.2 dB vs bf16 (passes both marks; 4 bits throughout: KL 0.016, 87.4 %, 1.52 dB, 8.2 dB) |
| 4bit | 4bit/vocab.safetensors |
803.8 MB | e2072b33387e |
mixed 4 / 6 bits: AR KL 0.0074 nats (p95 0.0141), top-1 agreement 91.8 %; audio 0.78 dB log-mel, MR-STFT 0.319, SNR 14.2 dB vs bf16 (passes both marks; 4 bits throughout: KL 0.016, 87.4 %, 1.52 dB, 8.2 dB) |
| 4bit | 4bit/yue2_generation_config.json |
0.5 kB | 203830cebde6 |
mixed 4 / 6 bits: AR KL 0.0074 nats (p95 0.0141), top-1 agreement 91.8 %; audio 0.78 dB log-mel, MR-STFT 0.319, SNR 14.2 dB vs bf16 (passes both marks; 4 bits throughout: KL 0.016, 87.4 %, 1.52 dB, 8.2 dB) |
How to run
YuE2-3B (m-a-p's music model: lyrics and a style prompt in, a full song with vocals out, 48 kHz stereo) converted to
MLX for Apple Silicon, in three sizes. Each folder is complete and runs with the mlx_model.py inside it: no PyTorch.
These files were converted and quantized by Moonbox from m-a-p/YuE2-3B
(revision 8f24312f) and m-a-p/YuE2-Vae (revision 152733a1). They are
not an official m-a-p release. The weights are CC BY-NC 4.0: non-commercial use only, with attribution.
How to run
Needs mlx, numpy and tiktoken (soundfile only to write the file below). Download one folder:
from huggingface_hub import snapshot_download
folder = snapshot_download("moonbox-dev/YuE2-3B-MLX", allow_patterns="6bit/*") + "/6bit"
import sys; sys.path.insert(0, folder)
from mlx_model import Pipeline
pipe = Pipeline(folder)
audio, info = pipe.generate(
style="indie pop, warm female vocal, acoustic guitar, soft drums, hopeful",
lyrics="[Verse]\nMorning light across the kitchen floor\nCoffee steam and an open door\n\n[Chorus]\nSo here I go, here I go\n",
seconds=60, # an upper bound; leave it out and the model ends the song itself (at most 6 minutes)
seed=0,
)
import soundfile as sf
sf.write("song.flac", audio.T, pipe.sample_rate) # audio: float32 [2, samples], 48 kHz
print(info["abc"]) # the score the model wrote before the music
or from a shell: python 6bit/mlx_model.py 6bit --style "..." --lyrics @lyrics.txt --seconds 60 --out song.wav.
- Lyrics use section tags on their own lines (
[Intro],[Verse],[Pre-Chorus],[Chorus],[Bridge],[Interlude],[Outro]); English and Mandarin are what the model was trained on. Style is a comma-separated description (genre, mood, instruments, voice). cot="full"(default) makes the model write a chord-annotated ABC score first and then the music;"melody"writes a score without chords;"off"skips the score.cfg_scalesets classifier-free guidance of the music tokens (default: none with a score, 1.01 without, as the publisher's package).on_progress(stage, done, total)reports the four stages (plan: score tokens,compose: music tokens, 25 per second,render: 64 evaluations of the acoustic model,decode: audio tiles; for the first twototalis the upper bound).should_stop()is polled throughout; returning True raisesmlx_model.Cancelled.- The same seed gives the same song on the same build. It does not give the song the PyTorch package makes from that seed: the sampling rules are the publisher's, the random stream is numpy's.
secondscuts the song off; the model still plans a whole song from the lyrics, so give it lyrics that fit.
What is in a folder
| File | What |
|---|---|
ar.safetensors |
the "AR expert": the half of the 28-layer transformer that writes the score and the music tokens |
nar.safetensors |
the "NAR expert": the half that turns the tokens into audio latents by flow matching (32 midpoint steps), with its small latent and timestep layers |
vocab.safetensors |
token embedding and output head (184704 tokens), 8 bits; the same file in every folder |
vae.safetensors |
the YuE2-Vae decoder, float32; the same file in every folder |
qwen.tiktoken, yue2_generation_config.json |
the tokenizer and the publisher's sampling defaults, unchanged |
mlx_model.py |
the whole runtime: tokenizer, both experts, sampler, solver, decoder, pipeline |
config.json |
architecture, what is quantized at how many bits, source revisions |
The attention and feed-forward linears of both experts are quantized with MLX's affine quantization (group size 64); norms and the latent / timestep layers stay bf16, the VAE float32. Only one of the three large parts is in memory at a time.
The three builds
| Folder | Size | Bits | Token model vs bf16: KL (mean / p95), top-1 | Audio vs bf16: log-mel, MR-STFT, SNR | 60 s song on an M5 | Peak memory | Status |
|---|---|---|---|---|---|---|---|
8bit/ |
4.08 GB | 8 | 0.0009 / 0.0019 nats, 97.2 % | 0.16 dB, 0.089, 25.7 dB | 107 s | 2.6 GB | passes |
6bit/ |
3.37 GB | 6 | 0.0018 / 0.0034 nats, 96.0 % | 0.34 dB, 0.166, 17.9 dB | 133 s | 2.2 GB | passes |
4bit/ |
3.02 GB | 4, with 14 of each expert's 28 layers at 6 | 0.0074 / 0.0141 nats, 91.8 % | 0.78 dB, 0.319, 14.2 dB | 127 s | 2.0 GB | passes; clearly further from bf16 than 6bit/ |
Each size includes the two files every folder shares (0.80 GB vocab.safetensors, 0.27 GB vae.safetensors).
How they were checked. Two songs made by the same model are never the same, so the builds are not compared song
against song. Everything is measured against the unquantized (bf16) weights run through the same mlx_model.py:
- Token model. The bf16 model writes a score and 60 s of music tokens for an English and a Mandarin prompt. Each build then predicts every one of those 3000 music tokens given the true earlier ones. KL is the distance between the two predicted distributions (0 = identical), top-1 how often the most likely token is the same. Our mark: mean KL at most 0.01 nats and top-1 at least 90 %.
- Audio. Turning tokens into sound is deterministic once the noise is fixed, so here the waveforms are compared: each build renders the same tokens from the same noise as bf16. Log-mel distance is the mean difference of the two mel spectrograms in dB; SNR compares the waveforms. For scale, two bf16 renders of the same tokens from different noise (two takes of one song) are 4.62 dB, 1.36 and -2.9 dB apart. Our mark: at most a quarter of that log-mel distance (1.16 dB). Below 16 kHz no build shifts the long-term spectrum by more than 0.3 dB.
- Why
4bit/is not 4 bits throughout. At 4 bits everywhere the token model is at KL 0.016 / top-1 87.4 % and the audio 1.52 dB (SNR 8 dB) from bf16: under both marks. The error is spread over the layers, largest in the middle of the AR expert and at the end of the NAR expert, so4bit/keeps AR layers 7 - 20 and NAR layers 14 - 27 at 6 bits (config.jsonlists the bits per layer). That costs 0.36 GB and brings both under the marks. - The port.
mlx_model.pyrun in float32 reproduces m-a-p's PyTorch code on the original weights: token distributions to 1e-5 nats, audio latents to 3e-6, the decoded audio to 102 dB SNR. As shipped it computes in bf16 like the original (29 dB from float32 PyTorch, the same order as the8bit/build's distance from bf16). - Not checked: nobody listened, and the lyrics were not checked with a speech recogniser. One 60 s song per build was generated and measured: no clipping, no silence, -14 LUFS, and the tempo and key found in the audio are those of the score the model wrote for it (83 - 84 BPM; D major or its relative minor).
Times are for one 60 s song on an Apple M5 (24 GB, fanless MacBook Air; they vary by 20 % with heat and were not
taken back to back): writing the score takes 24 - 28 s (the model plans the whole song whatever seconds is), the
music tokens 27 - 29 s (50 - 60 tokens per second; one second of music is 25 tokens), rendering 49 - 67 s, decoding
5 - 6 s. Rendering grows faster than the song's length: a 118 s song took 272 s in all with 4bit/ (2.2 GB peak).
Peak memory is MLX's own count; the whole process peaks 0.4 GB higher.
How these were made
- Source:
model.safetensorsof m-a-p/YuE2-3B (3.62 B values in bf16, sha2561d55c42c...) and of m-a-p/YuE2-Vae. - Quantized with MLX's affine quantization (
mx.quantize, group size 64): the attention (q, k, v, o) and feed-forward (gate, up, down) linears of both experts at the folder's bits; the token embedding and the output head at 8 bits in every folder (measured: no change in the token model's numbers against keeping them bf16). Kept bf16: all norm weights, the latent input / output layers, the timestep MLP. The VAE decoder is float32 with its weight normalisation folded in (fp16 is 53 dB from float32: not transparent). - Left out: the VAE encoder (not needed to generate), and
latent_pos_embed.pe, a fixed sinusoid thatmlx_model.pyrecomputes. mlx_model.pysetsMLX_ENABLE_TF32=0when it is imported (unless the variable is already set): with MLX's reduced-precision float32 matmul on M5 GPUs the decoder is 46 dB from the reference instead of 110 dB.
How these were made
From YuE2-3B at revision 8f24312f18: the original weights loaded in MLX and stored as bf16 or fp16, or
quantized with MLX's affine quantization (8bit, 6bit, 4bit, group size 64; layers MLX can't quantize, such as
convolutions and norms, stay bf16). Every build was checked against the original before upload; the sha256 column
above is what the files must hash to.
License
CC-BY-NC-4.0 (weights); code Apache-2.0. These files are a derivative of the original model and carry its license: it is copied in this repository, and it governs your use of these files. The original model card follows unchanged.
Original model card
π€ YuE2-3B
Frontier music generation with editable scores
π§ Demo Β· π³οΈ Music Arena Β· π Quick start Β· ποΈ Cover Β· π€ Edit Β· β‘ Speed Β· π Benchmarks Β· π arXiv Β· PDF Β· π SheetSage2 arXiv Β· π Citation
YuE2 is an open music generation model with frontier song quality competitive with Suno v5/v6. Turn lyrics and a style prompt into a complete song with vocals and accompaniment, then shape its melody and chords through an editable score.
π§ YuE2 needs your ears
Help evaluate YuE2 alongside leading proprietary music models in our blind listening arena. Listen to anonymous music clips, choose your preference, and contribute to our listener preference study.
Listen & vote β Β· No account needed.
State-of-the-art results on WildSongBench. YuE2 (best-of-8) achieves the highest SongBench average among all evaluated open and proprietary models: 6.9632, compared with 6.8721 for Suno v5, 6.5562 for Suno v6, and 6.4195 for Suno v6 Wild.
Frontier song quality and text alignment on 192 WildSongBench prompts. YuE2 uses symbolic planning; Bo8 means best-of-8.
September 12, 2026 comparison: 17 settings, including Suno v6 and Suno v6 Wild. Vector PDF Β· SVG Β· All nine metrics and protocols Β· Full-precision CSV.
- Compose and edit: melody + chords, melody-only, or direct generation; bring your own ABC score.
- Edit with an agent: turn musical feedback into score, style and lyric revisions, then let YuE2 render the next version. Hear the editing process.
- Run locally: 48 kHz stereo songs on a 24GB GPU, without quantization.
- Build on it: Hugging Face loading, text guidance (CFG), and separate planning and synthesis APIs.
One ARβNAR Mixture-of-Transformers backbone writes the score and semantic tokens, then generates acoustic latents through flow matching. The VAE turns them into stereo audio.
π§ Listen to YuE2
πΆ Text-to-music
Original songs generated from lyrics and a style prompt.
Cyber Metal Β· English Β· 5:00
δ»ζδΈη Β· Mandarin funk / nu-disco Β· 3:24
Passion Β· English rock Β· 3:55
All three songs use π€ YuE2-Vae.
ποΈ Cover songs
Existing songs reimagined in a new style.
Auld Lang Syne Β· Jazz-funk cover Β· 3:10
ζη«ζ°ζι£ Β· Ballad cover Β· 4:45
Jingle Bells Β· Heavy metal cover Β· 1:09
π€ Agentic editing
Explore the agentic editing demo β
Follow The Last Train through 9 steps and 14 versions, from Mandarin pop to English jazz with modern harmony and a saxophone solo built around two complete statements of βTwinkle, Twinkle, Little Star.β Hear the full songs and inspect the conversation, scores, prompts and lyrics at each step.
π Quick start
Linux Β· Python 3.10+ Β· 24GB NVIDIA GPU with BF16 support. Install the inference package:
python -m pip install huggingface-hub==0.36.2
hf download m-a-p/YuE2-3B yue2_infer-0.1.5-py3-none-any.whl --local-dir .
python -m pip install ./yue2_infer-0.1.5-py3-none-any.whl
Create Β· Cover Β· Edit & agentic edit
Load the pipeline once for the examples below:
from pathlib import Path
from yue2 import YuE2Pipeline
pipe = YuE2Pipeline.from_pretrained("m-a-p/YuE2-3B", device="cuda")
πΆ Create
Turn a style prompt and lyrics into a complete song with vocals and accompaniment.
Use the style and full lyrics from δ»ζδΈη , the funk / nu-disco demo above.
import json
from huggingface_hub import hf_hub_download
repo = "m-a-p/YuE2-3B"
prompt_path = hf_hub_download(repo, "examples/tonight-awake.json")
demo = json.loads(Path(prompt_path).read_text(encoding="utf-8"))
style, lyrics = demo["style"], demo["lyrics"]
song = pipe(style=style, lyrics=lyrics, cot="full", seed=demo["seed"])
song.save("song.flac")
song.save_artifacts("outputs/song") # ABC, tokens, latents, audio and settings
Defaults are ready to use: cot="full" and π€ YuE2-Vae.
| Option | What it does |
|---|---|
cot="full" |
Melody + chord planning (default) |
cot="melody" |
Melody-only planning; recommended for covers |
cot="off" |
Generate without a symbolic plan |
cfg_scale=1.2 |
Experiment with stronger text guidance |
ποΈ Cover
Start from an existing recording and give it a new arrangement.
For covers, we recommend melody-only mode (cot="melody").
- Get the score: transcribe the existing song with π€ SheetSage2 and save its melody ABC without chord symbols as
melody.abc. - Get the lyrics: ask an agent to find them online, or transcribe the singing with Qwen3-ASR or the Gemini API. Check the words, organize them into sections matching the recording, and save them as
cover_lyrics.txt. - Choose a target style and generate: review or edit the score and lyrics, then supply them to YuE2 with
cot="melody"and your target style prompt.
Run the transcription tools in their own environments, then use the YuE2 pipeline:
cover = pipe(
style="Jazz-funk, warm lead vocal, Rhodes piano, electric bass, tight drums",
lyrics=Path("cover_lyrics.txt").read_text(encoding="utf-8"),
abc=Path("melody.abc").read_text(encoding="utf-8"),
cot="melody", seed=831001,
)
cover.save("cover.flac")
cover.save_artifacts("outputs/cover")
cot="melody" does not remove chord symbols automatically. Use cot="full" if you want to supply the original or edited harmony as well.
πΌ Edit & agentic edit
Edit the ABC yourself, or give an agent the score, original prompt and lyrics, and your requested changes. The agent can reharmonize, develop a solo, or adapt the lyrics and style; YuE2 renders each revision. Hear the multi-turn editing demo.
For the song from Create, copy outputs/song/score.abc to edited.abc. To obtain a plan before generating audio, use the same prompt and seed:
plan = pipe.plan(style=style, lyrics=lyrics, cot="full", seed=demo["seed"])
plan.save("original_plan")
# Keep the original and edit a copy as edited.abc.
For strict reharmonization, ask the agent to preserve melody pitches and rhythm and check sustained notes against the new chords. Allow selected melody or lyric changes for a broader adaptation. After reviewing edited.abc, regenerate with the revised style; this example keeps the lyrics and seed from Create:
edited_style = (
"Jazz, expressive lead vocal, piano, tenor saxophone, upright bass, "
"brushed drums, no guitar, spacious modern harmony"
)
song = pipe(style=edited_style, lyrics=lyrics, cot="full", seed=demo["seed"],
abc=Path("edited.abc").read_text(encoding="utf-8"))
song.save("edited.flac")
song.save_artifacts("outputs/edited")
βοΈ CFG, individual stages, and decoder selection
Generation shows English progress messages by default, including the current stage, elapsed time, and token throughput. To disable them, use YuE2Pipeline.from_pretrained(repo, progress=False) in Python or yue2 generate --quiet / yue2 batch --quiet on the command line.
Each CoT mode selects its native instruction. Semantic CFG defaults to 1.0 for full/melody and 1.01 for off; ABC sampling uses no CFG.
pipe.plan() β pipe.generate_semantic(plan) β pipe.synthesize(semantic) β pipe.decode(latents). Call pipe.close() when finished. For the benchmark decoder, pass vae="m-a-p/YuE2-Vae-legacy" to from_pretrained.
β‘ Speed and resources
A 3.6-minute song in 71 seconds on an RTX 4090. The HF package uses PyTorch, CUDA graphs, and FlashAttention, with BF16 AR/NAR and FP32 VAE.
| GPU | CoT | Warm samples | LM tokens/s | Generation / audio | Peak VRAM |
|---|---|---|---|---|---|
| RTX 4090 24GB | full | 32 | 139.48 | 71.04 / 214.85 s | 11.18 GiB |
| RTX 4090 24GB | melody | 32 | 139.32 | 68.68 / 214.67 s | 11.02 GiB |
| RTX 4090 24GB | off | 32 | 121.07 | 57.91 / 196.88 s | 11.09 GiB |
| H800 80GB | full | 1 | 164.38 | 54.74 / 224.96 s | 10.34 GiB |
One song at a time. Use a 24GB GPU and 24GB available host RAM; maximum-context testing peaked at 14.08 GiB.
H800 server Β· vLLM 0.19 Β· full CoT. This separate serving runtime handles concurrent requests:
| AR concurrency limit | LM system tokens/s | Songs/hour | Peak VRAM |
|---|---|---|---|
| 1 | 378.42 | 119.43 | 78.55 GiB |
| 16 | 2418.63 | 340.38 | 78.66 GiB |
| 32 | 3231.74 | 373.53 | 76.61 GiB |
π Measurement details
HF: PyTorch 2.10, Transformers 4.57.6, no quantization, default YuE2-Vae. 4090 values average 32 warm requests per mode; H800 is a one-song HF download-and-generation check. Times are synchronized pipeline calls, excluding initial path resolution and saving. NVML records the full-run GPU peak.
Server: 32 songs per row; AR/NAR use PyTorch 2.10 and Triton 3.6, VAE uses PyTorch 2.6. Songs/hour is warm batch throughput through all stages, not request latency. TPS counts output tokens once, excluding prefixes, supplied ABC, and the second CFG branch. Memory includes reserved KV cache. This server is separate from the HF quick start.
π Benchmarks
π WildSongBench Β· full-song generation
π Open models
| Model | Musicality β | SongBench Avg β | MuLan β | AllMusicCaps β | Q3O β | PER β |
|---|---|---|---|---|---|---|
| YuE 1 | 4.0847 | 4.9165 | 0.2623 | 0.2882 | 3.7301 | 36.38% |
| SongBloom | 3.4493 | 4.2350 | 0.2697 | 0.1926 | 3.0287 | 19.19% |
| LeVo 2 | 5.4590 | 6.3247 | 0.3542 | 0.2680 | 3.9458 | 26.12% |
| ACE-Step 1.5 | 5.1588 | 6.0118 | 0.4372 | 0.3869 | 4.5809 | 7.46% |
| HeartMuLa | 5.4963 | 6.2483 | 0.3823 | 0.2786 | 3.4907 | 10.71% |
| DiffRhythm 2 | 4.4775 | 5.2428 | 0.3782 | 0.3255 | 4.0870 | 18.41% |
| Muse | 5.1692 | 6.0349 | 0.3937 | 0.3466 | 4.4038 | 33.42% |
| MiniMax Music 3 | 5.3482 | 6.2830 | 0.3928 | 0.3609 | 4.4362 | 6.27% |
| YuE2 | 5.9075 | 6.7316 | 0.5068 | 0.4054 | 4.6819 | 8.44% |
| YuE2 (best-of-8) | 6.2666 | 6.9632 | 0.5051 | 0.3980 | 4.7009 | 9.79% |
π Proprietary models
| Model | Musicality β | SongBench Avg β | MuLan β | AllMusicCaps β | Q3O β | PER β |
|---|---|---|---|---|---|---|
| Suno v5 | 5.9918 | 6.8721 | 0.5428 | 0.4353 | 4.5907 | 8.10% |
| Suno v4.5 | 5.8317 | 6.6995 | 0.5022 | 0.3873 | 4.4149 | 5.80% |
| Suno v5.5 | 5.8087 | 6.7150 | 0.5089 | 0.3917 | 4.5914 | 5.96% |
| Suno v6 | 5.6558 | 6.5562 | 0.4916 | 0.4305 | 4.6258 | 7.58% |
| Suno v6 Wild | 5.5644 | 6.4195 | 0.4999 | 0.4316 | 4.5898 | 7.45% |
| MiniMax Music 2.6 | 5.4437 | 6.3222 | 0.4251 | 0.3670 | 4.5688 | 24.55% |
| Mureka 9 | 6.0488 | 6.9377 | 0.4394 | 0.4102 | 4.6368 | 11.69% |
| YuE2 | 5.9075 | 6.7316 | 0.5068 | 0.4054 | 4.6819 | 8.44% |
| YuE2 (best-of-8) | 6.2666 | 6.9632 | 0.5051 | 0.3980 | 4.7009 | 9.79% |
192 prompts. Both YuE2 settings use symbolic planning and π€ YuE2-Vae-legacy. Standard YuE2 selects from two candidates; best-of-8 selects from eight.
π€ SHS100K Β· zero-shot cover generation
| Method | CLEWS mAP β | CLEWS Hit@1 β | VINet mAP β | MuLan β | Musicality β |
|---|---|---|---|---|---|
| SongEcho | 0.419 | 48.4% | 0.122 | 0.366 | 3.286 |
| ACE-Step 1.5 | 0.024 | 2.4% | 0.006 | 0.166 | 3.689 |
| YuE2 (full score) | 0.647 | 71.3% | 0.288 | 0.382 | 5.104 |
| YuE2 (without chords) | 0.598 | 67.3% | 0.179 | 0.417 | 5.490 |
| YuE2 (without score) | 0.006 | 0.3% | 0.004 | 0.474 | 5.691 |
948 works Γ two styles Γ two seeds: 3,792 songs per method, without candidate selection. The score-conditioned variants use supplied source scores; all YuE2 variants use YuE2-Vae-legacy.
π Evaluation protocols and metric definitions
WSB: SongBench Avg averages seven dimensions; Q3O measures prompt adherence on a 0β5 scale; PER is phoneme error rate. YuE2 selects the lower-PER candidate from two. Best-of-8 selects by Musicality β Q3O β PER. Each candidate's PER uses the lowest-PER of four ASR passes. A pipeline call generates one candidate; selection is separate. Q3O weights differ on 10 of 192 prompts between the two YuE2 settings. Bold marks the best value within each table. Open baselines, Suno v6, and Suno v6 Wild use two candidates and four ASR passes per candidate, followed by lower-PER selection; earlier proprietary systems retain their delivered-candidate protocols. MiniMax Music 3 uses its official caption rewriter. SongBloom uses a fixed audio prompt rather than a style-text input.
SHS100K: CLEWS and Discogs-VINet measure preserved song identity against 10,545 recordings after source exclusion. MuLan measures target-style similarity; Musicality is from SongBench. Identity and quality should be read together. Every displayed metric covers all 3,792 outputs per method; incomplete Q3O scores are omitted. Source-score extraction is separate from this kit.
The overview plot combines SongBench and SongEval for quality, and MuLan, AllMusicCaps, and Q3O for alignment. These are automatic benchmark results under the stated candidate-selection protocols.
π Choosing a VAE
In our comparisons, π€ YuE2-Vae-legacy achieves higher musicality scores on benchmarks, while π€ YuE2-Vae delivers better perceptual audio quality. We recommend YuE2-Vae by default; use YuE2-Vae-legacy when reproducing the paper's benchmark results.
π Citation
Please cite the YuE2 paper when using YuE2-3B in your research:
@article{yuan2026yue2,
title = {{YuE2}: Unifying Symbolic and Audio Music Generation at Frontier Quality},
author = {Yuan, Ruibin and Pan, Jiahao and Jiang, Junyan and Wu, Zhiyue and Zhou, Ziya and Sun, Jiankai and Li, Yizhi and Zhang, Ge and Gu, Yicheng and Tian, Zeyue and Dai, Junyu and Lin, Hanfeng and Li, Kai and Wu, Shangda and Liu, Xuanjie and Wang, Jiaming and Liu, Zihan and Wang, Yue and Ma, Yinghao and Yin, Hanzhi and Chen, Kangrui and Zhang, Xinyue and Ma, Ziyang and Liao, Mengqi and Zhao, Hejia and Huang, Guowei and Yan, Chao and Ke, Lei and Yu, Jianwei and Liu, Bei and Guo, Joe and Xue, Liumeng and Xia, Gus and Xue, Wei and Guo, Yike},
journal = {arXiv preprint arXiv:2609.33757},
year = {2026},
eprint = {2609.33757},
archivePrefix = {arXiv},
primaryClass = {eess.AS},
url = {https://arxiv.org/abs/2609.33757}
}
When using SheetSage2 for transcription, covers, or score editing, please also cite the SheetSage2 paper:
@misc{jiang2026sheetsage2,
title = {{SheetSage2}: Coherent Lead-Sheet Transcription with Synthetic Supervision},
author = {Jiang, Junyan and Yuan, Ruibin and Pan, Jiahao and Xue, Wei and Guo, Yike and Xia, Gus and LeCun, Yann},
year = {2026},
eprint = {2610.05336},
archivePrefix = {arXiv},
primaryClass = {cs.SD},
url = {https://arxiv.org/abs/2610.05336}
}
For the original YuE model, please cite:
@article{yuan2025yue,
title = {{YuE}: Scaling Open Foundation Models for Long-Form Music Generation},
author = {Yuan, Ruibin and Lin, Hanfeng and Guo, Shuyue and Zhang, Ge and Pan, Jiahao and Zang, Yongyi and Liu, Haohe and Liang, Yiming and Ma, Wenye and Du, Xingjian and Du, Xinrun and Ye, Zhen and Zheng, Tianyu and Jiang, Zhengxuan and Ma, Yinghao and Liu, Minghao and Tian, Zeyue and Zhou, Ziya and Xue, Liumeng and Qu, Xingwei and Li, Yizhi and Wu, Shangda and Shen, Tianhao and Ma, Ziyang and Zhan, Jun and Wang, Chunhui and Wang, Yatian and Chi, Xiaowei and Zhang, Xinyue and Yang, Zhenzhu and Wang, Xiangzhou and Liu, Shansong and Mei, Lingrui and Li, Peng and Wang, Junjie and Yu, Jianwei and Pang, Guojian and Li, Xu and Wang, Zihao and Zhou, Xiaohuan and Yu, Lijun and Benetos, Emmanouil and Chen, Yong and Lin, Chenghua and Chen, Xie and Xia, Gus and Zhang, Zhaoxiang and Zhang, Chao and Chen, Wenhu and Zhou, Xinyu and Qiu, Xipeng and Dannenberg, Roger and Liu, Jiaheng and Yang, Jian and Huang, Wenhao and Xue, Wei and Tan, Xu and Guo, Yike},
journal = {arXiv preprint arXiv:2503.08638},
year = {2025},
eprint = {2503.08638},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2503.08638}
}
Weights: CC BY-NC 4.0. Third-party code licenses.
Quantized
Model tree for moonbox-dev/YuE2-3B-MLX
Base model
m-a-p/YuE2-3B
