LeVo: High-Quality Song Generation with Multi-Preference Alignment
LeVo framework significantly enhances song generation quality through multi-preference alignment.
Key Findings
Methodology
LeVo combines LeLM and Music Codec, using mixed and dual-track tokens for parallel modeling. It employs Direct Preference Optimization (DPO) for multi-preference alignment, addressing diverse human preferences.
Key Results
- LeVo outperforms open-source methods in FAD, MuQ-T, and MuQ-A metrics, achieving FAD of 2.68, MuQ-T of 0.34, and MuQ-A of 0.83.
- In subjective evaluations, LeVo achieves a MOS score of 3.42, significantly higher than other models.
- Ablation studies show performance drops without the AR decoder, confirming its importance.
Significance
LeVo framework has significant impact in academia and industry, addressing long-standing pain points in audio quality and instruction following.
Technical Contribution
LeVo optimizes language models through a three-stage training strategy, significantly enhancing music generation diversity and quality, offering new engineering possibilities.
Novelty
LeVo is the first to apply multi-preference DPO in song generation, significantly improving musicality and instruction following compared to existing methods.
Limitations
- Uneven data quality affects model training, leading to instability in some generated results.
- High computational cost in long-sequence generation.
Future Work
Future work can explore more diverse music styles and emotional expressions, further optimizing model computational efficiency.
AI Executive Summary
The LeVo framework combines innovative language models and music codecs to significantly enhance lyrics-to-song generation quality. Existing methods fall short in audio quality and instruction following, which LeVo addresses through multi-preference alignment. Experimental results show LeVo outperforms open-source methods across multiple metrics and competes with industry systems. In the future, LeVo is expected to break new ground in diverse music styles and emotional expressions.
Deep Analysis
Background
The music generation field has evolved from symbolic music to end-to-end generation. Early works like Jukebox and SongCreator improved musicality but faced challenges in audio quality and instruction following.
Core Problem
Existing methods struggle to generate high-quality vocals and accompaniments, especially in maintaining musicality and instruction following in long sequences.
Innovation
LeVo introduces parallel modeling of mixed and dual-track tokens, combined with DPO for multi-preference alignment, significantly enhancing music generation quality and diversity.
Methodology
- �� LeLM models mixed and dual-track tokens in parallel. • AR decoder captures finer details. • DPO method achieves multi-preference alignment.
Experiments
Trained on 2 million songs, evaluated using FAD and PER metrics. Ablation studies confirm component importance.
Results
LeVo outperforms open-source methods in FAD, MuQ-T, and MuQ-A metrics; highest MOS score in subjective evaluations.
Applications
LeVo can be used for automated music creation and lyrics-to-song generation, with broad industry application potential.
Limitations & Outlook
High computational cost in long-sequence generation; uneven data quality affects generation stability.
Plain Language Accessible to non-experts
Imagine you're in a kitchen cooking a meal. LeVo is like a smart chef assistant that automatically prepares delicious dishes based on your recipes and taste preferences. It not only follows your instructions accurately but also adjusts the flavors based on your preferences, just like LeVo adjusts music generation based on lyrics and musical styles.
ELI14 Explained like you're 14
Hey, imagine you're playing a music game. LeVo is like a super smart game assistant that automatically generates cool songs based on your lyrics and music style requests. It not only follows your instructions perfectly but also adjusts the music style based on your preferences, just like game characters change based on your choices. Isn't that awesome?
Glossary
LeLM (Language Model)
LeLM is a language model used for parallel modeling of mixed and dual-track tokens.
Core component in LeVo framework for song generation.
Music Codec
Music Codec reconstructs tokens into high-fidelity music audio.
Key component in LeVo framework for audio reconstruction.
DPO (Direct Preference Optimization)
DPO method handles diverse human preferences through semi-automatic data construction.
Used to enhance musicality and instruction following.
FAD (Fréchet Audio Distance)
FAD evaluates the similarity between generated audio and real audio.
Metric for objective model performance evaluation.
AR Decoder (Autoregressive Decoder)
AR decoder predicts dual-track tokens in parallel, capturing finer details.
Component in LeVo framework for detailed modeling.
Open Questions Unanswered questions from this research
- 1 How to further improve computational efficiency in long-sequence generation?
- 2 How to optimize model performance across different music styles?
Applications
Immediate Applications
Automated Music Creation
Musicians can use LeVo to quickly generate high-quality songs, reducing creation time.
Long-term Vision
Music Education
LeVo can be used in music education to help students understand the music creation process.
Abstract
Recent advances in large language models (LLMs) and audio language models have significantly improved music generation, particularly in lyrics-to-song generation. However, existing approaches still struggle with the complex composition of songs and the scarcity of high-quality data, leading to limitations in audio quality, musicality, instruction following, and vocal-instrument harmony. To address these challenges, we introduce LeVo, a language model based framework consisting of LeLM and Music Codec. LeLM is capable of parallel modeling of two types of tokens: mixed tokens, which represent the combined audio of vocals and accompaniment to achieve better vocal-instrument harmony, and dual-track tokens, which separately encode vocals and accompaniment for high-quality song generation. It employs two decoder-only transformers and a modular extension training strategy to prevent interference between different token types. To further enhance musicality and instruction following ability, we introduce a multi-preference alignment method based on Direct Preference Optimization (DPO). This method handles diverse human preferences through a semi-automatic data construction process and post-training. Experimental results demonstrate that LeVo significantly outperforms existing open-source methods in both objective and subjective metrics, while performing competitively with industry systems. Ablation studies further justify the effectiveness of our designs. Audio examples and source code are available at https://levo-demo.github.io and https://github.com/tencent-ailab/songgeneration.