Adaptable Symbolic Music Infilling with MIDI-RWKV
MIDI-RWKV model enables controllable symbolic music infilling, enhancing music creation efficiency.
Key Findings
Methodology
The study developed the MIDI-RWKV model using the RWKV-7 architecture, achieving style transfer through state tuning. The model was trained on the GigaMIDI dataset and fine-tuned on the POP909 dataset.
Key Results
- MIDI-RWKV excels in symbolic music infilling tasks, especially in long sequence processing, outperforming models like MIDI-GPT and MIDI-Mistral.
- State tuning surpasses LoRA fine-tuning in low-sample style transfer tasks.
- The model shows high success rates in attribute control effectiveness tests.
Significance
This research provides a new tool for computer-assisted music composition, addressing shortcomings in controllability and context awareness in existing systems, advancing the field of music generation.
Technical Contribution
MIDI-RWKV achieves long sequence processing and effective attribute control through the RWKV-7 architecture, offering a novel method for style transfer, significantly enhancing model adaptability.
Novelty
First application of RWKV-7 architecture in symbolic music infilling, achieving style transfer through state tuning, filling gaps in existing research.
Limitations
- The model may perform poorly with extremely complex music styles, requiring further optimization.
- Attribute control may sometimes conflict with musical context.
Future Work
Future exploration of more complex music style transfer and applications in other music generation tasks.
AI Executive Summary
Existing automatic music generation systems struggle to meet composers' iterative needs, particularly in terms of controllability and context awareness. To address this issue, researchers developed the MIDI-RWKV model, based on the RWKV-7 architecture, capable of effectively processing long sequences and achieving style transfer through state tuning.
MIDI-RWKV, trained on the GigaMIDI dataset, demonstrated excellent performance in symbolic music infilling tasks. Experimental results show that the model surpasses existing symbolic music generation models, such as MIDI-GPT and MIDI-Mistral, in long sequence processing and attribute control.
While MIDI-RWKV excels in many aspects, challenges remain in handling extremely complex music styles. Future research directions include further optimizing the model to handle more complex music styles and exploring its applications in other music generation tasks.
Deep Analysis
Background
The field of automatic music generation has made significant progress, with many studies using autoregressive Transformer models to achieve music sequence modeling. However, existing systems still lack controllability and context awareness, making them difficult to meet composers' practical needs.
Core Problem
Existing symbolic music generation models struggle to regenerate parts of compositions and perform poorly in long sequence processing. Achieving style transfer with limited samples is a significant challenge.
Innovation
MIDI-RWKV achieves long sequence processing through the RWKV-7 architecture and uses state tuning for style transfer. This method enables effective style adaptation in low-sample scenarios.
Methodology
- �� Use REMI+ encoding to convert symbolic music data into sequences.
- �� Achieve style transfer through state tuning, optimizing initial state vectors.
- �� Train the base model on the GigaMIDI dataset and fine-tune on the POP909 dataset.
Experiments
Experiments used the GigaMIDI and POP909 datasets, comparing the performance of MIDI-RWKV with other symbolic music generation models. Evaluation metrics included content preservation, groove similarity, and pitch class histogram entropy.
Results
MIDI-RWKV excels in symbolic music infilling tasks, especially in long sequence processing. State tuning surpasses LoRA fine-tuning in low-sample style transfer tasks.
Applications
MIDI-RWKV can be used for computer-assisted music composition, helping composers iterate on existing works.
Limitations & Outlook
The model may perform poorly with extremely complex music styles, requiring further optimization. Attribute control may sometimes conflict with musical context.
Plain Language Accessible to non-experts
Imagine you're in a kitchen cooking. You have a basic recipe but want to add your own style. MIDI-RWKV is like a smart assistant that can adjust ingredient proportions based on your requests, even when you have only a few samples. It helps you create unique flavors based on existing recipes.
ELI14 Explained like you're 14
Imagine you're playing a music game where you can choose different music styles and rhythms. MIDI-RWKV is like a superpower in the game, helping you quickly adjust music styles to make your creations cooler! Even if you have only a few songs, it can help you create new music pieces. Isn't that amazing?
Glossary
RWKV-7
A linear architecture capable of efficiently processing long sequences.
Used as the foundational architecture for the MIDI-RWKV model.
State Tuning
Achieving style transfer by optimizing initial state vectors.
Used for style adaptation in the MIDI-RWKV model.
Symbolic Music Infilling
Reconstructing musical content from surrounding material.
The primary task of MIDI-RWKV.
REMI+ Encoding
A method of encoding musical data, extending REMI encoding.
Used to convert symbolic music data into sequences.
Attribute Control
Achieving controllability in music generation by controlling musical attributes.
Used for music generation control in the MIDI-RWKV model.
Open Questions Unanswered questions from this research
- 1 How to optimize MIDI-RWKV for extremely complex music styles?
- 2 How to improve consistency between attribute control and musical context?
Applications
Immediate Applications
Computer-Assisted Composition
Helps composers iterate on existing works, enhancing creation efficiency.
Long-term Vision
Music Generation System Optimization
Explores generation of more complex music styles, advancing the field of music generation.
Abstract
Existing work in automatic music generation has mostly focused on end-to-end systems that generate either entire compositions or continuations of pieces, which are difficult for composers to iterate on. The area of computer-assisted composition, where generative models integrate into existing creative workflows, remains comparatively underexplored. In this study, we address the tasks of model style adaptation and multi-track, long-context, and controllable symbolic music infilling to enhance the process of computer-assisted composition. We present MIDI-RWKV, a small foundation model based on the RWKV-7 linear architecture, to enable efficient and coherent musical cocreation on edge devices. We also demonstrate that MIDI-RWKV admits an effective method of finetuning its initial state for style adaptation in the very-low-sample regime. We evaluate MIDI-RWKV and its state tuning on several quantitative and qualitative metrics with respect to existing models, and release model weights and code at https://github.com/christianazinn/MIDI-RWKV.