Discogs-VI: A Musical Version Identification Dataset Based on Public Editorial Metadata

Music version recognition has evolved from simple audio fingerprinting methods like MFCC and chroma-based similarity to deep learning approaches utilizing CNNs and transformers. Prior datasets such as SHS100K and Da-TACOS provided initial benchmarks but suffered from limited scale and metadata richness. The advent of community-driven databases like Discogs offers an unprecedented opportunity to leverage detailed artist, release, and style information for large-scale dataset construction. This shift addresses longstanding challenges in capturing the diversity of musical versions, including live performances, remixes, and edits, which are crucial for real-world applications like copyright enforcement and personalized recommendations.

Despite advances, music version identification remains hindered by insufficient data scale, metadata inconsistency, and the difficulty of accurately mapping textual descriptions to audio content. Existing datasets lack the diversity needed to train models capable of handling complex scenarios, such as multiple editions, live recordings, and remixes. Additionally, the reliance on platform-specific URLs introduces instability, as videos may be removed or altered. These issues collectively limit the performance and generalization of current systems, necessitating a new approach that can leverage community-maintained metadata to build a comprehensive, scalable dataset.

This work introduces a novel framework that constructs a large-scale music version dataset solely from Discogs metadata, utilizing string normalization, artist ID expansion, and multi-stage filtering to form high-quality cliques. A high-precision mapping algorithm links these cliques to official YouTube uploads, ensuring data authenticity. The neural network model trained with triplet loss on this dataset demonstrates that simple architectures can achieve competitive performance, emphasizing the importance of data scale and quality. This approach overcomes the limitations of previous datasets by providing a diverse, richly annotated, and publicly accessible resource for music IR research.

• �� Data acquisition: Download the July 2024 Discogs dump, extract relevant metadata fields such as track title, artist, alias, genre, and release info. • String normalization: Remove diacritics, articles, punctuation, and parentheses to standardize track titles. • Artist ID expansion: Use Discogs artist IDs and aliases to unify artist identities across releases. • Clique formation: Group tracks with identical normalized titles and shared writer artists into cliques, representing versions. • High-precision mapping: Generate search queries combining artist and track info, query YouTube, and match top results based on metadata similarity. • Data filtering: Keep only official uploads, discard videos longer than 20 minutes, and ensure at least two versions per clique are available. • Model training: Use CQT features, apply triplet loss, and optimize embedding space with online hard-negative mining, training for 50 epochs on a single GPU.

The dataset was split into training, validation, and test sets based on clique intersections with Da-TACOS and SHS100K benchmarks. Hyperparameters included 512-dimensional embeddings, batch size of 48 cliques, and two versions per clique. The model was trained with AdamW optimizer, initial learning rate 1e-3, decayed exponentially, for 50 epochs. Evaluation metrics included MAP and MR1, with ablation studies on input length and feature types. Experiments demonstrated that the model trained on large-scale data outperformed baselines, confirming the effectiveness of the dataset. Additional tests on external datasets validated robustness and generalization.

The trained Discogs-VINet achieved MAP 0.443 and MR1 614.1 on the Discogs-YT test set, surpassing many existing models. The dataset's diversity enabled the model to handle complex scenarios like live versions and remixes. On Da-TACOS, the model scored MAP 0.607, MR1 24; on SHS100K, MAP 0.863, MR1 19, indicating strong generalization. The results underscore the importance of large, richly annotated datasets for deep learning in music IR. The ablation studies confirmed that even without data augmentation, the model maintained high performance, highlighting the dataset's contribution to robustness.

The dataset and model can be directly applied to copyright enforcement, music streaming platforms, and content recommendation systems. They enable accurate identification of different versions, aiding in rights management and personalized user experiences. The approach also facilitates musicological research by providing detailed version annotations. Future integration with audio content analysis will further enhance system accuracy and robustness, supporting industry-wide efforts in digital music management.

Dependence on metadata quality may lead to missed or misclassified versions, especially with inconsistent naming conventions. The YouTube mapping relies on platform stability, which could change over time, causing URL invalidation. The current approach does not incorporate audio content features, limiting discrimination in subtle cases. Handling multilingual metadata and non-English titles remains challenging, requiring further adaptation. Future work should address these issues by combining content-based features and expanding platform coverage.