Audio melody extraction

Extracting melody from audio recordings is a difficult and important research problem, because melody is one of the most important descriptors of (Western) music. Successful audio melody extraction would bridge the gap between a large number of researches made on symbolic (MIDI) data and audio, and enable a number of applications from retrieval in audio collections, segmentation, summarization, voice removal, music learning etc.

Melody extraction algorithm

Our approach to melody extraction is based on finding significant melodic fragments throughout the analyzed piece of music and clustering these fragments according to their timbral similarity. Fragments in each cluster are taken to represent fragments belonging to the same melodic line, different clusters contain different melodic lines. Holes between significant fragments within each cluster are filled-in by a shortest-path approach over all melodic fragments.

The algorithm was also submitted to MIREX 2005 Audio Melody Extraction task.

For more details, see:

  • [PDF] M. Marolt, "Audio melody extraction based on timbral similarity of melodic fragments," in Proceedings / EUROCON 2005 - The International Conference on "Computer as a Tool", Belgrade, Serbia and Montenegro, November 21-24, 2005, 2005, pp. 1288-1291.
    [Bibtex]
    @conference{5063252,
    author={Matija Marolt},
    year={2005},
    pages={1288-1291},
    title={Audio melody extraction based on timbral similarity of melodic fragments},
    booktitle={Proceedings / EUROCON 2005 - The International Conference on "Computer as a Tool", Belgrade, Serbia and Montenegro, November 21-24, 2005},
    }
  • [PDF] M. Marolt, "Gaussian mixture models for extraction of melodic lines from audio recordings," in ISMIR 2004 : proceedings, 2004, pp. 80-83.
    [Bibtex]
    @conference{4414548,
    author={Matija Marolt},
    year={2004},
    pages={80-83},
    title={Gaussian mixture models for extraction of melodic lines from audio recordings},
    booktitle={ISMIR 2004 : proceedings},
    }
  • [PDF] M. Marolt, "On finding melodic lines in audio recordings," in Proceedings / 7th International Conference on Digital Audio Effects, DAFx´04, Naples, October 5-8, 2004, 2004, pp. 217-221.
    [Bibtex]
    @conference{4415060,
    author={Matija Marolt},
    year={2004},
    pages={217-221},
    title={On finding melodic lines in audio recordings},
    booktitle={Proceedings / 7th International Conference on Digital Audio Effects, DAFx´04, Naples, October 5-8, 2004},
    }
Some examples (see papers for evaluation)

Grouping melodic lines with similar timbre