Muskits-ESPnet：新范式歌唱语音合成综合工具包

arXiv - EE - Audio and Speech Processing Pub Date : 2024-09-11 DOI:arxiv-2409.07226

Yuning Wu, Jiatong Shi, Yifeng Yu, Yuxun Tang, Tao Qian, Yueqian Lin, Jionghao Han, Xinyi Bai, Shinji Watanabe, Qin Jin

{"title":"Muskits-ESPnet：新范式歌唱语音合成综合工具包","authors":"Yuning Wu, Jiatong Shi, Yifeng Yu, Yuxun Tang, Tao Qian, Yueqian Lin, Jionghao Han, Xinyi Bai, Shinji Watanabe, Qin Jin","doi":"arxiv-2409.07226","DOIUrl":null,"url":null,"abstract":"This research presents Muskits-ESPnet, a versatile toolkit that introduces\nnew paradigms to Singing Voice Synthesis (SVS) through the application of\npretrained audio models in both continuous and discrete approaches.\nSpecifically, we explore discrete representations derived from SSL models and\naudio codecs and offer significant advantages in versatility and intelligence,\nsupporting multi-format inputs and adaptable data processing workflows for\nvarious SVS models. The toolkit features automatic music score error detection\nand correction, as well as a perception auto-evaluation module to imitate human\nsubjective evaluating scores. Muskits-ESPnet is available at\n\\url{https://github.com/espnet/espnet}.","PeriodicalId":501284,"journal":{"name":"arXiv - EE - Audio and Speech Processing","volume":"110 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Muskits-ESPnet: A Comprehensive Toolkit for Singing Voice Synthesis in New Paradigm\",\"authors\":\"Yuning Wu, Jiatong Shi, Yifeng Yu, Yuxun Tang, Tao Qian, Yueqian Lin, Jionghao Han, Xinyi Bai, Shinji Watanabe, Qin Jin\",\"doi\":\"arxiv-2409.07226\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"This research presents Muskits-ESPnet, a versatile toolkit that introduces\\nnew paradigms to Singing Voice Synthesis (SVS) through the application of\\npretrained audio models in both continuous and discrete approaches.\\nSpecifically, we explore discrete representations derived from SSL models and\\naudio codecs and offer significant advantages in versatility and intelligence,\\nsupporting multi-format inputs and adaptable data processing workflows for\\nvarious SVS models. The toolkit features automatic music score error detection\\nand correction, as well as a perception auto-evaluation module to imitate human\\nsubjective evaluating scores. Muskits-ESPnet is available at\\n\\\\url{https://github.com/espnet/espnet}.\",\"PeriodicalId\":501284,\"journal\":{\"name\":\"arXiv - EE - Audio and Speech Processing\",\"volume\":\"110 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - EE - Audio and Speech Processing\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.07226\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - EE - Audio and Speech Processing","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.07226","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 0

摘要

本研究介绍了多功能工具包 Muskits-ESPnet，该工具包通过应用连续和离散方法中经过训练的音频模型，为歌声合成（SVS）引入了新的范例。具体而言，我们探索了从 SSL 模型和音频编解码器中衍生出来的离散表示法，在多功能性和智能性方面具有显著优势，支持多格式输入和适用于各种 SVS 模型的数据处理工作流。该工具包具有自动乐谱错误检测和纠正功能，以及一个感知自动评估模块，可模仿人类对乐谱的主观评估。Muskits-ESPnet的网址为：url{https://github.com/espnet/espnet}。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Muskits-ESPnet: A Comprehensive Toolkit for Singing Voice Synthesis in New Paradigm

This research presents Muskits-ESPnet, a versatile toolkit that introduces new paradigms to Singing Voice Synthesis (SVS) through the application of pretrained audio models in both continuous and discrete approaches. Specifically, we explore discrete representations derived from SSL models and audio codecs and offer significant advantages in versatility and intelligence, supporting multi-format inputs and adaptable data processing workflows for various SVS models. The toolkit features automatic music score error detection and correction, as well as a perception auto-evaluation module to imitate human subjective evaluating scores. Muskits-ESPnet is available at \url{https://github.com/espnet/espnet}.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

arXiv - EE - Audio and Speech Processing

自引率

0.00%

发文量