Pathological Voice Detection and Classification Based on Multimodal Transmission Network

IF 2.5 4区医学 Q1 AUDIOLOGY & SPEECH-LANGUAGE PATHOLOGY

Journal of Voice Pub Date : 2025-05-01 DOI:10.1016/j.jvoice.2022.11.018

Lei Geng , Yan Liang , Hongfeng Shan , Zhitao Xiao , Wei Wang , Mei Wei

{"title":"Pathological Voice Detection and Classification Based on Multimodal Transmission Network","authors":"Lei Geng , Yan Liang , Hongfeng Shan , Zhitao Xiao , Wei Wang , Mei Wei","doi":"10.1016/j.jvoice.2022.11.018","DOIUrl":null,"url":null,"abstract":"<div><h3>Objectives</h3><div><span><span>Describing pronunciation features from multiple perspectives can help doctors accurately diagnose the pathological type of a patient's voice. According to the two modal information of sound signal and electroglottography (EGG) signal, this paper proposes a pathological voice detection and </span>classification algorithm based on multimodal </span>transmission network.</div></div><div><h3>Methods</h3><div>Firstly, we used the short-time Fourier transform (STFT) to map the features of the two signals, and designed the Mel filter to obtain the Mel spectogram. Then, the constructed multimodal transmission network extracted features from Mel spectogram and applied Multimodal Transfer Module (MMTM) module. Finally, the fusion layer can integrate multimodal information, and the full connection layer diagnoses and classifies voice pathology according to the fused features.</div></div><div><h3>Results</h3><div>The experiment was based on 1179 subjects in Saarbrücken voice database (SVD), and the average accuracy, recall, specificity and F1 score of pathological voice classification reached 98.02%, 98.23%, 97.82% and 97.95% respectively. Compared with other algorithms, the classification accuracy is significantly improved.</div></div><div><h3>Conclusions</h3><div>The proposed model can integrate multiple modal information to obtain more comprehensive and stable voice features and improve the accuracy of pathological voice classification. Future research will further explore in reducing the time-consuming and complexity of the model.</div></div>","PeriodicalId":49954,"journal":{"name":"Journal of Voice","volume":"39 3","pages":"Pages 591-601"},"PeriodicalIF":2.5000,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Voice","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0892199722003708","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUDIOLOGY & SPEECH-LANGUAGE PATHOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Objectives

Describing pronunciation features from multiple perspectives can help doctors accurately diagnose the pathological type of a patient's voice. According to the two modal information of sound signal and electroglottography (EGG) signal, this paper proposes a pathological voice detection and classification algorithm based on multimodal transmission network.

Methods

Firstly, we used the short-time Fourier transform (STFT) to map the features of the two signals, and designed the Mel filter to obtain the Mel spectogram. Then, the constructed multimodal transmission network extracted features from Mel spectogram and applied Multimodal Transfer Module (MMTM) module. Finally, the fusion layer can integrate multimodal information, and the full connection layer diagnoses and classifies voice pathology according to the fused features.

Results

The experiment was based on 1179 subjects in Saarbrücken voice database (SVD), and the average accuracy, recall, specificity and F1 score of pathological voice classification reached 98.02%, 98.23%, 97.82% and 97.95% respectively. Compared with other algorithms, the classification accuracy is significantly improved.

Conclusions

The proposed model can integrate multiple modal information to obtain more comprehensive and stable voice features and improve the accuracy of pathological voice classification. Future research will further explore in reducing the time-consuming and complexity of the model.

查看原文本刊更多论文

基于多模态传输网络的病态语音检测与分类

目的：从多角度描述发音特征可以帮助医生准确诊断患者嗓音的病理类型。本文根据声音信号和声门电图（EGG）信号两种模态信息，提出了一种基于多模态传输网络的病态嗓音检测与分类算法：方法：首先，利用短时傅里叶变换（STFT）对两种信号的特征进行映射，并设计 Mel 滤波器得到 Mel 频谱图。然后，构建的多模态传输网络从梅尔谱图中提取特征，并应用多模态传输模块（MMTM）。最后，融合层可以整合多模态信息，全连接层根据融合后的特征对语音病理进行诊断和分类：实验基于萨尔布吕肯语音数据库（SVD）中的 1179 名受试者，病理语音分类的平均准确率、召回率、特异性和 F1 分数分别达到 98.02%、98.23%、97.82% 和 97.95%。与其他算法相比，分类准确率明显提高：结论：所提出的模型可以整合多种模态信息，获得更全面、更稳定的语音特征，提高病理语音分类的准确性。未来的研究将进一步探索如何降低模型的耗时和复杂度。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Voice 医学-耳鼻喉科学

CiteScore

4.00

自引率

13.60%

发文量

395

审稿时长

59 days

期刊介绍： The Journal of Voice is widely regarded as the world''s premiere journal for voice medicine and research. This peer-reviewed publication is listed in Index Medicus and is indexed by the Institute for Scientific Information. The journal contains articles written by experts throughout the world on all topics in voice sciences, voice medicine and surgery, and speech-language pathologists'' management of voice-related problems. The journal includes clinical articles, clinical research, and laboratory research. Members of the Foundation receive the journal as a benefit of membership.