A multiscale feature extraction algorithm for dysarthric speech recognition_Journal of Biomedical Engineering

Authors：

ZHAO Jianxing ,  XUE Peiyun , BAI Jing , SHI Chenkang , YUAN Bo , SHI Tongtong

School of Information and Computer Science, Taiyuan University of Technology, Taiyuan 030024, P. R. China;

Corresponding author：

XUE Peiyun, Email: 236139168@qq.com

Keywords：

Dysarthric; Fbank characteristics; Speech recognition; Empirical mode decomposition

DOI：

10.7507/1001-5515.202205049

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

In this paper, we propose a multi-scale mel domain feature map extraction algorithm to solve the problem that the speech recognition rate of dysarthria is difficult to improve. We used the empirical mode decomposition method to decompose speech signals and extracted Fbank features and their first-order differences for each of the three effective components to construct a new feature map, which could capture details in the frequency domain. Secondly, due to the problems of effective feature loss and high computational complexity in the training process of single channel neural network, we proposed a speech recognition network model in this paper. Finally, training and decoding were performed on the public UA-Speech dataset. The experimental results showed that the accuracy of the speech recognition model of this method reached 92.77%. Therefore, the algorithm proposed in this paper can effectively improve the speech recognition rate of dysarthria.

Citation： ZHAO Jianxing, XUE Peiyun, BAI Jing, SHI Chenkang, YUAN Bo, SHI Tongtong. A multiscale feature extraction algorithm for dysarthric speech recognition. Journal of Biomedical Engineering, 2023, 40(1): 44-50. doi: 10.7507/1001-5515.202205049 Copy

1.	Yu Q, Ma Y, Li Y. Enhancing speech recognition for parkinson’s disease patient using transfer learning technique. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 90-98.
2.	张涛, 蒋培培, 张亚娟, 等. 基于时频混合域局部统计的帕金森病语音障碍分析方法研究. 生物医学工程学杂志, 2021, 38(1): 21-29.
3.	Liu S, Hu S, Xie X, et al. Recent progress in the CUHK dysarthric speech recognition system. IEEE/ACM T Audio Spe, 2021, 29(99): 2267-2281.
4.	梁正友, 黎雨星, 孙宇, 等. 基于多特征组合的构音障碍语音识别. 计算机工程与设计, 2022, 43(2): 567-572.
5.	Revathi A, Nagakrishnan R, Sasikaladevi N. Comparative analysis of dysarthric speech recognition: multiple features and robust templates. Multimedia Tools Appl, 2022, 81(22): 31245-31259.
6.	Calvo I, Tropea P, Viganò M, et al. Evaluation of an automatic speech recognition platform for dysarthric speech. Folia Phoniatr Logo, 2020, 73(5): 1-10.
7.	Al-Qatab B A, Mustafa M B. Classification of dysarthric speech according to the severity of impairment: an analysis of acoustic features. IEEE Access, 2021(9): 18183-18194.
8.	Christabel S, Chellu A, Kannan P. Isolated word recognition for dysarthric patients. Commun Appl Electron, 2016, 5(2): 14-17.
9.	Chandrashekar H M, Karjigi V, Sreedevi N. Investigation of different time-frequency representations for intelligibility assessment of dysarthric speech. IEEE T Neur Sys Reh, 2020, 28(12): 2880-2889.
10.	Asemi A, Salim S, Shahamiri S R, et al. Adaptive neuro-fuzzy inference system for evaluating dysarthric automatic speech recognition (ASR) systems: a case study on mvml-based ASR. Soft comput, 2019, 23(10): 3529-3544.
11.	Selouani S A, Yakoub M S, O’Shaughnessy D. Alternative speech communication system for persons with severe speech disorders. Eurasip J Adv Sig Pr, 2009(2009): 1-12.
12.	郑纯军, 王春立, 贾宁. 语音任务下声学特征提取综述. 计算机科学, 2020, 47(5): 110-119.
13.	Zaidi B F, Boudraa M, Selouani S A, et al. Interface of an automatic recognition system for dysarthric speech. Int J Adv Comput Sci Appl, 2018, 9(9): 560-564.
14.	Ren J, Liu M. An automatic dysarthric speech recognition approach using deep neural networks. Int J Adv Comput Sci Appl, 2017, 8(12): 48-52.
15.	Misbullah A, Lin H H, Chang C Y, et al. Improving acoustic models for dysarthric speech recognition using time delay neural networks// 2020 International Conference on Electrical Engineering and Informatics (ICELTICs). Aceh: IEEE, 2020: 1-4.
16.	Chandrakala S, Rajeswari N. Representation learning based speech assistive system for persons with dysarthria. IEEE T Neur Sys Reh, 2017, 25(9): 1510-1517.
17.	王晴, 白静, 薛珮芸, 等. 听障学生和健听学生鼻韵母声学及运动学的分析研究. 生物医学工程学杂志, 2018, 35(2): 198-205.
18.	Zaidi B F, Selouani S A, Boudraa M, et al. Deep neural network architectures for dysarthric speech analysis and recognition. Neural Comput Appl, 2021, 33(15): 9089-9108.
19.	Mohammed S Y, Sid-ahmed S, Brahim-Fares Z, et al. Improving dysarthric speech recognition using empirical mode decomposition and convolutional neural network. Eurasip J Audio Spee, 2020, 2020(1): 1-7.
20.	Joy N M, Umesh S. Improving acoustic models in torgo dysarthric speech database. IEEE T Neur Sys Reh, 2018, 26(3): 637-645.
21.	Rajeswari N, Chandrakala S. Generative model-driven feature learning for dysarthric speech recognition. Biocybern Biomed Eng, 2016, 36(4): 553-561.
22.	Yue Z, Loweimi E, Christensen H, et al. Acoustic modelling from raw source and filter components for dysarthric speech recognition. IEEE/ACM T Audio Spe, 2022(30): 2968-2980.
23.	Bouchair A, Selouani S A, Amrouche A, et al. Improved empirical mode decomposition using optimal recursive averaging noise estimation for speech enhancement. Circ Syst Signal Pr, 2022, 41(1): 196-223.
24.	Fritsch J, Magimai-Doss M. Utterance verification-based dysarthric speech intelligibility assessment using phonetic posterior features. IEEE Signal Proc Let, 2021(28): 224-228.
25.	Martin L, Matus P, Eva K, et al. Efficient acoustic detector of gunshots and glass breaking. Multimed Tools Appl, 2016, 75(17): 10441-10469.
26.	Li D, Sun L, Xu X, et al. BLSTM and CNN stacking architecture for speech emotion recognition. Neural Process Lett, 2021, 53(6): 4097-4115.
27.	Haase D, Amthor M. Rethinking depthwise separable convolutions: how intra-kernel correlations lead to improved mobileNets// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2020: 14588-14597.
28.	Kim H, Hasegawa-Johnson M, Perlman A, et al. Dysarthric speech database for universal access research// INTERSPEECH 2008. Brisbane: DBLP, 2008: 1741-1744.
29.	张顺, 龚怡宏, 王进军. 深度卷积神经网络的发展及其在计算机视觉领域的应用. 计算机学报, 2019, 42(3): 453-482.
30.	Shahamiri S R, Salim S. Artificial neural networks as speech recognisers for dysarthric speech: identifying the best-performing set of MFCC parameters and studying a speaker-independent approach. Adv Eng Inform, 2014, 28(1): 102-110.
31.	Shahamiri S R. Speech vision: an end-to-end deep learning-based dysarthric automatic speech recognition system. IEEE T Neur Sys Reh, 2021(29): 852-861.

1. Yu Q, Ma Y, Li Y. Enhancing speech recognition for parkinson’s disease patient using transfer learning technique. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 90-98.
2. 张涛, 蒋培培, 张亚娟, 等. 基于时频混合域局部统计的帕金森病语音障碍分析方法研究. 生物医学工程学杂志, 2021, 38(1): 21-29.
3. Liu S, Hu S, Xie X, et al. Recent progress in the CUHK dysarthric speech recognition system. IEEE/ACM T Audio Spe, 2021, 29(99): 2267-2281.
4. 梁正友, 黎雨星, 孙宇, 等. 基于多特征组合的构音障碍语音识别. 计算机工程与设计, 2022, 43(2): 567-572.
5. Revathi A, Nagakrishnan R, Sasikaladevi N. Comparative analysis of dysarthric speech recognition: multiple features and robust templates. Multimedia Tools Appl, 2022, 81(22): 31245-31259.
6. Calvo I, Tropea P, Viganò M, et al. Evaluation of an automatic speech recognition platform for dysarthric speech. Folia Phoniatr Logo, 2020, 73(5): 1-10.
7. Al-Qatab B A, Mustafa M B. Classification of dysarthric speech according to the severity of impairment: an analysis of acoustic features. IEEE Access, 2021(9): 18183-18194.
8. Christabel S, Chellu A, Kannan P. Isolated word recognition for dysarthric patients. Commun Appl Electron, 2016, 5(2): 14-17.
9. Chandrashekar H M, Karjigi V, Sreedevi N. Investigation of different time-frequency representations for intelligibility assessment of dysarthric speech. IEEE T Neur Sys Reh, 2020, 28(12): 2880-2889.
10. Asemi A, Salim S, Shahamiri S R, et al. Adaptive neuro-fuzzy inference system for evaluating dysarthric automatic speech recognition (ASR) systems: a case study on mvml-based ASR. Soft comput, 2019, 23(10): 3529-3544.
11. Selouani S A, Yakoub M S, O’Shaughnessy D. Alternative speech communication system for persons with severe speech disorders. Eurasip J Adv Sig Pr, 2009(2009): 1-12.
12. 郑纯军, 王春立, 贾宁. 语音任务下声学特征提取综述. 计算机科学, 2020, 47(5): 110-119.
13. Zaidi B F, Boudraa M, Selouani S A, et al. Interface of an automatic recognition system for dysarthric speech. Int J Adv Comput Sci Appl, 2018, 9(9): 560-564.
14. Ren J, Liu M. An automatic dysarthric speech recognition approach using deep neural networks. Int J Adv Comput Sci Appl, 2017, 8(12): 48-52.
15. Misbullah A, Lin H H, Chang C Y, et al. Improving acoustic models for dysarthric speech recognition using time delay neural networks// 2020 International Conference on Electrical Engineering and Informatics (ICELTICs). Aceh: IEEE, 2020: 1-4.
16. Chandrakala S, Rajeswari N. Representation learning based speech assistive system for persons with dysarthria. IEEE T Neur Sys Reh, 2017, 25(9): 1510-1517.
17. 王晴, 白静, 薛珮芸, 等. 听障学生和健听学生鼻韵母声学及运动学的分析研究. 生物医学工程学杂志, 2018, 35(2): 198-205.
18. Zaidi B F, Selouani S A, Boudraa M, et al. Deep neural network architectures for dysarthric speech analysis and recognition. Neural Comput Appl, 2021, 33(15): 9089-9108.
19. Mohammed S Y, Sid-ahmed S, Brahim-Fares Z, et al. Improving dysarthric speech recognition using empirical mode decomposition and convolutional neural network. Eurasip J Audio Spee, 2020, 2020(1): 1-7.
20. Joy N M, Umesh S. Improving acoustic models in torgo dysarthric speech database. IEEE T Neur Sys Reh, 2018, 26(3): 637-645.
21. Rajeswari N, Chandrakala S. Generative model-driven feature learning for dysarthric speech recognition. Biocybern Biomed Eng, 2016, 36(4): 553-561.
22. Yue Z, Loweimi E, Christensen H, et al. Acoustic modelling from raw source and filter components for dysarthric speech recognition. IEEE/ACM T Audio Spe, 2022(30): 2968-2980.
23. Bouchair A, Selouani S A, Amrouche A, et al. Improved empirical mode decomposition using optimal recursive averaging noise estimation for speech enhancement. Circ Syst Signal Pr, 2022, 41(1): 196-223.
24. Fritsch J, Magimai-Doss M. Utterance verification-based dysarthric speech intelligibility assessment using phonetic posterior features. IEEE Signal Proc Let, 2021(28): 224-228.
25. Martin L, Matus P, Eva K, et al. Efficient acoustic detector of gunshots and glass breaking. Multimed Tools Appl, 2016, 75(17): 10441-10469.
26. Li D, Sun L, Xu X, et al. BLSTM and CNN stacking architecture for speech emotion recognition. Neural Process Lett, 2021, 53(6): 4097-4115.
27. Haase D, Amthor M. Rethinking depthwise separable convolutions: how intra-kernel correlations lead to improved mobileNets// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2020: 14588-14597.
28. Kim H, Hasegawa-Johnson M, Perlman A, et al. Dysarthric speech database for universal access research// INTERSPEECH 2008. Brisbane: DBLP, 2008: 1741-1744.
29. 张顺, 龚怡宏, 王进军. 深度卷积神经网络的发展及其在计算机视觉领域的应用. 计算机学报, 2019, 42(3): 453-482.
30. Shahamiri S R, Salim S. Artificial neural networks as speech recognisers for dysarthric speech: identifying the best-performing set of MFCC parameters and studying a speaker-independent approach. Adv Eng Inform, 2014, 28(1): 102-110.
31. Shahamiri S R. Speech vision: an end-to-end deep learning-based dysarthric automatic speech recognition system. IEEE T Neur Sys Reh, 2021(29): 852-861.

Journal of Biomedical Engineering

A multiscale feature extraction algorithm for dysarthric speech recognition

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content