Recognition of S1 and S2 heart sounds with two-stream convolutional neural networks_Journal of Biomedical Engineering

Authors：

SHEN Yujing ¹ ,  WANG Xun ^2,3 , TANG Min ¹ , LIANG Jinfu ⁴

1. Center of Arrhythmia, Fuwai Hospital, Chinese Academy of Medical Sciences, Beijing 100037, P.R.China;
2. School of Intelligent Manufacturing and CDKIP, Shanghai Dianji University, Shanghai 201306, P.R.China;
3. Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics, Chinese Academy of Sciences, Beijing 100190, P.R.China;
4. School of Physics and Electronic Science, Guizhou Normal University, Guiyang 550025, P.R.China;

Corresponding author：

WANG Xun, Email: wangxunweihua@163.com

Keywords：

deep learning; convolutional neural network; short-time Fourier transform; heart sound recognition

DOI：

10.7507/1001-5515.201909071

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Auscultation of heart sounds is an important method for the diagnosis of heart conditions. For most people, the audible component of heart sound are the first heart sound (S1) and the second heart sound (S2). Different diseases usually generate murmurs at different stages in a cardiac cycle. Segmenting the heart sounds precisely is the prerequisite for diagnosis. S1 and S2 emerges at the beginning of systole and diastole, respectively. Locating S1 and S2 accurately is beneficial for the segmentation of heart sounds. This paper proposed a method to classify the S1 and S2 based on their properties, and did not take use of the duration of systole and diastole. S1 and S2 in the training dataset were transformed to spectra by short-time Fourier transform and be feed to the two-stream convolutional neural network. The classification accuracy of the test dataset was as high as 91.135%. The highest sensitivity and specificity were 91.156% and 92.074%, respectively. Extracting the features of the input signals artificially can be avoid with the method proposed in this article. The calculation is not complicated, which makes this method effective for distinguishing S1 and S2 in real time.

Citation： SHEN Yujing, WANG Xun, TANG Min, LIANG Jinfu. Recognition of S1 and S2 heart sounds with two-stream convolutional neural networks. Journal of Biomedical Engineering, 2021, 38(1): 138-144, 153. doi: 10.7507/1001-5515.201909071 Copy

1.	Mcleod G, Shum K, Gupta T, et al. Echocardiography in congenital heart disease. Prog Cardiovasc Dis, 2018, 61(5-6): 468-475.
2.	Aquaro G D, Habtemicael Y G, Camastra G, et al. Prognostic value of repeating cardiac magnetic resonance in patients with acute myocarditis. J Am Coll Cardiol, 2019, 74(20): 2439-2448.
3.	Rosmini S, Treibel T A, Bandula S, et al. Cardiac computed tomography for the detection of cardiac amlyoidosis. J Cardiovasc Comput, 2017, 11: 155-156.
4.	Patidar S, Pachori R B. Segmentation of cardiac sound signals by removing murmurs using constrained tunable-Q wavelet transform. Biomed Signal Proces, 2013, 8: 559-567.
5.	Cheng X, Ma Y, Liu C, et al. Research on heart sound identification technology. Sci China Inf Sci, 2012, 55(2): 281-292.
6.	Lim K H, Shin Y D, Park S H, et al. Correlation of blood pressure and the ratio of S1 to S2 as measured by esophageal stethoscope and wireless bluetooth transmission. Pak J Med Sci, 2013, 29(4): 1023-1027.
7.	成谢锋, 马勇, 刘陈, 等. 心音身份识别技术的研究. 中国科学: 信息科学, 2012, 42(2): 237-251.
8.	侯雷静, 郭婷婷, 孙燕, 等. 面向心音分割的个性化高斯混合建模方法. 声学学报, 2019, 44(1): 20-27.
9.	Papadaniil C D, Hadjileontiadis L J. Efficient heart sound segmentation and Extraction using ensemble empirical mode decomposition and kurtosis features. IEEE J Biomed Health, 2014, 18(4): 1138-1152.
10.	谢稳, 姚泽阳, 邱海龙, 等. 人工智能在先天性心脏病学中的应用. 中国胸心血管外科临床杂志, 2020, 27(3): 343-353.
11.	Moukadem A, Dieterlen A, Hueber N, et al. A robust heart sounds segmentation module based on S-transform. Biomed Signal Proces, 2013, 8: 273-281.
12.	Kumar D, Carvalho P, Antunes M, et al. Wavelet transform and simplicity based heart murmur segmentation. Comput Cardiol, 2006, 33: 173-176.
13.	周润景. 模式识别与人工智能(基于MATLAB). 北京: 清华大学出版社, 2018.
14.	侯雷静. 心音信号的周期分析与状态分割. 北京: 中国科学院大学硕士学位论文, 2018.
15.	Chen T E, Yang S I, Ho L T, et al. S1 and S2 heart sound recognition using deep neural networks. IEEE T Bio-Med Eng, 2017, 64(2): 372-380.
16.	Tsao Y, Lin T H, Chen F, et al. Robust S1 and S2 heart sound recognition based on spectral restoration and multi-style training. Biomed Signal Proces, 2019, 49: 173-180.
17.	Cui K, Wu W, Huang J, et al. DOA estimation of LFM signals based on STFT and multiple invariance ESPRIT. AEU-Int J Electron C, 2017, 77: 10-17.
18.	Liu K, Ma P, An J, et al. Endpoint detection of distributed fiber sensing systems based on STFT algorithm. Opt Laser Technol, 2019, 114: 122-126.
19.	Scanlan A G. Low power & mobile hardware accelerators for deep convolutional neural network. Integration, 2019, 65: 110-127.
20.	Feng X, Qin B, Liu T. A language-independent neural network for event detection. Sci China Inf Sci, 2018, 61: 092106.
21.	Shaughnessy D O. Recognition and processing of speech signals using neural networks. Circ Syst Signal Process, 2019, 38: 3454-3481.
22.	Brahimi S, Aoun N B, Amar C B. Boosted convolutional neural network for object recognition at large scale. Neurocomputing, 2019, 33: 337-354.
23.	李端, 张洪欣, 刘知青, 等. 基于深度残差卷积神经网络的心电信号心律不齐识别. 生物医学工程学杂志, 2019, 36(2): 189-198.
24.	Acharya U R, Oh S L, Hagiwara Y, et al. A deep convolutional neural network model to classify heartbeats. Comput Biol Med, 2017, 89: 389-396.
25.	左东奇, 韩霖, 陈科, 等. 基于卷积神经网络提取超声图像甲状腺结点钙化点的研究. 生物医学工程学杂志, 2018, 35(5): 679-687.
26.	朱莉, 张丽英, 韩云涛, 等. 基于卷积神经网络的注意缺陷多动障碍分类研究. 生物医学工程学杂志, 2017, 34(1): 99-105.
27.	Qiao H, Lu C Y. Signal-background discrimination with convolutionan neural network in the PandaX-III experiment using MC simulation. Sci China Phys Mech Astron, 2018, 61(10): 101007.
28.	Zhao Z, Zhang Y, Deng Y, et al. ECG authentication system design incorporating a convolutional neural network and generalized S-Transformation. Comput Biol Med, 2018, 102: 168-179.
29.	He D, Xu K, Wang D. Design of multi-scale receptive field convolutional neural network for surface inspection of hot rolled steels. Image Vision Comput, 2019, 89: 12-20.
30.	Wang S, Chen H. A novel deep learning method for the classification of power quality disturbances using deep convolutional neural network. Appl Energ, 2019, 235: 1126-1140.
31.	陈诗慧, 刘维湘, 秦璟, 等. 基于深度学习和医学图像的癌症计算机辅助诊断研究进展. 生物医学工程学杂志, 2017, 34(2): 314-319.
32.	Mitra V, Sivaraman G, Nam H, et al. Hybrid convolutional neural networks for articulatory and acoustic information based speech recognition. Speech Commun, 2017, 89: 103-112.
33.	Lu S, Lu Z, Zhang Y D. Pathological brain detection based on AlexNet and transfer learning. J Comput Sci-Neth, 2019, 30: 41-47.
34.	Teerakawanich N, Leelaruji T, Pichetjamroen A. Short term prediction of sun coverage using optical flow with GoogLeNet. Energy Rep, 2020, 6: 526-531.
35.	Wei J, Ibrahim Y, Qian S, et al. Analyzing the impact of soft errors in VGG networks implemented on GPUs. Microelectr Reliabil, 2020, 110: 113648.
36.	Kawaguchi K, Bengio Y. Depth with nonlinearity creates no bad local minima in ResNets. Neur Netw, 2019, 118: 167-174.
37.	Kocak Y, Siray G U. New activation functions for signal layer feedforward neural network. Expert Syst Appl, 2021, 164: 113977.
38.	Jiang J, Zhang Z, Dong Q, et al. Characterization and identification of asphalt mixtures based on Convolutional Neural Network methods using X-ray scanning images. Constr Build Mater, 2018, 174: 72-80.
39.	Cao J, Pang Y, Li X, et al. Randomly translational activation inspired by the input distributions of ReLU. Neurocomputing, 2018, 275: 859-868.
40.	Wu H, Zhao J. Deep convolutional neural network model based chemical process fault diagnosis. Comput Chem Eng, 2018, 115: 185-197.
41.	Zhang Z, Jaiswal P, Rai R. FeatureNet: Machining feature recognition based on 3D Convolution Neural Network. Comput Aided Design, 2018, 101: 12-22.
42.	Goldberger A L, Amaral L A N, Glass L, et al. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 2003, 101(23): e215-e220.
43.	谭志向, 张懿, 曾德平, 等. 基于希尔伯特-黄变换的心音包络提取在LabVIEW上的实现. 生物医学工程学杂志, 2015, 32(2): 263-268.
44.	卢誉声. 移动平台深度神经网络实战: 原理、架构与优化. 北京: 机械工业出版社, 2019.

1. Mcleod G, Shum K, Gupta T, et al. Echocardiography in congenital heart disease. Prog Cardiovasc Dis, 2018, 61(5-6): 468-475.
2. Aquaro G D, Habtemicael Y G, Camastra G, et al. Prognostic value of repeating cardiac magnetic resonance in patients with acute myocarditis. J Am Coll Cardiol, 2019, 74(20): 2439-2448.
3. Rosmini S, Treibel T A, Bandula S, et al. Cardiac computed tomography for the detection of cardiac amlyoidosis. J Cardiovasc Comput, 2017, 11: 155-156.
4. Patidar S, Pachori R B. Segmentation of cardiac sound signals by removing murmurs using constrained tunable-Q wavelet transform. Biomed Signal Proces, 2013, 8: 559-567.
5. Cheng X, Ma Y, Liu C, et al. Research on heart sound identification technology. Sci China Inf Sci, 2012, 55(2): 281-292.
6. Lim K H, Shin Y D, Park S H, et al. Correlation of blood pressure and the ratio of S1 to S2 as measured by esophageal stethoscope and wireless bluetooth transmission. Pak J Med Sci, 2013, 29(4): 1023-1027.
7. 成谢锋, 马勇, 刘陈, 等. 心音身份识别技术的研究. 中国科学: 信息科学, 2012, 42(2): 237-251.
8. 侯雷静, 郭婷婷, 孙燕, 等. 面向心音分割的个性化高斯混合建模方法. 声学学报, 2019, 44(1): 20-27.
9. Papadaniil C D, Hadjileontiadis L J. Efficient heart sound segmentation and Extraction using ensemble empirical mode decomposition and kurtosis features. IEEE J Biomed Health, 2014, 18(4): 1138-1152.
10. 谢稳, 姚泽阳, 邱海龙, 等. 人工智能在先天性心脏病学中的应用. 中国胸心血管外科临床杂志, 2020, 27(3): 343-353.
11. Moukadem A, Dieterlen A, Hueber N, et al. A robust heart sounds segmentation module based on S-transform. Biomed Signal Proces, 2013, 8: 273-281.
12. Kumar D, Carvalho P, Antunes M, et al. Wavelet transform and simplicity based heart murmur segmentation. Comput Cardiol, 2006, 33: 173-176.
13. 周润景. 模式识别与人工智能(基于MATLAB). 北京: 清华大学出版社, 2018.
14. 侯雷静. 心音信号的周期分析与状态分割. 北京: 中国科学院大学硕士学位论文, 2018.
15. Chen T E, Yang S I, Ho L T, et al. S1 and S2 heart sound recognition using deep neural networks. IEEE T Bio-Med Eng, 2017, 64(2): 372-380.
16. Tsao Y, Lin T H, Chen F, et al. Robust S1 and S2 heart sound recognition based on spectral restoration and multi-style training. Biomed Signal Proces, 2019, 49: 173-180.
17. Cui K, Wu W, Huang J, et al. DOA estimation of LFM signals based on STFT and multiple invariance ESPRIT. AEU-Int J Electron C, 2017, 77: 10-17.
18. Liu K, Ma P, An J, et al. Endpoint detection of distributed fiber sensing systems based on STFT algorithm. Opt Laser Technol, 2019, 114: 122-126.
19. Scanlan A G. Low power & mobile hardware accelerators for deep convolutional neural network. Integration, 2019, 65: 110-127.
20. Feng X, Qin B, Liu T. A language-independent neural network for event detection. Sci China Inf Sci, 2018, 61: 092106.
21. Shaughnessy D O. Recognition and processing of speech signals using neural networks. Circ Syst Signal Process, 2019, 38: 3454-3481.
22. Brahimi S, Aoun N B, Amar C B. Boosted convolutional neural network for object recognition at large scale. Neurocomputing, 2019, 33: 337-354.
23. 李端, 张洪欣, 刘知青, 等. 基于深度残差卷积神经网络的心电信号心律不齐识别. 生物医学工程学杂志, 2019, 36(2): 189-198.
24. Acharya U R, Oh S L, Hagiwara Y, et al. A deep convolutional neural network model to classify heartbeats. Comput Biol Med, 2017, 89: 389-396.
25. 左东奇, 韩霖, 陈科, 等. 基于卷积神经网络提取超声图像甲状腺结点钙化点的研究. 生物医学工程学杂志, 2018, 35(5): 679-687.
26. 朱莉, 张丽英, 韩云涛, 等. 基于卷积神经网络的注意缺陷多动障碍分类研究. 生物医学工程学杂志, 2017, 34(1): 99-105.
27. Qiao H, Lu C Y. Signal-background discrimination with convolutionan neural network in the PandaX-III experiment using MC simulation. Sci China Phys Mech Astron, 2018, 61(10): 101007.
28. Zhao Z, Zhang Y, Deng Y, et al. ECG authentication system design incorporating a convolutional neural network and generalized S-Transformation. Comput Biol Med, 2018, 102: 168-179.
29. He D, Xu K, Wang D. Design of multi-scale receptive field convolutional neural network for surface inspection of hot rolled steels. Image Vision Comput, 2019, 89: 12-20.
30. Wang S, Chen H. A novel deep learning method for the classification of power quality disturbances using deep convolutional neural network. Appl Energ, 2019, 235: 1126-1140.
31. 陈诗慧, 刘维湘, 秦璟, 等. 基于深度学习和医学图像的癌症计算机辅助诊断研究进展. 生物医学工程学杂志, 2017, 34(2): 314-319.
32. Mitra V, Sivaraman G, Nam H, et al. Hybrid convolutional neural networks for articulatory and acoustic information based speech recognition. Speech Commun, 2017, 89: 103-112.
33. Lu S, Lu Z, Zhang Y D. Pathological brain detection based on AlexNet and transfer learning. J Comput Sci-Neth, 2019, 30: 41-47.
34. Teerakawanich N, Leelaruji T, Pichetjamroen A. Short term prediction of sun coverage using optical flow with GoogLeNet. Energy Rep, 2020, 6: 526-531.
35. Wei J, Ibrahim Y, Qian S, et al. Analyzing the impact of soft errors in VGG networks implemented on GPUs. Microelectr Reliabil, 2020, 110: 113648.
36. Kawaguchi K, Bengio Y. Depth with nonlinearity creates no bad local minima in ResNets. Neur Netw, 2019, 118: 167-174.
37. Kocak Y, Siray G U. New activation functions for signal layer feedforward neural network. Expert Syst Appl, 2021, 164: 113977.
38. Jiang J, Zhang Z, Dong Q, et al. Characterization and identification of asphalt mixtures based on Convolutional Neural Network methods using X-ray scanning images. Constr Build Mater, 2018, 174: 72-80.
39. Cao J, Pang Y, Li X, et al. Randomly translational activation inspired by the input distributions of ReLU. Neurocomputing, 2018, 275: 859-868.
40. Wu H, Zhao J. Deep convolutional neural network model based chemical process fault diagnosis. Comput Chem Eng, 2018, 115: 185-197.
41. Zhang Z, Jaiswal P, Rai R. FeatureNet: Machining feature recognition based on 3D Convolution Neural Network. Comput Aided Design, 2018, 101: 12-22.
42. Goldberger A L, Amaral L A N, Glass L, et al. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 2003, 101(23): e215-e220.
43. 谭志向, 张懿, 曾德平, 等. 基于希尔伯特-黄变换的心音包络提取在LabVIEW上的实现. 生物医学工程学杂志, 2015, 32(2): 263-268.
44. 卢誉声. 移动平台深度神经网络实战: 原理、架构与优化. 北京: 机械工业出版社, 2019.

Journal of Biomedical Engineering

Recognition of S1 and S2 heart sounds with two-stream convolutional neural networks

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content