Research progress of microphone array based front-end speech enhancement technology for cochlear implant_Journal of Biomedical Engineering

Authors：

 CHEN Yousheng , CHEN Weifang , ZHANG Pu , CHEN Peipei

Shenzhen Institute of Information Technology, Shenzhen, Guangdong 518000, P.R.China;

Corresponding author：

CHEN Yousheng, Email: chenyoushengtsinghua@aliyun.com

Keywords：

cochlear implant; microphone array; speech enhancement; beamforming

DOI：

10.7507/1001-5515.201805050

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Microphone array based methods are gradually applied in the front-end speech enhancement and speech recognition improvement for cochlear implant in recent years. By placing several microphones in different locations in space, this method can collect multi-channel signals containing a lot of spatial position and orientation information. Microphone array can also yield specific beamforming mode to enhance desired signal and suppress ambient noise, which is particularly suitable to be applied in face-to-face conversation for cochlear implant users. And its application value has attracted more and more attention from researchers. In this paper, we describe the principle of microphone array method, analyze the microphone array based speech enhancement technologies in present literature, and further present the technical difficulties and development trend.

Citation： CHEN Yousheng, CHEN Weifang, ZHANG Pu, CHEN Peipei. Research progress of microphone array based front-end speech enhancement technology for cochlear implant. Journal of Biomedical Engineering, 2019, 36(4): 696-704. doi: 10.7507/1001-5515.201805050 Copy

1.	World Hearth Organization (WHO). Deafness and hearing loss[EB/OL]. (2018-03-15)[2019-02-20]. http://www.who.int/en/news-room/fact-sheets/detail/deafness-and-hearing-loss.
2.	银力, 屠文河, 高姗仙, 等. 耳聋与助听设备的选择. 中国医疗器械信息, 2016(5): 23-29, 63.
3.	向琳. 儿童人工耳蜗植入后康复效果及影响因素研究. 长春: 吉林大学, 2017.
4.	National Institute on Deafness and Other Communication Disorders (NIDCD). Cochlear implants[EB/OL]. (2017-03-06)[2019-02-20]. https://www.nidcd.nih.gov/health/cochlear-implants.
5.	Lu C K, Wang S W. Peak-triggered sampling circuitry for a fine-structure-aware cochlear implant//IEEE Region 10 Conference (TENCON). Penang, Malaysia: IEEE, 2017: 31-34.
6.	Langner F, Saoji A A, Büchner A, et al. Adding simultaneous stimulating channels to reduce power consumption in cochlear implants. Hear Res, 2017, 345: 96-107.
7.	Padilla M, Stupak N, Landsberger D M. Pitch ranking with different virtual channel configurations in electrical hearing. Hear Res, 2017, 348: 54-62.
8.	Guan T, Yang M, Wei Z, et al. Simulation of the optical stimulation mechanism of cochlear nerves. Journal of Tsinghua University, 2017, 57(10): 1102-1105.
9.	Jiang Bin, Xia Nan, Wang Xing, et al. Auditory responses to short-wavelength infrared neural stimulation of the rat cochlear nucleus//39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’17). Seogwipo, South Korea: IEEE, 2017: 1942-1945.
10.	Wang Jingxuan, Lu Jianren, Tian Lan. Effect of fiberoptic collimation technique on 808 nm wavelength laser stimulation of cochlear neurons. Photomed Laser Surg, 2016, 34(6): 252-257.
11.	Anderson S R, Kan A, Thakkar T, et al. Pitch magnitude estimation can predict across-ear pitch comparisons in cochlear-implant users. J Acoust Soc Amer, 2017, 141(5): 3815.
12.	Van Eyndhoven S, Francart T, Bertrand A. EEG-informed attended speaker extraction from recorded speech mixtures with application in neuro-steered hearing prostheses. IEEE Trans Biomed Eng, 2017, 64(5): 1045-1056.
13.	Ma X J, Sudanthi W, Zhou Y, et al. Simulation for training cochlear implant electrode insertion//30th IEEE International Symposium on Computer-Based Medical Systems (IEEE CBMS 2017). Thessaloniki, Greece: IEEE, 2017: 1–6.
14.	Chen Yousheng, Chen Weifang. Research on fractional delay filter and mismatch feature based on least mean square rule for CI device//9th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC). Hangzhou, China: IEEE, 2017: 308-311.
15.	Arora S V, Vig R. Comparison of speech intelligibility parameter in cochlear implants by spatial filtering and coherence function methods. International Conference on Micro-Electronics and Telecommunication Engineering (ICMETE). Ghaziabad, India: IEEE, 2016: 573-577.
16.	Wimmer W, Kompis M, Stieger C, et al. Directional microphone contralateral routing of signals in cochlear implant users: a within-subjects comparison. Ear Hear, 2017, 38(3): 368-373.
17.	Mosnier I, Mathias N, Flament J, et al. Benefit of the UltraZoom beamforming technology in noise in cochlear implant users. Eur Arch Otorhinolaryngol, 2017, 274(9): 3335-3342.
18.	Gong Qin, Chen Yousheng. Parameter selection methods of delay and beamforming for cochlear implant speech enhancement. Acoust Phys, 2011, 57(4): 542-550.
19.	Li Xingxing, Wang Dangwei, Ma Xiaoyan, et al. Robust adaptive beamforming using iterative variable loaded sample matrix inverse. Electron Lett, 2018, 54(9): 546-548.
20.	Zohourian M, Enzner G, Martin R. Binaural speaker localization integrated into an adaptive beamformer for hearing aids. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2018, 26(3): 515-528.
21.	Xiao Jinjun, Luo Zhiquan, Merks I, et al. A robust adaptive binaural beamformer for hearing devices//2017 51st Asilomar Conference on Signals, Systems, and Computers. Pacific Grove, USA: IEEE, 2017: 1885-1889.
22.	Zeng Fangang. Challenges in improving cochlear implant performance and accessibility. IEEE Trans Biomed Eng, 2017, 64(8): 1662-1664.
23.	Lockwood M E, Jones D L, Bilger R C, et al. Performance of time- and frequency-domain binaural beamformers based on recorded signals from real rooms. J Acoust Soc Am, 2004, 115(1): 379-391.
24.	Ehlers E, Goupell M J, Zheng Yi, et al. Binaural sensitivity in children who use bilateral cochlear implants. J Acoust Soc Am, 2017, 141(6): 4264-4277.
25.	Lopez-Poveda E A, Eustaquio-Martín A, Stohl J S, et al. Intelligibility in speech maskers with a binaural cochlear implant sound coding strategy inspired by the contralateral medial olivocochlear reflex. Hear Res, 2017, 348: 134-137.
26.	Sheffield B M, Schuchman G, Bernstein J G. Pre- and postoperative binaural unmasking for bimodal cochlear implant listeners. Ear Hear, 2017, 38(5): 554-567.
27.	Goupell M J, Stakhovskaya O A, Bernstein J G. Contralateral interference caused by binaurally presented competing speech in adult bilateral cochlear-implant users. Ear Hear, 2018, 39(1): 110-123.
28.	罗鑫, 傅前杰, 王仁华. 联合使用助听器和增强电子耳蜗的使用者的中文语音识别. 北京生物医学工程, 2005, 24(4): 250-253, 267.
29.	Kates J M, Weiss M R. A comparison of hearing-aid array-processing techniques. J Acoust Soc Am, 1996, 99(5): 3138-3148.
30.	Chen Yousheng, Gong Qin. Real-time spectrum estimation-based dual-channel speech-enhancement algorithm for cochlear implant. Biomed Eng Online, 2012, 11(74). DOI: 10.1186/1475-925X-11-74.
31.	Kim J, Lee H, Hoonteak L, et al. A micromachined microphone based on the electret membrane and field-effect-transistor mechano-electrical transduction. J Acoust Soc Am, 2017, 142(4): 2567.
32.	朱振岭, 陈日林, 杨亦春. 差分传声器阵列低频特性优化研究. 应用声学, 2016, 35(6): 505-510.
33.	Duan X, Giddings R P, Mansoor S, et al. Performance tolerance of IMDD DFMA PONs to channel frequency response roll-off. IEEE Photonics Technology Letters, 2017, 29(19): 1655-1658.
34.	Chen Yousheng, Gong Qin. Broadband beamforming compensation algorithm in CI front-end acquisition. Biomed Eng Online, 2013, 12(18). DOI: 10.1186/1475-925X-12-18.

1. World Hearth Organization (WHO). Deafness and hearing loss[EB/OL]. (2018-03-15)[2019-02-20]. http://www.who.int/en/news-room/fact-sheets/detail/deafness-and-hearing-loss.
2. 银力, 屠文河, 高姗仙, 等. 耳聋与助听设备的选择. 中国医疗器械信息, 2016(5): 23-29, 63.
3. 向琳. 儿童人工耳蜗植入后康复效果及影响因素研究. 长春: 吉林大学, 2017.
4. National Institute on Deafness and Other Communication Disorders (NIDCD). Cochlear implants[EB/OL]. (2017-03-06)[2019-02-20]. https://www.nidcd.nih.gov/health/cochlear-implants.
5. Lu C K, Wang S W. Peak-triggered sampling circuitry for a fine-structure-aware cochlear implant//IEEE Region 10 Conference (TENCON). Penang, Malaysia: IEEE, 2017: 31-34.
6. Langner F, Saoji A A, Büchner A, et al. Adding simultaneous stimulating channels to reduce power consumption in cochlear implants. Hear Res, 2017, 345: 96-107.
7. Padilla M, Stupak N, Landsberger D M. Pitch ranking with different virtual channel configurations in electrical hearing. Hear Res, 2017, 348: 54-62.
8. Guan T, Yang M, Wei Z, et al. Simulation of the optical stimulation mechanism of cochlear nerves. Journal of Tsinghua University, 2017, 57(10): 1102-1105.
9. Jiang Bin, Xia Nan, Wang Xing, et al. Auditory responses to short-wavelength infrared neural stimulation of the rat cochlear nucleus//39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’17). Seogwipo, South Korea: IEEE, 2017: 1942-1945.
10. Wang Jingxuan, Lu Jianren, Tian Lan. Effect of fiberoptic collimation technique on 808 nm wavelength laser stimulation of cochlear neurons. Photomed Laser Surg, 2016, 34(6): 252-257.
11. Anderson S R, Kan A, Thakkar T, et al. Pitch magnitude estimation can predict across-ear pitch comparisons in cochlear-implant users. J Acoust Soc Amer, 2017, 141(5): 3815.
12. Van Eyndhoven S, Francart T, Bertrand A. EEG-informed attended speaker extraction from recorded speech mixtures with application in neuro-steered hearing prostheses. IEEE Trans Biomed Eng, 2017, 64(5): 1045-1056.
13. Ma X J, Sudanthi W, Zhou Y, et al. Simulation for training cochlear implant electrode insertion//30th IEEE International Symposium on Computer-Based Medical Systems (IEEE CBMS 2017). Thessaloniki, Greece: IEEE, 2017: 1–6.
14. Chen Yousheng, Chen Weifang. Research on fractional delay filter and mismatch feature based on least mean square rule for CI device//9th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC). Hangzhou, China: IEEE, 2017: 308-311.
15. Arora S V, Vig R. Comparison of speech intelligibility parameter in cochlear implants by spatial filtering and coherence function methods. International Conference on Micro-Electronics and Telecommunication Engineering (ICMETE). Ghaziabad, India: IEEE, 2016: 573-577.
16. Wimmer W, Kompis M, Stieger C, et al. Directional microphone contralateral routing of signals in cochlear implant users: a within-subjects comparison. Ear Hear, 2017, 38(3): 368-373.
17. Mosnier I, Mathias N, Flament J, et al. Benefit of the UltraZoom beamforming technology in noise in cochlear implant users. Eur Arch Otorhinolaryngol, 2017, 274(9): 3335-3342.
18. Gong Qin, Chen Yousheng. Parameter selection methods of delay and beamforming for cochlear implant speech enhancement. Acoust Phys, 2011, 57(4): 542-550.
19. Li Xingxing, Wang Dangwei, Ma Xiaoyan, et al. Robust adaptive beamforming using iterative variable loaded sample matrix inverse. Electron Lett, 2018, 54(9): 546-548.
20. Zohourian M, Enzner G, Martin R. Binaural speaker localization integrated into an adaptive beamformer for hearing aids. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2018, 26(3): 515-528.
21. Xiao Jinjun, Luo Zhiquan, Merks I, et al. A robust adaptive binaural beamformer for hearing devices//2017 51st Asilomar Conference on Signals, Systems, and Computers. Pacific Grove, USA: IEEE, 2017: 1885-1889.
22. Zeng Fangang. Challenges in improving cochlear implant performance and accessibility. IEEE Trans Biomed Eng, 2017, 64(8): 1662-1664.
23. Lockwood M E, Jones D L, Bilger R C, et al. Performance of time- and frequency-domain binaural beamformers based on recorded signals from real rooms. J Acoust Soc Am, 2004, 115(1): 379-391.
24. Ehlers E, Goupell M J, Zheng Yi, et al. Binaural sensitivity in children who use bilateral cochlear implants. J Acoust Soc Am, 2017, 141(6): 4264-4277.
25. Lopez-Poveda E A, Eustaquio-Martín A, Stohl J S, et al. Intelligibility in speech maskers with a binaural cochlear implant sound coding strategy inspired by the contralateral medial olivocochlear reflex. Hear Res, 2017, 348: 134-137.
26. Sheffield B M, Schuchman G, Bernstein J G. Pre- and postoperative binaural unmasking for bimodal cochlear implant listeners. Ear Hear, 2017, 38(5): 554-567.
27. Goupell M J, Stakhovskaya O A, Bernstein J G. Contralateral interference caused by binaurally presented competing speech in adult bilateral cochlear-implant users. Ear Hear, 2018, 39(1): 110-123.
28. 罗鑫, 傅前杰, 王仁华. 联合使用助听器和增强电子耳蜗的使用者的中文语音识别. 北京生物医学工程, 2005, 24(4): 250-253, 267.
29. Kates J M, Weiss M R. A comparison of hearing-aid array-processing techniques. J Acoust Soc Am, 1996, 99(5): 3138-3148.
30. Chen Yousheng, Gong Qin. Real-time spectrum estimation-based dual-channel speech-enhancement algorithm for cochlear implant. Biomed Eng Online, 2012, 11(74). DOI: 10.1186/1475-925X-11-74.
31. Kim J, Lee H, Hoonteak L, et al. A micromachined microphone based on the electret membrane and field-effect-transistor mechano-electrical transduction. J Acoust Soc Am, 2017, 142(4): 2567.
32. 朱振岭, 陈日林, 杨亦春. 差分传声器阵列低频特性优化研究. 应用声学, 2016, 35(6): 505-510.
33. Duan X, Giddings R P, Mansoor S, et al. Performance tolerance of IMDD DFMA PONs to channel frequency response roll-off. IEEE Photonics Technology Letters, 2017, 29(19): 1655-1658.
34. Chen Yousheng, Gong Qin. Broadband beamforming compensation algorithm in CI front-end acquisition. Biomed Eng Online, 2013, 12(18). DOI: 10.1186/1475-925X-12-18.

Previous Article
Regulation of tumor cell glycometabolism and tumor therapy

Journal of Biomedical Engineering

Research progress of microphone array based front-end speech enhancement technology for cochlear implant

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Format

Content