Design and experiment of a multi-modal electroencephalogram-near infrared spectroscopy helmet for simultaneously acquiring at the same brain area_Journal of Biomedical Engineering

Authors：

XIONG Xin ¹ ,  FU Yunfa ¹ , ZHANG Xiabing ¹ , LI Song ¹ , XU Baolei ² , YIN Xuxian ²

1. School of Automation and Information Engineering, Kunming University of Science and Technology, Kunming 650500, P.R.China;
2. State Key Laboratory of Robotics, Shenyang Institute of Automation (SIA), Chinese Academy of Sciences (CAS), Shenyang 110016, P.R.China;

Corresponding author：

FU Yunfa, Email: fyf@ynu.edu.cn

Keywords：

multi-modal; electroencephalogram-near infrared spectroscopy helmet; electroencephalogram; near infrared spectroscopy; brain-computer interface

DOI：

10.7507/1001-5515.201611025

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Multi-modal brain-computer interface and multi-modal brain function imaging are developing trends for the present and future. Aiming at multi-modal brain-computer interface based on electroencephalogram-near infrared spectroscopy (EEG-NIRS) and in order to simultaneously acquire the brain activity of motor area, an acquisition helmet by NIRS combined with EEG was designed and verified by the experiment. According to the 10-20 system or 10-20 extended system, the diameter and spacing of NIRS probe and EEG electrode, NIRS probes were aligned with C3 and C4 as the reference electrodes, and NIRS probes were placed in the middle position between EEG electrodes to simultaneously measure variations of NIRS and the corresponding variation of EEG in the same functional brain area. The clamp holder and near infrared probe were coupled by tightening a screw. To verify the feasibility and effectiveness of the multi-modal EEG-NIRS helmet, NIRS and EEG signals were collected from six healthy subjects during six mental tasks involving the right hand clenching force and speed motor imagery. These signals may reflect brain activity related to hand clenching force and speed motor imagery in a certain extent. The experiment showed that the EEG-NIRS helmet designed in the paper was feasible and effective. It not only could provide support for the multi-modal motor imagery brain-computer interface based on EEG-NIRS, but also was expected to provide support for multi-modal brain functional imaging based on EEG-NIRS.

Citation： XIONG Xin, FU Yunfa, ZHANG Xiabing, LI Song, XU Baolei, YIN Xuxian. Design and experiment of a multi-modal electroencephalogram-near infrared spectroscopy helmet for simultaneously acquiring at the same brain area. Journal of Biomedical Engineering, 2018, 35(2): 290-296. doi: 10.7507/1001-5515.201611025 Copy

1.	Wolpaw J R, Wolpaw E W. Brain-computer interfaces: principles and practice. New York: Oxford University Press, 2012: 66.
2.	伏云发, 徐保磊, 李永程. 脑-机接口: 原理与实践. 北京: 国防工业出版社, 2016: 104.
3.	Gao Shangkai. Grand challenges in EEG based brain-computer interface [EB/OL]. (2015-09-04) [2016-11-12]. http://lifesciences.ieee.org/lsg-cc/2012-ieee-life-sciences-grand-challenges-conference/presentations/session-1/September20, 2015.
4.	Fu Yunfa, Xiong Xin, Jiang Changhao, et al. Imagined hand clenching force and speed modulate brain activity and are classified by NIRS combined with EEG. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(9): 1641-1652.
5.	Izzetoglu M, Izzetoglu K, Bunce S, et al. Functional near-infrared neuroimaging. IEEE Trans Neural Syst Rehabil Eng, 2005, 13(2): 153-159.
6.	Hoshi Y. Functional near-infrared spectroscopy: current status and future prospects. J Biomed Opt, 2007, 12(6): 24-32.
7.	Coyle S M, Ward T E, Markham C M. Brain-computer interface using a simplified functional near-infrared spectroscopy system. J Neural Eng, 2007, 4(3): 219-226.
8.	Yin Xuxian, Xu Baolei, Jiang Changhao, et al. A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching. J Neural Eng, 2015, 12(3): 036004.
9.	胡汉彬, 祝晔, 蒋田仔. 基于功能近红外光谱技术的脑—机接口研究. 生物医学工程研究, 2010, 29(1): 23-26.
10.	Wriessnegger S C, Kurzmann J, Neuper C. Spatio-temporal differences in brain oxygenation between movement execution and imagery: a multichannel near-infrared spectroscopy study. Int J Psychophysiol, 2008, 67(1): 54-63.
11.	Karahan E, Rojas-Lopez P A, Bringas-Vega M L, et al. Tensor analysis and fusion of multimodal brain images. Proceedings of the IEEE, 2015, 103(9, SI): 1531-1559.
12.	Ribeiro A S, Lacerda L M, Ferreira H A. Multimodal imaging brain connectivity analysis (MIBCA) toolbox. PEERJ, 2015, 3: e1078.
13.	Selvaraj S, Mouchlianitis E, Faulkner P, et al. Presynaptic serotoninergic regulation of emotional processing: a multimodal brain imaging study. Biol Psychiatry, 2015, 78(8): 563-571.
14.	Lahat D, Adali T, Jutten C. Multimodal data fusion: an overview of methods, challenges, and prospects. Proceedings of the IEEE, 2015, 103(9, SI): 1449-1477.
15.	尧德中, 雷旭. 同步脑电-核磁共振原理与技术. 北京: 科学出版社, 2013: 1-20.
16.	Pfurtscheller G, Allison B Z, Brunner C, et al. The hybrid BCI. Front Neurosci, 2010, 4: 30. DOI: 10.3389/fnpro.2010.00003..
17.	Lee M H, Fazli S, Mehnert J, et al. Subject-dependent classification for robust idle state detection using multi-modal neuroimaging and data-fusion techniques in BCI. Pattern Recognit, 2015, 48(8): 2725-2737.
18.	Fazli S, Daehne S, Samek W, et al. Learning from more than one data source: data fusion techniques for sensorimotor rhythm-based brain-computer interfaces. Proceedings of the IEEE, 2015, 103(6, SI): 891-906.
19.	高上凯. 浅谈脑-机接口的发展现状与挑战. 中国生物医学工程学报, 2007, 26(6): 801-803, 809.
20.	尧德中, 刘铁军, 雷旭, 等. 基于脑电的脑—机接口:关键技术和应用前景. 电子科技大学学报, 2009, 38(5): 550-554.
21.	Fazli S, Mehnert J, Steinbrink J, et al. Enhanced performance by a hybrid NIRS-EEG brain computer interface. Neuroimage, 2012, 59(1): 519-529.
22.	Koo B, Lee H G, Nam Y, et al. A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery. J Neurosci Methods, 2015, 244: 26-32.
23.	Wallois F, Mahmoudzadeh M, Patil A, et al. Usefulness of simultaneous EEG-NIRS recording in language studies. Brain Lang, 2012, 121(2): 110-123.
24.	Machado A, Lina J M, Tremblay J, et al. Detection of hemodynamic responses to epileptic activity using simultaneous Electro-EncephaloGraphy (EEG)/Near Infra Red Spectroscopy (NIRS) acquisitions. Neuroimage, 2011, 56(1): 114-125.
25.	Wallois F, Patil A, Héberlé C, et al. EEG-NIRS in epilepsy in children and neonates. Neurophysiol Clin, 2010, 40(5/6): 281-292.
26.	Tomita Y, Vialatte F B, Dreyfus G, et al. Bimodal BCI using simultaneously NIRS and EEG. IEEE Trans Biomed Eng, 2014, 61(4): 1274-1284.
27.	Blokland Y, Spyrou L, Thijssen D, et al. Combined EEG-fNIRS decoding of motor attempt and imagery for brain switch control: an offline study in patients with tetraplegia. IEEE Trans Neural Syst Rehabil Eng, 2014, 22(2): 222-229.
28.	Khan M J, Hong M J, Hong K S. Decoding of four movement directions using hybrid NIRS-EEG brain-computer interface. Front Hum Neurosci, 2014, 8(244): 244.
29.	Klem G H, Luders H O, Jasper H H, et al. The ten-twenty electrode system of the International Federation of Clinical Neurophysiology. Electroencephalogr Clin Neurophysiol Suppl, 1999, 52: 3-6.
30.	Safaie J, Grebe R, Abrishami Moghaddam H, et al. Toward a fully integrated wireless wearable EEG-NIRS bimodal acquisition system. J Neural Eng, 2013, 10(5): 056001.

1. Wolpaw J R, Wolpaw E W. Brain-computer interfaces: principles and practice. New York: Oxford University Press, 2012: 66.
2. 伏云发, 徐保磊, 李永程. 脑-机接口: 原理与实践. 北京: 国防工业出版社, 2016: 104.
3. Gao Shangkai. Grand challenges in EEG based brain-computer interface [EB/OL]. (2015-09-04) [2016-11-12]. http://lifesciences.ieee.org/lsg-cc/2012-ieee-life-sciences-grand-challenges-conference/presentations/session-1/September20, 2015.
4. Fu Yunfa, Xiong Xin, Jiang Changhao, et al. Imagined hand clenching force and speed modulate brain activity and are classified by NIRS combined with EEG. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(9): 1641-1652.
5. Izzetoglu M, Izzetoglu K, Bunce S, et al. Functional near-infrared neuroimaging. IEEE Trans Neural Syst Rehabil Eng, 2005, 13(2): 153-159.
6. Hoshi Y. Functional near-infrared spectroscopy: current status and future prospects. J Biomed Opt, 2007, 12(6): 24-32.
7. Coyle S M, Ward T E, Markham C M. Brain-computer interface using a simplified functional near-infrared spectroscopy system. J Neural Eng, 2007, 4(3): 219-226.
8. Yin Xuxian, Xu Baolei, Jiang Changhao, et al. A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching. J Neural Eng, 2015, 12(3): 036004.
9. 胡汉彬, 祝晔, 蒋田仔. 基于功能近红外光谱技术的脑—机接口研究. 生物医学工程研究, 2010, 29(1): 23-26.
10. Wriessnegger S C, Kurzmann J, Neuper C. Spatio-temporal differences in brain oxygenation between movement execution and imagery: a multichannel near-infrared spectroscopy study. Int J Psychophysiol, 2008, 67(1): 54-63.
11. Karahan E, Rojas-Lopez P A, Bringas-Vega M L, et al. Tensor analysis and fusion of multimodal brain images. Proceedings of the IEEE, 2015, 103(9, SI): 1531-1559.
12. Ribeiro A S, Lacerda L M, Ferreira H A. Multimodal imaging brain connectivity analysis (MIBCA) toolbox. PEERJ, 2015, 3: e1078.
13. Selvaraj S, Mouchlianitis E, Faulkner P, et al. Presynaptic serotoninergic regulation of emotional processing: a multimodal brain imaging study. Biol Psychiatry, 2015, 78(8): 563-571.
14. Lahat D, Adali T, Jutten C. Multimodal data fusion: an overview of methods, challenges, and prospects. Proceedings of the IEEE, 2015, 103(9, SI): 1449-1477.
15. 尧德中, 雷旭. 同步脑电-核磁共振原理与技术. 北京: 科学出版社, 2013: 1-20.
16. Pfurtscheller G, Allison B Z, Brunner C, et al. The hybrid BCI. Front Neurosci, 2010, 4: 30. DOI: 10.3389/fnpro.2010.00003..
17. Lee M H, Fazli S, Mehnert J, et al. Subject-dependent classification for robust idle state detection using multi-modal neuroimaging and data-fusion techniques in BCI. Pattern Recognit, 2015, 48(8): 2725-2737.
18. Fazli S, Daehne S, Samek W, et al. Learning from more than one data source: data fusion techniques for sensorimotor rhythm-based brain-computer interfaces. Proceedings of the IEEE, 2015, 103(6, SI): 891-906.
19. 高上凯. 浅谈脑-机接口的发展现状与挑战. 中国生物医学工程学报, 2007, 26(6): 801-803, 809.
20. 尧德中, 刘铁军, 雷旭, 等. 基于脑电的脑—机接口:关键技术和应用前景. 电子科技大学学报, 2009, 38(5): 550-554.
21. Fazli S, Mehnert J, Steinbrink J, et al. Enhanced performance by a hybrid NIRS-EEG brain computer interface. Neuroimage, 2012, 59(1): 519-529.
22. Koo B, Lee H G, Nam Y, et al. A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery. J Neurosci Methods, 2015, 244: 26-32.
23. Wallois F, Mahmoudzadeh M, Patil A, et al. Usefulness of simultaneous EEG-NIRS recording in language studies. Brain Lang, 2012, 121(2): 110-123.
24. Machado A, Lina J M, Tremblay J, et al. Detection of hemodynamic responses to epileptic activity using simultaneous Electro-EncephaloGraphy (EEG)/Near Infra Red Spectroscopy (NIRS) acquisitions. Neuroimage, 2011, 56(1): 114-125.
25. Wallois F, Patil A, Héberlé C, et al. EEG-NIRS in epilepsy in children and neonates. Neurophysiol Clin, 2010, 40(5/6): 281-292.
26. Tomita Y, Vialatte F B, Dreyfus G, et al. Bimodal BCI using simultaneously NIRS and EEG. IEEE Trans Biomed Eng, 2014, 61(4): 1274-1284.
27. Blokland Y, Spyrou L, Thijssen D, et al. Combined EEG-fNIRS decoding of motor attempt and imagery for brain switch control: an offline study in patients with tetraplegia. IEEE Trans Neural Syst Rehabil Eng, 2014, 22(2): 222-229.
28. Khan M J, Hong M J, Hong K S. Decoding of four movement directions using hybrid NIRS-EEG brain-computer interface. Front Hum Neurosci, 2014, 8(244): 244.
29. Klem G H, Luders H O, Jasper H H, et al. The ten-twenty electrode system of the International Federation of Clinical Neurophysiology. Electroencephalogr Clin Neurophysiol Suppl, 1999, 52: 3-6.
30. Safaie J, Grebe R, Abrishami Moghaddam H, et al. Toward a fully integrated wireless wearable EEG-NIRS bimodal acquisition system. J Neural Eng, 2013, 10(5): 056001.

Previous Article
Novel type of unperturbed sleep monitoring scheme under pillow based on hidden Markov model
Next Article
Design of longitudinal auto-tracking of the detector on X-ray in digital radiography

Journal of Biomedical Engineering

Design and experiment of a multi-modal electroencephalogram-near infrared spectroscopy helmet for simultaneously acquiring at the same brain area

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content