Direct brain-controlled multi-robot cooperation task_Journal of Biomedical Engineering

Authors：

ZHANG Chao , XIONG Xin , REN Hongjin ,  FU Yunfa

School of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, P.R.China;

Corresponding author：

FU Yunfa, Email: fyf@ynu.edu.cn

Keywords：

brain control; brain-robot interaction; multi-robot cooperation; brain switch; steady-state visual evoked potentials

DOI：

10.7507/1001-5515.201802022

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Brain control is a new control method. The traditional brain-controlled robot is mainly used to control a single robot to accomplish a specific task. However, the brain-controlled multi-robot cooperation (MRC) task is a new topic to be studied. This paper presents an experimental research which received the "Innovation Creative Award" in the brain-computer interface (BCI) brain-controlled robot contest at the World Robot Contest. Two effective brain switches were set: total control brain switch and transfer switch, and BCI based steady-state visual evoked potentials (SSVEP) was adopted to navigate a humanoid robot and a mechanical arm to complete the cooperation task. Control test of 10 subjects showed that the excellent SSVEP-BCI can be used to achieve the MRC task by appropriately setting up the brain switches. This study is expected to provide inspiration for the future practical brain-controlled MRC task system.

Citation： ZHANG Chao, XIONG Xin, REN Hongjin, FU Yunfa. Direct brain-controlled multi-robot cooperation task. Journal of Biomedical Engineering, 2018, 35(6): 943-952. doi: 10.7507/1001-5515.201802022 Copy

1.	Mcfarland D J, Vaughan T M. BCI in practice. Prog Brain Res, 2016, 228: 389-404.
2.	王行愚, 金晶, 张宇, 等. 脑控:基于脑-机接口的人机融合控制. 自动化学报, 2013, 39(3): 208-221.
3.	伏云发, 王越超, 李洪谊, 等. 直接脑控机器人接口技术. 自动化学报, 2012, 38(8): 1229-1246.
4.	明东, 蒋晟龙, 王忠鹏, 等. 基于人机信息交互的助行外骨骼机器人技术进展. 自动化学报, 2017, 43(7): 1089-1100.
5.	李松, 熊馨, 伏云发. 基于脑电信号神经反馈控制智能小车的研究. 生物医学工程学杂志, 2018, 35(1): 15-24.
6.	He Shenghong, Zhang Rui, Wang Qihong, et al. A P300-based threshold-free brain switch and its application in wheelchair control. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(6): 715-725.
7.	李松, 伏云发, 杨秋红, 等. 基于左右手运动想象单通道脑电信号的预处理研究. 生物医学工程学杂志, 2016, 33(5): 862-866.
8.	尧德中, 刘铁军, 雷旭, 等. 基于脑电的脑-机接口:关键技术和应用前景. 电子科技大学学报, 2009, 38(5): 550-554.
9.	Jin Jing, Allison B Z, Sellers E W, et al. Optimized stimulus presentation patterns for an event-related potential EEG-based brain-computer interface. Med Biol Eng Comput, 2011, 49(2): 181-191.
10.	Yan Zheng, Gao Xiaorong, Gao Shangkai. Right-and-left visual field stimulation: a frequency and space mixed coding method for SSVEP based brain-computer interface. Science China Information Sciences, 2011, 54(12): 2492-2498.
11.	Zou H L, Li Y Q, Long J Y, et al. Integrated control system based on brain-computer interfaces. Computer Engineering and Applicattons, 2012, 48(s): 76-78.
12.	Wang H T, Li Y Q, Yu T Y. Coordinated control of an intelligent wheelchair based on a brain-computer interface and speech recognition. Journal of Zhejiang University-Science C-Computers & Electronics, 2014, 15(10): 832-838.
13.	伏云发, 郭衍龙, 李松, 等. 基于SSVEP直接脑控机器人方向和速度研究. 自动化学报, 2016, 42(11): 1630-1640.
14.	Bin G, Gao Xiaorong, Yan Zheng, et al. An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. J Neural Eng, 2009, 6(4): 046002.
15.	闫铮, 宾光宇, 高小榕. 基于左右视野双频率刺激的SSVEP脑-机接口. 清华大学学报: 自然科学版网络, 2009, 49(12): 2017-2020.
16.	Cao Lei, Ju Zhengyu, Li Jie, et al. Sequence detection analysis based on canonical correlation for steady-state visual evoked potential brain computer interfaces. J Neurosci Methods, 2015, 253: 10-17.
17.	Zhang Yu, Zhou Guoxu, Jin Jing, et al. SSVEP recognition using common feature analysis in brain-computer interface. J Neurosci Methods, 2015, 244: 8-15.
18.	Chang M H, Lee J S, Heo J, et al. Eliciting dual-frequency SSVEP using a hybrid SSVEP-P300 BCI. J Neurosci Methods, 2016, 258: 104-113.
19.	Qiu Shiyuan, Li Zhijun, He Wei, et al. Brain-machine interface and visual compressive Sensing-Based teleoperation control of an exoskeleton robot. IEEE Transactions on Fuzzy Systems, 2017, 25(1): 58-69.
20.	Wang Yijun, Chen Xiaogang, Gao Xiaorong, et al. A benchmark dataset for SSVEP-based brain-computer interfaces. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(10): 1746-1752.
21.	吴正华, 尧德中. 不同颜色单色光产生的稳态视觉诱发电位的比较. 生物医学工程学杂志, 2008, 25(5): 1021-1024.
22.	Sengelmann M, Engel A K, Maye A. Maximizing information transfer in SSVEP-based brain-computer interfaces. IEEE Trans Biomed Eng, 2017, 64(2): 381-394.
23.	Zhao Xing, Zhao Dechun, Wang Xia, et al. A SSVEP stimuli encoding method using trinary frequency-shift keying encoded SSVEP (TFSK-SSVEP). Front Hum Neurosci, 2017, 11: 278.
24.	Punsawad Y, Wongsawat Y. A multi-command SSVEP-based BCI system based on single flickering frequency half-field steady-state visual stimulation. Med Biol Eng Comput, 2017, 55(6): 965-977.
25.	Li Yun, Bin Guangyu, Gao Xiaorong, et al. Analysis of phase coding SSVEP based on canonical correlation analysis (CCA). 5th International IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Conference on Neural Engineering (NER), Cancun, 2011. DOI: 10.1109/NER.2011.5910563.
26.	Merino L M, Nayak T, Hall G, et al. Detection of control or idle state with a likelihood ratio test in asynchronous SSVEP-based brain-computer interface systems. Conf Proc IEEE Eng Med Biol Soc, 2016, 2016: 1568-1571.
27.	Ramli R, Arof H, Ibrahim F, et al. Using finite state machine and a hybrid of EEG signal and EOG artifacts for an asynchronous wheelchair navigation. Expert Syst Appl, 2015, 42(5): 2451-2463.
28.	Liu Y H, Wang S H, Hu M R. A self-paced P300 healthcare brain-computer interface system with SSVEP-based switching control and kernel FDA + SVM-based detector. Applied Sciences, 2016, 6(5): 142.
29.	Spyrou L, Blokland Y, Farquhar J, et al. Optimal multitrial prediction combination and Subject-Specific adaptation for minimal training brain switch designs. IEEE Trans Neural Syst Rehabil Eng, 2016, 24(6): 700-709.
30.	Blokland Y, Spyrou L, Thijssen D, et al. Combined EEG-fNIRS decoding of motor attempt and imagery for brain switch control: an offline study in patients with tetraplegia. IEEE Trans Neural Syst Rehabil Eng, 2014, 22(2): 222-229.
31.	Lim J H, Kim Y W, Lee J H, et al. An emergency call system for patients in locked-in state using an SSVEP-based brain switch. Psychophysiology, 2017, 54(11): 1632-1643.
32.	Pan Jiahui, Li Yuanqing, Zhang Rui, et al. Discrimination between control and idle states in asynchronous SSVEP-based brain switches: a pseudo-key-based approach. IEEE Trans Neural Syst Rehabil Eng, 2013, 21(3): 435-443.

1. Mcfarland D J, Vaughan T M. BCI in practice. Prog Brain Res, 2016, 228: 389-404.
2. 王行愚, 金晶, 张宇, 等. 脑控:基于脑-机接口的人机融合控制. 自动化学报, 2013, 39(3): 208-221.
3. 伏云发, 王越超, 李洪谊, 等. 直接脑控机器人接口技术. 自动化学报, 2012, 38(8): 1229-1246.
4. 明东, 蒋晟龙, 王忠鹏, 等. 基于人机信息交互的助行外骨骼机器人技术进展. 自动化学报, 2017, 43(7): 1089-1100.
5. 李松, 熊馨, 伏云发. 基于脑电信号神经反馈控制智能小车的研究. 生物医学工程学杂志, 2018, 35(1): 15-24.
6. He Shenghong, Zhang Rui, Wang Qihong, et al. A P300-based threshold-free brain switch and its application in wheelchair control. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(6): 715-725.
7. 李松, 伏云发, 杨秋红, 等. 基于左右手运动想象单通道脑电信号的预处理研究. 生物医学工程学杂志, 2016, 33(5): 862-866.
8. 尧德中, 刘铁军, 雷旭, 等. 基于脑电的脑-机接口:关键技术和应用前景. 电子科技大学学报, 2009, 38(5): 550-554.
9. Jin Jing, Allison B Z, Sellers E W, et al. Optimized stimulus presentation patterns for an event-related potential EEG-based brain-computer interface. Med Biol Eng Comput, 2011, 49(2): 181-191.
10. Yan Zheng, Gao Xiaorong, Gao Shangkai. Right-and-left visual field stimulation: a frequency and space mixed coding method for SSVEP based brain-computer interface. Science China Information Sciences, 2011, 54(12): 2492-2498.
11. Zou H L, Li Y Q, Long J Y, et al. Integrated control system based on brain-computer interfaces. Computer Engineering and Applicattons, 2012, 48(s): 76-78.
12. Wang H T, Li Y Q, Yu T Y. Coordinated control of an intelligent wheelchair based on a brain-computer interface and speech recognition. Journal of Zhejiang University-Science C-Computers & Electronics, 2014, 15(10): 832-838.
13. 伏云发, 郭衍龙, 李松, 等. 基于SSVEP直接脑控机器人方向和速度研究. 自动化学报, 2016, 42(11): 1630-1640.
14. Bin G, Gao Xiaorong, Yan Zheng, et al. An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. J Neural Eng, 2009, 6(4): 046002.
15. 闫铮, 宾光宇, 高小榕. 基于左右视野双频率刺激的SSVEP脑-机接口. 清华大学学报: 自然科学版网络, 2009, 49(12): 2017-2020.
16. Cao Lei, Ju Zhengyu, Li Jie, et al. Sequence detection analysis based on canonical correlation for steady-state visual evoked potential brain computer interfaces. J Neurosci Methods, 2015, 253: 10-17.
17. Zhang Yu, Zhou Guoxu, Jin Jing, et al. SSVEP recognition using common feature analysis in brain-computer interface. J Neurosci Methods, 2015, 244: 8-15.
18. Chang M H, Lee J S, Heo J, et al. Eliciting dual-frequency SSVEP using a hybrid SSVEP-P300 BCI. J Neurosci Methods, 2016, 258: 104-113.
19. Qiu Shiyuan, Li Zhijun, He Wei, et al. Brain-machine interface and visual compressive Sensing-Based teleoperation control of an exoskeleton robot. IEEE Transactions on Fuzzy Systems, 2017, 25(1): 58-69.
20. Wang Yijun, Chen Xiaogang, Gao Xiaorong, et al. A benchmark dataset for SSVEP-based brain-computer interfaces. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(10): 1746-1752.
21. 吴正华, 尧德中. 不同颜色单色光产生的稳态视觉诱发电位的比较. 生物医学工程学杂志, 2008, 25(5): 1021-1024.
22. Sengelmann M, Engel A K, Maye A. Maximizing information transfer in SSVEP-based brain-computer interfaces. IEEE Trans Biomed Eng, 2017, 64(2): 381-394.
23. Zhao Xing, Zhao Dechun, Wang Xia, et al. A SSVEP stimuli encoding method using trinary frequency-shift keying encoded SSVEP (TFSK-SSVEP). Front Hum Neurosci, 2017, 11: 278.
24. Punsawad Y, Wongsawat Y. A multi-command SSVEP-based BCI system based on single flickering frequency half-field steady-state visual stimulation. Med Biol Eng Comput, 2017, 55(6): 965-977.
25. Li Yun, Bin Guangyu, Gao Xiaorong, et al. Analysis of phase coding SSVEP based on canonical correlation analysis (CCA). 5th International IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Conference on Neural Engineering (NER), Cancun, 2011. DOI: 10.1109/NER.2011.5910563.
26. Merino L M, Nayak T, Hall G, et al. Detection of control or idle state with a likelihood ratio test in asynchronous SSVEP-based brain-computer interface systems. Conf Proc IEEE Eng Med Biol Soc, 2016, 2016: 1568-1571.
27. Ramli R, Arof H, Ibrahim F, et al. Using finite state machine and a hybrid of EEG signal and EOG artifacts for an asynchronous wheelchair navigation. Expert Syst Appl, 2015, 42(5): 2451-2463.
28. Liu Y H, Wang S H, Hu M R. A self-paced P300 healthcare brain-computer interface system with SSVEP-based switching control and kernel FDA + SVM-based detector. Applied Sciences, 2016, 6(5): 142.
29. Spyrou L, Blokland Y, Farquhar J, et al. Optimal multitrial prediction combination and Subject-Specific adaptation for minimal training brain switch designs. IEEE Trans Neural Syst Rehabil Eng, 2016, 24(6): 700-709.
30. Blokland Y, Spyrou L, Thijssen D, et al. Combined EEG-fNIRS decoding of motor attempt and imagery for brain switch control: an offline study in patients with tetraplegia. IEEE Trans Neural Syst Rehabil Eng, 2014, 22(2): 222-229.
31. Lim J H, Kim Y W, Lee J H, et al. An emergency call system for patients in locked-in state using an SSVEP-based brain switch. Psychophysiology, 2017, 54(11): 1632-1643.
32. Pan Jiahui, Li Yuanqing, Zhang Rui, et al. Discrimination between control and idle states in asynchronous SSVEP-based brain switches: a pseudo-key-based approach. IEEE Trans Neural Syst Rehabil Eng, 2013, 21(3): 435-443.

Previous Article
Drug-target protein interaction prediction based on AdaBoost algorithm
Next Article
An endpoint-detection algorithm of surface electromyography insensitive to electrocardiogram interference

Journal of Biomedical Engineering

Direct brain-controlled multi-robot cooperation task

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content