Research progress of brain-computer interface application paradigms based on rapid serial visual presentation_Journal of Biomedical Engineering

Authors：

SUN Jingmin ¹ ,  MENG Jiayuan ^1,2 , YOU Jia ¹ , YANG Mingming ² , JIANG Jing ³ , XU Minpeng ^1,2 , MING Dong ^1,2

1. School of Precision Instruments and Opto-Electronic Engineering, Tianjin University, Tianjin 300072, P. R. China;
2. Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, P. R. China;
3. National Key Laboratory of Human Factors Engineering, China Astronaut Research and Training Center, Beijing 100094, P. R. China;

Corresponding author：

MENG Jiayuan, Email: mengjiayuan@tju.edu.cn

Keywords：

Rapid serial visual presentation; Brain-computer interface; Electroencephalography; Paradigm design

DOI：

10.7507/1001-5515.202305061

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Rapid serial visual presentation (RSVP) is a type of psychological visual stimulation experimental paradigm that requires participants to identify target stimuli presented continuously in a stream of stimuli composed of numbers, letters, words, images, and so on at the same spatial location, allowing them to discern a large amount of information in a short period of time. The RSVP-based brain-computer interface (BCI) can not only be widely used in scenarios such as assistive interaction and information reading, but also has the advantages of stability and high efficiency, which has become one of the common techniques for human-machine intelligence fusion. In recent years, brain-controlled spellers, image recognition and mind games are the most popular fields of RSVP-BCI research. Therefore, aiming to provide reference and new ideas for RSVP-BCI related research, this paper reviewed the paradigm design and system performance optimization of RSVP-BCI in these three fields. It also looks ahead to its potential applications in cutting-edge fields such as entertainment, clinical medicine, and special military operations.

Citation： SUN Jingmin, MENG Jiayuan, YOU Jia, YANG Mingming, JIANG Jing, XU Minpeng, MING Dong. Research progress of brain-computer interface application paradigms based on rapid serial visual presentation. Journal of Biomedical Engineering, 2023, 40(6): 1235-1241, 1248. doi: 10.7507/1001-5515.202305061 Copy

1.	Xu M P, He F, Jung P, et al. Current challenges for the practical application of electroencephalography-based brain-computer interfaces. Engineering, 2021, 7(12): 1710-1712..
2.	Torres E P, Torres E A, Hernandez-Alvarez M, et al. EEG-based BCI emotion recognition: A survey. Sensors, 2020, 20(18): 36..
3.	Meng J Y, Xu M P, Wang K, et al. Separable EEG features induced by timing prediction for active brain-computer interfaces. Sensors, 2020, 20(12): 3588..
4.	Li Z, Lan Z, Tang D, et al. Survey on ERP-based target detection. Comput Eng Appl, 2021, 57(23): 37-49..
5.	Grootswagers T, Zhou I, Robinson A K, et al. Human EEG recordings for 1,854 concepts presented in rapid serial visual presentation streams. Sci Data, 2022, 9(1): 3..
6.	Won K, Kwon M, Ahn M, et al. EEG dataset for RSVP and P300 speller brain-computer interfaces. Sci Data, 2022, 9(1): 388..
7.	Liu S, Wang W, Sheng Y, et al. Improving the cross-subject performance of the ERP-based brain-computer interface using rapid serial visual presentation and correlation analysis rank. Front Hum Neurosci, 2020, 14: 00296..
8.	Mijani A M, Shamsollahi M B, Hassani M S, et al. Comparison between single, dual and triple rapid serial visual presentation paradigms for P300 speller// 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) - Human Genomics. Madrid: IEEE, 2018: 2635-2638..
9.	Lin Z, Zhang C, Zeng Y, et al. A novel P300 BCI speller based on the triple RSVP paradigm. Sci Rep, 2018, 8(1): 3350..
10.	Mijani A M, Shamsollahi M B, Hassani M S. A novel dual and triple shifted RSVP paradigm for P300 speller. J Neurosci Methods, 2019, 328: 108420..
11.	Lees S, Mccullagh P, Payne P, et al. Speed of rapid serial visual presentation of pictures, numbers and words affects event-related potential-based detection accuracy. IEEE Trans Neural Syst Rehabil Eng, 2020, 28(1): 113-122..
12.	Fernandez-Rodriguez A, Darves-Bornoz A, Velasco-Alvarez F, et al. Effect of stimulus size in a visual ERP-based BCI under RSVP. Sensors, 2022, 22(23): 9505..
13.	Ahani A, Moghadamfalahi M, Erdogmus D. Language-model assisted and icon-based communication through a brain-computer interface with different presentation paradigms. IEEE Trans Neural Syst Rehabil Eng, 2018, 26(9): 1835-1844..
14.	Fernandez-Rodriguez A, Medina-Julia M T, Velasco-Alvarez F, et al. Different effects of using pictures as stimuli in a P300 brain-computer interface under rapid serial visual presentation or row-column paradigm. Med Biol Eng Comput, 2021, 59(4): 869-881..
15.	Ron-Angevin R, Medina-Julia M T, Fernandez-Rodriguez A, et al. Performance analysis with different types of visual stimuli in a BCI-based speller under an RSVP paradigm. Front Comput Neurosci, 2021, 14: 587702..
16.	Won D O, Hwang H J, Kim D M, et al. Motion-based rapid serial visual presentation for gaze-independent brain-computer interfaces. IEEE Trans Neural Syst Rehabil Eng, 2018, 26(2): 334-343..
17.	Jalilpour S, Hajipour Sardouie S, Mijani A. A novel hybrid BCI speller based on RSVP and SSVEP paradigm. Comput Methods Programs Biomed, 2020, 187: 105326..
18.	Gonzalez-Navarro P, Celik B, Moghadamfalahi M, et al. Feedback related potentials for EEG-based typing systems. Front Hum Neurosci, 2022, 15: 13..
19.	Zisk A H, Borgheai S B, Mclinden J, et al. P300 latency jitter and its correlates in people with amyotrophic lateral sclerosis. Clin Neurophysiol, 2021, 132(2): 632-642..
20.	Li B, Lin Y, Gao X, et al. Enhancing the EEG classification in RSVP task by combining interval model of ERPs with spatial and temporal regions of interest. J Neural Eng, 2021, 18(1): 016008..
21.	Zhang C, Qiu S, Wang S, et al. Target detection using ternary classification during a rapid serial visual presentation task using magnetoencephalography data. Front Comput Neurosci, 2021, 15: 619508..
22.	Lin Z, Zeng Y, Wang X, et al. EEG-based target detection during a multi-rapid serial visual presentation// 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER). Shanghai: IEEE, 2017: 556-559..
23.	Hou Y, Gu Z, Yu Z L, et al. Enhancement of lower limb motor imagery ability via dual-level multimodal stimulation and sparse spatial pattern decoding method. Front Hum Neurosci, 2022, 16: 975410..
24.	迟新一, 崔红岩, 陈小刚. 结合稳态视觉诱发电位的多模态脑机接口研究进展. 中国生物医学工程学报, 2022, 41(2): 204-213..
25.	Onishi A. Brain-computer interface with rapid serial multimodal presentation using artificial facial images and voice. Comput Biol Med, 2021, 136: 104685..
26.	Mao J, Qiu S, Li D, et al. A cross-modal guiding and fusion method for multi-modal RSVP-based image retrieval// 2021 International Joint Conference on Neural Networks (IJCNN). Shenzhen: IEEE, 2021: 1-7..
27.	Mao J, Qiu S, Wei W, et al. Cross-modal guiding and reweighting network for multi-modal RSVP-based target detection. Neural Networks, 2023, 161: 65-82..
28.	Song X, Zeng Y, Tong L, et al. A collaborative brain-computer interface framework for enhancing group detection performance of dynamic visual targets. Comput Intell Neurosci, 2022, 2022: 4752450..
29.	韩锦, 董博文, 刘邈, 等. 基于P300-SSVEP的双人协同脑-控机械臂汉字书写系统. 数据采集与处理, 2022, 37(6): 1401-1411..
30.	Bhattacharyya S, Valeriani D, Cinel C, et al. Anytime collaborative brain-computer interfaces for enhancing perceptual group decision-making. Sci Rep, 2021, 11(1): 17008..
31.	Zhang H, Zhu L, Xu S, et al. Two brains, one target: Design of a multi-level information fusion model based on dual-subject RSVP. J Neurosci Methods, 2021, 363: 109346..
32.	Valeriani D, O’Flynn L C, Worthley A, et al. Multimodal collaborative brain-computer interfaces aid human-machine team decision-making in a pandemic scenario. J Neural Eng, 2022, 19(5): 056036..
33.	Salvatore C, Valeriani D, Piccialli V, et al. Optimized collaborative brain-computer interfaces for enhancing face recognition. IEEE Trans Neural Syst Rehabil Eng, 2022, 30: 1223-1232..
34.	Zhang S, Sun L, Mao X, et al. Review on EEG-based authentication technology. Comput Intell Neurosci, 2021, 2021: 5229576..
35.	Wang H W, Qi Y, Yu H, et al. RCIT: An RSVP-based concealed information test framework using EEG signals. IEEE Trans Cognit Dev Syst, 2022, 14(2): 541-551..
36.	Jangwan N S, Ashraf G M, Ram V, et al. Brain augmentation and neuroscience technologies: Current applications, challenges, ethics and future prospects. Front Syst Neurosci, 2022, 16: 1000495..
37.	Zhang S, Chen X, Wang Y, et al. Visual field inhomogeneous in brain-computer interfaces based on rapid serial visual presentation. J Neural Eng, 2022, 19(1): 016015..
38.	Zokaei N, Gillebert C R, Chauvin J J, et al. Temporal orienting in Parkinson’s disease. Eur J Neurosci, 2021, 53(8): 2713-2725..
39.	Shalbaf A, Bagherzadeh S, Maghsoudi A. Transfer learning with deep convolutional neural network for automated detection of schizophrenia from EEG signals. Phys Eng Sci Med, 2020, 43(4): 1229-1239..
40.	Yi W, Qiu S, Fan X, et al. Evaluation of mental workload associated with time pressure in rapid serial visual presentation tasks. IEEE Trans Cognit Dev Syst, 2022, 14(2): 608-616..
41.	Nayak T, Ko L-W, Jung T-P, et al. Target classification in a novel SSVEP-RSVP based BCI gaming system// 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). Bari: IEEE, 2019: 4194-4198..
42.	Ko L-W, Sankar D S V, Huang Y, et al. SSVEP-assisted RSVP brain-computer interface paradigm for multi-target classification. J Neural Eng, 2021, 18(1): 016021..
43.	Diddi S V S, Ko L W. Course-grained multi-scale EMD based fuzzy entropy for multi-target classification during simultaneous SSVEP-RSVP hybrid BCI paradigm. Int J Fuzzy Syst, 2022, 24(5): 2157-2173..
44.	陈景霞, 郝为, 张鹏伟,等. RSVP与SSVEP混合脑电信号刺激与多类事件检测. 计算机工程与应用, 2020, 56(15): 132-139..

1. Xu M P, He F, Jung P, et al. Current challenges for the practical application of electroencephalography-based brain-computer interfaces. Engineering, 2021, 7(12): 1710-1712..
2. Torres E P, Torres E A, Hernandez-Alvarez M, et al. EEG-based BCI emotion recognition: A survey. Sensors, 2020, 20(18): 36..
3. Meng J Y, Xu M P, Wang K, et al. Separable EEG features induced by timing prediction for active brain-computer interfaces. Sensors, 2020, 20(12): 3588..
4. Li Z, Lan Z, Tang D, et al. Survey on ERP-based target detection. Comput Eng Appl, 2021, 57(23): 37-49..
5. Grootswagers T, Zhou I, Robinson A K, et al. Human EEG recordings for 1,854 concepts presented in rapid serial visual presentation streams. Sci Data, 2022, 9(1): 3..
6. Won K, Kwon M, Ahn M, et al. EEG dataset for RSVP and P300 speller brain-computer interfaces. Sci Data, 2022, 9(1): 388..
7. Liu S, Wang W, Sheng Y, et al. Improving the cross-subject performance of the ERP-based brain-computer interface using rapid serial visual presentation and correlation analysis rank. Front Hum Neurosci, 2020, 14: 00296..
8. Mijani A M, Shamsollahi M B, Hassani M S, et al. Comparison between single, dual and triple rapid serial visual presentation paradigms for P300 speller// 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) - Human Genomics. Madrid: IEEE, 2018: 2635-2638..
9. Lin Z, Zhang C, Zeng Y, et al. A novel P300 BCI speller based on the triple RSVP paradigm. Sci Rep, 2018, 8(1): 3350..
10. Mijani A M, Shamsollahi M B, Hassani M S. A novel dual and triple shifted RSVP paradigm for P300 speller. J Neurosci Methods, 2019, 328: 108420..
11. Lees S, Mccullagh P, Payne P, et al. Speed of rapid serial visual presentation of pictures, numbers and words affects event-related potential-based detection accuracy. IEEE Trans Neural Syst Rehabil Eng, 2020, 28(1): 113-122..
12. Fernandez-Rodriguez A, Darves-Bornoz A, Velasco-Alvarez F, et al. Effect of stimulus size in a visual ERP-based BCI under RSVP. Sensors, 2022, 22(23): 9505..
13. Ahani A, Moghadamfalahi M, Erdogmus D. Language-model assisted and icon-based communication through a brain-computer interface with different presentation paradigms. IEEE Trans Neural Syst Rehabil Eng, 2018, 26(9): 1835-1844..
14. Fernandez-Rodriguez A, Medina-Julia M T, Velasco-Alvarez F, et al. Different effects of using pictures as stimuli in a P300 brain-computer interface under rapid serial visual presentation or row-column paradigm. Med Biol Eng Comput, 2021, 59(4): 869-881..
15. Ron-Angevin R, Medina-Julia M T, Fernandez-Rodriguez A, et al. Performance analysis with different types of visual stimuli in a BCI-based speller under an RSVP paradigm. Front Comput Neurosci, 2021, 14: 587702..
16. Won D O, Hwang H J, Kim D M, et al. Motion-based rapid serial visual presentation for gaze-independent brain-computer interfaces. IEEE Trans Neural Syst Rehabil Eng, 2018, 26(2): 334-343..
17. Jalilpour S, Hajipour Sardouie S, Mijani A. A novel hybrid BCI speller based on RSVP and SSVEP paradigm. Comput Methods Programs Biomed, 2020, 187: 105326..
18. Gonzalez-Navarro P, Celik B, Moghadamfalahi M, et al. Feedback related potentials for EEG-based typing systems. Front Hum Neurosci, 2022, 15: 13..
19. Zisk A H, Borgheai S B, Mclinden J, et al. P300 latency jitter and its correlates in people with amyotrophic lateral sclerosis. Clin Neurophysiol, 2021, 132(2): 632-642..
20. Li B, Lin Y, Gao X, et al. Enhancing the EEG classification in RSVP task by combining interval model of ERPs with spatial and temporal regions of interest. J Neural Eng, 2021, 18(1): 016008..
21. Zhang C, Qiu S, Wang S, et al. Target detection using ternary classification during a rapid serial visual presentation task using magnetoencephalography data. Front Comput Neurosci, 2021, 15: 619508..
22. Lin Z, Zeng Y, Wang X, et al. EEG-based target detection during a multi-rapid serial visual presentation// 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER). Shanghai: IEEE, 2017: 556-559..
23. Hou Y, Gu Z, Yu Z L, et al. Enhancement of lower limb motor imagery ability via dual-level multimodal stimulation and sparse spatial pattern decoding method. Front Hum Neurosci, 2022, 16: 975410..
24. 迟新一, 崔红岩, 陈小刚. 结合稳态视觉诱发电位的多模态脑机接口研究进展. 中国生物医学工程学报, 2022, 41(2): 204-213..
25. Onishi A. Brain-computer interface with rapid serial multimodal presentation using artificial facial images and voice. Comput Biol Med, 2021, 136: 104685..
26. Mao J, Qiu S, Li D, et al. A cross-modal guiding and fusion method for multi-modal RSVP-based image retrieval// 2021 International Joint Conference on Neural Networks (IJCNN). Shenzhen: IEEE, 2021: 1-7..
27. Mao J, Qiu S, Wei W, et al. Cross-modal guiding and reweighting network for multi-modal RSVP-based target detection. Neural Networks, 2023, 161: 65-82..
28. Song X, Zeng Y, Tong L, et al. A collaborative brain-computer interface framework for enhancing group detection performance of dynamic visual targets. Comput Intell Neurosci, 2022, 2022: 4752450..
29. 韩锦, 董博文, 刘邈, 等. 基于P300-SSVEP的双人协同脑-控机械臂汉字书写系统. 数据采集与处理, 2022, 37(6): 1401-1411..
30. Bhattacharyya S, Valeriani D, Cinel C, et al. Anytime collaborative brain-computer interfaces for enhancing perceptual group decision-making. Sci Rep, 2021, 11(1): 17008..
31. Zhang H, Zhu L, Xu S, et al. Two brains, one target: Design of a multi-level information fusion model based on dual-subject RSVP. J Neurosci Methods, 2021, 363: 109346..
32. Valeriani D, O’Flynn L C, Worthley A, et al. Multimodal collaborative brain-computer interfaces aid human-machine team decision-making in a pandemic scenario. J Neural Eng, 2022, 19(5): 056036..
33. Salvatore C, Valeriani D, Piccialli V, et al. Optimized collaborative brain-computer interfaces for enhancing face recognition. IEEE Trans Neural Syst Rehabil Eng, 2022, 30: 1223-1232..
34. Zhang S, Sun L, Mao X, et al. Review on EEG-based authentication technology. Comput Intell Neurosci, 2021, 2021: 5229576..
35. Wang H W, Qi Y, Yu H, et al. RCIT: An RSVP-based concealed information test framework using EEG signals. IEEE Trans Cognit Dev Syst, 2022, 14(2): 541-551..
36. Jangwan N S, Ashraf G M, Ram V, et al. Brain augmentation and neuroscience technologies: Current applications, challenges, ethics and future prospects. Front Syst Neurosci, 2022, 16: 1000495..
37. Zhang S, Chen X, Wang Y, et al. Visual field inhomogeneous in brain-computer interfaces based on rapid serial visual presentation. J Neural Eng, 2022, 19(1): 016015..
38. Zokaei N, Gillebert C R, Chauvin J J, et al. Temporal orienting in Parkinson’s disease. Eur J Neurosci, 2021, 53(8): 2713-2725..
39. Shalbaf A, Bagherzadeh S, Maghsoudi A. Transfer learning with deep convolutional neural network for automated detection of schizophrenia from EEG signals. Phys Eng Sci Med, 2020, 43(4): 1229-1239..
40. Yi W, Qiu S, Fan X, et al. Evaluation of mental workload associated with time pressure in rapid serial visual presentation tasks. IEEE Trans Cognit Dev Syst, 2022, 14(2): 608-616..
41. Nayak T, Ko L-W, Jung T-P, et al. Target classification in a novel SSVEP-RSVP based BCI gaming system// 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). Bari: IEEE, 2019: 4194-4198..
42. Ko L-W, Sankar D S V, Huang Y, et al. SSVEP-assisted RSVP brain-computer interface paradigm for multi-target classification. J Neural Eng, 2021, 18(1): 016021..
43. Diddi S V S, Ko L W. Course-grained multi-scale EMD based fuzzy entropy for multi-target classification during simultaneous SSVEP-RSVP hybrid BCI paradigm. Int J Fuzzy Syst, 2022, 24(5): 2157-2173..
44. 陈景霞, 郝为, 张鹏伟,等. RSVP与SSVEP混合脑电信号刺激与多类事件检测. 计算机工程与应用, 2020, 56(15): 132-139..

Previous Article
Research advances in neuromodulation techniques for blood glucose regulation and diabetes intervention
Next Article
Research progress of biomarkers of hepatitis B virus and clinical significance

Journal of Biomedical Engineering

Research progress of brain-computer interface application paradigms based on rapid serial visual presentation

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content