Review of joint attention deficit intervention based on virtual reality for children with autism_Journal of Biomedical Engineering

Authors：

ZHAO Xin ¹ , WANG Heting ¹ ,  GUO Xinmeng ² , ZHANG Ludan ² , LIU Wei ³ ,  LIU Shuang ² , MING Dong ^1,2

1. School of Precision Instrument and Opto-electronics Engineering, Tianjin University, Tianjin 300072, P. R. China;
2. Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, P. R. China;
3. Children’s Hospital, Tianjin University, Tianjin 300134, P. R. China;

Corresponding author：

GUO Xinmeng, Email: guoxinmeng@tju.edu.cn

Keywords：

Joint attention deficit; Virtual reality; Children with autism; Scene intervention

DOI：

10.7507/1001-5515.202305029

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Joint attention deficit is one of the core disorders in children with autism, which seriously affects the development of multiple basic skills such as language and communication. Virtual reality scene intervention has great potential in improving joint attention skills in children with autism due to its good interactivity and immersion. This article reviewed the application of virtual reality based social and nonsocial scenarios in training joint attention skills for children with autism in recent years, summarized the problems and challenges of this intervention method, and proposed a new joint paradigm for social scenario assessment and nonsocial scenario training. Finally, it looked forward to the future development and application prospects of virtual reality technology in joint attention skill training for children with autism.

Citation： ZHAO Xin, WANG Heting, GUO Xinmeng, ZHANG Ludan, LIU Wei, LIU Shuang, MING Dong. Review of joint attention deficit intervention based on virtual reality for children with autism. Journal of Biomedical Engineering, 2024, 41(3): 612-619. doi: 10.7507/1001-5515.202305029 Copy

1.	Amat A Z, Zhao H, Swanson A, et al. Design of an interactive virtual reality system, InViRS, for joint attention practice in autistic children. IEEE Trans Neural Syst Rehabil Eng, 2021, 29: 1866-1876.
2.	Meindl J N, Cannella-Malone H I. Initiating and responding to joint attention bids in children with autism: A review of the literature. Res Devl Disabil, 2011, 32(5): 1441-1454.
3.	康一奇, 种霞, 吴南. ASD早期预警: JA和共情的发生发展及影响因素. 心理科学进展, 2018, 26(7): 1223-1231.
4.	Kelty-Stephen E, Fein D A, Naigles L R. Children with ASD use joint attention and linguistic skill in pronoun development. Lang Acq, 2020, 27(4): 410-433.
5.	孙美琦. ASD谱系障碍患儿共同注意干预的个案研究. 大连: 辽宁师范大学, 2021.
6.	Liu X, Wu Q, Zhao W, et al. Technology-facilitated diagnosis and treatment of individuals with autism spectrum disorder: An engineering perspective. Appl Sci, 2017, 7(10): 1051.
7.	张万丰, 王园春, 刘圆, 等. 视频示范教学法在自闭症谱系障碍者职业技能训练中的应用. 现代特殊教育, 2021(4): 75-80.
8.	Bar Yehuda S, Bauminger-Zviely N. Social-motor coordination between peers: Joint action developmental trajectories in ASD and TD. J Autism Dev Disord, 2024, 54(3): 811-828.
9.	Abdelmohsen M, Arafa Y. Training social skills of children with ASD through social virtual robot// 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Lisbon: IEEE, 2021: 314-319.
10.	Cao H L, Simut R E, Desmet N, et al. Robot-assisted joint attention: A comparative study between children with autism spectrum disorder and typically developing children in interaction with NAO. IEEE Access, 2020, 8: 223325-223334.
11.	杨玲. 基于虚拟现实技术的计算机三维仿生图像特征增强策略. 电子技术与软件工程, 2021(13): 142-143.
12.	Kuriakose S, Lahiri U. Understanding the psycho-physiological implications of interaction with a virtual reality-based system in adolescents with autism: A feasibility study. IEEE Trans Neural Syst Rehabil Eng, 2015, 23(4): 665-675.
13.	Mesa-Gresa P, Gil-Gómez H, Lozano-Quilis J A, et al. Effectiveness of virtual reality for children and adolescents with autism spectrum disorder: an evidence-based systematic review. Sensors, 2018, 18(8): 2486.
14.	Rosli M I, Che Embi Z, Abdullah J. A non-immersive virtual reality application for children with autism spectrum disorder// International Conference on Intelligent Tutoring Systems. Cham: Springer, 2021: 519-525.
15.	Frolli A, Savarese G, Di Carmine F, et al. Children on the autism spectrum and the use of virtual reality for supporting social skills. Children, 2022, 9(2): 181.
16.	Chen Y Q, Lin F A, Yang T Y, et al. A VR-based training and intelligent assessment system integrated with multi-modal sensing for children with autism spectrum disorder// 2021 IEEE 3rd Eurasia Conference on IOT, Communication and Engineering (ECICE). Yunlin: IEEE, 2021: 191-195.
17.	张领, 朱原雨润, 王晶仪. 基于人脸表情识别的ASD患儿辅助诊断. 数字技术与应用, 2022, 40(7): 124-126.
18.	马宏森. 线索提示对ASD谱系障碍患儿图画书社会信息加工的影响. 漳州: 闽南师范大学, 2020.
19.	Grynszpan O, Nadel J, Carbonell N, et al. A new virtual environment paradigm for high functioning autism intended to help attentional disengagement in a social context bridging the gap between relevance theory and executive dysfunction// 2009 Virtual Rehabilitation International Conference. Haifa: IEEE, 2009: 51-58.
20.	Yurkovic-Harding J, Lisandrelli G, Shaffer R C, et al. Children with ASD establish joint attention during free-flowing toy play without face looks. Curr Biol, 2022, 32(12): 2739-2746. e4.
21.	Cardoso T V, Michelassi G C, Franco F O, et al. Autism Spectrum Disorder diagnosis based on trajectories of eye tracking data// 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS). Aveiro: IEEE, 2021: 50-55.
22.	Lahiri U, Warren Z, Sarkar N. Design of a gaze-sensitive virtual social interactive system for children with autism. IEEE Trans Neural Syst Rehabil Eng, 2011, 19(4): 443-452.
23.	Lahiri U, Bekele E, Dohrmann E, et al. Design of a virtual reality based adaptive response technology for children with autism. IEEE Trans Neural Syst Rehabil Eng, 2012, 21(1): 55-64.
24.	毋嫘, 李昕雨, 张凯帆, 等. ASD患儿面部表情识别缺损的内在机制. 中国健康心理学杂志, 2022, 30(8): 1270-1276.
25.	Akhtar Z, Guha T. Computational analysis of gaze behavior in autism during interaction with virtual agents// ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Brighton: IEEE, 2019, 20: 1075-1079.
26.	Bekele E, Zheng Z, Swanson A, et al. Understanding how adolescents with autism respond to facial expressions in virtual reality environments. IEEE Trans Visual Com Grap, 2013, 19(4): 711-720.
27.	Derakshan N, Smyth S, Eysenck M W. Effects of state anxiety on performance using a task-switching paradigm: An investigation of attentional control theory. Psychon Bull Rev, 2009, 16(6): 1112-1117.
28.	Pradeep Raj K B, Lahiri U. Design of eyegaze-sensitive virtual reality based social communication platform for individuals with autism// 2016 7th International Conference on Intelligent Systems, Modelling and Simulation (ISMS). Bangkok: IEEE, 2016: 301-306.
29.	Babu P R K, Oza P, Lahiri U. Gaze-sensitive virtual reality based social communication platform for individuals with autism. IEEE T Affect Comput, 2017, 9(4): 450-462.
30.	Peters C, Qureshi A. A head movement propensity model for animating gaze shifts and blinks of virtual characters. Comput Graph, 2010, 34(6): 677-687.
31.	Courgeon M, Rautureau G, Martin J C, et al. Joint attention simulation using eye-tracking and virtual humans. IEEE T Affect Comput, 2014, 5(3): 238-250.
32.	Jyoti V, Lahiri U. Human-computer interaction based joint attention cues: Implications on functional and physiological measures for children with autism spectrum disorder. CHB, 2020, 104: 106163.
33.	Jyoti V, Lahiri U. Virtual reality based joint attention task platform for children with autism. IEEE T Learn Technol, 2019, 13(1): 198-210.
34.	Jyoti V, Gupta S, Lahiri U. Understanding the role of objects in joint attention task framework for children with autism. IEEE T Cogn Dev Syst, 2020, 13(3): 524-534.
35.	Little G E, Bonnar L, Kelly S W, et al. Gaze contingent joint attention with an avatar in children with and without ASD// 2016 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob). Cergy-Pontoise: IEEE, 2016: 15-20.
36.	Kim K, Mundy P. Joint attention, social-cognition, and recognition memory in adults. Front Hum Neurosci, 2012, 6: 172.
37.	Mei C, Zahed B T, Mason L, et al. Towards joint attention training for children with ASD-a VR game approach and eye gaze exploration// 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). Tuebingen/Reutlingen: IEEE, 2018: 289-296.
38.	吴潇萍. ASD患儿音乐教学中集体活动课模式探析. 试题与研究, 2022(19): 185-186.
39.	Kulikovskiy R, Sochanski M, Eaton M, et al. Can therapists design robot-mediated interventions and teleoperate robots using VR to deliver interventions for ASD?// 2021 IEEE International Conference on Robotics and Automation (ICRA). Xi’an: IEEE, 2021: 3669-3676.
40.	Shahab M, Taheri A, Mokhtari M, et al. Utilizing social virtual reality robot (V2R) for music education to children with high-functioning autism. Educ Inf Technol, 2022, 27(1): 819-843.
41.	Hopkins W D, Staes N, Guevara E E, et al. Vasopressin, and not oxytocin, receptor gene methylation is associated with individual differences in receptive joint attention in chimpanzees (Pan troglodytes). Autism Res, 2023, 16(4): 713-722.
42.	Kourassanis-Velasquez J, Jones E A. Increasing joint attention in children with autism and their peers. Behav Anal Pract, 2019, 12: 78-94.
43.	沙庆庆, 陈东帆, 于新宇, 等. 面向ASD患儿的虚拟现实游戏设计与开发. 中国教育技术装备, 2018(2): 37-40.
44.	陆凯, 岳康, 胡昊辰, 等. 神经反馈训练中的虚拟现实技术综述. 计算机辅助设计与图形学学报, 2023, 35(8): 1150-1161.

1. Amat A Z, Zhao H, Swanson A, et al. Design of an interactive virtual reality system, InViRS, for joint attention practice in autistic children. IEEE Trans Neural Syst Rehabil Eng, 2021, 29: 1866-1876.
2. Meindl J N, Cannella-Malone H I. Initiating and responding to joint attention bids in children with autism: A review of the literature. Res Devl Disabil, 2011, 32(5): 1441-1454.
3. 康一奇, 种霞, 吴南. ASD早期预警: JA和共情的发生发展及影响因素. 心理科学进展, 2018, 26(7): 1223-1231.
4. Kelty-Stephen E, Fein D A, Naigles L R. Children with ASD use joint attention and linguistic skill in pronoun development. Lang Acq, 2020, 27(4): 410-433.
5. 孙美琦. ASD谱系障碍患儿共同注意干预的个案研究. 大连: 辽宁师范大学, 2021.
6. Liu X, Wu Q, Zhao W, et al. Technology-facilitated diagnosis and treatment of individuals with autism spectrum disorder: An engineering perspective. Appl Sci, 2017, 7(10): 1051.
7. 张万丰, 王园春, 刘圆, 等. 视频示范教学法在自闭症谱系障碍者职业技能训练中的应用. 现代特殊教育, 2021(4): 75-80.
8. Bar Yehuda S, Bauminger-Zviely N. Social-motor coordination between peers: Joint action developmental trajectories in ASD and TD. J Autism Dev Disord, 2024, 54(3): 811-828.
9. Abdelmohsen M, Arafa Y. Training social skills of children with ASD through social virtual robot// 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Lisbon: IEEE, 2021: 314-319.
10. Cao H L, Simut R E, Desmet N, et al. Robot-assisted joint attention: A comparative study between children with autism spectrum disorder and typically developing children in interaction with NAO. IEEE Access, 2020, 8: 223325-223334.
11. 杨玲. 基于虚拟现实技术的计算机三维仿生图像特征增强策略. 电子技术与软件工程, 2021(13): 142-143.
12. Kuriakose S, Lahiri U. Understanding the psycho-physiological implications of interaction with a virtual reality-based system in adolescents with autism: A feasibility study. IEEE Trans Neural Syst Rehabil Eng, 2015, 23(4): 665-675.
13. Mesa-Gresa P, Gil-Gómez H, Lozano-Quilis J A, et al. Effectiveness of virtual reality for children and adolescents with autism spectrum disorder: an evidence-based systematic review. Sensors, 2018, 18(8): 2486.
14. Rosli M I, Che Embi Z, Abdullah J. A non-immersive virtual reality application for children with autism spectrum disorder// International Conference on Intelligent Tutoring Systems. Cham: Springer, 2021: 519-525.
15. Frolli A, Savarese G, Di Carmine F, et al. Children on the autism spectrum and the use of virtual reality for supporting social skills. Children, 2022, 9(2): 181.
16. Chen Y Q, Lin F A, Yang T Y, et al. A VR-based training and intelligent assessment system integrated with multi-modal sensing for children with autism spectrum disorder// 2021 IEEE 3rd Eurasia Conference on IOT, Communication and Engineering (ECICE). Yunlin: IEEE, 2021: 191-195.
17. 张领, 朱原雨润, 王晶仪. 基于人脸表情识别的ASD患儿辅助诊断. 数字技术与应用, 2022, 40(7): 124-126.
18. 马宏森. 线索提示对ASD谱系障碍患儿图画书社会信息加工的影响. 漳州: 闽南师范大学, 2020.
19. Grynszpan O, Nadel J, Carbonell N, et al. A new virtual environment paradigm for high functioning autism intended to help attentional disengagement in a social context bridging the gap between relevance theory and executive dysfunction// 2009 Virtual Rehabilitation International Conference. Haifa: IEEE, 2009: 51-58.
20. Yurkovic-Harding J, Lisandrelli G, Shaffer R C, et al. Children with ASD establish joint attention during free-flowing toy play without face looks. Curr Biol, 2022, 32(12): 2739-2746. e4.
21. Cardoso T V, Michelassi G C, Franco F O, et al. Autism Spectrum Disorder diagnosis based on trajectories of eye tracking data// 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS). Aveiro: IEEE, 2021: 50-55.
22. Lahiri U, Warren Z, Sarkar N. Design of a gaze-sensitive virtual social interactive system for children with autism. IEEE Trans Neural Syst Rehabil Eng, 2011, 19(4): 443-452.
23. Lahiri U, Bekele E, Dohrmann E, et al. Design of a virtual reality based adaptive response technology for children with autism. IEEE Trans Neural Syst Rehabil Eng, 2012, 21(1): 55-64.
24. 毋嫘, 李昕雨, 张凯帆, 等. ASD患儿面部表情识别缺损的内在机制. 中国健康心理学杂志, 2022, 30(8): 1270-1276.
25. Akhtar Z, Guha T. Computational analysis of gaze behavior in autism during interaction with virtual agents// ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Brighton: IEEE, 2019, 20: 1075-1079.
26. Bekele E, Zheng Z, Swanson A, et al. Understanding how adolescents with autism respond to facial expressions in virtual reality environments. IEEE Trans Visual Com Grap, 2013, 19(4): 711-720.
27. Derakshan N, Smyth S, Eysenck M W. Effects of state anxiety on performance using a task-switching paradigm: An investigation of attentional control theory. Psychon Bull Rev, 2009, 16(6): 1112-1117.
28. Pradeep Raj K B, Lahiri U. Design of eyegaze-sensitive virtual reality based social communication platform for individuals with autism// 2016 7th International Conference on Intelligent Systems, Modelling and Simulation (ISMS). Bangkok: IEEE, 2016: 301-306.
29. Babu P R K, Oza P, Lahiri U. Gaze-sensitive virtual reality based social communication platform for individuals with autism. IEEE T Affect Comput, 2017, 9(4): 450-462.
30. Peters C, Qureshi A. A head movement propensity model for animating gaze shifts and blinks of virtual characters. Comput Graph, 2010, 34(6): 677-687.
31. Courgeon M, Rautureau G, Martin J C, et al. Joint attention simulation using eye-tracking and virtual humans. IEEE T Affect Comput, 2014, 5(3): 238-250.
32. Jyoti V, Lahiri U. Human-computer interaction based joint attention cues: Implications on functional and physiological measures for children with autism spectrum disorder. CHB, 2020, 104: 106163.
33. Jyoti V, Lahiri U. Virtual reality based joint attention task platform for children with autism. IEEE T Learn Technol, 2019, 13(1): 198-210.
34. Jyoti V, Gupta S, Lahiri U. Understanding the role of objects in joint attention task framework for children with autism. IEEE T Cogn Dev Syst, 2020, 13(3): 524-534.
35. Little G E, Bonnar L, Kelly S W, et al. Gaze contingent joint attention with an avatar in children with and without ASD// 2016 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob). Cergy-Pontoise: IEEE, 2016: 15-20.
36. Kim K, Mundy P. Joint attention, social-cognition, and recognition memory in adults. Front Hum Neurosci, 2012, 6: 172.
37. Mei C, Zahed B T, Mason L, et al. Towards joint attention training for children with ASD-a VR game approach and eye gaze exploration// 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). Tuebingen/Reutlingen: IEEE, 2018: 289-296.
38. 吴潇萍. ASD患儿音乐教学中集体活动课模式探析. 试题与研究, 2022(19): 185-186.
39. Kulikovskiy R, Sochanski M, Eaton M, et al. Can therapists design robot-mediated interventions and teleoperate robots using VR to deliver interventions for ASD?// 2021 IEEE International Conference on Robotics and Automation (ICRA). Xi’an: IEEE, 2021: 3669-3676.
40. Shahab M, Taheri A, Mokhtari M, et al. Utilizing social virtual reality robot (V2R) for music education to children with high-functioning autism. Educ Inf Technol, 2022, 27(1): 819-843.
41. Hopkins W D, Staes N, Guevara E E, et al. Vasopressin, and not oxytocin, receptor gene methylation is associated with individual differences in receptive joint attention in chimpanzees (Pan troglodytes). Autism Res, 2023, 16(4): 713-722.
42. Kourassanis-Velasquez J, Jones E A. Increasing joint attention in children with autism and their peers. Behav Anal Pract, 2019, 12: 78-94.
43. 沙庆庆, 陈东帆, 于新宇, 等. 面向ASD患儿的虚拟现实游戏设计与开发. 中国教育技术装备, 2018(2): 37-40.
44. 陆凯, 岳康, 胡昊辰, 等. 神经反馈训练中的虚拟现实技术综述. 计算机辅助设计与图形学学报, 2023, 35(8): 1150-1161.

Previous Article
The effect of surface modification strategies on biological activity of titanium implant
Next Article
In vivo tumor imaging and therapy based on near-infrared cadmium-free quantum dots

Journal of Biomedical Engineering

Review of joint attention deficit intervention based on virtual reality for children with autism

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content