A review of studies on visual behavior analysis aided diagnosis of autism spectrum disorders_Journal of Biomedical Engineering

Authors：

LI Xiang ,  MA Xin , LI Yibin

School of Control Science and Engineering, Shandong University, Jinan 250061, P. R. China;

Corresponding author：

Keywords：

Autism spectrum disorder; Diagnosis; Visual behavior analysis; Interactive scene; Video dataset

DOI：

10.7507/1001-5515.202204038

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by deficits in social communication and repetitive behaviors. With the rapid development of computer vision, visual behavior analysis aided diagnosis of ASD has got more and more attention. This paper reviews the research on visual behavior analysis aided diagnosis of ASD. First, the core symptoms and clinical diagnostic criteria of ASD are introduced briefly. Secondly, according to clinical diagnostic criteria, the interaction scenes are classified and introduced. Then, the existing relevant datasets are discussed. Finally, we analyze and compare the advantages and disadvantages of visual behavior analysis aided diagnosis methods for ASD in different interactive scenarios. The challenges in this research field are summarized and the prospects of related research are presented to promote the clinical application of visual behavior analysis in ASD diagnosis.

Citation： LI Xiang, MA Xin, LI Yibin. A review of studies on visual behavior analysis aided diagnosis of autism spectrum disorders. Journal of Biomedical Engineering, 2023, 40(4): 812-819. doi: 10.7507/1001-5515.202204038 Copy

1.	Maenner M J, Shaw K A, Baio J, et al. Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2016. MMWR Surveillance Summaries, 2020, 69(4): 1-12.
2.	袁玉琢, 骆方. 人工智能辅助的自闭症早期患者的筛查与诊断. 心理科学进展, 2022, 30(10): 2303-2320.
3.	Fuller E A, Oliver K, Vejnoska S F, et al. The effects of the early start denver model for children with autism spectrum disorder: a meta-analysis. Brain Sciences, 2020, 10(6): 368-384.
4.	Lebersfeld J B, Swanson M, Clesi C D, et al. Systematic review and meta-analysis of the clinical utility of the ADOS-2 and the ADI-R in diagnosing autism spectrum disorders in children. Journal of Autism and Developmental Disorders, 2021, 51(11): 4101-4114.
5.	Rosen N E, Lord C, Volkmar F R. The diagnosis of autism: from Kanner to DSM-III to DSM-5 and beyond. Journal of Autism and Developmental Disorders, 2021, 51(12): 4253-4270.
6.	Song D Y, Kim S Y, Bong G, et al. The use of artificial intelligence in screening and diagnosis of autism spectrum disorder: a literature review. Soa Chongsonyon Chongsin Uihak, 2019, 30(4): 145-152.
7.	Leblanc E, Washington P, Varma M, et al. Feature replacement methods enable reliable home video analysis for machine learning detection of autism. Scientific Reports, 2020, 10(1): 21245.
8.	Sapiro G, Hashemi J, Dawson G. Computer vision and behavioral phenotyping: an autism case study. Current Opinion in Biomedical Engineering, 2019, 9: 14-20.
9.	Kojovic N, Ben Hadid L, Franchini M, et al. Sensory processing issues and their association with social difficulties in children with autism spectrum disorders. Journal of Clinical Medicine, 2019, 8(10): 1508-1523.
10.	Hiremath C S, Sagar K J V, Yamini B, et al. Emerging behavioral and neuroimaging biomarkers for early and accurate characterization of autism spectrum disorders: a systematic review. Translational Psychiatry, 2021, 11(1): 42.
11.	Holloway J M, Tomlinson S M, Hardwick D D. Strategies to support learning of gross motor tasks in children with autism spectrum disorder: a scoping review. Physical & Occupational Therapy In Pediatrics, 2023, 43(1): 17-33.
12.	Taverna E C, Huedo-Medina T B, Fein D A, et al. The interaction of fine motor, gesture, and structural language skills: the case of autism spectrum disorder. Research in Autism Spectrum Disorders, 2021, 86: 101824.
13.	Ohara R, Kanejima Y, Kitamura M, et al. Association between social skills and motor skills in individuals with autism spectrum disorder: a systematic review. European Journal of Investigation in Health, Psychology and Education, 2019, 10(1): 276-296.
14.	Al-Jubouri A A, Ali I H, Rajihy Y. Generating 3D dataset of gait and full body movement of children with autism spectrum disorders collected by Kinect v2 camera. Compusoft, 2020, 9(8): 3791-3797.
15.	Rajagopalan S, Dhall A, Goecke R. Self-stimulatory behaviours in the wild for autism diagnosis// Proceedings of the IEEE International Conference on Computer Vision Workshops. Sydney: IEEE, 2013: 755-761.
16.	Zunino A, Morerio P, Cavallo A, et al. Video gesture analysis for autism spectrum disorder detection// 2018 24th International Conference on Pattern Recognition. Beijing: IEEE, 2018: 3421-3426.
17.	Liu J, He K, Wang Z, et al. A computer vision system to assist the early screening of autism spectrum disorder// International Conference on Cognitive Systems and Signal Processing. Beijing: Springer, 2018: 27-38.
18.	Qin H, Wang Z, Liu J, et al. Vision-based pointing estimation and evaluation in toddlers for autism screening// International Conference on Intelligent Robotics and Applications. Yantai: Springer, 2021: 177-185.
19.	Wang Z, Liu J, He K, et al. Screening early children with autism spectrum disorder via response-to-name protocol. IEEE Transactions on Industrial Informatics, 2021, 17(1): 587-595.
20.	Hashemi J, Tepper M, Vallin Spina T, et al. Computer vision tools for low-cost and noninvasive measurement of autism-related behaviors in infants. Autism Research and Treatment, 2014, 2014: 935686.
21.	Moghadas M, Moradi H. Analyzing human-robot interaction using machine vision for autism screening// 2018 6th RSI International Conference on Robotics and Mechatronics. Tehran: IEEE, 2018: 572-576.
22.	Cook A, Mandal B, Berry D, et al. Towards automatic screening of typical and atypical behaviors in children with autism// 2019 IEEE International Conference on Data Science and Advanced Analytics. Washington: IEEE, 2019: 504-510.
23.	Vyas K, Ma R, Rezaei B, et al. Recognition of atypical behavior in autism diagnosis from video using pose estimation over time// 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing. Pittsburgh: IEEE, 2019: 1-6.
24.	Tian Y, Min X, Zhai G, et al. Video-based early asd detection via temporal pyramid networks// 2019 IEEE International Conference on Multimedia and Expo. Shanghai: IEEE, 2019: 272-277.
25.	Negin F, Ozyer B, Agahian S, et al. Vision-assisted recognition of stereotype behaviors for early diagnosis of autism spectrum disorders. Neurocomputing, 2021, 446: 145-155.
26.	Marinoiu E, Zanfir M, Olaru V, et al. 3D human sensing, action and emotion recognition in robot assisted therapy of children with autism// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2158-2167.
27.	Campbell K, Carpenter K L, Hashemi J, et al. Computer vision analysis captures atypical attention in toddlers with autism. Autism, 2019, 23(3): 619-628.
28.	Pandey P, Prathosh A, Kohli M, et al. Guided weak supervision for action recognition with scarce data to assess skills of children with autism// Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2020: 463-470.
29.	Kojovic N, Natraj S, Mohanty S P, et al. Using 2D video-based pose estimation for automated prediction of autism spectrum disorders in young children. Scientific Reports, 2021, 11(1): 15069.
30.	Ali A, Negin F F, Bremond F F, et al. Video-based behavior understanding of children for objective diagnosis of autism. Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. SciTePress, 2022: 475-484.
31.	Sun Kangbo, Li Lin, Li Lianqiang, et al. Spatial attentional bilinear 3D convolutional network for video-based autism spectrum disorder detection// ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: IEEE, 2020: 3387-3391.
32.	Lakkapragada A, Kline A, Mutlu O C, et al. Classification of abnormal hand movement for aiding in autism detection: machine learning study. JMIR Biomed Eng, 2022, 7(1): e33771.
33.	Washington P, Kline A, Mutlu O C, et al. Activity recognition with moving cameras and few training examples: applications for detection of autism-related headbanging// CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. Yokohama: ACM, 2021.

1. Maenner M J, Shaw K A, Baio J, et al. Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2016. MMWR Surveillance Summaries, 2020, 69(4): 1-12.
2. 袁玉琢, 骆方. 人工智能辅助的自闭症早期患者的筛查与诊断. 心理科学进展, 2022, 30(10): 2303-2320.
3. Fuller E A, Oliver K, Vejnoska S F, et al. The effects of the early start denver model for children with autism spectrum disorder: a meta-analysis. Brain Sciences, 2020, 10(6): 368-384.
4. Lebersfeld J B, Swanson M, Clesi C D, et al. Systematic review and meta-analysis of the clinical utility of the ADOS-2 and the ADI-R in diagnosing autism spectrum disorders in children. Journal of Autism and Developmental Disorders, 2021, 51(11): 4101-4114.
5. Rosen N E, Lord C, Volkmar F R. The diagnosis of autism: from Kanner to DSM-III to DSM-5 and beyond. Journal of Autism and Developmental Disorders, 2021, 51(12): 4253-4270.
6. Song D Y, Kim S Y, Bong G, et al. The use of artificial intelligence in screening and diagnosis of autism spectrum disorder: a literature review. Soa Chongsonyon Chongsin Uihak, 2019, 30(4): 145-152.
7. Leblanc E, Washington P, Varma M, et al. Feature replacement methods enable reliable home video analysis for machine learning detection of autism. Scientific Reports, 2020, 10(1): 21245.
8. Sapiro G, Hashemi J, Dawson G. Computer vision and behavioral phenotyping: an autism case study. Current Opinion in Biomedical Engineering, 2019, 9: 14-20.
9. Kojovic N, Ben Hadid L, Franchini M, et al. Sensory processing issues and their association with social difficulties in children with autism spectrum disorders. Journal of Clinical Medicine, 2019, 8(10): 1508-1523.
10. Hiremath C S, Sagar K J V, Yamini B, et al. Emerging behavioral and neuroimaging biomarkers for early and accurate characterization of autism spectrum disorders: a systematic review. Translational Psychiatry, 2021, 11(1): 42.
11. Holloway J M, Tomlinson S M, Hardwick D D. Strategies to support learning of gross motor tasks in children with autism spectrum disorder: a scoping review. Physical & Occupational Therapy In Pediatrics, 2023, 43(1): 17-33.
12. Taverna E C, Huedo-Medina T B, Fein D A, et al. The interaction of fine motor, gesture, and structural language skills: the case of autism spectrum disorder. Research in Autism Spectrum Disorders, 2021, 86: 101824.
13. Ohara R, Kanejima Y, Kitamura M, et al. Association between social skills and motor skills in individuals with autism spectrum disorder: a systematic review. European Journal of Investigation in Health, Psychology and Education, 2019, 10(1): 276-296.
14. Al-Jubouri A A, Ali I H, Rajihy Y. Generating 3D dataset of gait and full body movement of children with autism spectrum disorders collected by Kinect v2 camera. Compusoft, 2020, 9(8): 3791-3797.
15. Rajagopalan S, Dhall A, Goecke R. Self-stimulatory behaviours in the wild for autism diagnosis// Proceedings of the IEEE International Conference on Computer Vision Workshops. Sydney: IEEE, 2013: 755-761.
16. Zunino A, Morerio P, Cavallo A, et al. Video gesture analysis for autism spectrum disorder detection// 2018 24th International Conference on Pattern Recognition. Beijing: IEEE, 2018: 3421-3426.
17. Liu J, He K, Wang Z, et al. A computer vision system to assist the early screening of autism spectrum disorder// International Conference on Cognitive Systems and Signal Processing. Beijing: Springer, 2018: 27-38.
18. Qin H, Wang Z, Liu J, et al. Vision-based pointing estimation and evaluation in toddlers for autism screening// International Conference on Intelligent Robotics and Applications. Yantai: Springer, 2021: 177-185.
19. Wang Z, Liu J, He K, et al. Screening early children with autism spectrum disorder via response-to-name protocol. IEEE Transactions on Industrial Informatics, 2021, 17(1): 587-595.
20. Hashemi J, Tepper M, Vallin Spina T, et al. Computer vision tools for low-cost and noninvasive measurement of autism-related behaviors in infants. Autism Research and Treatment, 2014, 2014: 935686.
21. Moghadas M, Moradi H. Analyzing human-robot interaction using machine vision for autism screening// 2018 6th RSI International Conference on Robotics and Mechatronics. Tehran: IEEE, 2018: 572-576.
22. Cook A, Mandal B, Berry D, et al. Towards automatic screening of typical and atypical behaviors in children with autism// 2019 IEEE International Conference on Data Science and Advanced Analytics. Washington: IEEE, 2019: 504-510.
23. Vyas K, Ma R, Rezaei B, et al. Recognition of atypical behavior in autism diagnosis from video using pose estimation over time// 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing. Pittsburgh: IEEE, 2019: 1-6.
24. Tian Y, Min X, Zhai G, et al. Video-based early asd detection via temporal pyramid networks// 2019 IEEE International Conference on Multimedia and Expo. Shanghai: IEEE, 2019: 272-277.
25. Negin F, Ozyer B, Agahian S, et al. Vision-assisted recognition of stereotype behaviors for early diagnosis of autism spectrum disorders. Neurocomputing, 2021, 446: 145-155.
26. Marinoiu E, Zanfir M, Olaru V, et al. 3D human sensing, action and emotion recognition in robot assisted therapy of children with autism// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2158-2167.
27. Campbell K, Carpenter K L, Hashemi J, et al. Computer vision analysis captures atypical attention in toddlers with autism. Autism, 2019, 23(3): 619-628.
28. Pandey P, Prathosh A, Kohli M, et al. Guided weak supervision for action recognition with scarce data to assess skills of children with autism// Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2020: 463-470.
29. Kojovic N, Natraj S, Mohanty S P, et al. Using 2D video-based pose estimation for automated prediction of autism spectrum disorders in young children. Scientific Reports, 2021, 11(1): 15069.
30. Ali A, Negin F F, Bremond F F, et al. Video-based behavior understanding of children for objective diagnosis of autism. Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. SciTePress, 2022: 475-484.
31. Sun Kangbo, Li Lin, Li Lianqiang, et al. Spatial attentional bilinear 3D convolutional network for video-based autism spectrum disorder detection// ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: IEEE, 2020: 3387-3391.
32. Lakkapragada A, Kline A, Mutlu O C, et al. Classification of abnormal hand movement for aiding in autism detection: machine learning study. JMIR Biomed Eng, 2022, 7(1): e33771.
33. Washington P, Kline A, Mutlu O C, et al. Activity recognition with moving cameras and few training examples: applications for detection of autism-related headbanging// CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. Yokohama: ACM, 2021.

Previous Article
Research progress on biocomposites based on bioactive glass
Next Article
Research progress on attention level evaluation based on electroencephalogram signals

Journal of Biomedical Engineering

A review of studies on visual behavior analysis aided diagnosis of autism spectrum disorders

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content