Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by deficits in social communication and repetitive behaviors. With the rapid development of computer vision, visual behavior analysis aided diagnosis of ASD has got more and more attention. This paper reviews the research on visual behavior analysis aided diagnosis of ASD. First, the core symptoms and clinical diagnostic criteria of ASD are introduced briefly. Secondly, according to clinical diagnostic criteria, the interaction scenes are classified and introduced. Then, the existing relevant datasets are discussed. Finally, we analyze and compare the advantages and disadvantages of visual behavior analysis aided diagnosis methods for ASD in different interactive scenarios. The challenges in this research field are summarized and the prospects of related research are presented to promote the clinical application of visual behavior analysis in ASD diagnosis.