Leukemia is a common, multiple and dangerous blood disease, whose early diagnosis and treatment are very important. At present, the diagnosis of leukemia heavily relies on morphological examination of blood cell images by pathologists, which is tedious and time-consuming. Meanwhile, the diagnostic results are highly subjective, which may lead to misdiagnosis and missed diagnosis. To address the gap above, we proposed an improved Vision Transformer model for blood cell recognition. First, a faster R-CNN network was used to locate and extract individual blood cell slices from original images. Then, we split the single-cell image into multiple image patches and put them into the encoder layer for feature extraction. Based on the self-attention mechanism of the Transformer, we proposed a sparse attention module which could focus on the discriminative parts of blood cell images and improve the fine-grained feature representation ability of the model. Finally, a contrastive loss function was adopted to further increase the inter-class difference and intra-class consistency of the extracted features. Experimental results showed that the proposed module outperformed the other approaches and significantly improved the accuracy to 91.96% on the Munich single-cell morphological dataset of leukocytes, which is expected to provide a reference for physicians’ clinical diagnosis.