The judgment of the type of arrhythmia is the key to the prevention and diagnosis of early cardiovascular disease. Therefore, electrocardiogram (ECG) analysis has been widely used as an important basis for doctors to diagnose. However, due to the large differences in ECG signal morphology among different patients and the unbalanced distribution of categories, the existing automatic detection algorithms for arrhythmias have certain difficulties in the identification process. This paper designs a variable scale fusion network model for automatic recognition of heart rhythm types. In this study, a variable-scale fusion network model was proposed for automatic identification of heart rhythm types. The improved ECG generation network (EGAN) module was used to solve the imbalance of ECG data, and the ECG signal was reproduced in two dimensions in the form of gray recurrence plot (GRP) and spectrogram. Combined with the branching structure of the model, the automatic classification of variable-length heart beats was realized. The results of the study were verified by the Massachusetts institute of technology and Beth Israel hospital (MIT-BIH) arrhythmia database, which distinguished eight heart rhythm types. The average accuracy rate reached 99.36%, and the sensitivity and specificity were 96.11% and 99.84%, respectively. In conclusion, it is expected that this method can be used for clinical auxiliary diagnosis and smart wearable devices in the future.