Arrhythmia is a kind of common cardiac electrical activity abnormalities. Heartbeats classification based on electrocardiogram (ECG) is of great significance for clinical diagnosis of arrhythmia. This paper proposes a feature extraction method based on manifold learning, neighborhood preserving embedding (NPE) algorithm, to achieve the automatic classification of arrhythmia heartbeats. With classification system, we obtained low dimensional manifold structure features of high dimensional ECG signals by NPE algorithm, then we inputted the feature vectors into support vector machine (SVM) classifier for heartbeats diagnosis. Based on MIT-BIH arrhythmia database, we clustered 14 classes of arrhythmia heartbeats in the experiment, which yielded a high overall classification accuracy of 98.51%. Experimental result showed that the proposed method was an effective classification method for arrhythmia heartbeats.