To solve the problem of real-time arrhythmia classification, this paper proposes a real-time arrhythmia classification algorithm using deep learning with low latency, high practicality, and high reliability, which can be easily applied to a real-time arrhythmia classification system. In the algorithm, a classifier detects the QRS complex position in real time for heartbeat segmentation. Then, the ECG_RRR feature is constructed according to the heartbeat segmentation result. Finally, another classifier classifies the arrhythmia in real time using the ECG_RRR feature. This article uses the MIT-BIH arrhythmia database and divides the 44 qualified records into two groups (DS1 and DS2) for training and evaluation, respectively. The result shows that the recall rate, precision rate, and overall accuracy of the algorithm’s interpatient QRS complex position prediction are 98.0%, 99.5%, and 97.6%, respectively. The overall accuracy for 5-class and 13-class interpatient arrhythmia classification is 91.5% and 75.6%, respectively. Furthermore, the real-time arrhythmia classification algorithm proposed in this paper has the advantages of practicability and low latency. It is easy to deploy the algorithm since the input is the original ECG signal with no feature processing required. And, the latency of the arrhythmia classification is only the duration of one heartbeat cycle.