Journal Pre-proof

Opportunities and challenges of deep learning methods for electrocardiogram data: A

systematic review

Shenda Hong, Yuxi Zhou, Junyuan Shang, Cao Xiao, Jimeng Sun

PII: S0010-4825(20)30169-4
DOI: https://doi.org/10.1016/j.compbiomed.2020.103801
Reference: CBM 103801

To appear in: Computers in Biology and Medicine

Received Date: 28 December 2019

Revised Date: 30 April 2020
Accepted Date: 30 April 2020

Please cite this article as: S. Hong, Y. Zhou, J. Shang, C. Xiao, J. Sun, Opportunities and challenges
of deep learning methods for electrocardiogram data: A systematic review, Computers in Biology and
Medicine (2020), doi: https://doi.org/10.1016/j.compbiomed.2020.103801.

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition
of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of
record. This version will undergo additional copyediting, typesetting and review before it is published
in its final form, but we are providing this version to give early visibility of the article. Please note that,
during the production process, errors may be discovered which could affect the content, and all legal
disclaimers that apply to the journal pertain.

© 2020 Published by Elsevier Ltd.

Opportunities and Challenges of Deep Learning Methods for
Electrocardiogram Data: A Systematic Review
Shenda Honga , Yuxi Zhoub,c , Junyuan Shangb,c , Cao Xiaod and Jimeng Sune
a Department of Computational Science and Engineering, Georgia Institute of Technology, Atlanta, USA
b School of Electronics Engineering and Computer Science, Peking University, Beijing, China
c Key Laboratory of Machine Perception (Ministry of Education), Peking University, Beijing, China
d Analytics Center of Excellence, IQVIA, Cambridge, USA
e Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, USA

ARTICLE INFO ABSTRACT

Keywords: Background: The electrocardiogram (ECG) is one of the most commonly used diagnostic tools in
Deep Learning medicine and healthcare. Deep learning methods have achieved promising results on predictive health-
Deep Neural Network(s) care tasks using ECG signals.
Electrocardiogram (ECG/EKG) Objective: This paper presents a systematic review of deep learning methods for ECG data from both
Systematic Review modeling and application perspectives.
Methods: We extracted papers that applied deep learning (deep neural network) models to ECG data
that were published between January 1st of 2010 and February 29th of 2020 from Google Scholar,
PubMed, and the Digital Bibliography & Library Project. We then analyzed each article according to
three factors: tasks, models, and data. Finally, we discuss open challenges and unsolved problems in
this area.
Results: The total number of papers extracted was 191. Among these papers, 108 were published
after 2019. Different deep learning architectures have been used in various ECG analytics tasks, such
as disease detection/classification, annotation/localization, sleep staging, biometric human identifica-
tion, and denoising.
Conclusion: The number of works on deep learning for ECG data has grown explosively in recent
years. Such works have achieved accuracy comparable to that of traditional feature-based approaches
and ensembles of multiple approaches can achieve even better results. Specifically, we found that a
hybrid architecture of a convolutional neural network and recurrent neural network ensemble using
expert features yields the best results. However, there are some new challenges and problems related
to interpretability, scalability, and efficiency that must be addressed. Furthermore, it is also worth
investigating new applications from the perspectives of datasets and methods.
Significance: This paper summarizes existing deep learning research using ECG data from multi-
ple perspectives and highlights existing challenges and problems to identify potential future research
directions.

1. Introduction Traditionally, automatic ECG analysis has relied on di-

agnostic golden rules. As shown in the top of Figure 1, this
The electrocardiogram (ECG/EKG) is one of the most is a two-stage method that requires human experts to engi-
commonly used non-invasive diagnostic tools for recording neer useful features based on raw ECG data, which are re-
the physiological activities of the heart over a period of time. ferred to as “expert features”, and then deploy decision rules
ECG data can aid in the diagnosis of many cardiovascular or other machine learning methods to generate final results.
abnormalities, such as premature contractions of the atria
Expert features can be categorized [69] into statistical fea-
(PAC) or ventricles (PVC), atrial fibrillation (AF), myocar-
tures (such as heart rate variability [19], sample entropy [3],
dial infarction (MI), and congestive heart failure (CHF). In and coefficients of variation and density histograms [172]),
recent years, we have witnessed the rapid development of frequency-domain features [151, 106], and time-domain fea-
portable ECG monitors in the medical field, such as the Holter tures (such as the Philips 12 lead ECG Algorithm [139]).
monitor [129], and wearable devices in various healthcare
In practice, expert features are automatically extracted using
areas, such as the Apple Watch. As a result, the amount
computer-based algorithms. However, they are still insuf-
of ECG data requiring analysis has grown too rapidly for ficient because they are limited by data quality and human
human cardiologists to keep up. Therefore, analyzing ECG expert knowledge [157, 161, 50].
data automatically and accurately has become a hot research Recently, deep learning methods have achieved promis-
topic. Additionally, many emerging applications, such as ing results in many application areas, such as speech recogni-
biometric human identification and sleep staging, can be im-
tion, computer vision, and natural language processing [89].
plemented based on ECG data.
The main advantage of deep learning methods is that they
sdhong1503@gmail.com (S. Hong); joy_yuxi@pku.edu.cn (Y. Zhou); do not require an explicit feature extraction step using hu-
sjy1203@pku.edu.cn (J. Shang); cao.xiao@iqvia.com (C. Xiao); man experts, as shown in the bottom of Figure 1. Instead,
jimeng.sun@gmail.com (J. Sun) feature extraction is performed automatically and implicitly
ORCID (s):
by deep learning models based on their powerful data learn-

Shenda Hong et al.: Preprint submitted to Elsevier Page 1 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

Expert Machine Learning Models 2.2. Study Selection

We only included published peer-reviewed papers and
Traditional Expert
Results excluded preprints. We focused on papers from the follow-
Methods Features
ing journals/conferences:
ECG Data

Deep Learning • Medical Information and Biomedical Engineering (MI

Results
Methods & BME) : Circulation, Journal of the American College
of Cardiology, Nature Medicine, Nature Biomedical En-
Deep Neural Networks
gineering, American Medical Informatics Association,
Figure 1: Comparisons between traditional methods and deep Journal of American Medical Informatics Association,
learning methods. Journal of American Biomedical Informatics, Transac-
tions on Biomedical Engineering, Computers in Biology
and Medicine, Biomedical Signal Procession and Con-
ing capabilities and flexible processing architectures. Some trol, Computing in Cardiology, Physiological Measure-
studies have experimentally demonstrated that deep learning ment, IEEE Journal of Biomedical and Health Informat-
features are more informative than expert features for ECG ics, Computer Methods and Programs in Biomedicine,
data [67, 69]. The performance of deep learning methods Journal of Electrocardiology, International Conference
is also superior to that of traditional methods on many ECG of the IEEE Engineering in Medicine and Biology So-
analysis tasks, such as disease detection [30] and sleep stag- ciety.
ing [42]. • Artificial Intelligence and Data Mining (AI & DM):
Although some papers have reviewed machine learning Nature Machine Intelligence, ACM SIGKDD Interna-
methods for ECG data [121] (2019), cardiac arrhythmia de- tional Conference on Knowledge Discovery & Data Min-
tection using deep learning [136] (2019), and deep learn- ing, AAAI Conference on Artificial Intelligence, Inter-
ing methods for ECG data [135] (2018), there have no sys- national Joint Conference on Artificial Intelligence, Neu-
tematic reviews focusing on deep learning methods, which ral Information Processing Systems, IEEE Transactions
we consider to be promising methods for mining ECG data. on Pattern Analysis and Machine Intelligence, Interna-
Therefore, we believe it is crucial to conduct a systematic tional Conference on Acoustics, Speech and Signal Pro-
review of existing deep learning methods for ECG data from cessing, IEEE Transactions on Neural Networks and Learn-
the perspectives of model architectures and application tasks. ing Systems, IEEE Transactions on Knowledge and Data
Challenges and problems related to the current research sta- Engineering, IEEE Transactions on Cybernetics, Neuro-
tus are discussed, which should provide inspiration and in- computing, Knowledge-Based System, Expert Systems
sights for future work. with Applications, Pattern Recognition Letters.
• Interdisciplinary Area: Nature Scientific Reports, PLoS
2. Method One, IEEE Access.
2.1. Search Strategy The process of literature searching and selection is illus-
To conduct a comprehensive review, we searched for pa- trated in Figure 2. It is a four-stage process consisting of
pers that deployed deep learning methods (deep neural net- identification, screening, eligibility, and inclusion.
works (DNNs)) for ECG data on Google Scholar, PubMed, The number of papers collected in the identification step
and the Digital Bibliography & Library Project from January is 1,621. After removing duplicates, the number of papers
1st of 2010 to February 29th of 2020. is 1,224. Next, paper eligibility assessment was conducted
Inspired by [136], the following general search terms in coarse-to-fine manner by two independent reviewers: one
were compiled for each database: ("electrocardiogram" OR reviewer screening titles and abstracts, and another reviewer
"electrocardiology" OR "electrocardiography" OR "ECG" OR reading full texts. The exclusion criteria are 1) not in En-
"EKG" OR "arrhythmia") AND ("deep learning" OR "deep glish, 2) not focusing on ECG data, 3) not using deep learn-
neural network" OR "deep neural networks" OR "convolu- ing methods, and 4) no quantitative evaluations. As a result,
tional neural network" OR "cnn" OR "recurrent neural net- 928 papers were excluded by screening and 105 papers were
work" OR "rnn" OR "long short term memory" OR "lstm" excluded by full text assessment. This left 191 papers among
OR "autoencoder" OR "deep belief network" OR "dbn"). All the initial 1,621 papers.
keywords are case insensitive.
To avoid missing papers that do not explicitly mention 2.3. Data Extraction and Analysis
these keywords in their titles, we expanded our search to in- Each paper was analyzed according to the following three
clude all fields in each article. It should be noted that many aspects:
unrelated papers mentioned some of the keywords in their in-
troduction sections or related work sections, which resulted • Task: the targeted application tasks were (1) disease de-
in a large initial set of papers. tection (e.g., specific diseases such as AF, MI, CHF, ST
elevation, or general diagnostic arrhythmia), (2) anno-
tation or localization (e.g., QRS complex annotation, P-

Shenda Hong et al.: Preprint submitted to Elsevier Page 2 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

for these papers are presented in Figure 4. Among the in-

("deep learning" OR "deep neural network" OR "deep neural networks" OR
"convolutional neural network" OR "cnn" OR "recurrent neural network" cluded papers, 108 (approximately 57%) were published af-
OR "rnn" OR "long short term memory" OR "lstm" OR "autoencoder" OR ter 2019, 112 were published by the medical information and
Identification

"deep belief network" OR "dbn")

biomedical engineering community, and only 25 (approxi-
AND Search all fields
in the article mately 13%) were published by the artificial intelligence and
("electrocardiogram" OR "electrocardiology" OR
data mining community. We provide a detailed summary
"electrocardiography" OR "ECG" OR "EKG" OR "arrhythmia") of each paper on our GitHub at https://github.com/hsd1503/
(n = 1,621) DL-ECG-Review.

3.1. Task
Records after removing duplicates 3.1.1. Disease Detection
(n = 1,224) The goal of developing a deep learning model for disease
detection is to map input ECG data to output disease targets
Screening

through multiple layers of neural networks [66]. For exam-

ple, the detection of cardiac arrhythmias (e.g., atrial flutter,
Records screened Records excluded supraventricular tachyarrhythmia, and ventricular trigeminy)
(n = 1,224) (n = 928)
[57] is one of the most common tasks for deep learning mod-
els based on ECG signals. AF detection can be regarded
as a special case of cardiac arrhythmia detection, where all
Eligibility

Full-text assessed Records excluded non-AF rhythms are grouped together [173, 229]. Addition-
(n = 296) (n = 105) ally, a deep learning technique was introduced to monitor
ST changes in ECG data [134]. In [113] and [181], convo-
• Not in English
• Not focus on ECG data
lutional neural networks were applied to automate the detec-
• Not deep learning methods tion of MI and CHF, respectively.
Included

Final list • No quantitative evaluation

(n = 191)
From the perspective of the volume of classification re-
sults, multiple modalities (e.g., ECG, transthoracic echocar-
diogram [8]) can be employed for binary classification tasks
Figure 2: Framework for literature searching and selection. (e.g., patients with paroxysmal AF or healthy patients [143]),
multi-class classification tasks (e.g., detection of acute cog-
nitive stress [61] and decompensation of patient detection
wave annotation, localizing the origin of ventricular ac- [205]), and multi-task classification tasks (e.g., detection of
tivation), (3) sleep staging, (4) biometric human identi- prevalent hypertension, sleep apnea, and diabetes [174]).
fication, (5) denoising, and (6) others.
3.1.2. Localization and Annotation
• Model: deep model architectures include (1) convolu- The localization and annotation of specific waves in ECG
tional neural networks (CNNs), (2) recurrent neural net- signals are of great importance for cardiologists attempting
works (RNNs), (3) combinations of CNNs and RNNs to diagnose cardiac diseases, such as AF. For instance, AEs [51]
(CRNNs), (4) autoencoders (AEs), (5) generative adver- and RNNs [52] have been employed to localize the exit of
sarial networks (GANs), (6) fully connected neural net- ventricular tachycardia in 12-lead ECG data automatically.
works (FCs), and (6) others. We also determined (7) Some studies have focused on localizing of the origins of
whether each paper included traditional expert features premature ventricular issues [207] and MI [220].
or integrated expert knowledge when constructing a deep Most studies that have applied deep learning methods to
model. ECG annotation have focused on using deep learning meth-
• Data: data statistics are (1) the size of the dataset, (2) ods for annotating the fetal QRS complex (detecting Q-waves,
number of channels (number of electrode leads), (3) du- R-waves, and S-waves, and calculating heart rate), which
ration, (4) annotations, (5) sources, (6) collection year, is critical for identifying various arrhythmias, in the MIT-
and (7) number of used papers. BIH arrhythmia dataset [93]. Some studies have used the
QT database of PhysioNet [46] to explore other methods for
We summarized all papers from the perspectives of mod- annotating ECG waves, including P-wave [138] and T-wave
els and tasks, as shown in Figure 3. Finally, we discuss the annotation [10]. Another annotated 12-lead ECG dataset
challenges and problems that existing models cannot handle named Lobachevsky University database (LUDB) is provided
well. by [77].

3. Results 3.1.3. Sleep Staging

Understanding sleep patterns is critical for improving health-
We included 191 papers in our survey and analyzed them care because sleep is a key aspect of our wellbeing. Sleep
based on the aspects of tasks, models, and data. An overview disorders can lead to catastrophes in personal medicine or
of our analysis is presented in Figure 3. General statistics public health [118]. In [97], a sparse AE (SAE) and hidden

Shenda Hong et al.: Preprint submitted to Elsevier Page 3 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

AF patient
QRS complex Awake
Task
P wave

ID

Biometric Human
Disease Detection Annotation Sleep Staging Denoising
Identification

Raw ECG Data

Data
Healthcare Devices Collect Collect
Medical Devices

Input

AE Model
FC
CNN RNN

CRNN RNN
Discriminator +
CNN CNN CNN Generator GAN NN with Expert Features

Figure 3: Overview of analysis based on the aspects of tasks, models, and data.

100 94 120 112 University.

100
Number of Papers

Number of Papers

80
3.1.4. Human Identification
80
60 54 Based on the rapid development of information technol-
60 54 ogy, body sensor networks are reshaping people’s daily lives,
40
24 40 particularly in the context of smart health applications. Biometric-
25
20 14 20 based human identification is a promising technology for au-
5 tomatic and accurate individual recognition based on various
0 0
e2016 2017 2018 2019 9th 2020 BME AI & DM ciplinary body sensor data, including heart rate, temperature, and ac-
Befor MI & is
Feb 2 Interd tivity. Zhang et al. constructed a CNN-based biometric hu-
man identification system, evaluated their system using eight
Figure 4: General statistics for all papers. datasets from PhysioNet (CEBSDB, WECG, FANTASIA,
NSRDB, STDB, MITDB, AFDB, and VFDB), and achieved
state-of-the-art performance using a CNN. Similarly, the PTB
Markov model (HMM) were combined to detect obstruc- diagnostic dataset (549 15 channel ECG records from 290
tive sleep apnea (OSA) using the PhysioNet challenge 2000 subjects) [86] and CYBHi dataset (65 subjects between 21.64
dataset. A CNN has also been used to detect OSA [118]. and 40.56 y old) [54] were also used to evaluate CNN-based
In addition to OSA detection, CNN-based deep learn- biometric human identification methods.
ing architectures have also been employed for the multi-class Some studies have combined CNNs with other approaches. Zhao
classification of OSA with hypopnea [176], which is the most et al. proposed a novel ECG biometric authentication system
common sleep-related breathing disorder, based on single- that incorporates the generalized S-transformation and CNN
lead ECG data. Some studies have focused on sleep stage techniques. A secure multimodal biometric system using a
identification outside of OSA detection. For example, a long CNN and Q-Gaussian multi-support vector machine (SVM)
short-term memory (LSTM) network was employed in [188] based on a different levels of fusion was developed in [54].
to analyze a multi-channel physiological signal dataset (elec- Additionally, residual networks have been employed to
troencephalogram, electrooculogram, and electromyogram develop ECG biometric authentication methods that can im-
signals), which was collected by the Sleep Disorder Diagno- prove generalization for ECG signals sampled in the differ-
sis Center of Xijing Hospital at the Fourth Military Medical

Shenda Hong et al.: Preprint submitted to Elsevier Page 4 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

Disease Detection Annotation Sleep Biometric Denoising Others

or Localiza- Staging Identification
tion
CNN [80, 82, 212, 113, 1, 67, 201, 5, 191, 153, [207, 20, [118, 176, [219, 225, 54, [223, 27, [156, 9,
105, 43, 81, 143, 196, 192, 211, 7, 41, 47, 226, 25, 214, 35] 26, 29, 86, 39, 40] 120, 11,
102, 216, 100, 137, 200, 141, 167, 78, 168, 93, 53, 76, 221, 65, 101] 160, 6,
128, 195, 154, 18, 75, 159, 31, 62, 189, 71, 175, 166, 74, 208, 162]
63, 193, 169, 95, 94, 61, 21, 134, 103, 164, 124]
8, 57, 98, 69, 130, 178, 224, 160, 227, 13,
16, 88, 190, 58, 170, 72, 131, 177, 59, 55]
RNN [117, 158, 209, 23, 173, 147, 111, 228, 24, [10, 138, 52] [188, 35] N.A. [144] [60]
31, 62, 179, 99, 45, 220, 202, 155]
CRNN [231, 186, 174, 12, 37, 171, 132, 146, 163, [215] N.A. [114] N.A. [34, 148,
187, 197, 4, 110, 125, 181, 182, 68, 229, 140, 205, 49,
73, 184, 210, 96, 198, 2, 222, 112, 126, 104] 230, 142]
AE [36, 194, 179, 218, 210] [52, 51] [97] N.A. [27, 199] [44, 213,
84]
GAN [182, 45, 227] N.A. N.A. N.A. [180] [90, 230,
208, 60,
162]
NN with Ex- [67, 5, 191, 117, 43, 47, 71, 99, 173, 78, 69, N.A. N.A. [54] [223] N.A.
pert Features 155, 206, 96, 88, 58, 141, 167, 137]
FC & Others [203, 204, 119, 133, 116, 79, 38, 127, 206] N.A. [35] [92] N.A. [217, 85]

Table 1
Summary of papers from the perspectives of models and tasks.

ent environments for similar tasks [29]. In [92], features dencies (e.g., classifying pulsed rhythm or pulseless electri-
were extracted using a principal component analysis network cal activity [34], detecting hypoglycemic events in healthy
(PCANet). Next, a robust Eigen ECG network was applied individuals [142], predicting the need for urgent revascu-
to the time-frequency representations of ECGs for personal larization [49], and classifying driver stress levels [148]).
identification. A bidirectional gated recurrent unit (GRU)- A combination of a CNN and HMM was used to segment
based method was also proposed for human identification ECG data into standard component waveforms and intervals
based on ECG data [114]. [175].
Additionally, AEs [44] have been used for the recon-
3.1.5. Denoising struction and analysis of biomedical signals. GANs have
The ECG signal acquisition process is often accompa- been employed for ECG generation [230] and synthesis [90],
nied by a large amount of noise, which negatively affects the as well as anomaly beat detection [227]. Some studies have
accuracy of diagnosis, particularly in telemedicine environ- used DNNs to process raw ECG waveform data for future
ments. In [40], an encoder-decoder CNN, which is a widely risk prediction (e.g., assessing the risk of future cardiac dis-
used deep learning technique, was employed for ECG de- ease [120], sudden cardiac arrest risk prediction [160], mor-
noising. Similarly, Xiong et al. and Chiang et al. proposed tality prediction [145], and physiological measures for health
fully-convolutional-network-based denoising AE (DAE) meth- prediction [6]).
ods for ECG signal denoising. In [91], a bidirectional recur-
rent DAE was used to perform photoplethysmography fea- 3.2. Models
ture accentuation for pulse waveform analysis. 3.2.1. CNN
Additionally, GANs have been employed to accumulate CNNs represent a class of DNNs that are widely applied
knowledge regarding the distribution of ECG noise continu- for image classification, natural language processing, and
ously through a minimax game between a generator and dis- signal analysis. Such networks can automatically extract hi-
criminator, where the quality of denoised signals was evalu- erarchical patterns in data using stacked trainable small fil-
ated versus an SVM algorithm [180]. Zhao et al. addressed ters or kernels, meaning they require relatively little prepro-
noise by developing a noise rejection method based on a cessing compared to handcrafted features. A typical CNN
combination of a modified frequency slice wavelet transform is composed of several convolutional layers followed by a
(WT) and CNN. batch normalization layer, nonlinear activation layer, dropout
layer, pooling layer, and classification layer (e.g., fully con-
3.1.6. Others nected layer), as discussed in [57]. In some works, SVMs,
The CNNs used in other types of studies focused on emo- boosting classifier trees, and RNNs have been used as alter-
tion detection [156], drug assessment [9], data compression native fully connected layers to summarize global features
[217], transfer learning in ECG classification from human in CNNs. CNNs can achieve superior performance and fast
data to horse data [177], etc. Combinations of CNNs and computation based on shared-weight architectures and par-
LSTM networks have been used to learn long-term depen- allelization.

Shenda Hong et al.: Preprint submitted to Elsevier Page 5 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

Two types of CNN are commonly used for ECG clas- 3.2.3. CRNN
sification, namely the 1D CNN and 2D CNN. A 1D CNN A CRNN, as the name suggests, combines CNN and RNN
operates by applying kernels along the temporal dimension modules. This is a preferred architecture for handling long
of raw ECG data, whereas a 2D CNN typically operates on ECG signals with varied sequence lengths and multi-channel
transformed ECG data, such as distance distribution matri- inputs. A 1D CNN [62] or 2D CNN [163] is used to extract
ces based on entropy calculations [102], gray-level co-occurrencelocal features from an ECG sequence. An RNN then sum-
matrices [169], or combined features, such as morphologies, marizes local features along the time dimension to generate
RR intervals, and beat-to-beat correlations [95]. However, global features.
there is some controversy regarding the use of multi-head DeepHeart [12] follows the CRNN framework to per-
ECG or time-frequency ECG spectrograms extracted using form cardiovascular risk prediction for heart rate sequences
the WT, fast Fourier transform (FFT), or short-term Fourier extracted from ECG data and utilizes an AE model (see Sec. 3.2.4)
transform. Some works, such as [71], have directly applied to initialize model weights to achieve enhanced performance.
2D-CNNs to such spectrograms, but this introduces issues To provide interpretable diagnosis, MINA [68] incorporates
related to varying frequency resolutions, meaning most sig- a CRNN with a multi-level attention mechanism with beat-
nal characteristics are reflected by intra-component patterns, level, rhythm-level, and frequency-level expert features based
rather than inter-component behaviors. To overcome this is- on medical domain knowledge.
sue, a shared 1D CNN was adopted in [80, 113]. Similarly,
a multi-scale 1D CNN was adopted in [219] for biometric 3.2.4. AE
human identification. An AE is a type of neural network composed of an en-
Additionally, a 2D CNN can be applied to a one-head coder module and decoder module for learning embeddings
ECG signal, which is treated as an image. In such cases, in an unsupervised manner. The goal of an AE is to learn a
a pre-trained ResNet, DenseNet, or Inception-Net trained on reduced dimensional representation using the encoder mod-
an ImageNet dataset [32] can be fine tuned on an ECG dataset ule while the decoder attempts to reconstruct an original in-
to perform heartbeat disease detection [189] and analyze ST put from this reduced representation. There are three com-
changes [182]. Particularly for localization tasks, such as monly used variants of the AE, namely the DAE, SAE, and
QRS detection, a fully convolutional network with a large contractive AE (CAE). DAEs take partially corrupted inputs
kernel size can be applied to an ECG image to reduce that and are trained to recover original undistorted inputs. SAEs
image’s height to one while maintaining an output length and CAEs utilize different regularization methods, such as
equal to the input length to derive a QRS window label [93]. KL-divergence and the Frobenius norm of the Jacobian ma-
Some advanced techniques, such as an atrous spatial pyra- trix, to learn robust representations.
mid pooling module, can be used to exploit multi-scale fea- Stacked DAEs [194], SAEs [213, 218], and CAEs [199]
tures in ECG data. Additionally, active learning [193] and are widely used for ECG denoising purposes because ECG
data augmentation [182, 229] can be incorporated into CNN signals are often contaminated by various types of noise,
frameworks to handle imbalance problems and further im- such as baseline wandering, electrode contact noise, and mo-
prove accuracy. tion artifacts, which may lead to inaccurate interpretations.
In practice, the effectiveness of an AE is determined by the
3.2.2. RNN choice of encoder and decoder modules. As introduced in
An RNN is a type of neural network designed to model Sections 3.2.1, 3.2.2, and 3.2.3, CNNs, RNNs, and CRNNs
sequential data, such as time series, event sequences, and are typically combined to serve as encoder and decoder mod-
natural language. In an RNN, the output from the previous ules. The authors of [27] utilized fully convolutional net-
step is used as the input for the current step. By iteratively works as an encoder and decoder while [87] used BiLSTM.
updating hidden states and memory, an RNN is capable of To improve classification performance, the authors of [44,
remembering information in sequential order. 147, 210] simultaneously conducted the reconstruction and
In particular, for ECG data, RNNs are a logical choice for classification procedures. For examples, the authors of [210]
both capturing temporal dependencies and handling inputs utilized a 1D CNN AE to first compress large ECG signals
of various lengths. GRU/LSTM and bidirectional-LSTM with minimum loss, and then fed the compressed signals into
(BiLSTM) are commonly used RNN variants that handle a an LSTM network to recognize arrhythmias automatically.
critical problem called the vanishing gradient, which nega-
tively affects classical RNNs. Two small LSTM networks 3.2.5. GAN
were employed in [155] to combine raw ECG features and A GAN is a class of neural network framework that was
WT features for continuous real-time execution on wearable invented by Goodfellow et al. [48]. This type of model con-
devices. In [99], an attention mechanism was incorporated sists of two sub-models: a generative model 𝐺 that captures
into a BiLSTM network to improve performance and inter- the data distribution of a training dataset in a latent repre-
pretability by visualizing attention weights. sentation and a discriminative model 𝐷 that determines the
probability that a sample produced by the generator comes
from the true data distribution. These two models are trained
iteratively to conduct a minimax game.

Shenda Hong et al.: Preprint submitted to Elsevier Page 6 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

GAN application has increased rapidly, particularly in I

areas such as image generation [17] and language genera-
II
tion [107]. Recently, GANs have been applied to tackle the
data imbalance challenge in ECG data. For example, the III
authors of [182] proposed an abnormality detection model aVR
for ECG signals based on a CRNN framework and used a
aVL
GAN composed of multiple 1D CNNs to perform data aug-
mentation. Their model achieved excellent performance for aVF

class-imbalanced datasets. The authors of [189] utilized a V1

GAN to perform denoising of ECG data and the authors
V2
of [230] proposed a GAN composed of a BiLSTM network
(generator) and CNN (discriminator) to generate synthetic V3

ECG data to train a deep learning model. Similarly, a model V4

called PGAN [45] was proposed to perform personalized V5
ECG classification, where subject-specific labeled data is
V6
sparse. Specifically, a GAN was optimized using a special-
ized loss function and trained to generate personalized syn- VX
thetic ECG signals for different arrhythmias. VY

3.2.6. NN with Expert Features VZ

0 250 500 750 1000 1250 1500 1750 2000

ENCASE [67] suggests that expert features can be di- Figure 5: Example 15 lead ECG data collected from medical
vided into three categories: 1) statistical features (e.g., count, devices. The figure presents 10 s of 15 lead ECG data from an
mean, maximum, and minimum), 2) signal processing fea- MI patient in the PTB Diagnostic ECG Database.
tures that transform ECG data from the time domain into the
frequency domain (e.g., FFT, WT, and Shannon entropy), 1.25

and 3) medical features based on medical domain knowl- 1.00

0.75
0.50

edge (e.g., features based on P-, Q-, R-, S- and T-waves, 0.25
0.00
0.25

sample entropy, and the coefficient of variation and density

0 2000 4000 6000 8000

histograms). Figure 6: Example single-lead ECG data collected from health-

All of these methods can benefit significantly from ex- care devices. The figure presents 30 s of single-lead ECG data
pert features, although additional effort is required to extract from an AF patient in the PhysioNet Computing in Cardiology
such features compared to raw morphological features. En- Challenge 2017 dataset.
semble methods [67, 71, 191, 141, 167, 137] combine ex-
pert features with raw morphological features and incorpo-
rate various tree models with DNNs to achieved better results ing to the state-of-the-art baselines discussed above.
compared to individual models.
3.3. Data
3.2.7. FC & Others Most of the works identified in this review (150 out of
Some works have relied on FCs for disease detection and 191) used open-source datasets, which makes it easier to
classification [38, 79], particularly for extremely short ECG perform follow-up works and reproduction. A summary of
sequences (e.g., 10 RR intervals) [79]. Stacked restricted the 10 most frequently used open-source databases is pro-
Boltzmann machines, which are a type of generative stochas- vided in Table 2. We list the following parameters for each
tic DNN, are also used to process raw ECG data in combi- database, which can be used to classify the databases into
nation with beat alignment processing to classify heartbeat further subtypes:
types [203].
• Sources. Most databases were collected from medical
Other works have borrowed ideas from the computer vi-
devices, but a few were collected from healthcare de-
sion field. For example, U-net [152], which is based on
vices. The biggest difference between such devices is
a fully convolutional network and consists of a contracting
that medical device data typically have more leads than
path and expansive path, is widely used for image segmenta-
healthcare device data, meaning medical device data are
tion tasks. The authors of [133] proposed a modified U-net
more informative. However, medical device data are also
to handle various lengths of ECG sequences for classifica-
more difficult to collect. Healthcare ECG monitor de-
tion and R-peak detection. PCANet [22], which is an im-
vices, such as smart hardware and wristbands, are be-
age classification model based on cascaded principal com-
coming increasingly common.
ponent analysis, binary hashing, and block-wise histograms,
was modified for biometric human identification tasks in a • Number of leads. A standard 12 lead ECG (or even 15-
non-stationary ECG noise environment [92]. However, the lead ECG) system can observe more abnormalities than a
effectiveness of this method must still be evaluated accord- single-lead ECG (similar to lead I in a 12 lead ECG). For
example, posterior wall MI can only be detected by chest

Shenda Hong et al.: Preprint submitted to Elsevier Page 7 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

Database Records Leads Duration Annotations Source Year Papers

MIT-BIH Arrhythmia 47 2 30 minutes Beat Level, rhythm level. (annotations Boston’s Beth Is- 1975- 54
Database 1 are keeping updating, please refer to rael Hospital 1979
https://archive.physionet.org/physiobank/annotations.shtml)
PhysioNet Computing 8528 train, 1 30 seconds Rhythm level: normal, AF, other, noise AliveCor health- 2017 33
in Cardiology Chal- 3658 test care device
lenge 2017 2
PTB Diagnostic ECG 549 15 Several Rhythm level: Myocardial infarction, Cardiomy- National Metrol- 1995 16
Database 3 minutes opathy/Heart failure, Bundle branch block, Dys- ogy Institute of
rhythmia, Myocardial hypertrophy, Valvular heart Germany
disease, Myocarditis, Miscellaneous, Healthy con-
trols
MIT-BIH Atrial Fibril- 25 2 10 hours Rhythm level: AF, atrial flutter (AFL), AV junc- Boston’s Beth Is- 1983 8
lation Database 4 tional rhythm, and others rael Hospital
2018 China Physiolog- 6877 train, 12 15 seconds Rhythm level: AF, I-AVB, LBBB, RBBB, PAC, 11 hospitals 2018 7
ical Signal Challenge 5 2954 test PVC, STD, STE
QT Database 6 105 2 15 minutes Onset, peak, and end markers for P, QRS, T, and Compiled from 1997 6
U waves several existing
databases
MIT-BIH Normal Si- 18 2 24 hours Beat Level: normal Boston’s Beth Is- N.A. 5
nus Rhythm Database rael Hospital
7

St Petersburg IN- 75 12 30 minutes Rhythm level: Acute MI, Transient ischemic at- St. Petersburg 2003 3
CART 12-lead Ar- tack (angina pectoris), Prior MI, Coronary artery Institute of Cardi-
rhythmia Database disease with hypertension, Sinus node dysfunc- ological Technics
8 tion, Supraventricular ectopy, Atrial fibrillation or
SVTA, WPW, AV block, Bundle branch block
MIT-BIH Malignant 22 2 30 minutes ventricular tachycardia, ventricular flutter, and Compiled from 1986 3
Ventricular Ectopy ventricular fibrillation two separate
Database 9 databases
CU Ventricular Tach- 35 1 8 minutes ventricular tachycardia, ventricular flutter, and Creighton Univer- 1986 3
yarrhythmia Database ventricular fibrillation sity Cardiac Cen-
10 ter

Table 2
Summary of databases.

leads (V1 to V4) and no abnormalities will be detected by The following databases were used by many of the se-
a single lead. Examples of 15 lead ECG data from med- lected papers:
ical devices and single lead ECG data from healthcare
devices are presented in Figures 5 and 6, respectively. • The MIT-BIH Arrhythmia Database [123] (54 papers)
consists of 48 half-hour ECG records from 47 subjects at
• Duration. Short-term ECG data (less than several min-
Boston’s Beth Israel Hospital (now the Beth Israel Dea-
utes) and long-term ECG data can complement each other.
coness Medical Center). Each ECG data sequence has
Short-term ECG data is cheaper and easier to collect.
an 11 bit resolution over a 10 mV range with a sampling
Many cardiac diseases can be detected based on short-
frequency of 360 Hz. This dataset is fully annotated with
term ECG data, so such data represent the primary di-
both beat-level and rhythm-level diagnoses.
agnostic tool in outpatient departments. However, long-
term ECG can help to detect diseases with intermittent • The PhysioNet Computing in Cardiology Challenge 2017
symptoms, such as paroxysmal ventricular fibrillation (VF) dataset [30] (33 papers) contains 8,528 de-identified ECG
and AF. recordings with durations ranging from 9 s to just over
60 s that were sampled at 300 Hz by an AliveCor health-
• Annotations. Annotations include ECG measurement an-
care device. Among these recordings, 5154 are normal,
notations (onset, peak, and end markers for P-, QRS-, T-,
717 recordings are AF, 2,557 recordings are others, and
and U-waves), beat-level annotations (PAC, PVC, etc.),
46 recordings are noise. Additionally, 3,658 test record-
and rhythm-level annotations (covers both beat-level an-
ings are private for scoring. This dataset was collected
notations and other diseases such as AF and VF). Anno-
by healthcare devices.
tation requires huge effort by medical experts.
• The PTB Diagnostic ECG Database [15] (16 papers) con-
1 https://physionet.org/content/mitdb/1.0.0/ tains 549 15 channel ECG records from 290 subjects.
2 https://www.physionet.org/content/challenge-2017/1.0.0/ The sampling rate reaches as high as 10 kHz. Among
3 https://physionet.org/content/ptbdb/1.0.0/
these subjects, 216 have one of eight types of heart dis-
4 https://physionet.org/content/afdb/1.0.0/
5 http://2018.icbeb.org/Challenge.html
ease and 52 are healthy control, while 22 are unknown.
6 https://physionet.org/content/qtdb/1.0.0/ • The MIT-BIH Atrial Fibrillation Database [122] (8 pa-
7 https://physionet.org/content/nsrdb/1.0.0/
pers) includes 25 10 h long-term 2 lead ECG recordings
8 https://physionet.org/content/incartdb/1.0.0/
9 https://physionet.org/content/vfdb/1.0.0/ with a sampling rate of 250 Hz for human subjects with
10 https://physionet.org/content/cudb/1.0.0/ AF (mostly paroxysmal). The original recordings were
collected at Boston’s Beth Israel Hospital using ambula-

Shenda Hong et al.: Preprint submitted to Elsevier Page 8 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

Method Model 𝐹1𝑁 𝐹1𝐴 𝐹1𝑂 𝐹1𝑃 𝐹1𝑁𝐴𝑂𝑃 𝐹1𝑁𝐴𝑂

but they both focused on short-term ECG recordings. Re-
searchers would welcome a new high-quality long-term ECG
[173] RNN + Expert Features 0.9030 0.8547 0.7366 0.5622 0.7641 0.8314
[57] CNN 0.91 0.84 0.74 NA NA 0.83
[141]
[67]
CNN + Expert Features
CRNN + Expert Features
0.9151
0.9117
0.8247
0.8128
0.7437
0.7505
NA
0.5671
NA
0.7605
0.8278
0.8250
dataset with annotations and such a dataset would certainly
[167]
[231]
CNN + Expert Features
CRNN
0.9142
0.9090
0.8153
0.8221
0.7370
0.7319
NA
0.5676
NA
0.7577
0.8222
0.8210
inspire new innovative studies.
[187] CRNN 0.9028 0.8221 0.7324 NA NA 0.8191
[200] CNN 0.9031 0.8203 0.7310 0.5251 0.7449 0.8181
[137] CNN + Expert Features 0.9056 0.8284 0.7204 NA NA 0.8181 4.2. Interpretability
Deep learning models are often considered to be black-
Table 3
box models because they typically have many model param-
Comparison of deep learning methods on the PhysioNet Com-
eters or complex model architectures, which makes it dif-
puting in Cardiology Challenge 2017 dataset. We only include
methods with an 𝐹1 score over 0.8 reported for the hidden test ficult for a human to understand why a particular result is
set. generated by such a model. This challenge is much more se-
vere in the medical domain because diagnoses without any
explanation are not acceptable for medical experts.
tory ECG recorders with a 0.1 to 40 Hz recording band- There have been a few works focusing on enhancing the
width. interpretability of ECG deep learning methods. For exam-
ple, some works have [99, 69] explicitly added interpretable
• 2018 The China Physiological Signal Challenge dataset
expert features to deep learning models that can be used for
[109] (seven papers) contains 6,877 (3178 female, 3699
partial interpretation. Others have used multi-level atten-
male) 12 lead ECG recordings with durations ranging
tion weights [68] or attribute scores [168] to generate salient
from 6 s to just over 60 s, which were collected at 11
maps based on raw ECG data. There have also been sev-
hospitals with a sampling rate of 500 Hz. Among these
eral works [163, 69] on generating lower-dimensional em-
recordings, 918 are normal, 1,098 recordings are AF, 704
beddings using t-distributed stochastic neighbor embeddings
are first-degree atrioventricular block, 207 recordings are
[115] to derive interpretable results.
left-bundle branch block (LBBB), 1,695 are right-bundle
There are two worthwhile research directions regarding
branch block (RBBB), 556 are PAC, 672 recordings are
interpretability. The first is how to interpret complex deep
PVC, 825 are ST segment depression, and 202 are ST
learning models using relatively simple models. For exam-
segment elevation. Additionally, 2,954 test recordings
ple, one can first construct a black-box deep learning model
are private for scoring.
for a specific task, and then construct a separate interpretable
A summary of the deep learning methods tested on the simple model that matches the deep learning model’s predic-
PhysioNet Computing in Cardiology Challenge 2017 dataset tions, and then interpret prediction results based on the sim-
is provided in Table 3. We only include methods with an ple model [149, 150]. The second one is how to construct an
𝐹1 score over 0.8 reported for the hidden test set. One can interpretable deep model directly. For example, when de-
find the official leaderboards for the PhysioNet Computing in signing a deep model architecture, one can borrow neuron
Cardiology Challenge 2017 at https://physionet.org/content/ connection concepts from tree-based models [183] or add at-
challenge-2017/1.0.0/, 2018 China Physiological Signal Chal- tention mechanisms to hidden layers [28, 68] because such
lenge at http://2018.icbeb.org/Challenge.html, and PhysioNet mechanisms can be more easily understood by humans.
Computing in Cardiology Challenge 2020 at https://physionetchallenges.
github.io/2020/.
4.3. Efficiency
Because deep models are complex, it is difficult to de-
ploy large models on portable healthcare devices, which is
4. Discussion of Opportunities and Challenges a major obstacle to applying deep learning models in real-
In this section, we will discuss the current challenges and world applications. In this context, one promising research
problems related to deep learning based on ECG data. Ad- direction is the model compression technique. For example,
ditionally, potential opportunities are also identified in the knowledge distillation is commonly used to transform large
context of these challenges and problems. and powerful models into simpler models with a minor de-
crease in accuracy [64]. Additionally, one can use quantiza-
4.1. Data Collection tion, weight sharing, and careful coding of network weights
As shown in Table 2, there is no standard regarding col- [56] to compress a large model.
lection procedures. Different studies have used various num-
bers of leads, durations, sources (subject backgrounds), etc. 4.4. Integration with Traditional Methods
This makes it difficult to compare results between different Most existing deep learning models are trained in an end-
datasets fairly. Additionally, high-quality data and annota- to-end manner, making them difficult to integrate with tradi-
tions are difficult to acquire, so many current works are still tional expert-feature-based methods after model training is
using the MIT-BIH Arrhythmia Database, which was col- completed [135].
lected over 40 y ago. The most recent single-lead PhysioNet There are two main research directions for tackling this
Computing in Cardiology Challenge 2017 and 12 lead 2018 issue. The first is to use existing expert knowledge to design
China Physiological Signal Challenge used high-quality data, DNN architectures [70]. For example, the authors of [68]
proposed guiding multilevel attention weights using expert

Shenda Hong et al.: Preprint submitted to Elsevier Page 9 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

features for modeling ECG data. The second is to consider 5. Conclusion

deep models as feature extractors and explicitly extract la-
tent embeddings from deep learning models. To some ex- In this paper, we systematically reviewed existing deep
tent, neural networks with expert features represent a step in learning (DNN) methods for ECG data from the perspec-
this direction. Therefore, one can easily combine expert fea- tives of models, data, and tasks. We found that deep learning
tures with deep features and construct traditional machine methods can generally achieve better performance than tra-
learning methods on such features. ditional methods for ECG modeling. However, there are still
some unresolved challenges and problems related to these
4.5. Imbalanced Labels deep learning methods. Our main contributions are twofold.
ECG disease labels are very likely to have very biased First, we provided a systematic overview of various deep
distributions because most severe diseases occur rarely, but learning methods that can be employed in real applications.
are very important. It is difficult to train an effective deep Second, we highlighted some potential future research op-
learning method with a large number of model parameters portunities.
using small datasets of disease labels.
There are two main methods for handling this problem. Acknowledgement
The first is data augmentation, such as data preprocessing us-
ing the side-and-cut technique, or generating synthetic train- This work was partially supported by the National Sci-
ing datasets using generative models, such as variational AEs ence Foundation awards IIS-1418511, CCF-1533768 and IIS-
[33] or GANs [48]. The second is to design new loss func- 1838042, and the National Institute of Health awards NIH
tions, such as focal loss [108], or new model training schema, R01 1R01NS107291-01 and R56HL138415.
such as few-shot learning [185].
For example, the method proposed in [229] uses a skewness- References
driven dynamic data augmentation technique to balance data [1] Acharya, U.R., Oh, S.L., Hagiwara, Y., Tan, J.H., Adam, M., Ger-
distributions. The authors of [227] and [45] deployed GANs tych, A., San Tan, R., 2017. A deep convolutional neural network
to improve classification accuracy and anomaly detection, model to classify heartbeats. Computers in biology and medicine
respectively. The method proposed in [75] eliminates the 89, 389–396.
[2] Al Rahhal, M.M., Bazi, Y., Almubarak, H., Alajlan, N., Al Zuair,
negative effects of imbalanced data from the perspectives of M., 2019. Dense convolutional networks with focal loss and im-
resampling, data features, and algorithms. age generation for electrocardiogram classification. IEEE Access 7,
182225–182237.
4.6. Multimodal Data [3] Alcaraz, R., Abásolo, D., Hornero, R., Rieta, J.J., 2010. Optimal
Currently, most works have only considered ECG data parameters study for sample entropy-based atrial fibrillation organi-
zation analysis. computer methods and programs in biomedicine 99,
for analysis. However, a few works have considered joint 124–132.
analysis with other data sources. For example, the method [4] Andersen, R.S., Peimankar, A., Puthusserypady, S., 2019. A deep
proposed in [205] predicts intensive care unit patient moral- learning approach for real-time detection of atrial fibrillation. Expert
ities based on a combination of interventions, lab tests, vi- Systems with Applications 115, 465–473.
tal signs, and ECG data. The authors of [54] designed a [5] Andreotti, F., Carr, O., Pimentel, M.A., Mahdi, A., De Vos, M.,
2017. Comparing feature-based classifiers and convolutional neu-
multi-modal biometric authentication system using a fusion ral networks to detect arrhythmia from short segments of ecg, in:
of ECG and fingerprint data. 2017 Computing in Cardiology (CinC), IEEE. pp. 1–4.
Based on the development of medical and healthcare de- [6] Attia, Z.I., Friedman, P.A., Noseworthy, P.A., Lopez-Jimenez, F.,
vices, many vital signs, such as temperature, respiratory rate, Ladewig, D.J., Satam, G., Pellikka, P.A., Munger, T.M., Asirvatham,
and blood pressure can be collected simultaneously with ECG S.J., Scott, C.G., et al., 2019a. Age and sex estimation using artificial
intelligence from standard 12-lead ecgs. Circulation: Arrhythmia
data. However, these data are not always synchronized with and Electrophysiology 12, e007284.
ECG timelines and their sampling frequencies vary signif- [7] Attia, Z.I., Kapa, S., Lopez-Jimenez, F., Asirvatham, S.J., Friedman,
icantly, meaning they can be regarded as multimodal data. P.A., Noseworthy, P.A., 2018a. Electrocardiographic screening for
There is a potential opportunity to study how to design mod- atrial fibrillation while in sinus rhythm using deep learning. Circu-
els that are capable of utilizing such multimodal data simul- lation 138, A15871–A15871.
[8] Attia, Z.I., Kapa, S., Lopez-Jimenez, F., McKie, P.M., Ladewig,
taneously to improve task performance compared to models D.J., Satam, G., Pellikka, P.A., Enriquez-Sarano, M., Noseworthy,
trained on any individual modality. P.A., Munger, T.M., et al., 2019b. Screening for cardiac contrac-
tile dysfunction using an artificial intelligence–enabled electrocar-
4.7. Emerging Interdisciplinary Studies diogram. Nature medicine 25, 70.
Finally, there have been a few innovative interdisciplinary [9] Attia, Z.I., Sugrue, A., Asirvatham, S.J., Ackerman, M.J., Kapa, S.,
Friedman, P.A., Noseworthy, P.A., 2018b. Noninvasive assessment
studies, a few of which are listed below. 1) Safe driving in-
of dofetilide plasma concentration using a deep learning (neural net-
tensity and cardiac reaction time assessment based on ECG work) analysis of the surface electrocardiogram: A proof of concept
signals [83]. 2) Emotion detection based on ECG signals study. PloS one 13, e0201059.
[165]. 3) Mammalian ECG analysis [14]. 4) Estimating age [10] Attin, M., Cogliati, A., Duan, Z., 2017. Annotating ecg signals with
and gender based on ECG data [6]. The key to the success deep neural networks. Circulation 136, A19056–A19056.
[11] Baalman, S.W., Ramos, L.A., Lopes, R.R., Van Der Stuijt, W.,
of such studies is adequate data support.
Bleijendaal, H., Brouwer, T.F., Marquering, H.A., Driessen, A.H.,

Shenda Hong et al.: Preprint submitted to Elsevier Page 10 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

Knops, R.E., de Groot, J.R., 2019. A deep learning model to pre- ral Information Processing Systems 2016, December 5-10, 2016,
dict outcome after thoracoscopic surgery for atrial fibrillation using Barcelona, Spain, pp. 3504–3512.
single beat electrocardiographic samples. Circulation 140, A15889– [29] Chu, Y., Shen, H., Huang, K., 2019. Ecg authentication method
A15889. based on parallel multi-scale one-dimensional residual network with
[12] Ballinger, B., Hsieh, J., Singh, A., Sohoni, N., Wang, J., Tison, G.H., center and margin loss. IEEE Access 7, 51598–51607.
Marcus, G.M., Sanchez, J.M., Maguire, C., Olgin, J.E., et al., 2018. [30] Clifford, G.D., Liu, C., Moody, B., Li-wei, H.L., Silva, I., Li, Q.,
Deepheart: semi-supervised sequence learning for cardiovascular Johnson, A., Mark, R.G., 2017. Af classification from a short single
risk prediction, in: Thirty-Second AAAI Conference on Artificial lead ecg recording: the physionet/computing in cardiology challenge
Intelligence. 2017, in: 2017 Computing in Cardiology (CinC), IEEE. pp. 1–4.
[13] Baloglu, U.B., Talo, M., Yildirim, O., San Tan, R., Acharya, U.R., [31] Dang, H., Sun, M., Zhang, G., Qi, X., Zhou, X., Chang, Q., 2019. A
2019. Classification of myocardial infarction with multi-lead ecg novel deep arrhythmia-diagnosis network for atrial fibrillation clas-
signals and deep cnn. Pattern Recognition Letters 122, 23–30. sification using electrocardiogram signals. IEEE Access .
[14] Behar, J.A., Rosenberg, A.A., Weiser-Bitoun, I., Shemla, O., [32] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L., 2009.
Alexandrovich, A., Konyukhov, E., Yaniv, Y., 2018. Physiozoo: a Imagenet: A large-scale hierarchical image database, in: 2009 IEEE
novel open access platform for heart rate variability analysis of mam- conference on computer vision and pattern recognition, Ieee. pp.
malian electrocardiographic data. Frontiers in physiology 9, 1390. 248–255.
[15] Bousseljot, R., Kreiseler, D., Schnabel, A., 1995. Nutzung der ekg- [33] Doersch, C., 2016. Tutorial on variational autoencoders. arXiv
signaldatenbank cardiodat der ptb über das internet. Biomedizinis- preprint arXiv:1606.05908 .
che Technik/Biomedical Engineering 40, 317–318. [34] Elola, A., Aramendi, E., Irusta, U., Picón, A., Alonso, E., Owens, P.,
[16] Brisk, R., Bond, R., Banks, E., Piadlo, A., Finlay, D., McLaugh- Idris, A., 2018. Deep learning for pulse detection in out-of-hospital
lin, J., McEneaney, D., 2019. Deep learning to automatically inter- cardiac arrest using the ecg, in: 2018 Computing in Cardiology Con-
pret images of the electrocardiogram: Do we need the raw samples? ference (CinC), IEEE. pp. 1–4.
Journal of electrocardiology 57, S65. [35] Erdenebayar, U., Kim, Y.J., Park, J.U., Joo, E.Y., Lee, K.J., 2019.
[17] Brock, A., Donahue, J., Simonyan, K., 2018. Large scale gan Deep learning approaches for automatic detection of sleep apnea
training for high fidelity natural image synthesis. arXiv preprint events from an electrocardiogram. Computer methods and programs
arXiv:1809.11096 . in biomedicine 180, 105001.
[18] Cai, W., Chen, Y., Guo, J., Han, B., Shi, Y., Ji, L., Wang, J., Zhang, [36] Farhadi, J., Attarodi, G., Dabanloo, N.J., Mohandespoor, M., Es-
G., Luo, J., 2019. Accurate detection of atrial fibrillation from 12- lamizadeh, M., 2018. Classification of atrial fibrillation using
lead ecg using deep neural network. Computers in Biology and stacked auto encoders neural networks, in: 2018 Computing in Car-
Medicine , 103378. diology Conference (CinC), IEEE. pp. 1–3.
[19] Camm, A.J., Malik, M., Bigger, J.T., Breithardt, G., Cerutti, S., Co- [37] Faust, O., Shenfield, A., Kareem, M., San, T.R., Fujita, H., Acharya,
hen, R.J., Coumel, P., Fallen, E.L., Kennedy, H.L., Kleiger, R., et al., U.R., 2018. Automated detection of atrial fibrillation using long
1996. Heart rate variability: standards of measurement, physiologi- short-term memory network with rr interval signals. Computers in
cal interpretation and clinical use. task force of the european society biology and medicine 102, 327–335.
of cardiology and the north american society of pacing and electro- [38] Feng, N., Xu, S., Liang, Y., Liu, K., 2019. A probabilistic process
physiology . neural network and its application in ecg classification. IEEE Access
[20] Camps, J., Rodríguez, B., Mincholé, A., 2018. Deep learning based 7, 50431–50439.
qrs multilead delineator in electrocardiogram signals, in: 2018 Com- [39] Fotiadou, E., Konopczyński, T., Hesser, J., Vullings, R., 2019. Deep
puting in Cardiology Conference (CinC), IEEE. pp. 1–4. convolutional encoder-decoder framework for fetal ecg signal de-
[21] Cao, X.C., Yao, B., Chen, B.Q., 2019. Atrial fibrillation detec- noising, in: 2019 Computing in Cardiology (CinC), IEEE. pp. Page–
tion using an improved multi-scale decomposition enhanced residual 1.
convolutional neural network. IEEE Access 7, 89152–89161. [40] Fotiadou, E., Konopczyński, T., Hesser, J.W., Vullings, R., 2020.
[22] Chan, T.H., Jia, K., Gao, S., Lu, J., Zeng, Z., Ma, Y., 2015. Pcanet: End-to-end trained cnn encoder-decoder network for fetal ecg signal
A simple deep learning baseline for image classification? IEEE denoising. Physiological Measurement .
transactions on image processing 24, 5017–5032. [41] Gadaleta, M., Rossi, M., Steinhubl, S.R., Quer, G., 2018. Deep
[23] Chang, Y.C., Wu, S.H., Tseng, L.M., Chao, H.L., Ko, C.H., 2018. Af learning to detect atrial fibrillation from short noisy ecg segments
detection by exploiting the spectral and temporal characteristics of measured with wireless sensors. Circulation 138, A16177–A16177.
ecg signals with the lstm model, in: 2018 Computing in Cardiology [42] Ghassemi, M.M., Moody, B.E., Lehman, L.W.H., Song, C., Li, Q.,
Conference (CinC), IEEE. pp. 1–4. Sun, H., Mark, R.G., Westover, M.B., Clifford, G.D., 2018. You
[24] Chauhan, S., Vig, L., Ahmad, S., 2019. Ecg anomaly class identifi- snooze, you win: the physionet/computing in cardiology challenge
cation using lstm and error profile modeling. Computers in biology 2018, in: 2018 Computing in Cardiology Conference (CinC), IEEE.
and medicine 109, 14–21. pp. 1–4.
[25] Chen, M., Wang, G., Xie, P., Sang, Z., Lv, T., Zhang, P., Yang, H., [43] Ghiasi, S., Abdollahpur, M., Madani, N., Kiani, K., Ghaffari, A.,
2018. Region aggregation network: Improving convolutional neu- 2017. Atrial fibrillation detection using feature based algorithm and
ral network for ecg characteristic detection, in: 2018 40th Annual deep convolutional neural network, in: 2017 Computing in Cardiol-
International Conference of the IEEE Engineering in Medicine and ogy (CinC), IEEE. pp. 1–4.
Biology Society (EMBC), IEEE. pp. 2559–2562. [44] Gogna, A., Majumdar, A., Ward, R., 2016. Semi-supervised
[26] Chen, Y., Chen, W., 2018. Finger ecg based two-phase authentica- stacked label consistent autoencoder for reconstruction and analysis
tion using 1d convolutional neural networks, in: 2018 40th Annual of biomedical signals. IEEE Transactions on Biomedical Engineer-
International Conference of the IEEE Engineering in Medicine and ing 64, 2196–2205.
Biology Society (EMBC), IEEE. pp. 336–339. [45] Golany, T., Radinsky, K., 2019. Pgans: Personalized generative ad-
[27] Chiang, H.T., Hsieh, Y.Y., Fu, S.W., Hung, K.H., Tsao, Y., Chien, versarial networks for ecg synthesis to improve patient-specific deep
S.Y., 2019. Noise reduction in ecg signals using fully convolutional ecg classification, in: Proceedings of the AAAI Conference on Ar-
denoising autoencoders. IEEE Access 7, 60806–60813. tificial Intelligence, pp. 557–564.
[28] Choi, E., Bahadori, M.T., Sun, J., Kulas, J., Schuetz, A., Stewart, [46] Goldberger, A.L., Amaral, L.A., Glass, L., Hausdorff, J.M., Ivanov,
W.F., 2016. RETAIN: an interpretable predictive model for health- P.C., Mark, R.G., Mietus, J.E., Moody, G.B., Peng, C.K., Stanley,
care using reverse time attention mechanism, in: Advances in Neural H.E., 2000. Physiobank, physiotoolkit, and physionet: components
Information Processing Systems 29: Annual Conference on Neu- of a new research resource for complex physiologic signals. Circu-

Shenda Hong et al.: Preprint submitted to Elsevier Page 11 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

lation 101, e215–e220. Ecg biometric recognition: template-free approaches based on deep
[47] Golrizkhatami, Z., Acan, A., 2018. Ecg classification using three- learning, in: 2019 41st Annual International Conference of the IEEE
level fusion of different feature descriptors. Expert Systems with Engineering in Medicine and Biology Society (EMBC), IEEE. pp.
Applications 114, 54–64. 2633–2636.
[48] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, [66] Hong, S., Fu, Z., Zhou, R., Yu, J., Li, Y., Wang, K., Cheng, G.,
D., Ozair, S., Courville, A., Bengio, Y., 2014. Generative adversar- 2020. Cardiolearn: A cloud deep learning service for cardiac disease
ial nets, in: Advances in neural information processing systems, pp. detection from electrocardiogram, in: Companion Proceedings of
2672–2680. the Web Conference 2020, pp. 148–152.
[49] Goto, S., Kimura, M., Katsumata, Y., Goto, S., Kamatani, T., Ichi- [67] Hong, S., Wu, M., Zhou, Y., Wang, Q., Shang, J., Li, H., Xie, J.,
hara, G., Ko, S., Sasaki, J., Fukuda, K., Sano, M., 2019. Artificial in- 2017. Encase: An ensemble classifier for ecg classification using
telligence to predict needs for urgent revascularization from 12-leads expert features and deep neural networks, in: 2017 Computing in
electrocardiography in emergency patients. PloS one 14, e0210103. Cardiology (CinC), IEEE. pp. 1–4.
[50] Guglin, M.E., Thatai, D., 2006. Common errors in computer electro- [68] Hong, S., Xiao, C., Ma, T., Li, H., Sun, J., 2019b. Mina: Multi-
cardiogram interpretation. International journal of cardiology 106, level knowledge-guided attention for modeling electrocardiography
232–237. signals, in: Tweenty-Eighth International Joint Conference on Arti-
[51] Gyawali, P.K., Chen, S., Liu, H., Horacek, B.M., Sapp, J.L., Wang, ficial Intelligence (IJCAI), pp. 5888–5894.
L., 2017. Automatic coordinate prediction of the exit of ventricular [69] Hong, S., Zhou, Y., Wu, M., Shang, J., Wang, Q., Li, H., Xie, J.,
tachycardia from 12-lead electrocardiogram, in: 2017 Computing in 2019c. Combining deep neural networks and engineered features
Cardiology (CinC), IEEE. pp. 1–4. for cardiac arrhythmia detection from ecg recordings. Physiological
[52] Gyawali, P.K., Horacek, B.M., Sapp, J.L., Wang, L., 2019. Sequen- measurement 40, 054009.
tial factorized autoencoder for localizing the origin of ventricular [70] Hu, Z., Ma, X., Liu, Z., Hovy, E.H., Xing, E.P., 2016. Harnessing
activation from 12-lead electrocardiograms. IEEE Transactions on deep neural networks with logic rules, in: Proceedings of the 54th
Biomedical Engineering . Annual Meeting of the Association for Computational Linguistics,
[53] Habib, A., Karmakar, C., Yearwood, J., 2019. Impact of ecg dataset ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long
diversity on generalization of cnn model for detecting qrs complex. Papers. URL: https://www.aclweb.org/anthology/P16-1228/.
IEEE access 7, 93275–93285. [71] Huang, J., Chen, B., Yao, B., He, W., 2019. Ecg arrhythmia classi-
[54] Hammad, M., Liu, Y., Wang, K., 2018. Multimodal biometric au- fication using stft-based spectrogram and convolutional neural net-
thentication systems using convolution neural network based on dif- work. IEEE Access 7, 92871–92880.
ferent level fusion of ecg and fingerprint. IEEE Access 7, 26527– [72] Jalali, A., Lee, M., 2019. Atrial fibrillation prediction with residual
26542. network using sensitivity & orthogonality constraints. IEEE Journal
[55] Han, C., Shi, L., 2020. Ml–resnet: A novel network to detect and of Biomedical and Health Informatics .
locate myocardial infarction using 12 leads ecg. Computer methods [73] Jia, D., Zhao, W., Hu, J., Wang, H., Yan, C., Li, Z., 2019a. Detection
and programs in biomedicine 185, 105138. of first-degree atrioventricular block on variable-length electrocar-
[56] Han, S., Mao, H., Dally, W.J., 2016. Deep compression: Compress- diogram via a multimodal deep learning method, in: 2019 Comput-
ing deep neural network with pruning, trained quantization and huff- ing in Cardiology (CinC), IEEE. pp. Page–1.
man coding, in: 4th International Conference on Learning Represen- [74] Jia, D., Zhao, W., Li, Z., Hu, J., Yan, C., Wang, H., You, T., 2019b.
tations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Confer- An electrocardiogram delineator via deep segmentation network, in:
ence Track Proceedings. URL: http://arxiv.org/abs/1510.00149. 2019 41st Annual International Conference of the IEEE Engineering
[57] Hannun, A.Y., Rajpurkar, P., Haghpanahi, M., Tison, G.H., Bourn, in Medicine and Biology Society (EMBC), IEEE. pp. 1913–1916.
C., Turakhia, M.P., Ng, A.Y., 2019. Cardiologist-level arrhythmia [75] Jiang, J., Zhang, H., Pi, D., Dai, C., 2019. A novel multi-module
detection and classification in ambulatory electrocardiograms using neural network system for imbalanced heartbeats classification. Ex-
a deep neural network. Nature medicine 25, 65. pert Systems with Applications: X 1, 100003.
[58] Hao, C., Wibowo, S., Majmudar, M., Rajput, K.S., 2019. Spectro- [76] Jimenez-Perez, G., Alcaine, A., Camara, O., 2019. U-net architec-
temporal feature based multi-channel convolutional neural network ture for the automatic detection and delineation of the electrocardio-
for ecg beat classification, in: 2019 41st Annual International Con- gram, in: 2019 Computing in Cardiology (CinC), IEEE. pp. Page–1.
ference of the IEEE Engineering in Medicine and Biology Society [77] Kalyakulina, A.I., Yusipov, I.I., Moskalenko, V.A., Nikolskiy,
(EMBC), IEEE. pp. 5642–5645. A.V., Kozlov, A.A., Kosonogov, K.A., Zolotykh, N.Y., Ivanchenko,
[59] Hao, P., Gao, X., Li, Z., Zhang, J., Wu, F., Bai, C., 2020. Multi- M.V., 2018. Lu electrocardiography database: a new open-
branch fusion network for myocardial infarction screening from 12- access validation tool for delineation algorithms. arXiv preprint
lead ecg images. Computer Methods and Programs in Biomedicine arXiv:1809.03393 .
184, 105286. [78] Kamaleswaran, R., Mahajan, R., Akbilgic, O., 2018. A robust deep
[60] Harada, S., Hayashi, H., Uchida, S., 2019. Biosignal generation convolutional neural network for the classification of abnormal car-
and latent variable analysis with recurrent generative adversarial net- diac rhythm using single lead electrocardiograms of variable length.
works. IEEE Access 7, 144292–144302. Physiological measurement 39, 035006.
[61] He, J., Li, K., Liao, X., Zhang, P., Jiang, N., 2019a. Real-time detec- [79] Kimura, T., Takatsuki, S., Yamashita, H., Miyama, H., Fujisawa, T.,
tion of acute cognitive stress using a convolutional neural network Nakajima, K., Katsumata, Y., Nishiyama, T., Aizawa, Y., Fukuda,
from electrocardiographic signal. IEEE Access 7, 42710–42717. K., 2018. A 10-rr-interval-based rhythm classifier using a deep neu-
[62] He, R., Liu, Y., Wang, K., Zhao, N., Yuan, Y., Li, Q., Zhang, H., ral network. Circulation 138, A12693–A12693.
2019b. Automatic cardiac arrhythmia classification using combina- [80] Kiranyaz, S., Ince, T., Gabbouj, M., 2015a. Real-time patient-
tion of deep residual network and bidirectional lstm. IEEE Access specific ecg classification by 1-d convolutional neural networks.
7, 102119–102135. IEEE Transactions on Biomedical Engineering 63, 664–675.
[63] He, W., Wang, G., Hu, J., Li, C., Guo, B., Li, F., 2019c. Simultane- [81] Kiranyaz, S., Ince, T., Gabbouj, M., 2017. Personalized monitor-
ous human health monitoring and time-frequency sparse representa- ing and advance warning system for cardiac arrhythmias. Scientific
tion using eeg and ecg signals. IEEE Access 7, 85985–85994. reports 7, 9270.
[64] Hinton, G.E., Vinyals, O., Dean, J., 2015. Distilling the knowledge [82] Kiranyaz, S., Ince, T., Hamila, R., Gabbouj, M., 2015b. Convo-
in a neural network. CoRR abs/1503.02531. URL: http://arxiv. lutional neural networks for patient-specific ecg classification, in:
org/abs/1503.02531, arXiv:1503.02531. 2015 37th Annual International Conference of the IEEE Engineering
[65] Hong, P.L., Hsiao, J.Y., Chung, C.H., Feng, Y.M., Wu, S.C., 2019a. in Medicine and Biology Society (EMBC), IEEE. pp. 2608–2611.

Shenda Hong et al.: Preprint submitted to Elsevier Page 12 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

[83] Koh, D.W., Lee, S.G., 2019. An evaluation method of safe driving network with svm architecture. IEEE Access 7, 77849–77856.
for senior adults using ecg signals. Sensors 19, 2828. [104] Li, Z., Zhou, D., Wan, L., Li, J., Mou, W., 2020b. Heartbeat classifi-
[84] Kuznetsov, V., Moskalenko, V., Zolotykh, N.Y., 2020. Electrocar- cation using deep residual convolutional neural network from 2-lead
diogram generation and feature extraction using a variational autoen- electrocardiogram. Journal of Electrocardiology 58, 105–112.
coder. arXiv preprint arXiv:2002.00254 . [105] Limam, M., Precioso, F., 2017. Atrial fibrillation detection and ecg
[85] Kwon, J.m., Kim, K.H., Jeon, K.H., Lee, S.E., Lee, H.Y., Cho, H.J., classification based on convolutional recurrent neural network, in:
Choi, J.O., Jeon, E.S., Kim, M.S., Kim, J.J., et al., 2019. Artificial 2017 Computing in Cardiology (CinC), IEEE. pp. 1–4.
intelligence algorithm for predicting mortality of patients with acute [106] Lin, C.H., 2008. Frequency-domain features for ecg beat discrimi-
heart failure. PloS one 14. nation using grey relational analysis-based classifier. Computers &
[86] Labati, R.D., Muñoz, E., Piuri, V., Sassi, R., Scotti, F., 2019. Deep- Mathematics with Applications 55, 680–690.
ecg: convolutional neural networks for ecg biometric recognition. [107] Lin, K., Li, D., He, X., Zhang, Z., Sun, M.T., 2017a. Adversarial
Pattern Recognition Letters 126, 78–85. ranking for language generation, in: Advances in Neural Information
[87] Laguna, P., Mark, R.G., Goldberg, A., Moody, G.B., 1997. A Processing Systems, pp. 3155–3165.
database for evaluation of algorithms for measurement of qt and [108] Lin, T., Goyal, P., Girshick, R.B., He, K., Dollár, P., 2017b. Fo-
other waveform intervals in the ecg, in: Computers in cardiology cal loss for dense object detection, in: IEEE International Confer-
1997, IEEE. pp. 673–676. ence on Computer Vision, ICCV 2017, Venice, Italy, October 22-29,
[88] Lai, D., Zhang, X., Bu, Y., Su, Y., Ma, C.S., 2019. An automatic sys- 2017, pp. 2999–3007. URL: https://doi.org/10.1109/ICCV.2017.
tem for real-time identifying atrial fibrillation by using a lightweight 324, doi:10.1109/ICCV.2017.324.
convolutional neural network. IEEE access 7, 130074–130084. [109] Liu, F., Liu, C., Zhao, L., Zhang, X., Wu, X., Xu, X., Liu, Y., Ma,
[89] LeCun, Y., Bengio, Y., Hinton, G.E., 2015. Deep learning. Nature C., Wei, S., He, Z., et al., 2018. An open access database for eval-
521, 436–444. URL: https://doi.org/10.1038/nature14539, doi:10. uating the algorithms of electrocardiogram rhythm and morphology
1038/nature14539. abnormality detection. Journal of Medical Imaging and Health In-
[90] Lee, J., Oh, K., Kim, B., Yoo, S.K., 2019a. Synthesis of electro- formatics 8, 1368–1373.
cardiogram v lead signals from limb lead measurement using r peak [110] Liu, F., Zhou, X., Cao, J., Wang, Z., Wang, H., Zhang, Y., 2019a.
aligned generative adversarial network. IEEE journal of biomedical A lstm and cnn based assemble neural network framework for ar-
and health informatics . rhythmias classification, in: ICASSP 2019-2019 IEEE International
[91] Lee, J., Sun, S., Yang, S., Sohn, J., Park, J., Lee, S., Kim, H.C., Conference on Acoustics, Speech and Signal Processing (ICASSP),
2018. Bidirectional recurrent auto-encoder for photoplethysmogram IEEE. pp. 1303–1307.
denoising. IEEE journal of biomedical and health informatics . [111] Liu, M., Kim, Y., 2018. Classification of heart diseases based on
[92] Lee, J.N., Kwak, K.C., 2019. Personal identification using a robust ecg signals using long short-term memory, in: 2018 40th Annual
eigen ecg network based on time-frequency representations of ecg International Conference of the IEEE Engineering in Medicine and
signals. IEEE Access 7, 48392–48404. Biology Society (EMBC), IEEE. pp. 2707–2710.
[93] Lee, J.S., Lee, S.J., Choi, M., Seo, M., Kim, S.W., 2019b. Qrs de- [112] Liu, W., Wang, F., Huang, Q., Chang, S., Wang, H., He, J., 2019b.
tection method based on fully convolutional networks for capacitive Mfb-cbrnn: a hybrid network for mi detection using 12-lead ecgs.
electrocardiogram. Expert Systems with Applications 134, 66–78. IEEE journal of biomedical and health informatics .
[94] Li, F., Wu, J., Jia, M., Chen, Z., Pu, Y., 2019a. Automated heartbeat [113] Liu, W., Zhang, M., Zhang, Y., Liao, Y., Huang, Q., Chang, S.,
classification exploiting convolutional neural network with channel- Wang, H., He, J., 2017. Real-time multilead convolutional neu-
wise attention. IEEE Access . ral network for myocardial infarction detection. IEEE journal of
[95] Li, F., Xu, Y., Chen, Z., Liu, Z., 2019b. Automated heartbeat classi- biomedical and health informatics 22, 1434–1444.
fication using 3-d inputs based on convolutional neural network with [114] Lynn, H.M., Pan, S.B., Kim, P., 2019. A deep bidirectional gru
multi-fields of view. IEEE Access 7, 76295–76304. network model for biometric electrocardiogram classification based
[96] Li, H., Wang, X., Liu, C., Wang, Y., Li, P., Tang, H., Yao, L., Zhang, on recurrent neural networks. IEEE Access 7, 145395–145405.
H., 2019c. Dual-input neural network integrating feature extraction [115] Maaten, L.v.d., Hinton, G., 2008. Visualizing data using t-sne. Jour-
and deep learning for coronary artery disease detection using electro- nal of machine learning research 9, 2579–2605.
cardiogram and phonocardiogram. IEEE Access 7, 146457–146469. [116] Maidens, J., Slamon, N.B., 2018. Artificial intelligence detects pe-
[97] Li, K., Pan, W., Li, Y., Jiang, Q., Liu, G., 2018a. A method to detect diatric heart murmurs with cardiologist-level accuracy. Circulation
sleep apnea based on deep neural network and hidden markov model 138, A12591–A12591.
using single-lead ecg signal. Neurocomputing 294, 94–101. [117] Maknickas, V., Maknickas, A., 2017. Atrial fibrillation classifica-
[98] Li, Q., Liu, C., Li, Q., Shashikumar, S.P., Nemati, S., Shen, Z., Clif- tion using qrs complex features and lstm, in: 2017 Computing in
ford, G.D., 2019d. Ventricular ectopic beat detection using a wavelet Cardiology (CinC), IEEE. pp. 1–4.
transform and a convolutional neural network. Physiological mea- [118] Malik, J., Lo, Y.L., Wu, H.t., 2018. Sleep-wake classification via
surement 40, 055002. quantifying heart rate variability by convolutional neural network.
[99] Li, R., Zhang, X., Dai, H., Zhou, B., Wang, Z., 2019e. Interpretabil- Physiological measurement 39, 085004.
ity analysis of heartbeat classification based on heartbeat activity’s [119] Mathews, S.M., Kambhamettu, C., Barner, K.E., 2018. A novel ap-
global sequence features and bilstm-attention neural network. IEEE plication of deep learning for single-lead ecg classification. Com-
Access 7, 109870–109883. puters in biology and medicine 99, 53–62.
[100] Li, Y., Pang, Y., Wang, J., Li, X., 2018b. Patient-specific ecg classi- [120] Miller, A.C., Obermeyer, Z., Mullainathan, S., 2019. A comparison
fication by deeper cnn from generic to dedicated. Neurocomputing of patient history-and ekg-based cardiac risk scores. AMIA Summits
314, 336–346. on Translational Science Proceedings 2019, 82.
[101] Li, Y., Pang, Y., Wang, K., Li, X., 2020a. Toward improving ecg bio- [121] Mincholé, A., Camps, J., Lyon, A., Rodríguez, B., 2019. Machine
metric identification using cascaded convolutional neural networks. learning in the electrocardiogram. Journal of electrocardiology .
Neurocomputing . [122] Moody, G., 1983. A new method for detecting atrial fibrillation using
[102] Li, Y., Zhang, Y., Zhao, L., Zhang, Y., Liu, C., Zhang, L., Zhang, rr intervals. Computers in Cardiology , 227–230.
L., Li, Z., Wang, B., Ng, E., et al., 2018c. Combining convolutional [123] Moody, G.B., Mark, R.G., 2001. The impact of the mit-bih arrhyth-
neural network and distance distribution matrix for identification of mia database. IEEE Engineering in Medicine and Biology Magazine
congestive heart failure. IEEE Access 6, 39734–39744. 20, 45–50.
[103] Li, Z., Feng, X., Wu, Z., Yang, C., Bai, B., Yang, Q., 2019f. Classifi- [124] Moskalenko, V., Zolotykh, N., Osipov, G., 2020. Deep learning
cation of atrial fibrillation recurrence based on a convolution neural for ecg segmentation, in: Kryzhanovsky, B., Dunin-Barkowski, W.,

Shenda Hong et al.: Preprint submitted to Elsevier Page 13 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

Redko, V., Tiumentsev, Y. (Eds.), Advances in Neural Computation, for the classification of holter ecg. Physiological measurement 39,
Machine Learning, and Cognitive Research III, Springer Interna- 094002.
tional Publishing, Cham. pp. 246–254. [142] Porumb, M., Stranges, S., Pescapè, A., Pecchia, L., 2020. Precision
[125] Mousavi, S., Afghah, F., 2019. Inter-and intra-patient ecg heart- medicine and artificial intelligence: A pilot study on deep learning
beat classification for arrhythmia detection: a sequence to sequence for hypoglycemic events detection based on ecg. Scientific Reports
deep learning approach, in: ICASSP 2019-2019 IEEE International 10, 1–16.
Conference on Acoustics, Speech and Signal Processing (ICASSP), [143] Pourbabaee, B., Roshtkhari, M.J., Khorasani, K., 2017. Deep con-
IEEE. pp. 1308–1312. volutional neural networks and learning ecg features for screening
[126] Mousavi, S., Fotoohinasab, A., Afghah, F., 2020. Single-modal paroxysmal atrial fibrillation patients. IEEE Transactions on Sys-
and multi-modal false arrhythmia alarm reduction using attention- tems, Man, and Cybernetics: Systems 48, 2095–2104.
based convolutional and recurrent neural networks. PloS one 15, [144] Qiu, Y., Xiao, F., Shen, H., 2017. Elimination of power line inter-
e0226990. ference from ecg signals using recurrent neural networks, in: 2017
[127] Mukherjee, A., Choudhury, A.D., Datta, S., Puri, C., Banerjee, R., 39th Annual International Conference of the IEEE Engineering in
Singh, R., Ukil, A., Bandyopadhyay, S., Pal, A., Khandelwal, S., Medicine and Biology Society (EMBC), IEEE. pp. 2296–2299.
2019. Detection of atrial fibrillation and other abnormal rhythms [145] Raghunath, S.M., Ulloa Cerna, A., Jing, L., vanMaanen, D., Stough,
from ecg using a multi-layer classifier architecture. Physiological J.V., Hartzel, D., Leader, J., Kirchner, H.L., Good, C., Patel, A.,
measurement 40, 054006. et al., 2019. Deep neural networks can predict 1-year mortality di-
[128] Nguyen, M.T., Van Nguyen, B., Kim, K., 2018. Deep feature learn- rectly from ecg signal, even when clinically interpreted as normal.
ing for sudden cardiac arrest detection in automated external defib- Circulation 140, A14425–A14425.
rillators. Scientific reports 8, 17196. [146] Rajan, D., Beymer, D., Narayan, G., 2018. Generalization studies
[129] Nikolic, G., Bishop, R., Singh, J., 1982. Sudden death recorded of neural network models for cardiac disease detection using limited
during holter monitoring. Circulation 66, 218–225. channel ecg, in: 2018 Computing in Cardiology Conference (CinC),
[130] Niu, J., Tang, Y., Sun, Z., Zhang, W., 2019. Inter-patient ecg clas- IEEE. pp. 1–4.
sification with symbolic representations and multi-perspective con- [147] Rajan, D., Thiagarajan, J.J., 2018. A generative modeling approach
volutional neural networks. IEEE journal of biomedical and health to limited channel ecg classification, in: 2018 40th Annual Interna-
informatics . tional Conference of the IEEE Engineering in Medicine and Biology
[131] Noseworthy, P.A., Attia, Z.I., Brewer, L.C., Hayes, S.N., Yao, X., Society (EMBC), IEEE. pp. 2571–2574.
Kapa, S., Friedman, P.A., Lopez-Jimenez, F., 2020. Assessing and [148] Rastgoo, M.N., Nakisa, B., Maire, F., Rakotonirainy, A., Chan-
mitigating bias in medical artificial intelligence: The effects of race dran, V., 2019. Automatic driver stress level classification using
and ethnicity on a deep learning model for ecg analysis. Circulation: multimodal deep learning. Expert Systems with Applications 138,
Arrhythmia and Electrophysiology . 112793.
[132] Oh, S.L., Ng, E.Y., San Tan, R., Acharya, U.R., 2018. Automated di- [149] Ribeiro, M.T., Singh, S., Guestrin, C., 2016. "why should I trust
agnosis of arrhythmia using combination of cnn and lstm techniques you?": Explaining the predictions of any classifier, in: Proceedings
with variable length heart beats. Computers in biology and medicine of the 22nd ACM SIGKDD International Conference on Knowledge
102, 278–287. Discovery and Data Mining, San Francisco, CA, USA, August 13-
[133] Oh, S.L., Ng, E.Y., San Tan, R., Acharya, U.R., 2019. Auto- 17, 2016, pp. 1135–1144. URL: https://doi.org/10.1145/2939672.
mated beat-wise arrhythmia diagnosis using modified u-net on ex- 2939778, doi:10.1145/2939672.2939778.
tended electrocardiographic recordings with heterogeneous arrhyth- [150] Ribeiro, M.T., Singh, S., Guestrin, C., 2018. Anchors: High-
mia types. Computers in biology and medicine 105, 92–101. precision model-agnostic explanations, in: Proceedings of the
[134] Park, Y., Yun, I.D., Kang, S.H., 2019. Preprocessing method for per- Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-
formance enhancement in cnn-based stemi detection from 12-lead 18), the 30th innovative Applications of Artificial Intelligence
ecg. IEEE Access 7, 99964–99977. (IAAI-18), and the 8th AAAI Symposium on Educational Advances
[135] Parvaneh, S., Rubin, J., 2018. Electrocardiogram monitoring and in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA,
interpretation: from traditional machine learning to deep learning, February 2-7, 2018, pp. 1527–1535. URL: https://www.aaai.org/
and their combination, in: 2018 Computing in Cardiology Confer- ocs/index.php/AAAI/AAAI18/paper/view/16982.
ence (CinC), IEEE. pp. 1–4. [151] Romero, I., Serrano, L., 2001. Ecg frequency domain features
[136] Parvaneh, S., Rubin, J., Babaeizadeh, S., Xu-Wilson, M., 2019. Car- extraction: A new characteristic for arrhythmias classification, in:
diac arrhythmia detection using deep learning: A review. Journal of 2001 Conference Proceedings of the 23rd Annual International Con-
electrocardiology . ference of the IEEE Engineering in Medicine and Biology Society,
[137] Parvaneh, S., Rubin, J., Rahman, A., Conroy, B., Babaeizadeh, S., IEEE. pp. 2006–2008.
2018. Analyzing single-lead short ecg recordings using dense convo- [152] Ronneberger, O., Fischer, P., Brox, T., 2015. U-net: Convolutional
lutional neural networks and feature-based post-processing to detect networks for biomedical image segmentation, in: International Con-
atrial fibrillation. Physiological measurement 39, 084003. ference on Medical image computing and computer-assisted inter-
[138] Peimankar, A., Puthusserypady, S., 2019. An ensemble of deep re- vention, Springer. pp. 234–241.
current neural networks for p-wave detection in electrocardiogram, [153] Rubin, J., Parvaneh, S., Rahman, A., Conroy, B., Babaeizadeh, S.,
in: ICASSP 2019-2019 IEEE International Conference on Acous- 2017. Densely connected convolutional networks and signal qual-
tics, Speech and Signal Processing (ICASSP), IEEE. pp. 1284–1288. ity analysis to detect atrial fibrillation using short single-lead ecg
[139] Philips, 2003. The Philips 12-Lead Algorithm Physician’s Guide. recordings, in: 2017 Computing in Cardiology (CinC), IEEE. pp.
URL: http://incenter.medical.philips.com/doclib/enc/fetch/ 1–4.
2000/4504/577242/577243/577245/577817/577818/12-Lead_Algorithm_ [154] Rubin, J., Parvaneh, S., Rahman, A., Conroy, B., Babaeizadeh, S.,
Physician_s_Guide_for_Algorithm_Version_PH080A 2018. Densely connected convolutional networks for detection of
[140] Picon, A., Irusta, U., Álvarez-Gila, A., Aramendi, E., Alonso- atrial fibrillation from short single-lead ecg recordings. Journal of
Atienza, F., Figuera, C., Ayala, U., Garrote, E., Wik, L., Kramer- electrocardiology 51, S18–S21.
Johansen, J., et al., 2019. Mixed convolutional and long short-term [155] Saadatnejad, S., Oveisi, M., Hashemi, M., 2019. Lstm-based ecg
memory network for the detection of lethal ventricular arrhythmia. classification for continuous monitoring on personal wearable de-
PloS one 14, e0216756. vices. IEEE journal of biomedical and health informatics .
[141] Plesinger, F., Nejedly, P., Viscor, I., Halamek, J., Jurak, P., 2018. Par- [156] Santamaria-Granados, L., Munoz-Organero, M., Ramirez-
allel use of a convolutional neural network and bagged tree ensemble Gonzalez, G., Abdulhay, E., Arunkumar, N., 2018. Using

Shenda Hong et al.: Preprint submitted to Elsevier Page 14 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

deep convolutional neural network for emotion detection on a risk stratification using off-the-shelf wearables and a multi-task deep
physiological signals dataset (amigos). IEEE Access 7, 57–67. learning algorithm. Circulation 136, A21042–A21042.
[157] Schläpfer, J., Wellens, H.J., 2017. Computer-interpreted electrocar- [175] Tison, G.H., Zhang, J., Delling, F.N., Deo, R.C., 2019. Automated
diograms: benefits and limitations. Journal of the American College and interpretable patient ecg profiles for disease detection, tracking,
of Cardiology 70, 1183–1192. and discovery. Circulation: Cardiovascular Quality and Outcomes
[158] Schwab, P., Scebba, G.C., Zhang, J., Delai, M., Karlen, W., 2017. 12, e005289.
Beat by beat: Classifying cardiac arrhythmias with recurrent neural [176] Urtnasan, E., Park, J.U., Lee, K.J., 2018. Multiclass classification of
networks, in: 2017 Computing in Cardiology (CinC), IEEE. pp. 1–4. obstructive sleep apnea/hypopnea based on a convolutional neural
[159] Sellami, A., Hwang, H., 2019. A robust deep convolutional neural network from a single-lead electrocardiogram. Physiological mea-
network with batch-weighted loss for heartbeat classification. Expert surement 39, 065003.
Systems with Applications 122, 75–84. [177] Van Steenkiste, G., van Loon, G., Crevecoeur, G., 2020. Transfer
[160] Selvalingam, A., Alhusseini, M., Rogers, A.J., Krummen, D.E., learning in ecg classification from human to horse using a novel par-
Abuzaid, F.M., Zaman, J.A., Baykaner, T., Clopton, P.L., Bailis, P., allel neural network architecture. Scientific Reports 10, 1–12.
Zaharia, M., et al., 2019. Developing convolutional neural networks [178] Vullings, R., 2019. Fetal electrocardiography and deep learning for
for deep learning of ventricular action potentials to predict risk for prenatal detection of congenital heart disease, in: 2019 Computing
ventricular arrhythmias. Circulation 140, A14675–A14675. in Cardiology (CinC), IEEE. pp. Page–1.
[161] Shah, A.P., Rubin, S.A., 2007. Errors in the computerized electro- [179] Wang, E.K., Zhang, X., Pan, L., 2019a. Automatic classification
cardiogram interpretation of cardiac rhythm. Journal of electrocar- of cad ecg signals with sdae and bidirectional long short-term term
diology 40, 385–390. network. IEEE Access .
[162] Shaker, A.M., Tantawi, M., Shedeed, H.A., Tolba, M.F., 2020. Gen- [180] Wang, J., Li, R., Li, R., Li, K., Zeng, H., Xie, G., Liu, L., 2019b.
eralization of convolutional neural networks for ecg classification us- Adversarial de-noising of electrocardiogram. Neurocomputing 349,
ing generative adversarial networks. IEEE Access 8, 35592–35605. 212–224.
[163] Shashikumar, S.P., Shah, A.J., Clifford, G.D., Nemati, S., 2018. De- [181] Wang, L., Zhou, W., Chang, Q., Chen, J., Zhou, X., 2019c. Deep
tection of paroxysmal atrial fibrillation using attention-based bidi- ensemble detection of congestive heart failure using short-term rr
rectional recurrent neural networks, in: Proceedings of the 24th intervals. IEEE Access .
ACM SIGKDD International Conference on Knowledge Discovery [182] Wang, P., Hou, B., Shao, S., Yan, R., 2019d. Ecg arrhythmias de-
& Data Mining, ACM. pp. 715–723. tection using auxiliary classifier generative adversarial network and
[164] Shen, Y., Voisin, M., Aliamiri, A., Avati, A., Hannun, A., Ng, A., residual network. IEEE Access 7, 100910–100922.
2019. Ambulatory atrial fibrillation monitoring using wearable pho- [183] Wang, S., Aggarwal, C.C., Liu, H., 2017. Using a random forest
toplethysmography with deep learning, in: Proceedings of the 25th to inspire a neural network and improving on it, in: Proceedings of
ACM SIGKDD International Conference on Knowledge Discovery the 2017 SIAM International Conference on Data Mining, Houston,
& Data Mining, ACM. pp. 1909–1916. Texas, USA, April 27-29, 2017., pp. 1–9. URL: https://doi.org/
[165] Shu, L., Xie, J., Yang, M., Li, Z., Li, Z., Liao, D., Xu, X., Yang, X., 10.1137/1.9781611974973.1, doi:10.1137/1.9781611974973.1.
2018. A review of emotion recognition using physiological signals. [184] Wang, Y., Xiao, B., Bi, X., Li, W., Zhang, J., Ma, X., 2019e. Pay
Sensors 18, 2074. attention and watch temporal correlation: A novel 1-d convolutional
[166] Smith, S.W., Walsh, B., Grauer, K., Wang, K., Rapin, J., Li, J., Fen- neural network for ecg record classification, in: 2019 Computing in
nell, W., Taboulet, P., 2019. A deep neural network learning algo- Cardiology (CinC), IEEE. pp. Page–1.
rithm outperforms a conventional algorithm for emergency depart- [185] Wang, Y., Yao, Q., 2019. Few-shot learning: A survey.
ment electrocardiogram interpretation. Journal of electrocardiology CoRR abs/1904.05046. URL: http://arxiv.org/abs/1904.05046,
52, 88–95. arXiv:1904.05046.
[167] Sodmann, P., Vollmer, M., Nath, N., Kaderali, L., 2018. A convolu- [186] Warrick, P., Homsi, M.N., 2017. Cardiac arrhythmia detection from
tional neural network for ecg annotation as the basis for classification ecg combining convolutional and long short-term memory networks,
of cardiac rhythms. Physiological measurement 39, 104005. in: 2017 Computing in Cardiology (CinC), IEEE. pp. 1–4.
[168] Strodthoff, N., Strodthoff, C., 2018. Detecting and interpreting my- [187] Warrick, P.A., Homsi, M.N., 2018. Ensembling convolutional and
ocardial infarction using fully convolutional neural networks. Phys- long short-term memory networks for electrocardiogram arrhythmia
iological measurement . detection. Physiological measurement 39, 114002.
[169] Sun, W., Zeng, N., He, Y., 2019. Morphological arrhythmia auto- [188] Wei, Y., Qi, X., Wang, H., Liu, Z., Wang, G., Yan, X., 2019. A
mated diagnosis method using gray-level co-occurrence matrix en- multi-class automatic sleep staging method based on long short-term
hanced convolutional neural network. IEEE Access . memory network using single-lead electrocardiogram signals. IEEE
[170] Tadesse, G.A., Zhu, T., Liu, Y., Zhou, Y., Chen, J., Tian, M., Clifton, Access 7, 85959–85970.
D., 2019. Cardiovascular disease diagnosis using cross-domain [189] Wołk, A., et al., 2019. Early and remote detection of possible heart-
transfer learning, in: 2019 41st Annual International Conference of beat problems with convolutional neural networks and multipart in-
the IEEE Engineering in Medicine and Biology Society (EMBC), teractive training. IEEE Access .
IEEE. pp. 4262–4265. [190] Wu, Z., Feng, X., Yang, C., 2019. A deep learning method to detect
[171] Tan, J.H., Hagiwara, Y., Pang, W., Lim, I., Oh, S.L., Adam, M., atrial fibrillation based on continuous wavelet transform, in: 2019
San Tan, R., Chen, M., Acharya, U.R., 2018. Application of stacked 41st Annual International Conference of the IEEE Engineering in
convolutional and long short-term memory network for accurate Medicine and Biology Society (EMBC), IEEE. pp. 1908–1912.
identification of cad ecg signals. Computers in biology and medicine [191] Xia, Y., Wulan, N., Wang, K., Zhang, H., 2017. Atrial fibrillation
94, 19–26. detection using stationary wavelet transform and deep learning, in:
[172] Tateno, K., Glass, L., 2001. Automatic detection of atrial fibrillation 2017 Computing in Cardiology (CinC), IEEE. pp. 1–4.
using the coefficient of variation and density histograms of rr and 𝛿rr [192] Xia, Y., Wulan, N., Wang, K., Zhang, H., 2018a. Detecting atrial
intervals. Medical and Biological Engineering and Computing 39, fibrillation by deep convolutional neural networks. Computers in
664–671. biology and medicine 93, 84–92.
[173] Teijeiro, T., García, C.A., Castro, D., Félix, P., 2018. Abductive rea- [193] Xia, Y., Xie, Y., 2019. A novel wearable electrocardiogram classifi-
soning as a basis to reproduce expert criteria in ecg atrial fibrillation cation system using convolutional neural networks and active learn-
identification. Physiological measurement 39, 084006. ing. IEEE Access 7, 7989–8001.
[174] Tison, G.H., Singh, A.C., Ohashi, D.A., Hsieh, J.T., Ballinger, B.M., [194] Xia, Y., Zhang, H., Xu, L., Gao, Z., Zhang, H., Liu, H., Li, S., 2018b.
Olgin, J.E., Marcus, G.M., Pletcher, M.J., 2017. Cardiovascular An automatic cardiac arrhythmia classification system with wearable

Shenda Hong et al.: Preprint submitted to Elsevier Page 15 of 17

 Opportunities and Challenges of Deep Learning for ECG Data

electrocardiogram. IEEE Access 6, 16529–16538. [214] Yu, R., Gao, Y., Duan, X., Zhu, T., Wang, Z., Jiao, B., 2018. Qrs
[195] Xiao, R., Xu, Y., Pelter, M.M., Fidler, R., Badilini, F., Mortara, detection and measurement method of ecg paper based on convo-
D.W., Hu, X., 2018a. Monitoring significant st changes through deep lutional neural networks, in: 2018 40th Annual International Con-
learning. Journal of electrocardiology 51, S78. ference of the IEEE Engineering in Medicine and Biology Society
[196] Xiao, R., Xu, Y., Pelter, M.M., Mortara, D.W., Hu, X., 2018b. A (EMBC), IEEE. pp. 4636–4639.
deep learning approach to examine ischemic st changes in ambula- [215] Yuen, B., Dong, X., Lu, T., 2019. Inter-patient cnn-lstm for qrs
tory ecg recordings. AMIA Summits on Translational Science Pro- complex detection in noisy ecg signals. IEEE Access 7, 169359–
ceedings 2018, 256. 169370.
[197] Xie, P., Wang, G., Zhang, C., Chen, M., Yang, H., Lv, T., Sang, Z., [216] Zhai, X., Tin, C., 2018. Automated ecg classification using dual
Zhang, P., 2018. Bidirectional recurrent neural network and convo- heartbeat coupling based on convolutional neural network. IEEE
lutional neural network (bircnn) for ecg beat classification, in: 2018 Access 6, 27465–27472.
40th Annual International Conference of the IEEE Engineering in [217] Zhang, B., Zhao, J., Chen, X., Wu, J., 2017a. Ecg data compression
Medicine and Biology Society (EMBC), IEEE. pp. 2555–2558. using a neural network model based on multi-objective optimization.
[198] Xie, Q., Tu, S., Wang, G., Lian, Y., Xu, L., 2019. Feature enrichment PloS one 12, e0182500.
based convolutional neural network for heartbeat classification from [218] Zhang, J., Lin, F., Xiong, P., Du, H., Zhang, H., Liu, M., Hou, Z.,
electrocardiogram. IEEE Access 7, 153751–153760. Liub, X., 2019a. Automated detection and localization of myocar-
[199] Xiong, P., Wang, H., Liu, M., Lin, F., Hou, Z., Liu, X., 2016. A dial infarction with staked sparse autoencoder and treebagger. IEEE
stacked contractive denoising auto-encoder for ecg signal denoising. Access .
Physiological measurement 37, 2214. [219] Zhang, Q., Zhou, D., Zeng, X., 2017b. Heartid: A multiresolution
[200] Xiong, Z., Nash, M.P., Cheng, E., Fedorov, V.V., Stiles, M.K., Zhao, convolutional neural network for ecg-based biometric human identi-
J., 2018. Ecg signal classification for the detection of cardiac arrhyth- fication in smart health applications. Ieee Access 5, 11805–11816.
mias using a convolutional recurrent neural network. Physiological [220] Zhang, X., Li, R., Dai, H., Liu, Y., Zhou, B., Wang, Z., 2019b.
measurement 39, 094006. Localization of myocardial infarction with multi-lead bidirectional
[201] Xiong, Z., Stiles, M.K., Zhao, J., 2017. Robust ecg signal classifica- gated recurrent unit neural network. IEEE Access 7, 161152–
tion for detection of atrial fibrillation using a novel neural network, 161166.
in: 2017 Computing in Cardiology (CinC), IEEE. pp. 1–4. [221] Zhang, Y., Xiao, Z., Guo, Z., Wang, Z., 2019c. Ecg-based personal
[202] Xu, S., Li, J., Liu, K., Wu, L., 2019a. A parallel gru recurrent net- recognition using a convolutional neural network. Pattern Recogni-
work model and its application to multi-channel time-varying signal tion Letters 125, 668–676.
classification. IEEE Access 7, 118739–118748. [222] Zhao, W., Hu, J., Jia, D., Wang, H., Li, Z., Yan, C., You, T., 2019a.
[203] Xu, S.S., Mak, M.W., Cheung, C.C., 2018a. Towards end-to-end ecg Deep learning based patient-specific classification of arrhythmia on
classification with raw signal extraction and deep neural networks. ecg signal, in: 2019 41st Annual International Conference of the
IEEE journal of biomedical and health informatics . IEEE Engineering in Medicine and Biology Society (EMBC), IEEE.
[204] Xu, S.S., Mak, M.W., Cheung, C.C., 2019b. I-vector based patient pp. 1500–1503.
adaptation of deep neural networks for automatic heartbeat classifi- [223] Zhao, Z., Liu, C., Li, Y., Li, Y., Wang, J., Lin, B.S., Li, J., 2019b.
cation. IEEE journal of biomedical and health informatics . Noise rejection for wearable ecgs using modified frequency slice
[205] Xu, Y., Biswal, S., Deshpande, S.R., Maher, K.O., Sun, J., 2018b. wavelet transform and convolutional neural networks. IEEE Access
Raim: Recurrent attentive and intensive model of multimodal pa- 7, 34060–34067.
tient monitoring data, in: Proceedings of the 24th ACM SIGKDD [224] Zhao, Z., Wang, X., Cai, Z., Li, J., Liu, C., 2019c. Pvc recognition
International Conference on Knowledge Discovery & Data Mining, for wearable ecgs using modified frequency slice wavelet transform
ACM. pp. 2565–2573. and convolutional neural network, in: 2019 Computing in Cardiol-
[206] Yang, T., Gregg, R.E., Babaeizadeh, S., 2020. Detection of strict left ogy (CinC), IEEE. pp. 1–4.
bundle branch block by neural network and a method to test detection [225] Zhao, Z., Zhang, Y., Deng, Y., Zhang, X., 2018. Ecg authentication
consistency. Physiological Measurement . system design incorporating a convolutional neural network and gen-
[207] Yang, T., Yu, L., Jin, Q., Wu, L., He, B., 2017. Localization of ori- eralized s-transformation. Computers in biology and medicine 102,
gins of premature ventricular contraction by means of convolutional 168–179.
neural network from 12-lead ecg. IEEE Transactions on Biomedical [226] Zhong, W., Liao, L., Guo, X., Wang, G., 2018. A deep learning ap-
Engineering 65, 1662–1671. proach for fetal qrs complex detection. Physiological measurement
[208] Ye, F., Zhu, F., Fu, Y., Shen, B., 2019. Ecg generation with sequence 39, 045004.
generative adversarial nets optimized by policy gradient. IEEE Ac- [227] Zhou, B., Liu, S., Hooi, B., Cheng, X., Ye, J., 2019a. Beatgan:
cess 7, 159369–159378. anomalous rhythm detection using adversarially generated time se-
[209] Yildirim, Ö., 2018. A novel wavelet sequence based on deep bidirec- ries, in: Proceedings of the 28th International Joint Conference on
tional lstm network model for ecg signal classification. Computers Artificial Intelligence, AAAI Press. pp. 4433–4439.
in biology and medicine 96, 189–202. [228] Zhou, X., Zhu, X., Nakamura, K., Mahito, N., 2018. Premature ven-
[210] Yildirim, O., Baloglu, U.B., Tan, R.S., Ciaccio, E.J., Acharya, U.R., tricular contraction detection from ambulatory ecg using recurrent
2019. A new approach for arrhythmia classification using deep neural networks, in: 2018 40th Annual International Conference of
coded features and lstm networks. Computer methods and programs the IEEE Engineering in Medicine and Biology Society (EMBC),
in biomedicine 176, 121–133. IEEE. pp. 2551–2554.
[211] Yıldırım, Ö., Pławiak, P., Tan, R.S., Acharya, U.R., 2018. Arrhyth- [229] Zhou, Y., Hong, S., Shang, J., Wu, M., Wang, Q., Li, H., Xie, J.,
mia detection using deep convolutional neural network with long 2019b. K-margin-based residual-convolution-recurrent neural net-
duration ecg signals. Computers in biology and medicine 102, 411– work for atrial fibrillation detection. IJCAI , 6057–6063.
420. [230] Zhu, F., Ye, F., Fu, Y., Liu, Q., Shen, B., 2019. Electrocardiogram
[212] Yin, W., Yang, X., Zhang, L., Oki, E., 2016. Ecg monitoring system generation with a bidirectional lstm-cnn generative adversarial net-
integrated with ir-uwb radar based on cnn. IEEE Access 4, 6344– work. Scientific reports 9, 6734.
6351. [231] Zihlmann, M., Perekrestenko, D., Tschannen, M., 2017. Convolu-
[213] Yin, Z., Zhao, M., Wang, Y., Yang, J., Zhang, J., 2017. Recog- tional recurrent neural networks for electrocardiogram classification,
nition of emotions using multimodal physiological signals and an in: 2017 Computing in Cardiology (CinC), IEEE. pp. 1–4.
ensemble deep learning model. Computer methods and programs in
biomedicine 140, 93–110.

Shenda Hong et al.: Preprint submitted to Elsevier Page 16 of 17

Background: The electrocardiogram (ECG) is one of the most commonly used diagnostic tools in
medicine and healthcare. Deep learning methods have achieved promising results on predictive
healthcare tasks using ECG signals.

Objective: This paper presents a systematic review of deep learning methods for ECG data from both
modeling and application perspectives.

Methods: We extracted papers that applied deep learning (deep neural network) models to ECG
data that were published between January 1st of 2010 and February 29th of 2020 from Google
Scholar, PubMed, and the Digital Bibliography & Library Project. We then analyzed each article
according to three factors: tasks, models, and data. Finally, we discuss open challenges and unsolved
problems in this area.

Results: The total number of papers extracted was 191. Among these papers, 108 were published
after 2019. Different deep learning architectures have been used in various ECG analytics tasks, such
as disease detection/classification, annotation/localization, sleep staging, biometric human
identification, and denoising.

Conclusion: The number of works on deep learning for ECG data has grown explosively in recent
years. Such works have achieved accuracy comparable to that of traditional feature-based
approaches and ensembles of multiple approaches can achieve even better results. Specifically, we
found that a hybrid architecture of a convolutional neural network and recurrent neural network
ensemble using expert features yields the best results. However, there are some new challenges and
problems related to interpretability, scalability, and efficiency that must be addressed. Furthermore,
it is also worth investigating new applications from the perspectives of datasets and methods.

Significance: This paper summarizes existing deep learning research using ECG data from multiple
perspectives and highlights existing challenges and problems to identify potential future research
directions.
Highlights:

- A systematic review of deep learning methods on Electrocardiogram data

- Including 191 papers from multiple research fields from 2010 to 2020

- Analyzing papers from perspectives of task, model and data

- Discussing 7 aspects of challenges and potential opportunities for future works

None Declared