This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.3008223, IEEE
Transactions on Industrial Informatics

A Data-driven Auto-CNN-LSTM Prediction Model

for Lithium-ion Battery Remaining Useful Life
Lei Ren, Jiabao Dong, Xiaokang Wang, Zihao Meng, Li Zhao, and M. Jamal Deen

Abstract—Integration of each aspect of the manufacturing At the same time, lithium-ion battery (LIB), as a green
process with the new generation of information technology such high-energy rechargeable electrical power resource of intel-
as the Internet of Things, big data and cloud computing, makes ligent manufacturing with prominent characteristics such as
industrial manufacturing systems more flexible and intelligent.
Industrial big data (IBD), recording all aspects of the industrial high capacity, reliability and safety, is widely used in in-
production process, contain the key value for industrial intel- telligent manufacturing involving computational engineering,
ligence. For industrial manufacturing, an essential and widely logistics and aerospace [6]. However, the degradation of LIBs
used electronic device is the lithium-ion battery (LIB). However, may significantly lead to a decline in the performance of
accurately predicting the remaining useful life (RUL) of LIB is electrical equipment, which increases the cost of unexpected
urgently needed to reduce unexpected maintenance and avoid
accidents. Due to insufficient amount of degradation data, the maintenance. As the discharging time increases, battery failure
prediction accuracy of data-driven methods is greatly limited. induced by degradation can shorten the LIB service life and
Besides, mathematical models established by model-driven meth- even cause serious accidents [7]. Therefore, the remaining
ods to represent degradation process are unstable because of useful life (RUL) of the LIB, referred as the length of time to
external factors like temperature. To solve this problem, a new the end of useful life [8], should be predicted accurately. This
Lithium-ion battery RUL prediction method, namely Auto-CNN-
LSTM, is proposed in this work. This method is developed based is also of great significance to the safety, stability and cost of
on deep convolution neural network (CNN) and long short-term the intelligent manufacturing.
memory (LSTM) to mine deeper information in finite data. In Therefore, we should analyze the operation data of LIB to
this method, a autoencoder is utilized to augment the dimensions predicate its remaining useful life. At present, the current pre-
of data for more effective training of CNN and LSTM. In order diction method of LIB mainly includes model-driven methods
to obtain continuous and stable output, a filter to smooth the
predicted value is used. Comparing with other commonly-used and data-driven methods. Model-based methods aim to estab-
methods, experiments on a real-world dataset demonstrate the lish mathematical models to represent the performance during
effectiveness of the proposed method. degradation process of LIB, such as physics of failure (PoF)
Index Terms—Autoencoder, CNN, Lithium-ion Battery, LSTM, modeling method [9], Kalman filtering (KF) method [10],
Remaining useful life prediction particle filtering (particle filter, PF) method [11]. Existing
I. I NTRODUCTION model-based methods mainly seek a mathematical model to
fit the degraded trajectory of the LIB, and then exploits
Intelligent manufacturing combines the new generation of
the particle filtering method for prediction. For example, a
information, computational and communication technologies
coupled numerical method was proposed in [12] for the life
such as the big data, cloud computing, edge computing,
prediction of LIB for electric vehicles. A double exponential
artificial intelligence and Internet-of-Things (IoT) into the
model was proposed to approximate the degraded trajectory
every aspects of manufacturing to achieve more intelligent and
of LIB capacity [13], and initialized model parameters with
automated manufacturing processes [1]–[3]. For this purpose,
Dempster-Shafer (DS) evidence theory, combining with parti-
intelligent manufacturing big data was selected as the research
cle filtering. An improved method based on interaction multi-
target [4], [5]. This is because intelligent manufacturing big
model particle filter (IMMPF) was developed [14], combining
data records all aspects of the production process including
various capacity models of LIBs. However, there are still
product parameters, production environment, logistics and
two significant shortcomings in existing models: (1) Due to
transportation, and after-sales services [2], [5]. These het-
complex internal electrochemical characteristics, the battery is
erogeneous and complex big data contain hidden valuable
susceptible to external factors such as temperature and humid-
information of intelligent manufacturing [1], [4], [5], which
ity, so establishing a general accurate mathematical or physical
brings unprecedented challenges for in-depth understanding
model is difficult [15]. (2) Due to the particle degeneracy
of intelligent manufacturing.
problems, the filtering method PF brings disadvantages for
L. Ren, J. Dong, Z. Meng, and L. Zhao are with School of Automation the RUL prediction accuracy.
Science and Electrical Engineering, Beihang University, Beijing, 100191, The data-driven methods analyze LIB life characteris-
China, Beijing Advanced Innovation Center for Big Data-based Precision
Medicine, Beihang University, Beijing, 100191, China. tic parameters [16] [17] such as capacity, current, voltage,
X. Wang is with Department of Computer Science, St. Francis Xavier impedance, by intelligent algorithm models such as neural
University, Antigonish, B2G 2W5, Canada. network [18], support vector machine (SVM) [19], Bayesian
M.J. Deen is with Department of Electrical and Computer Engineering,
McMaster University, Hamilton, L8S 4K1, Canada. regression [20]. An adaptive recurrent neural network RUL
L. Ren is the corresponding author, E-mail: renlei@buaa.edu.cn. prediction method for LIB impedance spectrum data was

1551-3203 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 03:02:59 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.3008223, IEEE
Transactions on Industrial Informatics

proposed in [21]. A method for predicting the remaining life time series, we propose a lithium-ion battery RUL prediction
of LIBs based on the nonlinear auto regressive (AR) model method based on CNN and LSTM. Moreover, the convolution
for its capacity, and that closely approaches its nonlinear layers of CNN require much data but the data of LIB are
degradation trajectory was presented in [22]. In [23], the insufficient. However, by using a autoencoder, we achieved
Gaussian Process Regression (GPR) method that takes the a significant increase in the dimensions of data that can be
capacity characteristics of LIBs into consideration was used transmitted to the CNN for more effective training. Also,
to describe its uncertainty of the prediction. Even though considering the noise presented in the model prediction curve,
the shortcomings of model-driven methods are avoided by we propose a post-smoothing method to reduce noise and
selecting data-driven methods, there are still limitations: (1) solve the discontinuity problem in the prediction results.
Some methods, such as GPR require large statistical datasets to This improves the interpretability of the prediction results by
complete the learning process of the model. (2) Most methods outputting smooth prediction curve. Algorithm 1 shows the
obtain only a single prediction result without uncertainty framework of our proposed Auto-CNN-LSTM method.
information, largely limiting their prediction accuracy.
To predict RUL with satisfactory accuracy, three main chal- Algorithm 1 Framework of Auto-CNN-LSTM proposed by
lenges have to be addressed. (1) Limitation of amount of data this Article
and noise in data are impeding offline training of data-driven Input: The set of original feature vectors with same di-
models. Meanwhile, information between the state of LIB in mensions d1 , representing each adjacent charging and
adjacent charge and discharge cycles can be used to train the discharging cycle from raw data, Pn ;
model. Besides, fluctuations among cycles are barely analyzed Output: Battery’s RUL prediction results corresponding to
in most models. (2) Due to the complex and unstructured each cycle in whole process, and the prediction curve.
characteristics of data, analyzing and processing data of LIB 1: Vectors with d1 dimensions in Pn are passed through
for industrial optimization and scheduling is challenging with the autoencoder, which augments the dimension of every
most data-driven approaches. 3) Most data-driven methods vector to be d2 ;
output a single-valued prediction result corresponding to the 2: Feature maps formed by feature vectors with d2 dimen-
number of charge and discharge cycles, and the prediction sions are stacked over as feature maps, which will be
curves are usually discontinuous. mined for deep information by the CNN;
Considering the limitations of previous research and meth- 3: The same feature maps with d2 dimensions are used to
ods, we propose a data-driven prediction model, namely name- extract correlation between adjacent cycles by RNN with
ly Auto-CNN-LSTM, for LIB RUL and our main contributions LSTM;
summarized as follows. (1) A model composed of autoen- 4: Features obtained by CNN and LSTM are concatenated
coder, convolution neural network (CNN) and long short-term to be a vector containing information both in time series
memory (LSTM) is proposed to predict RUL of LIB. CNN and and in depth; then, the vector will be operated by DNN
LSTM use correlation between adjacent cycles of the battery, with multiple layers to predict the final RUL;
and exploits features among sequences, which is beneficial for 5: The RUL prediction results are filtered to obtain smoother
attenuating data noise and enhancing prediction accuracy. (2) and more stable result;
An auto autoencoder is introduced to process and reconstruct
battery data in time series by augmenting dimensions of data, In the model, the LIB characteristics are initially extracted
which is used to extract more valuable information from raw from the raw data by the autoencoder, which augments the
data, preparing for more efficient training afterwards. (3) A number of data dimension from 21 to 50. Then, feature
post-smoothing method is proposed to solve the discontinuity expansion is performed by the CNN and LSTM model. The
problem in the prediction results. Filters of different types are CNN mines deep information and LSTM extracts the timing
compared, allowing us to achieve more accurate results than information between the data, and the features extracted by
models without filters. CNN and LSTM are combined into the fully connected layer.
The rest of this paper is organized as follows: Section II The RUL prediction of the LIB is output after a 7-layers Deep
describes related works in the field of LIB RUL prediction. Neural Network (DNN). Finally, in order to smooth the fitting
Section III describes the methodology proposed, followed by curve, the output is respectively filtered by first-order linear,
the procedure of the experiments. Section IV presents and second-order and third-order smoothing networks. Afterwards,
discusses results of experiments. Finally, the conclusions are the accuracy calculated by these networks are compared, and
presented in section V. the best accuracy is selected as the prediction result. Using
II. M ETHODOLOGY experiments with real data, the prediction results demonstrated
the effectiveness of our proposed method.
In this section, the framework (Figure 1) of the proposed
lithium-ion battery RUL prediction method, namely Auto- A. Autoencoder of Auto-CNN-LSTM
CNN-LSTM is presented. The autoencoder is an unsupervised neural network struc-
Due to ability of CNN to mine hidden information in ture mapping input to output [20]. The network structure of
limited data, and the ability of LSTM to process information in the autoencoder is shown in Figure 2. It is composed of input,

1551-3203 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 03:02:59 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.3008223, IEEE
Transactions on Industrial Informatics

Hidden Layer
Original with Dimension d2
Data RUL True Value Encoder with Decoder with
in all t Cycles Dimension d1 Dimension d1
Extract
Feature Features Feature Vectors
Extraction Autoencoder:
with Dimension d1
Augment
Features
Feature Vectors
with Dimension d2

...

...
Feature

...
CNN: Mine

...

...
Expansion Deep LSTM:
Information Extract Timing
Feature Map Information
Flattening
Feature Vectors Feature Vectors
with Dimension d3 with Dimension d4

Data Input for CNN and LSTM

Normalization
Feature Vectors with
RUL
DNN
Dimension d3+d4 Fig. 2: Autoencoder Structure in Auto-CNN-LSTM
Prediction
First-order RUL Prediction
Filter: Smooth Value in all t Cycles
will be enlarged to a 14 × 50 time domain feature map, which
Output RUL will be the input of CNN and LSTM.
Results
B. Convolution Neural Network (CNN)
Fig. 1: Framework for Lithium-ion Battery RUL Prediction
To the best of our knowledge, this is the first attempt at using
CNN for predicting the lifetime of LIBs. A typical CNN con-
sists of convolution layers, pooling layers and fully connected
hidden and output layers. The hidden layer is also referred as layers. A discrete two-dimensional convolution formula is:
the feature extraction layer. The forward conduction process of
the encoder is divided into two parts: encoding and decoding.
∑
s ∑
s
The encoding function expression is: A(i, j) = K(m, n) ∗ z(i + m, j + n). (3)
m=−s n=−s
z = E(x) = s(w1 x + b1 ), (1)
If we assume the kernel size of matrix K is (2s + 1) × (2s +
d1 ×1 1), then A(i,j) is the resulting matrix from convolution. Also,
where x∈R denotes the input data, and d1 denotes the
dimension of the input data; z∈Rr×1 denotes the characteristic z(i,j) denotes the input matrix which is the encoded matrix
expression of the hidden layer, and r denotes the number processed by encoder.
of neurons in the hidden layer; w1 ∈Rr×d1 denotes the input The CNN module can extract deep information from the
weight of the hidden layer; b1 ∈Rr×1 denotes the implicit input feature map output of the autoencoder. According to the results
bias; s denotes the activation function. of our extensive tests, we chose the most appropriate CNN
In the decoding process, the feature expression z of the structure. Fully connected layers are usually used to map the
hidden layer is mapped to the input data x through the features learned by convolution layers and pooled layers to
decoding process, and the function expression x̄ is: the sampling space. In our CNN, the fully connected layer
is discarded to reduce the amount of calculation, contributing
to a more light-weight and effective model without too many
x = D(z) = s(w2 x + b2 ), (2)
parameters to train. The network structure is shown in Figure
in which w2 ∈Rd1 ×r , b2 ∈Rd1 ×1 . 3.
This method applies autoencoder to expand the data, due
to limitation of battery data when conducting convolution
and pooling in CNN. In this way, autoencoder is used to
significantly augment the dimensions of the data that is used
by the CNN for more effective training.
Vectors with 21 dimensions (d1 ) of each adjacent 14
charging and discharging cycles are vertically stacked into
a time domain feature map with size of 14 × 21. The 21-
dimensional LIB data vectors are first input to the encoder,
then output with 50 dimensions (d2 ). After the encoding
process of autoencoder, the feature map from original dataset Fig. 3: CNN Structure in Auto-CNN-LSTM

1551-3203 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 03:02:59 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.3008223, IEEE
Transactions on Industrial Informatics

The CNN structure comprises 6 layers including 3 con- Dropout Dropout Dropout
volution layers, 2 pooling layers and a flattening layer. The
specific network structure is composed in the following or-
der, [Convolution, Max Pooling, Convolution, Max Pooling,
Convolution, Flatten]. The CNN utilizes the characteristics
of the LIB in each charging and discharging cycle, finding
the time-domain characteristics between adjacent charging
and discharging cycles that can be extracted by autoencoder.
During the forward propagation of the deep CNN, the LIB
feature map is gradually blurred, and the global information
of each feature map will be gradually highlighted.
Ultimately, information extracted by this section will be Fig. 4: LSTM Structure in Auto-CNN-LSTM
transformed into a feature vector containing 480 elements
by the flattening layer. This feature vector will be a part of
combination with LSTM, to be input into the DNN for final it = σ(Wi · [ht−1 , xt ] + bi ), (5)
prediction of the RUL. C t = tanh(Wg · [ht−1 , xt ] + bg ), (6)
C. Long Short-Term Memory (LSTM)
ot = σ(Wo · [ht−1 , xt ] + bo ), (7)
Because the data of the LIB are collected in charging
and discharging cycles, which are time series, the recurrent C t = f t ∗ C t−1 + it ∗ C t , (8)
neural network (RNN) is suitable to process time series
when making use of internal memory. There are both internal ht = ot · tanh(C t ), (9)
feedback connections and feedforward connections between its in which f denotes forgetting gate output, i denotes input gate
processing units, which contributes to the processing sequence output, o denotes output gate output, ht denotes t moment
of information by the RNN. However, since RNN can only hidden layer output, xt denotes input at time t, W denotes
store part of the sequence, its performance on a long sequence connection weight parameter, B denotes offset parameter, and
is worse than on a short sequence, resulting in a decrease in C t denotes the intermediate variable, which is multiplied by
accuracy for long sequences. In addition, the data sequence of the input gate result and sent to the state space.
LIBs in this experiment is relatively long and there are hun- With 14 adjacent charging and discharging cycles vertically
dreds of time series, so Long Short-Term Memory (LSTM) stacked into a time domain feature map, the input of the
is selected as an alternative to a conventional RNN. LSTM module is also composed of a sequence of time
The LSTM is a special kind of RNN. It overcomes the prob- domain features of these cycles. In this sequence, each element
lem that a RNN cannot handle long-distance dependences [24]. corresponds to the time domain characteristics obtained by
There is a certain correlation between the hidden layer nodes in autoencoder training in one specific charging and discharging
the LSTM. When data are sequentially input into the network, cycle.
the calculation of the hidden layer nodes depends not only on
The RNN can accurately capture the trend of each feature
the input of the current layer but also on the activation values
over time obtained by the autoencoder. We explore the use of
of nodes in hidden layer at previous moment. As for input
LSTM network structure to extract the time-domain character-
sequence, the LSTM network layer will work out the hidden
istics of LIBs, and incorporate the extracted features into the
layer sequence and the output sequence.
feature vectors for the next stage. In the forward propagation
The RNN module can extract amounts of timing information
process of the LSTM, the characteristics of the LIB output by
from the input feature sequences. Building on the work in [25]
the autoencoder are constructed into a sequence of features in
and our own experience, we chose LSTM to form the RNN
time sequence. After calculation of the network, the correlation
module. Part of the LSTM network structure used in our work
between moments in the input sequence will be gradually
is shown in Figure 4.
highlighted.
The RNN structure consists of 4 layers, including 2 LSTM
layers and 2 Dropout layers. The specific network structure D. Deep Neural Network (DNN)
is composed in the following module order, [LSTM, Dropout,
Due to its deep architecture, a deep neural network (DNN)
LSTM, Dropout], and the structure in Figure 4 is only half of
model consists of multiple network layers, which can be
the whole LSTM module. Although the CNN structure pro-
used to obtain features of the raw data from linear and
posed above can find the global characteristics of the feature
nonlinear operations [26]. With features containing sequential
map, the RNN structure is still preferred, which benefits from
information obtained by CNN and LSTM, the DNN is capable
its unique memory capability. The calculation formula for each
of predicting RUL with better accuracy.
gate function and state transfer process in the LSTM module
CNN and LSTM modules obtain and represent features
is as follows:
with a 484-dimensional vector from autoencoder. In order to
f t = σ(Wf · [ht−1 , xt ] + bf ), (4) establish an accurate mapping of the above eigenvectors to the

1551-3203 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 03:02:59 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.3008223, IEEE
Transactions on Industrial Informatics

battery RUL prediction, this method utilizes a 7-layer DNN to In order to explore the influence of filters with different
build the final LIB RUL regression model shown in Figure 5. order, we studied the results of predictor with no filtering and
In this 7-layers DNN structure, the input is a 484-dimensional with using a linear filter, a second-order filter and a third-order
feature vector, and the output is a one-dimensional battery filter. According to the analysis of the prediction errors in these
life prediction vector ranging from 0 to 1. Besides, in order to four cases, the output of the first-order filter is selected as the
established the LIB RUL regression model with generalization final predicted value of the LIB RUL. This choice meets the
properties, the Dropout and L2 regularization techniques are basic assumption that a general battery’s RUL value should be
used in the DNN structure. linear with operating time.
The first-order linear filter predicts the RUL of the LIB at
Fully Connected time t, which depends not only on the output of the DNN LIB
regression model at time t, but also at previous moments at
t-7, t-6, t-5, t-4, and output effects at time t-3, t-2, and t-1.
Experimental analysis suggests that the predicted output values
by the first-order filter and the predicted output values directly
without any filter have almost no difference in accuracy, but
the filtered curve is smoother and more stable.
...

...

...
...

...
...

TABLE I: Notations appearing in Methodology

...

Model Notation Meaning

x Input Feature Map for Encoder
Autoencoder z = E(x) Encoded Map output by Encoder
x = D(z) Feature Map output by Decoder
484 504 280 180 89 50 29 1 CNN A Convolution Result Operated on z
ft Forgetting Gate Output
Input Layer Hidden Layer Output Layer it Input Gate Output
LSTM Ct Intermediate Variable
ot Input Gate Output
Fig. 5: DNN Structure in Auto-CNN-LSTM
ht Hidden Layer Output

It is noted that the root mean square error (RMSE) between

the output value of the DNN network and the actual value III. E XPERIMENTS
of the battery RUL, is applied to measure the accuracy of
In this section, experiments will be carried out to demon-
prediction results.
strate the performance of the proposed model for LIBs RUL,
v in which the #5, #6 and #25, #26, #27 batteries data of the
u
u1 ∑ m
NASA PcoE will be selected as the training dataset, and the #7
RM SE = t (yi − ŷi )2 , (10)
m i=1 and #28 battery data set will be selected as the testing dataset.
A. Data Description
in which yi represents RUL prediction result corresponding to
i-th moment in a total of m moments, and ŷi represents true In this part, as shown in the Figure 6, we can obtain the
value at the same moment. data via the following charging and discharging processes.
Through the back propagation algorithm, the weights of
CNN, LSTM and DNN are updated to realize the network
learning function. The adam optimizer is selected for network
training so that the network parameters can quickly converge
to an acceptable range. After calculation of the DNN, the
output from this part is a one-dimensional RUL prediction
vector which needs to be smoothed.

E. Smoothing Method in Auto-CNN-LSTM

After directly processed by the LIB RUL regression model
composed of the DNN described above, the output is often
discontinuous, unlike true remaining life of the battery. In
view of this problem, we attempt to smooth the predicted Fig. 6: The Charging and Discharging Process in Lithium-ion
value by passing the output of the prediction model through Batteries
a filter for a more continuous and stable output. The method
we select to smooth the curve is the least squares polynomial (1)The first step is the charging process. The LIB will be
approximation. charged by a constant current (Constant Current, CC) at

1551-3203 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 03:02:59 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.3008223, IEEE
Transactions on Industrial Informatics

1.5A until its voltage equals to 4.2V. Then, the LIB will be TABLE II: Comparison of Accuracy with Different Output
maintained at a constant voltage (Constant Voltage, CV) at Size of Encoder. The Encoder outputting dimensions are set
4.2V until the charging current decreases to 20mA. to vary from 40 to 60 and the filters are selected as linear,
(2) The second step is the discharging process in which second-order and third-order filters.
different LIBs will be discharged with a constant current of Output Size Filters to Smooth RMSE(%) Accuracy(%)
2A until the voltage decreases to different values respectively. of Encoder
without filter 9.43 90.57
The experiments need to constantly repeat the charging and Linear filter 9.32 91.68
discharging cycles above to accelerate the battery degradation 40
Second-order filter 9.39 90.61
process. The experiment will end when the LIB life ends, that Third-order filter 9.39 90.61
without filter 8.04 91.96
is its capacity drops to 70%. Linear filter 7.98 92.02
According to data of these kinds of LIB, we took the 45
Second-order filter 8.03 91.97
measured voltage or current by observing how it changes Third-order filter 8.03 91.97
without filter 5.03 94.97
to some specified value. For instance, we took the value of Linear filter 4.84 95.16
largest measured voltage in every complete charging process. 50
Second-order filter 4.98 95.02
Meanwhile, we record the time and value when the charging Third-order filter 4.98 95.02
without filter 7.12 92.88
(discharging) current (voltage) gets its preset value described Linear filter 6.97 93.03
in the dataset document. In this way, a 21-dimensional vector 55
Second-order filter 7.09 92.81
is formed to include physical variants such as voltage mea- Third-order filter 7.09 92.81
sured at charge, temperature at discharge for every charging without filter 7.91 92.09
Linear filter 7.77 92.23
and discharging cycle. There are 168 cycles for #5, #6 and #7 60
Second-order filter 7.89 92.11
LIBs and 28 cycles for #25, #26, #27 and #28 LIBs. Third-order filter 7.89 92.11

B. Feature Extraction by Autoencoder

After the characteristics of each LIB are extracted as de- this feature vector will be input into the DNN for the final
scribed above, this method applies autoencoder to augment RUL prediction before smoothing.
the dimension of raw data, for LIBs #5, #6, #7, and #25, #26,
#27, #28. Vectors with 21 dimensions of each original adjacent D. Data Normalization
14 charging and discharging cycles are vertically stacked into In the feature vectors, different physical variants possess
a time domain feature map with a size of 14 × 21. The 21- different units such as Ampere, Volt and degree Celsius. The
dimensional LIB data vectors are input to the encoder to get value of these variants can also be extremely different so that
an output with 50 dimensions. After the encoding process of their weights on the final result vary a lot and reduces the
autoencoder, the feature maps from original dataset will be training efficiency. Consequently, data normalization is used in
enlarged to 14 × 50 time domain feature maps, which will be our model to unify all features to the same scale, which helps
the input of the CNN and the LSTM. every element predicting the RUL with the same weight. We
Note that, the augmented number of dimension was deter- transformed the range of the compressed features to [0,1] with
mined with a set of controlled experiments. In this set of the minimum-maximum normalization method, balancing the
experiments, the number of output dimensions was selected predicting weight for every variant with different magnitudes.
as 40, 45, 50, 55 and 60 respectively, while other conditions E. Training of Auto-CNN-LSTM Model
were kept the same. The final RUL prediction accuracy with
For autoencoder, there are three layers in which ReLu
different output dimensions of the encoder is compared as
and sigmoid function are respectively utilized as activation
follows. According to the comparison results in Table 2, the
function of the hidden layer and output layer.
dimension number of encoder is selected to be 50, which
For the CNN, its structure in this work is composed in
means the feature maps input for CNN and LSTM are 14×50.
the following order, [Convolution, Max Pooling, Convolution,
C. Feature Fusion for CNN and LSTM Max Pooling, Convolution, Flatten]. The kernel size of the
convolution layer is finally determined as [(3 × 3), (3 × 3),
In the forward propagation process of the LSTM, the
(2 × 2)], which can extract features among adjacent physical
characteristics of the LIB output by the autoencoder are
variables in a vector horizontally and extract timing features
constructed into a sequence of features in time sequence. The
among adjacent vectors vertically. At the same time, the largest
CNN module extracts deep information from the feature maps
size of the pooling layers is determined to be 2×2. Each layer
with size of 14 × 50 output from the autoencoder. During
selected ReLu as the activation function.
the forward propagation of the deep CNN, the LIB feature
map is gradually blurred, but the global information of each
f (x) = max(0, x) (11)
feature map or correlation between adjacent moments will
be gradually highlighted. Ultimately, information extracted The LSTM structure consists of 4 layers, including 2
by CNN and LSTM will be concatenated to be a feature LSTM layers and 2 Dropout layers in such sequence, [LSTM,
vector containing 484 elements by the flattening layer, and Dropout, LSTM, Dropout].

1551-3203 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 03:02:59 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.3008223, IEEE
Transactions on Industrial Informatics

The DNN parameters were trained with normalized training

data. In order to establish an accurate mapping of the above
eigenvectors to the battery RUL prediction, a 7-layer DNN
is used for the final LIB RUL regression prediction. In this
7-layers DNN structure, the input of the DNN model is a
concatenated feature vector extracted by CNN and LSTM, and
the output is the final RUL prediction vector ranging from 0 to
1. Also, in order to enable the established LIB RUL regression
model with generalization properties, the Dropout and L2
澳 澳
regularization techniques are used in the DNN structure. The
number of neurons in each layer is determined as [504, 280, (a) ADNN (b) SVM
180, 89, 50, 29, 1] according to extensive experiments. The
Fig. 9: Prediction Results of ADNN and SVM for #7 LIB
parameters of Dropout are determined to be [0.59, 0.59, 0.589,
0.591, 0.59, 0.59], and the coefficient of L2 regularization is
0.001. The activation function of each layer selects ReLU.
7 (d) shows the results obtained by output through the first-
After training, the DNN model outputs RUL of the LIB while
order linear, second-order and third-order filters. In terms of
the input is from the combination of CNN and LSTM.
RMSE, corresponding value after filter in sequence of the three
F. Prediction and Evaluation of RUL filters above are 4.84%, 4.98% and 4.98%. By comparing
prediction errors in four cases, the output of the first-order
To measure the performance of the proposed model, the data filter is finally selected as the final predicted value of the LIB’s
set of #7 and #28 LIB will be used as the testing dataset. We RUL. This also meets the basic assumption that the general
compared the performance of proposed model with two other battery RUL value is linear with operating time. In the case
models, which are autoencoder-deep neural network (ADNN) of first-order filtering, the accuracy rate reaches maximum of
and supporting vector machine (SVM). 95.16%. Compared with Fig. 7 (a) without filtering, the curve
IV. R ESULTS AND D ISCUSSION of Fig. 7 (b) is also smoother.

A. Analysis of Results C. Comparison with Other Models

After experiments based on process described above, we got In order to evaluate the performance of our model, we have
our results of our model. Figures 7 and 8 shows the results of compared experimental results with other two methods [27],
a #7 and #28 battery prediction with Auto-CNN-LSTM. As [28] using the same training and testing sets. As shown in Fig.
shown as Figures. 7 and 8, there are respectively 4 different 9 (a), the prediction of #7 LIB with ADNN model proposed
sub-figs. These sub-figures shows the charge and discharge in [27] is compared to the true value. The ADNN model is
cycles in remaining useful life of LIB, and !!! the normalized a data-driven method based on a autoencoder and DNN. In
values of remaining cycle times for LIB without a filter (a) Fig. 9 (a), The horizontal axis represents charge and discharge
and with three types of filters ((b)-(d)). The orange solid lines cycles in remaining useful life of LIB, and the vertical axis
are used to show the true values of the relationship between represents normalized values of remaining cycle times for LIB.
RUL and charge and discharge times, and the green dashed In this figure, the orange solid line shows the true value, and
lines show the predicted values. the green dashed line shows the predicted values using ADNN.
From the experimental results, the RUL prediction results From the curves shown in Fig. 9 (a), we could find that the
are satisfactory. The RMSE is 5.03%, and the accuracy is RUL prediction curve has a satisfactory performance with a
94.97%. The reason for the error may be that the amount RMSE of 11.80%, and an accuracy rate of 88.20%. However,
of data is still insufficient even though the original data is a simple fully connected neural network cannot give good
augmented with the autoencoder. Another reason may be results, and it is 6.96% less accurate than Auto-CNN-LSTM
the effect of noise in the data. Although the autoencoder is proposed in the current work described in this paper. Next,
added for noise reduction, the effect of data noise cannot be Fig. 9 (b) shows the prediction of #7 LIB with a SVM model
completely avoided. As shown in Fig. 7 (a) and 8 (a), the described in [28]. From the experimental results in Table 3, the
RUL prediction curves are well fitted, but the curve at 60th RMSE of SVM is 18.23% and the accuracy rate is 81.77%.
to 100th generation for #7 LIB (15th to 20th for #28 LIB) From these two comparisons using ADNN and SVM, it is
on the abscissa is still rough. Therefore, this paper adopts the shown that such traditional data-driven models are not ideal
idea of filtering and performs first-order linear, second-order for LIB RUL prediction.
and third-order filtering respectively. As shown in Table 3, the comparison results demonstrate
that Auto-CNN-LSTM gives better accuracy than ADNN or
B. Analysis of Results with Filters SVM for LIB RUL prediction. After filtering, the prediction
To explore the influence of different order filters on the accuracy is further improved and the result of linear filtering is
prediction model output, the following Figs. 7 (b), 7 (c) and the best. Also, note that the RMSE of the experimental results

1551-3203 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 03:02:59 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.3008223, IEEE
Transactions on Industrial Informatics

(a) without filter (b) first-order linear filter (c) second-order filter (d) third-order filter

Fig. 7: Prediction Results of Auto-CNN-LSTM with Filters for #7 LIB

(a) without filter (b) first-order linear filter (c) second-order filter (d) third-order filter

Fig. 8: Prediction Results of Auto-CNN-LSTM with Filters for #28 LIB

in the training data is 3.52%, which is slightly better than the Auto-CNN-LSTM prediction was an autoencoder to augment
same model performed with test data because of overfitting. the original data which are then mined for deep information
Overfitting may result in non-completely optimized parameters with a convolution neural network (CNN) to get the feature
or relatively a small L2 regularization coefficient. map. In parallel, we used long short-term memory (LSTM)
to get its time series which are combined with the feature
TABLE III: Comparison of Experiment Results with Testing map and normalized. Then a 7-layers deep neural network
Data (DNN) is used to predict the RUL of the LIB and the predicted
result is smoothed using a first-order filter. Using the proposed
Method RMSE(%) Accuracy(%)
Auto-CNN-LSTM model, and training with true datasets, the
Auto-CNN-LSTM 5.03 94.97
Linear Filter 4.84 95.16 predicted results have a root-mean-square-error of 4.8%, sig-
Second-order Filter 4.98 95.02 nificantly outperforming two other data-driven models based
Third-order Filter 4.98 95.02 on autoencoder and deep neural network (ADNN with error
ADNN 11.80 88.20
SVM 18.23 81.77 of 11.8%) and support vector machine (SVM with error of
18.2%). In future, we plan to investigate other techniques to
further improve the RUL prediction accuracy by optimizing
V. C ONCLUSION the number of neural network layers, dropout and L2 regu-
larization parameters as well as applying our model to other
Currently, we are witnessing significant advances in intel-
prediction problems where lifetime in time series is important.
ligent manufacturing due to the integration of information,
communication, big data and computing technologies in the VI. ACKNOWLEDGMENT
manufacturing process. For intelligent manufacturing systems,
The research is supported by The National Key Research
efficient and long lifetime energy sources such as lithium-ion
and Development Program of China No. 2018YFB1004001,
batteries (LIBs) are essential. However, an important problem
and the NSFC (National Science Foundation of China) project
with LIBs is how to accurately predict their remaining useful
No. 61572057 and 61836001.
life (RUL).
In this paper, we propose a three-step data-driven process R EFERENCES
– feature extractor, feature expansion and RUL prediction to [1] L. Ren, Y. Laili, X. Li, and X. Wang, “Coding-based large-scale task
determine the RUL of LIBs. Our proposed method, called the assignment for industrial edge intelligence,” IEEE Transactions on

1551-3203 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 03:02:59 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.3008223, IEEE
Transactions on Industrial Informatics

Network Science and Engineering, DOI: 10.1109/TNSE.2019.2942042, tion and electronics-rich systems,” IEICE ESS Fundamentals Review,
2019. vol. 3, no. 4, pp. 19–30, 2009.
[2] L. Ren, Z. Meng, X. Wang, R. Lu, and L. T. Yang, “A wide-deep- [22] D. Liu, Y. Luo, Y. Peng, X. Peng, and M. Pecht, “Lithium-ion battery
sequence model based quality prediction method in industrial process remaining useful life estimation based on nonlinear ar model combined
analysis,” IEEE Transactions on Neural Networks and Learning Systems, with degradation feature,” in Annual Conference of the Prognostics and
DOI: 10.1109/TNNLS.2020.3001602, 2020. Health Management Society, Denver, USA, Jun. 18-21, 2012, pp.1803-
[3] X. Wang, L. T. Yang, L. Song, H. Wang, L. Ren, and M. J. Deen, 1836.
“A tensor-based multi-attributes visual feature recognition method for [23] K. Goebel, B. Saha, A. Saxena, J. R. Celaya, and J. P. Christophersen,
industrial intelligence,” IEEE Transactions on Industrial Informatics, “Prognostics in battery health management,” IEEE instrumentation &
DOI: 10.1109/TII.2020.2999901, 2020. measurement magazine, vol. 11, no. 4, pp. 33–40, 2008.
[4] X. Wang, L. T. Yang, Y. Wang, L. Ren, and M. J. Deen, “Adtt: [24] Y. Zuo, Y. Wu, G. Min, C. Huang, and K. Pei, “An intelligent anomaly
A highly-efficient distributed tensor-train decomposition method for detection scheme for micro-services architectures with temporal and
iiot big data,” IEEE Transactions on Industrial Informatics, DOI: spatial data analysis,” IEEE Transactions on Cognitive Communications
10.1109/TII.2020.2967768, 2020. and Networking, 2020.
[5] L. Ren, Z. Meng, X. Wang, L. Zhang, and L. T. Yang, “A data-driven [25] Y. Zuo, Y. Wu, G. Min, and L. Cui, “Learning-based network path
approach of product quality prediction for complex production sys- planning for traffic engineering,” Future Generation Computer Systems,
tems,” IEEE Transactions on Industrial Informatics, DOI: 10.1109/TI- vol. 92, pp. 59–67, 2019.
I.2020.3001054, 2020. [26] X. Yuan, B. Huang, Y. Wang, C. Yang, and W. Gui, “Deep learning-
[6] H. Li, K. Ota, and M. Dong, “Energy cooperation in battery-free based feature representation and its application for soft sensor modeling
wireless communications with radio frequency energy harvesting,” ACM with variable-wise weighted sae,” IEEE Transactions on Industrial
Transactions in Embedded Computing Systems, vol. 17, no. 2, p. 44, Informatics, vol. 14, no. 7, pp. 3235–3243, 2018.
2018. [27] L. Ren, L. Zhao, S. Hong, S. Zhao, H. Wang, and L. Zhang, “Remaining
[7] S. Yuchen, L. Datong, H. Yandong, Y. Jinxiang, and P. Yu, “Satellite useful life prediction for lithium-ion battery: A deep learning approach,”
lithium-ion battery remaining useful life estimation with an iterative up- IEEE Access, vol. 6, pp. 50 587–50 598, 2018.
dated rvm fused with the kf algorithm,” Chinese Journal of Aeronautics, [28] J. Wei, G. Dong, and Z. Chen, “Remaining useful life prediction and
vol. 31, no. 1, pp. 31–40, 2018. state of health diagnosis for lithium-ion batteries using particle filter and
support vector regression,” IEEE Transactions on Industrial Electronics,
[8] D. Wang, J. Liu, and R. Srinivasan, “Data-driven soft sensor approach for
vol. 65, no. 7, pp. 5634–5643, 2017.
quality prediction in a refining process,” IEEE Transactions on Industrial
Informatics, vol. 6, no. 1, pp. 11–17, 2010.
[9] A. T. Shimamoto, B. R. Tanaka, and C. K. Tanaka, “A study on
evaluation method for the lithium-ion battery life performance for
stationary use,” in 2013 International Conference on Clean Electrical
Power (ICCEP), Alghero, Italy, Jun. 11-13, 2013, pp. 115-119. Lei Ren (Member, IEEE) received the Ph.D. degree
[10] S. Lee, J. Kim, J. Lee, and B. H. Cho, “State-of-charge and capacity in computer science from the Institute of Software,
estimation of lithium-ion battery using a new open-circuit voltage versus Chinese Academy of Sciences, Beijing, China, in
state-of-charge,” Journal of Power Sources, vol. 185, no. 2, pp. 1367– 2009. Currently, he is an Associate Professor and
1373, 2008. the Deputy Head with the Cloud Manufacturing
[11] R. Xiong, Y. Zhang, H. He, X. Zhou, and M. G. Pecht, “A double- Research Center, School of Automation Science and
scale, particle-filtering, energy state prediction algorithm for lithium-ion Electrical Engineering, Beihang University, China.
batteries,” IEEE Transactions on Industrial Electronics, vol. 65, no. 2, He has authored or co-authored over 60 papers and
pp. 1526–1538, 2017. got over 3000 citations according to Google Scholar.
[12] B. Saha, K. Goebel, S. Poll, and J. Christophersen, “Prognostics meth- His research interests include Cloud Manufacturing,
ods for battery health monitoring using a bayesian framework,” IEEE Industrial Internet-of-Things, AI and Big Data.
Transactions on Instrumentation and Measurement, vol. 58, no. 2, pp.
291–296, 2008.
[13] W. He, N. Williard, M. Osterman, and M. Pecht, “Prognostics of lithium-
ion batteries based on dempster–shafer theory and the bayesian monte
carlo method,” Journal of Power Sources, vol. 196, no. 23, pp. 10 314–
10 321, 2011. Jiabao Dong is a Ph.D candidate at School of
[14] X. Su, S. Wang, M. Pecht, L. Zhao, and Z. Ye, “Interacting multiple Automation Science and Electrical Engineering in
model particle filter for prognostics of lithium-ion batteries,” Microelec- Beihang University, China. His research interests
tronics Reliability, vol. 70, pp. 59–69, 2017. include Industry Intelligence, Robot Grasping Policy
[15] W. Waag, C. Fleischer, and D. U. Sauer, “Critical review of the methods and Semantic Segmentation.
for monitoring of lithium-ion batteries in electric and hybrid vehicles,”
Journal of Power Sources, vol. 258, pp. 321–339, 2014.
[16] C. Zhang, M. Dong, and K. Ota, “Enabling computational intelligence
for green internet of things: Data-driven adaptation in lpwa networking,”
IEEE Computational Intelligence Magazine, vol. 15, no. 1, pp. 32–43,
2020.
[17] X. Liu, A. Liu, T. Wang, K. Ota, M. Dong, Y. Liu, and Z. Cai, “Adaptive
data and verified message disjoint security routing for gathering big
data in energy harvesting networks,” Journal of Parallel and Distributed
Computing, vol. 135, pp. 140–155, 2020. Xiaokang Wang (Member, IEEE) received the
[18] H.-T. Lin, T.-J. Liang, and S.-M. Chen, “Estimation of battery state Ph.D degree in Computer System Architecture in
of health using probabilistic neural network,” IEEE Transactions on Huazhong University of Science and Technology,
Industrial Informatics, vol. 9, no. 2, pp. 679–685, 2012. Wuhan, China, in 2017. Currently, he is a Post-
[19] X. Chen, J. Yu, D. Tang, and Y. Wang, “Probabilistic residual life Doctoral Fellow with Department of Computer Sci-
prediction for lithium-ion batteries based on bayesian ls-svr,” Acta ence, St. Francis Xavier University, Canada. His re-
Aeronautica et Astronautica Sinica, vol. 34, no. 9, pp. 2219–2229, 2013. search interests are Cyber-Physical-Social Systems,
[20] M. Mishra, J. Martinsson, M. Rantatalo, and K. Goebel, “Bayesian hi- Big Data, Parallel and Distributed Computing, and
erarchical model-based prognostics for lithium-ion batteries,” Reliability Cloud-Edge Computing.
Engineering & System Safety, vol. 172, pp. 25–35, 2018.
[21] M. Pecht, “A prognostics and health management roadmap for informa-

1551-3203 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 03:02:59 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.3008223, IEEE
Transactions on Industrial Informatics

Zihao Meng is a Ph.D. candidate at School of

Automation Science and Electrical Engineering in
Beihang University, China. His research interests
include Industry Intelligence, Naturel Language Pro-
cessing and Soft Sensors.

Li Zhao is a graduate student at School of Automa-

tion Science and Electrical Engineering in Beihang
University. His research interests include Deep learn-
ing, Data-driven Methods, and Remaining Useful
Life (RUL) Prediction of Lithium-ion Battery.

M. Jamal Deen (Fellow, IEEE) was born in Guyana,

South America. He received his Ph.D. degree in
Electrical Engineering and Applied Physics from
Case Western Reserve University, Cleveland, USA,
in 1985. Currently, he is a Distinguished University
Professor, Senior Canada Research Chair, and Di-
rector of the Micro- and Nano-Systems Laboratory,
McMaster University, Hamilton, Canada. His current
research interests include nanoelectronics, optoelec-
tronics, smart sensors and data analytics, and their
emerging applications in health and environment.
His research records include more than 600 peer-reviewed articles and two
textbooks. Prof. Deen was the recipient of more than 75 awards and honors
including the McNaughton Gold Medalłthe highest recognition for engineers
from IEEE Canada. He was elected to Fellow status in 12 national academies
and professional societies including the Royal Society of Canada (RSC),
Chinese Academy of Sciences, National Academy of Sciences (India), and
American Physical Society. He served as the elected President of the Academy
of Science, RSC, between 2015 and 2017.

1551-3203 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 03:02:59 UTC from IEEE Xplore. Restrictions apply.