Open AccessArticle

Air Quality Prediction Based on Singular Spectrum Analysis and Artificial Neural Networks

Javier Linkolk López-Gonzales

^1,*

Rodrigo Salas

^2,3

Daira Velandia

^4,5 and

Paulo Canas Rodrigues

⁶

Escuela de Posgrado, Universidad Peruana Unión, Lima 15468, Peru

Biomedical Engineering School, Faculty of Engineering, Universidad de Valparaíso, Valparaíso 2362905, Chile

Center of Interdisciplinary Biomedical and Engineering Research for Health—MEDING, Universidad de Valparaíso, Valparaíso 2540064, Chile

⁴

Statistical Institute, Faculty of Science, Universidad de Valparaíso, Valparaíso 2362905, Chile

⁵

Center for Atmospheric Studies and Climate Change—CEACC, Universidad de Valparaíso, Valparaíso 2360102, Chile

⁶

Department of Statistics, Federal University of Bahia, Salvador 40170-110, Brazil

Author to whom correspondence should be addressed.

Entropy 2024, 26(12), 1062; https://doi.org/10.3390/e26121062

Submission received: 10 September 2024 / Revised: 19 November 2024 / Accepted: 29 November 2024 / Published: 6 December 2024

(This article belongs to the Special Issue Big Data Analytics and Information Science for Business and Biomedical Applications: Third Edition)

Download

Browse Figures

Figure 1
Long short-term memory network. Each neuron has a memory cell and three gates: input, output, and forget. The function of the gates is to safeguard information by stopping or allowing its flow. The input gate determines how much information from the previous layer is stored in the cell. The output gate determines how much the next layer knows about the cell. For its part, the forgetting gate decides whether to keep or forget the information. The LSTM can learn complex sequences. "> Figure 2
Network long short-term memory detailed. (A): cell state; (B): forget gate; (C): input gate; and (D): output gate. "> Figure 3
A time series in (A). It is executed according to the singular spectrum analysis mechanism (B). The residuals are captured and treated according to the characteristics of the long short-term memory network. Finally in (C), the prediction is made by adopting properties of both SSA and the LSTM. "> Figure 4
Prediction horizons. Data were normalized within the interval [0, 1]. Then, they were divided into two matrices: training and test. The models were trained with <math display="inline"><semantics> <mrow> <mi>N</mi> <mo>−</mo> <mn>48</mn> <mo>+</mo> <mn>1</mn> </mrow> </semantics></math> data and tested with the last 48 data. For the time series, 10 simulations were carried out with the adapted approach (singular spectrum analysis and long short-term memory), and the model with the best performance was selected. The results for all models correspond to the prediction of one, six, twelve, and twenty-four steps ahead of the test set. Likewise, <a href="#entropy-26-01062-t001" class="html-table">Table 1</a> shows the three metrics that evaluated the performance of the adapted methodology. "> Figure 5
Time series air pollution and the description of the behavior of the contaminant through the histogram. "> Figure 6
Eigenvalues—SSA. "> Figure 7
Eigenvectors—SSA. The first 7 components have useful information and the remaining components can be considered noise. "> Figure 8
Pairs of eigenvectors—SSA. It is mainly observed that with the pair 6 (0.42%) vs. 7 (0.42%), the size of the seasonality is predicted since it can be differentiated that there is a seasonality of 12 by the number of sides reported by the figure. This is summarized here, in that the pairs 4 (0.69%) vs. 7 (0.69%) and 6 (0.42%) vs. 7 (0.42%) are components that are related to each other; that is, they are close to each other. "> Figure 9
Correlation matrix—SSA. As observed, the diagonal indicates the autocorrelation of each component; in this case, there are 100 components. For example, the black squares represent a w-correlation of 1. Focusing on components 4, 5, 6, and 7, a high correlation is observed, the same as was observed in previous reports. They were similar in terms of magnitude and behavior. This similarity means that they can be grouped together. This report aims to maximize the correlations within the signal and within the noise, and minimize the correlations between the signal and the noise. "> Figure 10
Reconstructed series—SSA. Extraction of information from the time series: trend, seasonality, and residuals. "> Figure 11
Reconstructed series II—SSA. Clusters of the different seasonalities. "> Figure 12
Reconstructed series III—SSA. Here, the original series, the signal, which describes its behavior based on the composition of both trend and seasonality, and finally, the noise (residuals), are integrated. "> Figure 13
Signal and noise components after the SSA decomposition. The signal includes the deterministic part of the series. The noise includes the stochastic unstructured residuals of the series, which is not white noise. "> Figure 14
Accuracy measures (a) RMSE, (b) MAPE, and (c) MAE for all methods. "> Figure 15
Behavior of the hybrid model between the observed PM10 values (represented by the black lines) and the predictions’ generated red lines). ">

Versions Notes

Abstract

Singular spectrum analysis is a powerful nonparametric technique used to decompose the original time series into a set of components that can be interpreted as trend, seasonal, and noise. For their part, neural networks are a family of information-processing techniques capable of approximating highly nonlinear functions. This study proposes to improve the precision in the prediction of air quality. For this purpose, a hybrid adaptation is considered. It is based on an integration of the singular spectrum analysis and the recurrent neural network long short-term memory; the SSA is applied to the original time series to split signal and noise, which are then predicted separately and added together to obtain the final forecasts. This hybrid method provided better performance when compared with other methods.

Keywords:

air quality; singular spectrum analysis; artificial neural networks; hybrid method

1. Introduction

1.1. Deep Learning Approach in Air Pollution Prediction

Deep learning models have grown substantially due to their excellent performance and flexibility. In particular, the development of the models for the prediction of air quality [1] has become an active research field, highlighting recurrent neural networks (RNNs) and their variations [2] as the tools of modeling with better results in this field. Furthermore, these deep learning methods have been successfully applied in time series modeling to predict air quality [3,4,5,6,7,8,9].

Air pollutants present temporary correlations in their emission and diffusion. Therefore, it is crucial to consider this factor related to the objective variable in quantifying air quality [10]. In this sense, the works of [11,12,13] introduce the temporal correlation of the data to improve the performance of deep learning models. On the other hand, for the prediction of meteorological variables, Pak [10] proposed a convolutional neural network (CNN) and long short-term memory (LSTM) network model, CNN-LSTM, and in the extended LSTM model (LSTME) proposed by [12], the correlations based on PM_2.5 were captured. A Deep Belief Network (Geoi-DBN) model, with additional variables to PM_2.5, like temperature, humidity, and wind speed, was presented in [11]. In the deep neural network model proposed by Soh [13], temporal correlation, spatiotemporal, and geographic features were extracted using the LSTM, CNN, and integration of K-nearest neighbors (KNNs) and artificial neural networks (ANNs), respectively; the extracted data were finally assembled into an ANN to produce the prediction value.

1.2. Hybrid and Ensemble Methods

Hybrid methods and ensemble methods have been shown to perform significantly better than their counterparts, both in terms of classification and prediction, than single conventional methods such as multilayer perceptron, support vector machines, decision trees, discriminant analysis, and logistic regression, among others [14,15,16]. Moreover, ensemble methods can substantially improve model accuracy; in turn, they are considered supervised learning algorithms, benefiting different training algorithms to achieve higher test accuracy [14,17,18]. For example, Singh [19] applied an ensemble method and principal component analysis (PCA) algorithm to integrate air quality data and forecast air quality index (AQI) values. On the other hand, Xayasouk [20] developed models to predict pollutant concentrations for ten days ahead using the LSTM and a deep autoencoder (DAE). Likewise, Tao [2] proposed a short-term forecasting model that combines 1D convolutional neural networks and bidirectional closed recurrent units (CBGRUs), and Qi [21] proposed a hybrid model integrating graphical convolutional networks and long short-term memory (GC-LSTM) networks to model and forecast the temporal variation of PM_2.5 concentrations. Recently, Qiao [22] proposed using a self-organized fuzzy neural network to predict the hourly concentration of PM_2.5 using data from multiple sources.

From the approach of the time series, Hou et al. [23] developed an advanced hybrid wavelet neural network integrated by an autoregressive integrated moving average (ARIMA) model using a fuzzy method, and Díaz-Robles, L.A. et al. [24] proposed a hybrid model that combines ARIMA and an ANN to improve forecast accuracy in meteorological data. Meanwhile, Du et al. [25] developed a hybrid method for estimation and optimization that includes an empirical suite integrated with adaptive filters to remove noise and extract key features from the original data, followed by a wavelet neural network optimized for high estimation accuracy. Also, Albalawi, F., et al. [26] developed a hybrid HybPAS that integrates the linear regression–deep neural network model and statistics based on signal processing as input values. On the other hand, Siwek, K. and Osowski, S. [27] proposed a system for predicting the PM₁₀ concentration in the air using a combination of wavelet transformation and some predictors. They included the least successful predictors, extracted their most important information, and used them in the final forecasting procedure. Likewise, they applied an additional neural network for joint integration. Also, Liu et al. [28], from the bias–variance compensation of the prediction model, introduced the Gradient Boosting Decision Tree in the Bagging training framework to improve the base learning strength and then established the Bagging-GBDT model, using air quality and historical weather data to train the forecast model. In a similar approach, performed in [29], it was concluded that using ensemble ANNs, combined with the multilayer perceptron neural network, leads to better one-step forecast accuracy. On the other hand, Li et al. [30] developed a hybrid QPSO-SVR model to predict the atmospheric concentrations of PM_2.5 and NO₂ in the short term, with the QPSO algorithm used to select the optimal parameters that influence SVR performance. Finally, de Mattos Neto, P.S. [31] improved the performance of a hybrid system composed of a genetic algorithm and ANN from the modeling of a residual carried out by an MLP neural network and their hybrid system. This system uses as many series of residuals as necessary to obtain a residual with white noise behavior.

1.3. Nonparametric Statistical Methods in Time Series

Statistical methods for time series analysis have been extensively developed and discussed in the literature. However, most of these methods require specialized knowledge to approach their use in correctly modeling data. For example, in the classical context, the series may show a trend. Holt’s exponential smoothing is an option to model linear trend, and without seasonality [32,33], and if, in addition, the series presents seasonality, its modeling is through the smoothing of the Holt–Winters [34,35]. On the other hand, Golyandina [36,37] discussed the singular spectrum analysis (SSA), a nonparametric technique for time series analysis and forecasting, which incorporates elements of classical time series analysis, multivariate statistics, and matrix algebra. Its main objective is to decompose the original time series into a set of components: seasonal, trend, and noise.

Different proposals have been presented in conjunction with machine learning and deep learning algorithms based on this theory. One of the essential issues when analyzing time series is the forecast of new observations. Mahmoudvand [38] proposed a new parsimonious recurrent forecasting model that uses optimal coefficients on the linear combination of the recurrent SSA, and Rodrigues and Kazemi [39,40] developed an improvement in the model fitting performance of the SSA method when the time series is contaminated with outlying observations: a robust SSA (robSSA) approach, where a robust singular value decomposition (SVD) algorithm replaces the standard SVD in the second decomposition step. Furthermore, Rodrigues [41] proposed a method to first smooth out the noise in the tainted data and then calculate the dependency between the variables in the smoothed data. On the other hand, Wang [42] presented a hybrid model that integrates SSA and support machine vector regression (SVR) to forecast failure time series data. Sulandari [43] proposed modeling time series data by obtaining the forecast values with SSA, considering the Linear Recurring Formula (LRF), and then modeling the irregular component by fuzzy systems and neural networks (NNs), and Liu [44] proposed a new hybrid model for short-term prediction based on SSA, a convolutional neural network (CNN), GRU, and SVR, seeing that the SSA decomposes the original series into several components and the CNNGRU predicts the component of trend. In contrast, the SVR predicts the detailed components. Likewise, De [45] proposed the nonlinear combination method, highlighting one for the time series and the other for the residuals, so that both models are combined using a multilayer perceptron (MLP) since it is capable of performing linear and nonlinear combinations. In addition, Samal [46] identified the utility of analytical models to build a system capable of giving a rough estimate of future pollution levels within a considerable confidence interval. Various methods and techniques have been presented to improve the prediction of the concentration of air pollutants.

1.4. Comparison with Previous Studies

Traditional models, such as ARIMA and simple variants of neural networks (e.g., MLP), have limitations in capturing nonlinear and stochastic patterns in time series. Compared to studies such as Singh’s [19] that use methods such as PCA and ensemble techniques, this SSA+LSTM proposal goes further by decomposing the series temporally and addressing specific components such as noise, which other studies do not explicitly separate. Recent hybrid methods such as CNN-LSTM [6] or GC-LSTM [21] mainly focus on spatial and temporal integration to improve PM_2.5 or PM₁₀ prediction. This study stands out for the application of SSA as a robust preprocessor that improves signal decomposition and reconstruction before prediction, a strategy less explored in other studies. In works such as Sulandari et al. [43], SSA is used to predict climate variables, but usually with traditional methods (fuzzy systems or SVMs). This study excels by combining SSA with the LSTM, incorporating advanced deep learning techniques to model both linear and nonlinear components.

Likewise, Pak [6] integrates convolutional networks and LSTM neural networks to capture spatiotemporal correlations in the prediction of PM_2.5. However, it depends on spatially structured data, which can limit its applicability in areas with lower density of monitoring stations. For their part, Qi [21] used graphical convolution networks to model spatial and temporal relationships. This method is effective in regions with a high interaction of pollutants, but its complexity increases with the number of nodes (regions), which can be computationally expensive. Given this, our study presents some alternatives, such as time series preprocessing, since the decomposition of signals into trends, seasonality, and noise improves the predictive capacity by feeding the model with cleaner data. In addition, the versatility without spatial dependence, that is, it does not depend on specific spatial data, makes it more flexible to be applied in regions with fewer resources. Finally, the computational efficiency increases by separating components before using the LSTM, and the computational load is reduced compared with methods such as the GC-LSTM, which process complex data directly. Hou [23] combined ARIMA for linear trends and the MLP for nonlinear ones. However, this method depends on a strict parameterization for ARIMA, which makes it less adaptable. On the other hand, Tao [2] used wavelet-based methods that are effective for decomposing signals, but do not address specific noise modeling as in this research. Thus, our proposal is more flexible, since SSA does not require parametric assumptions, and the LSTM can handle more complex relationships than MLP or wavelet-based methods.

1.5. Innovation Points of This Study

This research seeks to consider a new hybrid method that considers the nonlinear interaction of temporal pollution patterns and meteorological variables to improve air quality prediction. This hybrid method was adapted using a nonparametric statistical algorithm and a deep learning approach. This adaptation involves the singular spectrum analysis responsible for separating the trend and seasonality components, i.e., the signal, from the noise components, using the linear recurrent algorithm to forecast the signal. For its part, the long short-term memory neural network will be responsible for addressing the residual part, which is not white noise and presents a stochastic behavior that is not captured in the signal through SSA. In general, the prediction integrates a sum of both what was predicted by the signal component and predicted by the noise component. It is demonstrated how advanced preprocessing methods (SSA) can significantly improve the predictive capacity of neural networks in nonlinear and noisy domains. Accurate air quality prediction is critical in urban contexts to design public policies and risk mitigation strategies. In this sense, this study also proposes the hybrid model as a potential to be implemented in real-time monitoring systems to improve early warning on air pollution.

2. Theoretical Foundation

2.1. Singular Spectrum Analysis

The singular spectrum analysis algorithm can be written in two complementary stages, decomposition and reconstruction of the univariate time series [36,37], followed by a forecasting stage.

2.1.1. First Stage: Decomposition

First step: Embedding. Let $y_{1}, \dots, y_{N}$ be a time series of length N. Considering a window length L, the result of this step is a $L \times K$ matrix $Y = [Y_{1} : \dots : Y_{K}]$ , where $K = N - L + 1$ and $Y_{i} = {(y_{i}, \dots, y_{i + L - 1})}^{T}, 1 \leq i \leq K$ .
Second step: Singular value decomposition (SVD). In this step, the matrix $Y$ will be decomposed using SVD as $Y = Y_{1} + \dots + Y_{L}$ , where $Y_{i} = {\sqrt{λ}}_{i} U_{i} {V_{i}}^{T}$ , $Y_{i} = 0$ when $λ_{i} = 0$ , and $V_{i} = Y^{T} U_{i} / {\sqrt{λ}}_{i}$ with $λ_{1}, \dots, λ_{L},$ and the eigenvalues of $S = Y Y^{T}$ and $U_{1}, \dots, U_{L},$ are the corresponding eigenvectors.

2.1.2. Second Stage: Reconstruction

Third step: Grouping. The grouping step corresponds to splitting the elementary matrices into m disjunct subsets $I_{1}, \dots, I_{m}$ and summing the matrices within each group. In our application we will focus on $m = 2$ , i.e., only two groups. $I_{1} = {1, \dots, r}$ and $I_{2} = {r + 1, \dots, L}$ which are associated with the signal and noise components, respectively.
Fourth step: Diagonal averaging. This step transforms each matrix $Y_{I_{j}}$ into a new series of length N. Using diagonal averaging, $Y = {\tilde{Y}}_{I_{1}} + \dots + {\tilde{Y}}_{I_{m}}$ , where ${\tilde{Y}}_{I_{j}}$ is the Hankelized form of $Y_{I_{j}}$ , $j = 1, \dots, m$ . Considering ${\tilde{y}}_{m, n}^{(I_{j})}$ the ${(m, n)}^{t h}$ entry of the estimated matrix ${\tilde{Y}}_{I_{j}}$ and denoting by $\{{\tilde{y}}_{j_{1}}, \dots, {\tilde{y}}_{j_{N}}\}$ , the reconstructed components in the matrix ${\tilde{Y}}_{I_{j}}$ , $j = 1, \dots, m,$ applying diagonal averaging, it follows that

${\tilde{y}}_{j_{l}} = \{\begin{matrix} \frac{1}{j_{ℓ} - 1} \sum_{n = 1}^{j_{ℓ} - 1} {\tilde{y}}_{n, j_{ℓ} - n}^{(I_{j})} & 2 \leq j_{ℓ} \leq L - 1, \\ \frac{1}{L} \sum_{n = 1}^{L} {\tilde{y}}_{n, j_{ℓ} - n}^{(I_{j})} & L \leq j_{ℓ} \leq K + 1, \\ \frac{1}{K + L - j_{ℓ} + 1} \sum_{n = n - K}^{L} {\tilde{y}}_{n, j_{ℓ} - n}^{(I_{j})} & K + 2 \leq j_{ℓ} \leq K + L . \end{matrix}$

(1)

2.1.3. Third Stage: Forecasting

Although there are several versions of SSA forecasting algorithms (e.g., [36,47]), we consider here the recurrent SSA forecasting algorithm, one of the most widely used.

Let

U_{j}^{▽}

be the vector of the first

L - 1

components of

U_{j}

, the jth eigenvector of

Y Y^{'}

π_{j}

denote the last component of

U_{j}

j = 1, \dots, r

, and r denote the number of eigenvalues used for reconstruction. We can define the coefficient vector

\hat{a}

\hat{a} = {({\hat{a}}_{L - 1}, \dots, {\hat{a}}_{1})}^{'} = \frac{1}{1 - ν^{2}} \sum_{j = 1}^{r} π_{j} U_{j}^{▽},

(2)

where

ν^{2} = \sum_{j = 1}^{r} π_{j}^{2}

. Considering the above notation, the h steps ahead out-of-sample recurrent SSA forecasts

{\hat{y}}_{N + 1}, \dots, {\hat{y}}_{N + h}

can be obtained as

{\hat{y}}_{t} = \{\begin{matrix} {\tilde{y}}_{t} & for t = 1, \dots, N, \\ \sum_{j = 1}^{L - 1} {\hat{a}}_{j} {\hat{y}}_{t - j} & for t = N + 1, \dots, N + h, \end{matrix},

(3)

where

{\tilde{y}}_{1}, \dots, {\tilde{y}}_{N}

, are the fitted values for the reconstructed time series as obtained from Equation (1).

2.2. Long Short-Term Memory Recurrent Neural Networks

The idea of introducing good loops to produce trajectories in which the gradient can flow for long periods is a fundamental contribution to the initial LSTM model [48]. A crucial addition is that the weight of this loop itself is context-bound rather than fixed [49]. The integration time scale can be changed dynamically by making the weight of this automatic loop closed (controlled by another hidden unit). Even for an LSTM with fixed parameters, the integration time scale may change depending on the input sequence because the model generates the time constants. The LSTM has successfully performed in many applications, such as unrestricted handwriting recognition [50], speech recognition [51,52], handwriting generation by handwriting [53], machine translation [54], image captions [55,56,57], and syntactic analysis [56]. Figure 1 shows a perspective view of the LSTM. They have a link

h (t)

from one unit to the next, in addition to a cell state,

C (t)

, which is the main flow of information through the LSTM. The cell state is the long-term memory of the model. All that happens afterward is that different filters decide what should be kept or added to the cell state. Cell status is emphasized in Figure 2A. The LSTM also adds and removes information from the cell with the so-called gates, making up the rest of the unit in an LSTM. Gates are a combination of addition, multiplication, and nonlinearities. Nonlinearities are used to “squash” information. For example, the sigmoid function (sigm) is used to “squash” the information to values between 0 and 1, and the hyperbolic tangent (tanh) for values between

- 1

y 1.

The first gate, so called because it comes from the analogy with logic gates, is the forget gate, emphasized in Figure 2B. The weight of the loop itself (or the associated time constant) is controlled by a forget gate unit

f_{i}^{(t)}

(for time step t and cell i), which sets this weight to a value between 0 and 1 through a sigmoid unit. Intuitively, it controls how much of the raw input and previous hidden state will be remembered. The forget gate in the unit t is expressed as follows:

f (t) = σ (w_{f} (x (t) + h (t - 1))),

(4)

where

x (t)

is the current input vector and

h (t)

is the currently hidden layer vector containing the outputs of all LSTM cells. Regarding the weights, consider the one that divides

w_{h}

into several different weights,

w_{f}

w_{f f}

w_{C}

, and

w_{f f f}

. It is worth mentioning that each block must have its own set of weights. All the weight in a complete neural network is trained together with backpropagation, and joint training makes a neural network into a connected whole. The next gate (shown in Figure 2C), called the input gate, precisely decides what to put in the cell state. It comprises another forget gate (denoted

f f (t)

) but with different weights. It also has an additional module that creates candidates to be added to the cell state. The

f f (t)

can be thought of as a saving mechanism that controls the amount of input that we will save in the cell state. Then,

f f (t) = σ (w_{f f} (x (t) + h (t - 1))),

(5)

and

i (t) = f f (t) \cdot C^{*} (t) .

(6)

The computation for candidates is

C^{*} (t) = t a n h (w_{C} \cdot (x (t) + h (t - 1)))

. As seen above, an LSTM unit has three outputs:

C (t), y (t)

, and

h (t)

. With this, the current state of cell

C (t)

(shown in Figure 2A) can be calculated as follows:

C (t) = f (t) \cdot C (t - 1) + i (t) .

(7)

Let

y (t) = g_{o} (w_{o} \cdot h (t))

(where

g_{o}

is a nonlinear component). For its part, to calculate

h (t)

, a third copy of the forget gate

(f f f (t))

is needed, which will decide which parts of the inputs and how much to include in

h (t)

(as a “focus” mechanism that tries to tell which is the most critical part of the cell’s state). We can write

f f f (t)

f f f (t) : = σ (w_{f f f} (x (t) + h (t - 1))) .

(8)

Now, the only thing left for a complete output gate (whose result is not

o (t)

but

h (t)

, emphasized in Figure 2D), is to multiply

f f f (t)

by the current state of the cell squashed between

- 1

and 1. Then,

h (t) : = f f f (t) \cdot τ (C (t)),

(9)

This allows us to have the complete LSTM network. LSTM networks have been shown to learn long-term dependencies more quickly than simple recursive architectures, first on artificial datasets designed to test the ability to learn long-term dependencies [48,58,59], then on challenging sequence processing tasks where state-of-the-art performance was obtained [51,54].

3. SSA+LSTM Hybrid Method

The architecture of the SSA+LSTM method is represented in Figure 3. Detailed descriptions are given below.

Sequence 1. Model and forecast the time series using the SSA algorithm.
- Set the length of the window and find the trajectory matrix.
- Decompose the trajectory matrix using SVD.
- Define the two groups of separable matrices, both signal and noise.
- Reconstruct each matrix in a time series using the diagonal averaging algorithm.
- Use the recurrent forecasting algorithm to compute forecast values.
Sequence 2. Model the residuals with the LSTM neural network.
- Get the residual generated in the above sequence (SSA-RF).
- Set the neural network architecture to determine the inputs, nodes, and activation functions used in the hidden layer and the output. For univariate time series forecasting, one output unit is considered.
- Obtain the weights that connect each input node with each hidden node, and those that connect each hidden node with each output node using the backpropagation algorithm through time, the same one that allows the solving of two possible consequences related to the gradient, the tendency to infinity when the value is high, or degradation when the value is too small, due to recurring connections.
- Calculate the predicted values for the residual generated by the SSA-RF using the LSTM neural network.
Sequence 3. Forecast the time series with the hybrid method SSA+LSTM.
- Compute the final values through the sum between the predicted values obtained by the SSA-RF and the predicted residuals obtained by the LSTM network.
- Compare the RMSEs, MAEs, and MAPEs obtained from the predictive method (see Figure 4 for details).

Parameter Settings and Experimental Software Platform

In this study, different functions from both R Studio (2024.09.1+394) and Python (3.11.10) were used. For data imputation, the mice R package’s mice function was used. Also, the MinMaxScaler function from sklearn.preprocessing was used to scale the data. Subsequently, the pytorch, typical of Python, was used to support all the machine learning algorithms. The ssa function of the Rssa R package was also used to model the signal after signal and noise decomposition. On the other hand, the model parameters of the ARIMA model were estimated with the auto.arima function of the R package forecast. In this scenario, the parameters configured for the LSTM and MLP are detailed as follows: hidden.layer.size=10, dropout.prob=0.1, epochs equal to 150 and optimizer=torch.optim.Adam.

4. Results and Discussion

This section presents the statistical analysis on the original time series of PM₁₀ concentration (see Figure 5). Likewise, the evaluation of the models is included. The dataset comprises two years (from 1 January 2017 to 31 December 2018) and includes PM₁₀ (

μ

g/m³) concentrations for the year, month, day, and hour. These data were obtained from the Servicio Nacional de Meteorología e Hidrología website of Peru, which provides air quality data for the entire country.

Figure 6 presents the singular values or eigenvalues of the matrix X. The singular values were obtained through the singular value decomposition in step two of the decomposition stage, and their behaviors can be observed through the spectrum of the singular values of the trajectory matrix. One of the essential characteristics to observe is that the first singular value, the one with the highest absolute value, corresponds to the trend component of the series. Other helpful information is provided by checking for breaks in the singular value spectrum. In general terms, a purely residual noise series produces a slowly decreasing sequence of singular values. Since a slight and continuous decrease of the singular values is visualized, it can characterize the beginning of the noise of the time series. It is a way to validate the importance of each component. The main idea is to choose components that allow for better separability between signal and noise. In this case, we consider the first 14 components for the signal and the remainder for the noise.

The SSA calibration depends on two parameters: the window length L, and the number of eigentriples used for reconstruction r. The choice of improper values for the parameters L or r yields incomplete reconstruction, and the forecasting results might be misleading [38,60]. Although the choice of these parameters is of crucial importance, no theoretical solution has been proposed for their choosing. An overall agreeable suggestion to choose the window length is that it be close to the middle of the time series and proportional to the number of observations per period (e.g., to 12 for monthly time series). However, this choice does not guarantee the best predictions [38,60] for making a parameter choice according to the available data and intended analysis.

Note that the first singular value (eigenvalue of most significant magnitude, rank 1) corresponds to the trend component of the series. Once this trend component has been identified, the paired periodic components are determined. In this way, in the spectrum of the singular values (Figure 6), the periodic paired components composed, mainly by 4–5 and 6–7, are observed, which constitute the seasonality pattern of the original series. From rank 15 onwards, a slight and continuous decrease of the singular values is displayed. Also, there is a break in Figure 6 (component 14); this establishes the boundary between signal and noise. In this context, Figure 7 complements what is observed in Figure 6, since it demonstrates the similarity of components 4, 5, and 6, 7 are also complemented by Figure 8 (seasonality, since the phenomena that occur during a time are repeated in each identical time frame; that is, phenomena that occur daily at a particular time, because the scale is hourly). On the contrary, the rest of the reports are considered trends.

For its part, Figure 9 reports the weighted correlation derived from the concept of separability. This is linked to how to choose parameters to build a suitable decomposition of the time series. Furthermore, it characterizes how well the different components can be separated from each other. Finally, it is worth mentioning that the decomposition of the time series will only be successful if the resulting additive components of the series are separable from each other. In this way, the singular value decomposition and eigentriple grouping steps rely on this property called separability. According to [36], in practice, there is no exact separability, only approximate ones. Typically, separability quality is assessed through a natural measure of dependency between the substrings. This measure is called weighted correlation or W–correlation (Figure 9). This plot can be used to check the quality of separability. In turn, it is considered a grouping technique of the components resulting from SVD. What the SSA seeks is to decompose the original series into a sum of series that allows each component to be identified. These components can be observed in Figure 9. This report is based on separability, which allows the choosing of parameters to build an adequate decomposition of the series. SVD and eigentriple grouping rely on this concept [36,61].

The components of the series can be differentiated as trend, seasonality, and residuals. This allows for evaluating the grouping of seasonalities (Figure 10) according to the previously reported components. On the other hand, in Figure 11, it is observed that this report clusters the different seasonalities seen in Figure 10. The same components of the time series are observed in a classified manner. This is because the seasonalities can be grouped by similarity, only defining the original series, the trend, seasonality, and residuals. Figure 12 shows the integration of the original time series and the two fundamental components of this process: signal and residuals. The signal will obviously be used within the recurrent forecasting, and the noise will enter the LSTM (both observed in Figure 13). In this context, it is emphasized that the reported noise is not white noise. It is a stochastic behavior not captured in the signal using singular spectrum analysis. The critical episodes in the time series become complex components for the SSA to capture.

4.1. Accuracy Measures

In order to evaluate the predictive performance of the different air pollution concentration prediction methods, three performance metrics were used in the study, which are described in Equations (10)–(12). The Mean Absolute Percentage Error (MAPE) and the Mean Absolute Error (MAE) can be used to measure the distance between the forecast data and the actual data; the Root Mean Squared Error (RMSE) is helpful in showing more significant deviations.

\begin{matrix} RMSE & = \sqrt{\frac{1}{N} \sum_{i = 1}^{N} {(y_{i} - {\hat{y}}_{i})}^{2}} \end{matrix}

(10)

\begin{matrix} MAE & = \frac{1}{N} \sum_{i = 1}^{N} |y_{i} - {\hat{y}}_{i}| \end{matrix}

(11)

\begin{matrix} MAPE & = \frac{1}{N} \sum_{i = 1}^{N} |\frac{y_{i} - {\hat{y}}_{i}}{y_{i}}| \times 100 \end{matrix}

(12)

where

y_{i}

is the actual value and

{\hat{y}}_{i}

is the prediction derived from the forecast model and N is the number of predictions of air pollution concentrations.

4.2. Predictionwith Hybrid Method SSA+LSTM

The performances of the hybrid approach and those used as reference are summarized in Table 1. As can be seen, the

{SSA}_{signal}

{LSTM}_{noise}

method achieved better results compared with the other methods. This highlights the importance of considering the integration of residual modeling using the LSTM network when forecasting PM₁₀ concentrations. With a larger prediction amplitude, the errors in all the models increased; however, this was at a considerably slower rate for the predictive method. This reveals the prediction capacity of the hybrid method in different periods (short, medium, and long).

To further verify the effectiveness of the SSA+LSTM forecasting method, several popular air pollution forecasting methods, such as the MLP, LSTM, and MPL with signal and noise, and the LSTM with signal and noise, were considered for the forecasting performance comparison. Table 1 shows the RMSE, MAE, and MAPE of different forecasting methods (see also Figure 14a–c). It can be seen from Table 1 that (1) SSA+LSTM has the highest forecasting accuracy than other methods. (2) MLP and MLP with signal and noise have larger RMSE, MAE, and RMSE values in both one-step ahead and multi-step ahead predictions. (3) The proximity of the RMSE, MAE, and RMSE values for the method from the forecast for 6 h ahead is due to one of the properties of the LSTM network related to data memory; that is, it can learn long-term sequences of PM₁₀ concentrations.

Figure 15 shows the behavior of the hybrid method. The relationship between the observed PM₁₀ values (represented by the black lines) and the predictions generated by the proposed hybrid method is illustrated, which offered, according to the performance metrics, the best performance. This proposed method consistently captures the general trends of the time series, reflecting the ability of the approach to model both seasonal patterns and fluctuations. Although some deviations could be observed at specific points, the proximity between the lines indicates a high prediction accuracy, highlighting the effectiveness of the SSA+LSTM method in addressing the inherent complexity of PM₁₀ ambient pollution data. This performance suggests that the model is robust in accurately predicting PM₁₀ concentrations, which can be crucial for environmental monitoring and management.

5. Conclusions and Future Scope

In this study, ten competing methods are presented. Seven of these adopt spectrum analysis components for air pollution forecasting for 1 h, 6 h, 12 h, and 24 h ahead. The methods were evaluated with the PM₁₀ hourly time series between 2017 and 2019. The SSA preprocesses the data to extract the time series trend, seasonal, and noise components. Once the components were extracted, they were predicted with seven methods: multilayer perceptron with signal, multilayer perceptron with signal + multilayer perceptron with noise, long short-term memory with signal, long short-term memory with signal + long short-term memory with noise, singular spectrum analysis (signal only), singular spectrum analysis with signal + multilayer perceptron with noise, and finally, singular spectrum analysis with signal + long short-term memory with noise.

The hybrid method showed high precision in predicting PM₁₀ and superiority concerning the individual methods (without preprocessing). Likewise, this method derived some variations based on SSA, which were superior to the rest of the methods presented with a conventional window length. The high gain achieved with this hybrid method lies in the SSA, which deals with the parts of trend and seasonality (linear part), and the LSTM network takes care of the nonlinear part (noise), having as its main virtue long-term dependence. This is observed not only an hour ahead but also six, twelve, and twenty-four hours ahead of the forecast. The evaluation through the performance metrics confirms the superiority of the hybrid method singular spectrum analysis plus long short-term memory over the individual methods.

The increasing availability of historical data and computational resources has facilitated the development of better-performing methods for air pollution prediction. In this study, a hybrid method based on nonparametric statistics and deep artificial neural networks for air pollution concentration forecasting was developed. This method is based on an integration of singular spectrum analysis and long short-term memory neural networks to take into account the modeling of the residuals extracted from its matrix decomposition and to evaluate it by adhering to the signal component in the prediction model. The hybrid method was first trained to confirm network parameters and then justified by verification with test sample data. The experimental results showed that the predictive method outperformed the other methods in predicting PM₁₀ values. The performance advantage of the method is consistent for both the prediction and the error minimum fluctuation trend. With RMSE, MAE, and MAPE in the prediction of 1 h, 6 h, 12 h, and 24 h, it is verified that the hybrid method is an effective statistical method to estimate the concentration of PM₁₀. Due to the promising results, this method will be extrapolated and evaluated with other time series related to air pollution and other areas of knowledge. As future research complements this developed predictive approach, addressing classification approaches and simultaneously developing other hybrid classification methods for air pollution concentrations have been considered. We can also develop a proposal that includes spatial covariates in the hybrid method and likewise create prediction intervals for all the methods addressed in this investigation. Finally, using the multivariate SSA followed by the LSTM, we can consider multivariate series. Indeed, there is a multivariate version of the singular spectrum analysis (multivariate SSA) that can be used in the first part of the hybrid methodology. However, its usefulness, compared with the univariate SSA, is highly dependent on the “correlation/relation” between the variables [47]. The assessment of this “correlation/relation” can be complex to find as we are dealing with time series, and measures such as dynamic time warping are more appropriate than the standard Pearson correlation. In future work, we will work on a deep study where these correlations between time series are considered and incorporated into the multivariate SSA to improve the results in a multivariate hybrid methodology. The potential of the SSA+LSTM model to adapt to different regions and pollutants positions it as a promising solution for air quality management and monitoring, and can also be extended to other scenarios with different datasets [62,63,64].

Author Contributions

Conceptualization, J.L.L.-G. and P.C.R.; methodology, J.L.L.-G. and P.C.R.; software, J.L.L.-G. and P.C.R.; validation, R.S., D.V. and P.C.R.; formal analysis, J.L.L.-G. and P.C.R.; investigation, J.L.L.-G.; resources, J.L.L.-G. and P.C.R.; data curation, J.L.L.-G. and P.C.R.; writing—original draft preparation, J.L.L.-G., R.S., D.V. and P.C.R.; writing—review and editing, J.L.L.-G., R.S., D.V. and P.C.R.; visualization, J.L.L.-G. and P.C.R.; supervision, P.C.R. and R.S.; project administration, J.L.L.-G.; funding acquisition, R.S. and P.C.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by ANID ANILLO “One Health Network: Dengue and Emerging medically Important Arboviruses (DEMIV-OH)” grant number ATE 220020. P.C. Rodrigues acknowledges financial support from the CNPq grant “bolsa de produtividade PQ-2” 309359/2022-8, Federal University of Bahia and CAPES-PRINT-UFBA, under the topic Modelos Matemáticos, Estatísticos e Computacionais Aplicados às Ciências da Natureza. We also acknowledge the Universidad Peruana Unión.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

The datasets are available in the repository https://www.senamhi.gob.pe/site/descarga-datos/, accessed on 15 January 2023.

Conflicts of Interest

The authors declare no conflicts of interest.

References

Wang, P.; Zhang, G.; Chen, F.; He, Y. A hybrid-wavelet model applied for forecasting PM_2.5 concentrations in Taiyuan city, China. Atmos. Pollut. Res. 2019, 10, 1884–1894. [Google Scholar] [CrossRef]
Tao, Q.; Liu, F.; Li, Y.; Sidorov, D. Air pollution forecasting using a deep learning model based on 1D convnets and bidirectional GRU. IEEE Access 2019, 7, 76690–76698. [Google Scholar] [CrossRef]
Ahn, J.; Shin, D.; Kim, K.; Yang, J. Indoor air quality analysis using deep learning with sensor data. Sensors 2017, 17, 2476. [Google Scholar] [CrossRef] [PubMed]
Athira, V.; Geetha, P.; Vinayakumar, R.; Soman, K. DeepAirNet: Applying Recurrent Networks for Air Quality Prediction. Procedia Comput. Sci. 2018, 132, 1394–1403. [Google Scholar] [CrossRef]
Ong, B.T.; Sugiura, K.; Zettsu, K. Dynamic pre-training of deep recurrent neural networks for predicting environmental monitoring data. In Proceedings of the 2014 IEEE International Conference on Big Data (Big Data), Washington, DC, USA, 27–30 October 2014; pp. 760–765. [Google Scholar]
Pak, U.; Kim, C.; Ryu, U.; Sok, K.; Pak, S. A hybrid model based on convolutional neural networks and long short-term memory for ozone concentration prediction. Air Qual. Atmos. Health 2018, 11, 883–895. [Google Scholar] [CrossRef]
Salman, A.G.; Heryadi, Y.; Abdurahman, E.; Suparta, W. Single layer & multi-layer long short-term memory (LSTM) model with intermediate variables for weather forecasting. Procedia Comput. Sci. 2018, 135, 89–98. [Google Scholar]
Tsai, Y.-T.; Zeng, Y.-R.; Chang, Y.-S. Air pollution forecasting using RNN with LSTM. In Proceedings of the 2018 IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), Athens, Greece, 12–15 August 2018; pp. 1074–1079. [Google Scholar]
Zhang, C.Y.; Chen, C.P.; Gan, M.; Chen, L. Predictive deep Boltzmann machine for multiperiod wind speed forecasting. IEEE Trans. Sustain. Energy 2015, 6, 1416–1425. [Google Scholar] [CrossRef]
Pak, U.; Ma, J.; Ryu, U.; Ryom, K.; Juhyok, U.; Pak, K.; Pak, C. Deep learning-based PM_2.5 prediction considering the spatiotemporal correlations: A case study of Beijing, China. Sci. Total Environ. 2020, 699, 133561. [Google Scholar] [CrossRef]
Li, T.; Shen, H.; Yuan, Q.; Zhang, X.; Zhang, L. Estimating ground-level PM_2.5 by fusing satellite and station observations: A geo-intelligent deep learning approach. Geophys. Res. Lett. 2017, 44, 11985–11993. [Google Scholar] [CrossRef]
Li, X.; Peng, L.; Yao, X.; Cui, S.; Hu, Y.; You, C.; Chi, T. Long short-term memory neural network for air pollutant concentration predictions: Method development and evaluation. Environ. Pollut. 2017, 231, 997–1004. [Google Scholar] [CrossRef]
Soh, P.W.; Chang, J.W.; Huang, J.W. Adaptive deep learning-based air quality prediction model using the most relevant spatial-temporal relations. IEEE Access 2018, 6, 38186–38199. [Google Scholar] [CrossRef]
Dietterich, T.G. Ensemble methods in machine learning. In Multiple Classifier Systems, Proceedings of the First International Workshop on Multiple Classifier Systems, Cagliari, Italy, 21–23 June 2000; Springer: Berlin/Heidelberg, Germany, 2000; pp. 1–15. [Google Scholar]
Kazienko, P.; Lughofer, E.; Trawiński, B. Hybrid and ensemble methods in machine learning J. UCS special issue. J. Univers. Comput. Sci. 2013, 19, 457–461. [Google Scholar]
Qu, Y.; Quan, P.; Lei, M.; Shi, Y. Review of bankruptcy prediction using machine learning and deep learning techniques. Procedia Comput. Sci. 2019, 162, 895–899. [Google Scholar] [CrossRef]
Allende, H.; Moraga, C.; Ñanculef, R.; Salas, R. Ensembles Methods for Machine Learning Pattern Recognition and Machine Vision; River Publisher Series in Information Science and Technology; River Publisher: Aalborg, Denmark, 2010; pp. 247–261. [Google Scholar]
Singh, H.; Rana, P.S.; Singh, U. Prediction of drug synergy in cancer using ensemble-based machine learning techniques. Mod. Phys. Lett. B 2018, 32, 1850132. [Google Scholar] [CrossRef]
Singh, K.P.; Gupta, S.; Rai, P. Identifying pollution sources and predicting urban air quality using ensemble learning methods. Atmos. Environ. 2013, 80, 426–437. [Google Scholar] [CrossRef]
Xayasouk, T.; Lee, H.; Lee, G. Air Pollution Prediction Using Long Short-Term Memory (LSTM) and Deep Autoencoder (DAE) Models. Sustainability 2020, 12, 2570. [Google Scholar] [CrossRef]
Qi, Y.; Li, Q.; Karimian, H.; Liu, D. A hybrid model for spatiotemporal forecasting of PM_2.5 based on graph convolutional neural network and long short-term memory. Sci. Total Environ. 2019, 664, 1–10. [Google Scholar] [CrossRef]
Qiao, J.; He, Z.; Du, S. Prediction of PM_2.5 concentration based on multi-source data and self-organizing fuzzy neural network. SN Appl. Sci. 2020, 2, 612. [Google Scholar] [CrossRef]
Hou, Q.; Leng, J.; Ma, G.; Liu, W.; Cheng, Y. An adaptive hybrid model for short-term urban traffic flow prediction. Phys. A Stat. Mech. Its Appl. 2019, 527, 121065. [Google Scholar] [CrossRef]
Díaz-Robles, L.A.; Ortega, J.C.; Fu, J.S.; Reed, G.D.; Chow, J.C.; Watson, J.G.; Moncada-Herrera, J.A. A hybrid ARIMA and artificial neural networks model to forecast particulate matter in urban areas: The case of Temuco, Chile. Atmos. Environ. 2008, 42, 8331–8340. [Google Scholar] [CrossRef]
Du, P.; Wang, J.; Yang, W.; Niu, T. A novel hybrid model for short-term wind power forecasting. Appl. Soft Comput. 2019, 80, 93–106. [Google Scholar] [CrossRef]
Albalawi, F.; Chahid, A.; Guo, X.; Albaradei, S.; Magana-Mora, A.; Jankovic, B.R.; Uludag, M.; Van Neste, C.; Essack, M.; Laleg-Kirati, T.M.; et al. Hybrid model for efficient prediction of poly (A) signals in human genomic DNA. Methods 2019, 166, 31–39. [Google Scholar] [CrossRef]
Siwek, K.; Osowski, S. Improving the accuracy of prediction of PM₁₀ pollution by the wavelet transformation and an ensemble of neural predictors. Eng. Appl. Artif. Intell. 2012, 25, 1246–1258. [Google Scholar] [CrossRef]
Liu, X.; Tan, W.; Tang, S. A Bagging-GBDT ensemble learning model for city air pollutant concentration prediction. IOP Conf. Ser. Earth Environ. Sci. 2019, 237, 022027. [Google Scholar] [CrossRef]
Neto, P.S.D.M.; Firmino, P.R.A.; Siqueira, H.; Tadano, Y.D.S.; Alves, T.A.; De Oliveira, J.F.L.; Marinho, M.H.D.N.; Madeiro, F. Neural-based ensembles for particulate matter forecasting. IEEE Access 2021, 9, 14470–14490. [Google Scholar] [CrossRef]
Li, X.; Luo, A.; Li, J.; Li, Y. Air pollutant concentration forecast based on support vector regression and quantum-behaved particle swarm optimization. Environ. Model. Assess. 2019, 24, 205–222. [Google Scholar] [CrossRef]
de Mattos Neto, P.S.; Cavalcanti, G.D.; Madeiro, F.; Ferreira, T.A. An approach to improve the performance of PM forecasters. PLoS ONE 2015, 10, e0138507. [Google Scholar] [CrossRef]
Maia, A.L.S.; de Carvalho, F.d.A. Holt’s exponential smoothing and neural network models for forecasting interval-valued time series. Int. J. Forecast. 2011, 27, 740–759. [Google Scholar] [CrossRef]
Gelper, S.; Fried, R.; Croux, C. Robust forecasting with exponential and Holt–Winters smoothing. J. Forecast. 2010, 29, 285–300. [Google Scholar] [CrossRef]
Chatfield, C.; Yar, M. Holt-Winters forecasting: Some practical issues. J. R. Stat. Soc. Ser. D (Stat.) 1988, 37, 129–140. [Google Scholar] [CrossRef]
Kalekar, P.S. Time series forecasting using holt-winters exponential smoothing. Kanwal Rekhi Sch. Inf. Technol. 2004, 4329008, 1–13. [Google Scholar]
Golyandina, N.; Nekrutkin, V.; Zhigljavsky, A.A. Analysis of Time Series Structure: SSA and Related Techniques; Chapman and Hall/CRC: New York, NY, USA, 2001; 320p. [Google Scholar]
Golyandina, N.; Zhigljavsky, A. Singular Spectrum Analysis for Time Series; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2013. [Google Scholar]
Mahmoudvand, R.; Rodrigues, P.C. A new parsimonious recurrent forecasting model in singular spectrum analysis. J. Forecast. 2018, 37, 191–200. [Google Scholar] [CrossRef]
Rodrigues, P.C.; Lourenço, V.; Mahmoudvand, R. A robust approach to singular spectrum analysis. Qual. Reliab. Eng. Int. 2018, 34, 1437–1447. [Google Scholar] [CrossRef]
Kazemi, M.; Rodrigues, P.C. Robust singular spectrum analysis: Comparison between classical and robust approaches for model fit and forecasting. Comput. Stat. 2023, 1–33. [Google Scholar] [CrossRef]
Rodrigues, P.C.; Mahmoudvand, R. Correlation analysis in contaminated data by singular spectrum analysis. Qual. Reliab. Eng. Int. 2016, 32, 2127–2137. [Google Scholar] [CrossRef]
Wang, X.; Wu, J.; Liu, C.; Wang, S.; Niu, W. A hybrid model based on singular spectrum analysis and support vector machines regression for failure time series prediction. Qual. Reliab. Eng. Int. 2016, 32, 2717–2738. [Google Scholar] [CrossRef]
Sulandari, W.; Lee, M.H.; Rodrigues, P.C. Indonesian electricity load forecasting using singular spectrum analysis, fuzzy systems and neural networks. Energy 2020, 190, 116408. [Google Scholar] [CrossRef]
Liu, H.; Mi, X.; Li, Y.; Duan, Z.; Xu, Y. Smart wind speed deep learning based multi-step forecasting model using singular spectrum analysis, convolutional Gated Recurrent Unit network and Support Vector Regression. Renew. Energy 2019, 143, 842–854. [Google Scholar] [CrossRef]
de Mattos Neto, P.S.; Cavalcanti, G.D.; Madeiro, F. Nonlinear combination method of forecasters applied to PM time series. Pattern Recognit. Lett. 2017, 95, 65–72. [Google Scholar] [CrossRef]
Samal, K.K.R.; Babu, K.S.; Das, S.K.; Acharaya, A. Time series based air pollution forecasting using SARIMA and prophet model. In Proceedings of the 2019 International Conference on Information Technology and Computer Communications, Singapore, 16–18 August 2019; pp. 80–85. [Google Scholar]
Rodrigues, P.C.; Mahmoudvand, R. The benefits of multivariate singular spectrum analysis over the univariate version. J. Frankl. Inst. 2018, 355, 544–564. [Google Scholar] [CrossRef]
Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef] [PubMed]
Gers, F.A.; Schmidhuber, J.; Cummins, F. Learning to forget: Continual prediction with LSTM. Neural Comput. 2000, 12, 2451–2471. [Google Scholar] [CrossRef]
Graves, A.; Liwicki, M.; Fernández, S.; Bertolami, R.; Bunke, H.; Schmidhuber, J. A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2008, 31, 855–868. [Google Scholar] [CrossRef]
Graves, A.; Mohamed, A.r.; Hinton, G. Speech recognition with deep recurrent neural networks. In Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, 26–31 May 2013; pp. 6645–6649. [Google Scholar]
Graves, A.; Jaitly, N. Towards end-to-end speech recognition with recurrent neural networks. In Proceedings of the International Conference on Machine Learning, PMLR, Beijing, China, 22–24 June 2014; pp. 1764–1772. [Google Scholar]
Graves, A. Generating sequences with recurrent neural networks. arXiv 2013, arXiv:1308.0850. [Google Scholar]
Sutskever, I.; Vinyals, O.; Le, Q.V. Sequence to sequence learning with neural networks. In Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; pp. 3104–3112. [Google Scholar]
Kiros, R.; Salakhutdinov, R.; Zemel, R.S. Unifying visual-semantic embeddings with multimodal neural language models. arXiv 2014, arXiv:1411.2539. [Google Scholar]
Vinyals, O.; Kaiser, Ł.; Koo, T.; Petrov, S.; Sutskever, I.; Hinton, G. Grammar as a foreign language. Adv. Neural Inf. Process. Syst. 2015, 28, 2773–2781. [Google Scholar]
Xu, K.; Ba, J.; Kiros, R.; Cho, K.; Courville, A.; Salakhudinov, R.; Zemel, R.; Bengio, Y. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the International Conference on Machine Learning, PMLR, Lille, France, 6–11 July 2015; pp. 2048–2057. [Google Scholar]
Bengio, Y.; Simard, P.; Frasconi, P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 1994, 5, 157–166. [Google Scholar] [CrossRef] [PubMed]
Hochreiter, S.; Younger, A.S.; Conwell, P.R. Learning to learn using gradient descent. In Artificial Neural Networks—ICANN 2001, Proceedings of the International Conference on Artificial Neural Networks, Vienna, Austria, 21–25 August 2001; Springer: Berlin/Heidelberg, Germany, 2001; pp. 87–94. [Google Scholar]
Rodrigues, P.C.; Mahmoudvand, R. A new approach for the vector forecast algorithm in singular spectrum analysis. Commun. Stat.-Simul. Comput. 2020, 49, 591–605. [Google Scholar] [CrossRef]
Golyandina, N. On the choice of parameters in singular spectrum analysis and related subspace-based methods. arXiv 2010, arXiv:1005.4374. [Google Scholar] [CrossRef]
Iftikhar, H.; Turpo-Chaparro, J.E.; Canas Rodrigues, P.; López-Gonzales, J.L. Day-Ahead Electricity Demand Forecasting Using a Novel Decomposition Combination Method. Energies 2023, 16, 6675. [Google Scholar] [CrossRef]
López-Gonzales, J.L.; Castro Souza, R.; Leite Coelho da Silva, F.; Carbo-Bustinza, N.; Ibacache-Pulgar, G.; Calili, R.F. Simulation of the energy efficiency auction prices via the markov chain monte carlo method. Energies 2020, 13, 4544. [Google Scholar] [CrossRef]
Carbo-Bustinza, N.; Iftikhar, H.; Belmonte, M.; Cabello-Torres, R.J.; De La Cruz, A.R.H.; López-Gonzales, J.L. Short-term forecasting of Ozone concentration in metropolitan Lima using hybrid combinations of time series models. Appl. Sci. 2023, 13, 10514. [Google Scholar] [CrossRef]

Figure 1. Long short-term memory network. Each neuron has a memory cell and three gates: input, output, and forget. The function of the gates is to safeguard information by stopping or allowing its flow. The input gate determines how much information from the previous layer is stored in the cell. The output gate determines how much the next layer knows about the cell. For its part, the forgetting gate decides whether to keep or forget the information. The LSTM can learn complex sequences.

Figure 2. Network long short-term memory detailed. (A): cell state; (B): forget gate; (C): input gate; and (D): output gate.

Figure 3. A time series in (A). It is executed according to the singular spectrum analysis mechanism (B). The residuals are captured and treated according to the characteristics of the long short-term memory network. Finally in (C), the prediction is made by adopting properties of both SSA and the LSTM.

Figure 4. Prediction horizons. Data were normalized within the interval [0, 1]. Then, they were divided into two matrices: training and test. The models were trained with

N - 48 + 1

data and tested with the last 48 data. For the time series, 10 simulations were carried out with the adapted approach (singular spectrum analysis and long short-term memory), and the model with the best performance was selected. The results for all models correspond to the prediction of one, six, twelve, and twenty-four steps ahead of the test set. Likewise, Table 1 shows the three metrics that evaluated the performance of the adapted methodology.

Figure 4. Prediction horizons. Data were normalized within the interval [0, 1]. Then, they were divided into two matrices: training and test. The models were trained with

N - 48 + 1

Figure 5. Time series air pollution and the description of the behavior of the contaminant through the histogram.

Figure 6. Eigenvalues—SSA.

Figure 7. Eigenvectors—SSA. The first 7 components have useful information and the remaining components can be considered noise.

Figure 8. Pairs of eigenvectors—SSA. It is mainly observed that with the pair 6 (0.42%) vs. 7 (0.42%), the size of the seasonality is predicted since it can be differentiated that there is a seasonality of 12 by the number of sides reported by the figure. This is summarized here, in that the pairs 4 (0.69%) vs. 7 (0.69%) and 6 (0.42%) vs. 7 (0.42%) are components that are related to each other; that is, they are close to each other.

Figure 9. Correlation matrix—SSA. As observed, the diagonal indicates the autocorrelation of each component; in this case, there are 100 components. For example, the black squares represent a w-correlation of 1. Focusing on components 4, 5, 6, and 7, a high correlation is observed, the same as was observed in previous reports. They were similar in terms of magnitude and behavior. This similarity means that they can be grouped together. This report aims to maximize the correlations within the signal and within the noise, and minimize the correlations between the signal and the noise.

Figure 10. Reconstructed series—SSA. Extraction of information from the time series: trend, seasonality, and residuals.

Figure 11. Reconstructed series II—SSA. Clusters of the different seasonalities.

Figure 12. Reconstructed series III—SSA. Here, the original series, the signal, which describes its behavior based on the composition of both trend and seasonality, and finally, the noise (residuals), are integrated.

Figure 13. Signal and noise components after the SSA decomposition. The signal includes the deterministic part of the series. The noise includes the stochastic unstructured residuals of the series, which is not white noise.

Figure 14. Accuracy measures (a) RMSE, (b) MAPE, and (c) MAE for all methods.

Figure 15. Behavior of the hybrid model between the observed PM₁₀ values (represented by the black lines) and the predictions’ generated red lines).

Table 1. RMSE, MAE, and MAPE for all considered methods for 1 h, 6 h, 12 h, and 24 h ahead of out-of-sample forecasts. These values are based on a 10-fold cross-validation experiment.

Time	Method	$RMSE$	$MAE$	$MAPE$
$+ 1 h$	ARIMA	$6.10$	$20.50$	$39.43$
	${MLP}_{orig}$	$5.50$	$19.81$	$38.90$
	${MLP}_{sig}$	$4.97$	$14.93$	$25.95$
	${MLP}_{sig}$ + ${MLP}_{noi}$	$4.11$	$13.40$	$25.40$
	${LSTM}_{orig}$	$4.79$	$15.17$	$29.05$
	${LSTM}_{sig}$	$4.38$	$14.88$	$24.70$
	${LSTM}_{sig}$ + ${LSTM}_{noi}$	$4.03$	$12.84$	$24.09$
	SSA	$4.41$	$11.21$	$21.52$
	${SSA}_{sig}$ + ${MLP}_{noi}$	$3.56$	$10.42$	$22.74$
	${SSA}_{sig}$ + ${LSTM}_{noi}$	$3.04$	$9.98$	$20.74$
$+ 6 h$	ARIMA	$7.02$	$21.3$	$40.32$
	${MLP}_{orig}$	$6.91$	$21.15$	$50.99$
	${MLP}_{sig}$	$5.63$	$15.89$	$31.71$
	${MLP}_{sig}$ + ${MLP}_{noi}$	$4.69$	$15.55$	$29.11$
	${LSTM}_{orig}$	$5.56$	$20.12$	$35.18$
	${LSTM}_{sig}$	$5.02$	$15.29$	$29.89$
	${LSTM}_{sig}$ + ${LSTM}_{noi}$	$4.83$	$15.07$	$29.44$
	SSA	$7.33$	$14.57$	$26.48$
	${SSA}_{sig}$ + ${MLP}_{noi}$	$4.58$	$13.14$	$26.21$
	${SSA}_{sig}$ + ${LSTM}_{noi}$	$4.02$	$12.07$	$24.69$
$+ 12 h$	ARIMA	$8.20$	$24.97$	$60.58$
	${MLP}_{orig}$	$6.97$	$23.99$	$71.72$
	${MLP}_{sig}$	$6.63$	$18.99$	$33.55$
	${MLP}_{sig}$ + ${MLP}_{noi}$	$5.28$	$16.59$	$30.68$
	${LSTM}_{orig}$	$5.77$	$23.21$	$48.19$
	${LSTM}_{sig}$	$4.92$	$17.74$	$29.13$
	${LSTM}_{sig}$ + ${LSTM}_{noi}$	$4.85$	$16.88$	$29.02$
	SSA	$9.84$	$17.47$	$28.67$
	${SSA}_{sig}$ + ${MLP}_{noi}$	$4.81$	$16.17$	$28.51$
	${SSA}_{sig}$ + ${LSTM}_{noi}$	$4.62$	$14.57$	$27.05$
$+ 24 h$	ARIMA	$8.80$	$27.45$	$78.43$
	${MLP}_{orig}$	$7.49$	$26.38$	$79.57$
	${MLP}_{sig}$	$5.97$	$19.89$	$35.37$
	${MLP}_{sig}$ + ${MLP}_{noi}$	$5.77$	$19.44$	$33.02$
	${LSTM}_{orig}$	$5.88$	$24.39$	$48.31$
	${LSTM}_{sig}$	$4.99$	$18.84$	$30.12$
	${LSTM}_{sig}$ + ${LSTM}_{noi}$	$4.89$	$19.37$	$30.02$
	SSA	$12.91$	$22.74$	$33.49$
	${SSA}_{sig}$ + ${MLP}_{noi}$	$4.87$	$19.16$	$29.94$
	${SSA}_{sig}$ + ${LSTM}_{noi}$	$4.65$	$15.79$	$27.62$

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

López-Gonzales, J.L.; Salas, R.; Velandia, D.; Canas Rodrigues, P. Air Quality Prediction Based on Singular Spectrum Analysis and Artificial Neural Networks. Entropy 2024, 26, 1062. https://doi.org/10.3390/e26121062

AMA Style

López-Gonzales JL, Salas R, Velandia D, Canas Rodrigues P. Air Quality Prediction Based on Singular Spectrum Analysis and Artificial Neural Networks. Entropy. 2024; 26(12):1062. https://doi.org/10.3390/e26121062

Chicago/Turabian Style

López-Gonzales, Javier Linkolk, Rodrigo Salas, Daira Velandia, and Paulo Canas Rodrigues. 2024. "Air Quality Prediction Based on Singular Spectrum Analysis and Artificial Neural Networks" Entropy 26, no. 12: 1062. https://doi.org/10.3390/e26121062

APA Style

López-Gonzales, J. L., Salas, R., Velandia, D., & Canas Rodrigues, P. (2024). Air Quality Prediction Based on Singular Spectrum Analysis and Artificial Neural Networks. Entropy, 26(12), 1062. https://doi.org/10.3390/e26121062

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Air Quality Prediction Based on Singular Spectrum Analysis and Artificial Neural Networks

Abstract

1. Introduction

1.1. Deep Learning Approach in Air Pollution Prediction

1.2. Hybrid and Ensemble Methods

1.3. Nonparametric Statistical Methods in Time Series

1.4. Comparison with Previous Studies

1.5. Innovation Points of This Study

2. Theoretical Foundation

2.1. Singular Spectrum Analysis

2.1.1. First Stage: Decomposition

2.1.2. Second Stage: Reconstruction

2.1.3. Third Stage: Forecasting

2.2. Long Short-Term Memory Recurrent Neural Networks

3. SSA+LSTM Hybrid Method

Parameter Settings and Experimental Software Platform

4. Results and Discussion

4.1. Accuracy Measures

4.2. Predictionwith Hybrid Method SSA+LSTM

5. Conclusions and Future Scope

Author Contributions

Funding

Institutional Review Board Statement

Data Availability Statement

Conflicts of Interest

References

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI