Short-Term Wind Speed Prediction via Sample Entropy: A Hybridisation Approach against Gradient Disappearance and Explosion
<p>The time series and Q-Q plots of minutely averaged wind speed data for the CSIR (<b>a</b>), NUST (<b>b</b>), RVD (<b>c</b>), and Venda (<b>d</b>) stations. Blue lines represent QQ lines, while grey boxes indicate interquartile ranges.</p> "> Figure 1 Cont.
<p>The time series and Q-Q plots of minutely averaged wind speed data for the CSIR (<b>a</b>), NUST (<b>b</b>), RVD (<b>c</b>), and Venda (<b>d</b>) stations. Blue lines represent QQ lines, while grey boxes indicate interquartile ranges.</p> "> Figure 2
<p>Level three MODWT results for minutely averaged wind speed data for CSIR (<b>top left panel</b>), NUST (<b>top right panel</b>), Venda (<b>bottom left panel</b>) and RVD (<b>bottom right panel</b>). D1–D3 denote the detailed coefficients at different decomposition levels and A3 denotes the approximate signal of <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>Y</mi> </mrow> <mrow> <mi>t</mi> </mrow> </msub> </mrow> </semantics></math>.</p> "> Figure 3
<p>A typical NNAR (<span class="html-italic">p, k</span>) architecture consists of an input layer, a hidden layer, and an output layer [<a href="#B33-computation-12-00163" class="html-bibr">33</a>]. The values <math display="inline"><semantics> <mrow> <msub> <mrow> <mo>{</mo> <mi>y</mi> </mrow> <mrow> <mi>t</mi> <mo>−</mo> <mn>1</mn> </mrow> </msub> <mo>,</mo> <msub> <mrow> <mi>y</mi> </mrow> <mrow> <mi>t</mi> <mo>−</mo> <mn>2</mn> </mrow> </msub> <mo>,</mo> <mo>.</mo> <mo>.</mo> <mo>.</mo> <mo>,</mo> <msub> <mrow> <mi>y</mi> </mrow> <mrow> <mi>t</mi> <mo>−</mo> <mi>s</mi> </mrow> </msub> <mo>,</mo> <msub> <mrow> <mi>y</mi> </mrow> <mrow> <mi>t</mi> <mo>−</mo> <mn>2</mn> <mi mathvariant="normal">s</mi> </mrow> </msub> <mo>,</mo> <msub> <mrow> <mi>y</mi> </mrow> <mrow> <mi>t</mi> <mo>−</mo> <mi>p</mi> </mrow> </msub> <mo>}</mo> </mrow> </semantics></math> represent the lagged inputs of order <math display="inline"><semantics> <mrow> <mi>p</mi> </mrow> </semantics></math> with <math display="inline"><semantics> <mrow> <mi>s</mi> </mrow> </semantics></math> being the seasonality multiple. Number of neurons in the hidden layer are denoted by <math display="inline"><semantics> <mrow> <mi>k</mi> </mrow> </semantics></math> and the resultant output at time <math display="inline"><semantics> <mrow> <mi>t</mi> </mrow> </semantics></math> is given by <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>y</mi> </mrow> <mrow> <mi>t</mi> </mrow> </msub> </mrow> </semantics></math>.</p> "> Figure 4
<p>Schematic representation of an LSTM cell.</p> "> Figure 5
<p>Proposed WT-NNAR-LSTM-GBM model.</p> "> Figure 6
<p>Model comparisons using performance metrics for CSIR (<b>top left panel</b>), NUST (<b>top right panel</b>), RVD (<b>bottom left panel</b>), and Venda (<b>bottom right panel</b>).</p> "> Figure 7
<p>Comparison of 288 min predictions and actual wind speed data for CSIR (<b>Top panel</b>), NUST (<b>Second top panel</b>), RVD (<b>Second bottom panel</b>) and Venda (<b>Bottom panel</b>).</p> "> Figure 8
<p>Distributions of the residuals for CSIR (<b>top left panel</b>), NUST (<b>top right panel</b>), RVD (<b>bottom left panel</b>), and Venda (<b>bottom right panel</b>).</p> ">
Abstract
:1. Introduction
1.1. Overview
1.2. Literature Review
1.3. Innovations and Contributions
- The proposed hybrid model for wind speed forecasting is predicated upon a multi-model ensemble approach, incorporating data decomposition through WTs, complexity classification through SampEn, individual subseries modelling and prediction using NNAR and LSTM techniques, and forecast combination through GBM strategies. This model presents a comprehensive solution for wind speed forecasting, leveraging the strengths of several techniques to provide a refined and accurate forecast.
- WTs play a crucial role in data transformation and decomposition, as they offer exceptional efficiency while minimising random fluctuations in data sequences, thus improving models’ prediction accuracy. As such, these techniques are highly recommended for breaking down irregular wind speed data into low- and high-frequency signals. Significantly, the decomposed signals exhibit more apparent trends and patterns with less variability than the original wind speed signal. This allows for more efficient training and prediction, to a certain extent.
- The concept of randomness pertains to the frequency of distinct digits appearing in a given sequence. To gauge randomness, statistical metrics such as mean, standard deviation, skewness, kurtosis, etc., are often employed in the literature. Nevertheless, traditional approaches fall short of addressing certain complexities that emerge when attempting to examine randomness using this methodology (see [22] for details). To the contrary, the SampEn criterion enables efficient and effective classification/judgment of decomposed signals based on their analogous complex properties and deterministic properties. Consequently, the most suitable modelling and forecasting approach is employed to improve prediction accuracy.
- Besides identifying patterns, NNAR models are resistant to non-stationarity and outliers. These nonlinear approximators leverage the SampEn criterion to precisely predict less random and deterministic subseries.
- To curb the gradient disappearances and explosions to which NNARs are susceptible, subseries classified as more complex or highly random by the SampEn criterion are modelled and predicted using more reliable, optimised, and robust stateless LSTMs. Different from the stateful LSTM, a stateless LSTM can effectively and accurately learn patterns in unstable random time series data such as wind speed. Furthermore, stateless LSTMs are preferred over stateful LSTMs for this time series prediction task because of their higher stability, simplicity, and accuracy.
- To capture the complex nonlinear structure embedded in wind speed subseries forecasts, it is imperative to employ a nonlinear forecast combination method. Therefore, a highly scalable, robust, nonlinear GBM model is preferred over a linear combination model for combining nonlinear wind speed subseries forecasts.
1.4. Paper Structure
2. Materials and Methods
2.1. Data Description
2.2. Exploratory Analysis
2.3. Wavelet Transformation
2.4. Sample Entropy
2.5. Neural Network Autoregression
2.6. Long Short-Term Memory Networks
Stateless LSTM Prediction Approach
- Step 1: Data preparation
- Step 2: Data normalisation
- Step 3: Training LSTM Network
- Step 4: Predictions Using the LSTM Network
2.7. Gradient Boosting Machines
Hyperparameter Setting for GBM
2.8. K-Nearest Neighbours
2.9. Proposed Predictive Approach
WT-NNAR-LSTM-GBM Model
Algorithm 1: WT-NNAR-LSTM-GBM |
INPUT: Wind speed data [] |
|
OUTPUT:, MAE, RMSE, MAD, CRPS, and PIW |
2.10. Predictive Performance Assessment
2.10.1. Point Prediction Metrics
2.10.2. Probabilistic Prediction Metrics
3. Results
3.1. Evaluation of Point Predictions
3.2. Residual Analysis
3.3. Evaluation of Probabilistic Predictions
4. Conclusions
- (i)
- (ii)
- SampEn reduced the complexity of predictions and improved predictive performance by tailoring the modelling and forecasting methods to the specific attributes of subseries data.
- (iii)
- The implementation of NNARs has proved to be successful in predicting less random and deterministic subseries. NNARs are known to be resistant to non-stationarity and outliers, which makes them an effective tool for this purpose. While highly computationally efficient, NNAR is the least accurate based on both point and probabilistic metrics.
- (iv)
- Using a stateless LSTM significantly enhanced the predictive accuracy of hybrid models by capturing extreme wind speed values associated with turbulence and producing more accurate, reliable, and robust predictions. Similar results were found in [36].
- (v)
- Although computationally intensive [6,8], LSTMs are valuable for wind speed prediction due to their advanced pattern recognition and gradient explosion handling capabilities, and the ensemble of predictions using nonlinear GBM is efficient and robust across different seasons and station locations. These results are analogous to those found in [5,22].
- (vi)
- According to our thorough overall comparative analysis of point evaluation metrics (MAE and RMSE), as well as probabilistic evaluation metrics (MAD and PIW), in addition to residual analysis, we have found that the WT-LSTM-NNAR-GBM model (followed by the better-calibrated WT-LSTM-KNN-GBM based on CPRS) is the most accurate, sharpest, robust, and reliable option for modelling and predicting all four datasets across varying locations with differing weather patterns.
- (vii)
- The performance of the models is influenced by both station location and season. Additionally, we found that WT-LSTM-NNAR-GBM was less sensitive (to station location and season of the year) compared to individual models such as NNAR and LSTM, which displayed higher sensitivity.
- (viii)
- The proposed hybrid approach shows promising results for short-term wind speed forecasting, and it can be utilised for various purposes, such as achieving uniform wind power distribution, optimising wind power output, and ensuring smooth grid operations in real time.
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Wang, X.; Guo, P.; Huang, X. A Review of Wind Power Forecasting Models. Energy Procedia 2011, 12, 770–777. [Google Scholar] [CrossRef]
- Hanifi, X.; Liu, Z.; Lin, S.; Lotfian, A. Critical Review of Wind Power Forecasting Methods-Past, Present and Future. Energies 2020, 13, 3764. [Google Scholar] [CrossRef]
- Aasim; Singh, S.N.; Abheejeet, M. Repeated wavelet transform based ARIMA Model for very shortterm wind speed forecasting. Renew. Energy 2019, 136, 758–768. [Google Scholar] [CrossRef]
- Sivhugwana, K.S.; Ranganai, E. An Ensemble Approach to Short-Term Wind Speed Predictions Using Stochastic Methods, Wavelets and Gradient Boosting Decision Trees. Wind 2024, 4, 44–67. [Google Scholar] [CrossRef]
- Xiang, J.; Qiu, Z.; Hao, Q.; Cao, H. Multi-time scale wind speed prediction based on WT-bi-LSTM. MATEC Web Conf. 2020, 309, 05011. [Google Scholar] [CrossRef]
- Xie, A.; Yang, H.; Chen, V.; Sheng, L.; Zhang, Q. A Short-Term Wind Speed Forecasting Model Based on a Multi-Variable Long Short-Term Memory Network. Atmosphere 2021, 12, 651. [Google Scholar] [CrossRef]
- Berrezzek, F.; Khelil, K.; Bouadjila, T. Efficient wind speed forecasting using discrete wavelet transform and artificial neural networks. Rev. D’Intell. Artif. 2019, 33, 447–452. [Google Scholar] [CrossRef]
- Delgado, I.; Fahim, M. Wind Turbine Data Analysis and LSTM-Based Prediction in SCADA System. Energies 2021, 14, 125. [Google Scholar] [CrossRef]
- Ibrahim, M.; Alsheikh, M.; Al-Hindawi, O.; Al-Dahidi, S.; ElMoaqet, H. Short-Time Wind Speed Forecast Using Artificial Learning-Based Algorithms. Comput. Intell. Neurosci. 2020, 2020, 8439719. [Google Scholar] [CrossRef]
- Chen, N.; Qian, Z.; Meng, X. Multistep Wind Speed Forecasting Based on Wavelet and Gaussian Processes. Math. Probl. Eng. 2013, 2013, 461983. [Google Scholar] [CrossRef]
- Fuad, N.; Gamal, A.; Ammar, A.; Ali, A.; Sieh Kiong, T.; Nasser, A.; Janaka, E.; Ahmed, A. Multistep short-term wind speed prediction using nonlinear auto-regressive neural network with exogenous variable selection. Alex. Eng. J. 2021, 60, 1221–1229. [Google Scholar] [CrossRef]
- Hua, Y.; Zhao, Z.; Li, R.; Chen, X.; Liu, Z.; Zhang, H. Deep Learning with Long Short-Term Memory for Time Series Prediction. IEEE Commun. Mag. 2018, 57, 114–119. [Google Scholar] [CrossRef]
- Liu, Y.; Guan, L.; Hou, C.; Han, H.; Liu, Z.; Sun, Y.; Zheng, M. Wind Power Short-Term Prediction Based on LSTM and Discrete Wavelet Transform. Appl. Sci. 2019, 9, 1108. [Google Scholar] [CrossRef]
- Wang, A. Hybrid Wavelet Transform Based Short-Term Wind Speed Forecasting Approach. Sci. World J. 2014, 2014, 914127. [Google Scholar] [CrossRef] [PubMed]
- Mutavhatsindi, T.; Sigauke, C.; Mbuvha, R. Forecasting Hourly Global Horizontal Solar Irradiance in South Africa Using Machine Learning Models. IEEE Access 2020, 8, 198872–198885. [Google Scholar] [CrossRef]
- Feng, T.; Yang, S.; Han, F. Chaotic time series prediction using wavelet transform and multi-model hybrid method. J. Vibro Eng. 2019, 21, 983–1999. [Google Scholar] [CrossRef]
- Yu, Y.; Cao, J.; Zhu, J. An LSTM Short-Term Solar Irradiance Forecasting Under Complicated Weather Conditions. IEEE Access 2019, 7, 145651–145666. [Google Scholar] [CrossRef]
- Jin, Y.; Guo, H.; Wang, J.; Song, A. A Hybrid System Based on LSTM for Short-Term Power Load Forecasting. Energies 2020, 13, 6241. [Google Scholar] [CrossRef]
- Zhang, J.; Wei, Y.; Tan, Z.F.; Ke, W.; Tian, W. A Hybrid Method for Short-Term Wind Speed Forecasting. Sustainability 2017, 9, 596. [Google Scholar] [CrossRef]
- Adamowski, K.; Prokoph, A.; Adamowski’, J. Development of a new method of wavelet aided trend detection and estimation. Hydrol. Process. 2009, 23, 2686–2696. [Google Scholar] [CrossRef]
- Saroha, S.; Aggarwal, S. Wind power forecasting using wavelet transforms and neural networks with tapped delay. J. Power Energy Syst. 2018, 4, 197–209. [Google Scholar] [CrossRef]
- Wei, Q.; Liu, D.-H.; Wang, K.-H.; Liu, Q.; Abbod, M.F.; Jiang, B.C.; Chen, K.-P.; Wu, C.; Shieh, J.-S. Multivariate multiscale entropy applied to center of pressure signals analysis: An effect of vibration stimulation of shoes. Entropy 2012, 14, 2157–2172. [Google Scholar] [CrossRef]
- Chen, J.; Zeng, G.Q.; Zhou, W.; Du, W.; Lu, K.D. Wind speed forecasting using nonlinear-learning ensemble of deep learning time series prediction and extremal optimization. Energy Convers. Manag. 2018, 165, 681–695. [Google Scholar] [CrossRef]
- Dghais, A.A.; Ismail, M.T. A Comparative Study between Discrete Wavelet Transform and Maximal Overlap Discrete Wavelet Transform for Testing Stationarity. Int. J. Math. Comput. Phys. Electr. Comput. Eng. 2013, 7, 1677–1681. [Google Scholar]
- Cornish, C.R.; Bretherton, C.S.; Percival, D.B. Maximal Overlap Wavelet Statistical Analysis With Application to Atmospheric Turbulence. Bound.-Layer Meteorol. 2006, 119, 339–374. [Google Scholar] [CrossRef]
- Rodrigues, D.V.Q.; Zuo, D.; Li, C. A MODWT-Based Algorithm for the Identification and Removal of Jumps/Short-Term Distortions in Displacement Measurements Used for Structural Health Monitoring. IoT 2022, 3, 60–72. [Google Scholar] [CrossRef]
- Paramasivam, S.; Pl, S.A.; Sathyamoorthi, P. Maximal overlap discrete wavelet transform-based power trace alignment algorithm against random delay countermeasure. ETRI J. 2022, 44, 512–523. [Google Scholar] [CrossRef]
- Delgado-Bonal, A.; Marshak, A. Approximate Entropy and Sample Entropy: A Comprehensive Tutorial. Entropy 2019, 21, 541. [Google Scholar] [CrossRef] [PubMed]
- Lake, E.D.; Richman, S.J.; Griffin, J.M.; Moorman, P.M. Sample entropy analysis of neonatal heart rate variability. Am. J. Physiol.-Regul. Integr. Comp. Physiol. 2002, 283, R789–R797. [Google Scholar] [CrossRef] [PubMed]
- Richman, J.S.; Randall Moorman, J.R. Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol.-Heart Circ. Physiol. 2000, 278, H2039–H2049. [Google Scholar] [CrossRef] [PubMed]
- Ronak, B.; Na, H.; Yi, S.; Neil, D.; Tony, S.; David, M. Efficient Methods for Calculating Sample Entropy in Time Series Data Analysis. Procedia Comput. Sci. 2018, 145, 97–104. [Google Scholar] [CrossRef]
- Celeux, G.; Soromenho, G. An entropy criterion for assessing the number of clusters in a mixture model. J. Classif. 1996, 13, 195–212. [Google Scholar] [CrossRef]
- Hyndman, R.J.; Athanasopoulos, G. Forecasting Principles and Practice, 2nd ed.; OTexts: Melbourne, Australia, 2021. [Google Scholar]
- Hochreiter, S.; Schmidhuber, J. Long Short-term Memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef] [PubMed]
- Yadav, A.; Jha, C.K.; Sharan, A. Optimizing LSTM for time series prediction in Indian stock market. Procedia Comput. Sci. 2020, 167, 2091–2100. [Google Scholar] [CrossRef]
- Saha, S. Comprehensive Forecasting-Based Analysis of Hybrid and Stacked Stateful/Stateless Models. arXiv 2024. [Google Scholar] [CrossRef]
- Friedman, J.H. Stochastic gradient boosting. Comput. Stat. Data Anal. 2019, 38, 367–378. [Google Scholar] [CrossRef]
- Singh, U.; Rizwan, M.; Alaraj, M.; Alsaidan, I. A Machine Learning-Based Gradient Boosting Regression Approach for Wind Power Production Forecasting: A Step towards Smart Grid Environments. Energies 2021, 14, 5196. [Google Scholar] [CrossRef]
- Huang, C.M.; Chen, S.J.; Yang, S.P.; Chen, H.-J. One-Day-Ahead Hourly Wind Power Forecasting Using Optimized Ensemble Prediction Methods. Energies 2023, 16, 2688. [Google Scholar] [CrossRef]
- Martínez, F.; Frías, M.P.; Charte, F.; Rivera, A.J. Time Series Forecasting with KNN in R: The tsfknn Package. R J. 2019, 11, 229–242. [Google Scholar] [CrossRef]
- Gensler, A. Wind Power Ensemble Forecasting: Performance Measures and Ensemble Architectures for Deterministic and Probabilistic Forecasts. Ph.D. Thesis, University of Kussel, Kassel, Hessen, Germany, 21 September 2018. [Google Scholar]
- Funk, S.; Camacho, A.; Kucharski, A.J.; Lowe, R.; Eggo, R.M.; Edmunds, W.J. Assessing the performance of real-time epidemic forecasts: A case study of Ebola in the Western Area region of Sierra Leone, 2014–2015. PLoS Comput. Biol. 2009, 15, e1006785. [Google Scholar] [CrossRef]
- Gneiting, T.; Katzfuss, M. Probabilistic Forecasting. Annu. Rev. Stat. Appl. 2014, 1, 125–151. [Google Scholar] [CrossRef]
- Gneiting, T.; Raftery, A.E. Strictly Proper Scoring Rules, Prediction, and Estimation. J. Am. Stat. Assoc. 2007, 102, 359–378. [Google Scholar] [CrossRef]
Station | Longitude | Latitude | Elevation (m) | Topography |
---|---|---|---|---|
CSIR | −25.746519 | 28.278739 | 1400 | The roof of a building |
NUST | −22.565000 | 17.075001 | 1683 | The roof of the engineering building |
RVD | −28.560841 | 16.761459 | 141 | Inside enclosure in a desert region |
Venda | −23.131001 | 30.423910 | 628 | Vuwani Science Research Centre |
Station | Season | Month | Sample Size | Training Set | Testing Set |
---|---|---|---|---|---|
CSIR | Winter | 15 August 2019 | 1440 | 1152 | 288 |
NUST | Autumn | 15 May 2019 | 1440 | 1152 | 288 |
RVD | Spring | 7 September 2019 | 1440 | 1152 | 288 |
Venda | Summer | 31 January 2019 | 1440 | 1152 | 288 |
Station | Min | Q1 | Median | Mean | St. Dev. | Q3 | Max | Skewness | Kurtosis |
---|---|---|---|---|---|---|---|---|---|
CSIR | 0.000 | 1.812 | 2.725 | 2.692 | 1.116 | 3.512 | 8.050 | 0.143 | 2.792 |
NUST | 0.000 | 0.6465 | 1.1760 | 1.2772 | 0.861 | 1.7820 | 5.2890 | 0.769 | 3.641 |
RVD | 0.036 | 3.832 | 7.588 | 7.125 | 3.603 | 10.030 | 14.400 | −0.218 | 1.947 |
Venda | 0.000 | 0.962 | 1.454 | 1.561 | 0.812 | 2.048 | 4.136 | 0.579 | 3.008 |
Station | D1 (r) | D2 (r) | D3 (r) | A3 (r) |
---|---|---|---|---|
CSIR | (0.0691) | (0.0665) | (0.0657) | (0.1905) |
NUST | (0.0510) | (0.0516) | (0.0482) | (0.1485) |
RVD | (0.0486) | (0.0559) | (0.0605) | (0.7141) |
Venda | (0.0472) | (0.0507) | (0.0478) | (0.1386) |
Station | D1 | D2 | D3 | A3 |
---|---|---|---|---|
CSIR | 1.7127 | 1.4321 | 0.7617 | 0.3292 |
NUST | 0.9138 | 0.9038 | 0.7038 | 0.3309 |
RVD | 1.2615 | 1.1175 | 0.6430 | 0.0716 |
Venda | 1.3662 | 1.1764 | 0.7288 | 0.3743 |
Station | ) | ) | ) | ) |
---|---|---|---|---|
CSIR | (0.3455, 0.0549) | (0.3324, 0.0946) | (0.3287, −0.1607) | (0.9525, −0.1555) |
NUST | (0.2550, 0.1959) | (0.2579, 0.2359) | (0.2412, −0.0412) | (0.7424, 0.3316) |
RVD | (0.2430, 0.2374) | (0.2794, 0.0740) | (0.3026, −0.5158) | (3.5706, −0.2514) |
Venda | (0.2361, 0.0793) | (0.2537, 0.1026) | (0.2388, −0.0025) | (0.6928, 0.4834) |
Station | D1 (p-k-1) | D2 (p-k-1) | D3 (p-k-1) | A3 (p-k-1) | (p-k-1) |
---|---|---|---|---|---|
CSIR | 17-9-1 | 10-6-1 | 22-12-1 | 16-8-1 | 16-8-1 |
NUST | 8-4-1 | 10-6-1 | 23-12-1 | 15-8-1 | 9-5-1 |
RVD | 6-4-1 | 10-6-1 | 20-10-1 | 4-2-1 | 17-9-1 |
Venda | 7-5-1 | 8-6-1 | 14-8-1 | 12-6-1 | 20-10-1 |
Hyperparameter | Values |
---|---|
Distribution | Gaussian |
Trees | 463–812 |
Interaction depth | 3–7 |
Learning rate | 5%–6% |
Loss function | root mean square error (RMSE) |
Cross-validation | 10% |
Model | Goal of Each Model on the Proposed Hybrid Strategy |
---|---|
WTs | WTs are of paramount importance as a denoising and transform technique, as they are designed to minimise random fluctuations in the sequence of data and enhance prediction accuracy. Consequently, these techniques are endorsed for the deconstruction of wind speed data into low-frequency and several high-frequency signals. |
SampEn | Besides being highly efficient and simple, SampEn can determine the randomness of a wind speed series of data without any previous knowledge of the source data. Hence, we unleash the power of SampEn and employ it to determine the level of complexity for each of the decomposed signals, thereby ensuring that the most appropriate modelling and forecasting is the approach employed for improved prediction accuracy. |
NNAR | NNARs are not only employed to detect trends, but they are not prone to non-stationarity and outlier effects. Consequently, these nonlinear approximators are employed to precisely detect and model nonlinear features within those subseries that have been recognised as less random (or deterministic) through the application of the SampEn criterion. |
LSTM | To circumvent the drawbacks of gradient disappearance and explosion to which the NNARs are vulnerable, those subseries that are considered to be more complex (or highly random) based on the SampEn criterion (i.e., SampEn values closer or greater than 1 (i.e., at least 0.9)), are modelled using a more robust and reliable stateless LSTM. This time series prediction task is best performed using stateless LSTMs over stateful LSTMs due to their stability and accuracy. |
GBM | In addition to its robustness and scalability, a nonlinear GBM model is preferred over a linear combination (such as conventional direct summation) model for prediction fusion because it is highly accurate. In addition, it takes into account the non-linear structure of subseries forecast in the combination of predictions, thus enhancing predictive performance. |
Model | Decomposition | Entropy | D1 | D2 | D3 | A3 | Fusion |
---|---|---|---|---|---|---|---|
WT-NNAR-LSTM-GBM | WT | SampEn | LSTM | LSTM | NNAR | NNAR | GBM |
WT-NNAR-KNN-GBM | WT | SampEn | KNN | KNN | NNAR | NNAR | GBM |
WT-KNN-LSTM-GBM | WT | SampEn | LSTM | LSTM | KNN | KNN | GBM |
Hyperparameter | Values |
---|---|
Activation function | Hyperbolic |
Number of layers | 3 |
Loss function | MSE |
Optimiser | ADAM |
Learning rate | 1%–2% |
Epochs (D1, D2, D3, A3, ) | 25–30 |
M1 | M2 | M3 | M4 | M5 | |
---|---|---|---|---|---|
CSIR | |||||
(%) | 54.8610 | 48.6111 | 51.3889 | 47.5694 | 52.0833 |
(%) | 45.1389 | 51.3889 | 48.6111 | 52.4306 | 47.9167 |
Std.Dev (m/s) | 0.5229 | 0.5093 | 0.0934 | 0.1064 | 0.0954 |
Skewness (m/s) | 3.6226 | 0.0048 | −0.0472 | 0.0679 | 0.0242 |
AD* () | <0.0001 | 0.06522 | 0.1147 | 0.6743 | 0.05911 |
NUST | |||||
(%) | 46.1806 | 64.5833 | 50.0000 | 49.3056 | 49.3056 |
(%) | 53.8194 | 35.4167 | 50.0000 | 50.6944 | 50.6944 |
Std.Dev (m/s) | 0.1558 | 0.2829 | 0.0781 | 0.1118 | 0.0918 |
Skewness (m/s) | 0.7140 | 0.7507 | 0.2284 | 0.3877 | 0.2161 |
AD* (α = 0.05) | <0.0001 | <0.0001 | 0.0043 | 0.0059 | 0.0522 |
RVD | |||||
(%) | 55.5556 | 52.4306 | 47.5694 | 47.5694 | 46.8750 |
(%) | 44.4444 | 47.5694 | 52.4306 | 52.4306 | 53.1250 |
Std.Dev (m/s) | 0.6257 | 0.1913 | 0.1132 | 0.0774 | 0.1150 |
Skewness (m/s) | 4.3280 | 0.1282 | −6.2854 | −0.0108 | −5.8903 |
AD* (α = 0.05) | <0.0001 | 0.01416 | <0.0001 | 0.01404 | <0.0001 |
VENDA | |||||
(%) | 51.7361 | 57.6389 | 49.6528 | 48.2638 | 46.8750 |
(%) | 48.2639 | 42.3611 | 50.3472 | 52.4306 | 53.1250 |
Std.Dev (m/s) | 0.2996 | 0.2668 | 0.1281 | 0.1483 | 0.1466 |
Skewness (m/s) | 3.4968 | 0.0625 | 0.0669 | 0.1281 | 0.2871 |
AD* (α = 0.05) | <0.0001 | 0.00654 | 0.5505 | 0.1815 | 0.1412 |
M1 | M2 | M3 | M4 | M5 | Mean | |
---|---|---|---|---|---|---|
CSIR | ||||||
MAD (m/s) | 0.0886 | 0.4808 | 0.0778 | 0.1006 | 0.0834 | 0.1662 |
St.Dev PIW (m/s) | 0.5903 | 0.1373 | 0.0012 | 0.0118 | 0.0005 | 0.1482 |
OL (count) | 25 | 30 | 27 | 29 | 27 | 28 |
CRPS (m/s) | 0.4017 | 0.3541 | 0.3439 | 0.3358 | 0.3423 | 0.3555 |
NUST | ||||||
MAD (m/s) | 0.1106 | 0.1527 | 0.0595 | 0.0950 | 0.0829 | 0.1021 |
St.Dev PIW (m/s) | 0.0950 | 0.4450 | 0.0074 | 0.0118 | 0.0247 | 0.1178 |
OL (count) | 28 | 28 | 28 | 29 | 28 | 28 |
CRPS (m/s) | 0.3551 | 0.3634 | 0.3540 | 0.3503 | 0.3412 | 0.3528 |
RVD | ||||||
MAD (m/s) | 0.0582 | 0.1664 | 0.0521 | 0.0640 | 0.0575 | 0.0796 |
St.Dev PIW (m/s) | 0.7608 | 0.0569 | 0.0075 | 0.0283 | 0.0075 | 0.1722 |
OL (count) | 28 | 26 | 26 | 30 | 28 | 28 |
CRPS (m/s) | 0.6869 | 0.6368 | 0.6247 | 0.6141 | 0.6238 | 0.6373 |
VENDA | ||||||
MAD (m/s) | 0.0187 | 0.2390 | 0.1110 | 0.1316 | 0.1354 | 0.1045 |
St.Dev PIW (m/s) | 0.3564 | 0.1485 | 0.0143 | 0.0066 | 0.0284 | 0.1108 |
OL (count) | 28 | 27 | 27 | 27 | 32 | 28 |
CRPS (m/s) | 0.3740 | 0.3138 | 0.2873 | 0.2746 | 0.2754 | 0.3050 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Sivhugwana, K.S.; Ranganai, E. Short-Term Wind Speed Prediction via Sample Entropy: A Hybridisation Approach against Gradient Disappearance and Explosion. Computation 2024, 12, 163. https://doi.org/10.3390/computation12080163
Sivhugwana KS, Ranganai E. Short-Term Wind Speed Prediction via Sample Entropy: A Hybridisation Approach against Gradient Disappearance and Explosion. Computation. 2024; 12(8):163. https://doi.org/10.3390/computation12080163
Chicago/Turabian StyleSivhugwana, Khathutshelo Steven, and Edmore Ranganai. 2024. "Short-Term Wind Speed Prediction via Sample Entropy: A Hybridisation Approach against Gradient Disappearance and Explosion" Computation 12, no. 8: 163. https://doi.org/10.3390/computation12080163