[go: up one dir, main page]
Skip to main content
SpringerLink
Log in

Fuzzy wavelet neural network based on fuzzy clustering and gradient techniques for time series prediction

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper presents the development of fuzzy wavelet neural network system for time series prediction that combines the advantages of fuzzy systems and wavelet neural network. The structure of fuzzy wavelet neural network (FWNN) is proposed, and its learning algorithm is derived. The proposed network is constructed on the base of a set of TSK fuzzy rules that includes a wavelet function in the consequent part of each rule. A fuzzy c-means clustering algorithm is implemented to generate the rules, that is the structure of FWNN prediction model, automatically, and the gradient-learning algorithm is used for parameter identification. The use of fuzzy c-means clustering algorithm with the gradient algorithm allows to improve convergence of learning algorithm. FWNN is used for modeling and prediction of complex time series and prediction of foreign-exchange rates. Exchange rates are dynamic process that changes every day and have high-order nonlinearity. The statistical data for the last 2 years are used for the development of FWNN prediction model. Effectiveness of the proposed system is evaluated with the results obtained from the simulation of FWNN-based systems and with the comparative simulation results of previous related models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Box GEP (1970) Time series analysis, forecasting and control. Holden Day, San Francisco

    MATH  Google Scholar 

  2. So MKP, Lam K, Li WK (1999) Forecasting exchange rate volatility using autoregressive random variance model. Appl Financial Econ 9:583–591

    Article  Google Scholar 

  3. Hsieh DA (1989) Modeling heteroscedasticity in daily foreign-exchange rates. J Bus Econ Stat 7:307–317

    Article  Google Scholar 

  4. Bollerslev T (1990) Modeling the coherence in short-run nominal exchange rates: a multivariate generalized ARCH model. Rev Econ Stat 72:498–505

    Article  Google Scholar 

  5. Huang W, Lai KK, Nakamori Y, Wang S (2004) Forecasting the foreign exchange rates with artificial neural networks: a review. Intl J Inf Tech Decis Mak 3(1):145–165

    Article  Google Scholar 

  6. Maddala GS (1996) Introduction to econometrics. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  7. Chen Y, Yang B, Dong J, Abraham A (2004) Nonlinear system modelling via optimal design of neural trees. Int J Neural Syst 14(2):125–137

    Article  Google Scholar 

  8. Kim K-j, Lee WB (2004) Stock market prediction using artificial neural networks with optimal feature transformation. Neural Comput Appl 13(3):255–260

    Article  Google Scholar 

  9. Sfetsos A, Siriopoulos C (2004) Combinatorial time series forecasting based on clustering algorithms and neural networks. Neural Comput Appl 13(1):56–64

    Article  Google Scholar 

  10. Maqsood I, Khan MR, Abraham A (2004) An ensemble of neural networks for weather forecasting. Neural Comput Appl 13(2):112–122

    Google Scholar 

  11. Thomas B, Soleimani-Mohseni M (2007) Artificial neural network models for indoor temperature prediction: investigations in two buildings. Neural Comput Appl 16(1):81–89

    Google Scholar 

  12. Górriz JM, Puntonet CG, Salmerón M, de la Rosa JJG (2004) A new model for time-series forecasting using radial basis functions and exogenous data. Neural Comput Appl 13(2):101–111

    Google Scholar 

  13. Sun YF, Liang YC, Zhang WL, Lee HP, Lin WZ, Cao LJ (2005) Optimal partition algorithm of the RBF neural network and its application to financial time series forecasting. Neural Comput Appl 14(1):36–44

    Article  Google Scholar 

  14. Garg S, Pal SK (2007) Evaluation of the performance of backpropagation and radial basis function neural networks in predicting the drill flank wear. Neural Comput Appl 16(4–5):407–417

    Google Scholar 

  15. Hocaoglu FO, Oysal Y, Kurban M (2009) Missing wind data forecasting with adaptive neuro-fuzzy inference system. Neural Comput Appl 18(3):207–212

    Google Scholar 

  16. Gholipour A, Lucas C, Araabi BN, Mirmomeni M, Shafiee M (2007) Extracting the main patterns of natural time series for long-term neurofuzzy prediction. Neural Comput Appl 16(4–5):383–393

    Google Scholar 

  17. Weigend AS, Huberman BA, Rumelhart DE (1992) Predicting sunspots and exchange rates with connectionist networks. In: Casdagli M, Eubank S (eds) Nonlinear modeling and forecasting. Addison-Wesley, Redwood City, CA, pp 395–432

    Google Scholar 

  18. Refenes AN (1993) Constructive learning and its application to currency exchange rate forecasting. In: Trippi R, Turban E (eds) Neural networks in finance and investing: using artificial intelligence to improve real-world performance. Probus, Chicago, pp 777–805

    Google Scholar 

  19. Refenes AN, Azema-Barac M, Chen L, Karoussos SA (1993) Currency exchange rate prediction and neural network design strategies. Neural Comput Appl 1:46–58

    Article  Google Scholar 

  20. Kuan CM, Liu T (1995) Forecasting exchange rates using feedforward and recurrent neural networks. J Appl Econom 10:347–364

    Article  Google Scholar 

  21. Hann TH, Steurer E (1996) Much ado about nothing? Exchange rate forecasting: neural networks versus linear models using monthly and weekly data. Neurocomputing 10:323–339

    Article  MATH  Google Scholar 

  22. Episcopos A, Davis J (1996) Predicting returns on Canadian exchange rates with artificial neural networks and EGARCHM-M model. Neural Comput Appl 4:168–174

    Article  Google Scholar 

  23. Yager RR, Zadeh LA (eds) (1994) Fuzzy sets, neural networks and soft computing. Van Nostrand Reinhold, New York

    MATH  Google Scholar 

  24. Jang J-SR, Sun Ch-T, Muzutani E (1997) Neuro-fuzzy and soft computing: A computational approach to learning and machine intelligence. Prentice Hall, Upper Saddle River, NJ

  25. Kugarajah T, Zhang Q (1995) Multidimensional wavelet frames. IEEE Trans Neural Netw 6:1552–1556

    Article  Google Scholar 

  26. Zhang Q, Benviste A (1995) Wavelet networks. IEEE Trans Neural Netw 3:889–898

    Article  Google Scholar 

  27. Zhang J, Walter GG, Wayne Lee WN (1995) Wavelet neural networks for function learning. IEEE Trans Signal Process 43(6):1485–1497

    Article  Google Scholar 

  28. Postalcioglu S, Becerikli Y (2007) Wavelet networks for nonlinear system modelling. Neural Comput Appl 16(4–5):433–441

    Google Scholar 

  29. Lotric U, Dobnikar A (2005) Predicting time series using neural networks with wavelet-based denoising layers. Neural Comput Appl 14(1):11–17

    Article  Google Scholar 

  30. Cao L, Hong Y, Fang H, He G (1995) Predicting chaotic time series with wavelet networks. Physica D 85:225–238

    Article  MATH  Google Scholar 

  31. Chang PR, Weihui F, Minjun Y (1998) Short term load forecasting using wavelet networks. Eng Intell Syst Electr Eng Commun 6:217–230

    Google Scholar 

  32. Khao TQD, Phuong LM, Binh PTT, Lien NTH (2004) Application of wavelet and neural network to long-term load forecasting. International Conference on Power System technology, POWERCON 2004, pp 840–844, Singapore

  33. Thuillard M (2000) Fuzzy logic in the wavelet framework. Proc Toolmet’2000, April 13–14, Oulu

  34. Thuillard M (2001) Wavelets in softcomputing. World Scientific Press, Singapore

    Book  Google Scholar 

  35. Lin CK, Wang SD (1996) Fuzzy modelling using wavelet transform. Electron Lett 32:2255–2256

    Article  Google Scholar 

  36. Lin Y, Wang FY (2005) Predicting chaotic time series using adaptive wavelet-fuzzy inference system. In: Proceeding of IEEE intelligent vehicles symposium, Las Vegas, Nevada, USA, pp 888–893

  37. Guo QJ, Yu HB, Xu AD (2005) Wavelet fuzzy network for fault diagnosis. In: Proceedings of international conference on communications, circuits and systems. IEEE Press, pp 993–998

  38. Daniel WCH, Ping-An Z, Jinhua X (2001) Fuzzy wavelet networks for function learning. IEEE Trans Fuzzy Syst 9(1):200–211

    Article  Google Scholar 

  39. Abiyev RH, Kaynak O (2008) Fuzzy wavelet neural networks for identification and control of dynamic plants—a novel structure and a comparative study. IEEE Trans Ind Electron 55(8):3133–3140

    Article  Google Scholar 

  40. Abiyev RH (2005) Controller based of fuzzy wavelet neural network for control of technological processes CIMSA 2005. In: IEEE international conference on computational intelligence for measurement systems and applications, Giardini Naxos, Italy, pp 215–219

  41. Abiyev RH (2006) Time series prediction using fuzzy wavelet neural network model. Lecture Notes in Computer Sciences, Springer, Berlin, pp 191–200

  42. Abiyev RH (2009) Fuzzy wavelet neural network for prediction of electricity consumption. AIEDAM: Artif Intell Eng Des Anal Manuf 23(2):109–118

    Google Scholar 

  43. Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum Press, New York

    MATH  Google Scholar 

  44. Chiu SL (1994) Fuzzy model identification based on cluster estimation. J Intell Fuzzy Syst 2:267–278

    MathSciNet  Google Scholar 

  45. Yager RR, Filev DP (1994) Generation of fuzzy rules by mountain clustering. J Intell Fuzzy Syst 2:267–278

    Google Scholar 

  46. Demirli K, Muthukumaran P (2000) Higher order fuzzy system identification using subtractive clustering. J Intell Fuzzy Syst 9:129–158

    Google Scholar 

  47. Kasabov NK (2002) DENFIS: Dynamic evolving neural-fuzzy inference system and its application for time-series. IEEE Trans Syst Fuzzy Syst 10(2):144–154

    Article  Google Scholar 

  48. Wang LX (1997) A course in fuzzy systems and control. Prentice Hall, NJ, pp 183–189

    MATH  Google Scholar 

  49. Wang LX, Wei C (2000) Approximation accuracy of some neuro-fuzzy systems. IEEE Trans Fuzzy Syst 8(4):470–478

    Article  Google Scholar 

  50. Juang C-F (2002) A TSK-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithm. IEEE Trans Fuzzy Syst 10:155–170

    Article  Google Scholar 

  51. Juang CF, Lin CT (1998) An on-line self-constructing neural fuzzy inference network and its applications. IEEE Trans Fuzzy Syst 6(1):12–31

    Article  Google Scholar 

  52. Zadeh LA (1975) The concept of linguistic variable and its application to approximate reasoning. Inf Sci 8

  53. Szu H, Telfer B, Garcia J (1996) Wavelet transforms and neural networks for compression and recognition. Neural Netw 9:695–708

    Article  Google Scholar 

  54. Ku C-C, Lee KY (1995) Diagonal recurrent neural networks for dynamic systems control. IEEE Trans Neural Netw 6:144–156

    Article  Google Scholar 

  55. Tong RM (1980) The evaluation of fuzzy models derived from experimental data. Fuzzy Sets Syst 4:1–12

    Article  MATH  Google Scholar 

  56. Pedtycz W (1984) An identification algorithm in fuzzy relational systems. Fuzzy Sets Syst 13:53–167

    Google Scholar 

  57. Xu CW, Lu YZ (1987) Fuzzy model identification and self-learning for dynamic systems. IEEE Trans Syst Man Cybernet 17:683–689

    Article  MATH  Google Scholar 

  58. Sugeno M, Yasukawa T (1993) A fuzzy logic based approach to qualitative modelling. IEEE Trans Fuzzy Syst 1:7–31

    Article  Google Scholar 

  59. Sugeno M, Tanaka K (1991) Successive identification of a fuzzy model and its application to prediction of complex system. Fuzzy Sets and Syst 42:315–334

    Article  MATH  MathSciNet  Google Scholar 

  60. Lin Y, Cunningham GA III (1995) A new approach to fuzzy-neural system modelling. IEEE Trans Fuzzy Syst 3:190–198

    Article  Google Scholar 

  61. Kim E, Park M, Ji S, Park M (1997) A new approach to fuzzy modelling. IEEE Trans Fuzzy Syst 5:328–337

    Article  Google Scholar 

  62. Kim E, Park M, Kim S, Park M (1998) A transformed input-domain approach to fuzzy modelling. IEEE Trans Fuzzy Syst 6:596–604

    Article  Google Scholar 

  63. Kim J, Kasabov NK (1999) HyFIS: adaptive neuro-fuzzy inference systems and their application to nonlinear dynamical systems. Neural Netw 12:1301–1319

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Near East University, Lefkosa, North Cyprus, Mersin-10, Turkey

    Rahib H. Abiyev

Authors
  1. Rahib H. Abiyev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rahib H. Abiyev.

Appendix

Appendix

As mentioned, the time varying learning rate is used in this paper. Let us consider the derivation of optimal learning rate using Lyapunov function [54]. γ(t) denotes learning rates for weight update formulas (10, 11) at discreet time t.

Let γ(t) is learning rate for the weights W = [w j , a ij , b ij , c ij , o ij ] and FWNN is trained using (10, 11). The convergence is guaranteed if the following conditions are satisfied

$$ 0 < \gamma (t) < {\frac{2}{{\left( {\mathop {\max }\limits_{t} \left\| {{\frac{\partial u(t)}{\partial W}}} \right\|} \right)^{2} }}} $$
(18)

Above statement can be proved by choosing a Lyapunov function \( V(t) = \frac{1}{2}e^{2} (t),\,{\text{where}}\,e(t) = (u^{d} (t) - u(t)) \).

Let us define Lyapunov function as,

$$ V(t) = \frac{1}{2}e^{2} (t) $$
(19)

Here, e(k) represents error function calculated in learning processes.

The change of the Lyapunov function is

$$ \begin{aligned} \Updelta V(t) & = V(t + 1) - V(t) = \frac{1}{2}(e^{2} (t + 1) - e^{2} (t)) = \frac{1}{2}((e(t) + \Updelta e(t))^{2} - e^{2} (t)) \\ & = \frac{1}{2}(2e(t) \cdot \Updelta e(t) + \Updelta e^{2} (t)) = \frac{1}{2}\Updelta e(t)(2e(t) + \Updelta e(t)) \\ \end{aligned} $$
(20)

The error difference is determined as

$$ \Updelta e(t) = {\frac{\partial e(t)}{\partial W}}\Updelta W = {\frac{{\partial (u^{d} (t) - u(t))}}{\partial W}}\Updelta W = - {\frac{\partial u(t)}{\partial W}}\Updelta W $$
(21)

From the update formula (10, 11)

$$ \begin{aligned} \Updelta W & = & - \gamma {\frac{\partial E}{\partial W}} = \gamma e(t){\frac{\partial u(t)}{\partial W}} \\ {\frac{\partial E}{\partial W}} & = & {\frac{\partial }{\partial W}}\left[ {\frac{1}{2}e^{2} (t)} \right] = e(t){\frac{\partial e(t)}{\partial W}} = - e(t){\frac{\partial u(t)}{\partial W}} \\ \end{aligned} $$
(22)
$$ \begin{aligned} \Updelta V(t) & = \frac{1}{2}\Updelta e(t)(2e(t) + \Updelta e(t)) = - \frac{1}{2}\left[ {{\frac{\partial u(t)}{\partial W}}} \right]^{T} \gamma (t)e(t){\frac{\partial u(t)}{\partial W}}\left( {2e(t) - \left[ {{\frac{\partial u(t)}{\partial W}}} \right]^{T} \gamma (t)e(t){\frac{\partial u(t)}{\partial W}}} \right) \\ \, & = \frac{1}{2}\gamma (t)e^{2} (t)\left\| {{\frac{\partial u(t)}{\partial W}}} \right\|^{2} \left( {\gamma (t)\left\| {{\frac{\partial u(t)}{\partial W}}} \right\|^{2} - 2} \right) \\ \end{aligned} $$
(23)

From the Lyapunov stability theorem, asymptotic stability is granted if ΔV(t) < 0, for all t. The initial values of the learning rates for the parameters {c1 ij , c2 ij , o ij , a ij , b ij , w j , q} can be taken differently. In the paper, the learning rates for all parameters W = {c1 ij , c2 ij , o ij , a ij , b ij , w j , q} are chosen to be the same initially, i.e., γ = γc = γo = γa = γb = γw. According to stability condition, from (23), the sufficient condition for convergence can be derived.

$$ 0 < \gamma (t) < {\frac{2}{{\left( {\mathop {\max }\limits_{t} \left\| {{\frac{\partial u(t)}{\partial W}}} \right\|} \right)^{2} }}} $$
(24)

From (24), it is seen that the upper bound of learning rate is found from an epoch. However, the learning rate that guaranties most rapid or optimal convergence is \( \gamma (t) = {1 \mathord{\left/ {\vphantom {1 {\left\| {{\frac{\partial u(t)}{\partial W}}} \right\|^{2} }}} \right. \kern-\nulldelimiterspace} {\left\| {{\frac{\partial u(t)}{\partial W}}} \right\|^{2} }} \). The error at the discrete time t + 1 can be represented as

$$ e(t + 1) = e(t) + \Updelta e(t) \approx e(t) + \left[ {{\frac{\partial e(t)}{\partial W}}} \right]^{T} \Updelta W = e(t) - \left[ {{\frac{\partial u(k)}{\partial W}}} \right]^{T} \gamma e(t){\frac{\partial u(t)}{\partial W}} = e(t)\left( {1 - \gamma \left\| {{\frac{\partial u(t)}{\partial W}}} \right\|^{2} } \right) $$
(25)

If we solve (25) for γ(t) in order to minimize the output error e(t + 1), we can get

$$ \gamma (t) = {\frac{1}{{\left\| {{\frac{\partial u(t)}{\partial W}}} \right\|^{2} }}} $$
(26)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Abiyev, R.H. Fuzzy wavelet neural network based on fuzzy clustering and gradient techniques for time series prediction. Neural Comput & Applic 20, 249–259 (2011). https://doi.org/10.1007/s00521-010-0414-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-010-0414-4

Keywords

Search

Navigation