[go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Fully Complex-Valued Wirtinger Conjugate Neural Networks with Generalized Armijo Search

  • Conference paper
  • First Online:
Intelligent Computing Methodologies (ICIC 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10956))

Included in the following conference series:

Abstract

Conjugate gradient (CG) method has been verified to be one effective strategy for training neural networks in terms of its low memory requirements and fast convergence. In this paper, an efficient CG method is proposed to train fully complex neural networks based on Wirtinger calculus. We adopt two ways to enhance the training performance. One is to construct a sufficient descent direction during training by designing a fine tuning conjugate coefficient. Another technique is to pursue the optimal learning rate instead of a fixed constance in each iteration which is determined by employing a generalized Armijo search. To verify the effectiveness and the convergent behavior of the proposed algorithm, the illustrated simulation has been performed on the complex benchmark noncircular signal.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Chen, S., Hong, X., Khalaf, E., Alsaadi, F.E., Harris, C.J.: Complex-valued B-spline neural network and its application to iterative frequency-domain decision feedback equalization for Hammerstein communication systems. In: 2016 International Joint Conference on Neural Networks, pp. 4097–4104 (2016)

    Google Scholar 

  2. Liu, Y.S., Huang, H., Huang, T.W., Qian, X.S.: An improved maximum spread algorithm with application to complex-valued RBF neural networks. Neurocomputing 216, 261–267 (2016)

    Article  Google Scholar 

  3. Fink, O., Zio, E., Weidmann, U.: Predicting component reliability and level of degradation with complex-valued neural networks. Reliab. Eng. Syst. Saf. 121, 198–206 (2014)

    Article  Google Scholar 

  4. Aizenberg, I.: Complex-Valued Neural Networks with Multivalued Neurons. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20353-4

    Book  MATH  Google Scholar 

  5. Nitta, T.: Solving the XOR problem and the detection of symmetry using a single complex-valued neuron. Neural Netw. 16, 1101–1105 (2003)

    Article  Google Scholar 

  6. Kim, T., Adali, T.: Approximation by fully complex multilayer perceptrons. Neural Comput. 15, 1641–1666 (2003)

    Article  Google Scholar 

  7. Savitha, R., Suresh, S., Sundararajan, N.: Metacognitive learning in a fully complex-valued radial basis function neural network. Neural Comput. 24, 1297–1328 (2012)

    Article  MathSciNet  Google Scholar 

  8. Li, M., Huang, G., Saratchandran, P., Sundararajan, N.: Fully complex extreme learning machine. Neurocomputing 68, 306–314 (2005)

    Article  Google Scholar 

  9. Nitta, T.: An extension of the back-propagation algorithm to complex numbers. Neural Netw. 10, 1391–1415 (1997)

    Article  Google Scholar 

  10. Zhao, Z.Z., Xu, Q.S., Jia, M.P.: Improved shuffled frog leaping algorithm-based BP neural network and its application in bearing early fault diagnosis. Neural Comput. Appl. 27, 375–385 (2016)

    Article  Google Scholar 

  11. Xie, L.: The heat load prediction model based on BP neural network-markov model. Procedia Comput. Sci. 107, 296–300 (2017)

    Article  Google Scholar 

  12. Li, Z.K., Zhao, X.H.: BP artificial neural network based wave front correction for sensor-less free space optics communication. Opt. Commun. 385, 219–228 (2017)

    Article  Google Scholar 

  13. Li, H., Adali, T.: Complex-valued adaptive signal processing using nonlinear functions. EURASIP J. Adv. Sig. Process. 2008, 1–9 (2008)

    MATH  Google Scholar 

  14. Zhang, H.S., Liu, X.D., Xu, D.P., Zhang, Y.: Convergence analysis of fully complex backpropagation algorithm based on Wirtinger calculus. Cogn. Neurodyn. 8(3), 261–266 (2014)

    Article  Google Scholar 

  15. Xu, D.P., Zhang, H.S., Mandic, D.P.: Convergence analysis of an augmented algorithm for fully complex-valued neural networks. Neural Netw. 69, 44–50 (2015)

    Article  Google Scholar 

  16. Zhang, H.S., Xu, D.P., Zhang, Y.: Boundedness and convergence of split-complex back-propagation algorithm with momentum and penalty. Neural Process Lett. 39(3), 297–307 (2014)

    Article  Google Scholar 

  17. Zhang, H.S., Zhang, C., Wu, W.: Convergence of batch split-complex backpropagation algorithm for complex-valued neural networks. Discrete Dyn. Nat. Soc. 1–16 (2009)

    MATH  Google Scholar 

  18. Papalexopoulos, A.D., Hao, S.Y., Peng, T.M.: An implementation of a neural-network-based load forecasting-model for the EMS. IEEE Trans. Power Syst. 9, 1956–1962 (1994)

    Article  Google Scholar 

  19. Lu, C.N., Wu, H.T., Vemuri, S.: Neural network based short-term load forecasting. Trans. Power Syst. 8, 336–342 (1993)

    Article  Google Scholar 

  20. Saini, L.M., Soni, M.K.: Artificial neural network-based peak load forecasting using conjugate gradient methods. IEEE Trans. Power Syst. 17, 907–912 (2002)

    Article  Google Scholar 

  21. Goodband, J.H., Haas, O.C.L., Mills, J.A.: A comparison of neural network approaches for on-line prediction in IGRT. Med. Phys. 35, 1113–1122 (2008)

    Article  Google Scholar 

  22. Hagan, M.T., Demuth, H.B., Beale, M.H.: Neural Network Design. PWS Publisher, Boston (1996)

    Google Scholar 

  23. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (2006). https://doi.org/10.1007/978-0-387-40065-5

    Book  MATH  Google Scholar 

  24. Hestenes, M.R., Stiefel, E.L.: Method of Conjugate Gradients for Solving Linear Systems. National Bureau of Standards, Washington (1952)

    MATH  Google Scholar 

  25. Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)

    Article  MathSciNet  Google Scholar 

  26. Liu, H., Li, X.Y.: A modified HS conjugate gradient method. In: 2011 International Conference on Multimedia Technology, pp. 5699–5702 (2011)

    Google Scholar 

  27. Polak, E., Ribiere, G.: Note sur la convergence de directions conjugates. Rev. Francaise d’Informatique et de Rech. Operationnelle 16, 35–43 (1969)

    Google Scholar 

  28. Polyak, B.T.: The conjugate gradient method in extremal problems. USSR Comput. Math. Math. Phys. 9, 94–112 (1969)

    Article  Google Scholar 

  29. Wan, Z., Hu, C., Yang, Z.: A spectral PRP conjugate gradient methods for nonconvex optimization problem based on modified line search. Discrete Cont. Dyn. Syst. Ser. B 16, 1157–1169 (2017)

    Article  MathSciNet  Google Scholar 

  30. Sun, Q.Y., Liu, X.H.: Global convergence results of a new three terms conjugate gradient method with generalized armijo step size rule. Math. Numer. Sinica 26, 25–36 (2004)

    MathSciNet  Google Scholar 

  31. Magoulas, G.D., Vrahatis, M.N., Androulakis, G.S.: Effective backpropagation training with variable stepsize. Neural Netw. 10, 69–82 (1997)

    Article  Google Scholar 

  32. Wang, A.P., Chen, Z.: Global convergence of a modified HS conjugate gradient method under wolfe-type line search. J. Anhui Univ. 2, 150–156 (2015)

    MATH  Google Scholar 

  33. Dong, X.L., Yang, X.M., Huang, Y.Y.: Global convergence of a new conjugate gradient method with Armijo search. J. Henan Normal Univ. 43(6), 25–29 (2015)

    MATH  Google Scholar 

  34. Wang, J., Zhang, B.J., Sun, Z.Q., Hao, W.X., Sun, Q.Y.: A novel conjugate gradient method with generalized Armijo search for efficient training of feedforward neural networks. Neurocomputing 275, 308–316 (2018)

    Article  Google Scholar 

  35. Needham, T.: Visual complex analysis. Am. Math. Mon. 105, 195–196 (1998)

    Article  MathSciNet  Google Scholar 

  36. Novey, M.P.: Complex ICA using nonlinear functions. IEEE Trans. Sig. Process. 56(9), 4536–4544 (2008)

    Article  MathSciNet  Google Scholar 

  37. Wirtinger, W.: Zur formalen theorie der funktionen von mehr komplexen ver盲nderlichen. Mathematische Annalen 97, 357–375 (1927)

    Google Scholar 

  38. Kreutz-Delgado, K.: The complex gradient operator and the CR-calculus. Mathematics 1–74 (2009)

    Google Scholar 

  39. Mandic, D.P., Goh, S.L.: Complex Valued Nonlinear Adaptive Filters: Noncircularity. Widely Linear and Neural Models. Wiley, Hoboken (2009)

    Book  Google Scholar 

  40. Brandwood, D.H.: A complex gradient operator and its application in adaptive array theory. In: IEE Proceedings H - Microwaves, Optics and Antennas, vol. 130, pp. 11–16 (1983)

    Article  MathSciNet  Google Scholar 

  41. Orozco-Henao, C., Bretas, A.S., Chouhy-Leborgne, R., Herrera-Orozco, A.R., Marín-Quintero, J.: Active distribution network fault location methodology: a minimum fault reactance and Fibonacci search approach. Electr. Power Energy Syst. 84, 232–241 (2017)

    Article  Google Scholar 

  42. Vieira, D.A.G., Lisboa, A.C.: Line search methods with guaranteed asymptotical convergence to an improving local optimum of multimodal functions. Eur. J. Oper. Res. 235, 38–46 (2014)

    Article  MathSciNet  Google Scholar 

  43. Xia, Y., Jelfs, B., Hulle, M.M.V., Principe, J.C., Mandic, D.P.: An augmented echo state network for nonlinear adaptive filtering of complex noncircular signals. IEEE Trans. Neural Netw. 22, 74–83 (2011)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (No. 61305075), the Natural Science Foundation of Shandong Province (No. ZR2015A-L014, ZR201709220208) and the Fundamental Research Funds for the Central Universities (No. 15CX08011A, 18CX02036A).

Author information

Authors and Affiliations

  1. College of Science, China University of Petroleum, Qingdao, 266580, China

    Bingjie Zhang, Shujun Wu, Jian Wang & Huaqing Zhang

  2. College of Computer and Communication Engineering, China University of Petroleum, Qingdao, 266580, China

    Junze Wang

Authors
  1. Bingjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junze Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shujun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Huaqing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huaqing Zhang .

Editor information

Editors and Affiliations

  1. Tongji University, Shanghai, China

    De-Shuang Huang

  2. Indian Institute of Technology Madras, Chennai, India

    M. Michael Gromiha

  3. Inha University, Incheon, Korea (Republic of)

    Kyungsook Han

  4. Liverpool John Moores University, Liverpool, United Kingdom

    Abir Hussain

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, B., Wang, J., Wu, S., Wang, J., Zhang, H. (2018). Fully Complex-Valued Wirtinger Conjugate Neural Networks with Generalized Armijo Search. In: Huang, DS., Gromiha, M., Han, K., Hussain, A. (eds) Intelligent Computing Methodologies. ICIC 2018. Lecture Notes in Computer Science(), vol 10956. Springer, Cham. https://doi.org/10.1007/978-3-319-95957-3_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-95957-3_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-95956-6

  • Online ISBN: 978-3-319-95957-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics