Batch gradient neuro-fuzzy learning method with smoothing L0 regularization for the first-order Takagi-Sugeno system
Pages 11 - 15
Abstract
In this paper, we propose a batch gradient neuro-fuzzy learning algorithm with smoothing regularization (BGNFSL0) for the first-order Takagi-Sugeno system. The regularization method usually tends to produce the sparsest solution, however, its solving is an NP-hard problem, and it cannot be directly used in designing the regularized gradient neuro-fuzzy learning method. By exploiting a series of smoothing functions to approximate the regularizer, the proposed BGNFSL0 successfully avoids the NP-hard nature of the original regularization method, while inheriting the advantage in producing the sparsest solution. In this way, BGNFSL0 can prune the network efficiently during the learning procedure and thus improve the generalization capability of the networks. By conducting simulations to compare it with several other popular regularization learning methods, it is found that BGNFSL0 exhibits the best performance in generating the parsimonious network as well as the generalization capability.
References
[1]
A. Riid and E. Rustern, "Interpretability of fuzzy systems and its application to process control," 2007 IEEE International Fuzzy Systems Conference,2007, pp. 1-6.
[2]
R. Paiva and A. Dourado, "Interpretability and learning in neuro-fuzzy systems," Fuzzy Sets and Systems, vol.147, pp.17-38, 2004.
[3]
B. Cao, J. Zhao, Z. Lv, Y. Gu, P. Yang, and S. K. Halgamuge, "Multiobjective evolution of fuzzy rough neural network via distributed parallelism for stock prediction," IEEE Transactions on Fuzzy Systems, vol. 28, pp. 939-952, 2020.
[4]
J. Fei and T. Wang, "Adaptive fuzzy-neural-network based on RBFNN control for active power filter," International Journal of Machine Learning and Cybernetics, 2018.
[5]
J. Tang, F. Liu, W. Zhang, R. Ke and Y. Zou, "Lane-changes prediction based on adaptive fuzzy neural network," Expert Systems with Application, 2018.
[6]
W. Shen and J. Wang, "Adaptive fuzzy sliding mode control based on pi-sigma fuzzy neutral network for hydraulic hybrid control system using new hydraulic transformer," International Journal of Control, Automation and Systems, vol. 17, pp. 1708-1716, 2019.
[7]
Y. Shin and J. Ghosh, "The pi-sigma network: an efficient higher-order neural network for pattern classification and function approximation," in IJCNN-91-Seattle International Joint Conference on Neural Networks, 1991, pp. 13-18 vol.1.
[8]
T. Takagi and M. Sugeno, "Fuzzy identification of systems and its applications to modeling and control," Systems, Man and Cybernetics, IEEE Transactions on, vol. SMC-15, pp. 116-132, 1985.
[9]
L. Li, Z. Long, H. Ying, and Z. Qiao, "An online gradient-based parameter identification algorithm for the neuro-fuzzy systems," Fuzzy Sets and Systems, vol. 426, pp. 27-45, 2022-01-01 2022.
[10]
T. Gao, Z. Zhang, Q. Chang, X. Xie, P. Ren, and J. Wang, "Conjugate gradient-based Takagi-Sugeno fuzzy neural network parameter identification and its convergence analysis," Neurocomputing, vol. 364, pp. 168-181, 2019.
[11]
Y. Liu, D. Yang, L. Li, and J. Yang, "A split-complex valued gradient-based descent neuro-fuzzy algorithm for TS system and its convergence," Neural Processing Letters, vol. 50, pp. 1589-1609, 2019.
[12]
Y. Lu, W. Li and H. Wang, "A batch variable learning rate gradient descent algorithm with the smoothing L1/2 regularization for Takagi-Sugeno models," IEEE Access, vol. 8, pp. 100185-100193, 2020.
[13]
X. Yu and Q. Chen, "Convergence of gradient method with penalty for ridge polynomial neural network," Neurocomputing, vol. 97, pp. 405-409, 2012.
[14]
H. Shao and G. Zheng, "Boundedness and convergence of online gradient method with penalty and momentum," Neurocomputing, vol. 74, pp. 765-770, 2011.
[15]
H. Shao, J. Wang, L. Liu, D. Xu, and W. Bao, "Relaxed conditions for convergence of batch BPAP for feedforward neural networks," NEUROCOMPUTING, vol. 153, pp. 174-179, 2015.
[16]
Y. Liu and D. Yang, "Convergence analysis of the batch gradient-based neuro-fuzzy learning algorithm with smoothing L1/2 regularization for the first-order Takagi–Sugeno system," Fuzzy Sets & Systems, vol. 319, pp. 28-49, 2016.
[17]
R. Tibshirani, "Regression shrinkage and selection via the lasso," Journal of the Royal Statistical Society Series: B-Methodological, vol. 58, pp. 267-288, 1996.
[18]
T. Zhang, "Approximation bounds for some sparse kernel regression algorithms," Neural Computation, vol. 14, pp. 3013-3042, 2002.
[19]
G. Gnecco and M. Sanguineti, "Regularization techniques and suboptimal solutions to optimization problems in learning from data," Neural Computation, vol. 22, pp. 793-829, 2010.
[20]
T. Zhang, "Analysis of multi-stage convex relaxation for sparse regularization," Journal of Machine Learning Research, vol. 11, pp. 1081-1107, 2010.
[21]
G. Gnecco, R. Morisi and A. Bemporad, "Sparse solutions to the average consensus problem via various regularizations of the fastest mixing markov-chain problem," IEEE Transactions on Network Science and Engineering, vol. 2, pp. 97-111, 2015.
Recommendations
Batch gradient training method with smoothing $$\boldsymbol{\ell}_{\bf 0}$$ℓ0 regularization for feedforward neural networks
This paper considers the batch gradient method with the smoothing $$\ell _0$$ℓ0 regularization (BGSL0) for training and pruning feedforward neural networks. We show why BGSL0 can produce sparse weights, which are crucial for pruning networks. We prove ...
Multi-task l0 gradient minimization for visual tracking
In most object tracking algorithms based on sparse representation, the optimization problem is often formulated as an l1 or l2 minimization problem, because its primal l0-norm minimization problem is NP-hard. In this paper, a visual tracking method is ...
Convergence of online gradient method for feedforward neural networks with smoothing L 1/2 regularization penalty
Minimization of the training regularization term has been recognized as an important objective for sparse modeling and generalization in feedforward neural networks. Most of the studies so far have been focused on the popular L"2 regularization penalty. ...
Comments
Information & Contributors
Information
Published In
January 2022
159 pages
ISBN:9781450396219
DOI:10.1145/3514105
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 April 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
icWCSN 2022
icWCSN 2022: 2022 9th international Conference on Wireless Communication and Sensor Networks
January 11 - 13, 2022
Dalian, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 19Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)0
Reflects downloads up to 04 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format