Feature Mining and Sensitivity Analysis with Adaptive Sparse Attention for Bearing Fault Diagnosis
<p>LSTM memory unit.</p> "> Figure 2
<p>(<b>a</b>) Attention structure. (<b>b</b>) Adaptive sparse attention structure.</p> "> Figure 3
<p>A deep learning architecture for fault diagnosis based on the adaptive sparse attention mechanism.</p> "> Figure 4
<p>Flowchart of fault diagnosis method.</p> "> Figure 5
<p>CWRU bearing test bench.</p> "> Figure 6
<p>Test results of each test. (where cps is the diagnostic accuracy with parameter Convolution parameter sharing, cpi is the diagnostic accuracy with parameter Convolution parameter independence, cps-cross is the diagnostic accuracy with parameter cps and across working conditions, and cpi-cross is the diagnostic accuracy with parameter cpi and across working conditions).</p> "> Figure 7
<p>Cross-condition diagnostic performance of different methods under three input data.</p> "> Figure 8
<p>Loss trend graph of the training set.</p> "> Figure 9
<p>The average attention of each method to the bearing envelope spectrum.</p> "> Figure 10
<p>Average attention of each working condition.</p> "> Figure 11
<p>Visualization of attention weights for each working condition.</p> ">
Abstract
:1. Introduction
- A new adaptive sparse attention network, ASAN, is proposed, which uses a soft threshold to filter attention weight sequences and ignores redundant sequences through sparse operation, thus paying more attention to corresponding features.
- Considering the influence of the independent and shared settings of the convolution parameters of each sequence on network performance, the comparison results of the cross-condition diagnosis performance show that the diagnosis method with the independent settings of the convolution parameters of each sequence has better generalization.
- The effectiveness of the proposed algorithm is verified on the CWRU bearing dataset. The attention mechanism mainly captures 1I, 1I side frequency, 6I of IF, 1-3x, 1B, 2B of BF, and 1O, 2O, 3O of OF, which is consistent with the rule of fault diagnosis knowledge (IF is the inner ring fault, BF is the ball fault, OF is the outer ring fault, 1X is the power frequency of the bearing, 1I is one time of the characteristic frequency of the bearing with inner ring fault, and 1B and 1O are the same). The proposed method can locate the fault feature region and has better interpretability and visualization effect.
2. Introduction of Relevant Basic Models
2.1. Convolutional Neural Network
2.2. BiLSTM Network
2.3. Attentional Mechanism
3. The Proposed Adaptive Sparse Attention Network (ASAN)
3.1. Adaptive Sparse Attention Network
3.2. The Network Structure
3.3. The Diagnosis Process
4. Model Analysis and Comparative Validation
4.1. Introduction to Dataset
4.2. Weight Sharing and Weight Independent Comparison Test
4.3. Model Performance Comparison
4.4. Visualization of Fault Information
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Yang, J.; Wu, C.; Shan, Z.; Liu, H.; Yang, C. Extraction and enhancement of unknown bearing fault feature in the strong noise under variable speed condition. Meas. Sci. Technol. 2021, 32, 105021. [Google Scholar] [CrossRef]
- Xiong, X.; Hongkai, J.; Li, X.; Niu, M. A wasserstein gradient-penalty generative adversarial network with deep auto-encoder for bearing intelligent fault diagnosis. Meas. Sci. Technol. 2020, 31, 045006. [Google Scholar] [CrossRef]
- Wang, L.; Zhang, X.; Liu, Z.; Wang, J. Sparsity-based fractional spline wavelet denoising via overlapping group shrinkage with non-convex regularization and convex optimization for bearing fault diagnosis. Meas. Sci. Technol. 2020, 31, 055003. [Google Scholar] [CrossRef]
- Zhang, H.; He, Q. Tacholess bearing fault detection based on adaptive impulse extraction in the time domain under fluctuant speed. Meas. Sci. Technol. 2020, 31, 074004. [Google Scholar] [CrossRef]
- Xiong, Q.; Zhang, X.; Wang, J.; Liu, Z. Sparse representations for fault signatures via hybrid regularization in adaptive undecimated fractional spline wavelet transform domain. Meas. Sci. Technol. 2021, 32, 045107. [Google Scholar] [CrossRef]
- Wang, L.; Xiang, J.; Liu, Y. A time–frequency-based maximum correlated kurtosis deconvolution approach for detecting bearing faults under variable speed conditions. Meas. Sci. Technol. 2019, 30, 125005. [Google Scholar] [CrossRef]
- Gong, J.; Yang, X.; Feng, K.; Liu, W.; Zhou, F.; Liu, Z. An integrated health condition detection method for rolling bearings using time-shift multi-scale amplitude-aware permutation entropy and uniform phase empirical mode decomposition. Meas. Sci. Technol. 2021, 32, 125103. [Google Scholar] [CrossRef]
- Zou, Y.; Shi, K.; Liu, Y.; Ding, G.; Ding, K. Rolling bearing transfer fault diagnosis method based on adversarial variational autoencoder network. Meas. Sci. Technol. 2021, 32, 115017. [Google Scholar] [CrossRef]
- Sun, W.; Yao, B.; Zeng, N.; Chen, B.; He, Y.; Cao, X.; He, W. An intelligent gear fault diagnosis methodology using a complex wavelet enhanced convolutional neural network. Materials 2017, 10, 790. [Google Scholar] [CrossRef] [Green Version]
- Eren, L.; Ince, T.; Kiranyaz, S. A generic intelligent bearing fault diagnosis system using compact adaptive 1d cnn classifier. J. Signal Process. Syst. 2019, 91, 179–189. [Google Scholar] [CrossRef]
- Sohaib, M.; Kim, C.-H.; Kim, J.-M. A hybrid feature model and deep-learning-based bearing fault diagnosis. Sensors 2017, 17, 2876. [Google Scholar] [CrossRef] [Green Version]
- Iannace, G.; Ciaburro, G.; Trematerra, A. Fault diagnosis for uav blades using artificial neural network. Robotics 2019, 8, 59. [Google Scholar] [CrossRef] [Green Version]
- Zuo, L.; Zhang, L.; Zhang, Z.-H.; Luo, X.-L.; Liu, Y. A spiking neural network-based approach to bearing fault diagnosis. J. Manuf. Syst. 2021, 61, 714–724. [Google Scholar] [CrossRef]
- Wen, L.; Li, X.; Gao, L.; Zhang, Y. A new convolutional neural network-based data-driven fault diagnosis method. IEEE Trans. Ind. Electron. 2017, 65, 5990–5998. [Google Scholar] [CrossRef]
- Lipton, Z.C. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 2018, 16, 31–57. [Google Scholar] [CrossRef]
- Liu, Y.; Li, P.; Hu, X. Language. Combining context-relevant features with multi-stage attention network for short text classification. Comput. Speech. Lang. 2022, 71, 101268. [Google Scholar] [CrossRef]
- Chan, W.; Jaitly, N.; Le, Q.; Vinyals, O. In Listen, attend and spell: A neural network for large vocabulary conversational speech recognition. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, 20–25 March 2016; pp. 4960–4964. [Google Scholar]
- Yang, Z.; Yang, D.; Dyer, C.; He, X.; Smola, A.; Hovy, E. Hierarchical attention networks for document classification. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA, 12–17 June 2016; pp. 1480–1489. [Google Scholar]
- Greff, K.; Srivastava, R.K.; Koutník, J.; Steunebrink, B.R.; Schmidhuber, J. Lstm: A search space odyssey. IEEE Trans. Neural. Netw. Learn. Syst. 2016, 28, 2222–2232. [Google Scholar] [CrossRef] [Green Version]
- Yang, Z.-b.; Zhang, J.-P.; Zhao, Z.-b.; Zhai, Z.; Chen, X.-F. Interpreting network knowledge with attention mechanism for bearing fault diagnosis. Appl. Soft. Comput. 2020, 97, 106829. [Google Scholar] [CrossRef]
- Xu, K.; Ba, J.; Kiros, R.; Cho, K.; Courville, A.; Salakhudinov, R.; Zemel, R.; Bengio, Y. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the International Conference on Machine Learning, Lille, France, 6–11 July 2015; pp. 2048–2057. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. Proc. Adv. Neural Inf. Process. Syst. 2017, 30, 5998–6008. [Google Scholar]
- Luong, M.-T.; Pham, H.; Manning, C.D. Effective approaches to attention-based neural machine translation. arXiv 2015, arXiv:1508.04025. [Google Scholar]
- Xue, L.; Li, X.; Zhang, N.L. Not all attention is needed: Gated attention network for sequence data. In Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, 7–12 February 2020; pp. 6550–6557. [Google Scholar]
- Zhao, M.; Zhong, S.; Fu, X.; Tang, B.; Pecht, M. Deep residual shrinkage networks for fault diagnosis. IEEE Trans. Ind. Inform. 2020, 16, 4681–4690. [Google Scholar] [CrossRef]
- Guo, X.; Chen, L.; Shen, C. Hierarchical adaptive deep convolution neural network and its application to bearing fault diagnosis. Measurement 2016, 93, 490–502. [Google Scholar] [CrossRef]
- Sun, W.; Zhao, R.; Yan, R.; Shao, S.; Chen, X. Convolutional discriminative feature learning for induction motor fault diagnosis. IEEE Trans. Ind. Inf. 2017, 13, 1350–1359. [Google Scholar] [CrossRef]
- Li, X.; Zhang, W.; Ding, Q. Deep learning-based remaining useful life estimation of bearings using multi-scale feature extraction. Reliab. Eng. Syst. Saf. 2019, 182, 208–218. [Google Scholar] [CrossRef]
- Ciaburro, G.; Iannace, G. Improving smart cities safety using sound events detection based on deep neural network algorithms. Informatics 2020, 7, 23. [Google Scholar] [CrossRef]
- Fang, W.; Zhang, F.; Sheng, V.S.; Ding, Y. A method for improving cnn-based image recognition using dcgan. CMC-Comput. Mater. Conin. 2018, 57, 167–178. [Google Scholar] [CrossRef]
- Ambrożkiewicz, B.; Litak, G.; Georgiadis, A.; Meier, N.; Gassner, A. Analysis of dynamic response of a two degrees of freedom (2-dof) ball bearing nonlinear model. Appl. Sci. 2021, 11, 787. [Google Scholar] [CrossRef]
- Syta, A.; Czarnigowski, J.; Jakliński, P. Detection of cylinder misfire in an aircraft engine using linear and non-linear signal analysis. Measurement 2021, 174, 108982. [Google Scholar] [CrossRef]
- Zaremba, W.; Sutskever, I.; Vinyals, O. Recurrent neural network regularization. arXiv 2014, arXiv:1409.2329. [Google Scholar]
- Graves, A. Generating sequences with recurrent neural networks. arXiv 2013, arXiv:1308.0850. [Google Scholar]
- Lei, Y.; Jia, F.; Lin, J.; Xing, S.; Ding, S.X. An intelligent fault diagnosis method using unsupervised feature learning towards mechanical big data. IEEE Trans. Ind. Electron. 2016, 63, 3137–3147. [Google Scholar] [CrossRef]
- Sun, W.; Shao, S.; Zhao, R.; Yan, R.; Zhang, X.; Chen, X. A sparse auto-encoder-based deep neural network approach for induction motor faults classification. Measurement 2016, 89, 171–178. [Google Scholar] [CrossRef]
- Zhu, H.; Rui, T.; Wang, X.; Zhou, Y.; Fang, H. Fault diagnosis of hydraulic pump based on stacked autoencoders. In Proceedings of the 2015 12th IEEE International Conference on Electronic Measurement & Instruments (ICEMI), Qingdao, China, 16–18 July 2015; pp. 58–62. [Google Scholar]
- Hinton, G.; Vinyals, O.; Dean, J. Distilling the knowledge in a neural network. arXiv 2015, arXiv:1503.02531. [Google Scholar]
- Smith, W.A.; Randall, R.B. Rolling element bearing diagnostics using the case western reserve university data: A benchmark study. Mech. Syst. Signal Process. 2015, 64–65, 100–131. [Google Scholar] [CrossRef]
- Li, X.; Zhang, W.; Ding, Q. Understanding and improving deep learning-based rolling bearing fault diagnosis with attention mechanism. Signal Process. 2019, 161, 136–154. [Google Scholar] [CrossRef]
Parameters | Value |
---|---|
Number of training | 300 |
Learning rate | 0.001 |
Conv1 convolution kernel size | 3 |
Conv1 number of convolution kernels | 8 |
Conv2 convolution kernel size | 3 |
Conv2 number of convolution kernels | 16 |
LSTM hidden layer unit | 16 |
Dropout | 0.4 |
Description | Functional Parameters |
---|---|
Electric motor | 2 hp (1 hp = 746 W) |
Torque transducer & encoder | Measuring torque and speed |
Electronic controller | Adjusting torque |
Signal type | Vibration acceleration signal |
Data logger | 16 channels |
Sampling frequency | 12,000 Hz |
Measuring bearing position | drive end, fan end, base |
Code of the Working Condition | The Fault Location | The Fault Size | Class Label |
---|---|---|---|
Health | Health | 0 [mil] | 1 |
IF7 | IF | 7 [mil] | 2 |
BF7 | BF | 7 [mil] | 3 |
OF7 | OF | 7 [mil] | 4 |
IF14 | IF | 14 [mil] | 5 |
BF14 | BF | 14 [mil] | 6 |
OF14 | OF | 14 [mil] | 7 |
IF21 | IF | 21 [mil] | 8 |
BF21 | BF | 21 [mil] | 9 |
OF21 | OF | 21 [mil] | 10 |
Load | Speed | IF | BF | OF |
---|---|---|---|---|
0 hp | 1797 [rpm] | 162.19 [Hz] | 141.17 [Hz] | 107.36 [Hz] |
3 hp | 1730 [rpm] | 156.14 [Hz] | 135.90 [Hz] | 103.36 [Hz] |
Parameters | Value |
---|---|
Number of Samples | 500 (100%) |
Number of train data | 300 (60%) |
Number of validation data | 100 (20%) |
Number of test data | 100 (20%) |
Number of features | 8192 |
Number of Classes | 10 |
Experiment Method | Mean | Std |
---|---|---|
CPS | 100.00% | 0.00% |
CPI | 100.00% | 0.00% |
CPS-cross | 50.80% | 3.60% |
CPI-cross | 87.30% | 9.00% |
Diagnosis Method | Input | Diagnostic Accuracy | Diagnostic Accuracy across Working Conditions | Convergence Time (s) |
---|---|---|---|---|
CPI | Envelope spectrum | 1 | 0.873 | 213.0368 |
Global | Envelope spectrum | 1 | 0.980 | 389.7785 |
Local | Envelope spectrum | 1 | 0.764 | 310.0060 |
ASAN | Envelope spectrum | 1 | 0.976 | 196.5402 |
Frequency Range | 0–93.75 | 93.75–187.5 | 187.5–281.25 | 281.25–375 | 375–468.75 | 468.75–562.5 | 843.75–937.5 | 937.5–1031.25 |
---|---|---|---|---|---|---|---|---|
health | 1.000 | 0.762 | 0.055 | 0.010 | 0.007 | 0.009 | 0.581 | 0.440 |
IF | 0.703 | 0.991 | 0.308 | 0.019 | 0.007 | 0.009 | 0.114 | 0.338 |
BF | 0.987 | 0.948 | 0.626 | 0.062 | 0.025 | 0.098 | 0.079 | 0.115 |
OF | 0.670 | 0.996 | 0.486 | 0.214 | 0.056 | 0.035 | 0.098 | 0.171 |
Corresponding eigenfrequency | 1X, 2X, 3X | 1I, 1B, 1O | 2B, 2O | 2B, 2I, 3O | 3B, 4O | 3I, 4B, 5O | 6B, 8O | 6I, 7B, 9O |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Jiang, Q.; Bao, B.; Hou, X.; Huang, A.; Jiang, J.; Mao, Z. Feature Mining and Sensitivity Analysis with Adaptive Sparse Attention for Bearing Fault Diagnosis. Appl. Sci. 2023, 13, 718. https://doi.org/10.3390/app13020718
Jiang Q, Bao B, Hou X, Huang A, Jiang J, Mao Z. Feature Mining and Sensitivity Analysis with Adaptive Sparse Attention for Bearing Fault Diagnosis. Applied Sciences. 2023; 13(2):718. https://doi.org/10.3390/app13020718
Chicago/Turabian StyleJiang, Qinglei, Binbin Bao, Xiuqun Hou, Anzheng Huang, Jiajie Jiang, and Zhiwei Mao. 2023. "Feature Mining and Sensitivity Analysis with Adaptive Sparse Attention for Bearing Fault Diagnosis" Applied Sciences 13, no. 2: 718. https://doi.org/10.3390/app13020718