Abstract
Micron-level flaws on component surface can seriously affect the optical and mechanical properties of optics, so it is significant to accurately classify and repair surface flaws. However, the small scale, multiple categories, and complex characteristics of surface flaws make the accurate classification a great challenge in the field of precision optical engineering. In this work, a novel automated classification approach is proposed for multi-classification of flaws. Images of flaws illuminated by multi-light sources are automatically acquired by classification equipment to enrich the characteristics of flaws, and a series of automated workflows are set up to accomplish flaw detection, classification, and mitigation. SingleCNN-MergeRGB, a feature extraction structure based on a single convolutional neural network (CNN) and multi-light source fusion, is constructed to extract the integrated features of flaws under different illumination. The experimental results show that SingleCNN-MergeRGB has stronger capability of feature description compared with single light source and other feature extraction networks. The integrated deep learning architecture SingleCNN-MergeRGB + SVM and prediction probability threshold adjustment strategy are proposed to improve the accuracy of flaw classification and correct failure cases. Validation results on the test set show that the proposed flaw classification method obtains an accuracy of 96.34% and a recall rate of 100%. The experiments demonstrate the effectiveness of the automatic classification equipment and methods in this work in the classification of surface flaws in large-aperture optical components.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdulla, G. M., Kegelmeyer, L. M., Liao, Z. M., & Carr, W. (2010). Effective and efficient optics inspection approach using machine learning algorithms. In 42nd Annual Laser Damage Symposium, Boulder, CO, 2010, (vol. 7842).
Ammar, M., Daho, M. E., Harrar, K., & Laidi, A. (2022). Feature extraction using CNN for peripheral blood cells recognition. Eai Endorsed Transactions on Scalable Information Systems, 9(34), e12.
Baisden, P. A., et al. (2016). Large optics for the national ignition facility. Fusion Science and Technology, 69(1), 295–351.
Bhardwaj, S., & Mittal, A. (2012). A Survey on Various edge detector techniques. In 2nd International Conference on Computer, Communication, Control and Information Technology (C3IT), India, 2012, (vol. 4, pp. 220–226).
Blagus, R., & Lusa, L. (2013). SMOTE for high-dimensional class-imbalanced data. BMC Bioinformatics. https://doi.org/10.1186/1471-2105-14-106
Conder, A., Chang, J., Kegelmeyer, L., Spaeth, M., & Whitman, P. (2010). Final Optics Damage Inspection (FODI) for the National Ignition Facility. In Conference on Optics and Photonics for Information Processing IV, San Diego, CA, 2010, (vol. 7797).
Geyl, R. (2004). Large lenses, mirrors and filters for astronomy. In Conference on Optical Fabrication, Metrology and Material Advancements for Telescopes, Glasgow, Scotland, 2004, (vol. 5494, pp. 393–397).
He, K. M., Zhang, X. Y., Ren, S. Q., & Sun, J. (2016). Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, 2016, (pp. 770–778).
Jia, A. D. Y., Li, B. Z. Y., & Zhang, C. C. W. (2020). Detection of cervical cancer cells based on strong feature CNN-SVM network. Neurocomputing, 411, 112–127.
Koklu, M., Unlersen, M. F., Ozkan, I. A., Aslan, M. F., & Sabanci, K. (2022). A CNN-SVM study based on selected deep features for grapevine leaves classification. Measurement, 188, 110425.
Kotsiantis, S. B. (2007). Supervised machine learning: A review of classification techniques. Informatica.
Kozlowski, M. R., & Demos, S. G. (2000) .Properties of modified silica detected within laser-induced damage sites. In Conference on Inorganic Optical Materials II, San Diego, Ca, 2000, Bellingham: Spie-Int Soc Optical Engineering, (vol. 4102, pp. 106–111).
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the Acm, 60(6), 84–90.
Li, L., et al. (2014). Automated discrimination between digs and dust particles on optical surfaces with dark-field scattering microscopy. Applied Optics, 53(23), 5131–5140.
Loshchilov, I., & Hutter, F. (2016). SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv preprint.
Ma, C., Dang, H. F., Du, J., He, P. F., Jiang, M. B., & Wei, Z. Y. (2021). Research on automated defect classification based on visual sensing and convolutional neural network-support vector machine for GTA-assisted droplet deposition manufacturing Process. Metals, 11(4), 639.
Natarajan, V., Hung, T. Y., Vaikundam, S., & Chia L. T. (2017). Convolutional Networks for Voting-based Anomaly Classification in Metal Surface Inspection," in IEEE International Conference on Industrial Technology (ICIT), Toronto, Canada, 2017, (pp. 986–991).
Niu, X. X., & Suen, C. Y. (2012). A novel hybrid CNN-SVM classifier for recognizing handwritten digits. Pattern Recognition, 45(4), 1318–1325.
Norton et al. M. A. (2007). Growth of laser damage on the input surface of SiO2 at 351 nm. In Annual Boulder Damage Conference on Laser Induced Damage in Optical Materials, Boulder, CO, 2006 (vol. 6403).
Pedregosa, F., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
Ramaneswaran, S., Srinivasan, K., Vincent, P., & Chang, C. Y. (2021). Hybrid inception v3 XGBoost model for acute lymphoblastic leukemia classification. Computational and Mathematical Methods in Medicine, 2021, 2577375.
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020). Grad-CAM: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision, 128(2), 336–359.
Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(1), 48.
Simonyan, K., & Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. Computer Science.
Singh, K. (2016). Speech recognition: A review of literature. International Journal of Engineering Trends Technology, 37(6), 302–310.
Singh, A., Thakur, N., & Sharma, A. (2016) .A Review of Supervised Machine Learning Algorithms. In 3rd International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India, 2016, (pp. 1310–1315).
Singh, S. A., & Desai, K. A. (2022). Automated surface defect detection framework using machine vision and convolutional neural networks,". Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-021-01878-w
Spaeth, M. L., et al. (2016). Optics recycle loop strategy for NIF operations above UV laser-induced damage threshold. Fusion Science and Technology, 69(1), 265–294.
Stokowski, S., & Vaez-Iravani, M. (1998). Wafer inspection technology challenges for ULSI manufacturing. In International Conference on Characterization and Metrology for ULSI Technology, Gaithersburg, Md, 1998, (vol. 449, pp. 405–415).
Stolz, C. J. (2012). The National ignition facility: The path to a carbon-free energy future. Philosophical Transactions of the Royal Society a-Mathematical Physical and Engineering Sciences, 370(1973), 4115–4129.
Sun, J., Wang, P., Luo, Y. K., & Li, W. (2019). Surface defects detection based on adaptive multiscale image collection and convolutional neural networks. Ieee Transactions on Instrumentation and Measurement, 68(12), 4787–4797.
Szegedy, C. et al. (2015). Going Deeper with Convolutions. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, 2015, (pp. 1–9).
Tao, X., Zhang, Z. T., Zhang, F., & Xu, D. (2015). A novel and effective surface flaw inspection instrument for large-aperture optical elements. Ieee Transactions on Instrumentation and Measurement, 64(9), 2530–2540.
van der Maaten, L. (2014). Accelerating t-SNE using tree-based algorithms. Journal of Machine Learning Research, 15, 3221–3245.
Wei, F. P., et al. (2018). Automatic classification of true and false laser-induced damage in large aperture optics. Optical Engineering, 57(5), 053112.
Xie, S. N., Girshick, R., Dollar, P., Tu, Z. W., & He, K. M. (2017). Aggregated Residual Transformations for Deep Neural Networks. In 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017, (pp. 5987–5995).
Xu, Y., Li, Z. X., Wang, S. Q., Li, W. H., Sarkodie-Gyan, T., & Feng, S. Z. (2021). A hybrid deep-learning model for fault diagnosis of rolling bearings. Measurement, 169, 108502.
Xuan, Q., et al. (2018). Automatic pearl classification machine based on a multistream convolutional neural network. Ieee Transactions on Industrial Electronics, 65(8), 6538–6547.
Yang, Z. X., Tang, L. L., Zhang, K., & Wong, P. K. (2018). Multi-view CNN feature aggregation with ELM auto-encoder for 3D shape recognition. Cognitive Computation, 10(6), 908–921.
Zou, Z., Shi, Z., Guo, Y., & Ye, J. (2019). Object Detection in 20 Years: A Survey.
Funding
This work was supported in part by National Natural Science Foundation of China (Nos. 52235010, 52175389); Consolidation Program for Fundamental Research (No. 2019JCJQZDXX00); Natural Science Foundation of Heilongjiang Province (No. YQ2021E021); Self-Planned Task (Nos. SKLRS201718A, SKLRS201803B) of State Key Laboratory of Robotics and System (HIT).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yin, Z., Chen, M., Zhao, L. et al. A novel automatic classification approach for micro-flaws on the large-aperture optics surface based on multi-light source fusion and integrated deep learning architecture. J Intell Manuf 35, 413–428 (2024). https://doi.org/10.1007/s10845-022-02053-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-022-02053-5