[go: up one dir, main page]
Skip to main content

Advertisement

SpringerLink
Log in

MaGNIFIES: Manageable GAN Image Augmentation Framework for Inspection of Electronic Systems

  • Published:
Journal of Hardware and Systems Security Aims and scope Submit manuscript

Abstract

Electronic counterfeiting is a long-lasting problem that continues to cost original manufacturers billions, fund organized crime, and jeopardize national security and mission-critical infrastructures. Manual inspection is a popular and standardized way to detect counterfeit electronic components, but it is time-consuming and requires subject matter experts for classification. State-of-the-art machine learning, deep learning, and computer vision-based physical inspection methods are promising to alleviate these issues. However, the main bottleneck for doing so is a lack of high-quality, publicly available counterfeit image data for training. Producing such datasets is also time-consuming and often requires expensive equipment. In addition, most test labs are not allowed to freely publish images taken from their customer’s chips. One solution to this data shortage bottleneck can be addressed by augmenting synthetic data. In this paper, (i) data multiplication using Progressive GAN, StyleGAN, and classical methods in counterfeit data domain is explored; (ii) a novel framework, named MaGNIFIES, is proposed; and (iii) an efficient Convolutional Neural Network architecture is proposed, which can detect defective parts by training only on the synthetic dataset generated using (i) or (ii). For proof of concept, we have used low-quality images of resistors and capacitors with and without scratch defects as counterfeit and golden components respectively. We have also illustrated how our approach using MaGNIFIES addresses the shortcomings of the existing augmentation methods. Separate data augmentation detection models are trained with each type of augmented data generated using MaGNIFIES, as well as existing techniques, and tested on a test set of real data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Mehta D, Lu H, Paradis OP et al (2020) The big hack explained: detection and prevention of pcb supply chain implants. ACM J Emerg Technol Comput Syst (JETC) 16(4):1–25

    Article  Google Scholar 

  2. Karazuba P (2020) Combating counterfeit chips. https://semiengineering.com/combating-counterfeit-chips/

  3. Frontier economics (2016) The economic impacts of counterfeiting and piracy. Report prepared for BASCAP and INTA. International Chamber of Commerce

  4. Daniel B (2020) Counterfeit electronic parts: A multibillion-dollar black market. https://www.trentonsystems.com/

  5. Cardoso B (2021) The dark side of the chip shortage: Counterfeits. X-ray News

  6. Guin U, DiMase D, Tehranipoor M (2014) Counterfeit integrated circuits: Detection, avoidance, and the challenges ahead. J Electron Test 30(1):9–23

    Article  Google Scholar 

  7. Guin U, Forte D, Tehranipoor M (2013) Anti-counterfeit techniques: From design to resign. 2013 14th International workshop on microprocessor test and verification. IEEE, pp 89–94

  8. Ghosh P, Chakraborty RS (2019) Recycled and remarked counterfeit integrated circuit detection by image-processing-based package texture and indent analysis. IEEE Trans Industr Inf 15(4):1966–1974. https://doi.org/10.1109/TII.2018.2860953

    Article  Google Scholar 

  9. Ghosh P, Chakraborty RS (2017) Counterfeit IC detection by image texture analysis. 2017 Euromicro Conference on Digital System Design (DSD), pp 283–286. https://doi.org/10.1109/DSD.2017.10

  10. Ghosh P, Bhattacharya A, Forte D et al (2019) Automated defective pin detection for recycled microelectronics identification. Journal of Hardware and Systems Security 3(3):250–260

    Article  Google Scholar 

  11. Ghosh P, Botero UJ, Ganji F, Woodard D, Chakraborty RS, Forte D (2020) Automated detection and localization of counterfeit chip defects by texture analysis in infrared (ir) domain. 2020 IEEE Physical Assurance and Inspection of Electronics (PAINE). IEEE, pp 1–6

  12. Ghosh P, Forte D, Woodard DL, Chakraborty RS (2018) Automated detection of pin defects on counterfeit microelectronics. ISTFA 2018: Proceedings from the 44th International Symposium for Testing and Failure Analysis. ASM International, p 57

  13. Ghosh P, Ganji F, Forte D et al (2019) Automated framework for unsupervised counterfeit integrated circuit detection by physical inspection

  14. Asadizanjani N, Tehranipoor M, Forte D (2017) Counterfeit electronics detection using image processing and machine learning. J Phys Conf Ser 787:012023. IOP Publishing

    Article  Google Scholar 

  15. Mahmood K, Carmona PL, Shahbazmohamadi S et al (2015) Real-time automated counterfeit integrated circuit detection using x-ray microscopy. Appl Opt 54(13):D25–D32

    Article  Google Scholar 

  16. Shahbazmohamadi S, Forte D, Tehranipoor M (2014) Advanced physical inspection methods for counterfeit IC detection. ISTFA 2014: Conference Proceedings from the 40th International Symposium for Testing and Failure Analysis. ASM International, p 55

  17. Kuo C-W, Ashmore JD, Huggins D, Kira Z (2019) Data-efficient graph embedding learning for pcb component detection. 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, pp 551–560

  18. Lu H, Mehta D, Paradis OP et al (2020) Fics-pcb: A multi-modal image dataset for automated printed circuit board visual inspection. IACR Cryptol ePrint Arch 2020:366

    Google Scholar 

  19. Jessurun N, Dizon-Paradis OP, Harrison J, Ghosh S, Tehranipoor MM, Woodard DL, Asadizanjani N (2022) FPIC: a novel semantic dataset for optical PCB assurance. arXiv preprint arXiv:2202.08414

  20. Nathan Jessurun, Daniel E. Capecci, Olivia P. Dizon Paradis et al (2022) Semi-supervised semantic annotator (S3A): toward efficient semantic labeling. In: Agarwal M, Calloway C, Niederhut D, Shupe D (eds) Proceedings of the 21st Python in Science Conference, pp 7–12. https://doi.org/10.25080/majora-212e5952-001

  21. Fridman Y, Rusanovsky M, Oren G (2021) Changechip: A reference-based unsupervised change detection for pcb defect detection. 2021 IEEE physical assurance and inspection of electronics (PAINE). IEEE, pp 1–8

  22. Ganapathy P, Gupta A (2021) Defect detection and classification in manufacturing using Amazon Lookout for Vision and Amazon Rekognition Custom Labels. AWS Machine Learning Blog. https://aws.amazon.com/blogs/machine-learning/defect-detection-and-classification-in-manufacturing-using-amazon-lookout-for-vision-and-amazon-rekognition-custom-labels/

  23. Huang W, Wei P (2019) A pcb dataset for defects detection and classification. arXiv preprint arXiv:1901.08204

  24. Karanth N (2022) PCBexperiment. https://www.kaggle.com/datasets/namrathakaranth/pcbexperiment

  25. Anzai Y (2012) Pattern recognition and machine learning. Elsevier

    Google Scholar 

  26. Goodfellow I, Pouget-Abadie J, Mirza M et al (2020) Generative adversarial networks. Commun ACM 63(11):139–144

    Article  MathSciNet  Google Scholar 

  27. Hindupur A (2018) The gan zoo. GitHub. https://github.com/hindupuravinash/the-gan-zoo/blob/master/gans.tsv

  28. Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196

  29. Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J, Aila T (2020) Analyzing and improving the image quality of stylegan. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8110–8119

  30. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134

  31. Noble WS (2006) What is a support vector machine? Nat Biotechnol 24(12):1565–1567

    Article  Google Scholar 

  32. Wright RE (1995) Logistic regression. American Psychological Association, pp 217–244

    Google Scholar 

  33. Ramchoun H, Ghanou Y, Ettaouil M, Janati Idrissi MA (2016) Multilayer perceptron: Architecture optimization and training. Int J Interact Multimed Artif Intell

  34. Karras T, Laine S, Aila T (2018) A style-based generator architecture for generative adversarial networks. CoRR abs/1812.04948. https://arxiv.org/abs/arXiv:1812.04948

  35. Sathiaseelan MAM, Paradis OP, Mehta D, Lu H, Agrawal S, Roberts A, Jessurun N, Woodard DL, Chawla P, Tehranipoor M, Asadi N (2016) PCB images. TrustHub. https://trust-hub.org/#/data/PCB-Images

  36. Mittal A, Moorthy AK, Bovik AC (2011) Blind/referenceless image spatial quality evaluator. 2011 conference record of the forty fifth asilomar conference on signals, systems and computers (ASILOMAR). IEEE, pp 723–727

  37. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708. IEEE

    Article  MathSciNet  Google Scholar 

  38. Mittal A, Soundararajan R, Bovik AC (2012) Making a “completely blind” image quality analyzer. IEEE Signal Process Lett 20(3):209–212. IEEE

    Article  Google Scholar 

  39. Ferzli R, Karam LJ (2009) A no-reference objective image sharpness metric based on the notion of just noticeable blur (jnb). IEEE Trans Image Process 18(4):717–728

    Article  MathSciNet  Google Scholar 

  40. Birdal T (2021) Sharpness estimation from image gradients. MATLAB Central File Exchange. IEEE. https://www.mathworks.com/matlabcentral/fileexchange/32397-sharpness-estimation-from-image-gradients

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Florida, 968 Center Dr, Gainesville, 32603, FL, USA

    Pallabi Ghosh, Gijung Lee, Mengdi Zhu, Olivia P. Dizon-Paradis, Damon L. Woodard & Domenic Forte

  2. MIT Lincoln Laboratory, 244 Wood St, Lexington, 02421, MA, USA

    Ulbert J. Botero

Authors
  1. Pallabi Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gijung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengdi Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Olivia P. Dizon-Paradis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ulbert J. Botero
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Damon L. Woodard
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Domenic Forte
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Pallabi Ghosh or Domenic Forte.

Ethics declarations

Funding

The work was supported by National Science Foundation (NSF) Awards — 1821780 and 2131480.

Ethics Approval

Not applicable.

Competing Interests

Not applicable.

Author Contributions

Dr. Domenic Forte and Dr. Damon L. Woodard provided supervision throughout the work. Data labeling done by Pallabi Ghosh and Gijung Lee. Coding and data generation done by Pallabi Ghosh, Gijung Lee and Mengdi Zhu. Pallabi Ghosh wrote the manuscript. Olivia P. Dizon-Paradis provided supervision throughout the writing process and contributed text in the literature reviews to the paper. All authors reviewed the paper.

Availability of Data and Materials

Dataset and Code will be made available in Github after acceptance of the paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghosh, P., Lee, G., Zhu, M. et al. MaGNIFIES: Manageable GAN Image Augmentation Framework for Inspection of Electronic Systems. J Hardw Syst Secur 8, 44–59 (2024). https://doi.org/10.1007/s41635-024-00145-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41635-024-00145-7

Keywords

Search

Navigation