Background
With the development of industries such as electronic products and communication equipment, the soldering technology plays an important role in various industries. The welding gun main board is used as a core component of welding equipment, the production quality of the welding gun main board directly influences the welding effect and the quality of products, in the traditional welding gun main board production, the problems of high cost, low efficiency, difficulty in finding tiny defects and the like exist in manual detection, defective products enter the market, the quality of the products and brand image are influenced, and along with the continuous progress of an image processing technology, the popularization of an automatic detection system can improve the production efficiency, reduce the production cost and improve the production consistency and the stability. In the welding process, a cold joint defect is generated due to improper temperature control or too short welding time, and the cold joint defect refers to that the surface of a welding spot seems to be welded, but the actual internal connection is poor, which can lead to unstable electric contact between a pin of an electronic element and a welding disc and even complete disconnection, thereby influencing the welding quality and the product performance. The existing method for detecting the potential false welding defects on the surface of the welding gun main board is a maximum entropy method, can automatically calculate an optimal threshold value, is suitable for complex images, and has strong self-adaptability.
The patent application document with the publication number of CN115760830A provides a fabric flaw detection method, which comprises the steps of denoising a fabric image by adopting a self-adaptive median filter, carrying out image enhancement on the image by adopting a fuzzy domain image enhancement method, calculating edge gradient amplitudes of the image in the horizontal, vertical, 45-degree and 135-degree directions by using a Sobel edge difference operator, carrying out non-maximum suppression by adopting a linear interpolation method according to the edge gradient amplitude, segmenting and fusing the image by adopting an iteration method and a maximum entropy method, calculating a high-low threshold value of the compressed image, and extracting the flaw image edge after suppressing isolated pixel points according to the high-low threshold value.
However, flux residues may exist around the welding spot on the surface of the welding gun main board, and although the flux residues do not affect the reliability of electrical connection, the flux residues may bring similar appearance to the welding spot surface, so that the accuracy of detection of potential false welding defects is affected, meanwhile, the basic idea of the maximum entropy method in image segmentation is to determine a proper threshold value by maximizing the entropy value of an image gray level histogram, in general, the maximum entropy method uses a global threshold value, namely, the gray level histogram of the whole image is analyzed, and a global threshold value is selected for image segmentation, but when a fixed global threshold value is used for segmentation of each segmented area on the image, the segmentation result is inaccurate, and important information is lost.
Disclosure of Invention
In order to solve the problem that important information is lost due to inaccurate segmentation results of each segmented image of an image by using a fixed global threshold when a maximum entropy method is used and the accuracy of cold joint detection is reduced due to residual soldering flux, the invention provides a method and a system for detecting defects of a welding gun main board based on image processing.
In a first aspect, the present invention provides a method for detecting defects of a welding gun motherboard based on image processing, which adopts the following technical scheme:
A welding gun main board defect detection method based on image processing comprises the steps of extracting gray images of a welding gun main board surface image by means of semantic segmentation to obtain a main board area image, equally dividing the main board area image into a plurality of image blocks according to preset sizes, recording any one of the image blocks as a target image block, calculating the gray level value of each gray pixel point in the target image block, the gray average value of the target image block and the variance of the number of the gray pixels corresponding to each gray level in a gray level histogram corresponding to the target image block, obtaining all edges and edge pixel points of each image block by means of edge detection, identifying the directions of the edge pixel points by means of chain codes to obtain the direction number value of the edge pixel points, calculating the potential false welding degree of the target image block based on the gray level value of each gray pixel point in the target image block and the difference of the direction number value of all adjacent two edge pixel points on each edge of the target image block, and determining the potential false welding defect degree of each false welding point in sequence by means of the chain code to obtain the maximum false welding defect detection threshold value of each welding gun defect region.
The method has the advantages that fine defects on the surface of the welding gun main board, such as false welding, can be effectively identified through gray level difference degree analysis on the image blocks and combining an edge detection method and a chain code identification method, so that accuracy of the false welding detection is improved, influences of residual soldering flux on detection results can be effectively reduced through gray level value, mean value and histogram variance analysis, misjudgment and missed judgment caused by residual substances are avoided, each defect block is subjected to self-adaptive threshold segmentation through a maximum entropy method, segmentation threshold can be dynamically adjusted according to characteristics of different image areas, information loss caused by a fixed global threshold is avoided, accurate defect area identification is guaranteed, quality of the welding gun main board can be estimated from multiple dimensions through comprehensive analysis of gray level difference, edge direction and potential false welding degree, and accordingly defect detection comprehensiveness and accuracy are improved.
Further, the semantic segmentation uses DeepLab models.
The deep neural network model for semantic segmentation has the advantages that the deep neural network model for semantic segmentation can accurately segment the surface of the welding gun main board and accurately identify the defect area, has stronger adaptability to complex backgrounds, and ensures stable performance under complex conditions.
Further, the gray scale difference degree satisfies the following relation:
In the formula (I), in the formula (II), Is the firstThe degree of gray scale difference of individual image patches,Is the firstThe number of gray-scale pixels within a single image tile,Is the firstIntra-block first of individual imagesThe gray value of each gray pixel point,Is the firstThe gray-scale average of each image patch,Is the firstThe image blocks correspond to the variance of the number of gray pixel points corresponding to each gray level in the gray histogram,Is an absolute value sign.
The method has the advantages that local gray level change can be accurately captured through calculating the difference between the gray level value and the gray level average value of each pixel point in the image block, the image analysis accuracy is improved, the gray level histogram variance is combined to weight the change of gray level distribution, the area with larger gray level change is more prominent, the sensitivity to the fine difference is increased, noise and real change are effectively distinguished through absolute value difference calculation, false detection is reduced, and the robustness is improved.
Further, the edge detection adopts a Canny operator.
Further, the chain code adopts an 8-neighborhood chain code.
Further, the potential cold joint degree satisfies the following relation:
In the formula (I), in the formula (II), Is the firstThe degree of potential cold joint of the individual image tiles,Is the firstThe degree of gray scale difference of individual image patches,Is the firstThe number of categories of direction number values of edge pixels of the individual image patches,Is the firstThe number of edges of the individual image tiles,Is the firstThe first image blockThe number of edge pixels on the strip edge,AndRespectively the firstThe first image blockFirst on the strip edgeAnd (b)The direction number value of each edge pixel point,As a function of the normalization,Is an absolute value sign.
The method has the advantages that potential false soldering areas can be effectively identified by combining gray level differences, edge quantity and edge direction changes, so that the accuracy of false soldering detection is improved, the gray level differences and edge characteristics are comprehensively considered, the detection process is more comprehensive, fine changes in images can be accurately captured, and the scale differences of different image blocks are reduced by normalization processing, so that the detection results are more stable and consistent.
Further, determining the defect block according to the potential cold joint degree comprises determining the image block as the defect block in response to the potential cold joint degree being greater than a preset defect threshold.
The method has the advantages that the defect threshold is set, when the potential cold joint degree exceeds the value, the potential cold joint degree is determined to be the defect block, erroneous judgment and missed judgment are effectively avoided, the image block can be automatically divided into the defect area and the non-defect area, the detection efficiency is improved, and manual intervention is reduced.
In a second aspect, the present invention provides a welding gun motherboard defect detection system based on image processing, which adopts the following technical scheme:
The welding gun main board defect detection system based on image processing comprises a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are executed by the processor to realize the welding gun main board defect detection method based on the image processing.
By adopting the technical scheme, the welding gun main board defect detection method based on image processing generates a computer program, and the computer program is stored in the memory to be loaded and executed by the processor, so that terminal equipment is manufactured according to the memory and the processor, and the welding gun main board defect detection method based on image processing is convenient to use.
The invention has the following technical effects:
The main board area image of the welding gun is subjected to block processing, gray features are independently analyzed in each block image, so that the potential rosin joint degree is estimated, and the image segmentation is carried out by independently applying a maximum entropy method to each block with the large potential rosin joint degree, so that the subtle changes of the areas can be captured more accurately, the appearance rosin joint caused by the factors of the rosin flux residue and the like and the problem of inaccurate segmentation caused by using the global threshold are avoided, the segmentation effect and the sensitivity of defect detection are improved, and the risk of important information loss is reduced.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be understood that when the terms "first," "second," and the like are used in the claims, the specification and the drawings of the present invention, they are used merely for distinguishing between different objects and not for describing a particular sequential order. The terms "comprises" and "comprising" when used in the specification and claims of the present invention are taken to specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The invention aims to perform threshold segmentation treatment on potential virtual welding defects on the surface of a welding gun mainboard by using a maximum entropy method, perform blocking treatment on the welding gun mainboard image before segmentation, and screen out each image block needing to be processed by using the maximum entropy method independently according to the analysis of gray features in each image block.
The embodiment of the invention discloses a welding gun main board defect detection method based on image processing, which comprises the following steps of S1-S6 with reference to FIG. 1:
and S1, extracting a gray level image of a surface image of a mainboard of the welding gun by utilizing semantic segmentation to obtain a mainboard area image.
Specifically, the semantic segmentation uses DeepLab models.
The method comprises the steps of carrying out vertical shooting on the surface of a welding gun mainboard through a high-definition camera to obtain a welding gun mainboard surface image, carrying out gray-scale treatment on the welding gun mainboard surface image to obtain a gray-scale image of the welding gun mainboard surface image, and extracting a welding gun mainboard region in the welding gun mainboard surface image by using a deep convolutional neural network version 3 (Deep Convolutional Neural Network for SEMANTIC IMAGE Segmentation version, deep LabV 3) model for semantic image segmentation in order to facilitate analysis of gray-scale characteristics of the welding gun mainboard surface in the subsequent steps.
S2, equally dividing the image of the main board area into a plurality of image blocks according to a preset size.
The image blocking processing is performed on the welding gun main board area image so as to facilitate the analysis of the subsequent steps, and the blocking rule of the welding gun main board area image is that the welding gun main board area image is equally divided into a plurality of image blocks, and gray pixel points which are positioned in the edge area of the welding gun main board and are insufficient for constructing one image block are integrated into the image block closest to the Euclidean distance of the gray pixel points for analysis.
The practitioner can set the size of the image tiles, e.g., 7 x 7, depending on the particular implementation.
And S3, marking any image block as a target image block, and calculating the gray level difference degree of the target image block.
When the index is analyzed, the more the gray value of the gray pixel point in each image block deviates from the gray average value in the block and the larger the number difference of the corresponding gray pixel point in each gray level in the gray histogram, the more the gray value distribution in the image block is scattered, the larger the gray difference degree is, and the possibility of potential false welding defects in the image block is increased.
And calculating the gray level difference degree of the target image block based on the gray level value of each gray level pixel point in the target image block, the gray level average value of the target image block and the variance of the number of gray level pixel points corresponding to each gray level in the gray level histogram corresponding to the target image block.
Specifically, the gray scale difference degree satisfies the following relation:
;
In the formula, Is the firstThe degree of gray scale difference of individual image patches,Is the firstThe number of gray-scale pixels within a single image tile,Is the firstIntra-block first of individual imagesThe gray value of each gray pixel point,Is the firstThe gray-scale average of each image patch,Is the firstThe image blocks correspond to the variance of the number of gray pixel points corresponding to each gray level in the gray histogram,Is an absolute value sign.
Wherein, Represent the firstThe larger the gray value of all gray pixel points in each image block is compared with the gray average value of the image block, the larger the value is, which means that the larger the offset is, which can be described as the firstThe more discrete the gray value distribution in each image block, the greater the gray difference degree, and the greater the possibility of potential cold joint defects in the image block; The larger can be explained by The larger the difference of the number of the corresponding pixel points on each gray level in the gray level histogram of each image block, the more the difference can be verifiedThe larger the deviation of the gray value of all gray pixel points in each image block compared with the gray average value of the image block, the higher the reliability, namely the description of the firstThe more discrete the gray value distribution within each image block, the greater the gray difference degree, and the further description of the firstThe greater the likelihood of potential cold joint defects in the individual image segments.
And S4, calculating the potential cold joint degree of the target image block.
It should be noted that, in order to distinguish the potential cold joint defect from the residual soldering flux region in the surface image of the welding gun main board, the gray scale variation characteristic in each image block is further analyzed, and then the gray scale difference degree in each image block is analyzed and calculated to obtain the potential cold joint degree in each image block, before the index analysis, the welding point with the potential cold joint usually has irregular or incomplete edges, possibly has a missing part, which causes the non-smooth, incomplete or obvious fracture of the edge of the welding point, while the residual soldering flux usually does not change the edge shape of the welding point, and has no obvious fracture or abrupt change, so that when the potential cold joint degree in each image block is analyzed, the difference value of the direction numbers between the adjacent pixel points in all edges in one image block is larger, the greater the difference value of the direction numbers between the adjacent pixel points in one image block is, the greater the potential cold joint defect can be indicated, and the greater the potential cold joint defect can be corresponding to the greater the potential cold joint degree.
And calculating the potential false welding degree of the target image block based on the gray level difference degree, the edge direction number of the target image block and the difference between the direction number values of all adjacent two edge pixel points on each edge of the target image block, wherein the edge direction number is the kind number of all the direction number values in the target image block.
Specifically, the edge detection adopts a Canny operator.
Specifically, the chain code adopts an 8-neighborhood chain code.
Specifically, the potential cold joint degree satisfies the following relation:
;
In the formula, Is the firstThe degree of potential cold joint of the individual image tiles,Is the firstThe degree of gray scale difference of individual image patches,Is the firstThe number of categories of direction number values of edge pixels of the individual image patches,Is the firstThe number of edges of the individual image tiles,Is the firstThe first image blockThe number of edge pixels on the strip edge,AndRespectively the firstThe first image blockFirst on the strip edgeAnd (b)The direction number value of each edge pixel point,As a function of the normalization,Is an absolute value sign.
Wherein, The larger the firstThe larger the gray level difference degree of each image block is, the greater the possibility that potential cold joint defects exist in the image block is, and the greater the corresponding potential cold joint degree is; the larger the gray level variation within the one image patch, the more the possibility of the potential cold joint defect within the one image patch, the greater the corresponding potential cold joint degree, the more the image patch without the potential cold joint defect, The potential false soldering degree is 0, and the image block does not need to be processed by a maximum entropy method independently in the follow-up process; The larger the difference value of the direction numbers between the adjacent two pixel points on all edges in one image block can be described, the more disordered the gray level change in the one image block can be verified, and the greater the potential cold joint degree of the one image block can be further described.
And S5, determining defect blocking according to the potential cold joint degree.
Specifically, the determining the defect block according to the magnitude of the potential cold joint degree includes:
and in response to the potential cold joint degree being greater than a preset defect threshold, identifying the image block as a defect block.
The practitioner may set a defect threshold, for example, 0.5, depending on the particular implementation.
And S6, sequentially carrying out threshold segmentation processing on each defect block by using a maximum entropy method to obtain a virtual welding area of each defect block, and finishing the defect detection of the welding gun main board based on image processing.
For image segmentation without independent maximum entropy processing, as the potential cold joint degree of the areas is lower, the potential cold joint defects are not likely to exist, meanwhile, obvious gray level change does not exist in the areas, the distribution of gray level histograms is generally smooth, the areas do not need to be processed by the maximum entropy processing, and for all the defect segmentation with independent maximum entropy processing, threshold segmentation processing is performed by the maximum entropy processing, the corresponding segmentation threshold values are obtained, and the cold joint areas are obtained, wherein the obtained cold joint areas are the potential cold joint defects on the surface of a welding gun main board. The threshold segmentation process of the specific maximum entropy method is a known technology, and will not be described in detail herein.
The embodiment of the invention also discloses a welding gun main board defect detection system based on image processing, which comprises a processor and a memory, wherein the memory stores computer program instructions, and the welding gun main board defect detection method based on the image processing is realized when the computer program instructions are executed by the processor.
The above system further comprises other components well known to those skilled in the art, such as a communication bus and a communication interface, the arrangement and function of which are known in the art and therefore are not described in detail herein.
In the context of this patent, the foregoing memory may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium may be any suitable magnetic or magneto-optical storage medium, or any other medium that can be used to store the desired information and that can be accessed by an application, a module, or both. Any such computer storage media may be part of, or accessible by, or connectable to, the device.
While various embodiments of the present invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Many modifications, changes, and substitutions will now occur to those skilled in the art without departing from the spirit and scope of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in practicing the invention.
The above embodiments are not intended to limit the scope of the invention, so that the equivalent changes of the structure, shape and principle of the invention are covered by the scope of the invention.