CN117437636B - Method and system for improving defect labeling effect based on image comparison - Google Patents

Abstract

The invention discloses a method and a system for improving defect labeling effect based on image comparison, and relates to the technical field of image processing, wherein the method comprises the following steps: obtaining a standard image of a standard product and a target image of a product to be detected, and dividing the standard image and the target image into a plurality of sub-standard images and a plurality of sub-target images respectively; acquiring a first component element in each sub-standard image and a second component element in each sub-target image; obtaining a reconstructed image based on the first component element and the second component element; performing difference on the standard image and the reconstructed image to obtain a structural similarity index set; obtaining a plurality of defect images based on the set of structural similarity indexes; performing defect extraction on each defect image to obtain a plurality of defect coordinates; and marking the target image based on all the defect coordinates and the reconstruction images, so that the problem that the defect areas cannot be accurately marked when the non-defect areas have a slight gap can be solved.

Description

Method and system for improving defect labeling effect based on image comparison

Technical Field

The invention relates to the technical field of image processing, in particular to a method and a system for improving defect labeling effect based on image comparison.

Background

PCB (PrintedCircuitBoard) the Chinese name printed circuit board, also called printed circuit board, printed circuit board is an important electronic component, which is a support for electronic components and is a provider for electrical connection of electronic components. It is called a printed circuit board because it is made using electronic printing. In order to ensure the quality of the PCB, it is important to extract marks of defects on the PCB.

The prior Chinese patent with the patent number of CN115984244A discloses a panel defect marking method, a device, a storage medium, equipment and a program product, wherein the method comprises the following steps: generating masks on the target image respectively to obtain a first mask image and a second mask image; the mask comprises a first mask and a second mask, wherein the first mask is positioned on the first mask image, the second mask is positioned on the second mask image, the first mask and the second mask cover different areas on the target image respectively, and the areas where the first mask and the second mask are positioned can completely cover defects on the target image; repairing the first mask image and the second mask image respectively, so that the area where the mask is positioned is repaired into a defect-free product image corresponding to the target image, and a first repairing image and a second repairing image are obtained; respectively carrying out pixel difference on the first repair image and the second repair image and the target image to obtain a first difference image and a second difference image; combining the defect information on the first difference image and the second difference image to obtain target defect information; and marking the defects on the target image according to the target defect information.

Although the problem of lower quality of defect labeling is solved by the technical scheme of the patent, a defect-free product image is obtained by an image restoration technology and is inaccurate, and when a restoration image and a template image or an original image have a slight gap in a non-detection area, a great amount of interference can be generated by directly adopting a scheme of making a difference between pixels, so that a defect area which is really needed to be detected cannot be well extracted, and the defect area cannot be accurately labeled.

Disclosure of Invention

In order to solve the problem that when a template diagram and a product diagram have a slight gap in a non-detection area, a difference making scheme is directly adopted to generate a large amount of interference, so that a defect area cannot be accurately marked. The invention provides a method for improving defect labeling effect based on image comparison, which comprises the following steps:

s1, acquiring a standard image of a standard product and a target image of a product to be detected; dividing the standard image and the target image into a plurality of sub-standard images and a plurality of sub-target images, wherein the sub-standard images and the sub-target images are in one-to-one correspondence;

s2, acquiring a plurality of first component elements in each sub-standard image, wherein each first component element comprises a first lead, a first component, a first via hole and a first copper layer, and each first component element corresponds to a first element coordinate and a preset range threshold; obtaining a simulation standard image based on all the first element coordinates, all the preset range thresholds and all the sub-standard images; obtaining first coordinates of a defect based on the simulated standard image and the target image;

S3, acquiring a plurality of second component elements in each sub-target image, wherein each second component element comprises a second wire, a second component, a second via hole and a second copper layer, each second component element corresponds to a second component coordinate, and each first component element corresponds to a second component element; obtaining a reconstructed image based on all of the first element coordinates and all of the second element coordinates;

s4, dividing the reconstructed image into a plurality of sub-reconstructed images, wherein the sub-reconstructed images correspond to the sub-standard images one by one; performing difference on the standard image and the reconstructed image to obtain a structure similarity index set, wherein the structure similarity index set comprises a plurality of structure similarity indexes;

s5, judging whether the structural similarity index in the structural similarity index set is smaller than a preset index; if yes, acquiring a plurality of sub-reconstruction images corresponding to the plurality of structural similarity indexes, and acquiring a plurality of defect images; performing defect extraction on each defect image to obtain a plurality of defect coordinates;

and S6, labeling the target image based on the first coordinates, all the defect coordinates and the reconstruction image.

The principle of the invention is as follows: the method comprises the steps of obtaining a standard image of a standard product and a target image of a product to be detected, dividing the standard image and the target image into a sub-standard image and a sub-target image which are in one-to-one correspondence, obtaining a first component element on each sub-standard image, and maximizing the size of the first component element on the image to obtain a simulation standard image through a preset range threshold value and coordinates corresponding to the first component element, wherein the preset range threshold value refers to a check standard of the first component element, such as a qualified range of the position, thickness and size of a component; comparing the simulation standard image with the target image to obtain a first component element which does not meet the inspection standard, and obtaining a coordinate corresponding to the first component element, namely a first coordinate of the defect; acquiring a second component element and corresponding coordinates thereof on each sub-target image, moving the second component element to the corresponding coordinates of the first component element on each sub-target image, and aligning the second component element with the position of the first component element so as to acquire a reconstructed image; dividing the reconstructed image into a plurality of sub-reconstructed images corresponding to the standard images, and performing difference between the reconstructed image and the corresponding sub-standard images to obtain a plurality of structural similarity indexes, wherein all the structural similarity indexes obtain a structural similarity index set; judging whether the structural similarity index in the structural similarity index set is smaller than a preset index or not; if yes, obtaining a sub-reconstruction image corresponding to the structural similarity index, so as to obtain a defect image; carrying out defect extraction on each defect image to obtain a plurality of defect coordinates; and labeling the target image through the corresponding relation among the first coordinates, all the defect coordinates and the reconstructed image.

Maximizing the size of the first component element on the image according to the detection standard to obtain a simulation standard image, and comparing the simulation standard image with the target image to obtain a first component element which does not meet the detection standard, so that the component element which does not meet the detection standard can be found; the second component element is aligned with the first component element, so that interference caused by position deviation can be reduced in the subsequent difference making step, a defect area which is really needed to be detected can be better extracted, the target image is restored by adopting an image restoration technology and then difference making is carried out, the condition that the target image restored by adopting the image restoration technology is possibly wrong in restoration is inaccurate, the method adopts comparison with a standard image, only the positions of the component elements are moved, the shape of the component elements is not changed, and the like, so that the accuracy is improved, and the error is reduced; the standard image and the reconstructed image are subjected to difference, so that interference of a non-detection area is reduced, the structural similarity index is more accurate, and the accuracy of extracting the defect area is improved.

In order to maximize the size of the first component element, the method acquires the maximum value and the minimum value in the threshold value of the preset range, and enlarges the size based on the maximum value and the minimum value, so that the component element which does not meet the detection standard can be better detected.

Further, in step S2, the specific step of obtaining the simulated standard image based on all the first element coordinates, all the preset range thresholds, and all the sub-standard images includes:

obtaining an extremum in each preset range threshold, wherein the extremum comprises a maximum value and a minimum value; obtaining a plurality of extreme value coordinates corresponding to all the first component elements based on all the first element coordinates and all the extreme values; the simulated standard image is obtained based on all the extremum coordinates and all the first constituent elements.

In order to obtain the component elements which do not meet the detection standard more obviously, the method sets the simulation standard image and the target image to be different colors respectively, and then fuses the simulation standard image and the target image, so that the component elements which do not meet the detection standard can be obtained more accurately through color distinction.

Further, in step S2, the specific step of obtaining the first coordinates of the defect based on the simulated standard image and the target image includes:

dividing the simulation standard image into a plurality of sub-simulation images, wherein the sub-simulation images correspond to the sub-target images one by one; setting all the sub-simulation images and all the sub-target images to a first color and a second color respectively; fusing each sub-simulation image and the sub-target image corresponding to the sub-simulation image to obtain a fused image; the first coordinates of the defect are obtained based on the fused image.

Considering that other colors may be generated by overlapping colors, the method divides the sub-target image and the sub-simulation image into two image layers, places the sub-target image on the bottom layer, covers the sub-simulation image on the sub-target image, fuses the sub-target image to obtain a fused image, and if the component elements on the sub-target image do not meet the detection standard, the corresponding colors are displayed on the fused image, and the defect area and coordinates can be accurately positioned by acquiring the colors.

Further, the specific step of fusing each sub-simulation image and the sub-target image corresponding to the sub-simulation image to obtain a fused image includes: and placing each sub-target image on a bottom layer, and overlaying the sub-simulation image corresponding to the sub-target image on the sub-target image to fuse to obtain the fused image.

Further, the specific step of obtaining the first coordinates of the defect based on the fused image includes:

and judging whether the second color exists on the fusion image, if so, acquiring coordinates corresponding to the second color to acquire the first coordinates.

After the positions of the component elements are considered to be aligned, the sizes of the component elements do not correspond to each other and a large amount of interference is generated in the subsequent difference making step.

Further, in step S3, the specific step of obtaining a reconstructed image based on all the first element coordinates and all the second element coordinates includes:

respectively calculating the areas corresponding to all the first element coordinates and all the second element coordinates to obtain a plurality of first areas and a plurality of second areas; obtaining a number of scale values based on all of the first areas and all of the second areas; updating all the second constituent elements based on all the proportion values to obtain a plurality of third constituent elements; obtaining a plurality of third coordinates corresponding to all the third constituent elements based on all the first element coordinates; the reconstructed image is obtained based on all of the third constituent elements, all of the third coordinates, and the target image.

The brightness information of the object surface is related to illumination and reflection coefficient, the structure of the object in the scene is independent of illumination, the reflection coefficient is related to the object, the structure information in one image can be explored by separating the influence of illumination on the object, but the brightness and the contrast in one scene are always changed, so that the method can compare the brightness, the contrast and the structure of the local image, and can obtain a comparison result more accurately, thereby extracting the defect region more accurately.

Further, in step S4, the specific step of obtaining the structural similarity index set includes:

calculating the brightness similarity, the contrast similarity and the structural similarity of each sub-standard image and the sub-reconstruction image corresponding to the sub-standard image to obtain the structural similarity index; the set of structural similarity indices is obtained based on all of the structural similarity indices.

Considering that the redundant or missing part of the component element occurs in the manufacturing process and the deformation of the component element, such as the situation that the substrate is concave-convex, etc., the method is difficult to find out due to the thickness or density of the component element, etc., the shadow patterns of the product under a plurality of angles are obtained, and whether the redundant or missing part or the deformation part exists can be judged by judging whether the shadows correspond and are consistent, so that the redundant or missing part of the component element can be further accurately found out.

Further, the method further comprises:

s7, respectively acquiring a plurality of standard shadow images and a plurality of target shadow images of the standard product and the product to be detected, wherein the standard shadow images and the target shadow images are in one-to-one correspondence; dividing each standard shadow image and each target shadow image to obtain a plurality of sub-standard shadow images and a plurality of sub-target shadow images, wherein the sub-standard shadow images correspond to the sub-target shadow images one by one;

S8, respectively acquiring each sub-standard shadow image and a plurality of standard shadow areas and a plurality of target shadow areas of the sub-target shadow image corresponding to the sub-standard shadow image, wherein the standard shadow areas are in one-to-one correspondence with the first component elements; obtaining a defect area of a defect based on all the standard shadow areas and all the target shadow areas, and obtaining a second coordinate corresponding to the defect area; and labeling the target image based on the second coordinates and all the target shadow images.

Further, in step S8, the specific step of obtaining a defective area of the defect based on all the standard shadow areas and all the target shadow areas includes:

respectively acquiring coordinates of each standard shadow region and the target shadow region corresponding to the standard shadow region to acquire a plurality of first region coordinates and a plurality of second region coordinates; judging whether the standard shadow areas correspond to the target shadow areas one by one or not based on all the first area coordinates and all the second area coordinates; and if not, acquiring the second region coordinates to obtain the defective region of the defect.

Considering that the heights of the constituent elements are difficult to directly see from an image, the constituent elements which do not meet the detection standard in height cannot be obtained, but the heights of the constituent elements are different in the same scene, the width of shadow areas is different, and whether the heights of the constituent elements have defects or not is judged by calculating whether the areas of the shadow areas are in the detection standard range or not.

Further, the method further comprises:

s9, deleting the second region coordinates corresponding to the defect region in the second region coordinates to obtain a plurality of third region coordinates, and deleting the target shadow region corresponding to the defect region in the target shadow region to obtain a plurality of third regions;

s10, a shadow mapping table is obtained based on all the first region coordinates and all the third region coordinates, the shadow mapping table comprises the standard shadow regions and the third regions, and the standard shadow regions are in one-to-one correspondence with the third regions; calculating a shadow area range corresponding to each standard shadow area based on all the preset range thresholds to obtain a plurality of maximum shadow areas and a plurality of minimum shadow areas, wherein the maximum shadow areas and the minimum shadow areas are in one-to-one correspondence; calculating the shadow area of each third area to obtain a plurality of third shadow areas;

S11, obtaining an area mapping table based on the shadow mapping table, wherein the area mapping table comprises the maximum shadow area, the minimum shadow area and the third shadow area, and the third shadow area corresponds to the maximum shadow area and the minimum shadow area one by one; judging whether each third shadow area is larger than or equal to the minimum shadow area corresponding to the third shadow area and smaller than or equal to the maximum shadow area corresponding to the third shadow area based on the area mapping table; if not, acquiring the third area corresponding to the third shadow area to acquire a defect shadow area; acquiring the second element coordinates of the second component element corresponding to the defect shadow area to acquire third coordinates of a defect; and labeling the target image based on the third coordinate.

The invention also provides a system for improving the defect labeling effect based on image comparison, which comprises:

the acquisition module is used for: the method comprises the steps of acquiring a standard image of a standard product and a target image of a product to be detected; dividing the standard image and the target image into a plurality of sub-standard images and a plurality of sub-target images, wherein the sub-standard images and the sub-target images are in one-to-one correspondence; acquiring a plurality of first component elements in each sub-standard image, wherein each first component element comprises a first lead, a first component, a first via hole and a first copper layer, and each first component element corresponds to a first element coordinate and a preset range threshold; acquiring a plurality of second component elements in each sub-target image, wherein each second component element comprises a second lead, a second component, a second via hole and a second copper layer, each second component element corresponds to a second element coordinate, and each first component element corresponds to a second component element;

And (3) an analog module: the simulation standard image is obtained based on all the first element coordinates, all the preset range thresholds and all the sub-standard images;

and (3) a reconstruction module: for obtaining a reconstructed image based on all of the first element coordinates and all of the second element coordinates;

and a difference making module: the method comprises the steps of dividing the reconstructed image into a plurality of sub-reconstructed images, wherein the sub-reconstructed images correspond to the sub-standard images one by one; performing difference on the standard image and the reconstructed image to obtain a structure similarity index set, wherein the structure similarity index set comprises a plurality of structure similarity indexes;

the calculation module: the method comprises the steps of obtaining first coordinates of defects based on the simulation standard image and the target image, and judging whether the structural similarity index in the structural similarity index set is smaller than a preset index; if yes, acquiring a plurality of sub-reconstruction images corresponding to the plurality of structural similarity indexes, and acquiring a plurality of defect images; performing defect extraction on each defect image to obtain a plurality of defect coordinates;

and the marking module is used for: the method comprises the steps of marking the target image based on the first coordinates, all the defect coordinates and the reconstructed image.

The principle of the system is as follows: the acquisition module acquires a standard image of a standard product and a target image of a product to be detected and divides the standard image and the target image into sub-standard images and sub-target images which are in one-to-one correspondence, and acquires a first component element on each sub-standard image, a second component element on each sub-target image and corresponding coordinates thereof; the simulation module maximizes the size of the first component element on the image through a preset range threshold value and coordinates corresponding to the first component element to obtain a simulation standard image; the reconstruction module moves the second component element to the coordinates of the corresponding first component element on each sub-target image, and aligns the second component element with the position of the first component element so as to obtain a reconstructed image; the difference making module divides the reconstruction image into a plurality of sub-reconstruction images corresponding to the standard images, and makes differences between the sub-reconstruction images and the corresponding sub-standard images to obtain a plurality of structure similarity indexes, and all the structure similarity indexes obtain a structure similarity index set; the calculation module compares the simulation standard image with the target image to obtain a first component element which does not meet the inspection standard, obtains a coordinate corresponding to the first component element to be a first coordinate of the defect, and judges whether the structural similarity index in the structural similarity index set is smaller than a preset index; if yes, obtaining a sub-reconstruction image corresponding to the structural similarity index, so as to obtain a defect image; carrying out defect extraction on each defect image to obtain a plurality of defect coordinates; the labeling module labels the target image through the corresponding relation among the first coordinates, all the defect coordinates and the reconstructed image.

Maximizing the size of the first component element on the image according to the detection standard to obtain a simulation standard image, and comparing the simulation standard image with the target image to obtain a first component element which does not meet the detection standard, so that the component element which does not meet the detection standard can be found; the second component element is aligned with the first component element, so that interference caused by position deviation can be reduced in the subsequent difference making step, a defect area which is really needed to be detected can be better extracted, the target image is restored by adopting an image restoration technology and then difference making is carried out, the condition that the target image restored by adopting the image restoration technology is possibly wrong in restoration is inaccurate, the method adopts comparison with a standard image, only the positions of the component elements are moved, the shape of the component elements is not changed, and the like, so that the accuracy is improved, and the error is reduced; the standard image and the reconstructed image are subjected to difference, so that interference of a non-detection area is reduced, the structural similarity index is more accurate, and the accuracy of extracting the defect area is improved.

The one or more technical schemes provided by the invention have at least the following technical effects or advantages:

1. the size of the first component element is maximized on the image to obtain a simulation standard image and the first component element which does not meet the inspection standard is obtained by comparing the simulation standard image with the target image, so that the component element which does not meet the inspection standard can be found. 2. The second component element is aligned with the first component element, so that the interference caused by position deviation can be reduced in the subsequent difference making step, the defect area which is really needed to be detected can be better extracted, the target image is not repaired by adopting an image repairing technology and then subjected to difference making, the situation of repairing errors can not occur, the method adopts comparison with a standard image, only the positions of the component elements are moved, the shape and the like of the component elements are not changed, the accuracy is improved, and the errors are reduced; the standard image and the reconstructed image are subjected to difference, so that interference of a non-detection area is reduced, the structural similarity index is more accurate, and the accuracy of extracting the defect area is improved. 3. Maximizing the size of the first component element is achieved, and components that do not meet the detection criteria can be better detected. 4. By calculating the area of the component elements and then calculating the area ratio, the second component element is scaled up or down in equal proportion according to the area ratio, the shape of the second component element is not changed, and the interference generated in the difference making step due to the size difference of the component elements is reduced, so that the difference making result is more accurate. 5. The brightness, contrast and structure of the local image are compared, and a comparison result can be obtained more accurately, so that the defect area is extracted more accurately. 6. The shadow patterns of the product under a plurality of angles are obtained, and whether redundant, missing or deformed parts exist or not is judged by judging whether the shadows correspond and are consistent, so that the component elements of the redundant or missing parts can be further accurately found out. 7. And judging whether the heights of the component elements have defects or not by calculating whether the areas of the shadow areas are within the shadow areas corresponding to the detection standard range.

Drawings

The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention;

FIG. 1 is a flow chart of a method for improving defect labeling effect based on image comparison according to the present invention;

FIG. 2 is a schematic flow chart of obtaining a fused image in a method for improving the defect labeling effect based on image comparison in the present invention;

FIG. 3 is a schematic flow chart of obtaining a defect region in a method for improving the defect labeling effect based on image comparison according to the present invention;

FIG. 4 is a schematic flow chart of obtaining a defect shadow area in a method for improving the defect labeling effect based on image comparison according to the present invention;

FIG. 5 is a flow chart of a system for improving the defect labeling effect based on image comparison according to the present invention.

Detailed Description

In order that the above-recited objects, features and advantages of the present invention will be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without collision.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than within the scope of the description, and the scope of the invention is therefore not limited to the specific embodiments disclosed below.

Example 1

Referring to fig. 1 and 2, the present embodiment provides a method for improving defect labeling effect based on image comparison, the method includes:

s1, acquiring a standard image of a standard product and a target image of a product to be detected; dividing the standard image and the target image into a plurality of sub-standard images and a plurality of sub-target images, wherein the sub-standard images and the sub-target images are in one-to-one correspondence;

s2, acquiring a plurality of first component elements in each sub-standard image, wherein each first component element comprises a first lead, a first component, a first via hole and a first copper layer, and each first component element corresponds to a first element coordinate and a preset range threshold; obtaining a simulation standard image based on all the first element coordinates, all the preset range thresholds and all the sub-standard images; obtaining first coordinates of a defect based on the simulated standard image and the target image; in this embodiment, the first component may further include copper foil, connectors, pads, and an annular ring;

In step S2, the specific step of obtaining the simulated standard image based on all the first element coordinates, all the preset range thresholds, and all the sub-standard images includes:

obtaining an extremum in each preset range threshold, wherein the extremum comprises a maximum value and a minimum value; obtaining a plurality of extreme value coordinates corresponding to all the first component elements based on all the first element coordinates and all the extreme values; the simulated standard image is obtained based on all the extremum coordinates and all the first constituent elements. The preset range threshold of a component includes the length: 1 cm.+ -. 15mm, width: and (3) calculating the maximum value and the minimum value respectively corresponding to 0.5cm plus or minus 5mm, adding or subtracting the maximum value and the minimum value according to the coordinates of the component to obtain corresponding extremum coordinates, amplifying the size of the component to correspond to the extremum coordinates, and processing all the first component elements by the method to obtain the simulation standard image.

Wherein, in step S2, the specific step of obtaining the first coordinates of the defect based on the simulated standard image and the target image includes:

dividing the simulation standard image into a plurality of sub-simulation images, wherein the sub-simulation images correspond to the sub-target images one by one; setting all the sub-simulation images and all the sub-target images to a first color and a second color respectively; fusing each sub-simulation image and the sub-target image corresponding to the sub-simulation image to obtain a fused image; the first coordinates of the defect are obtained based on the fused image. If all sub-analog images and all sub-target images are respectively set to be black and red, fusion of corresponding areas is carried out on the sub-analog images and the sub-target images, and fusion images are obtained.

The specific step of fusing each sub-simulation image and the sub-target image corresponding to the sub-simulation image to obtain a fused image comprises the following steps: and placing each sub-target image on a bottom layer, and overlaying the sub-simulation image corresponding to the sub-target image on the sub-target image to fuse to obtain the fused image. If the sub-target image is placed on the bottom layer, red is placed on the bottom layer, then the sub-simulation image corresponding to the red is covered on the bottom layer, black is used as the top layer, and the two images are overlapped to obtain a fusion image.

Wherein the specific step of obtaining the first coordinates of the defect based on the fused image includes:

and judging whether the second color exists on the fusion image, if so, acquiring coordinates corresponding to the second color to acquire the first coordinates. If the fusion image has red color, the second component element on the sub-target image does not meet the detection standard, and the coordinates of the red area are obtained to obtain the first coordinates of the defect.

S3, acquiring a plurality of second component elements in each sub-target image, wherein each second component element comprises a second wire, a second component, a second via hole and a second copper layer, each second component element corresponds to a second component coordinate, and each first component element corresponds to a second component element; obtaining a reconstructed image based on all of the first element coordinates and all of the second element coordinates; in this embodiment, the second component may further include a welding point, copper filling, hole burying, and the like;

Wherein in step S3, the specific step of obtaining a reconstructed image based on all the first element coordinates and all the second element coordinates includes:

respectively calculating the areas corresponding to all the first element coordinates and all the second element coordinates to obtain a plurality of first areas and a plurality of second areas; obtaining a number of scale values based on all of the first areas and all of the second areas; updating all the second constituent elements based on all the proportion values to obtain a plurality of third constituent elements; obtaining a plurality of third coordinates corresponding to all the third constituent elements based on all the first element coordinates; the reconstructed image is obtained based on all of the third constituent elements, all of the third coordinates, and the target image. If the coordinates of the first element corresponding to the first element are (0, 0), (0, 1), (1, 0) and (1, 1) respectively, and the coordinates of the second element corresponding to the second element are (0, 0), (0,1.1), (1, 0) and (1, 1.1), the length and the width of the element can be obtained according to the coordinates, and then according to the area formula: the area is equal to the length multiplied by the width, the first area of the first component and the second area of the second component are calculated to be 1 and 1.1 respectively, the ratio value is 1/1.1=10/11, the second component is uniformly reduced by 10/11 according to the ratio value to obtain a third component, the third component is moved to the first component coordinate, namely the first component coordinate is the third coordinate of the third component, namely the size of the second component is enlarged or reduced to the size of the first component, the size of the second component is only changed without changing the shape of the second component, the second component is moved to the coordinate corresponding to the first component, the position of the first component and the position of the third component are accurately corresponding, and all the second component components are processed as above to obtain the reconstructed image.

S4, dividing the reconstructed image into a plurality of sub-reconstructed images, wherein the sub-reconstructed images correspond to the sub-standard images one by one; the standard image and the reconstructed image are subjected to difference according to a structural similarity principle (SSIM, structural similarity index) to obtain a structural similarity index set, wherein the structural similarity index set comprises a plurality of structural similarity indexes;

in step S4, the specific step of obtaining the structural similarity index set includes:

because the statistical features of the images are generally unevenly distributed in space, the distortion condition of the images is changed in space and in the normal sight distance, people can only focus the sight line in one area of the images, the local processing is more in accordance with the characteristics of a human visual system and the local quality detection can obtain a mapping matrix of the image space quality change, so that the effect of locally solving SSIM indexes is better than that of the whole world, and the embodiment calculates the brightness similarity, the contrast similarity and the structural similarity of each sub-standard image and the sub-reconstructed image corresponding to the sub-standard image according to an SSIM calculation formula to obtain a structural similarity index; a set of structural similarity indices is obtained based on all of the structural similarity indices. In this embodiment, the SSIM calculation formula adopts an existing calculation formula.

S5, judging whether the structural similarity index in the structural similarity index set is smaller than a preset index; if yes, acquiring a plurality of sub-reconstruction images corresponding to the plurality of structural similarity indexes, and acquiring a plurality of defect images; performing defect extraction on each defect image to obtain a plurality of defect coordinates; if the preset index is set to be 0.95 and the structural similarity index is set to be 0.946<0.95, obtaining a sub-reconstruction image corresponding to the structural similarity index, obtaining a sub-target image which does not meet the condition according to the corresponding relation between the sub-reconstruction image and the sub-target image, obtaining a defect image, and obtaining the coordinates of the defect image to obtain the coordinates of the defect.

And S6, marking the target image through the first coordinates, all the defect coordinates and the corresponding relation between the reconstructed image and the target image.

Example 2

Referring to fig. 3 and fig. 4, in this embodiment, on the basis of the first embodiment, the method further includes:

s7, respectively acquiring a plurality of standard shadow images and a plurality of target shadow images of the standard product and the product to be detected under different angles, wherein the standard shadow images and the target shadow images are in one-to-one correspondence; dividing each standard shadow image and each target shadow image to obtain a plurality of sub-standard shadow images and a plurality of sub-target shadow images, wherein the sub-standard shadow images correspond to the sub-target shadow images one by one;

S8, respectively acquiring each sub-standard shadow image and a plurality of standard shadow areas and a plurality of target shadow areas of the sub-target shadow image corresponding to the sub-standard shadow image, wherein the standard shadow areas are in one-to-one correspondence with the first component elements; obtaining a defect area of a defect based on all the standard shadow areas and all the target shadow areas, and obtaining a second coordinate corresponding to the defect area; and labeling the target image based on the second coordinates and all the target shadow images.

Wherein, in step S8, the specific step of obtaining a defective area of the defect based on all the standard shadow areas and all the target shadow areas includes:

respectively acquiring coordinates of each standard shadow region and the target shadow region corresponding to the standard shadow region to acquire a plurality of first region coordinates and a plurality of second region coordinates; judging whether the standard shadow areas correspond to the target shadow areas one by one or not based on all the first area coordinates and all the second area coordinates; and if not, acquiring the second region coordinates to obtain the defective region of the defect.

If under a certain angle, a standard shadow area a and a first area coordinate a are acquired on a standard shadow image a, and a target shadow area B and a first area coordinate B are acquired on a target shadow image B, the standard shadow area a corresponds to a first component element a, and the detection standards of the size and the height of the first component element a are respectively: the method comprises the steps of obtaining the maximum volume and the minimum volume of a first component A according to the maximum value and the minimum value of the length + -0.3 mm, the width + -0.2 mm and the height + -0.1 mm, obtaining the maximum shadow area and the minimum shadow area of the first component A according to simulation of a model tool, converting the maximum shadow area and the minimum shadow area into a coordinate range according to a first region coordinate A, judging whether a first region coordinate B is in the coordinate range or not, if yes, enabling a standard shadow region A to correspond to a target shadow region B, otherwise, enabling the target shadow region B to be an excessive shadow region, obtaining the first region coordinate B to be a second coordinate, and marking the target image according to the corresponding relation between the second coordinate and the target shadow image B and the target image B. In this embodiment, the simulation tool may be a modelable tool such as CAD or 3 DMaxS.

Example 3

With reference to fig. 3 and fig. 4, in this embodiment, the method further includes:

s9, deleting shadow areas and coordinates corresponding to the manufactured second component elements, namely deleting the second area coordinates corresponding to the defect areas in all the second area coordinates to obtain a plurality of third area coordinates, and deleting the target shadow areas corresponding to the defect areas in all the target shadow areas to obtain a plurality of third areas;

s10, a shadow mapping table { standard shadow areas and third areas } is obtained based on all the first area coordinates and all the third area coordinates, wherein the shadow mapping table comprises the standard shadow areas and the third areas, and the standard shadow areas are in one-to-one correspondence with the third areas; calculating a shadow area range corresponding to each standard shadow area based on all the preset range thresholds to obtain a plurality of maximum shadow areas and a plurality of minimum shadow areas, wherein the maximum shadow areas are in one-to-one correspondence with the minimum shadow areas, and if the standard shadow areas A are corresponding to the first component elements A, the preset range thresholds of the first component elements A are respectively: the length is +/-0.3 mm, the width is +/-0.2 mm and the height is +/-0.1 mm, the maximum volume and the minimum volume of the first component A are obtained according to the maximum value and the minimum value of the first component A, and the maximum shadow area and the minimum shadow area of the first component A are obtained according to simulation of a model tool; calculating the shadow area of each third area to obtain a plurality of third shadow areas; in this embodiment, the simulation tool may be a modelable tool such as CAD or 3 DMaxS;

S11, obtaining an area mapping table { maximum shadow area, minimum shadow area and third shadow area }, wherein the area mapping table comprises the maximum shadow area, the minimum shadow area and the third shadow area, and the third shadow area corresponds to the maximum shadow area and the minimum shadow area one by one; judging whether the area relation is based on the corresponding relation of the area mapping table: the minimum shadow area is less than or equal to the third shadow area and less than or equal to the maximum shadow area; if not, acquiring the third area corresponding to the third shadow area to acquire a defect shadow area; acquiring the second element coordinates of the second component element corresponding to the defect shadow area to acquire third coordinates of a defect; and labeling the target image based on the third coordinate.

Example 4

Referring to fig. 5, the present embodiment provides a system for improving defect labeling effect based on image comparison, the system includes:

the acquisition module is used for: the method comprises the steps of acquiring a standard image of a standard product and a target image of a product to be detected; dividing the standard image and the target image into a plurality of sub-standard images and a plurality of sub-target images, wherein the sub-standard images and the sub-target images are in one-to-one correspondence; acquiring a plurality of first component elements in each sub-standard image, wherein each first component element comprises a first lead, a first component, a first via hole and a first copper layer, and each first component element corresponds to a first element coordinate and a preset range threshold; acquiring a plurality of second component elements in each sub-target image, wherein each second component element comprises a second lead, a second component, a second via hole and a second copper layer, each second component element corresponds to a second element coordinate, and each first component element corresponds to a second component element;

And (3) an analog module: the simulation standard image is obtained based on all the first element coordinates, all the preset range thresholds and all the sub-standard images; the specific steps can be as follows: obtaining an extremum in each preset range threshold, wherein the extremum comprises a maximum value and a minimum value; obtaining a plurality of extreme value coordinates corresponding to all the first component elements based on all the first element coordinates and all the extreme values; obtaining the simulated standard image based on all the extremum coordinates and all the first constituent elements;

and (3) a reconstruction module: for obtaining a reconstructed image based on all of the first element coordinates and all of the second element coordinates; the specific steps can be as follows: respectively calculating the areas corresponding to all the first element coordinates and all the second element coordinates to obtain a plurality of first areas and a plurality of second areas; obtaining a number of scale values based on all of the first areas and all of the second areas; updating all the second constituent elements based on all the proportion values to obtain a plurality of third constituent elements; obtaining a plurality of third coordinates corresponding to all the third constituent elements based on all the first element coordinates; obtaining the reconstructed image based on all of the third constituent elements, all of the third coordinates, and the target image;

And a difference making module: the method comprises the steps of dividing the reconstructed image into a plurality of sub-reconstructed images, wherein the sub-reconstructed images correspond to the sub-standard images one by one; the standard image and the reconstruction image are subjected to difference according to an SSIM principle, and a structure similarity index set is obtained, wherein the structure similarity index set comprises a plurality of structure similarity indexes;

the calculation module: the method comprises the steps of obtaining first coordinates of defects based on the simulation standard image and the target image, and judging whether the structural similarity index in the structural similarity index set is smaller than a preset index; if yes, acquiring a plurality of sub-reconstruction images corresponding to the plurality of structural similarity indexes, and acquiring a plurality of defect images; performing defect extraction on each defect image to obtain a plurality of defect coordinates;

and the marking module is used for: the method comprises the steps of marking the target image based on the first coordinates, all the defect coordinates and the reconstructed image.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims (11)

1. A method for improving defect labeling effect based on image comparison, the method comprising:

s1, acquiring a standard image of a standard product and a target image of a product to be detected; dividing the standard image and the target image into a plurality of sub-standard images and a plurality of sub-target images, wherein the sub-standard images and the sub-target images are in one-to-one correspondence;

s2, acquiring a plurality of first component elements in each sub-standard image, wherein each first component element comprises a first lead, a first component, a first via hole and a first copper layer, and each first component element corresponds to a first element coordinate and a preset range threshold; obtaining a simulation standard image based on all the first element coordinates, all the preset range thresholds and all the sub-standard images; obtaining first coordinates of a defect based on the simulated standard image and the target image;

S3, acquiring a plurality of second component elements in each sub-target image, wherein each second component element comprises a second wire, a second component, a second via hole and a second copper layer, each second component element corresponds to a second component coordinate, and each first component element corresponds to a second component element; obtaining a reconstructed image based on all of the first element coordinates and all of the second element coordinates;

s4, dividing the reconstructed image into a plurality of sub-reconstructed images, wherein the sub-reconstructed images correspond to the sub-standard images one by one; performing difference on the standard image and the reconstructed image to obtain a structure similarity index set, wherein the structure similarity index set comprises a plurality of structure similarity indexes;

s5, judging whether the structural similarity index in the structural similarity index set is smaller than a preset index; if yes, acquiring a plurality of sub-reconstruction images corresponding to the plurality of structural similarity indexes, and acquiring a plurality of defect images; performing defect extraction on each defect image to obtain a plurality of defect coordinates;

and S6, labeling the target image based on the first coordinates, all the defect coordinates and the reconstruction image.

2. The method according to claim 1, wherein in step S2, the specific step of obtaining the simulated standard image based on all the first element coordinates, all the preset range thresholds and all the sub-standard images comprises:

obtaining an extremum in each preset range threshold, wherein the extremum comprises a maximum value and a minimum value; obtaining a plurality of extreme value coordinates corresponding to all the first component elements based on all the first element coordinates and all the extreme values; the simulated standard image is obtained based on all the extremum coordinates and all the first constituent elements.

3. The method for improving the effect of defect labeling based on image comparison according to claim 1, wherein in step S2, the specific step of obtaining the first coordinates of the defect based on the simulated standard image and the target image comprises:

dividing the simulation standard image into a plurality of sub-simulation images, wherein the sub-simulation images correspond to the sub-target images one by one; setting all the sub-simulation images and all the sub-target images to a first color and a second color respectively; fusing each sub-simulation image and the sub-target image corresponding to the sub-simulation image to obtain a fused image; the first coordinates of the defect are obtained based on the fused image.

4. A method for improving the effect of defect labeling based on image comparison according to claim 3, wherein the specific step of fusing each sub-simulation image with the sub-target image corresponding to the sub-simulation image to obtain a fused image comprises: and placing each sub-target image on a bottom layer, and overlaying the sub-simulation image corresponding to the sub-target image on the sub-target image to fuse to obtain the fused image.

5. The method for improving the labeling effect of defects based on image comparison according to claim 4, wherein the specific step of obtaining the first coordinates of defects based on the fused image comprises:

and judging whether the second color exists on the fusion image, if so, acquiring coordinates corresponding to the second color to acquire the first coordinates.

6. The method according to claim 1, wherein in step S3, the specific step of obtaining a reconstructed image based on all the first element coordinates and all the second element coordinates comprises:

respectively calculating the areas corresponding to all the first element coordinates and all the second element coordinates to obtain a plurality of first areas and a plurality of second areas; obtaining a number of scale values based on all of the first areas and all of the second areas; updating all the second constituent elements based on all the proportion values to obtain a plurality of third constituent elements; obtaining a plurality of third coordinates corresponding to all the third constituent elements based on all the first element coordinates; the reconstructed image is obtained based on all of the third constituent elements, all of the third coordinates, and the target image.

7. The method for improving the effect of defect labeling based on image comparison according to claim 1, wherein in step S4, the specific step of performing a difference between the standard image and the reconstructed image to obtain a structural similarity index set comprises:

calculating the brightness similarity, the contrast similarity and the structural similarity of each sub-standard image and the sub-reconstruction image corresponding to the sub-standard image to obtain the structural similarity index; the set of structural similarity indices is obtained based on all of the structural similarity indices.

8. The method for improving image alignment-based defect labeling of claim 1,

the method further comprises the steps of:

s7, respectively acquiring a plurality of standard shadow images and a plurality of target shadow images of the standard product and the product to be detected, wherein the standard shadow images and the target shadow images are in one-to-one correspondence; dividing each standard shadow image and each target shadow image to obtain a plurality of sub-standard shadow images and a plurality of sub-target shadow images, wherein the sub-standard shadow images correspond to the sub-target shadow images one by one;

S8, respectively acquiring each sub-standard shadow image and a plurality of standard shadow areas and a plurality of target shadow areas of the sub-target shadow image corresponding to the sub-standard shadow image, wherein the standard shadow areas are in one-to-one correspondence with the first component elements; obtaining a defect area of a defect based on all the standard shadow areas and all the target shadow areas, and obtaining a second coordinate corresponding to the defect area; and labeling the target image based on the second coordinates and all the target shadow images.

9. The method according to claim 8, wherein in step S8, the specific step of obtaining a defective area of the defect based on all the standard shadow areas and all the target shadow areas comprises:

respectively acquiring coordinates of each standard shadow region and the target shadow region corresponding to the standard shadow region to acquire a plurality of first region coordinates and a plurality of second region coordinates; judging whether the standard shadow areas correspond to the target shadow areas one by one or not based on all the first area coordinates and all the second area coordinates; and if not, acquiring the second region coordinates to obtain the defective region of the defect.

10. A method of improving image alignment-based defect labeling effects as in claim 9, further comprising:

s9, deleting the second region coordinates corresponding to the defect region in the second region coordinates to obtain a plurality of third region coordinates, and deleting the target shadow region corresponding to the defect region in the target shadow region to obtain a plurality of third regions;

s10, a shadow mapping table is obtained based on all the first region coordinates and all the third region coordinates, the shadow mapping table comprises the standard shadow regions and the third regions, and the standard shadow regions are in one-to-one correspondence with the third regions; calculating a shadow area range corresponding to each standard shadow area based on all the preset range thresholds to obtain a plurality of maximum shadow areas and a plurality of minimum shadow areas, wherein the maximum shadow areas and the minimum shadow areas are in one-to-one correspondence; calculating the shadow area of each third area to obtain a plurality of third shadow areas;

s11, obtaining an area mapping table based on the shadow mapping table, wherein the area mapping table comprises the maximum shadow area, the minimum shadow area and the third shadow area, and the third shadow area corresponds to the maximum shadow area and the minimum shadow area one by one; judging whether each third shadow area is larger than or equal to the minimum shadow area corresponding to the third shadow area and smaller than or equal to the maximum shadow area corresponding to the third shadow area based on the area mapping table; if not, acquiring the third area corresponding to the third shadow area to acquire a defect shadow area; acquiring the second element coordinates of the second component element corresponding to the defect shadow area to acquire third coordinates of a defect; and labeling the target image based on the third coordinate.

11. A system for improving defect labeling effect based on image comparison, the system comprising:

the acquisition module is used for: the method comprises the steps of acquiring a standard image of a standard product and a target image of a product to be detected; dividing the standard image and the target image into a plurality of sub-standard images and a plurality of sub-target images, wherein the sub-standard images and the sub-target images are in one-to-one correspondence; acquiring a plurality of first component elements in each sub-standard image, wherein each first component element comprises a first lead, a first component, a first via hole and a first copper layer, and each first component element corresponds to a first element coordinate and a preset range threshold; acquiring a plurality of second component elements in each sub-target image, wherein each second component element comprises a second lead, a second component, a second via hole and a second copper layer, each second component element corresponds to a second element coordinate, and each first component element corresponds to a second component element;

and (3) an analog module: the simulation standard image is obtained based on all the first element coordinates, all the preset range thresholds and all the sub-standard images;

And (3) a reconstruction module: for obtaining a reconstructed image based on all of the first element coordinates and all of the second element coordinates;

and a difference making module: the method comprises the steps of dividing the reconstructed image into a plurality of sub-reconstructed images, wherein the sub-reconstructed images correspond to the sub-standard images one by one; performing difference on the standard image and the reconstructed image to obtain a structure similarity index set, wherein the structure similarity index set comprises a plurality of structure similarity indexes;

the calculation module: the method comprises the steps of obtaining first coordinates of defects based on the simulation standard image and the target image, and judging whether the structural similarity index in the structural similarity index set is smaller than a preset index; if yes, acquiring a plurality of sub-reconstruction images corresponding to the plurality of structural similarity indexes, and acquiring a plurality of defect images; performing defect extraction on each defect image to obtain a plurality of defect coordinates;

and the marking module is used for: the method comprises the steps of marking the target image based on the first coordinates, all the defect coordinates and the reconstructed image.