CN117871545A

CN117871545A - Method and device for detecting defects of circuit board components, terminal and storage medium

Info

Publication number: CN117871545A
Application number: CN202311803529.2A
Authority: CN
Inventors: 花霖; 于非; 尹东富; 蔡昊杰; 韩笑
Original assignee: Guangdong Provincial Laboratory Of Artificial Intelligence And Digital Economy Shenzhen
Current assignee: Guangdong Provincial Laboratory Of Artificial Intelligence And Digital Economy Shenzhen
Priority date: 2023-12-25
Filing date: 2023-12-25
Publication date: 2024-04-12

Abstract

The application is applicable to the technical field of electronic components and provides a method, a device, a terminal and a storage medium for detecting defects of a circuit board component, wherein the method comprises the following steps: acquiring a first circuit board image, wherein the first circuit board image corresponds to a circuit board to be detected, and the circuit board to be detected comprises a plurality of components to be detected; extracting the content of the first circuit board image to obtain first component information; determining second component information in a reference circuit board with the similarity meeting requirements with the circuit board to be detected; and performing defect detection on the component to be detected based on the first component information and the second component information to obtain a detection result. The problem that the detection accuracy and the detection efficiency of defect detection are reduced due to the fact that the negative sample data acquisition period is long is solved.

Description

Method and device for detecting defects of circuit board components, terminal and storage medium

Technical Field

The application belongs to the technical field of electronic components, and particularly relates to a method, a device, a terminal and a storage medium for detecting defects of a circuit board component.

Background

Circuit boards (Printed Circuit Board, PCBs) are an indispensable component in electronic devices in the current technological age. In order to ensure stable operation of the electronic device, it is necessary to detect the quality of the circuit board, that is, whether the circuit board has defects, including detecting whether components in the circuit board have defects.

Currently, automated optical inspection (Automated Optical Inspection, AOI) techniques are commonly used to detect defects in circuit board components. The implementation of the AOI technology requires the support of a large amount of negative sample data, but the collection period of the negative sample data is long, so that the detection accuracy and the detection efficiency of defect detection are greatly reduced.

Disclosure of Invention

The embodiment of the application provides a method, a device, a terminal and a storage medium for detecting defects of a circuit board component, which are used for solving the problem that the detection accuracy and the detection efficiency of defect detection are greatly reduced due to long collection period of negative sample data in the prior art.

A first aspect of an embodiment of the present application provides a method for detecting a defect of a circuit board component, including:

acquiring a first circuit board image, wherein the first circuit board image corresponds to a circuit board to be detected, and the circuit board to be detected comprises a plurality of components to be detected;

Extracting the content of the first circuit board image to obtain first component information;

determining second component information in a reference circuit board with the similarity meeting requirements with the circuit board to be detected;

and performing defect detection on the component to be detected based on the first component information and the second component information to obtain a detection result.

A second aspect of the embodiments of the present application provides a device for detecting a defect of a circuit board component, including:

the device comprises an acquisition module, a detection module and a detection module, wherein the acquisition module is used for acquiring a first circuit board image, the first circuit board image corresponds to a circuit board to be detected, and the circuit board to be detected comprises a plurality of components to be detected;

the information extraction module is used for extracting the content of the first circuit board image to obtain first component information;

the determining module is used for determining second component information in the reference circuit board, wherein the similarity between the second component information and the circuit board to be detected meets the requirement;

and the detection module is used for carrying out defect detection on the component to be detected based on the first component information and the second component information to obtain a detection result.

A third aspect of the embodiments of the present application provides a terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the method according to the first aspect when executing the computer program.

A fourth aspect of the embodiments of the present application provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method according to the first aspect.

A fifth aspect of the present application provides a computer program product for causing a terminal to carry out the steps of the method of the first aspect described above when the computer program product is run on the terminal.

From the above, the present application firstly obtains a first circuit board image corresponding to a circuit board to be detected, the circuit board to be detected includes a plurality of components to be detected, the first circuit board image also corresponds to a plurality of images of components to be detected, and then content extraction is performed on the first circuit board image to obtain first component information corresponding to the components to be detected. And determining a reference circuit board similar to the circuit board to be detected, wherein the reference circuit board is a defect-free circuit board which can be referred to and contrasted by the circuit board to be detected. And acquiring second component information corresponding to the reference circuit board, and detecting defects of the component to be detected based on the first component information and the second component information to obtain a detection result. According to the method, the defect detection is not performed by using negative sample data, but a defect-free reference circuit board similar to the circuit board to be detected is selected for performing the defect detection, namely, the positive sample is used for performing the defect detection, the negative sample data is not collected any more, and the problem that the negative sample data collection period is long is solved. Meanwhile, during detection, the defects are detected by specific components rather than directly comparing through the circuit board, and the detection accuracy and efficiency are higher.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly introduce the drawings that are needed in the embodiments or the description of the prior art, it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of a method for detecting defects of a circuit board component according to an embodiment of the present application;

fig. 2 is a diagram of circuit board component mask division effect according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a dual-mode circuit board according to an embodiment of the present application;

fig. 4 is a schematic diagram of sub-circuit board division of a dual-mode circuit board according to an embodiment of the present application;

fig. 5 is a second schematic diagram of sub-circuit board division of a dual-mode circuit board according to an embodiment of the present application;

fig. 6 is a second flowchart of a method for detecting defects of a circuit board component according to an embodiment of the present application;

fig. 7 is a structural diagram of a defect detection device for a circuit board component according to an embodiment of the present application;

fig. 8 is a block diagram of a terminal according to an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

As used in this specification and the appended claims, the term "if" may be interpreted in context as "when …" or "upon" or "in response to a determination" or "in response to detection. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".

In particular implementations, the terminals described in embodiments of the present application include, but are not limited to, other portable devices such as mobile phones, laptop computers, or tablet computers having a touch-sensitive surface (e.g., a touch screen display and/or a touch pad). It should also be appreciated that in some embodiments, the device is not a portable communication device, but a desktop computer having a touch-sensitive surface (e.g., a touch screen display and/or a touch pad).

In the following discussion, a terminal including a display and a touch sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and/or joystick.

The terminal supports various applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk burning applications, spreadsheet applications, gaming applications, telephony applications, video conferencing applications, email applications, instant messaging applications, workout support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and/or digital video player applications.

Various applications that may be executed on the terminal may use at least one common physical user interface device such as a touch sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the terminal may be adjusted and/or changed between applications and/or within the corresponding applications. In this way, the common physical architecture (e.g., touch-sensitive surface) of the terminal may support various applications with user interfaces that are intuitive and transparent to the user.

It should be understood that the sequence number of each step in this embodiment does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

In particular, circuit boards are particularly important in the electronics field, and the quality of the circuit boards affects the operational stability of the electronic device. At present, various types of circuit boards exist in the market, and technicians are developing various novel circuit boards, and a plurality of components are arranged in each type of circuit board, and the categories of the components are five-in-eight. Components are classified into capacitors, inductors, resistors, potentiometers, transformers, diodes, triodes, optoelectronic devices, electroacoustic devices, display devices, field effect transistors, and the like, and are not specifically recited herein.

Specifically, surface mount technology (Surface Mounting Technology, SMT) is currently used to mount components on the surface of a circuit board, and various defects, such as skew, missing components, defects, and the like, of the components in the circuit board may occur due to a mounting error. In order to improve the quality of the circuit board, defect detection is required for the circuit board components. Initially, defect detection is performed by a manual visual inspection method, but detection fatigue is easily generated manually, so that the detection efficiency is low, and the detection accuracy rate may be reduced. The AOI technology can overcome the defects of manual visual inspection, but the AOI technology needs a large amount of negative sample data, namely the support of defect data, for the realization of defect detection, and is used for training a defect detection model. If the defect detection of different types of circuit board components is to be realized, negative sample data of each type of circuit board components need to be acquired respectively, the types of the circuit boards are various, the defect forms of the components in the circuit boards have randomness, the negative sample data acquisition period is prolonged, and complete negative sample data are difficult to obtain. The loss of the negative sample data makes part of defects undetectable, and the detection accuracy and detection efficiency of defect detection are reduced. In order to solve the problems, the application provides a method, a device, a terminal and a storage medium for detecting defects of a circuit board component.

In order to illustrate the technical solutions described in the present application, the following description is made by specific examples.

Referring to fig. 1, fig. 1 is a flowchart of a method for detecting defects of a circuit board component according to an embodiment of the present application. As shown in fig. 1, a method for detecting defects of a circuit board component includes the following steps:

step 101, a first circuit board image is obtained, wherein the first circuit board image corresponds to a circuit board to be detected, and the circuit board to be detected comprises a plurality of components to be detected.

Specifically, the first circuit board image is an acquired image containing the circuit board to be detected, the circuit board to be detected contains a plurality of components to be detected, and correspondingly, the first circuit board image also contains a plurality of images of the components to be detected.

Specifically, the object of defect detection in the application is to detect whether a component in a circuit board is defective or not. The defects of the components in the circuit board include skew offset, defect, missing components and the like. The method and the device for detecting the defects of the components in the circuit board to be detected are based on the first circuit board image, the detected objects are not limited to the components in the circuit board to be detected which are currently existing, and the components corresponding to the defects of the missing components can be detected.

Specifically, the image acquisition device acquires the circuit board image of the circuit board to be detected. When the image is collected, the light source intensity of the auxiliary light source can be strictly controlled, the collected image is prevented from being too dark, the reflection of the surface materials of parts of components is prevented, the extraction of subsequent content is affected, and the information loss occurs.

Specifically, the circuit board image of the circuit board to be detected directly acquired by the image acquisition device is called a third circuit board image. After the third circuit board image is acquired, the third circuit board image needs to be preprocessed, so that the quality of the image is improved, and the first circuit board image with higher image quality can be obtained.

Specifically, the user can check the first circuit board image and the third circuit board image through the display interface, so that the defects that the setting of a shooting area is unreasonable, the setting of light intensity is higher or lower and the like can be timely found, and staff is informed of timely checking, so that the influence on detection efficiency is avoided.

In one example, the preprocessing of the image is: and firstly, adjusting a Red Green Blue (RGB) color gamut threshold of the third circuit board image, then, performing closing operation processing on the adjusted image by using a morphological method to ensure that the edge characteristics are obvious, facilitating the subsequent content extraction, and finally, removing noise in the image by adopting Gaussian filtering to obtain the first circuit board image. Other image preprocessing operations, such as dilation, erosion, smoothing, sharpening, and image enhancement, are not described further herein.

And 102, extracting the content of the first circuit board image to obtain first component information.

Specifically, after the first circuit board image including the plurality of components to be detected is obtained, information of the components to be detected needs to be extracted from the first circuit board image, that is, the first component information is extracted.

Specifically, the first component information is a component information set, the first component information includes component information corresponding to all components to be detected extracted from the first circuit board image, that is, the first component information includes component information corresponding to a plurality of components to be detected.

Specifically, the application applies the everything segmentable model (Segment Anything Model, SAM) to defect detection of circuit board components and parts, and content extraction is achieved through the SAM model.

Specifically, in the application process of the AOI technology, besides the defect detection of circuit board components can be influenced by time and labor consumption in the collection of negative sample data, the detection accuracy and the detection efficiency can be influenced by model training realized based on the negative sample data. The current AOI equipment is in a customized training state, namely model training is carried out according to the type of a given circuit board, defect detection is realized based on a trained model, and the trained model has a better detection and identification effect when the defect detection is carried out on the circuit boards of the types participating in training, but once a new type of circuit board appears, the detection effect is greatly reduced. If the defect detection is performed on the new type of circuit board, a good detection and identification effect is obtained, and then image marking, data acquisition and model training are performed again, which is time-consuming and labor-consuming.

In the application, the SAM model can well solve the problem of multiple types of circuit boards. The SAM model is a very mature image segmentation model, can realize small sample and even zero sample segmentation, and can realize better image segmentation effect for some scenes which are not seen or relatively blurred. While the types of circuit boards are many, the types of components in the circuit boards are limited. By means of the SAM model with strong segmentation capability, the components to be detected in the first circuit board can be segmented very easily and accurately. The method does not need to carry out operations such as image marking, data acquisition, model training and the like, greatly reduces development time and cost, and simultaneously, can improve detection accuracy through excellent segmentation results.

Specifically, the SAM model is mainly composed of three parts, namely, an image encoder, a hint encoder, and a mask decoder. The application mainly uses an image encoder and a mask decoder in the defect detection process of circuit board components. Wherein the image encoder generates image embedding based on a mask-based, pre-trained, transducer-based visual processing (Vision Transformer, viT) architecture of the encoder (Mask Auto Encoder, MAE), the present application preferably performs image conversion based on the high resolution visual processing (Vision Transformer High resolution, viT-H) parameters of the transducer, converting the original image into a matrix profile, and how to convert it is not described in detail herein. The mask decoder may implement dynamic mask prediction, i.e., the embedded conversion of the image generated by the image encoder into a mask and output. And inputting the image into the SAM model to obtain the image segmentation mask output by the SAM model, the contour position of the image segmentation mask, the size area and other information.

Specifically, the SAM model is adopted to extract content of the first circuit board image, so as to obtain the first component information, wherein the first component information comprises information such as component masks, position coordinates of the component masks, component mask outlines, width and height of the component masks, area of the component masks and the like. And obtaining information such as the respective component masks of the components to be detected, the position coordinates of the component masks, the component mask outline, the width and height of the component masks, the area of the component masks and the like.

It should be noted that SAM models often use a 32×32 grid lattice to implement image recognition segmentation. However, after the SAM model is introduced into the defect detection of the circuit board components, it is found that some types of components are smaller, so that the grid lattice of 32×32 cannot cover all components in the circuit board, and further, some components are not successfully identified and segmented. In order to identify and divide all components in the circuit board, the grid lattice parameters of the SAM model are adjusted, namely the grid lattice parameters are increased. The application preferably uses a denser 40×40 grid lattice to realize image recognition segmentation, namely 40 points in each row and each column cover the whole graph, so that any component is avoided being omitted. All components in the circuit board can be identified through the 40X 40 grid lattice used in the method, the segmentation and extraction requirements are met, the segmentation effect is better, meanwhile, the calculated amount in the segmentation and extraction process is less, the time is shorter, namely, the cost is lower, and the efficiency is higher.

In addition, the SAM model can obtain, when extracting the content of the first circuit board image, first circuit board information of the circuit board to be detected, in addition to the first component information, the first circuit board information including information such as a circuit board mask, position coordinates of the circuit board mask, a circuit board mask outline, a width height of the circuit board mask, and an area of the circuit board mask.

Typically, the first circuit board information and the first component information are output substantially simultaneously. The data such as width, height, perimeter and area of the circuit board are far larger than corresponding data of the components, and if the output information is disordered, the output information can be classified through the data to obtain the first circuit board information and the first component information.

Specifically, in order to avoid that other irrelevant contents outside the circuit board area to be detected in the first circuit board image are mistaken for the component to be detected, the first circuit board information is extracted first, and then the first component information is extracted.

Specifically, the extracting the content of the first circuit board image to obtain first component information includes: carrying out circuit board identification on the first circuit board image to obtain first circuit board information; and carrying out component identification in an image area corresponding to the first circuit board information to obtain the first component information.

Specifically, the SAM model is utilized to carry out circuit board identification on the first circuit board image, and the first circuit board information is obtained through segmentation. And then determining an image area indicated by the first circuit board information, namely an area where the circuit board is located, based on the first circuit board information. And carrying out component identification in an image area corresponding to the first circuit board information, and dividing to obtain the first component information.

Specifically, as shown in fig. 2, fig. 2 is a circuit board component mask dividing effect diagram provided in the embodiment of the present application. In order to better show the segmentation effect, the components are marked by adopting colors with different depths in fig. 2. As can be seen from fig. 2, the edges of the component masks are each corresponding to a mask contour, and the SAM model precisely partitions the components in the circuit board. But also, it can be seen that the isolating columns at four corners of the circuit board and the connecting pieces at the connecting positions of the two circuit boards are also divided. In order to avoid other irrelevant contents, such as isolation columns and connecting pieces, outside the circuit board area to be detected in the first circuit board image, the components to be detected are mistakenly considered, circuit board identification is performed first, the first circuit board information is extracted, then component identification is performed in the image area corresponding to the first circuit board information, and the first component information is extracted.

And step 103, determining second component information in the reference circuit board, wherein the similarity between the second component information and the circuit board to be detected meets the requirement.

Specifically, the reference circuit board is similar to the circuit board to be detected, and can be used as a reference control circuit board in the defect detection process of the circuit board to be detected. More precisely, the reference circuit board is a circuit board that is qualified, defect-free, and of the same type as the circuit board to be inspected.

Specifically, in order to realize defect detection of the circuit board components, the reference circuit board for reference control needs to be provided for the circuit board to be detected, and defect detection is realized according to component information in the reference circuit board, namely the second component information.

Specifically, whether the similarity between other qualified and defect-free circuit boards and the circuit board to be detected meets the similarity requirement is judged to determine the reference circuit board, and then the second component information corresponding to the reference circuit board is determined to ensure that defect detection can be smoothly carried out.

Specifically, the determining the second component information in the reference circuit board with the similarity meeting the requirement with the circuit board to be detected includes: respectively calculating the information similarity between the first circuit board information and the second circuit board information corresponding to different types of circuit boards to obtain a plurality of circuit board information similarity calculation results; based on the similarity calculation result of the circuit board information, if the second circuit board information contains a plurality of target circuit board information which has the maximum similarity with the first circuit board information and has the same similarity value, determining specified circuit board information from the plurality of target circuit board information; taking a target type circuit board corresponding to the specified circuit board information as the reference circuit board, and taking the component information of the target type circuit board as the second component information; or based on the result of calculating the similarity of the circuit board information, if the second circuit board information contains the target circuit board information with the maximum similarity with the first circuit board information, taking the target type circuit board corresponding to the target circuit board information as the reference circuit board and taking the component information of the target type circuit board as the second component information.

Specifically, the first circuit board information of the circuit board to be detected and the second circuit board information corresponding to different types of circuit boards are obtained, and information similarity between the first circuit board information and the second circuit board information is calculated respectively, namely, the similarity between the circuit board mask outline, the circuit board mask width height and the circuit board mask area in the first circuit board information and the similarity between the circuit board mask outline, the circuit board mask width height and the circuit board mask area in the second circuit board information are calculated respectively, so that a plurality of circuit board information similarity calculation results are obtained. The number of the circuit board information similarity calculation results is consistent with the number of different types of circuit boards participating in calculation.

Specifically, the circuit board information similarity calculation result reflects the similarity between the circuit board to be detected and the circuit boards of different types for comparison. And determining the reference circuit board with the same type as the circuit board to be detected from different types of circuit boards based on the circuit board information similarity calculation result.

Specifically, the circuit board type determination is performed based on the circuit board information similarity calculation result, if the second circuit board information has a plurality of target circuit board information with the maximum similarity and the same similarity value as the first circuit board information, the screening is needed again, the designated circuit board information is determined from the plurality of target circuit board information, and the circuit board corresponding to the designated circuit board information is the circuit board with the type most similar to the type of the circuit board to be detected.

Specifically, after the specified circuit board information is compared and determined, the target type circuit board corresponding to the specified circuit board information is used as the reference circuit board, and the component information of the target type circuit board is used as the second component information.

Specifically, the circuit board type is determined based on the circuit board information similarity calculation result, if only one piece of target circuit board information with the maximum similarity with the first circuit board information exists in the second circuit board information, the target type circuit board corresponding to the target circuit board information is used as the reference circuit board, and the component information of the target type circuit board is used as the second component information.

Specifically, whether the target circuit board information is included or a plurality of target circuit board information is included, the subsequent screening and/or determining operation needs to be performed under the condition that the maximum similarity is greater than or equal to the set circuit board information similarity threshold value.

Specifically, based on the similarity calculation result of the circuit board information, if the second circuit board information includes a plurality of target circuit board information having the maximum similarity and the same similarity value as the first circuit board information, determining the designated circuit board information from the plurality of target circuit board information includes: acquiring third component information respectively associated with the first component information and the plurality of target circuit board information of the circuit board to be detected under the condition that the second circuit board information contains the plurality of target circuit board information which has the maximum similarity with the first circuit board information and has the same similarity value; respectively calculating the information similarity between the first component information and a plurality of third component information to obtain a plurality of component information similarity calculation results; and determining the designated circuit board information from the plurality of target circuit board information based on the component information similarity calculation result.

Specifically, if the second circuit board information includes a plurality of target circuit board information having the maximum similarity with the first circuit board information and the same similarity value, further screening is required, and the specified circuit board information is determined from the plurality of target circuit board information.

Specifically, when the second circuit board information includes a plurality of pieces of target circuit board information having the maximum similarity with the first circuit board information and the same similarity value, third component information associated with each of the first component information and the plurality of pieces of target circuit board information of the circuit board to be detected is obtained. That is, in the case of similar circuit board information, further screening determination is required in combination with the component information. The component layout information in the component information is used here.

Specifically, the information similarity between the first component layout information and the third component layout information is calculated respectively, and a plurality of component layout information similarity calculation results are obtained. And the target circuit board information corresponding to the component layout information similarity calculation result with the largest calculation result is the specified circuit board information. Up to this point, the specified circuit board information is determined from a plurality of the target circuit board information.

Specifically, when the component layout information is used, comparison can be performed sequentially by the category of the component. For example, the layout of the capacitors in the circuit board is compared, if the specified circuit board information is determined, the comparison is stopped, and if the specified circuit board information is not determined, the layout of the components such as the inductors or the resistors can be continuously compared until the specified circuit board information is determined. And when the layout information of the components is compared, the quantity, position coordinates and other data of the components are compared.

Specifically, since the circuit board to be tested may have a defect of component missing, it is not required to be completely identical when the component layout information is compared.

Specifically, as shown in fig. 3, fig. 3 is a schematic diagram of a dual-mode circuit board according to an embodiment of the present application. As can be seen from fig. 3, the layout of the upper and lower sub-circuit boards of the circuit board is the same, and the SAM model can also be used for dividing the upper and lower sub-circuit boards when dividing. Fig. 4 is a schematic diagram of a sub-circuit board division of a dual-mode circuit board according to an embodiment of the present application, fig. 5 is a schematic diagram of a sub-circuit board division of a dual-mode circuit board according to an embodiment of the present application, fig. 4 is a sub-circuit board of an upper half portion of the dual-mode circuit board of fig. 3, and fig. 5 is a sub-circuit board of a lower half portion of the dual-mode circuit board of fig. 3. When the information similarity of the circuit boards and the similarity of the layout information of the components are calculated, the calculation is carried out by using the related information of one sub circuit board. Meanwhile, defect detection can be realized based on the sub-circuit board.

Specifically, fig. 3 corresponds to fig. 2, and is an undivided original image corresponding to fig. 2. By comparing fig. 2 and 3, the component dividing effect can be more seen.

And 104, performing defect detection on the component to be detected based on the first component information and the second component information to obtain a detection result.

Specifically, through the above operation, the first component information and the second component information are obtained. And taking the second component information as a reference object to detect whether the to-be-detected component corresponding to the first component information has defects or not, and obtaining the detection result.

Specifically, the first component information and the second component information each include a plurality of component information, the plurality of component categories indicated by the component information are various, the defect detection is performed on the component to be detected based on the first component information and the second component information, so as to obtain a detection result, and the method includes: comparing the information similarity of the first component information and the second component information according to the component category to obtain a plurality of component information similarities; comparing the component information similarity with the component information similarity threshold corresponding to each category respectively to obtain a plurality of comparison results; and if the comparison results are all normal results, confirming that the detection of the components to be detected in the circuit board to be detected passes, wherein the normal results indicate that the similarity of the component information is not lower than the similarity threshold of the component information.

Specifically, the first component information is a component information set, and the first component information includes component information corresponding to all components to be detected extracted from the first circuit board image, that is, the first component information includes component information of a plurality of components to be detected. Correspondingly, the second component information is also a component information set, and the component information set comprises component information corresponding to a plurality of reference components. The categories of the plurality of components to be detected are various, and the categories of the plurality of reference components are also various.

Specifically, various types of components are attached to the circuit board, and the characteristics of the components in different types are different, so that all types of components cannot be judged according to the same standard. Therefore, classification calculation and classification comparison are required at the time of defect detection based on the first component information and the second component information.

Specifically, the classification is performed according to the component mask profile in the component information, and the component categories corresponding to the component information having the same component mask profile are the same. According to the method, the classification of the component information is realized. And calculating the information similarity of the component information belonging to the same component category in the first component information and the second component information to obtain a plurality of component information similarities. When the similarity is calculated, the similarity of the information such as the width and the height of the component masks, the area of the component masks, the inclination angle of the component masks and the like in the component information with the same or similar position coordinates of the component masks is calculated. And each component information similarity corresponds to one component.

Specifically, the similarity of the component information is compared with the similarity threshold value of the component information corresponding to each category, and a plurality of comparison results are obtained. The component information similarity threshold is a preset value, and each component category corresponds to the component information similarity threshold.

Optionally, calculating the similarity mean value of the components of the same class of multiple non-defective circuit boards, wherein the similarity mean value is used as the component information similarity threshold value. Or on the basis of the similarity mean value, setting the similarity threshold value of the component information according to the three-time standard deviation principle, namely introducing normal distribution into the setting standard of the similarity threshold value of the component information.

Specifically, the comparison result is a normal result or an abnormal result, the normal result indicates that the component information similarity is not lower than the component information similarity threshold, and the abnormal result indicates that the component information similarity is lower than the component information similarity threshold. The normal result indicates that the component is defect-free, and the abnormal result indicates that the component is defect-free.

Specifically, if the comparison results are the normal results, confirming that the components to be detected in the circuit board to be detected have no defects, and outputting a detection result, namely that the components to be detected have no defects, and detecting the components to be detected pass.

Specifically, the comparing the similarity of the plurality of component information with the similarity threshold of the component information corresponding to each category respectively, and after obtaining a plurality of comparison results, further includes: if an abnormal result exists in the comparison results, determining defect information based on the first component information and the second component information, wherein the abnormal result indicates that the component information similarity is lower than the component information similarity threshold; and performing defect labeling in the first circuit board image based on the defect information to obtain a defect labeling image.

Specifically, if the abnormal results exist in the plurality of comparison results, the defect of the to-be-detected component corresponding to the abnormal results is indicated. And determining the defect information according to the first component information and the second component information, wherein the defect information comprises defect type information and defect position information. And marking the position where the defect exists and why the defect exists according to the defect position information at the corresponding position in the first circuit board image, and obtaining the defect marking image. And displaying the defect labeling image as a detection result to a user.

Specifically, several examples of judging the defect type are given below. And if the similarity between the width and the height of the component mask and the area of the component mask is lower than a threshold value, indicating that the corresponding component to be detected has defects. If the comparison information for comparing with the certain information in the second component information is not found in the first component information, or if the comparison information for comparing with the certain information in the first component information is not found in the second component information, the fact that no comparable component exists at the same position of the circuit board to be detected and the reference circuit board, and the circuit board to be detected has component missing, namely component missing, or redundant components exist in the circuit board to be detected. If the difference of the inclination angles of the component masks is too large, the corresponding components to be detected are indicated to have skew offset defects, and the like.

Specifically, the user can intuitively know where the defects exist in the circuit board to be detected and what types of defects exist according to the defect labeling image, and the defect components in the circuit board to be detected can be quickly corrected.

Specifically, the similarity can be provided directly to the user for viewing, or a scoring criterion can be established in terms of similarity and presented to the user in the form of a score.

In the embodiment of the application, first circuit board images corresponding to the circuit board to be detected are obtained, the circuit board to be detected comprises a plurality of components to be detected, the first circuit board images also correspond to the images of the components to be detected, and then content extraction is performed on the first circuit board images to obtain first component information corresponding to the components to be detected. And determining a reference circuit board similar to the circuit board to be detected, wherein the reference circuit board is a defect-free circuit board which can be referred to and contrasted by the circuit board to be detected. And acquiring second component information corresponding to the reference circuit board, and detecting defects of the component to be detected based on the first component information and the second component information to obtain a detection result. According to the method, the defect detection is not performed by using negative sample data, but a defect-free reference circuit board similar to the circuit board to be detected is selected for performing the defect detection, namely, the positive sample is used for performing the defect detection, the negative sample data is not collected any more, and the problem that the negative sample data collection period is long is solved. Meanwhile, during detection, the defects are detected by specific components rather than directly comparing through the circuit board, and the detection accuracy and efficiency are higher.

Referring to fig. 6, fig. 6 is a second flowchart of a method for detecting defects of a circuit board component according to an embodiment of the present application. As shown in fig. 6, a method for detecting defects of a circuit board component includes the following steps:

step 201, obtaining second circuit board images corresponding to a plurality of different types of circuit boards, wherein each type of circuit board comprises a plurality of components, and the different types of circuit boards comprise reference circuit boards.

Specifically, before defect detection, a different type of defect-free circuit board is selected as a reference sample for defect detection. Each of the different types of circuit boards comprises a plurality of components, and correspondingly, the second circuit board image corresponding to each type of circuit board also comprises a plurality of components. These components are all qualified and defect-free components. The reference circuit board of the same type as the circuit board to be inspected is included in a plurality of different types of circuit boards.

Specifically, a circuit board image is acquired by the image acquisition device. And when the circuit board image is acquired, the light source intensity of the auxiliary light source can be strictly controlled, the acquired image is ensured not to be too dark, the reflection of the surface materials of part of components is avoided, the subsequent content extraction is affected, and the information loss condition occurs. The second circuit board image is an image subjected to image preprocessing, and the image preprocessing process is not repeated.

Specifically, after image acquisition and image preprocessing, the second circuit board images corresponding to the circuit boards of different types can be acquired. To circumvent the data imbalance problem, a plurality of said second circuit board images may be collected for each type of circuit board.

Step 202, extracting the content of the second circuit board images to obtain fourth device information; the fourth component information is component information subjected to information amplification processing, and the fourth component information comprises second component information corresponding to the reference circuit board.

Specifically, content extraction is performed on circuit board images corresponding to a plurality of different types of circuit boards, namely a plurality of second circuit board images, through a SAM model, so as to obtain fourth component information corresponding to each second circuit board image. The content extraction process is the same as that of the first circuit board image described above. The difference is that the fourth component information corresponding to the second circuit board images as the reference sample is the component information subjected to the information amplification processing. Image segmentation and feature extraction are carried out through the SAM model, and information amplification processing is carried out on the extracted component information to obtain fourth component information. Correspondingly, the fourth component information comprises the second component information corresponding to the reference circuit board, namely the second component information is also component information subjected to information amplification processing.

Specifically, when content extraction is performed based on the SAM model, circuit board information corresponding to the second circuit board image is also extracted, and the extraction process of the circuit board information and the content contained in the circuit board information are referred to above.

Specifically, in practical application, because components can present a little pose error in the production process of the circuit board, the outline of the original component mask is too close to the edge of the component, if image extraction is directly carried out, the edge characteristics of the components are easily lost, similarity calculation is affected, defect false evaluation is easily caused, and detection accuracy and detection efficiency are affected. Accordingly, the present application performs an information amplification processing operation for performing defect detection with the fourth device information after the information amplification processing as a reference sample.

Specifically, the procedure of the information amplification process is described with the reference circuit board as an example. And extracting the content of the circuit board image of the reference circuit board by adopting a SAM model to obtain initial component information corresponding to the reference circuit board, wherein the initial component information is component information which is not subjected to information amplification processing. And taking an initial component mask in the initial component information, performing mask amplification processing on the initial component mask, namely performing external rectangle operation on the initial component mask, and expanding mask area. More specifically, the information amplification processing mainly refers to mask amplification processing for an initial component mask of a component.

Specifically, after the initial component mask is subjected to mask processing, the component mask in the second component information corresponds to the component mask, which may also be referred to as a second component mask. And re-determining mask related information according to the second component mask, namely re-determining information such as position coordinates of the mask, mask outline, width height of the mask, area of the mask and the like, and finally obtaining the second component information of the reference component. Because the second component mask is a component mask after the mask expansion process, correspondingly, the information such as the position coordinates of the mask, the contour of the component mask, the width height of the mask, the area of the mask and the like is redetermined by the second component mask and is also the component information after the information expansion process.

Specifically, the information amplification processing procedure of other types of circuit boards as reference samples is the same as the above procedure.

Specifically, the reference sample information after the information amplification processing does not lose the edge contour information of the components, and allows the errors of slight inclination and partial defect of the components in the image due to the factors of production, placement, shooting and the like, namely, a space allowing the errors is provided, the performance of a circuit board is not influenced, meanwhile, the detection standard is properly widened, the detection accuracy and the detection efficiency are improved, invalid reworking is avoided, and the workload is reduced.

Specifically, the SAM model has a very excellent segmentation and extraction capability, and can segment and extract circuit board information and component information very accurately even for a new type of circuit board which is submitted for the first time as a reference sample.

Specifically, after a period of use, production equipment for producing circuit boards has a certain loss. The production state is not as good as the former information such as the position of the components in the produced circuit board, which is affected by the equipment loss, but the performance of the circuit board is not affected, and most of the circuit boards are qualified and defect-free.

However, if the defect detection is performed according to the initially given reference sample, these changes are likely to be determined as defects, resulting in reworking adjustment of the circuit board, which increases the workload. To avoid such a situation, it is necessary to replace these circuit boards as reference samples in accordance with a set period and to redefine the fourth component information, thereby reducing the influence due to the loss of production equipment.

Step 203, a first circuit board image is acquired, where the first circuit board image corresponds to a circuit board to be detected, and the circuit board to be detected includes a plurality of components to be detected.

The implementation process of this step is the same as that of step 101 in the foregoing embodiment, and will not be described here again.

And 204, extracting the content of the first circuit board image to obtain first component information.

The implementation process of this step is the same as that of step 102 in the foregoing embodiment, and will not be described here again.

Step 205, determining the second component information in the reference circuit board with the similarity meeting requirements with the circuit board to be detected.

The implementation process of this step is the same as that of step 103 in the foregoing embodiment, and will not be described here again.

And 206, performing defect detection on the component to be detected based on the first component information and the second component information to obtain a detection result.

The implementation process of this step is the same as that of step 104 in the foregoing embodiment, and will not be described here again.

In this embodiment, before acquiring the first circuit board image, first acquiring second circuit board images corresponding to a plurality of different types of circuit boards, and extracting content from the second circuit board images to obtain a plurality of fourth component information, where each type of circuit board includes a plurality of components, the different types of circuit boards include a reference circuit board, and the fourth component information includes second component information corresponding to the reference circuit board. I.e. the reference sample circuit board for defect detection is prepared in advance, facilitating defect detection. Meanwhile, the provided reference sample information, namely fourth device information, is the information of the components subjected to information amplification processing, and the detection standard is properly relaxed while the performance of the circuit board is not affected, so that the method is more flexible, the detection accuracy and the detection efficiency are improved, invalid reworking is avoided, and the workload is reduced.

Referring to fig. 7, fig. 7 is a structural diagram of a defect detecting device for circuit board components provided in an embodiment of the present application, and for convenience of explanation, only a portion related to the embodiment of the present application is shown.

The circuit board component defect detection apparatus 300 includes: the device comprises an acquisition module 301, an information extraction module 302, a determination module 303 and a detection module 304.

The obtaining module 301 is configured to obtain a first circuit board image, where the first circuit board image corresponds to a circuit board to be detected, and the circuit board to be detected includes a plurality of components to be detected.

And the information extraction module 302 is configured to extract content of the first circuit board image to obtain first component information.

And the determining module 303 is configured to determine second component information in the reference circuit board, where the similarity between the second component information and the circuit board to be detected meets the requirement.

And the detection module 304 is configured to detect a defect of the component to be detected based on the first component information and the second component information, so as to obtain a detection result.

The information extraction module 302 is specifically configured to:

carrying out circuit board identification on the first circuit board image to obtain first circuit board information;

and carrying out component identification in an image area corresponding to the first circuit board information to obtain the first component information.

The determining module 303 is specifically configured to:

respectively calculating the information similarity between the first circuit board information and the second circuit board information corresponding to different types of circuit boards to obtain a plurality of circuit board information similarity calculation results;

based on the similarity calculation result of the circuit board information, if the second circuit board information contains a plurality of target circuit board information which has the maximum similarity with the first circuit board information and has the same similarity value, determining specified circuit board information from the plurality of target circuit board information;

taking a target type circuit board corresponding to the specified circuit board information as the reference circuit board, and taking the component information of the target type circuit board as the second component information; or,

and based on the similarity calculation result of the circuit board information, if the second circuit board information contains the target circuit board information with the maximum similarity with the first circuit board information, taking the target type circuit board corresponding to the target circuit board information as the reference circuit board and taking the component information of the target type circuit board as the second component information.

Acquiring third component information respectively associated with the first component information and the plurality of target circuit board information of the circuit board to be detected under the condition that the second circuit board information contains the plurality of target circuit board information which has the maximum similarity with the first circuit board information and has the same similarity value;

respectively calculating the information similarity between the first component information and a plurality of third component information to obtain a plurality of component information similarity calculation results;

and determining the designated circuit board information from the plurality of target circuit board information based on the component information similarity calculation result.

Wherein, the detection module 304 is specifically configured to:

comparing the information similarity of the first component information and the second component information according to the component category to obtain a plurality of component information similarities;

comparing the component information similarity with the component information similarity threshold corresponding to each category respectively to obtain a plurality of comparison results;

and if the comparison results are all normal results, confirming that the detection of the components to be detected in the circuit board to be detected passes, wherein the normal results indicate that the similarity of the component information is not lower than the similarity threshold of the component information.

If an abnormal result exists in the comparison results, determining defect information based on the first component information and the second component information, wherein the abnormal result indicates that the component information similarity is lower than the component information similarity threshold;

and performing defect labeling in the first circuit board image based on the defect information to obtain a defect labeling image.

Specifically, the circuit board component defect detection apparatus 300 further includes a reference information determining module, configured to:

acquiring second circuit board images corresponding to a plurality of different types of circuit boards, wherein each type of circuit board comprises a plurality of components, and the reference circuit board is included in the plurality of different types of circuit boards;

extracting the content of the second circuit board images to obtain fourth device information; the fourth component information is component information subjected to information amplification processing, and the fourth component information comprises second component information corresponding to the reference circuit board.

The device for detecting the defects of the circuit board components can realize the processes of the embodiment of the method for detecting the defects of the circuit board components, can achieve the same technical effects, and is not repeated here.

Fig. 8 is a block diagram of a terminal according to an embodiment of the present application. As shown in the figure, the terminal 4 of this embodiment includes: at least one processor 40 (only one is shown in fig. 8), a memory 41 and a computer program 42 stored in the memory 41 and executable on the at least one processor 40, the processor 40 implementing the steps in any of the various method embodiments described above when executing the computer program 42.

The terminal 4 may be a computing device such as a desktop computer, a notebook computer, a palm computer, a cloud server, etc. The terminal 4 may include, but is not limited to, a processor 40, a memory 41. It will be appreciated by those skilled in the art that fig. 8 is merely an example of the terminal 4 and is not limiting of the terminal 4, and may include more or fewer components than shown, or may combine some components, or different components, e.g., the terminal may further include input and output devices, network access devices, buses, etc.

The processor 40 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 41 may be an internal storage unit of the terminal 4, such as a hard disk or a memory of the terminal 4. The memory 41 may also be an external storage device of the terminal 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the terminal 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the terminal 4. The memory 41 is used for storing the computer program as well as other programs and data required by the terminal. The memory 41 may also be used for temporarily storing data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal and method may be implemented in other manners. For example, the apparatus/terminal embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each method embodiment described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

The present application may implement all or part of the procedures in the methods of the above embodiments, and may also be implemented by a computer program product, which when run on a terminal causes the terminal to implement steps in the embodiments of the methods described above.

The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.

Claims

1. The method for detecting the defects of the components of the circuit board is characterized by comprising the following steps:

2. The method of claim 1, wherein the extracting the content of the first circuit board image to obtain the first component information includes:

3. The method of claim 2, wherein determining the second component information in the reference circuit board for which the similarity to the circuit board to be tested meets the requirement comprises:

4. The method of claim 3, wherein determining the specified circuit board information from the plurality of target circuit board information if the second circuit board information includes a plurality of target circuit board information having the greatest similarity with the first circuit board information and the same similarity value based on the circuit board information similarity calculation result, comprises:

5. The method of claim 1, wherein the first component information and the second component information each include a plurality of component information, the plurality of component categories indicated by the component information include a plurality of component categories, and the performing defect detection on the component to be detected based on the first component information and the second component information to obtain a detection result includes:

6. The method of claim 5, wherein comparing the plurality of component information similarities with respective component information similarity thresholds corresponding to respective categories, and after obtaining a plurality of comparison results, further comprises:

7. The method of claim 1, wherein prior to the acquiring the first circuit board image, further comprising:

8. A circuit board component defect detection apparatus, comprising:

9. A terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 7 when the computer program is executed.

10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the method according to any one of claims 1 to 7.