CN118657776A - Improved motherboard defect detection method, device and equipment based on anomaly detection model - Google Patents

Abstract

The application discloses a mainboard defect detection method, device and equipment based on an abnormality detection model improvement, and relates to the technical field of defect detection, wherein the method comprises the following steps: acquiring an original image of a main board to be detected, and extracting a key region to be detected from the original image of the main board to be detected; detecting a key region to be detected based on an improved Anomalib model to obtain abnormal probability distribution data of the key region to be detected; determining the defect position in the key region to be detected based on the abnormal probability distribution data of the key region to be detected; marking the key region to be detected and the defect position in the key region to be detected respectively to generate a result thermodynamic diagram of the main board to be detected. By the mode, the model structure is optimized, the calculated amount is reduced, the reasoning efficiency of the model is improved, meanwhile, the image key region is utilized for training and reasoning, the pertinence of the model is improved, the accuracy and the reasoning efficiency of the model are improved, and the accuracy and the efficiency of defect detection are improved.

Description

Improved motherboard defect detection method, device and equipment based on anomaly detection model

Technical Field

The present application relates to the technical field of defect detection, and in particular to a method, device and equipment for detecting motherboard defects based on an improved abnormality detection model.

Background Art

When the resolution of the motherboard image is too large, directly using the Anomalib model for defect detection is not very targeted and it is difficult to converge to the area of real concern, which affects the accuracy of defect detection. In addition, the detection takes a long time, which affects the efficiency of defect detection.

The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art.

Summary of the invention

The main purpose of this application is to provide a motherboard defect detection method, device and equipment based on an improved anomaly detection model, aiming to solve the technical problems of low accuracy and efficiency when the traditional Anomalib model in the prior art performs defect detection on large-resolution motherboard images.

To achieve the above objectives, the present application provides a method for detecting motherboard defects based on an improved anomaly detection model, the method comprising:

Acquire an original image of the motherboard to be detected, and extract a key area to be detected from the original image of the motherboard to be detected;

Based on the improved Anomalib model, the key area to be detected is detected to obtain abnormal probability distribution data of the key area to be detected, the improved Anomalib model includes a preprocessing unit, a Bottleneck unit that replaces the original feature extraction unit, and an abnormality recognition unit, the preprocessing unit is used to extract visual features in the key area to be detected, and input the visual features into the Bottleneck unit, the Bottleneck unit is used to generate dimensionality reduction features based on the visual features, and input the dimensionality reduction features into the abnormality recognition unit for abnormality recognition;

Determining the defect position in the key area to be detected based on the abnormal probability distribution data of the key area to be detected;

The key area to be inspected and the defect positions in the key area to be inspected are marked respectively to generate a result heat map of the motherboard to be inspected.

In one embodiment, the improved Anomalib model uses a new Relu4 activation function to replace the original Relu activation function.

In one embodiment, the step of acquiring an original image of the motherboard to be detected and extracting a key area to be detected from the original image of the motherboard to be detected includes:

Acquire an original image of the motherboard to be detected, and determine a transformation matrix based on the coordinates of a reference point on the motherboard to be detected in the original image and the coordinates of the reference point in the reference image;

Acquire the coordinates of the target area in the reference image, and determine the coordinates of the key area to be detected based on the transformation matrix and the coordinates of the target area in the reference image;

Based on the coordinates of the key area to be detected, the key area to be detected is extracted from the original image.

In one embodiment, the preprocessing unit includes at least a feature extractor, the abnormality identification unit includes at least a NormalizingFlow layer and a FastFlow layer, and the step of detecting the key area to be detected based on the improved Anomalib model to obtain abnormal probability distribution data of the key area to be detected includes:

Inputting the vector data corresponding to the key area to be detected into the feature extractor for feature extraction to obtain the visual features of the key area to be detected;

Inputting the visual features into the Bottleneck unit for feature extraction and dimensionality reduction processing to obtain the reduced dimensionality features;

Input the dimension reduction features into the NormalizingFlow layer for distribution normalization to obtain feature normal distribution data;

The characteristic normal distribution data is input into the FastFlow layer for anomaly recognition to obtain the abnormal probability distribution data of the key area to be detected.

In one embodiment, the Bottleneck unit includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer, and a fourth convolutional layer connected in sequence, the second convolutional layer uses a preset standard convolutional kernel, the first convolutional layer, the third convolutional layer, and the fourth convolutional layer all use a preset small-size convolutional kernel, the preset small-size convolutional kernel is smaller than the preset standard convolutional kernel, and the step of inputting the visual feature into the Bottleneck unit for feature extraction and dimensionality reduction, and obtaining the reduced dimensionality feature also includes:

Inputting the visual features into the first convolutional layer for dimensionality reduction processing to obtain dimensionality-reduced visual features;

Inputting the reduced-dimensional visual features into the second convolutional layer for feature extraction, and inputting the reduced-dimensional visual features after feature extraction into the third convolutional layer for dimensionality increase processing to obtain the increased-dimensional visual features;

The up-dimensional visual features are input into the pooling layer for feature extraction, and the up-dimensional visual features after feature extraction are input into the fourth convolutional layer for dimensionality reduction processing to obtain the reduced dimensionality features.

In one embodiment, the step of inputting the characteristic normal distribution data into the FastFlow layer for abnormality identification to obtain abnormal probability distribution data of the key area to be detected includes:

Inputting the characteristic normal distribution data into the FastFlow layer, and determining the abnormal scores of different positions in the key area to be detected based on the distance between the characteristic normal distribution data and the reference characteristic normal distribution data;

Determining the abnormality probabilities of different positions in the key area to be detected based on the abnormality scores of different positions in the key area to be detected;

Based on the abnormality probabilities at different positions in the key area to be detected, abnormality probability distribution data of the key area to be detected is determined.

In one embodiment, the method further comprises:

Acquire a training image, and determine a training transformation matrix based on coordinates of a training reference point in the training image and coordinates of the training reference point in the reference image;

Determining the coordinates of the training key area in the training image based on the training transformation matrix and the coordinates of the target area;

Extracting the training key area from the training image based on the coordinates of the training key area;

Based on the training key area, the improved Anomalib model is trained and the improved Anomalib model is converted and deployed to the mobile terminal.

In addition, to achieve the above purpose, the present application also proposes a motherboard defect detection device based on an improved abnormality detection model, and the motherboard defect detection device based on an improved abnormality detection model includes:

The region extraction module is used to obtain the original image of the motherboard to be detected, and extract the key region to be detected from the original image of the motherboard to be detected;

A defect detection module, configured to detect the key area to be detected based on an improved Anomalib model to obtain abnormal probability distribution data of the key area to be detected, wherein the improved Anomalib model includes a preprocessing unit, a Bottleneck unit replacing an original feature extraction unit, and an abnormality recognition unit, wherein the preprocessing unit is configured to extract visual features in the key area to be detected and input the visual features into the Bottleneck unit, and the Bottleneck unit is configured to generate a dimensionality reduction feature based on the visual features and input the dimensionality reduction feature into the abnormality recognition unit for abnormality recognition;

The defect detection module is further used to determine the defect position in the key area to be detected based on the abnormal probability distribution data of the key area to be detected;

The defect detection module is also used to mark the key area to be detected and the defect position in the key area to be detected respectively, and generate a result heat map of the motherboard to be detected.

In addition, to achieve the above-mentioned purpose, the present application also proposes a motherboard defect detection device improved based on anomaly detection model, and the motherboard defect detection device improved based on anomaly detection model includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the motherboard defect detection method improved based on anomaly detection model as described above.

In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the improved motherboard defect detection method based on the abnormality detection model as described above are implemented.

In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the improved motherboard defect detection method based on the abnormality detection model as described above.

The present application provides a mainboard defect detection method improved based on an anomaly detection model, which obtains an original image of a mainboard to be detected, and extracts a key area to be detected from the original image of the mainboard to be detected; based on an improved Anomalib model, the key area to be detected is detected to obtain abnormal probability distribution data of the key area to be detected, the improved Anomalib model includes a preprocessing unit, a Bottleneck unit that replaces the original feature extraction unit, and an abnormality recognition unit, the preprocessing unit is used to extract visual features in the key area to be detected, and input the visual features into the Bottleneck unit, the Bottleneck unit is used to generate dimensionality reduction features based on the visual features, and input the dimensionality reduction features into the abnormality recognition unit for abnormality recognition; based on the abnormal probability distribution data of the key area to be detected, the defect position in the key area to be detected is determined; the key area to be detected and the defect position in the key area to be detected are marked respectively, and a result heat map of the mainboard to be detected is generated. This application optimizes the model structure, uses the Bottleneck unit for dimensionality reduction, and adopts a new activation function with better robustness to reduce the amount of inference calculations, reduce the time spent on inference, and improve the inference efficiency of the model. At the same time, it uses the key areas of the image for training and inference to improve the model's pertinence, improve the model's accuracy and inference efficiency, thereby improving the accuracy and efficiency of defect detection. Even if the image resolution is large, the detection results can be obtained quickly and accurately, solving the technical problem of low accuracy and efficiency of the traditional Anomalib model when performing defect detection on large-resolution motherboard images.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

FIG1 is a flow chart of a first embodiment of a method for detecting motherboard defects based on an improved abnormality detection model of the present application;

FIG2 is a schematic diagram of the structure of an improved Anomalib model of a mainboard defect detection method based on an improved anomaly detection model provided in Example 1 of the present application;

FIG3 is a schematic diagram of the Bottleneck unit structure of the improved mainboard defect detection method based on the abnormality detection model provided in Example 1 of the present application;

FIG4 is a schematic diagram of a new activation function of the improved mainboard defect detection method based on anomaly detection model provided in Example 1 of the present application;

FIG5 is a schematic diagram of the module structure of a mainboard defect detection device improved based on an abnormality detection model according to an embodiment of the present application;

FIG6 is a schematic diagram of the device structure of the hardware operating environment involved in the improved mainboard defect detection method based on the abnormality detection model in an embodiment of the present application.

The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings.

DETAILED DESCRIPTION

It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

The main solution of the embodiment of the present application is: obtaining the original image of the mainboard to be detected, and extracting the key area to be detected from the original image of the mainboard to be detected; based on the improved Anomalib model, detecting the key area to be detected, and obtaining abnormal probability distribution data of the key area to be detected, the improved Anomalib model includes a connected preprocessing unit, a Bottleneck unit that replaces the original feature extraction unit, and an abnormality recognition unit, the preprocessing unit is used to extract visual features in the key area to be detected, and input the visual features into the Bottleneck unit, the Bottleneck unit is used to generate dimensionality reduction features based on the visual features, and input the dimensionality reduction features into the abnormality recognition unit for abnormality recognition; based on the abnormal probability distribution data of the key area to be detected, determining the defect position in the key area to be detected; marking the key area to be detected and the defect position in the key area to be detected respectively, and generating a result heat map of the mainboard to be detected.

At present, when the resolution of the motherboard image is too large, directly using the Anomalib model for defect detection is not very targeted and it is difficult to converge to the area of real concern, which affects the accuracy of defect detection. In addition, the detection takes a long time, which affects the efficiency of defect detection.

The present application provides a solution, optimizes the model structure, uses the Bottleneck unit for dimensionality reduction, and adopts a new activation function with better robustness, reduces the amount of inference calculations, reduces the time spent on inference, and improves the inference efficiency of the model. At the same time, the key areas of the image are used for training and inference to improve the model's pertinence, improve the model's accuracy and inference efficiency, thereby improving the accuracy and efficiency of defect detection. Even if the image resolution is large, the detection results can be obtained quickly and accurately, solving the technical problem of low accuracy and efficiency of the traditional Anomalib model when performing defect detection on large-resolution motherboard images.

It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a motherboard defect detection device improved based on anomaly detection model, etc., and this embodiment does not specifically limit this. The following takes the motherboard defect detection device improved based on anomaly detection model as an example to illustrate this embodiment and the following embodiments.

An embodiment of the present application provides a mainboard defect detection method based on an improved anomaly detection model. Referring to FIG. 1 , FIG. 1 is a flow chart of a first embodiment of the mainboard defect detection method based on an improved anomaly detection model of the present application.

In this embodiment, the improved mainboard defect detection method based on the abnormality detection model includes steps S10 to S40:

Step S10, obtaining an original image of the motherboard to be detected, and extracting a key area to be detected from the original image of the motherboard to be detected;

It should be noted that the motherboard to be tested refers to the motherboard that needs to be tested for defects. The specific number is determined according to the actual situation and is not specifically limited. The original image of the motherboard to be tested is the image of the motherboard to be tested directly taken. When acquiring the image, it is usually necessary to use a dedicated light box and camera. The key area to be tested refers to the part that needs to be focused on in the original image of the motherboard to be tested, which is usually the area where important components are located, and there is no specific limitation on this.

In a feasible implementation, step S10 may include steps S101 to S103:

Step S101, obtaining an original image of a motherboard to be inspected, and determining a transformation matrix based on the coordinates of a reference point on the motherboard to be inspected in the original image and the coordinates of the reference point in the reference image;

It should be noted that the reference point refers to the mark point set on the motherboard to be tested, which is usually set at a more obvious position, such as a hole. The coordinates of the reference point in the original image are the pixel coordinates of the reference point on the motherboard to be tested in the original image. The benchmark image refers to the template image set as a benchmark, and the coordinates of the reference point in the benchmark image are the pixel coordinates of the reference point on the motherboard to be tested in the benchmark image. In specific implementation, the reference point can be set as a feature point common to all motherboards to be tested.

It is understood that the conversion matrix refers to the calculation matrix required to convert the pixel coordinates in the reference image to the pixel coordinates in the original image. Generally speaking, the number of reference points needs to be greater than or equal to 2, so that the corresponding conversion matrix can be calculated based on the pixel coordinates of the reference points in the reference image and the original image.

Step S102, obtaining the coordinates of the target area in the reference image, and determining the coordinates of the key area to be detected based on the transformation matrix and the coordinates of the target area in the reference image;

It should be noted that the target area refers to the location of the defect that needs to be paid attention to at present, which can be flexibly selected according to actual needs. The area where all components on the motherboard are located can be selected, or the area where one or more components are located can be selected, and there is no specific limitation on this. The coordinates of the target area in the original image are the coordinates of the key area to be detected. Therefore, the coordinates of the target area in the reference image can be transformed using the transformation matrix to obtain the coordinates of the key area to be detected in the original image.

It is understandable that, since the locations of components are different, multiple target areas can be set. If multiple target areas are set, multiple key areas to be detected can be extracted accordingly.

Step S103: extracting the key area to be detected from the original image based on the coordinates of the key area to be detected.

It can be understood that all the key areas to be detected are extracted from the original image according to the coordinates of the key areas to be detected.

Step S20, based on the improved Anomalib model, the key area to be detected is detected to obtain abnormal probability distribution data of the key area to be detected, wherein the improved Anomalib model includes a preprocessing unit, a Bottleneck unit replacing the original feature extraction unit, and an abnormality recognition unit;

It should be noted that this embodiment uses an improved Anomalib model for defect detection. The improved Anomalib model includes a connected preprocessing unit, a Bottleneck unit that replaces the original feature extraction unit, and an anomaly recognition unit. Referring to FIG2 , the preprocessing unit, the Bottleneck unit, and the anomaly recognition unit are connected in sequence.

It can be understood that the preprocessing unit is used to extract visual features in the key area to be detected and input the visual features into the Bottleneck unit, the Bottleneck unit is used to generate dimensionality reduction features based on the visual features and input the dimensionality reduction features into the anomaly recognition unit for anomaly recognition.

In addition, it should be noted that the preprocessing unit at least includes a feature extractor, and the abnormality identification unit at least includes a NormalizingFlow layer and a FastFlow layer.

In a feasible implementation, the steps of training an improved Anomalib model include: acquiring a training image, and determining a training transformation matrix based on the coordinates of a training reference point in the training image and the coordinates of the training reference point in the reference image; determining the coordinates of a training key area in the training image based on the training transformation matrix and the coordinates of a target area; extracting the training key area from the training image based on the coordinates of the training key area; training the improved Anomalib model based on the training key area, and converting the improved Anomalib model and deploying it to a mobile terminal.

It should be noted that the training image, i.e. the initial training image, is usually a complete image of a defect-free motherboard. Since this embodiment performs defect detection on key areas in the motherboard image, the training image needs to further extract the corresponding key areas for model training. The training reference point refers to the mark point set on the defect-free motherboard used to shoot the training image, which is usually set in a more obvious position. The coordinates of the training reference point in the training image are the pixel coordinates of the training reference point in the training image, and the coordinates of the training reference point in the reference image are the pixel coordinates of the training reference point in the reference image. In the specific implementation, the training reference point can be set to a feature point common to these defect-free motherboards.

It can be understood that the training transformation matrix refers to the calculation matrix required to transform the pixel coordinates in the reference image to the pixel coordinates in the training image. Generally speaking, the number of training reference points needs to be greater than or equal to 2, so that the corresponding training transformation matrix can be calculated based on the pixel coordinates of the training reference points in the reference image and the training image. The coordinates of the target area in the training image are the coordinates of the training key area. Therefore, the training transformation matrix can be used to transform the coordinates of the target area in the reference image to obtain the coordinates of the training key area. The training key area is the key area extracted from the training image, which corresponds to the target area in the reference image and the key area to be detected in the original image.

It should be understood that the improved Anomalib model in this embodiment can be deployed on a mobile terminal, such as an Android terminal. In specific implementation, the Pytorch ckpt model can be trained using the training key area, and then converted into an onnx model to obtain the final improved Anomalib model, and finally deployed to the mobile terminal through onnxruntime.

In a feasible implementation, step S20 may include steps S201 to S204:

Step S201, inputting the vector data corresponding to the key area to be detected into the feature extractor for feature extraction to obtain the visual features of the key area to be detected;

It should be noted that the key area to be detected needs to be converted into corresponding vector data and input into the feature extractor, for example, converted into 1*3*512*512 vector data. The feature extractor can extract features from the key area to be detected and extract effective visual features for subsequent anomaly detection and positioning.

Step S202: input the visual features into the Bottleneck unit for feature extraction and dimensionality reduction processing to obtain the reduced dimensionality features.

It should be noted that the present embodiment uses the Bottleneck unit to replace the original feature extraction unit, which is the feature extraction unit used by the traditional Anomalib model and is not specifically limited. The Bottleneck unit can extract and reduce the dimension of the input visual features, and finally obtain the reduced dimension features, i.e., the reduced dimension features.

In addition, it should be noted that the Bottleneck unit includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer and a fourth convolutional layer connected in sequence. The second convolutional layer adopts a preset standard convolution kernel, and the convolution kernels of the first convolutional layer, the third convolutional layer and the fourth convolutional layer are the same, which are preset small-size convolution kernels. The preset standard convolution kernel is a standard convolution kernel, usually 3×3, and the preset small-size convolution kernel is a set convolution kernel of a smaller size, usually smaller than the preset standard convolution kernel, for example: 1×1. Exemplarily, referring to Figure 3, this embodiment introduces a new convolution operator ODConv (Omni-Dimensional Dynamic Convolution), and the preset small-size convolution kernel used is 1×1, and the preset standard convolution kernel used is 3×3.

In a feasible implementation, step S202 may include: inputting the visual features into the first convolution layer for dimensionality reduction processing to obtain reduced dimensionality visual features; inputting the reduced dimensionality visual features into the second convolution layer for feature extraction, and inputting the reduced dimensionality visual features after feature extraction into the third convolution layer for dimensionality increase processing to obtain increased dimensionality visual features; inputting the increased dimensionality visual features into the pooling layer for feature extraction, and inputting the increased dimensionality visual features after feature extraction into the fourth convolution layer for dimensionality reduction processing to obtain the reduced dimensionality features.

It can be understood that the first convolution layer with a convolution kernel of 1x1 is first used to reduce the dimension of the visual features to obtain preliminary reduced-dimensional visual features, i.e., reduced-dimensional visual features. Then, the second convolution layer with a convolution kernel of 3×3 is used to extract the reduced-dimensional visual features. Then, the third convolution layer with a convolution kernel of 1×1 is input to increase the dimension to obtain further processed visual features, i.e., increased-dimensional visual features. Then, the pooling layer is used to extract the increased-dimensional visual features. Finally, the fourth convolution layer with a convolution kernel of 1×1 is input to reduce the dimension to obtain the final required reduced-dimensional features.

It should be understood that the Bottleneck unit can be used to reduce the dimension of features, effectively reduce the amount of model calculations, reduce model inference time, and improve accuracy.

Step S203, inputting the dimension reduction features into the NormalizingFlow layer for distribution normalization to obtain feature normal distribution data;

It should be noted that the main function of the Normalizing Flow layer is to model the distribution of dimensionality reduction features through a two-dimensional normalizing flow, thereby mapping the features to a standard normal distribution that is easier to handle, allowing the model to detect and locate anomalies more accurately. Feature normal distribution data refers to the feature distribution obtained after mapping the dimensionality reduction features to a standard normal distribution.

Step S204: input the characteristic normal distribution data into the FastFlow layer for anomaly identification to obtain the anomaly probability distribution data of the key area to be detected.

In a feasible implementation, step S204 may include: inputting the characteristic normal distribution data into the FastFlow layer, and determining the abnormality scores of different positions in the key area to be detected based on the distance between the characteristic normal distribution data and the benchmark characteristic normal distribution data; determining the abnormality probability of different positions in the key area to be detected based on the abnormality scores of different positions in the key area to be detected; and determining the abnormal probability distribution data of the key area to be detected based on the abnormality probability of different positions in the key area to be detected.

It should be noted that the benchmark feature normal distribution data is the feature distribution used as a benchmark, which is usually the feature distribution of the key training area.

In addition, it should be noted that the anomaly score can quantitatively evaluate the degree of deviation between a data point and the normal data distribution, and can usually be used as a basis for judging whether the data is abnormal. The anomaly scores of different locations in the key area to be detected can be calculated by comparing the degree of difference between the characteristic normal distribution data and the benchmark characteristic normal distribution data. This difference can usually be quantified by a distance metric, such as Euclidean distance, Mahalanobis distance, etc. The larger the distance, the higher the degree of abnormality. It can be seen that the higher the anomaly score, the more likely it is abnormal. Exemplarily, the calculation relationship of the anomaly score is as follows:

in, is the characteristic normal distribution data of the key area to be detected, is the benchmark characteristic normal distribution data, is the distance between the characteristic normal distribution data and the benchmark characteristic normal distribution data, that is, the anomaly score.

It can be understood that the anomaly probability at different locations in the key area to be detected refers to the probability that the data point at that location belongs to the anomaly class, which is closely related to the anomaly score. Positions with higher anomaly scores correspond to higher anomaly probabilities, which means that the location is more likely to be a defect location. The anomaly probability is usually a probabilistic assessment further obtained based on these anomaly scores, and it is necessary to consider the interaction between features and other possible influencing factors to provide a more comprehensive anomaly evaluation. According to the anomaly probability at different locations in the key area to be detected, the distribution of the anomaly probability in the key area to be detected can be obtained, that is, the anomaly probability distribution data.

Furthermore, the improved Anomalib model adopts the new Relu4 activation function to replace the original Relu activation function.

It should be noted that the original Relu activation function is , the loss is large in the low-precision transformation process. Therefore, referring to the function diagram shown in FIG4 , this embodiment improves the Relu activation function and designs a new Relu4 activation function, namely , which can achieve better robustness in mobile terminals that require efficiency. In this embodiment, all Relu activation functions in the traditional Anomalib model are replaced by Relu4 activation functions.

It can be understood that since this embodiment uses the Bottleneck unit to replace the original feature extraction unit, the calculation amount of the model is greatly reduced, the model inference time is reduced, the model calculation efficiency is improved, and the new Relu4 activation function is used to replace the original Relu activation function, which can improve the robustness of the model when applied on the mobile terminal, so that the improved Anomalib model can be deployed on the mobile terminal for application.

It should be understood that the Anomalib model in this embodiment refers to an anomaly detection model that includes a feature extractor, a feature extraction unit, a NormalizingFlow layer and a FastFlow layer, and uses a Relu activation function, so that the Anomalib model can be optimized, the original feature extraction unit is replaced by a Bottleneck unit, and the original Relu activation function is replaced by a new Relu4 activation function to obtain an improved Anomalib model for improved motherboard defect detection based on the anomaly detection model.

Step S30, determining the defect position in the key area to be detected based on the abnormal probability distribution data of the key area to be detected;

It should be noted that the defect position is the site/area where the defect exists. The defect position in the key area to be detected can be screened out by setting a probability threshold. Usually, the position where the abnormal probability is greater than the probability threshold is considered to be the defect position. In specific implementation, the probability threshold can be set to 0.5, or it can be flexibly adjusted according to actual needs, and there is no specific limitation on this.

Step S40, marking the key area to be inspected and the defect positions in the key area to be inspected respectively, and generating a result heat map of the motherboard to be inspected.

It can be understood that this embodiment can ultimately output a heat map corresponding to the detection results, that is, a result heat map. In the result heat map, each key area to be detected can be marked, and the defect position of each key area to be detected can be marked with a bright color (for example: red, which needs to be different from the marking color of the key area to be detected).

The present embodiment provides a mainboard defect detection method improved based on an anomaly detection model, which obtains an original image of a mainboard to be detected, and extracts a key area to be detected from the original image of the mainboard to be detected; based on an improved Anomalib model, the key area to be detected is detected to obtain abnormal probability distribution data of the key area to be detected, and the improved Anomalib model includes a preprocessing unit, a Bottleneck unit that replaces an original feature extraction unit, and an abnormality recognition unit, the preprocessing unit is used to extract visual features in the key area to be detected, and input the visual features into the Bottleneck unit, the Bottleneck unit is used to generate a dimensionality reduction feature based on the visual features, and input the dimensionality reduction feature into the abnormality recognition unit for abnormality recognition; based on the abnormal probability distribution data of the key area to be detected, the defect position in the key area to be detected is determined; the key area to be detected and the defect position in the key area to be detected are marked respectively, and a result heat map of the mainboard to be detected is generated. The model structure is optimized, the Bottleneck unit is used for dimensionality reduction, and a new activation function with better robustness is adopted to reduce the amount of inference calculations, reduce the time spent on inference, and improve the inference efficiency of the model. At the same time, the key areas of the image are used for training and inference to improve the model's pertinence, accuracy and inference efficiency, thereby improving the accuracy and efficiency of defect detection. Even if the image resolution is large, the detection results can be obtained quickly and accurately.

It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the improved motherboard defect detection method based on the abnormal detection model of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

The present application also provides a mainboard defect detection device improved based on an anomaly detection model. Please refer to FIG. 5 . The mainboard defect detection device improved based on an anomaly detection model includes:

The region extraction module 10 is used to obtain the original image of the motherboard to be detected, and extract the key region to be detected from the original image of the motherboard to be detected.

The defect detection module 20 is used to detect the key area to be detected based on the improved Anomalib model to obtain the abnormal probability distribution data of the key area to be detected. The improved Anomalib model includes a preprocessing unit, a Bottleneck unit that replaces the original feature extraction unit, and an abnormality recognition unit. The preprocessing unit is used to extract the visual features in the key area to be detected and input the visual features into the Bottleneck unit. The Bottleneck unit is used to generate a dimensionality reduction feature based on the visual feature, and input the dimensionality reduction feature into the abnormality recognition unit for abnormality recognition.

The defect detection module is further used to determine the defect position in the key area to be detected based on the abnormal probability distribution data of the key area to be detected.

The defect detection module is also used to mark the key area to be detected and the defect position in the key area to be detected respectively, and generate a result heat map of the motherboard to be detected.

In a feasible implementation, the improved Anomalib model uses a new Relu4 activation function to replace the original Relu activation function.

In a feasible implementation manner, the region extraction module 10 is further used to obtain an original image of the motherboard to be detected, and determine a transformation matrix based on the coordinates of a reference point on the motherboard to be detected in the original image and the coordinates of the reference point in the reference image;

Acquire the coordinates of the target area in the reference image, and determine the coordinates of the key area to be detected based on the transformation matrix and the coordinates of the target area in the reference image;

Based on the coordinates of the key area to be detected, the key area to be detected is extracted from the original image.

In a feasible implementation manner, the preprocessing unit includes at least a feature extractor, the abnormality identification unit includes at least a NormalizingFlow layer and a FastFlow layer, and the defect detection module is further used to input the vector data corresponding to the key area to be detected into the feature extractor for feature extraction to obtain the visual features of the key area to be detected;

Inputting the visual features into the Bottleneck unit for feature extraction and dimensionality reduction processing to obtain the reduced dimensionality features;

Input the dimension reduction features into the NormalizingFlow layer for distribution normalization to obtain feature normal distribution data;

The characteristic normal distribution data is input into the FastFlow layer for anomaly recognition to obtain the abnormal probability distribution data of the key area to be detected.

In a feasible implementation manner, the Bottleneck unit includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer, and a fourth convolutional layer connected in sequence, the second convolutional layer adopts a preset standard convolutional kernel, the first convolutional layer, the third convolutional layer, and the fourth convolutional layer all adopt a preset small-size convolutional kernel, and the preset small-size convolutional kernel is smaller than the preset standard convolutional kernel, and the defect detection module is further used to input the visual feature into the first convolutional layer for dimensionality reduction processing to obtain a reduced-dimensional visual feature;

Inputting the reduced-dimensional visual features into the second convolutional layer for feature extraction, and inputting the reduced-dimensional visual features after feature extraction into the third convolutional layer for dimensionality increase processing to obtain the increased-dimensional visual features;

The up-dimensional visual features are input into the pooling layer for feature extraction, and the up-dimensional visual features after feature extraction are input into the fourth convolutional layer for dimensionality reduction processing to obtain the reduced dimensionality features.

In a feasible implementation manner, the defect detection module is further used to input the characteristic normal distribution data into the FastFlow layer, and determine the abnormality scores of different positions in the key area to be detected based on the distance between the characteristic normal distribution data and the reference characteristic normal distribution data;

Determining the abnormality probabilities of different positions in the key area to be detected based on the abnormality scores of different positions in the key area to be detected;

Based on the abnormality probabilities at different positions in the key area to be detected, abnormality probability distribution data of the key area to be detected is determined.

In a feasible implementation manner, the defect detection module is further used to obtain a training image, and determine a training transformation matrix based on coordinates of a training reference point in the training image and coordinates of the training reference point in the reference image;

Determining the coordinates of the training key area in the training image based on the training transformation matrix and the coordinates of the target area;

Extracting the training key area from the training image based on the coordinates of the training key area;

Based on the training key area, the improved Anomalib model is trained and the improved Anomalib model is converted and deployed to the mobile terminal.

The improved motherboard defect detection device based on the anomaly detection model provided in the present application adopts the improved motherboard defect detection method based on the anomaly detection model in the above-mentioned embodiment, which can solve the technical problem of low accuracy and efficiency when the traditional Anomalib model performs defect detection on a motherboard image with a large resolution. Compared with the prior art, the beneficial effects of the improved motherboard defect detection device based on the anomaly detection model provided in the present application are the same as the beneficial effects of the improved motherboard defect detection method based on the anomaly detection model provided in the above-mentioned embodiment, and the other technical features of the improved motherboard defect detection device based on the anomaly detection model are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

The present application provides a motherboard defect detection device improved based on an anomaly detection model, and the motherboard defect detection device improved based on an anomaly detection model includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the motherboard defect detection method improved based on the anomaly detection model in the above-mentioned embodiment one.

Reference is made to Figure 6 below, which shows a schematic diagram of the structure of a motherboard defect detection device based on an abnormality detection model improvement suitable for implementing an embodiment of the present application. The motherboard defect detection device based on an abnormality detection model improvement in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The motherboard defect detection device based on an abnormality detection model improvement shown in Figure 6 is only an example and should not bring any limitations to the functions and scope of use of the embodiments of the present application.

As shown in FIG6 , the improved motherboard defect detection device based on the abnormal detection model may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the improved motherboard defect detection device based on the abnormal detection model are also stored in RAM 1004. The processing device 1001, ROM 1002, and RAM 1004 are connected to each other via a bus 1005. An input/output (I/O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I/O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the improved motherboard defect detection device based on the abnormality detection model to communicate wirelessly or wired with other devices to exchange data. Although the improved motherboard defect detection device based on the abnormality detection model with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

The improved motherboard defect detection device based on the anomaly detection model provided in the present application adopts the improved motherboard defect detection method based on the anomaly detection model in the above-mentioned embodiment, which can solve the technical problem of low accuracy and efficiency when the traditional Anomalib model performs defect detection on a motherboard image with a large resolution. Compared with the prior art, the beneficial effects of the improved motherboard defect detection device based on the anomaly detection model provided in the present application are the same as the beneficial effects of the improved motherboard defect detection method based on the anomaly detection model provided in the above-mentioned embodiment, and the other technical features of the improved motherboard defect detection device based on the anomaly detection model are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the improved motherboard defect detection method based on the abnormality detection model in the above-mentioned embodiment.

The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

The above-mentioned computer-readable storage medium may be included in the mainboard defect detection device improved based on the abnormality detection model; or it may exist independently without being assembled into the mainboard defect detection device improved based on the abnormality detection model.

The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the motherboard defect detection device improved based on the anomaly detection model, the motherboard defect detection device improved based on the anomaly detection model: obtains the original image of the motherboard to be detected, and extracts the key area to be detected in the original image of the motherboard to be detected; based on the improved Anomalib model, detects the key area to be detected to obtain the abnormal probability distribution data of the key area to be detected, the improved Anomalib model includes a preprocessing unit, a Bottleneck unit that replaces the original feature extraction unit, and an abnormality recognition unit, the preprocessing unit is used to extract the visual features in the key area to be detected, and input the visual features into the Bottleneck unit, the Bottleneck unit is used to generate a dimensionality reduction feature based on the visual features, and input the dimensionality reduction feature into the abnormality recognition unit for abnormality recognition; based on the abnormal probability distribution data of the key area to be detected, the defect position in the key area to be detected is determined; the key area to be detected and the defect position in the key area to be detected are marked respectively, and a result heat map of the motherboard to be detected is generated.

Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and/or flow chart, and the combination of the boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The modules involved in the embodiments of the present application may be implemented by software or hardware, wherein the name of the module does not, in some cases, constitute a limitation on the unit itself.

The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned improved motherboard defect detection method based on the anomaly detection model, and can solve the technical problem of low accuracy and efficiency when the traditional Anomalib model performs defect detection on large-resolution motherboard images. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the improved motherboard defect detection method based on the anomaly detection model provided in the above-mentioned embodiment, and will not be repeated here.

The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned improved mainboard defect detection method based on anomaly detection model.

The computer program product provided by the present application can solve the technical problem of low accuracy and efficiency when the traditional Anomalib model performs defect detection on a motherboard image with a large resolution. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the improved motherboard defect detection method based on the anomaly detection model provided by the above embodiment, and will not be described in detail here.

The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct/indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims (10)

1. A method for detecting motherboard defects based on an improved anomaly detection model, characterized in that the method comprises: Acquire an original image of the motherboard to be detected, and extract a key area to be detected from the original image of the motherboard to be detected; Based on the improved Anomalib model, the key area to be detected is detected to obtain abnormal probability distribution data of the key area to be detected, the improved Anomalib model includes a preprocessing unit, a Bottleneck unit that replaces the original feature extraction unit, and an abnormality recognition unit, the preprocessing unit is used to extract visual features in the key area to be detected, and input the visual features into the Bottleneck unit, the Bottleneck unit is used to generate dimensionality reduction features based on the visual features, and input the dimensionality reduction features into the abnormality recognition unit for abnormality recognition; Determining the defect position in the key area to be detected based on the abnormal probability distribution data of the key area to be detected; The key area to be inspected and the defect positions in the key area to be inspected are marked respectively to generate a result heat map of the motherboard to be inspected. 2. The method according to claim 1, wherein the improved Anomalib model uses a new Relu4 activation function to replace the original Relu activation function. 3. The method according to claim 1, wherein the step of acquiring the original image of the motherboard to be detected and extracting the key area to be detected from the original image of the motherboard to be detected comprises: Acquire an original image of the motherboard to be detected, and determine a transformation matrix based on the coordinates of a reference point on the motherboard to be detected in the original image and the coordinates of the reference point in the reference image; Acquire the coordinates of the target area in the reference image, and determine the coordinates of the key area to be detected based on the transformation matrix and the coordinates of the target area in the reference image; Based on the coordinates of the key area to be detected, the key area to be detected is extracted from the original image. 4. The method according to claim 1, characterized in that the preprocessing unit includes at least a feature extractor, the abnormality identification unit includes at least a NormalizingFlow layer and a FastFlow layer, and the step of detecting the key area to be detected based on the improved Anomalib model to obtain the abnormal probability distribution data of the key area to be detected includes: Inputting the vector data corresponding to the key area to be detected into the feature extractor for feature extraction to obtain the visual features of the key area to be detected; Inputting the visual features into the Bottleneck unit for feature extraction and dimensionality reduction processing to obtain the reduced dimensionality features; Input the dimension reduction features into the NormalizingFlow layer for distribution normalization to obtain feature normal distribution data; The characteristic normal distribution data is input into the FastFlow layer for anomaly recognition to obtain the abnormal probability distribution data of the key area to be detected. 5. The method according to claim 4, characterized in that the Bottleneck unit comprises a first convolutional layer, a second convolutional layer, a third convolutional layer, a pooling layer and a fourth convolutional layer connected in sequence, and the step of inputting the visual features into the Bottleneck unit for feature extraction and dimensionality reduction to obtain the dimensionality reduction features further comprises: Inputting the visual features into the first convolutional layer for dimensionality reduction processing to obtain dimensionality-reduced visual features; Inputting the reduced-dimensional visual features into the second convolutional layer for feature extraction, and inputting the reduced-dimensional visual features after feature extraction into the third convolutional layer for dimensionality increase processing to obtain the increased-dimensional visual features; The up-dimensional visual features are input into the pooling layer for feature extraction, and the up-dimensional visual features after feature extraction are input into the fourth convolutional layer for dimensionality reduction processing to obtain the reduced dimensionality features. 6. The method as claimed in claim 5 is characterized in that the second convolution layer adopts a preset standard convolution kernel, and the first convolution layer, the third convolution layer and the fourth convolution layer all adopt a preset small-size convolution kernel, and the preset small-size convolution kernel is smaller than the preset standard convolution kernel. 7. The method according to claim 4, characterized in that the step of inputting the characteristic normal distribution data into the FastFlow layer for abnormality identification to obtain the abnormal probability distribution data of the key area to be detected comprises: Inputting the characteristic normal distribution data into the FastFlow layer, and determining the abnormal scores of different positions in the key area to be detected based on the distance between the characteristic normal distribution data and the reference characteristic normal distribution data; Determining the abnormality probabilities of different positions in the key area to be detected based on the abnormality scores of different positions in the key area to be detected; Based on the abnormality probabilities at different positions in the key area to be detected, abnormality probability distribution data of the key area to be detected is determined. 8. The method according to any one of claims 1 to 7, characterized in that the method further comprises: Acquire a training image, and determine a training transformation matrix based on coordinates of a training reference point in the training image and coordinates of the training reference point in the reference image; Determining the coordinates of the training key area in the training image based on the training transformation matrix and the coordinates of the target area; Extracting the training key area from the training image based on the coordinates of the training key area; Based on the training key area, the improved Anomalib model is trained and the improved Anomalib model is converted and deployed to the mobile terminal. 9. A motherboard defect detection device based on an improved anomaly detection model, characterized in that the motherboard defect detection device based on an improved anomaly detection model comprises: The region extraction module is used to obtain the original image of the motherboard to be detected, and extract the key region to be detected from the original image of the motherboard to be detected; A defect detection module, configured to detect the key area to be detected based on an improved Anomalib model to obtain abnormal probability distribution data of the key area to be detected, wherein the improved Anomalib model includes a preprocessing unit, a Bottleneck unit replacing an original feature extraction unit, and an abnormality recognition unit, wherein the preprocessing unit is configured to extract visual features in the key area to be detected and input the visual features into the Bottleneck unit, and the Bottleneck unit is configured to generate a dimensionality reduction feature based on the visual features and input the dimensionality reduction feature into the abnormality recognition unit for abnormality recognition; The defect detection module is further used to determine the defect position in the key area to be detected based on the abnormal probability distribution data of the key area to be detected; The defect detection module is also used to mark the key area to be detected and the defect position in the key area to be detected respectively, and generate a result heat map of the motherboard to be detected. 10. A motherboard defect detection device based on an improved anomaly detection model, characterized in that the device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the motherboard defect detection method based on an improved anomaly detection model as described in any one of claims 1 to 8.