CN116168388A - Litchi epidermis defect identification method and system - Google Patents

Abstract

The invention discloses a litchi epidermis defect identification method and system. After obtaining the litchi epidermis defect image to be detected, the target detection model built by adding the improved attention mechanism SimAM network into the main feature extraction network is called to perform the litchi epidermis defect image. For detection, first adjust the litchi skin defect image to the same resolution, and then perform grid division on the litchi skin defect image, and traverse the grids of each marked litchi skin defect image to obtain multiple bounding boxes in each grid, Identify and score the skin defects of litchi in each bounding box to obtain the credibility score of each bounding box. According to the credibility score of the bounding box, select the bounding box larger than the preset requirements as the final detection frame, and output the litchi according to the final detection frame. Defect position on the skin defect image, by using the improved algorithm to build a detection model to identify and classify litchi defects, and propose litchi skin defects, which can improve the accuracy of model detection.

Description

A method and system for identifying skin defects of litchi

technical field

The invention relates to the technical field of target detection, in particular to a method and system for identifying skin defects of litchi.

Background technique

The traditional appearance quality inspection of litchi mainly relies on manual sorting by visual inspection and fruit selection board comparison, which is labor-intensive and low in efficiency. The grading effect is poor due to irregular times; and the lychees will be damaged to varying degrees after manual selection, causing secondary pollution and affecting the food hygiene and safety of the product.

The current lychee grading machines in China sort lychees by controlling the gap between diameter and size. They cannot detect surface defects of lychees, and lychees are repeatedly rubbed against machinery during the grading process, which is prone to damage and degradation. In view of the above phenomena, a recognition method for the classification of litchi skin defects has been developed, which is of great significance to the reduction of labor costs after the fruit and vegetable industry, and to the development of agricultural products and agricultural economy. Machine vision is a key technology for the classification of litchi skin defects, and it provides a technical means for litchi quality inspection. Existing technologies that use computer vision to detect the quality of agricultural products.

Existing technologies include defect extraction algorithms based on color features and illuminance-reflection models, but these algorithms need to set thresholds, and the adaptability is not high. Some fruit defect areas have obvious edges with normal areas, so an edge-based The defect extraction algorithm of information has achieved good results on smooth-skinned fruits, but the effect on lychees is not ideal. In addition, there are defect extraction methods based on traditional supervised learning such as Bayesian classifiers and SVMs. According to the pixel value, the pixels are divided into two categories: "normal" and "defect", so as to realize defect segmentation. Spot-like protrusions, the above algorithm cannot be well applied to the extraction of epidermal defects of litchi.

Contents of the invention

The invention provides a litchi epidermis defect identification method and system. By using the improved backbone feature network to build a target detection model to identify and classify litchi defects, the accuracy and versatility of the target detection model are improved.

In order to solve the above technical problems, the embodiment of the present invention provides a litchi skin defect identification method and system, including:

Obtain an image of the skin defect of litchi to be detected;

Call the target detection model built by adding the improved attention mechanism SimAM network to the backbone feature extraction network to detect the marked litchi skin defect image, so that after the target detection model adjusts the litchi skin defect image to the same resolution, After meshing the litchi skin defect image, traverse the grids of each marked litchi skin defect image to obtain multiple bounding boxes in each grid, and identify and score the litchi skin defects in each bounding box to obtain each bounding box According to the credibility score of the bounding box, the bounding box larger than the preset requirement is screened out as the final detection frame, and the detection result of the litchi skin defect is output according to the final detection frame, wherein, the detection result of the litchi skin defect includes the litchi skin defect The location of the defect on the image.

Implement this embodiment, obtain the defect image of the litchi skin to be detected, and call the target detection model built by adding the improved attention mechanism SimAM network in the backbone feature extraction network to detect the defect image of the litchi skin. After the defect images are adjusted to the same resolution, after the litchi skin defect images are meshed, the grids of each litchi skin defect image are traversed to obtain multiple bounding boxes in each grid, and the litchi skin in each bounding box is Defects are identified and scored to obtain the reliability scores of each bounding box. According to the reliability scores of the bounding boxes, the bounding boxes larger than the preset requirements are selected as the final detection frame, and the defect position on the litchi skin defect image is output according to the final detection frame. By using the improved algorithm to build a detection model to identify and classify litchi defects, and to propose litchi skin defects, the accuracy of model detection can be improved.

As a preferred solution, the improved attention mechanism SimAM network is obtained by redefining the energy function, solution formula and feature extraction process, specifically:

Redefine the energy function of the attention mechanism SimAM network by using binary labels and adding regular terms, where the energy function formula is:

为t和x_i的线性变换，其中，t和x_i表示输入特征/>

的单个通道中的目标神经元和其他神经元，C，H和W分别为通道注意力的长，高和宽，i是指数除以空间维度，M＝H×W是通道上的神经元数量，ω_t和b_t分别表示变换的权值和偏置；in,

is the linear transformation of t and _xi , where t and _xi represent input features/>

The target neuron and other neurons in a single channel of , C, H and W are the length, height and width of the channel attention respectively, i is the index divided by the spatial dimension, M=H×W is the number of neurons on the channel , ω _t and b _t represent the weight and bias of the transformation, respectively;

Use the solution formula to update the energy function solution method, and the solution formula is:

μ_t表示对应通道中的除t以外的所有神经元的均值，/>

表示对应通道中的除t以外的所有神经元的方差，/>

表示能每个神经元的重要性。in,

μ _t represents the mean value of all neurons in the corresponding channel except t, />

Indicates the variance of all neurons in the corresponding channel except t, />

Indicates the importance of each neuron.

The defect feature is extracted by zooming operation, where the feature extraction process is:

在所有通道和空间维度的汇总，sigmoid表示约束过大的值。Among them, E means

Aggregated across all channels and spatial dimensions, sigmoid represents overly constrained values.

Implementing the embodiment of the present invention, by using binary labels and adding a regular term to redefine the energy function of the attention mechanism SimAM network, and using the updated solution formula to solve the energy function, the constructed target detection model can be detected In this method, the calculation efficiency can be improved, and the skin defect features of litchi can be extracted quickly.

As an optimal solution, identify and score the skin defects of litchi in the bounding box to obtain the credibility score of the bounding box, and filter out the bounding box larger than the preset requirement according to the credibility score of the bounding box as the final detection box, specifically:

Identify and score the litchi skin defects in the bounding box to obtain the confidence score of the bounding box;

After deleting the bounding boxes with scores lower than the first preset score, arrange the remaining bounding boxes according to the reliability scores from large to small to obtain bounding box arrangement results;

Filter out bounding boxes that are lower than the preset error after being fused with the bounding box ranked first in the bounding box arrangement results to obtain the first bounding box detection set, wherein the bounding boxes of the first bounding box detection set are based on the confidence score according to Arranged from largest to smallest;

After filtering out the bounding boxes that are lower than the preset error after the fusion of the first bounding box detection set and the second-ranked bounding box, the second bounding box detection set is obtained, and the first-ranked bounding box is filtered from the second bounding box detection set. The bounding box serves as the final detection box.

As a preferred solution, the lychee skin defects in the bounding box are identified and scored to obtain the confidence score of the bounding box, specifically:

Identifying the skin defect of litchi in the bounding box to obtain the recognition score;

The IOU score is obtained according to the coverage area of the bounding box and the actual defect box;

The confidence score is obtained by multiplying the recognition score and the IOU score.

As a preferred solution, the target detection model is obtained by training through a training method, and the training method is:

Obtain the image of litchi skin defect to be detected, and perform oversampling and data augmentation on each litchi defect image to obtain the litchi skin defect data set;

After labeling each litchi skin defect image in the litchi skin defect data set, an annotated litchi skin defect data set is obtained;

Call the target detection model built by adding the improved attention mechanism SimAM network to the backbone feature extraction network to detect the marked litchi skin defect image, so that after the target detection model adjusts the litchi skin defect image to the same resolution, After meshing the litchi skin defect image, traverse the grids of each marked litchi skin defect image to obtain the bounding box in each grid, and then identify and score the litchi skin defect in the bounding box to obtain the credibility of the bounding box Degree score, according to the confidence score of the bounding box, the bounding box larger than the preset requirement is screened out as the final detection box;

The loss function is used to calculate the error between the final detection frame and the expected result to evaluate the target detection model, and then update the model parameters. Select undetected litchi skin defect images in the litchi skin defect dataset for detection until the preset training times are reached to obtain the target detection model.

As a preferred solution, after labeling each litchi skin defect image in the litchi skin defect data set, the marked litchi skin defect data set is obtained, specifically:

Use a rectangular frame to mark out the litchi defect area in the image to obtain multiple bounding boxes for the litchi skin defect in the litchi skin defect image;

Set the corresponding label type according to the bounding box, where the label type includes black dot drop and crack.

As a preferred solution, in order to solve the same technical problem, an embodiment of the present invention also provides a litchi skin defect identification system, including a sample acquisition module, a labeling module and a detection module;

Wherein, the sample acquisition module is used to acquire the image of the skin defect of litchi to be detected;

The marking module is used to mark the litchi skin defect image to be detected to obtain the marked litchi skin defect image, wherein the marked litchi skin defect image includes a plurality of real frames;

The detection module is used to call the target detection model built by adding the improved attention mechanism SimAM network to the backbone feature extraction network to detect the marked litchi skin defect image, so that the target detection model adjusts the litchi skin defect image to the same After the resolution, after the litchi skin defect image is meshed, the grid of the litchi skin defect image after each label is traversed to obtain a plurality of bounding boxes in each grid, and the litchi skin defect in each bounding box is carried out Recognize and score to obtain the credibility scores of each bounding box, screen out the bounding boxes larger than the preset requirements according to the credibility scores of the bounding boxes as the final detection frame, and output the detection results of litchi skin defects according to the final detection frame, wherein, litchi The skin defect detection results include defect locations on the litchi skin defect image.

As a preferred solution, the detection module includes a credibility score calculation unit, a bounding box arrangement unit, a first screening unit and a second screening unit,

The credibility score calculation unit is used to identify and score the litchi skin defects in the bounding box to obtain the credibility score of the bounding box;

The bounding box arranging unit is used to delete the bounding boxes with scores lower than the first preset score, and then arrange the remaining bounding boxes according to the reliability scores in descending order to obtain the bounding box arrangement result;

The first screening unit is used to filter out bounding boxes that are lower than the preset error after being fused with the bounding box ranked first in the bounding box arrangement result to obtain the first bounding box detection set, wherein the bounding boxes of the first bounding box detection set According to the credibility scores, they are arranged in descending order;

The second screening unit is used to filter out the bounding boxes that are lower than the preset error after the fusion of the first bounding box detection set and the second-ranked bounding box to obtain the second bounding box detection set, and from the second bounding box detection set Filter out the top bounding box as the final detection box.

As a preferred solution, in order to solve the same technical problem, an embodiment of the present invention also provides a terminal device, including: a memory, a processor, and a computer program stored on the memory and operable on the processor, when the processor executes the program Realize the litchi epidermis defect recognition method as shown in the embodiment of the present invention.

As a preferred solution, in order to solve the same technical problem, the embodiment of the present invention also provides a computer-readable storage medium, the computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable the computer to execute the present invention. The steps of the litchi skin defect recognition method shown in the embodiment.

Obtain the image of litchi skin defect to be detected, and call the target detection model built by adding the improved attention mechanism SimAM network to the backbone feature extraction network to detect the marked litchi skin defect image. The target detection model first takes the litchi skin defect image After adjusting to the same resolution, after the litchi skin defect image is meshed, traverse the grids of each marked litchi skin defect image to obtain multiple bounding boxes in each grid, and for the litchi skin defect images in each bounding box Identify and score the skin defects to obtain the credibility scores of each bounding box, screen out the bounding boxes larger than the preset requirements according to the credibility scores of the bounding boxes as the final detection frame, and output the defect position on the litchi skin defect image according to the final detection frame . By using the improved algorithm to build a detection model to identify and classify litchi defects, and to propose litchi skin defects, the accuracy of model detection can be improved.

Description of drawings

Fig. 1: a schematic flow chart of an embodiment of the litchi skin defect identification method provided by the present invention;

Figure 2: Schematic diagram of the improved YOLOv7 backbone network structure of an embodiment of the litchi skin defect recognition method provided by the present invention;

Fig. 3: a schematic diagram of the model training process of an embodiment of the litchi skin defect recognition method provided by the present invention;

Fig. 4: A structural schematic diagram of another embodiment of the method for identifying skin defects of litchi provided by the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Embodiment one

Please refer to Fig. 1, which is a method for identifying skin defects of litchi provided by an embodiment of the present invention. The method for identifying skin defects of litchi includes steps 101 to 102, and each step is specifically as follows:

Step 101: Obtain an image of skin defects of litchi to be detected.

In this embodiment, an image of skin defects of litchi to be detected is acquired.

Step 102: Call the target detection model built by adding the improved attention mechanism SimAM network to the backbone feature extraction network to detect the marked litchi skin defect image, so that the target detection model adjusts the litchi skin defect image to the same resolution After performing grid division on the litchi skin defect image, traverse the grids of each marked litchi skin defect image to obtain a plurality of bounding boxes in each grid, and identify the litchi skin defect in each bounding box Score to get the credibility score of each bounding box, screen out the bounding box larger than the preset requirement according to the credibility score of the bounding box as the final detection frame, and output the detection result of litchi skin defect according to the final detection frame, among which, litchi skin defect detection The results include defect locations on the litchi skin defect image.

Optionally, the improved attention mechanism SimAM network is obtained by redefining the energy function, solution formula and feature extraction process, specifically:

Redefine the energy function of the attention mechanism SimAM network by using binary labels and adding regular terms, where the energy function formula is:

为t和x_i的线性变换，其中，t和x_i表示输入特征/>

的单个通道中的目标神经元和其他神经元，C，H和W分别为通道注意力的长，高和宽，i是指数除以空间维度，M＝H×W是通道上的神经元数量，ω_t和b_t分别表示变换的权值和偏置；in,

is the linear transformation of t and _xi , where t and _xi represent input features/>

The target neuron and other neurons in a single channel of , C, H and W are the length, height and width of the channel attention respectively, i is the index divided by the spatial dimension, M=H×W is the number of neurons on the channel , ω _t and b _t represent the weight and bias of the transformation, respectively;

Use the solution formula to update the energy function solution method, and the solution formula is:

μ_t表示对应通道中的除t以外的所有神经元的均值，/>

表示对应通道中的除t以外的所有神经元的方差，/>

表示能每个神经元的重要性。in,

μ _t represents the mean value of all neurons in the corresponding channel except t, />

Indicates the variance of all neurons in the corresponding channel except t, />

Indicates the importance of each neuron.

The defect feature is extracted by zooming operation, where the feature extraction process is:

Optionally, identify and score the litchi skin defects in the bounding box to obtain the confidence score of the bounding box, and filter out the bounding box larger than the preset requirement as the final detection box according to the credibility score of the bounding box, specifically:

Identify and score the litchi skin defects in the bounding box to obtain the confidence score of the bounding box;

After deleting the bounding boxes with scores lower than the first preset score, arrange the remaining bounding boxes according to the reliability scores from large to small to obtain bounding box arrangement results;

Filter out bounding boxes that are lower than the preset error after being fused with the bounding box ranked first in the bounding box arrangement results to obtain the first bounding box detection set, wherein the bounding boxes of the first bounding box detection set are based on the confidence score according to Arranged from largest to smallest;

After filtering out the bounding boxes that are lower than the preset error after the fusion of the first bounding box detection set and the second-ranked bounding box, the second bounding box detection set is obtained, and the first-ranked bounding box is filtered from the second bounding box detection set. The bounding box serves as the final detection box.

Optionally, identify and score the litchi skin defects in the bounding box to obtain the confidence score of the bounding box, specifically:

Identify the litchi skin defects in the bounding box to obtain the recognition score;

The IOU score is obtained according to the coverage area of the bounding box and the actual defect box;

The confidence score is obtained by multiplying the recognition score and the IOU score.

Specifically, in this embodiment, the backbone feature extraction network of the target detection model is an improved YOLOv7 backbone network, which is composed of the attention mechanism SimAM added to the YOLOv7 backbone network. The improved YOLOv7 backbone network structure is shown in Figure 2.

The attention mechanism SimAM does not need to add parameters to the original network, but infers the 3-D attention weights of the feature map in one layer, which can adopt the simplest method of finding important neurons: measuring the linear separability between neurons , the following energy function is defined for each neuron:

为t和x_i的线性变换，其中，t和x_i表示输入特征/>

的单个通道中的目标神经元和其他神经元，C，H和W分别为通道注意力的长，高和宽，i是指数除以空间维度，M＝H×W是通道上的神经元数量，ω_t和b_t分别表示变换的权值和偏置；in,

is the linear transformation of t and _xi , where t and _xi represent input features/>

The target neuron and other neurons in a single channel of , C, H and W are the length, height and width of the channel attention respectively, i is the index divided by the spatial dimension, M=H×W is the number of neurons on the channel , ω _t and b _t represent the weight and bias of the transformation, respectively;

When t= _t , the minimum value is obtained. By minimizing this equation, which is equivalent to finding the linear separability between the target neuron t and all other neurons in the same channel, binary labels can be taken for simplicity, and a regularization term added, the final energy function It is defined as follows:

为t和x_i的线性变换，其中，t和x_i表示输入特征/>

的单个通道中的目标神经元和其他神经元，C，H和W分别为通道注意力的长，高和宽，i是指数除以空间维度，M＝H×W是通道上的神经元数量，ω_t和b_t分别表示变换的权值和偏置；in,

is the linear transformation of t and _xi , where t and _xi represent input features/>

The target neuron and other neurons in a single channel of , C, H and W are the length, height and width of the channel attention respectively, i is the index divided by the spatial dimension, M=H×W is the number of neurons on the channel , ω _t and b _t represent the weight and bias of the transformation, respectively;

Because each channel has M energy functions, the analytical solution can be quickly obtained by using the following formula, which is as follows:

μ_t表示对应通道中的除t以外的所有神经元的均值，/>

表示对应通道中的除t以外的所有神经元的方差，/>

表示能每个神经元的重要性。in,

μ _t represents the mean value of all neurons in the corresponding channel except t, />

Indicates the variance of all neurons in the corresponding channel except t, />

Indicates the importance of each neuron.

From the above two formulas, it can be seen that the analytical solutions are all obtained on a single channel, so it is reasonable to speculate that other neurons of the same channel also satisfy the same distribution. Based on this assumption, the mean and variance are calculated on all neurons, and all neurons on the same channel can reuse this mean and variance. Therefore, the repeated calculation of μ and σ for each position can be avoided, and the calculation cost can be reduced. The final minimum energy can be calculated by the following formula:

μ_t表示对应通道中的除t以外的所有神经元的均值，/>

表示对应通道中的除t以外的所有神经元的方差，/>

表示能每个神经元的重要性。in,

μ _t represents the mean value of all neurons in the corresponding channel except t, />

Indicates the variance of all neurons in the corresponding channel except t, />

Indicates the importance of each neuron.

越低，神经元t和周围神经元的区别越大，在视觉处理中也越重要。因此，通过/>

来表示每个神经元的重要性。在特征提取中，采用缩放的操作来做特征提炼，提炼过程如下：Minimum Energy Formula Explanation, Energy

The lower it is, the greater the difference between neuron t and surrounding neurons, and the more important it is in visual processing. Therefore, by />

to represent the importance of each neuron. In feature extraction, the scaling operation is used for feature extraction, and the extraction process is as follows:

在所有通道和空间维度的汇总，sigmoid是用来约束过大的值，sigmoid是单调函数，不会影响每个神经元的相对大小。where E is

Aggregating across all channels and spatial dimensions, sigmoid is used to constrain overly large values, and sigmoid is a monotonic function that does not affect the relative size of each neuron.

After the target detection model is built, the target detection model is called to detect the detection image of litchi skin defects. First, the detection images of litchi skin defects are sorted into the same resolution, and then the litchi skin defect images are drawn into a grid, and the grid is traversed to find each Each grid unit is responsible for detecting a litchi skin defect area. The grid unit uses a bounding box to mark the defect area according to the system settings, and then scores the memory reliability of the bounding box. , the credibility calculation includes the product of the recognition score and the IOU covered by the bounding box, and the generated bounding box and classification information are stored in the matrix.

Then perform loss calculation. The loss mainly includes three parts. The first is to calculate the error of the bounding box position. When calculating the box position, select the bounding box with the highest reliability score for calculation. The second is the IOU error, which selects the highest scoring bounding box and the remaining bounding boxes for each litchi skin defect image to be detected.

After the memory reliability score of each bounding box is calculated, the bounding boxes lower than the preset value are deleted. Preferably, the preset value can be set at 0.2. Afterwards, according to the memory reliability score of the bounding box, sort the remaining bounding boxes from small to large to obtain the arrangement result, select the largest bounding box in the arrangement result, and then find the fusion with the largest bounding box that exceeds the second preset value other bounding boxes, use non-extreme suppression to delete these other bounding boxes that exceed the second preset value, and save the bounding box that is fused with the largest bounding box and exceeds the second preset value. Preferably, the second preset The setting value can be 0.5. Among the remaining bounding boxes, use the second-ranked bounding box as a comparison box, find and save other bounding boxes that are lower than the second preset value when fused with the second-ranked bounding box, and finally in the remaining bounding boxes , select a box with the largest score as the final detection box.

Optionally, the target detection model is obtained by training through a training method, and the training method is:

Obtain the image of litchi skin defect to be detected, and perform oversampling and data augmentation on each litchi defect image to obtain the litchi skin defect data set;

Annotated litchi skin defect data set is obtained after marking each of the litchi skin defect images in the litchi skin defect data set;

Call the target detection model built by adding the improved attention mechanism SimAM network to the backbone feature extraction network to detect the marked litchi skin defect image, so that after the target detection model adjusts the litchi skin defect image to the same resolution, After meshing the litchi skin defect image, traversing the grids of each marked litchi skin defect image to obtain the bounding box in each grid, identifying and scoring the litchi skin defect in the bounding box to obtain the bounding box Confidence score, according to the confidence score of the bounding box, the bounding box larger than the preset requirement is screened out as the final detection box;

Use the loss function to calculate the error between the final detection frame and the expected result. Evaluate the target detection model and update the model parameters. Select the undetected litchi skin defect image in the litchi skin defect data set for detection until the preset training times are reached to obtain the target detection. Model.

Optionally, after labeling each litchi skin defect image in the litchi skin defect data set, an annotated litchi skin defect data set is obtained, specifically:

Use a rectangular frame to mark out the litchi defect area in the image to obtain multiple bounding boxes for the litchi skin defect in the litchi skin defect image;

Set the corresponding label type according to the bounding box, where the label type includes black dot drop and crack.

In this embodiment, the target detection model is trained using litchi skin defect images. As shown in Figure 3, the litchi skin defect images are collected to create a litchi skin defect data set. Due to the small number of partial defect category images, the distribution of samples is uneven. Uniformity, which leads to poor training effect. In order to prevent the model from overfitting, it is necessary to oversample the image and augment the data to keep the litchi defect samples in balance. Oversampling is to copy samples with fewer categories. The specific measures for data augmentation are as follows:

(1) Random rotation: Randomly rotate the image according to the midpoint.

(2) Image brightness transformation: transform the brightness of the original image between preset brightness values.

As an example of this embodiment, the rotation angle may be set as (-20°, 20°), and the preset brightness value may be set as 0.5˜1.5.

Annotate each collected litchi skin defect image, and generate a TXT file to save the annotation results. Use the LabelImg labeling tool to classify and label litchi skin defects, use a rectangular frame to mark the litchi fruit area in the image, obtain the real frame, and set the corresponding label. The label types include dark spot, fall off and crack to distinguish different defects category of lychee, if a lychee skin defect image contains multiple defects, the corresponding number of bounding boxes are marked.

Call and build the litchi skin defect detection model to detect litchi defect images of different defect categories in sequence. First, organize the detected litchi skin defect images into the same resolution, and then draw a grid on the litchi skin defect image, and traverse the grid to find each The label corresponding to the defect area of litchi skin. Each grid unit is responsible for detecting a defect area of litchi skin. The grid unit uses the bounding box to mark the defect area according to the system settings, and then scores the memory reliability of the bounding box. The credibility calculation includes the product of the recognition score and the IOU covered by the bounding box, and the generated bounding box and classification information are stored in the matrix.

Then perform loss calculation. The loss mainly includes three parts. The first is to calculate the error of the bounding box position. When calculating the bounding box position, select the bounding box with the highest reliability score for calculation. The second is the IOU error, which selects the highest scoring bounding box and the remaining bounding boxes for each litchi skin defect image to be detected.

After the memory reliability score of each bounding box is calculated, the bounding boxes lower than the preset value are deleted. Preferably, the preset value can be set at 0.2. Afterwards, according to the memory reliability score of the bounding box, sort the remaining bounding boxes from small to large to obtain the arrangement result, select the largest bounding box in the arrangement result, and then find the fusion with the largest bounding box that exceeds the second preset value other bounding boxes, use non-extreme suppression to delete these other bounding boxes that exceed the second preset value, and save the bounding box that is fused with the largest bounding box and exceeds the second preset value. Preferably, the second preset The setting value can be 0.5. Among the remaining bounding boxes, use the second-ranked bounding box as a comparison box, find and save other bounding boxes that are lower than the second preset value when fused with the second-ranked bounding box, and finally in the remaining bounding boxes , select a box with the largest score as the final detection box.

Finally, the loss function is used to calculate the error between the final detection frame and the expected result. After evaluating the target detection model, the model parameters are updated, and the undetected litchi skin defect images in the litchi skin defect data set are selected for detection until the preset training times are reached to obtain the target detection model. .

After obtaining the litchi skin defect image to be detected, call the target detection model built by adding the improved attention mechanism SimAM network to the backbone feature extraction network to detect the litchi skin defect image. The target detection model first adjusts the litchi skin defect image to After the same resolution, after meshing the litchi skin defect image, traverse the grids of each litchi skin defect image to obtain multiple bounding boxes in each grid, and identify and score the litchi skin defects in each bounding box The confidence scores of each bounding box are obtained, and the bounding boxes larger than the preset requirements are selected according to the confidence scores of the bounding boxes as the final detection frame, and the defect position on the litchi skin defect image is output according to the final detection frame. By using the improved algorithm to build a detection model to identify and classify litchi defects, the skin defects of litchi were proposed, which improved the accuracy of model detection.

Embodiment two

Correspondingly, refer to FIG. 3 , which is a schematic structural diagram of a litchi skin defect recognition system provided by the present invention. As shown in the figure, the litchi skin defect recognition system includes: a sample acquisition module 401, a labeling module 402 and a detection module 403;

Wherein, the sample acquisition module 401 is used to acquire the image of the skin defect of litchi to be detected;

The marking module 402 is used to mark the litchi skin defect image to be detected and obtain the marked litchi skin defect image, wherein the marked litchi skin defect image includes a plurality of real frames;

The detection module 403 is used to call the target detection model built by adding the improved attention mechanism SimAM network in the backbone feature extraction network to detect the marked litchi skin defect image, so that the target detection model adjusts the litchi skin defect image to After the same resolution, after the litchi skin defect image is meshed, the grid of each marked litchi skin defect image is traversed to obtain multiple bounding boxes in each grid, and the litchi skin defect in each bounding box is Carry out recognition scoring to obtain the credibility score of each bounding box, screen out the bounding box larger than the preset requirement according to the credibility score of the bounding box as the final detection frame, and output the litchi skin defect detection result according to the final detection frame, wherein, The litchi skin defect detection result includes the defect location on the litchi skin defect image.

As a preferred solution, the detection module 403 includes a credibility score calculation unit 4031, a bounding box arrangement unit 4032, a first screening unit 4033 and a second screening unit 4034,

The credibility score calculation unit 4031 is used to identify and score the litchi skin defects in the bounding box to obtain the credibility score of the bounding box;

The bounding box arranging unit 4032 is used to delete the bounding boxes with scores lower than the first preset score, and then arrange the remaining bounding boxes according to the reliability scores in descending order to obtain the bounding box arrangement result;

The first screening unit 4033 is used to filter out bounding boxes that are lower than the preset error after being fused with the bounding box ranking first in the bounding box arrangement result to obtain the first bounding box detection set, wherein the boundary of the first bounding box detection set The boxes are arranged in descending order according to the credibility scores;

The second screening unit 4034 is used to filter out the bounding boxes that are lower than the preset error after the fusion of the first bounding box detection set and the second-ranked bounding box to obtain the second bounding box detection set, and detect from the second bounding box Set to filter out the top bounding box as the final detection box.

As a preferred solution, a terminal device provided by the present invention includes: a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the program, it implements the Litchi epidermis defect identification method.

As a preferred solution, the present invention provides a computer-readable storage medium, the computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable the computer to perform the identification of litchi skin defects as shown in Embodiment 1 of the present invention. method steps.

Compared with the prior art, after the present invention obtains the defect image of the litchi skin to be detected, it calls the target detection model built by adding the improved attention mechanism SimAM network in the backbone feature extraction network to detect the defect image of the litchi skin, and the target detection The model first adjusts the litchi skin defect image to the same resolution, and then performs grid division on the litchi skin defect image, and traverses the grids of each marked litchi skin defect image to obtain multiple bounding boxes in each grid. The litchi skin defects in the bounding box are identified and scored to obtain the credibility scores of each bounding box, and the bounding boxes larger than the preset requirements are selected according to the credibility scores of the bounding boxes as the final detection frame, and the litchi skin defects are output according to the final detection frame The position of the defect on the image, by using the improved algorithm to build a detection model to identify and classify litchi defects, and propose the skin defects of litchi, which can improve the accuracy of model detection.

The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the protection scope of the present invention. . In particular, for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. a litchi epidermis defect identification method, is characterized in that, comprises: Obtain an image of the skin defect of litchi to be detected; Call the target detection model built by adding the improved attention mechanism SimAM network in the backbone feature extraction network to detect the litchi skin defect image, so that the target detection model adjusts the litchi skin defect image to the same resolution After performing grid division on the litchi skin defect image, traverse the grids of each litchi skin defect image to obtain a plurality of bounding boxes in each grid, and for the litchi skin in each bounding box Defects are identified and scored to obtain the credibility scores of each bounding box, and the bounding boxes larger than the preset requirements are screened out according to the credibility scores of the bounding boxes as the final detection frame, and the litchi skin defects are output according to the final detection frame The detection result, wherein, the litchi skin defect detection result includes the defect position on the litchi skin defect image. 2. the lichee skin defect recognition method as claimed in claim 1, is characterized in that, described improved attention mechanism SimAM network obtains by redefining energy function, solving formula and feature extraction process, specifically: By adopting a binary label and adding a regular term to redefine the energy function of the attention mechanism SimAM network, wherein the energy function formula is:

in,

is the linear transformation of t and _xi , where t and _xi represent input features/>

The target neuron and other neurons in a single channel of , C, H and W are the length, height and width of the channel attention respectively, i is the index divided by the spatial dimension, M=H×W is the number of neurons on the channel , ω _t and b _t represent the weight and bias of the transformation, respectively; Utilize the solution formula to update the energy function solution method, the solution formula is:

in,

μ _t represents the mean value of all neurons in the corresponding channel except t, />

Indicates the variance of all neurons in the corresponding channel except t, />

Indicates the importance of each neuron; The defect feature is extracted by zooming operation, where the feature extraction process is:

Among them, E means

Aggregated across all channels and spatial dimensions, sigmoid represents overly constrained values. 3. the litchi epidermis defect recognition method as claimed in claim 1, is characterized in that, described litchi epidermis defect in described bounding box is identified and scored to obtain the credibility score of bounding box, according to described bounding box The confidence score of is to select the bounding box larger than the preset requirement as the final detection box, specifically: Identifying and scoring the litchi skin defects in the bounding box to obtain the reliability score of the bounding box; After deleting the bounding boxes with scores lower than the first preset score, arrange the remaining bounding boxes according to the reliability scores from large to small to obtain bounding box arrangement results; Filter out bounding boxes that are lower than the preset error after being fused with the bounding box ranking first in the bounding box arrangement result to obtain a first bounding box detection set, wherein the bounding boxes of the first bounding box detection set are based on the The above reliability scores are arranged in descending order; After filtering out the bounding boxes in the first bounding box detection set that are lower than the preset error after fusion with the second-ranked bounding box, a second bounding box detection set is obtained, and the second-ranked bounding box is filtered out from the second bounding box detection set. One of the bounding boxes is used as the final detection box. 4. the litchi epidermis defect recognition method as claimed in claim 3, is characterized in that, described litchi epidermis defect in described bounding box is identified and scored to obtain the credibility score of bounding box, specifically: Identifying the skin defect of litchi in the bounding box to obtain a recognition score; Obtain the IOU score according to the coverage area of the bounding box and the actual defect box; The reliability score is obtained by multiplying the recognition score and the IOU score. 5. litchi epidermis defect identification method as claimed in claim 1, is characterized in that, described target detection model is trained by training method and obtains, and described training method is: Obtaining an image of a litchi skin defect to be detected, and performing oversampling and data augmentation on each of the litchi defect images to obtain a litchi skin defect data set; After labeling each of the litchi skin defect images in the litchi skin defect data set, an annotated litchi skin defect data set is obtained; Call the target detection model built by adding the improved attention mechanism SimAM network in the backbone feature extraction network to detect the litchi skin defect image after the labeling, so that the target detection model adjusts the litchi skin defect image After being the same resolution, after the litchi skin defect image is meshed, after traversing the grids of the litchi skin defect images after each label to obtain the bounding boxes in each grid, the bounding box Identify and score the skin defects of litchi to obtain the credibility score of the bounding box, and screen out the bounding box greater than the preset requirement as the final detection box according to the credibility score of the bounding box; Use the loss function to calculate the error between the final detection frame and the expected result, evaluate the target detection model, update the model parameters, and select the undetected litchi skin defect image in the litchi skin defect data set for detection until reaching the preset training times to get the target detection model. 6. the litchi epidermis defect recognition method as claimed in claim 5, is characterized in that, the litchi epidermis defect data set obtained after the labeling is obtained after each described litchi epidermis defect image in the litchi epidermis defect data set, Specifically: Using a rectangular frame to mark out the litchi defect area in the image to obtain multiple bounding boxes for the litchi skin defect in the litchi skin defect image; A corresponding label type is set according to the bounding box, wherein the label type includes black dot drop and crack. 7. A litchi epidermal defect recognition system, comprising a sample acquisition module, a labeling module and a detection module; Wherein, the sample acquisition module is used to acquire the image of the skin defect of litchi to be detected; The marking module is used to mark the litchi skin defect image to be detected to obtain the marked litchi skin defect image, wherein the marked litchi skin defect image includes a plurality of real frames; The detection module is used to call the target detection model built by adding the improved attention mechanism SimAM network in the backbone feature extraction network to detect the litchi epidermis defect image after the labeling, so that the target detection model will After the skin defect image is adjusted to the same resolution, after the litchi skin defect image is meshed, traverse the grids of each marked litchi skin defect image to obtain multiple bounding boxes in each grid, and for each Identifying and scoring the litchi skin defects in the bounding boxes to obtain the credibility scores of each bounding box, and selecting the bounding boxes larger than the preset requirements according to the credibility scores of the bounding boxes as the final detection box, Output the detection result of litchi skin defect according to the final detection frame, wherein the litchi skin defect detection result includes the defect position on the litchi skin defect image. 8. litchi epidermis defect recognition system as claimed in claim 7, is characterized in that, described detection module comprises reliability score calculation unit, bounding box arrangement unit, the first screening unit and the second screening unit, The credibility score calculation unit is used to identify and score the litchi skin defects in the bounding box to obtain the credibility score of the bounding box; The bounding box arranging unit is used to delete the bounding boxes with scores lower than the first preset score, and then arrange the remaining bounding boxes according to the reliability scores in descending order to obtain the bounding box arrangement result; The first screening unit is used to filter out bounding boxes that are lower than a preset error after being fused with the bounding box ranked first in the bounding box arrangement result to obtain a first bounding box detection set, wherein the first bounding box The bounding boxes of the box detection set are arranged in descending order according to the credibility scores; The second screening unit is used to filter out the bounding boxes in the first bounding box detection set and the second-ranked bounding box after the fusion is lower than the preset error to obtain the second bounding box detection set, and from the second bounding box detection set The bounding box detection set screens out the first-ranked bounding box as the final detection box. 9. A terminal device, characterized in that, comprising: memory for storing computer programs; A processor configured to implement the method for identifying skin defects of litchi according to any one of claims 1 to 6 when executing the computer program. 10. A storage medium, characterized in that a computer program is stored on the storage medium, and when the computer program is executed by a processor, it realizes the steps of the litchi skin defect identification method according to any one of claims 1 to 6 .

Images