CN115222652A - Method for identifying, counting and centering end faces of bundled steel bars and memory thereof - Google Patents

Abstract

The invention relates to the technical field of machine vision, in particular to a method for identifying, counting and centering end faces of bundled reinforcing steel bars and a memory thereof, wherein the method comprises the steps of S1, shooting images of the end faces of the reinforcing steel bars, and obtaining images to be identified after processing; s2, performing data enhancement operation on the image to be recognized by adopting a first preset algorithm; s3, forming final detection frames in the image to be recognized by adopting a second preset algorithm with a lightweight convolutional neural network, and calculating the number of the final detection frames; and S4, generating a counting result. The invention solves the problems that the counting result is inaccurate and the practical requirement cannot be met because the conventional reinforcing steel bar end face identification technology is generally carried out by adopting a common machine vision algorithm.

Description

A method for identifying, counting and centering the end face of bundled steel bars and its memory

technical field

The invention relates to the technical field of machine vision, in particular to a method for identifying, counting and centering the end faces of bundled steel bars and a memory thereof.

Background technique

Machine vision is the use of robots to replace human eyes for measurement and judgment. The machine vision system converts the captured target into an image signal through its visual products, and transmits it to a dedicated image processing system to obtain the morphological information of the captured target, and converts it into a digital signal according to pixel distribution, brightness, color and other information; image The system performs various operations on these signals to extract the characteristics of the target, and then controls the on-site equipment actions according to the results of the discrimination.

The YOLOv3 algorithm can be used to solve "how to detect two objects of the same type that are close to each other, or objects of different types that are close to each other", which has good robustness to objects that are close to each other or small objects.

The existing reinforcement end face recognition technology is usually carried out by ordinary machine vision algorithms, but because the algorithm cannot accurately obtain the results, the application of machine vision in the field of reinforcement end face counting cannot obtain accurate results and cannot meet the practical needs. A method of identifying, counting and centering the end face of bundled steel bars and its storage came into being.

SUMMARY OF THE INVENTION

The content of the present invention is to provide a method for identifying, counting and centering the end faces of bundled steel bars and a memory thereof, which mainly solves the problem that the existing technology for identifying the end faces of steel bars is usually carried out by using ordinary machine vision algorithms, resulting in inaccurate counting results, which cannot meet the requirements of A question of practicality.

The invention provides a method for identifying, counting and centering the end faces of bundled steel bars, comprising the following steps:

S1, take an image of the end face of the steel bar, and obtain an image to be recognized after processing;

S2, using a first preset algorithm to perform a data enhancement operation on the to-be-recognized image;

S3, using a second preset algorithm with a lightweight convolutional neural network to form a final detection frame in the to-be-recognized image, and calculate the number of the final detection frame;

S4, generating a counting result.

Preferably, the step S3 specifically includes:

S31, pre-forming a second preset algorithm including a lightweight convolutional neural network;

S32, using the second preset algorithm to form a final detection frame in the to-be-recognized image, and calculate the number of the final detection frame.

Specifically, the second preset algorithm in step S31 is to replace the Darknet53 backbone feature extraction network in the original YoloV3 network with the Shfflenetv2 backbone feature extraction network by improving its backbone feature extraction network.

Preferably, the step S31 specifically includes:

S311, performing a clustering operation on the training image to form an anchor frame;

S312, dividing the training image and forming a plurality of small blocks;

S313, generating a plurality of rectangular frames in each small block, and the length and width of the rectangular frames are determined by the anchor frame;

S314, after fine-tuning a plurality of rectangular frames under the same small sub-block, a primary detection frame is formed;

S315, determine whether any of the small blocks contains target detection errors, and if so, calculate the IOU values between the multiple primary detection frames in the current small block and the real frame of the training image, if all the primary detection frames When the frame is greater than the set threshold, the primary detection frame with the largest IOU value is selected as a positive sample;

S316, after saving the frame shape of the positive sample, a second preset algorithm with a lightweight convolutional neural network is generated.

Preferably, the step S314 specifically includes:

S314a, obtaining multiple parameter values of the anchor frame;

S314b, forming a primary detection frame after adjusting the rectangular frame according to a plurality of the obtained parameter values;

Wherein, the plurality of parameter values in the step S314a include the coordinate values of the four corners, which are respectively denoted as t _x , _{ty , t w , th} , and also include the offset of the anchor frame relative to the training image quantity(c _x , c _y );

In the step S314b, the adjustment of the rectangular frame includes the following formula:

b _x =σ(t _x )+c _x ;

b _y =σ( _ty )+ _cy ;

所述预设坐标值为t^*。Note that the current width of the rectangular frame is p _w , and the height of the rectangular frame is p _h ; note that the real value of the coordinates of the rectangular frame is

The preset coordinate value is t ^* .

Preferably, after the step S315 and before the step S316, there are steps:

SX, the loss is calculated using the following formula,

Wbox=2.0-tw*th

Loss=lossbox+lossconf+lossclass

代表若在坐标[i,j]处的box有目标则其值为1否则为0，主要计算3个损失函数，分别是与真实框中心坐标，宽高之间的损失，以及预测框中是否包含检测物的损失，以及预测类别损失最终将3种损失进行相加作为一个层级的损失函数值，最终计算损失时采用三个层级损失函数的平均作为最终损失值。Among them, where S ² represents the training image size is S times S, B represents the box,

Represents that if the box at coordinates [i, j] has a target, its value is 1, otherwise it is 0, and three loss functions are mainly calculated, which are the loss between the center coordinates of the real box, the width and height, and whether the prediction box is Including the loss of the detection object and the loss of the prediction category, the three losses are finally added as the loss function value of one level, and the average of the three levels of loss functions is used as the final loss value when calculating the loss.

Preferably, in the step S1, the image to be recognized is obtained after the processing, including randomly flipping the image, randomly scaling, randomly cropping, and randomly changing the brightness and darkness.

Preferably, in the step S2, the first preset algorithm is used to perform a data enhancement operation on the to-be-recognized image, including performing Fmix enhancement and mixing on data sets such as dirt or texture.

The present invention also provides a computer-readable memory, the computer-readable memory comprising a stored computer program, wherein when the computer program runs, a device in which the computer-readable memory is located is controlled to execute the method described in the preceding claims .

As can be seen from the above, the following beneficial effects can be obtained by applying the technical solution provided by the present invention:

In the method proposed by the present invention, through model training, the second preset algorithm composed of the lightweight convolutional neural network can be used to form the detection frame in a targeted manner, and the technical result is ensured to be accurate.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Existing steel bar end face recognition technology is usually carried out by common machine vision algorithm, resulting in inaccurate counting results, which cannot meet the problem of practicality.

It should be emphasized that the counting and center positioning method proposed in this embodiment is not only used for the end face of the steel bar, but can be used in the counting field under different backgrounds and recognition objects.

In order to solve the above problems, the present embodiment proposes a method for identifying, counting and centering the end faces of bundled steel bars, which mainly includes the following steps:

S1, take an image of the end face of the steel bar, and obtain an image to be recognized after processing;

S2, using a first preset algorithm to perform a data enhancement operation on the to-be-recognized image;

S3, using a second preset algorithm with a lightweight convolutional neural network to form a final detection frame in the to-be-recognized image, and calculate the number of the final detection frame;

S4, generating a counting result.

Preferably, but not limitedly, in this embodiment, the processing of the shot image of the steel bar end face in step S1 includes random image flipping, random scaling, random cropping, random changing of brightness and darkness, etc. Sharp edges help with accurate counting.

Preferably, but not limitedly, in this embodiment, the first preset algorithm in step S2 performs a data enhancement operation on the image to be recognized, including performing Fmix enhancement and mixing on data sets such as dirt or texture. The mixing function is the Fmix function, and the implementation process of the Fmix function is as follows: 1) randomly extract a picture from the dirty data set; 2) perform threshold processing on the low-frequency grayscale image sampled by Fourier space to obtain a mask; The randomly acquired image in one step is mask-blended with the mask obtained in the second step. Among them, the Fourier transform and mask mixing function include:

That is, first sample a random complex tensor whose real and imaginary parts are independent and Gaussian distributed; then each component is scaled according to its frequency by the parameter λ, so that the higher value of λ is more important for the high frequency information. Increase the attenuation; perform inverse Fourier transform on the complex tensor, and take the real part to obtain a grayscale image; finally, through the set threshold (the top ratio of the image), the image is turned into a binary mask, and the setting higher than the threshold is 1, below the threshold is 0.

More specifically, the step S3 specifically includes:

S31, pre-forming a second preset algorithm including a lightweight convolutional neural network;

S32, a second preset algorithm is used to form a final detection frame in the image to be recognized, and the number of final detection frames is calculated.

Specifically, the second preset algorithm in step S31 is to replace the Darknet53 backbone feature extraction network in the original YoloV3 network with the Shfflenetv2 backbone feature extraction network by improving its backbone feature extraction network.

Preferably, the network in step S31 is an improved network based on the original yolov3, which mainly improves its backbone feature extraction network. By introducing channel split, the number of input and output channels of the network is the same, thereby reducing the memory access cost of the model; introducing point-by-point The grouped convolution is the convolution with grouping and the convolution kernel is 1×1, which reduces the amount of calculation. The function of the point-by-point convolution is to reduce the amount of parameters, and the grouped point-by-point convolution is based on the point-by-point convolution. Reduce the amount of calculation; in order to solve the problem that too many groups will reduce the network parallelism, channel shuffle is introduced, and its function is to enrich the information obtained by each group, thereby extracting more features;

Preferably, step S31 specifically includes:

S311, performing a clustering operation on the training image to form an anchor frame;

S312, dividing the training image and forming a plurality of small blocks;

S313, generate a plurality of rectangular frames in each small block, and the length and width of the rectangular frame are determined by the anchor frame;

S314, after fine-tuning a plurality of rectangular frames under the same small block, a primary detection frame is formed;

S315, determine whether any small block contains the target detection object, if so, calculate the IOU value between the multiple primary detection frames in the current small block and the real frame of the training image, if all the primary detection frames are greater than the set threshold value When , select the primary detection frame with the largest IOU value as the positive sample;

S316, after saving the frame shape of the positive sample, a second preset algorithm with a lightweight convolutional neural network is generated.

Preferably, but not limitedly, in this embodiment, before S3, a step is further provided for converting the preset data into a format that can be processed by the convolutional network. In this embodiment, the preset original data set is marked in VOC format. When the data is enhanced in step S1, when the size of the image changes, the corresponding marking coordinates also change. When the data is put into the convolution When using a neural network, image data and label data need to be processed separately. Image data needs to be normalized as a whole and converted into NCHW, that is, the format of number, number of channels, height, and width. The label data needs to be clustered by kmeans first, and 9 cluster centers are generated as the length and width of the anchor frame, so as to obtain the anchor frame used in training.

Preferably, but not limitedly, in this embodiment, the small blocks in step S312 can be set to an N×N format, and the number of rectangular frames selected in step S313 is three, and the shapes of the three rectangular frames are different, corresponding to , the anchor box size is divided into three groups from small to large according to the downsampling ratio of the network, and each group has three rectangular boxes from small to large.

In this embodiment, when the primary detection amount in step S315 is smaller than the set threshold, it is considered that the overlap of the part with the real frame is not high, and the small square does not contain an object, and it is set as a negative sample.

More specifically, step S314 specifically includes:

S314a, obtaining multiple parameter values of the anchor frame;

S314b, after adjusting the rectangular frame according to the obtained multiple parameter values, a primary detection frame is formed;

Among them, the multiple parameter values in step S314a include the coordinate values of the four corners, which are respectively recorded as t _x , _{ty , t w , th} , and also include the offset of the anchor frame relative to the training image (c _x , c _y );

In step S314b, the adjustment of the rectangular frame includes the following formula:

b _x =σ(t _x )+c _x ;

b _y =σ( _ty )+ _cy ;

b= ^pet ;

预设坐标值为t^*。Note that the width of the current rectangular frame is p _w , and the height of the rectangular frame is p _h ; note that the real value of the coordinates of the rectangular frame is

The preset coordinate value is t ^* .

More specifically, after step S315 and before step S316, there are steps:

SX, the loss is calculated using the following formula,

Wbox=2.0-tw*th

Loss=lossbox+lossconf+lossclass

代表若在坐标[i,j]处的box有目标则其值为1否则为0，主要计算3个损失函数，分别是与真实框中心坐标，宽高之间的损失，以及预测框中是否包含检测物的损失，以及预测类别损失最终将3种损失进行相加作为一个层级的损失函数值，最终计算损失时采用三个层级损失函数的平均作为最终损失值。Among them, where S ² represents the training image size is S times S, B represents the box,

Represents that if the box at coordinates [i, j] has a target, its value is 1, otherwise it is 0, and three loss functions are mainly calculated, which are the loss between the center coordinates of the real box, the width and height, and whether the prediction box is Including the loss of the detection object and the loss of the prediction category, the three losses are finally added as the loss function value of one level, and the average of the three levels of loss functions is used as the final loss value when calculating the loss.

In this embodiment, the coordinate values obtained according to a plurality of parameter values can be used to calibrate the coordinates of the steel bars in the current detection frame, and the preset coordinate values are determined coordinate values obtained by manual inspection or other methods of inspection on the image of the end face of the current steel bar. , the difference between the two can be used to determine whether the deviation of the current primary detection frame is within the allowable range, if so, the current primary detection frame can be selected as the detection standard, otherwise the primary detection frame is discarded.

It should be emphasized that the memory equipped with the aforementioned method also belongs to the protection scope of this embodiment.

To sum up, this embodiment proposes a method for identifying, counting and centering the end faces of bundles of steel bars, which mainly obtains the most suitable detection frame by iteratively replacing a plurality of detection frames of different scales. The number of recognized bars in the current image is judged to obtain the number of steel bar end faces in the current image, and its count is more accurate.

The above-mentioned embodiments do not constitute a limitation on the protection scope of the technical solution. Any modifications, equivalent replacements and improvements made within the spirit and principles of the above-mentioned embodiments shall be included within the protection scope of this technical solution.

Claims (8)

1. A method for identifying, counting and centering end faces of bundles of reinforcing steel bars is characterized by comprising the following steps:

s1, shooting an image of the end face of a steel bar, and processing to obtain an image to be identified;

s2, performing data enhancement operation on the image to be recognized by adopting a first preset algorithm;

s3, forming final detection frames in the image to be recognized by adopting a second preset algorithm with a lightweight convolutional neural network, and calculating the number of the final detection frames;

and S4, generating a counting result.

2. The method for identifying, counting and centering end faces of bundles of steel bars according to claim 1, wherein the step S3 specifically comprises:

s31, a second preset algorithm with a lightweight convolutional neural network is formed in advance;

s32, forming a final detection frame in the image to be recognized by adopting the second preset algorithm, and calculating the number of the final detection frames;

the second preset algorithm in step S31 is specifically to replace the Darknet53 backbone feature extraction network in the YoloV3 original network with the Shfflenetv2 backbone feature extraction network by improving the backbone feature extraction network.

3. The method as claimed in claim 2, wherein the step S31 specifically includes:

s311, clustering the training images to form an anchor frame;

s312, dividing the training image and forming a plurality of small blocks;

s313, generating a plurality of rectangular frames in each small block, wherein the length and the width of each rectangular frame are determined by the anchor frame;

s314, fine-tuning a plurality of rectangular frames under the same small block to form a primary detection frame;

s315, judging whether any small block contains a target detection object, if so, calculating IOU values between a plurality of primary detection frames in the small block and a real frame of a training image, and if all the primary detection frames are larger than a set threshold value, selecting the primary detection frame with the largest IOU value as a positive sample;

and S316, generating a second preset algorithm with the light-weight convolutional neural network after the frame shape of the positive sample is saved.

4. The method as claimed in claim 3, wherein the step S314 specifically includes:

s314a, acquiring a plurality of parameter values of the anchor frame;

s314b, adjusting the rectangular frame according to the acquired multiple parameter values to form a primary detection frame;

wherein, theThe parameter values in step S314a include coordinate values of four corners, which are respectively denoted as t _x ，t _y ，t _w ，t _h And an offset (c) of the anchor frame with respect to the training image _x ，c _y )；

In step S314b, the adjustment of the rectangular frame includes the following formula,

b _x ＝σ(t _x )+c _x ；

b _y ＝σ(t _y )+c _y ；

note that the width of the rectangular frame is p _w The height of the rectangular frame is p _h (ii) a Recording the real value of the coordinates of the rectangular frame as

The preset coordinate value is t ^* 。

5. The method for identifying, counting and centering end faces of reinforcing steel bundles according to claim 4, wherein after the step S315 and before the step S316, the steps of:

SX, which performs loss calculation by adopting the following formula,

Wbox＝2.0-tw*th

Loss＝lossbox+lossconf+lossclass

wherein S is ² Representing the training image size as S times S, B representing box,

represents if at coordinate [ i, j]And if the box has a target, the value of the box is 1, otherwise the box is 0, 3 loss functions are mainly calculated, namely the loss between the box and the center coordinate of the real box and the width and the height, whether the loss of the detection object is contained in the prediction box or not, and the 3 losses are finally added to be used as a loss function value of one level in the prediction type loss, and the average of the three-level loss functions is used as a final loss value in the final loss calculation.

6. The method for identifying, counting and centering the end faces of the steel bars in the bundle according to any one of claims 1 to 5, wherein the method comprises the following steps:

in the step S1, the image to be recognized is obtained after the processing, which includes randomly turning the image, randomly scaling, randomly cutting, and randomly changing brightness and darkness.

7. The method for identifying, calculating and centering end faces of bundles of steel bars according to any one of claims 1 to 5, wherein the method comprises the following steps:

in the step S2, the data enhancement operation is performed on the image to be recognized by using a first preset algorithm, which includes performing Fmix enhancement mixing on a dirty or texture data set.

8. A computer-readable memory, characterized in that: the computer-readable memory comprises a stored computer program, wherein the computer program when executed controls an apparatus in which the computer-readable memory is located to perform the method of any of claims 1-7.