CN112101343A - License plate character segmentation and recognition method - Google Patents

Abstract

The invention discloses a license plate character segmentation and recognition method, comprising the following steps: S1. Preprocessing the license plate image; S2. Removing spacers, removing upper and lower borders, and locating the coordinates of the right end of the second character; S3. Segmenting characters , and normalize the characters; S4. Divide the collected data set into training set and test set; S5. Construct a convolutional neural network suitable for license plate character image recognition; S6. Select training parameters, and use the training set to pair The set network is trained; S7. Use the test set to test the trained network to obtain the accuracy rate of the license plate recognition network. The invention can avoid the problem of character segmentation failure caused by character breakage, solve the problem of overfitting caused by less training samples, and further improve the convergence speed and the generalization ability of the model.

Description

A kind of license plate character segmentation and recognition method

technical field

The invention relates to the technical field of license plate recognition, in particular to a method for segmenting and recognizing license plate characters.

Background technique

The license plate number is an important sign of the vehicle, so the license plate recognition technology is of great significance to the traffic control. The license plate recognition system is an important link in the construction of an intelligent transportation system.

The traditional character segmentation methods include connected domain method and vertical projection method, which have better effect on character segmentation. Divide the word "Chuan" into 3 "1"s.

The traditional template matching license plate recognition method relies on the matching degree of the template characters and the characters to be tested, and has strict requirements on the clarity of the original image, and the recognition effect of this method is not good. In machine learning, the recognition algorithm based on support vector machine is a relatively classic algorithm. This method has strong robustness, but it overemphasizes the selection of character features.

At present, most of the license plate recognition algorithms are based on convolutional neural networks, which are continuously improved and optimized. The existing technology reduces the training error by increasing the complexity of the model, such as increasing the number of feature maps. However, when a complex model is still used when there are few training samples, it is prone to overfitting problems.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the deficiencies of the prior art, to provide a method that can avoid the problem of character segmentation failure caused by character breakage, solve the problem of overfitting caused by less training samples, and further improve the convergence speed and the generalization ability of the model. The segmentation and recognition method of license plate characters.

For achieving the above object, the technical scheme provided by the present invention is:

A license plate character segmentation and recognition method, comprising the following steps:

S1. Preprocess the license plate image;

S2. Remove the spacer, remove the upper and lower borders, and locate the coordinates of the right end of the second character;

S3. Split characters and normalize characters;

S4. Divide the collected data set into a training set and a test set;

S5. Construct a convolutional neural network suitable for license plate character image recognition;

S6. Select training parameters, and use the training set to train the set network;

S7. Use the test set to test the trained network to obtain the accuracy rate of the license plate recognition network.

Further, the step S1 specifically includes:

S11. Perform a grayscale operation on the color license plate image;

S12. Use the Otsu method to binarize the grayscale image.

Further, the step S2 specifically includes:

S21. Use the open operation to remove the area with an area less than 20;

S22. Count the number of transitions of pixels in each row from 1 to 0 or from 0 to 1, and obtain a statistical graph of transition times; search upward from 1/3 of the total number of rows, and obtain the transition times less than or equal to 13 Line coordinate x1; search down from 2/3 of the total number of lines, get the line coordinate x2 with the number of jumps less than or equal to 13, remove the area outside the interval [x1, x2]; get the character height x=(x2-x1 ), the width of a single character is a=x*(45/90), the character spacing b=x*(12/90), the second and third character spacing c=x*(34/90);

S23. Using the feature that the distance between the second character and the third character is the largest, use the vertical projection method to locate the coordinate z1 of the right end of the second character.

Further, the step S3 specifically includes:

S31. Use x, z1, a, b, and c to calculate the offsets of the nine division points y1 to y9 relative to z1, and then calculate the coordinates of each division point through the offsets;

S32. Divide 7 characters with 9 dividing points;

S33. Normalize the segmented characters, and transform the image size into 30*24.

Further, the convolutional neural network in the step S5 includes a convolution layer, a Batch normalization layer, a Relu activation layer, a maximum pooling layer, a Dropout layer, a fully connected layer, and a Softmax layer. Among them, the receptive field of the convolutional layer is the adjacent neuron area of the previous layer, which is used to extract the features of the image; the Batch normalization layer is used to enhance the generalization ability of the network and improve the accuracy of the network model; the Relu activation layer enables the network model to Close to any function; the maximum pooling layer is used to reduce the dimension of the data; the Dropout layer is used to suppress overfitting; the fully connected layer is used for linear transformation; the Softmax layer can obtain a multi-dimensional column vector for classification.

Further, the specific structure of the convolutional neural network is: an input layer, a first convolutional layer, a first Batchnormalization layer, a first Relu activation layer, a first maximum pooling layer, a second convolutional layer, and a second Batchnormalization layer layer, the second Relu activation layer, the second max pooling layer, the third convolution layer, the third Batchnormalization layer, the third Relu activation layer, the third max pooling layer, the first Dropout layer, the first fully connected layer, The second fully connected layer and the first Softmax layer.

Compared with the prior art, the principle and advantages of this scheme are as follows:

1) The characters are divided according to the spatial position of 7 characters, which avoids the problem of character segmentation failure caused by character breakage.

2) Convolutional neural network is used to avoid complex feature extraction and enhance robustness.

3) The convolutional neural network model is simplified, and by adding the Dropout layer and the Batch normalization layer, and through multiple experiments, the random inactivation probability of the Dropout layer is continuously adjusted to obtain the optimal value, which solves the overfitting and convergence speed. and other problems, improving the accuracy and convergence speed.

Description of drawings

In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the services required in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only For some embodiments of the present invention, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

Fig. 1 is the flow chart of a kind of license plate character segmentation and recognition method of the present invention;

2 is a schematic diagram of removing upper and lower borders according to an embodiment of the present invention;

3 is a schematic diagram of character segmentation according to an embodiment of the present invention;

FIG. 4 is a structural diagram of a convolutional neural network used in an embodiment of the present invention.

Detailed ways

Below in conjunction with specific embodiment, the present invention will be further described:

As shown in Figure 1, a method for character segmentation and recognition for license plate cloud DG7327 described in this embodiment includes the following steps:

S1. Preprocess the license plate image, specifically:

S11. Perform a grayscale operation on the color license plate image;

S12. Use the Otsu method to binarize the grayscale image.

S2. Remove the spacer, remove the upper and lower borders, and locate the coordinates of the right end of the second character; specifically:

S21. Use the open operation to remove the area with an area less than 20;

S22. As shown in Figure 2, count the number of transitions of each row of pixels from 1 to 0 or from 0 to 1, and obtain a statistical graph of the number of transitions; search upward from 1/3 of the total number of rows to obtain transitions Line coordinate x1 with times less than or equal to 13; search downward from 2/3 of the total number of lines, get line coordinate x2 with jump times less than or equal to 13, remove the area outside the interval [x1, x2]; get the character height again x=(x2-x1), single character width a=x*(45/90), character spacing b=x*(12/90), second and third character spacing c=x*(34/ 90);

S23. Using the feature that the distance between the second character and the third character is the largest, use the vertical projection method to locate the coordinate z1 of the right end of the second character.

S3. Split characters and normalize characters; specifically:

3, use x, z1, a, b, c to calculate the offset of 9 dividing points (y1 to y9) relative to z1, and then calculate the coordinates of each dividing point by the offset;

Then divide 7 characters with 9 dividing points;

Normalize the segmented characters and transform the image size to 30*24.

S4. Divide the collected data set into a training set and a test set. Specifically, the data set has a total of 9363 license plate character images, including 8430 in the training set and 933 in the test set; the training set accounts for 90% and the test set accounts for 90%. 10%.

S5. Construct a convolutional neural network suitable for license plate character image recognition;

Specifically, the convolutional neural network includes 3 layers of convolution layers, 3 layers of Batch normalization layers, 3 layers of Relu activation layers, 3 layers of maximum pooling layers, 1 layer of Dropout layers, 2 layers of fully connected layers, and 1 layer of Softmax layer. The sequence is the input layer, the first convolution layer, the first Batch normalization layer, the first Relu activation layer, the first maximum pooling layer, the second convolution layer, the second Batch normalization layer, the second Relu activation layer, the first The second maximum pooling layer, the third convolutional layer, the third Batch normalization layer, the third Relu activation layer, the third maximum pooling layer, the first Dropout layer, the first fully connected layer, the second fully connected layer, the first Softmax layer.

As shown in Figure 4, the input layer data is a grayscale image of license plate characters with a size of 30*24; the convolution kernel size of the first convolutional layer is 3*3, and the number of feature maps is 6; the first maximum pooling The sampling window size of the layer is 2*2; the convolution kernel size of the second convolutional layer is 3*3, and the number of feature maps is 16; the sampling window size of the second max pooling layer is 2*2; the third convolutional layer has a size of 2*2. The convolution kernel size of the layer is 3*3, and the number of feature maps is 128; the sampling window size of the third maximum pooling layer is 2*2; the random deactivation rate of the first dropout layer is 0.5; the first fully connected layer The number of neurons in the second fully connected layer is 128; the number of neurons in the second fully connected layer is 65.

S6. Select training parameters, and use the training set to train the set network; the training parameters specifically include: a model optimization algorithm, a learning rate, and the number of iterations. The model optimization algorithm is a stochastic gradient descent method with momentum; the learning rate is 0.01; and the number of iterations is 40.

S7. Use the test set to test the trained network to obtain the accuracy rate of the license plate recognition network.

Specifically, a total of 9363 samples are obtained in this embodiment, and 8430 training sets and 933 testing sets are obtained through step S4; 8430 training set samples are input into the convolutional neural network constructed in step 5 for training, and the trained The network identified 933 test set samples to obtain the performance of the network.

Table 1 is the test result of the present invention:

number letter Chinese character all error book 0 6 6 total 430 503 933 Accuracy 100% 98.81% 99.36%

Table I

From the test results in Table 1, it can be seen that this embodiment has strong robustness to license plate character recognition, and the addition of Dropout layer suppresses the over-fitting phenomenon, greatly improving the accuracy of license plate recognition, and the final recognition rate reaches 99.36% , the solution proposed in this embodiment has strong practical value.

The above-mentioned embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of implementation of the present invention. Therefore, any changes made according to the shape and principle of the present invention should be included within the protection scope of the present invention.

Claims (6)

1. a license plate character segmentation and recognition method, is characterized in that, comprises the following steps: S1. Preprocess the license plate image; S2. Remove the spacer, remove the upper and lower borders, and locate the coordinates of the right end of the second character; S3. Split characters and normalize characters; S4. Divide the collected data set into a training set and a test set; S5. Construct a convolutional neural network suitable for license plate character image recognition; S6. Select training parameters, and use the training set to train the set network; S7. Use the test set to test the trained network to obtain the accuracy rate of the license plate recognition network. 2. a kind of license plate character segmentation and recognition method according to claim 1, is characterized in that, described step S1 specifically comprises: S11. Perform a grayscale operation on the color license plate image; S12. Use the Otsu method to binarize the grayscale image. 3. a kind of license plate character segmentation and recognition method according to claim 1, is characterized in that, described step S2 specifically comprises: S21. Use the open operation to remove the area with an area less than 20; S22. Count the number of transitions of pixels in each row from 1 to 0 or from 0 to 1, and obtain a statistical graph of transition times; search upward from 1/3 of the total number of rows, and obtain the transition times less than or equal to 13 Line coordinate x1; search down from 2/3 of the total number of lines, get the line coordinate x2 with the number of jumps less than or equal to 13, remove the area outside the interval [x1, x2]; get the character height x=(x2-x1 ), the width of a single character is a=x*(45/90), the character spacing b=x*(12/90), the second and third character spacing c=x*(34/90); S23. Using the feature that the distance between the second character and the third character is the largest, use the vertical projection method to locate the coordinate z1 of the right end of the second character. 4. a kind of license plate character segmentation and recognition method according to claim 3, is characterized in that, described step S3 specifically comprises: S31. Use x, z1, a, b, and c to calculate the offsets of the nine division points y1 to y9 relative to z1, and then calculate the coordinates of each division point through the offsets; S32. Divide 7 characters with 9 dividing points; S33. Normalize the segmented characters, and transform the image size into 30*24. 5. a kind of license plate character segmentation and recognition method according to claim 1, is characterized in that, the convolutional neural network in described step S5 comprises convolution layer, Batch normalization layer, Relu activation layer, maximum pooling layer, Dropout layer, fully connected layer, Softmax layer; among them, the receptive field of the convolution layer is the adjacent neuron area of the previous layer, which is used to extract the features of the image; the batch normalization layer is used to enhance the generalization ability of the network and improve the network model. The Relu activation layer enables the network model to approach any function; the maximum pooling layer is used to reduce the dimension of the data; the Dropout layer is used to suppress overfitting; the fully connected layer is used for linear transformation; the Softmax layer can obtain a multi-dimensional Column vector, used for classification. 6. A kind of license plate character segmentation and recognition method according to claim 5, is characterized in that, the concrete structure of described convolutional neural network is: input layer, the first convolution layer, the first Batch normalization layer, the first Relu activation layer, first max pooling layer, second convolution layer, second Batch normalization layer, second Relu activation layer, second max pooling layer, third convolution layer, third Batch normalization layer, third Relu activation layer, third max pooling layer, first Dropout layer, first fully connected layer, second fully connected layer, first Softmax layer.

Images