CN111582272A - Double-row license plate recognition method, device and equipment and computer readable storage medium - Google Patents

Abstract

The invention provides a double-vehicle license plate recognition method, which relates to the technical field of image recognition. The method includes: acquiring a license plate image of a double-lane license plate to be recognized; using a preset feature extraction model to perform feature extraction on the license plate image to obtain a license plate feature matrix performing feature reorganization on the license plate feature matrix to obtain the target license plate feature; inputting the target license plate feature into a preset depth bidirectional recurrent neural network to obtain the license plate recognition result of the to-be-recognized dual-carriage license plate. The present invention also provides a dual-vehicle license plate recognition device, equipment and computer-readable storage medium. The invention can improve the recognition efficiency and accuracy of the double-vehicle license plate recognition result.

Description

Method, Apparatus, Device and Computer-readable Storage Medium for Recognition of Double License Plate

technical field

The present invention relates to the technical field of image recognition, and in particular, to a method, device, device, and computer-readable storage medium for recognizing dual-vehicle license plates.

Background technique

License plate recognition is one of the important components of modern intelligent transportation systems, and is widely used, such as unattended parking lots, restricted area security control, traffic law enforcement, congestion pricing, automatic toll collection, etc. According to my country's traffic regulations, license plates include single-lane license plates and multi-lane license plates. Among them, single-lane license plates are used for personal cars, police cars, military vehicles, embassy cars, trucks, buses, etc., while double-lane license plates are used for buses and passenger cars. , trucks, armed police cars, motorcycles, etc.

Currently, most license plate recognition methods only focus on the task of single-lane license plate recognition, and only a few methods consider double-lane license plates. These methods usually need to divide the double-column license plate into two parts, which are used as input respectively. Specifically, they require that each character of the license plate should be divided and correctly recognized. Therefore, the segmentation and recognition accuracy of each character will affect the accuracy of the final recognition result. sex. Therefore, in the existing dual-vehicle license plate recognition method, the task load of the recognition process is relatively large, the process is relatively complicated, and the accuracy of the recognition result is low.

SUMMARY OF THE INVENTION

The main purpose of the present invention is to provide a method, device, device and computer-readable storage medium for recognizing dual-vehicle license plates, aiming at improving the recognition efficiency and accuracy of the results of recognizing dual-vehicle license plates.

In order to achieve the above object, the present invention provides a method for recognizing a dual-vehicle license plate, which includes:

Obtain the license plate image of the double-carriage license plate to be recognized;

Using a preset feature extraction model to perform feature extraction on the license plate image to obtain a license plate feature matrix;

Perform feature reorganization on the license plate feature matrix to obtain the target license plate feature;

The feature of the target license plate is input into a preset depth bidirectional recurrent neural network to obtain the license plate recognition result of the double-carriage license plate to be recognized.

Optionally, the preset feature extraction model includes an input layer, a convolution layer, a pooling layer and a normalization layer; wherein,

The input layer is used to receive the license plate image of the to-be-recognized dual-vehicle license plate;

The convolution layer is used to extract the image feature matrix of the license plate image according to the convolution kernel;

The pooling layer is used for pooling the output of the convolutional layer;

The normalization layer is used for normalizing the output of the convolution layer.

Optionally, the step of performing feature reorganization on the license plate feature matrix to obtain the target license plate feature includes:

Splitting the license plate feature matrix according to the height of the license plate feature matrix to obtain a first feature matrix and a second feature matrix;

The first feature matrix and the second feature matrix are combined and spliced to obtain the target license plate feature.

Optionally, after the step of inputting the feature of the target license plate into a preset depth bidirectional recurrent neural network to obtain the license plate recognition result of the to-be-recognized dual-carriage license plate, the method further includes:

The license plate recognition result is processed through the connection time classification CTC algorithm to obtain the license plate information of the to-be-recognized double-carriage license plate.

Optionally, before the step of using a preset feature extraction model to perform feature extraction on the license plate image to obtain a license plate feature matrix, the method further includes:

Perform color statistics on the license plate image, and determine the color of the license plate number according to the statistical results;

Determine the pixel coordinates of the license plate number according to the color of the license plate number, and determine the effective area of the license plate number according to the pixel coordinates of the license plate number;

The license plate image is cropped according to the effective area of the license plate number to obtain a license plate number image;

The step of using a preset feature extraction model to perform feature extraction on the license plate image to obtain a license plate feature matrix includes:

A preset feature extraction model is used to perform feature extraction on the license plate number image to obtain a license plate feature matrix.

Optionally, before the step of using a preset feature extraction model to perform feature extraction on the license plate image to obtain a license plate feature matrix, the method further includes:

Perform image correction processing on the license plate image to obtain a corrected license plate image;

The step of using a preset feature extraction model to perform feature extraction on the license plate image to obtain a license plate feature matrix includes:

A preset feature extraction model is used to perform feature extraction on the corrected license plate image to obtain a license plate feature matrix.

Optionally, the step of performing image correction processing on the license plate image to obtain a corrected license plate image includes:

Perform nonlinear grayscale transformation on the license plate image through nonlinear index to obtain the transformed license plate image;

Perform edge detection on the transformed license plate image by using a preset edge detection algorithm to obtain a license plate character bounding frame;

The equation corresponding to the bounding box of the license plate character is established by straight line fitting, and the character inclination angle is obtained according to the equation;

The license plate image is rotated and corrected based on the character inclination angle to obtain a corrected license plate image.

In addition, in order to achieve the above purpose, the present invention also provides a dual-vehicle license plate recognition device, the dual-vehicle license plate recognition device includes:

The image acquisition module is used to acquire the license plate image of the double-carriage license plate to be recognized;

a feature extraction module, configured to perform feature extraction on the license plate image by using a preset feature extraction model to obtain a license plate feature matrix;

a feature reorganization module, used for feature reorganization of the license plate feature matrix to obtain the target license plate feature;

The license plate recognition module is used to input the feature of the target license plate into a preset depth bidirectional recurrent neural network to obtain the license plate recognition result of the dual-carriage license plate to be recognized.

In addition, in order to achieve the above object, the present invention also provides a dual-vehicle license plate recognition device, which includes a memory, a processor, and a dual-vehicle license plate stored in the memory and executed by the processor. A recognition program, wherein the dual-vehicle license plate recognition program, when executed by the processor, implements the steps of the dual-vehicle license plate recognition method as described above.

In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium, on which a dual-vehicle license plate recognition program is stored, wherein when the dual-vehicle license plate recognition program is executed by a processor, the The steps of the method for recognizing the dual-vehicle license plate as described above.

The present invention provides a dual-vehicle license plate recognition method, device, equipment and computer-readable storage medium. By acquiring a license plate image of a dual-lane license plate to be recognized, a preset feature extraction model is used to extract features from the license plate image to obtain a license plate feature matrix; Then, the feature recombination is performed on the license plate feature matrix to obtain the target license plate feature; and then the target license plate feature is input into the preset depth bidirectional recurrent neural network to obtain the license plate recognition result of the double-carriage license plate to be recognized. In this paper, the license plate image of the double-carriage license plate to be recognized can be directly input as a whole, and there is no need to segment and recognize each character of the license plate. Compared with the separate identification of each license plate character in the prior art, it can save work. to improve the recognition efficiency. At the same time, the present invention can avoid the whole license plate recognition error caused by the segmentation error of a single character, therefore, the present invention can improve the accuracy of the double-carriage license plate recognition result.

Description of drawings

1 is a schematic diagram of a device structure of a hardware operating environment involved in an embodiment of the present invention;

FIG. 2 is a schematic flowchart of the first embodiment of the method for recognizing dual-vehicle license plates according to the present invention;

FIG. 3 is a schematic diagram of the functional modules of the first embodiment of the dual-vehicle license plate recognition device according to the present invention.

The realization, functional characteristics and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments.

Detailed ways

It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Referring to FIG. 1 , FIG. 1 is a schematic diagram of a device structure of a hardware operating environment involved in an embodiment of the present invention.

The dual-vehicle license plate recognition device involved in the embodiment of the present invention may be a terminal device with display and processing functions, such as a PC (personal computer, personal computer), a notebook computer, and a server.

As shown in FIG. 1 , the dual-vehicle license plate recognition device may include: a processor 1001 , such as a CPU (Central Processing Unit, central processing unit), a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 . Wherein, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface . Optionally, the network interface 1004 can include a standard wired interface, a wireless interface (such as Wireless-Fidelity, Wi-Fi interface); the memory 1005 can be a high-speed random access memory (random access memory, RAM), or a stable A non-volatile memory, such as a disk memory, the memory 1005 may optionally be a storage device independent of the aforementioned processor 1001 . Those skilled in the art can understand that the structure of the dual-vehicle license plate recognition device shown in FIG. Or a different component arrangement.

Continuing to refer to FIG. 1 , the memory 1005 as a computer storage medium in FIG. 1 may include an operating system, a network communication module and a dual-vehicle license plate recognition program. In FIG. 1, the network communication module can be used to connect to the server and perform data communication with the server; and the processor 1001 can be used to call the dual-vehicle license plate recognition program stored in the memory 1005, and perform the following operations:

Obtain the license plate image of the double-carriage license plate to be recognized;

Using a preset feature extraction model to perform feature extraction on the license plate image to obtain a license plate feature matrix;

Perform feature reorganization on the license plate feature matrix to obtain the target license plate feature;

The feature of the target license plate is input into a preset depth bidirectional recurrent neural network to obtain the license plate recognition result of the double-carriage license plate to be recognized.

Further, the preset feature extraction model includes an input layer, a convolution layer, a pooling layer and a normalization layer; wherein,

The input layer is used to receive the license plate image of the to-be-recognized dual-vehicle license plate;

The convolution layer is used to extract the image feature matrix of the license plate image according to the convolution kernel;

The pooling layer is used for pooling the output of the convolutional layer;

The normalization layer is used for normalizing the output of the convolution layer.

Further, the processor 1001 can call the dual-vehicle license plate recognition program stored in the memory 1005, and also perform the following operations:

Splitting the license plate feature matrix according to the height of the license plate feature matrix to obtain a first feature matrix and a second feature matrix;

The first feature matrix and the second feature matrix are combined and spliced to obtain the target license plate feature.

Further, the processor 1001 can call the dual-vehicle license plate recognition program stored in the memory 1005, and also perform the following operations:

The license plate recognition result is processed through the connection time classification CTC algorithm to obtain the license plate information of the to-be-recognized double-carriage license plate.

Further, the processor 1001 can call the dual-vehicle license plate recognition program stored in the memory 1005, and also perform the following operations:

Perform color statistics on the license plate image, and determine the color of the license plate number according to the statistical results;

Determine the pixel coordinates of the license plate number according to the color of the license plate number, and determine the effective area of the license plate number according to the pixel coordinates of the license plate number;

The license plate image is cropped according to the effective area of the license plate number to obtain a license plate number image;

A preset feature extraction model is used to perform feature extraction on the license plate number image to obtain a license plate feature matrix.

Further, the processor 1001 can call the dual-vehicle license plate recognition program stored in the memory 1005, and also perform the following operations:

Perform image correction processing on the license plate image to obtain a corrected license plate image;

A preset feature extraction model is used to perform feature extraction on the corrected license plate image to obtain a license plate feature matrix.

Further, the processor 1001 can call the dual-vehicle license plate recognition program stored in the memory 1005, and also perform the following operations:

Perform nonlinear grayscale transformation on the license plate image through nonlinear index to obtain the transformed license plate image;

Perform edge detection on the transformed license plate image by using a preset edge detection algorithm to obtain a license plate character bounding frame;

The equation corresponding to the bounding box of the license plate character is established by straight line fitting, and the character inclination angle is obtained according to the equation;

The license plate image is rotated and corrected based on the character inclination angle to obtain a corrected license plate image.

Based on the above-mentioned hardware structure, various embodiments of the method for recognizing dual-vehicle license plates of the present invention are proposed.

The present invention provides a method for recognizing a double vehicle license plate.

Please refer to FIG. 2 . FIG. 2 is a schematic flowchart of a first embodiment of a method for recognizing dual-vehicle license plates according to the present invention.

In this embodiment, the dual-vehicle license plate recognition method includes:

Step S10, obtaining the license plate image of the double-carriage license plate to be recognized;

In this embodiment, the dual-vehicle license plate recognition method is implemented by a dual-lane license plate recognition device, which may be a PC, a notebook computer, a server, etc. The dual-lane license plate recognition device is described by taking a server as an example.

In this embodiment, the license plate image of the double-carriage license plate to be recognized is obtained first.

Step S20, using a preset feature extraction model to perform feature extraction on the license plate image to obtain a license plate feature matrix;

Then, a preset feature extraction model is used to perform feature extraction on the license plate image to obtain a license plate feature matrix. Wherein, the type of the preset feature extraction model can be selected as a convolutional neural network model, and the preset feature extraction model includes an input layer, a convolution layer, a pooling layer and a normalization layer; wherein,

The input layer is used to receive the license plate image of the to-be-recognized dual-vehicle license plate;

The convolution layer is used to extract the image feature matrix of the license plate image according to the convolution kernel;

The pooling layer is used for pooling the output of the convolutional layer;

The normalization layer is used for normalizing the output of the convolution layer.

Specifically, the preset feature extraction model (ie, the convolutional neural network model) includes 1 input layer (Input), 7 convolution layers (Conv), 4 pooling layers (Pool) and 2 normalization layers (BatchNorm), the dimensions and configuration parameters of each layer structure are shown in Table 1, where the dimensions represent the number of channels x height x width, respectively, and K, S, and P in the configuration parameters represent the size of the convolution kernel, step size, and padding (filling), respectively. )Number of. Among them, the input layer is used to receive the license plate image of the license plate to be recognized, the convolution layer is used to extract the image feature matrix of the license plate image according to the convolution kernel; the pooling layer is used to pool the output of the convolution layer, that is The image feature value that can best represent the local features of the image in each image feature matrix is extracted, and the normalization layer is used to normalize the output of the convolution layer. In addition, the convolutional neural network model can also add a zero-padding (zero-padding) layer to use the zero-padding method to perform zero-padding operations on the feature map to complement the boundary, thereby ensuring the output of the feature map from this layer. Size does not change.

After feature extraction is performed by the preset feature extraction model, the obtained license plate feature matrix has a dimension of 512x2x12, which is a matrix with a height of 2.

network layer type dimension configure Input 3x64x96 Conv1 64x64x96 K=3, S=1, P=1 Pool1 64x32x48 K=2, S=2 Conv2 128x32x48 K=3, S=1, P=1 Pool2 128x16x24 K=2, S=2 Conv3 256x16x24 K=3, S=1, P=1 BatchNorm 256x16x24 Conv4 256x16x24 K=3, S=1, P=1 Pool3 256x8x12 K=2, S=2 Conv5 512x8x12 K=3, S=1, P=1 BatchNorm 512x8x12 Conv6 512x8x12 K=3, S=1, P=1 Pool4 512x4x12 K=[2,1], S=[2,1] ZeroPadding 512x4x14 P=[0,1] Conv7 512x2x12 K=3, S=1

Table 1 Network structure of preset feature extraction model

Step S30, performing feature reorganization on the license plate feature matrix to obtain the target license plate feature;

After obtaining the license plate feature matrix, perform feature recombination on the license plate feature matrix to obtain the target license plate feature.

Specifically, step S30 includes:

Step a31, splitting the license plate feature matrix according to the height of the license plate feature matrix to obtain a first feature matrix and a second feature matrix;

Step a32, combining and splicing the first feature matrix and the second feature matrix to obtain the target license plate feature.

The license plate feature matrix is split according to the height of the license plate feature matrix to obtain the first feature matrix and the second feature matrix. That is, the obtained license plate feature matrix with a dimension of 512x2x12 is divided by height to obtain two feature matrices with a matrix dimension of 512x1x12, and the feature matrix obtained based on the first row of the license plate feature matrix is recorded as the first feature matrix, The feature matrix obtained based on the second row of the license plate feature matrix is denoted as the second feature matrix. Then, the first feature matrix and the second feature matrix are combined and spliced to obtain the target license plate feature. When combining and splicing, the first feature matrix and the second feature matrix are recombined horizontally, and the dimension of the recombined matrix is 512x1x24, that is, the target license plate feature with a height of 1 is obtained by recombination.

Step S40, inputting the feature of the target license plate into a preset depth bidirectional recurrent neural network to obtain the license plate recognition result of the dual-carriage license plate to be recognized.

Finally, the feature of the target license plate is input into the preset depth bidirectional recurrent neural network, and the license plate recognition result of the double-carriage license plate to be recognized is obtained. The preset depth bidirectional recurrent neural network is optionally a bidirectional long short-term memory network (Bi-directional Long Short-Term Memory, Bi-LSTM). It can be understood that Bi-LSTM has the ability to process feature information from left to right. Through context information, the processing results of the current frame are fused with the previous memory and forgetting results, so that predictions can be made and the structure can be passed to subsequent frames. The double-carriage license plate is also a character sequence after recombination, which has the time sequence characteristics from left to right, so the two-way recurrent neural network can be used for license plate character recognition.

It should be noted that the problem of the one-way Recurrent Neural Network (RNN) is that it can only use the information before time t, but sometimes it may also need to use the information in the future. The bidirectional RNN model can solve this problem. A bidirectional RNN maintains two hidden layers at all times, one for transmitting information from left to right, and another for recording information transmitted from right to left. Therefore, compared with using a one-way recurrent neural network, the embodiment of the present invention can improve the accuracy of the recognition result by using a preset depth two-way recurrent neural network.

The invention provides a double-vehicle license plate recognition method. By acquiring the license plate image of the double-vehicle license plate to be recognized, a preset feature extraction model is used to extract the characteristics of the license plate image to obtain a license plate feature matrix; then, the license plate feature matrix is reorganized, The target license plate feature is obtained; then the target license plate feature is input into the preset depth bidirectional recurrent neural network to obtain the license plate recognition result of the double-carriage license plate to be recognized. In this embodiment, the license plate image of the double-carriage license plate to be recognized can be directly input as a whole, and there is no need to segment and recognize each character of the license plate. Save workload and improve identification efficiency. At the same time, in the embodiment of the present invention, the entire license plate recognition error caused by the segmentation error of a single character can be avoided. Therefore, the embodiment of the present invention can improve the accuracy of the double-carriage license plate recognition result.

Further, based on the above-mentioned first embodiment, a second embodiment of the dual-vehicle license plate recognition method of the present invention is proposed.

In this embodiment, after step S40, the dual-vehicle license plate recognition method further includes:

In step A, the license plate recognition result is processed through the connection time classification CTC algorithm to obtain the license plate information of the dual-carriage license plate to be recognized.

In this embodiment, because the number of characters in the license plate image is different, the font style is different, or the size is different, the output may not necessarily correspond to each character one-to-one. Therefore, in order to further ensure the accuracy of the recognition result, CTC The (Connectionist Temporal Classification, connection time classification) algorithm processes the license plate recognition results to perform character alignment operations to obtain the final license plate information. Among them, the CTC algorithm can combine repeated elements and remove space characters, which can be used to solve the input Alignment of features and output labels.

In this embodiment, by combining the deep bidirectional recurrent neural network with the CTC algorithm, problems such as missed detection or multi-detection of dual-vehicle license plate information can be effectively avoided, thereby further improving the accuracy of license plate recognition results.

Further, based on the above-mentioned first embodiment, a third embodiment of the method for recognizing dual-vehicle license plates of the present invention is proposed.

In this embodiment, before step S20, it further includes:

Step B, performing color statistics on the license plate image, and determining the color of the license plate number according to the statistical result;

In this embodiment, after acquiring the license plate image of the double-carriage license plate to be recognized, color statistics are performed on the license plate image, and the color of the license plate number is determined according to the statistical result. Because the license plate image mainly contains two colors, one is the background color of the license plate, and the other is the color of the license plate number. Therefore, after performing color statistics, the color corresponding to the highest statistical value in the statistical results is the background color of the license plate, and the color corresponding to the second statistical value in the statistical results is the color of the license plate number.

When performing color statistics, the RGB (Red Green Blue, also known as RGB color mode) value of each pixel of the license plate image can be extracted through a preset image reading function, and then based on the value of each pixel of the license plate image RGB values perform color statistics on license plate images. Optionally, the preset image reading function is the imread() function. After reading the license plate image through the imread() function, a numpy (an open source scientific computing library of Python) array can be obtained, that is, each pixel of the license plate image corresponds to Then traverse the numpy array to perform color statistics on the RGB values of the extracted license plate image to obtain statistical results.

Step C, determine the pixel coordinates of the license plate number according to the color of the license plate number, and determine the effective area of the license plate number according to the pixel coordinates of the license plate number;

Then, the pixel coordinates of the license plate number are determined according to the color of the license plate number, and the valid area of the license plate number is determined according to the pixel coordinates of the license plate number. Specifically, the pixel coordinates corresponding to the RGB value of the license plate number color can be obtained, that is, the license plate number pixel coordinates, and then the outermost pixel coordinates are determined based on the license plate number pixel coordinates to form the license plate number valid area.

Step D, cutting the license plate image according to the effective area of the license plate number to obtain a license plate number image;

Then, the license plate image is cropped according to the effective area of the license plate number to obtain the license plate number image.

At this time, step S20 includes:

Step A21, using a preset feature extraction model to perform feature extraction on the license plate number image to obtain a license plate feature matrix.

After the license plate number image is obtained by cropping, a preset feature extraction model is used to extract the features of the license plate number image to obtain a license plate feature matrix, and then the subsequent steps are performed.

In this embodiment, by performing color statistics on the license plate image, and then determining the effective area of the license plate number, the license plate image is cropped to obtain the license plate number image, and then the identification is based on the license plate number image. In the above manner, the image recognition area can be reduced, so that the recognition efficiency can be further improved.

Further, based on the above-mentioned first embodiment, a fourth embodiment of the method for recognizing dual-vehicle license plates of the present invention is proposed.

In this embodiment, before step S20, it further includes:

Step E, performing image correction processing on the license plate image to obtain a corrected license plate image;

In this embodiment, due to the problem of the shooting angle, the license plate number in the obtained license plate image of the dual-vehicle license plate to be recognized may be inclined or distorted. Therefore, in order to further improve the accuracy of the license plate recognition result, the After acquiring the license plate image of the double-carriage license plate to be recognized, image correction processing is performed on the license plate image to obtain a corrected license plate image.

Specifically, step E includes:

Step E1, performing nonlinear grayscale transformation on the license plate image through a nonlinear index to obtain a transformed license plate image;

Step E2, performing edge detection on the transformed license plate image by using a preset edge detection algorithm to obtain a character bounding frame of the license plate;

Step E3, establishing an equation corresponding to the bounding box of the license plate character by straight line fitting, and obtaining the character inclination angle according to the equation;

Step E4, performing rotation correction on the license plate image based on the character inclination angle to obtain a corrected license plate image.

The process of image correction processing is as follows:

Firstly, the license plate image is subjected to nonlinear grayscale transformation through the nonlinear index to obtain the transformed license plate image. Through grayscale transformation, the high grayscale area of the license plate image can be stretched, so that the grayscale contrast of the image is more significant, which is convenient for subsequent identification. deal with. Then, edge detection is performed on the transformed license plate image through a preset edge detection algorithm to obtain the bounding frame of the license plate characters. The preset edge detection algorithm may be a sobel operator or a canny algorithm.

After obtaining the license plate character bounding box, the equation corresponding to the license plate character bounding box is established by straight line fitting, and the character inclination angle is obtained according to the equation. The calculation process of the character inclination angle is as follows: the outer frame of the license plate will have a certain radian due to shooting and other reasons, and the long straight line where the frame of the license plate is located may be broken into several straight line segments with different slopes. Therefore, the character is established by straight line fitting. The equation corresponding to the bounding box also includes multiple items. To obtain the equations of each straight line y=ax+b, based on the calculation formula of the inclination angle tanθ=-b/a, multiple angle values can be obtained, and the average value of each angle value of the inclination angle in the range of -10 to 10° can be obtained, Get the character inclination angle.

Finally, the license plate image is rotated and corrected based on the character inclination angle to obtain the corrected license plate image.

At this time, step S20 includes:

Step A22, using a preset feature extraction model to perform feature extraction on the corrected license plate image to obtain a license plate feature matrix.

After the license plate image of the double-pass license plate to be recognized is corrected, the preset feature extraction model is used to extract the feature of the corrected license plate image, and the license plate feature matrix is obtained, and then the subsequent steps are executed. No further elaboration here.

In this embodiment, by performing image correction processing on the license plate image, and then recognizing the corrected license plate image, the accuracy of the double-carriage license plate recognition result can be further improved.

The invention also provides a double-drive license plate recognition device.

Referring to FIG. 3 , FIG. 3 is a schematic diagram of the functional modules of the first embodiment of the dual-vehicle license plate recognition device of the present invention.

In this embodiment, the dual-vehicle license plate recognition device includes:

The image acquisition module 10 is used for acquiring the license plate image of the double-carriage license plate to be recognized;

A feature extraction module 20, configured to perform feature extraction on the license plate image by using a preset feature extraction model to obtain a license plate feature matrix;

The feature recombination module 30 is used to perform feature recombination on the license plate feature matrix to obtain the target license plate feature;

The license plate recognition module 40 is configured to input the feature of the target license plate into a preset depth bidirectional recurrent neural network to obtain the license plate recognition result of the dual-carriage license plate to be recognized.

Wherein, each virtual function module of the above-mentioned dual-vehicle license plate recognition device is stored in the memory 1005 of the dual-vehicle license plate recognition device shown in FIG. It can realize the recognition function of double license plate.

Further, the preset feature extraction model includes an input layer, a convolution layer, a pooling layer and a normalization layer; wherein,

The input layer is used to receive the license plate image of the to-be-recognized dual-vehicle license plate;

The convolution layer is used to extract the image feature matrix of the license plate image according to the convolution kernel;

The pooling layer is used for pooling the output of the convolutional layer;

The normalization layer is used for normalizing the output of the convolution layer.

Further, the feature reorganization module 30 includes:

a matrix splitting unit, configured to split the license plate feature matrix according to the height of the license plate feature matrix to obtain a first feature matrix and a second feature matrix;

The combined splicing unit is used for combining and splicing the first feature matrix and the second feature matrix to obtain the target license plate feature.

Further, the dual-vehicle license plate recognition device also includes:

The result processing module is used for processing the license plate recognition result through the connection time classification CTC algorithm to obtain the license plate information of the dual-carriage license plate to be recognized.

Further, the dual-vehicle license plate recognition device also includes:

The color statistics module is used to perform color statistics on the license plate image, and determine the color of the license plate number according to the statistical results;

an area determination module, configured to determine the pixel coordinates of the license plate number according to the color of the license plate number, and determine the effective area of the license plate number according to the pixel coordinates of the license plate number;

an image cropping module, configured to crop the license plate image according to the effective area of the license plate number to obtain a license plate number image;

The feature extraction module 20 is also used for:

A preset feature extraction model is used to perform feature extraction on the license plate number image to obtain a license plate feature matrix.

Further, the dual-vehicle license plate recognition device also includes:

an image correction module for performing image correction processing on the license plate image to obtain a corrected license plate image;

The feature extraction module 20 is also used for:

A preset feature extraction model is used to perform feature extraction on the corrected license plate image to obtain a license plate feature matrix.

Further, the image correction module includes:

The grayscale transformation unit is used to perform nonlinear grayscale transformation on the license plate image through the nonlinear index to obtain the transformed license plate image;

an edge detection unit, configured to perform edge detection on the transformed license plate image by using a preset edge detection algorithm to obtain a bounding frame of the license plate characters;

an angle acquisition unit, configured to establish an equation corresponding to the bounding box of the license plate character by straight line fitting, and obtain the character inclination angle according to the equation;

An image correction unit, configured to perform rotation correction on the license plate image based on the character inclination angle to obtain a corrected license plate image.

The function implementation of each module in the above-mentioned dual-vehicle license plate recognition device corresponds to each step in the above-mentioned embodiment of the dual-vehicle license plate recognition method, and the functions and implementation processes thereof will not be repeated here.

The present invention further provides a computer-readable storage medium, where a dual-vehicle license plate recognition program is stored on the computer-readable storage medium, and when the dual-vehicle license plate recognition program is executed by a processor, any of the above embodiments can be implemented. The steps of the dual-vehicle license plate recognition method.

The specific embodiments of the computer-readable storage medium of the present invention are basically the same as those of the above-mentioned embodiments of the method for recognizing two-way license plates, and will not be repeated here.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or system comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or system. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system that includes the element.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on such understanding, the technical solutions of the present invention can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products are stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disk), including several instructions to make a device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

The above are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields , are similarly included in the scope of patent protection of the present invention.

Claims (10)

1. A double-row license plate recognition method is characterized by comprising the following steps:

acquiring a license plate image of a double-row license plate to be identified;

extracting the characteristics of the license plate image by adopting a preset characteristic extraction model to obtain a license plate characteristic matrix;

performing characteristic recombination on the license plate characteristic matrix to obtain target license plate characteristics;

and inputting the target license plate characteristics into a preset depth bidirectional recurrent neural network to obtain a license plate recognition result of the double-row license plate to be recognized.

2. The method for recognizing a double-row license plate of claim 1, wherein the preset feature extraction model comprises an input layer, a convolution layer, a pooling layer and a normalization layer; wherein,

the input layer is used for receiving the license plate image of the double-row license plate to be recognized;

the convolution layer is used for extracting an image characteristic matrix of the license plate image according to a convolution kernel;

the pooling layer is used for pooling the output of the convolutional layer;

the normalization layer is used for performing normalization processing on the output of the convolution layer.

3. The method for recognizing a double-row license plate of claim 1, wherein the step of performing feature recombination on the license plate feature matrix to obtain the target license plate feature comprises:

splitting the license plate feature matrix according to the height of the license plate feature matrix to obtain a first feature matrix and a second feature matrix;

and combining and splicing the first characteristic matrix and the second characteristic matrix to obtain the target license plate characteristic.

4. The method for recognizing the double-row license plate of any one of claims 1 to 3, wherein after the step of inputting the target license plate feature into a bidirectional recurrent neural network with a preset depth to obtain the license plate recognition result of the double-row license plate to be recognized, the method further comprises:

and processing the license plate recognition result through a Connection Time Classification (CTC) algorithm to obtain the license plate information of the double-row license plate to be recognized.

5. The double-row license plate recognition method of any one of claims 1 to 3, wherein before the step of performing feature extraction on the license plate image by using a preset feature extraction model to obtain a license plate feature matrix, the method further comprises:

carrying out color statistics on the license plate image, and determining the color of the license plate according to the statistical result;

determining license plate number pixel coordinates according to the license plate number color, and determining a license plate number effective area according to the license plate number pixel coordinates;

cutting the license plate image according to the license plate number effective area to obtain a license plate number image;

the method comprises the following steps of adopting a preset feature extraction model to extract features of the license plate image to obtain a license plate feature matrix, wherein the license plate feature matrix comprises the following steps:

and performing feature extraction on the license plate number image by adopting a preset feature extraction model to obtain a license plate feature matrix.

6. The double-row license plate recognition method of any one of claims 1 to 3, wherein before the step of performing feature extraction on the license plate image by using a preset feature extraction model to obtain a license plate feature matrix, the method further comprises:

carrying out image correction processing on the license plate image to obtain a corrected license plate image;

the method comprises the following steps of adopting a preset feature extraction model to extract features of the license plate image to obtain a license plate feature matrix, wherein the license plate feature matrix comprises the following steps:

and performing feature extraction on the corrected license plate image by adopting a preset feature extraction model to obtain a license plate feature matrix.

7. The method for recognizing the double-row license plate of claim 6, wherein the step of performing image correction processing on the license plate image to obtain a corrected license plate image comprises:

carrying out nonlinear gray scale conversion on the license plate image through a nonlinear index to obtain a converted license plate image;

performing edge detection on the transformed license plate image through a preset edge detection algorithm to obtain a license plate character outer frame;

establishing an equation corresponding to the license plate character outer frame through linear fitting, and obtaining a character inclination angle according to the equation;

and carrying out rotation correction on the license plate image based on the character inclination angle to obtain a corrected license plate image.

8. A double-row license plate recognition device, characterized in that it comprises:

the image acquisition module is used for acquiring a license plate image of a double-row license plate to be identified;

the characteristic extraction module is used for extracting the characteristics of the license plate image by adopting a preset characteristic extraction model to obtain a license plate characteristic matrix;

the characteristic recombination module is used for performing characteristic recombination on the license plate characteristic matrix to obtain target license plate characteristics;

and the license plate recognition module is used for inputting the target license plate characteristics into a preset depth bidirectional recurrent neural network to obtain a license plate recognition result of the double-row license plate to be recognized.

9. A dual-line license plate recognition device comprising a memory, a processor, and a dual-line license plate recognition program stored on the memory and executable by the processor, wherein the dual-line license plate recognition program, when executed by the processor, implements the steps of the dual-line license plate recognition method of any one of claims 1 to 7.

10. A computer-readable storage medium, wherein a dual-line license plate recognition program is stored on the computer-readable storage medium, and wherein the dual-line license plate recognition program, when executed by a processor, implements the steps of the dual-line license plate recognition method of any one of claims 1 to 7.

Images