JP2022054632A - Program and device for recognizing license plate - Google Patents

Abstract

A license plate recognition program and a license plate recognition device that facilitate cutting out a license plate image are provided. Kind Code: A1 In an image data acquisition processing step 301, for example, front image data including a license plate of a vehicle is acquired from a camera. In black-and-white binary data conversion processing step 302, the acquired image data is converted into black-and-white binary data. Next, in a rectangular contour region extraction processing step 303, the contour of the black and white binarized data is extracted as a rectangular contour region. Next, in a number/point area extraction processing step 304, a number/point area representing a four-digit vehicle number number or a point is extracted from the four rectangular outline areas. Next, in vehicle number extraction processing step 305, the vehicle number is extracted from the number/dot area. Finally, in the license plate completion processing step 306, the vehicle number is modified by extracting the place name, the classification number and hiragana other than the vehicle number, and the extraction of the license plate is completed. [Selection drawing] Fig. 3

Description

The present invention relates to a number plate recognition program and a number plate recognition device.

It may be necessary to recognize the number plate of a stopped or running vehicle in a parking lot, on the road, or the like.

The conventional number plate recognition method recognizes the number plate information after cutting out the number plate image from the vehicle image taken by the camera (see: Patent Document 1).

Japanese Unexamined Patent Publication No. 2015-176248

However, in the above-mentioned conventional number plate recognition method, the number plate image largely depends on the environment of the vehicle, the color of the vehicle, and the color of the number plate, and there is a problem that it is difficult to cut out the number plate image.

In order to solve the above-mentioned problems, the number plate recognition program according to the present invention has an image data acquisition procedure for acquiring image data in front of or behind the vehicle, and black-and-white binomial data for converting image data into black-and-white binarization data. Conversion data conversion procedure, rectangular contour area extraction procedure to extract the concatenated black data or white data concatenated contour of black-and-white binarized data as a rectangular contour area, and numerical value to extract a numerical area and a point area from the rectangular contour area. -It includes a point area extraction procedure and a vehicle number extraction procedure for extracting the vehicle number of the vehicle from the numerical area and the point area.

Further, the number plate recognition device according to the present invention includes an image data acquisition means for acquiring image data in front of or behind the vehicle, a black-and-white binarization data conversion means for converting image data into black-and-white binarization data, and a black-and-white binarization data. A rectangular contour area extraction means for extracting a concatenated contour of black data or white data concatenated with binarized data as a rectangular contour area, and a number / point area extraction means for extracting a numerical area and a point area from the rectangular contour area. It is provided with a vehicle number extracting means for extracting a vehicle number of a vehicle from a numerical area and a point area.

According to the present invention, the connected contour is extracted as a rectangular contour region, the number / point region is extracted from the rectangular contour region, and the vehicle number of the number plate is extracted from the number / point region, so that the number plate can be easily cut out. can.

It is a block diagram which shows the system which includes the embodiment of the number plate recognition apparatus which concerns on this invention. It is a detailed block circuit diagram of a number plate recognition device. It is a flowchart for demonstrating the operation of the central processing unit of FIG. It is a figure for demonstrating steps 301 to 303 of FIG. It is a figure for demonstrating steps 304-306 of FIG. It is a figure which shows the example of the template and the rectangular contour area used for the extraction of the number / point area of the number / point area extraction processing step of FIG. It is a figure for demonstrating an example of the extraction method of the number / point area of the number / point area extraction processing step of FIG. It is a figure for demonstrating another example of the extraction method of the number / point area of the number / point area extraction processing step of FIG. It is a detailed flowchart of the vehicle number extraction processing step of FIG. It is a figure for demonstrating the vehicle number candidate extraction of FIG. It is a figure for demonstrating the number plate completion processing step of FIG. It is a block diagram which shows another example of the number plate recognition apparatus of FIG.

FIG. 1 is a block diagram showing a system including an embodiment of the number plate recognition device according to the present invention.

In FIG. 1, a camera (for example, a monocular camera) 2 provided in a parking lot or a road is connected to the number plate recognition device 1. The camera 2 captures the front or rear including the number plate of the stopped or running vehicle. Further, the number plate recognition device 1 is connected to the information terminal 4 or the like via the Internet 3 as needed. The operator can operate the number plate recognition device 1 via the information terminal 4 or the like or directly.

In general, the number plate is composed of the following four parts.
Vehicle number consisting of 4-digit number / dot (however, if there is a dot, the dot always exists before the number),
Place names such as Shinagawa and Adachi provided in the upper left of the vehicle number,
A 3-digit or 2-digit classification number provided on the upper right of the vehicle number,
Hiragana, etc. provided on the left side of the vehicle number (may be in English).

FIG. 2 is a detailed block circuit diagram of the number plate recognition device 1 of FIG.

In FIG. 2, the number plate recognition device 1 includes a central processing unit (CPU) 101, a read-only memory (ROM) (or flash memory) 102 for storing programs and the like, and a random access memory (RAM) for storing temporary data and the like. It is composed of 103, an image interface 104 connected to the camera 2, a communication interface 105 connected to the internet 3, and the like.

FIG. 3 is a flowchart for explaining the operation of the CPU 101 of FIG. 2, which is stored in the read-only memory (or flash memory) 102.

First, in the image data acquisition processing step 301, for example, front image data including the vehicle number plate shown in FIG. 4A is acquired from the camera 2. In this case, if the image data of the camera 2 is RGB color data, it is grayscaled.

Next, in the black-and-white binarization data conversion process step 302, the image data acquired in step 301 is converted into the black-and-white binarization data shown in FIG. 4B. The details of the black-and-white binarization data conversion processing step 302 will be described later.

Next, in the rectangular contour region extraction processing step 303, the contour of the black-and-white binarized data, for example, the contour of the concatenated black data or the white data is extracted as a rectangular contour region as shown in FIG. 4 (C). The details of the rectangular contour region extraction processing step 303 will be described later.

Next, in the number / point area extraction processing step 304, the number / point area shown in FIG. 5A representing the four-digit vehicle number or point is extracted from the four rectangular contour areas. The details of the number / point area extraction processing step 304 will be described later.

Next, in the vehicle number extraction processing step 305, the vehicle number shown in FIG. 5 (B) with four digits is extracted from the number / point area. The details of the vehicle number extraction processing step 305 will be described later.

Finally, in the number plate completion processing step 306, the place name, for example, "Adachi", the classification number, for example, "330" and the hiragana "ta" shown in FIG. 5 (C) other than the vehicle number are extracted and the vehicle number is modified. And complete the extraction of the number plate. The details of the number plate completion processing step 306 will be described later. Then, in step 307, the routine of FIG. 3 ends.

Next, the details of the black-and-white binarization data conversion processing step 302 of FIG. 3 will be described. First, a Gaussian blur process is performed to remove noise from the image data. The Gaussian blur process is a process of calculating the convolution sum using a two-dimensional Gaussian function whose value becomes smaller as the distance from the center increases. Next, an optimum binarization process is performed to establish appropriate shading data. In the optimal binarization process, the threshold value is not fixed, and the average value of a total of nine pixels of the pixel of interest and, for example, eight pixels around the pixel of interest is used as the threshold value, and the pixel of interest and this threshold value are used. By comparing the above, the pixel of interest is converted into black-and-white binarized data.

Next, the details of the rectangular contour region extraction processing step 303 of FIG. 3 will be described. First, detect the black outline. Next, the connected black contour is surrounded by a rectangle or a rectangle and extracted as a rectangular contour region. It should be noted that instead of the connected black contour portion, a connected white contour can also be used.

Next, extraction of the number / point area in the number / point area extraction processing step 304 will be described. In this case, the numbers and points in the point area are the numbers of the vehicle number, and the 11 dictionary numbers (templates) shown in FIG. 6A are stored in advance in the read-on memory (or flash memory) 102. back. Each of the 11 templates shown in FIG. 6 (A) and the binarized data of the image of the rectangular contour region shown in FIG. 6 (B) extracted in step 303 are compared using Boolean operations. Explaining an example of the Boolean operation with reference to FIG. 7, the template shown in (A) is 8 × 8 shown in (B), for example, the black pixel of “8” is T (true) and the white pixel is F (false). Pixel (actually 28 x 28 pixels) Convert to a Boolean template and save. On the other hand, it is converted into an 8 × 8 pixel Boolean image shown in (C) as black pixels T and white pixels F in the extracted rectangular contour area. Here, what is important is that the pixel format of the Boolean template shown in (B) and the pixel format of the Boolean image shown in (C) are the same, that is, 8 × 8 pixels in this case. As a result, when the 8 × 8 pixel Boolean template shown in (B) and the 8 × 8 pixel Boolean image shown in (C) are compared using the exclusive OR for each corresponding pixel, 8 shown in (D) is compared. Obtain a comparison result of × 8 pixels. In this case, the similarity is 59/64 = 92%. Similarly, to explain another example of the Boolean operation with reference to FIG. 8, in this case, the similarity is 5/64 = 8%. Generally, in the case of a number plate with a white background and a black number, as shown in FIG. 7, the larger the similarity, the more similar the Boolean image is to the Boolean template, but conversely, the background is black. In the case of a number plate with white numbers, as shown in FIG. 8, the smaller the similarity, the more similar the Boolean image to the Boolean template. Therefore, in order to match both, it may be determined that the image of the extracted rectangular contour region resembles the numbers of the template when the threshold value of the similarity is large, for example, 92% or more or the threshold value of 8% or less. ..

Next, the details of the vehicle number extraction processing step 305 of FIG. 3 will be described with reference to FIG.

First, in step 901, the average value of all heights h of the number / point regions already extracted in step 304 of FIG. 3 is calculated.

Next, in step 902, one of the number / point regions is selected as the first region A (see: FIG. 10).

Next, in step 903, a predetermined distance range on the right side of the first region A, for example, a number / point region in the range of 2h to 6h is selected as the second region B (see: FIG. 10).

Next, in step 904, the number / point regions between the first and second regions A and B are selected as the third regions C1, C2, ... (Reference: FIG. 10).

Next, in step 905, it is determined whether or not the number of the third regions C1, C2, ... Is 2, and when the number of the third regions is a predetermined number, for example, only two, in step 906, the vehicle number. Create as a candidate list i.
i = (A, C1, C2, B)

Step 907 repeats the flow of steps 902 to 906 for all the number / point areas.

In step 908, the most probable list is selected from the vehicle number candidate list. For example, the four areas of the list have the same background color, and the 1 to 3 point areas exist on the left side of the 3 to 1 number area, considering the variation in height between the number and point areas. select.

Then, in step 909, the routine of FIG. 9 ends.

Finally, the number plate completion processing step 306 of FIG. 3 will be described with reference to FIG. As shown in (A) of FIG. 11, the place name "Adachi" exists in the upper left of the vehicle number, the classification number "330" exists in the upper right of the vehicle number, and the hiragana "ta" exists in the left of the vehicle number. Is known. Therefore, these place names, classification numbers, and hiragana are also recognized using a template of 28 × 28 pixels, and as shown in FIG. 11B, the rectangular contour area of the place name “Adachi” and the rectangular contour of the classification number “330” are recognized. Extract the area and the rectangular outline area of the hiragana "ta". Then, as shown in FIG. 11 (C), after removing the noise in each rectangular contour area, as shown in FIG. 11 (D), the place name "Adachi", the classification number "330" and the classification number "330" are obtained from each rectangular contour area. Cut out the hiragana "ta". Then, as shown in FIG. 11 (E), the vehicle number "3582" is modified with the cut out place name "Adachi", the classification number "330" and the hiragana "ta" to complete the extraction of the number plate.

FIG. 12 shows the number plate recognition device of FIG. 1 configured by hardware.

In FIG. 12, the image data acquisition processing means 1201 acquires, for example, front image data including the vehicle number plate shown in FIG. 4A from the camera 2. The black-and-white binarization data conversion processing means 1202 converts the image data acquired by the image data acquisition means 1201 into the black-and-white binarization data shown in FIG. 4B. The rectangular contour region extraction processing means 1203 extracts the contour of the black-and-white binarized data, for example, the contour of the concatenated black data or the white data as a rectangular contour region as shown in FIG. 4 (C). The number / point area extraction processing means 1204 extracts the number / point area shown in FIG. 5 (A) representing a four-digit vehicle number or point from the four rectangular contour areas. The vehicle number extraction processing means 1205 extracts the vehicle number shown in FIG. 5 (B) of four digits from the number / point area. The number plate completion processing means 1206 extracts the place name, for example, "Adachi", the classification number, for example, "330" and the hiragana "ta" shown in FIG. 5 (C) other than the vehicle number, modifies the vehicle number, and modifies the number plate. Complete the extraction.

It should be noted that the present invention may be applied to any modification within the trivial scope of the embodiments described above.

The number plate recognition device according to the present invention can be used as a parking lot management system for preventing entry / exit troubles, a speed violation detection system for detecting speed violation vehicles, and the like.

1: Number plate recognition device 2: Camera 3: Internet
4: Information terminal
101: CPU
102: ROM
103: RAM
104: Image interface 105: Communication interface

Claims (8)

Image data acquisition procedure for acquiring image data in front of or behind the vehicle, and
The black-and-white binarization data conversion procedure for converting the image data into black-and-white binarization data, and
The rectangular contour area extraction procedure for extracting the concatenated black data or the concatenated contour of the white data of the black-and-white binarized data as a rectangular contour area, and the procedure for extracting the rectangular contour area.
A number / point area extraction procedure for extracting a number area and a point area from the rectangular contour area, and a procedure for extracting the number / point area.
A number plate recognition program comprising a vehicle number extraction procedure for extracting a vehicle number of the vehicle from the number area and the point area. moreover,
The number plate recognition program according to claim 1, further comprising a number plate completion procedure for extracting other number plate information of the vehicle number from the rectangular contour region around a predetermined area of the vehicle number and modifying the vehicle number. The black-and-white binarization data conversion procedure is as follows.
A Gaussian blur processing procedure for performing Gaussian blur processing for removing noise from the image data, and
The number plate recognition program according to claim 1, further comprising an optimum binarization processing procedure for performing an optimum binarization process after the Gaussian blurring process. The procedure for extracting the rectangular contour area is as follows.
The number plate recognition program according to claim 1, further comprising a procedure of surrounding the concatenated white contour or black contour of the image data with a rectangle or a rectangle and extracting the concatenated white contour or the black contour as a rectangular contour region. The procedure for extracting numbers / point areas is as follows.
A Boolean image conversion procedure for converting an image in the rectangular contour area into a Boolean image,
A claim comprising a number / point determination procedure for determining whether or not the rectangular contour region is a number / point region by comparing the Boolean image and the Boolean number template by a pixel-compatible Boolean operation. The number plate recognition program according to 1. The vehicle number extraction processing procedure is as follows.
An average value calculation procedure for calculating the average height of all the number / point areas, and
The first selection procedure for selecting one of the number / point areas as the first area, and
A second selection procedure for selecting the number / point area within a predetermined multiple of the average height on the right side of the first area as the second area, and
A third selection procedure for selecting the number / point area between the first and second areas as the third area, and a third selection procedure.
The number plate recognition program according to claim 5, further comprising a vehicle number candidate selection procedure in which the first, second, and third regions are vehicle number candidates when the number of the third region is a predetermined number. Image data acquisition means for acquiring image data in front of or behind the vehicle, and
A black-and-white binarization data conversion means for converting the image data into black-and-white binarization data,
A rectangular contour area extraction means for extracting the concatenated black data of the black-and-white binarized data or the concatenated contour of the white data as a rectangular contour area.
A number / point area extraction means for extracting a number area and a point area from the rectangular contour area,
A number plate recognition device including a vehicle number extracting means for extracting a vehicle number of the vehicle from the number area and the point area. moreover,
The number plate recognition device according to claim 7, further comprising a number plate completion means for extracting other number plate information of the vehicle number from the rectangular contour region around a predetermined area of the vehicle number and modifying the vehicle number.

Images