CN110298811B - Image preprocessing method, device, terminal and computer readable storage medium - Google Patents

Abstract

Embodiments of the present invention provide an image preprocessing method, device, terminal, and computer-readable storage medium. The method includes: converting a pre-acquired original text image into a grayscale image; performing expansion and corrosion processing on the grayscale image, and obtaining a processing After image; divide the processed image into several sub-image blocks of preset size; determine the inflection point of the gray histogram boundary between the foreground text and background shading in each sub-image block, and obtain the pixels at the inflection point of the gray histogram boundary value; binarize the sub-image block according to the pixel value at the inflection point of the gray histogram boundary to obtain the black and white binary sub-image block corresponding to the sub-image block; integrate all the black-and-white binary sub-image blocks into a A binary image corresponding to a grayscale image. The invention obviously eliminates the influence caused by factors such as a large number of interference moire fringes, uneven exposure, random noise interference and the like during image shooting, and obviously improves the accuracy of image OCR recognition.

Description

Image preprocessing method, device, terminal, and computer-readable storage medium

technical field

The present invention relates to the technical field of image processing, and in particular, to an image preprocessing method, device, terminal, and computer-readable storage medium.

Background technique

The rapid development of science and technology has brought a strong demand for information, and in today's society, information mainly exists as a medium of "paper". The traditional information storage method is to manually enter information and save it. The disadvantages of this method are the huge amount of manual labor, long time consumption and low efficiency. Therefore, in order to reduce people's labor and meet people's urgent needs for information, the concept of digitization came into being. Optical Character Recognition (OCR, Optical Character Recognition), as a text recognition technology, uses the computer to automatically analyze the text images input by imaging devices such as scanners and recognize the text in the images, and finally convert the text images into editable text. The digitization of document images is realized, and the function of text input can be quickly realized, replacing heavy manual labor.

A complete OCR system generally includes six modules: image acquisition, preprocessing, feature extraction, recognition and classification, post-processing and recognition results. When the OCR system recognizes low-quality document images, the key technology lies in the binarization algorithm used in image preprocessing. Whether an appropriate binarization algorithm can be selected determines the quality of the binarization result, which in turn affects subsequent recognition. Correct rate. With the continuous deepening of the digitalization process, fixed large-scale equipment such as scanners can no longer meet people's needs. Therefore, mobile devices such as digital cameras, mobile phones, and PDAs (Personal Digital Assistant, PDA for short) have become a better choice for people because of their portability, fast response, low price, and freedom from environmental restrictions. Although these portable devices are not limited in space like scanners, they also have certain disadvantages: 1) When shooting, due to the limitations of shooting technology and natural conditions, such as insufficient exposure, light changes, focal length changes, document A variety of situations such as unevenness often occur, resulting in large noises, uneven lighting, tilt, distortion and other deformations in the captured image. These deformations will affect the quality of the image to a certain extent; 2) When using a mobile phone or When a digital camera shoots a text document displayed on a computer screen, the shutter speed of the camera is inconsistent with the refresh rate of the screen, resulting in serious moire interference fringes in the photographed pictures, which seriously affects the recognition efficiency of OCR.

However, due to the low quality of the text document image captured in the above-mentioned situation, the character recognition rate of the current OCR technology is relatively low when the above-mentioned captured text document image is recognized, and may not be recognized at all in severe cases.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide an image preprocessing method, device, terminal, and computer-readable storage medium, so as to solve the problems in the prior art due to serious interference moiré, large noise, and poor illumination that appear in photographed images of text documents. The problem of poor image binarization effect caused by equal factors, thereby improving the character recognition rate of text document images.

A first aspect of the embodiments of the present invention provides an image preprocessing method, including:

Convert pre-acquired raw text images to grayscale images;

performing dilation and corrosion processing on the grayscale image to obtain a processed image;

dividing the processed image into several sub-image blocks of preset size;

Determine the inflection point of the grayscale histogram boundary between the foreground text and the background shading in each of the sub-image blocks, and obtain the pixel value at the boundary inflection point of the grayscale histogram;

Perform binarization processing on the sub-image block according to the pixel value at the inflection point of the boundary of the grayscale histogram to obtain a black and white binary sub-image block corresponding to the sub-image block;

All black and white binary sub-image blocks are integrated into a binary image corresponding to the grayscale image.

Optionally, the determining the inflection point of the grayscale histogram boundary between the foreground text and the background shading in each of the sub-image blocks includes:

Obtaining a grayscale histogram distribution corresponding to the sub-image block according to each of the sub-image blocks;

The distribution characteristics of the grayscale histogram distribution are obtained, and the inflection point of the grayscale histogram boundary between the foreground text and the background shading in each of the sub-image blocks is determined according to the distribution characteristics.

Optionally, determining the inflection point of the grayscale histogram boundary between the foreground text and the background shading in each of the sub-image blocks according to the distribution feature, including:

Determine the left boundary point Left and the preset marker point Point in the distribution of the grayscale histogram according to the distribution characteristics of the grayscale histogram;

确定所述灰度直方图分界拐点，其中，TH为灰度直方图分界拐点，Left为左侧边界点，Point为预设标记点；或者If the left boundary point Left is greater than the preset mark point Point, then according to the formula

Determine the inflection point of the grayscale histogram boundary, where TH is the inflection point of the grayscale histogram boundary, Left is the left boundary point, and Point is a preset marker point; or

If the left boundary point Left is smaller than the preset mark point Point, the inflection point of the gray histogram boundary is determined according to the formula TH=Point, where TH is the inflection point of the gray histogram boundary, and Point is the preset mark point.

Optionally, the determining the left boundary point in the distribution of the grayscale histogram according to the distribution feature of the grayscale histogram includes:

Determine the peak position of the grayscale histogram distribution according to the grayscale histogram distribution feature;

Search in the grayscale histogram distribution in the order of decreasing pixel number from the peak positions of the grayscale histogram distribution;

The number of pixels in each position that is searched is added in turn, until the ratio of the sum of the number of pixels to the sum of the number of all pixels reaches a preset pixel value threshold ratio;

The left position in the grayscale histogram distribution is determined as the left boundary point.

Optionally, the determining the peak position of the grayscale histogram distribution according to the grayscale histogram distribution features includes:

determining a position set consisting of preset positions in the grayscale histogram distribution, wherein the height of the preset position is greater than a preset height threshold;

Obtain the number of pixels at each position in the set of positions;

The position with the largest number of pixels in the position set is determined as the peak position of the grayscale histogram distribution.

Optionally, determining the preset marker points in the grayscale histogram distribution according to the grayscale histogram distribution features, including:

obtaining the change slope at the left boundary point;

If the change slope is less than 1, move the search to the right from the left boundary point in the grayscale histogram distribution, until the change slope at the searched position is greater than or equal to 1;

The position point found in the grayscale histogram distribution is used as the preset marker point; or,

If the change slope is greater than 1, move the search to the left from the left boundary point in the gray histogram distribution, until the change slope at the searched position is less than or equal to 1;

The position point found in the grayscale histogram distribution is used as the preset marker point.

Optionally, the acquiring the change slope at the left boundary point includes:

Obtain the pixel ratio value of the peak position relative to the distribution of the grayscale histogram, the number of left pixels corresponding to the fifth point located to the left of the left boundary point, and the number of pixels located to the right of the left boundary point. The number of pixels on the right side corresponding to the five points;

According to the pixel scale value, the number of pixels on the left and the number of pixels on the right, the change slope is determined by the following formula:

Among them, k is the change slope at the left boundary point, H(l-5) is the number of left pixels corresponding to the fifth point to the left of the left boundary point, and H(l+5) is the left boundary The number of pixels on the right corresponding to the fifth point to the right of the point, ratio is the pixel ratio value of the peak position relative to the distribution of the grayscale histogram.

Optionally, the obtaining the pixel ratio value of the peak position relative to the distribution of the grayscale histogram includes:

Obtain the number of pixels at the peak position;

The ratio of the number of pixels at the peak position to 256 is used as the pixel ratio value of the peak position relative to the grayscale histogram distribution.

Optionally, performing dilation and corrosion processing on the grayscale image includes:

Perform a convolution operation on the image area of the grayscale image and a preset kernel, wherein the kernel includes any one of a square, a rectangle, a diamond, or a hollow circle.

Optionally, the radius of the inner core is 2.

Optionally, the width and height of the sub-image block are both 256.

Optionally, the value range of the pixel value threshold ratio is between 85% and 95%.

A second aspect of the embodiments of the present invention provides an image preprocessing device, including:

A conversion module for converting the pre-acquired original text image into a grayscale image;

an expansion corrosion module, used for performing expansion corrosion processing on the grayscale image to obtain a processed image;

an image division module, configured to divide the processed image into several sub-image blocks of preset sizes;

A determination module, configured to determine the inflection point of the grayscale histogram boundary between the foreground text and the background shading in each of the sub-image blocks, and obtain the pixel value at the inflection point of the grayscale histogram boundary;

A binarization processing module, configured to perform a binarization process on the sub-image block according to the pixel value at the inflection point of the gray histogram boundary to obtain a black and white binary sub-image block corresponding to the sub-image block;

The integration module is used for integrating all the black and white binary sub-image blocks into a binary image corresponding to the grayscale image.

Optionally, the determining module is used for:

Obtaining a grayscale histogram distribution corresponding to the sub-image block according to each of the sub-image blocks;

The distribution characteristics of the grayscale histogram distribution are obtained, and the inflection point of the grayscale histogram boundary between the foreground text and the background shading in each of the sub-image blocks is determined according to the distribution characteristics.

Optionally, the determining module is further used for:

Determine the left boundary point Left and the preset marker point Point in the distribution of the grayscale histogram according to the distribution characteristics of the grayscale histogram;

确定所述灰度直方图分界拐点，其中，TH为灰度直方图分界拐点，Left为左侧边界点，Point为预设标记点；或者If the left boundary point Left is greater than the preset mark point Point, then according to the formula

Determine the inflection point of the grayscale histogram boundary, where TH is the inflection point of the grayscale histogram boundary, Left is the left boundary point, and Point is a preset marker point; or

If the left boundary point Left is smaller than the preset mark point Point, the inflection point of the gray histogram boundary is determined according to the formula TH=Point, where TH is the inflection point of the gray histogram boundary, and Point is the preset mark point.

Optionally, the determining module is further used for:

Determine the peak position of the grayscale histogram distribution according to the grayscale histogram distribution feature;

Search in the grayscale histogram distribution in the order of decreasing pixel number from the peak positions of the grayscale histogram distribution;

The number of pixels in each position that is searched is added in turn, until the ratio of the sum of the number of pixels to the sum of the number of all pixels reaches a preset pixel value threshold ratio;

The left position in the grayscale histogram distribution is determined as the left boundary point.

Optionally, the determining module is further used for:

determining a position set consisting of preset positions in the grayscale histogram distribution, wherein the height of the preset position is greater than a preset height threshold;

Obtain the number of pixels at each position in the set of positions;

The position with the largest number of pixels in the position set is determined as the peak position of the grayscale histogram distribution.

Optionally, the determining module is further used for:

obtaining the change slope at the left boundary point;

If the change slope is less than 1, move the search to the right from the left boundary point in the grayscale histogram distribution, until the change slope at the searched position is greater than or equal to 1;

The position point found in the grayscale histogram distribution is used as the preset marker point; or,

If the change slope is greater than 1, move the search to the left from the left boundary point in the gray histogram distribution, until the change slope at the searched position is less than or equal to 1;

The position point found in the grayscale histogram distribution is used as the preset marker point.

Optionally, the determining module is further used for:

Obtain the pixel ratio value of the peak position relative to the distribution of the grayscale histogram, the number of left pixels corresponding to the fifth point located to the left of the left boundary point, and the number of pixels located to the right of the left boundary point. The number of pixels on the right side corresponding to the five points;

According to the pixel scale value, the number of pixels on the left and the number of pixels on the right, the change slope is determined by the following formula:

Among them, k is the change slope at the left boundary point, H(l-5) is the number of left pixels corresponding to the fifth point to the left of the left boundary point, and H(l+5) is the left boundary The number of pixels on the right corresponding to the fifth point to the right of the point, ratio is the pixel ratio value of the peak position relative to the distribution of the grayscale histogram.

Optionally, the determining module is further used for:

Obtain the number of pixels at the peak position;

The ratio of the number of pixels at the peak position to 256 is used as the pixel ratio value of the peak position relative to the grayscale histogram distribution.

Optionally, the expansion corrosion module is used for:

Perform a convolution operation on the image area of the grayscale image and a preset kernel, wherein the kernel includes any one of a square, a rectangle, a diamond, or a hollow circle.

Optionally, the radius of the inner core is 2.

Optionally, the width and height of the sub-image block are both 256.

Optionally, the value range of the pixel value threshold ratio is between 85% and 95%.

A third aspect of the embodiments of the present invention provides an image preprocessing terminal, including:

memory;

processor; and

Computer program;

Wherein, the computer program is stored in the memory and configured to be executed by the processor to implement the image preprocessing method described in the first aspect above.

A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium on which a computer program is stored;

The computer program is executed by the processor to implement the image preprocessing method described in the first aspect above.

The image preprocessing method, device, terminal, and computer-readable storage medium provided by the embodiments of the present invention perform block processing on the image, and adaptively search for the image binarization threshold according to the distribution characteristics of the histogram of each sub-image block. , which can effectively distinguish the foreground area and the background area, obviously eliminate the influence of factors such as a large number of interference moire fringes, uneven exposure and random noise interference introduced in the process of image shooting, and significantly improve the OCR recognition of text images. The accuracy of the method ensures the practicability of the method and is beneficial to the promotion and application of the market.

Description of drawings

1 is a flowchart of an image preprocessing method provided by an embodiment of the present invention;

2a is a schematic diagram of the effect of a screen photographed image including moiré patterns provided by an embodiment of the present invention;

Fig. 2b is a schematic diagram of the effect of Fig. 2a after preliminary expansion corrosion treatment;

3a is a schematic diagram of the effect of taking a picture of a text document screen provided by an embodiment of the present invention;

Fig. 3b is a schematic diagram of the effect of a sub-image block after being divided into blocks in Fig. 3a;

Figure 3c is a schematic diagram of the effect of the distribution of the grayscale histogram corresponding to Figure 3b;

4 is a schematic diagram of finding a left boundary point in a grayscale histogram distribution according to an embodiment of the present invention;

5 is a schematic diagram of the effect of calculating the change slope k at the left boundary point in the grayscale histogram distribution according to an embodiment of the present invention;

6 is a schematic diagram of the effect of a document image obtained by taking a picture of a paper document with a mobile phone according to an embodiment of the present invention;

Fig. 7 is the result schematic diagram of direct OCR of Fig. 3a;

Fig. 8 is a schematic diagram of the result of the direct OCR of Fig. 6;

Figure 9 is a schematic diagram of the effect of Figure 3a after binarization preprocessing;

FIG. 10 is a schematic diagram of the effect of FIG. 6 after binarization preprocessing;

Fig. 11 is a schematic diagram of the identification result of OCR using Fig. 9;

Figure 12 is a schematic diagram of the identification result of OCR using Figure 10;

13 is a schematic structural diagram of an apparatus for preprocessing an image according to an embodiment of the present invention;

FIG. 14 is a schematic structural diagram of an image preprocessing terminal according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The terms "comprising" and "having" and any variations thereof in the description and claims of the present invention are intended to cover the non-exclusive inclusion, eg, a process or structure comprising a series of steps, not necessarily limited to those expressly listed Instead, those structures or steps may include other steps or structures not expressly listed or inherent to the processes or apparatus.

FIG. 1 is a flowchart of an image preprocessing method provided by an embodiment of the present invention. Referring to FIG. 1, an embodiment of the present invention provides an image preprocessing method, specifically based on the distribution of statistical grayscale histograms. A text image block binarization preprocessing method, which can be performed by an image preprocessing device, includes:

S1: Convert the pre-acquired original text image into a grayscale image I;

Wherein, the pre-acquired original text image may be an image obtained by photographing a mobile device, and the mobile device includes a digital camera, a mobile phone, a handheld computer, and the like.

S2: Perform expansion and corrosion processing on the grayscale image I to obtain the processed image I';

Among them, for the images obtained by taking pictures of electronic documents on the screen, due to the difference between the resolution of cameras in camera phones, digital cameras and other equipment and the screen refresh frequency of computer monitors, the document images after taking pictures on the screen cannot be seen in the background area. A large number of interference fringes are avoided, which seriously affects the recognition of characters. In order to eliminate a large number of interference fringes and at the same time improve the contrast between the foreground and background of the document image, it is necessary to perform the dilation and corrosion operation in the morphological sense for the grayscale image I.

Morphological operations are a series of image processing operations based on shape, which produce an output image by applying structural elements to the input image. The basic morphological operations include erosion and dilation. The dilation and erosion processing of the grayscale image here may include: convolving the image area A of the grayscale image I with a preset kernel B, where the kernel B includes a square , a rectangle, a rhombus or a hollow circle; more preferably, the radius r of the inner core B is set to 2. Note that the kernel has a definable anchor point, usually defined as the kernel center point.

The so-called dilation operation means that the kernel B is drawn across the image, the maximum pixel value of the area covered by the kernel B is extracted, and the pixels at the anchor point are replaced. Obviously this maximization causes the bright areas in the image to start "expanding".

The mathematical definition of the dilation operation for grayscale images is:

g(x,y)=dilate[f(x,y),B]=max{f(x-x',y-y')+B(x',y')|(x',y')∈Db}

Among them, g(x, y) is the expanded grayscale image, f(x,y) is the original grayscale image, and B is the kernel structure element.

The so-called corrosion refers to the extraction of the minimum value of the pixel covered by the kernel B; when performing the corrosion operation, the kernel B is drawn across the image, and the minimum pixel value of the area covered by the kernel B is extracted, and replaces the anchor position pixel, the result of the corrosion operation The bright areas of the picture become thinner and the black areas become larger.

The mathematical definition of the erosion operation for grayscale images is:

g(x,y)=erode[f(x,y),B]=min{f(x+x',y+y')-B(x',y')|(x',y') ∈Db}

Among them, g(x, y) is the grayscale image after corrosion, f(x,y) is the original grayscale image, and B is the kernel structure element.

The image I' is obtained by performing the expansion and erosion operation on the grayscale image I, and its purpose is to reduce the interference of interference fringes or noise while preserving the stroke structure of the font for the text document image after the screen is photographed. For example, Fig. 2a contains a schematic diagram of the effect of a screen photographed image with moiré patterns, and Fig. 2b is the result of the preliminary expansion and corrosion of Fig. 2a. It can be seen from Figure 2b that the texture effect of the moiré pattern of the grayscale image after expansion and corrosion is significantly weakened.

S3: Divide the processed image I' into several sub-image blocks of preset size;

Wherein, the present embodiment does not limit the number and division method of the sub-image blocks into which the processed image I′ is divided, and those skilled in the art can set it according to specific design requirements, wherein, in order to be able to adapt to photos with uneven light and dark For the image, preferably, the processed image I′ after the dilation and erosion is evenly divided into sub-image blocks Ψ of size m*m. It should be noted that when the processed image I' is divided into blocks, if the image blocks are small, the content contained in the sub-image blocks will be too single, and the information of the gray histogram distribution cannot be effectively counted; The sub-image block is too large, resulting in too much content in the sub-image block. For example, there may be a large difference in image brightness and darkness in the same sub-image block, which weakens the statistical characteristics that should appear in the subsequent grayscale histogram distribution. Therefore, the width and height of the sub-image blocks are optimal in the range of 200-350 pixels. In this embodiment, the width and height m of each sub-image block are equal to 256.

S4: Determine the inflection point of the boundary of the grayscale histogram of the foreground text and the background shading in each sub-image block, and obtain the pixel value at the inflection point of the boundary of the grayscale histogram;

The specific implementation process of determining the boundary point of the grayscale histogram between the foreground text and the background shading in each sub-image block may include: obtaining the grayscale histogram distribution corresponding to the sub-image block according to each sub-image block; obtaining the grayscale histogram distribution; According to the distribution characteristics of the histogram distribution, the inflection point of the gray histogram boundary between the foreground text and the background shading in each sub-image block is determined.

In specific applications, statistical analysis can be performed for each sub-image block Ψ, and its grayscale histogram distribution H(Ψ) can be obtained by calculation; for example, Fig. 3a is a schematic diagram of a text document screen photographing, and Fig. 3b is a block diagram of Fig. 3a. A sub-image block Ψ, Fig. 3c is a schematic diagram of the distribution effect of the grayscale histogram corresponding to Fig. 3a. It can be seen from the enlarged effect of Figure 3b that there is serious moiré interference in the background of the image captured by the screen.

Further, determining the inflection point of the gray histogram boundary between the foreground text and the background shading in each sub-image block according to the distribution feature may include:

Determine the left boundary point Left and the preset marker point Point in the grayscale histogram distribution according to the distribution characteristics of the grayscale histogram;

确定灰度直方图分界拐点，其中，TH为灰度直方图分界拐点，Left为左侧边界点，Point为预设标记点；或者If the left boundary point Left is greater than the preset mark point Point, then according to the formula

Determine the inflection point of the gray histogram boundary, where TH is the inflection point of the gray histogram boundary, Left is the left boundary point, and Point is the preset marker point; or

If the left boundary point Left is smaller than the preset marker point Point, the inflection point of the gray histogram boundary is determined according to the formula TH=Point, where TH is the inflection point of the gray histogram boundary, and Point is the preset marker point.

In this embodiment, the threshold inflection point is obtained through a simple histogram traversal search, the calculation method is simple, and the operation efficiency is high.

Further, in the above implementation process, determining the left boundary point in the grayscale histogram distribution according to the grayscale histogram distribution characteristics may include:

Determine the peak position of the gray histogram distribution according to the distribution characteristics of the gray histogram;

Specifically, determining the peak position of the grayscale histogram distribution according to the grayscale histogram distribution feature may include: determining a position set composed of preset positions in the grayscale histogram distribution, wherein the height of the preset position is greater than the preset height Height threshold; get the number of pixels at each location in the location set;

The position with the largest number of pixels in the position set is determined as the peak position of the grayscale histogram distribution; that is, by analyzing the distribution characteristics of the grayscale histogram corresponding to each sub-image block, the peak position Peak of the histogram is found, and the judgment condition is: The grayscale histogram at this position is the highest, that is, the number of pixels H (Peak) at the corresponding Peak is the largest, and the Peak is greater than the preset height threshold T1, T1 is 60, and this point is used as the center point of the statistical histogram.

After determining the peak position, you can search in the grayscale histogram distribution in the order of decreasing pixel number from the peak position of the grayscale histogram distribution; add the number of pixels in each searched position in turn, until the number of pixels reaches The ratio of the sum to the sum of all pixel numbers reaches a preset pixel value threshold ratio; the left position in the grayscale histogram distribution is determined as the left boundary point.

Specifically, from the highest peak position Peak, according to the order of the second largest distribution of gray histograms, for example, as shown in Figure 4, search for the second highest and the third highest in turn; according to the same method, all traversed gray histograms are distributed The number of pixels in the grayscale is added to obtain φ, until the proportion of φ to the total number of pixels contained in the entire grayscale histogram distribution reaches T2; finally, the histogram positions on the left and right sides are respectively recorded as Left and Right, and they are called the central area respectively. The left and right boundary points of , where Left and Right are integers between 0-255. The value range of T2 is between 85% and 95%. In this embodiment, T2 can be set to 92%.

Further, after the left boundary point is determined, the preset marker point may be determined according to the distribution feature of the grayscale histogram. Specifically, the determination of the preset marker point in the grayscale histogram distribution according to the distribution feature of the grayscale histogram may include:

Get the slope of change at the left boundary point;

As shown in Figure 5, l is the current point, l-5 and l+5 are the fifth point in front of l and the fifth point after l respectively; the k value to be calculated is two l-5 and l+5 The slope of the straight line segment formed by the points; that is, the specific steps of obtaining the change slope at the left boundary point may include: obtaining the pixel ratio value of the peak position relative to the distribution of the gray histogram, The number of pixels on the left side corresponding to the five points and the number of pixels on the right side corresponding to the fifth point located to the right of the left boundary point; according to the pixel ratio value, the number of pixels on the left side and the number of pixels on the right side, and determined by the following formula Slope of change:

Among them, k is the change slope at the left boundary point, H(l-5) is the number of left pixels corresponding to the fifth point to the left of the left boundary point, and H(l+5) is the left boundary The number of pixels on the right corresponding to the fifth point to the right of the point, ratio is the pixel ratio value of the peak position relative to the distribution of the grayscale histogram.

Wherein, obtaining the pixel ratio value of the peak position relative to the grayscale histogram distribution may include:

Obtain the number of pixels at the peak position; take the ratio of the number of pixels at the peak position to 256 as the pixel ratio value of the peak position relative to the grayscale histogram distribution. which is:

Among them, ratio is the pixel ratio value of the peak position relative to the gray histogram distribution, and H(peak) is the number of pixels at the peak position.

After the change slope is obtained, the change slope needs to be analyzed and processed. Specifically, the change slope is compared with 1;

If the change slope is less than 1, move the search from the left boundary point to the right in the gray histogram distribution until the change slope at the searched position is greater than or equal to 1; The position point is used as the preset mark point; that is, if k<1, move to the left from Left to search until k≥1, and record the position of this point as Point. or,

If the change slope is greater than 1, move the search from the left boundary point to the left in the grayscale histogram distribution until the change slope at the searched position is less than or equal to 1; The location point is used as the preset marker point. That is: if k>1, move to the left from Left to search until k≤1, and record the position of the point as Point.

S5: binarize the sub-image block according to the pixel value at the inflection point of the gray histogram boundary, to obtain a black and white binary sub-image block corresponding to the sub-image block;

S6: Integrate all the black and white binary sub-image blocks into a binary image corresponding to the grayscale image.

In order to illustrate the effect of the present embodiment, two images are taken as examples to display the effect: Fig. 2a includes a schematic diagram of the effect of a screen photographed image with moiré patterns; Fig. 6 is a schematic diagram of the effect of the document image obtained after the paper document is photographed by a mobile phone, wherein The paper has undergone aggressive manipulations such as manual creasing, tearing, and tea soaking, resulting in significantly uneven exposure and a large number of black spots due to defacement noise. Figure 7 is a schematic diagram of the result of direct OCR in Figure 3a, and Figure 8 is a schematic diagram of the effect of direct OCR in Figure 6. It can be seen that the OCR recognition rate in these two cases is relatively low. Using the binarization preprocessing method of the present invention, FIG. 9 is a schematic diagram of the effect after the binarization preprocessing in FIG. 3 a , and FIG. 10 is a schematic diagram of the effect after the binarization preprocessing in FIG. 6 . Correspondingly, the binarized preprocessed images are re-identified by OCR. FIG. 11 is a schematic diagram of the recognition result of OCR using FIG. 9 , and FIG. 12 is a schematic diagram of the recognition result of OCR performed using FIG. 10 . From the recognition renderings shown in Figures 11 and 12, it can be seen that the recognition rate of OCR is significantly improved.

In the image preprocessing method provided in this embodiment, the image is divided into blocks, and the image binarization threshold is adaptively searched according to the distribution characteristics of the histogram of each sub-image block, so that the foreground area and the background area can be effectively distinguished, The influence of factors such as a large number of interference moire fringes, uneven exposure and random noise interference introduced in the process of image shooting is obviously eliminated, the accuracy of OCR recognition of text images is significantly improved, and the practicability of the method is guaranteed. , which is conducive to the promotion and application of the market.

FIG. 13 is a schematic structural diagram of an image preprocessing apparatus provided by an embodiment of the present invention; with reference to FIG. 13 , this embodiment provides an image preprocessing apparatus, including:

The conversion module 1 is used to convert the pre-acquired original text image into a grayscale image;

The expansion and corrosion module 2 is used to perform expansion and corrosion processing on the grayscale image to obtain the processed image;

The image dividing module 3 is used to divide the processed image into several sub-image blocks of preset size; preferably, the width and height of the sub-image blocks are both 256.

Determining module 4 is used to determine the inflection point of the grayscale histogram boundary between the foreground text and the background shading in each sub-image block, and obtain the pixel value at the inflection point of the grayscale histogram boundary;

The binarization processing module 5 is used to perform binarization processing on the sub-image block according to the pixel value at the inflection point of the gray histogram boundary to obtain a black and white binary sub-image block corresponding to the sub-image block;

The integration module 6 is used to integrate all the black and white binary sub-image blocks into a binary image corresponding to the grayscale image.

Wherein, when the determination module 4 determines the inflection point of the gray histogram boundary between the foreground text and the background shading in each sub-image block, the determination module 4 is specifically used for:

Obtain the grayscale histogram distribution corresponding to the sub-image block according to each sub-image block; obtain the distribution characteristics of the gray-scale histogram distribution, and determine the gray-scale histogram of the foreground text and background shading in each sub-image block according to the distribution characteristics Demarcation point.

Specifically, when the determination module 4 determines the inflection point of the gray histogram boundary between the foreground text and the background shading in each sub-image block according to the distribution characteristics, the determination module 4 is also used for:

Determine the left boundary point Left and the preset marker point Point in the grayscale histogram distribution according to the distribution characteristics of the grayscale histogram;

确定灰度直方图分界拐点，其中，TH为灰度直方图分界拐点，Left为左侧边界点，Point为预设标记点；或者If the left boundary point Left is greater than the preset mark point Point, then according to the formula

Determine the inflection point of the gray histogram boundary, where TH is the inflection point of the gray histogram boundary, Left is the left boundary point, and Point is the preset marker point; or

If the left boundary point Left is smaller than the preset marker point Point, the inflection point of the gray histogram boundary is determined according to the formula TH=Point, where TH is the inflection point of the gray histogram boundary, and Point is the preset marker point.

Further, when the determination module 4 determines the left boundary point Left in the distribution of the grayscale histogram according to the distribution characteristics of the grayscale histogram, the determination module 4 is also used for:

Determine the peak position of the grayscale histogram distribution according to the distribution characteristics of the grayscale histogram; search in the grayscale histogram distribution in the order of decreasing number of pixels from the peak position of the grayscale histogram distribution; The number of pixels is added in turn until the ratio of the sum of the number of pixels to the sum of the total number of pixels reaches the preset pixel value threshold ratio; the left position in the grayscale histogram distribution is determined as the left boundary point; The value range of the value threshold ratio is between 85%-95%.

Wherein, when the determination module 4 determines the peak position of the grayscale histogram distribution according to the distribution characteristics of the grayscale histogram, the determination module 4 is specifically also used for:

Determine a position set composed of preset positions in the grayscale histogram distribution, wherein the height of the preset position is greater than the preset height threshold; obtain the number of pixels in each position in the position set; The position is determined as the peak position of the grayscale histogram distribution.

Further, when the determination module 4 determines the left boundary point Left and the preset marker point Point in the distribution of the grayscale histogram according to the distribution characteristics of the grayscale histogram, the determination module 4 is specifically also used for:

Obtain the change slope at the left boundary point; if the change slope is less than 1, move the search to the right from the left boundary point in the gray histogram distribution until the change slope at the searched position is greater than or equal to 1; The position points found in the grayscale histogram distribution are used as preset markers; or,

If the change slope is greater than 1, move the search from the left boundary point to the left in the grayscale histogram distribution until the change slope at the searched position is less than or equal to 1; The location point is used as the preset marker point.

Further, when the determination module 4 obtains the change slope at the left boundary point, the determination module 4 is also used for:

Obtain the pixel ratio value of the peak position relative to the grayscale histogram distribution, the number of left pixels corresponding to the fifth point to the left of the left boundary point, and the right side corresponding to the fifth point to the right of the left boundary point Number of pixels; the slope of change is determined from the pixel scale value, the number of pixels on the left, and the number of pixels on the right, using the following formula:

Among them, k is the change slope at the left boundary point, H(l-5) is the number of left pixels corresponding to the fifth point to the left of the left boundary point, and H(l+5) is the left boundary The number of pixels on the right corresponding to the fifth point to the right of the point, ratio is the pixel ratio value of the peak position relative to the distribution of the grayscale histogram.

Further, the determining module 4 is further used for: acquiring the number of pixels at the peak position; and taking the ratio of the number of pixels at the peak position to 256 as the pixel ratio value of the peak position relative to the grayscale histogram distribution.

Optionally, the dilation and corrosion module 2 is further configured to perform a convolution operation on the image area of the grayscale image and a preset kernel, wherein the kernel includes any one of a square, a rectangle, a diamond, or a hollow circle. Preferably, the radius of the inner core is 2.

The image preprocessing apparatus provided in this embodiment can be used to execute the method of the embodiment in FIG. 1 , and the specific execution manner and beneficial effects thereof are similar, which will not be repeated here.

The embodiment of the present invention also provides an image preprocessing terminal, including:

memory;

processor; and

Computer program;

Wherein, the computer program is stored in the memory and configured to be executed by the processor to realize the above-mentioned image preprocessing method.

Specifically, FIG. 14 is a schematic structural diagram of an image preprocessing terminal according to an embodiment of the present invention.

As shown, image preprocessing terminal 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, Sensor assembly 814 , and communication assembly 816 .

The processing component 802 generally controls the overall operation of the terminal 800 for image preprocessing, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to perform all or some of the steps of the methods described above. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

Memory 804 is configured to store various types of data to support operation of terminal 800 in image preprocessing. Examples of such data include instructions for any application or method operating on the terminal 800 for preprocessing images, contact data, phonebook data, messages, pictures, videos, and the like. Memory 804 may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power supply component 806 provides power to various components of image preprocessing terminal 800 . Power supply components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to preprocessing terminal 800 for images.

The multimedia component 808 includes a screen that provides an output interface between the image preprocessing terminal 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. A touch sensor can sense not only the boundaries of a touch or swipe action, but also the duration and pressure associated with the touch or swipe action.

Audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the image preprocessing terminal 800 is in operating modes, such as call mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 804 or transmitted via communication component 816 . In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing status assessment of various aspects to terminal 800 for image preprocessing. For example, the sensor component 814 can detect the open/closed state of the image preprocessing terminal 800, the relative positioning of components, such as the display and keypad of the image preprocessing terminal 800, the sensor component 814 can also detect the preprocessing of the image Change of position of terminal 800 or a component of image preprocessing terminal 800, presence or absence of user contact with image preprocessing terminal 800, image preprocessing terminal 800 orientation or acceleration/deceleration and temperature of image preprocessing terminal 800 Variety. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 814 may also include a camera assembly, which may employ, for example, a CMOS or CCD image sensor for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the image processing terminal 800 and other devices. The image preprocessing terminal 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, the image preprocessing terminal 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs) , Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation for performing the above method.

A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium on which a computer program is stored; the computer program is executed by a processor to implement an image preprocessing method according to the first aspect.

Finally, it should be noted that those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it may include the flow of the embodiments of the above-mentioned methods. The storage medium may be a magnetic disk, an optical disk, a read only memory (ROM) or a random access memory (RAM), and the like.

Each functional unit in this embodiment of the present invention may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disk, or the like.

The above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that the foregoing embodiments can still be used for The technical solutions described in the examples are modified, or some or all of the technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (18)

1. a preprocessing method of image, is characterized in that, comprises: Convert pre-acquired raw text images to grayscale images; performing dilation and corrosion processing on the grayscale image to obtain a processed image; dividing the processed image into several sub-image blocks of preset size; Determine the inflection point of the grayscale histogram boundary between the foreground text and the background shading in each of the sub-image blocks, and obtain the pixel value at the boundary inflection point of the grayscale histogram; Perform binarization processing on the sub-image block according to the pixel value at the inflection point of the boundary of the grayscale histogram to obtain a black and white binary sub-image block corresponding to the sub-image block; integrating all black and white binary sub-image blocks into a binary image corresponding to the grayscale image; Wherein, the determining the inflection point of the gray histogram boundary between the foreground text and the background shading in each of the sub-image blocks includes: Obtaining a grayscale histogram distribution corresponding to the sub-image block according to each of the sub-image blocks; Obtaining the distribution features of the grayscale histogram distribution, and determining the inflection point of the grayscale histogram boundary between the foreground text and the background shading in each of the sub-image blocks according to the distribution features; Wherein, determining the inflection point of the gray histogram boundary between the foreground text and the background shading in each of the sub-image blocks according to the distribution features includes: Determine the left boundary point Left and the preset marker point Point in the distribution of the grayscale histogram according to the distribution characteristics of the grayscale histogram; If the left boundary point Left is greater than the preset mark point Point, then according to the formula

Determine the inflection point of the grayscale histogram boundary, where TH is the inflection point of the grayscale histogram boundary, Left is the left boundary point, and Point is a preset marker point; or If the left boundary point Left is smaller than the preset mark point Point, then determine the inflection point of the gray histogram boundary according to the formula TH=Point, where TH is the inflection point of the gray histogram boundary, and Point is the preset mark point; Wherein, determining the left boundary point in the distribution of the grayscale histogram according to the distribution characteristics of the grayscale histogram includes: Determine the peak position of the grayscale histogram distribution according to the grayscale histogram distribution feature; Search in the grayscale histogram distribution in the order of decreasing pixel number from the peak positions of the grayscale histogram distribution; The number of pixels in each position that is searched is added in turn, until the ratio of the sum of the number of pixels to the sum of the number of all pixels reaches a preset pixel value threshold ratio; Determining the left position in the distribution of the grayscale histogram as the left boundary point; Wherein, determining the preset marker points in the distribution of the grayscale histogram according to the distribution characteristics of the grayscale histogram includes: obtaining the change slope at the left boundary point; If the change slope is less than 1, move the search to the right from the left boundary point in the grayscale histogram distribution, until the change slope at the searched position is greater than or equal to 1; The position point found in the grayscale histogram distribution is used as the preset marker point; or, If the change slope is greater than 1, move the search to the left from the left boundary point in the gray histogram distribution, until the change slope at the searched position is less than or equal to 1; The position point found in the grayscale histogram distribution is used as the preset marker point. 2 . The method according to claim 1 , wherein, determining the peak position of the grayscale histogram distribution according to the grayscale histogram distribution characteristics, comprising: 2 . determining a position set consisting of preset positions in the grayscale histogram distribution, wherein the height of the preset position is greater than a preset height threshold; Obtain the number of pixels at each position in the set of positions; The position with the largest number of pixels in the position set is determined as the peak position of the grayscale histogram distribution. 3. The method according to claim 1, wherein the acquiring the change slope at the left boundary point comprises: Obtain the pixel ratio value of the peak position relative to the distribution of the grayscale histogram, the number of left pixels corresponding to the fifth point located to the left of the left boundary point, and the fifth point located to the right of the left boundary point. The number of pixels on the right side corresponding to the point; According to the pixel scale value, the number of pixels on the left and the number of pixels on the right, the change slope is determined by the following formula:

Among them, k is the change slope at the left boundary point, H(l-5) is the number of left pixels corresponding to the fifth point to the left of the left boundary point, and H(l+5) is the left boundary The number of pixels on the right corresponding to the fifth point to the right of the point, ratio is the pixel ratio value of the peak position relative to the distribution of the grayscale histogram. 4. The method according to claim 3, wherein the obtaining the pixel ratio value of the peak position relative to the distribution of the grayscale histogram comprises: Obtain the number of pixels at the peak position; The ratio of the number of pixels at the peak position to 256 is used as the pixel ratio value of the peak position relative to the grayscale histogram distribution. 5. The method according to any one of claims 1-4, wherein the performing dilation and corrosion processing on the grayscale image comprises: Perform a convolution operation on the image area of the grayscale image and a preset kernel, wherein the kernel includes any one of a square, a rectangle, a diamond, or a hollow circle. 6. The method of claim 5, wherein the inner core has a radius of 2. 7 . The method according to claim 1 , wherein the width and height of the sub-image block are both 256. 8 . 8 . The method according to claim 2 , wherein the value range of the pixel value threshold ratio is between 85% and 95%. 9 . 9. An image preprocessing device, comprising: A conversion module for converting the pre-acquired original text image into a grayscale image; an expansion corrosion module, used for performing expansion corrosion processing on the grayscale image to obtain a processed image; an image division module, configured to divide the processed image into several sub-image blocks of preset sizes; A determination module, configured to determine the inflection point of the grayscale histogram boundary between the foreground text and the background shading in each of the sub-image blocks, and obtain the pixel value at the inflection point of the grayscale histogram boundary; A binarization processing module, configured to perform a binarization process on the sub-image block according to the pixel value at the inflection point of the gray histogram boundary to obtain a black and white binary sub-image block corresponding to the sub-image block; an integration module for integrating all black and white binary sub-image blocks into a binary image corresponding to the grayscale image; Wherein, the determining module is used for: Obtaining a grayscale histogram distribution corresponding to the sub-image block according to each of the sub-image blocks; Obtaining the distribution features of the grayscale histogram distribution, and determining the inflection point of the grayscale histogram boundary between the foreground text and the background shading in each of the sub-image blocks according to the distribution features; The determining module is also used for: Determine the left boundary point Left and the preset marker point Point in the distribution of the grayscale histogram according to the distribution characteristics of the grayscale histogram; If the left boundary point Left is greater than the preset mark point Point, then according to the formula

Determine the inflection point of the grayscale histogram boundary, where TH is the inflection point of the grayscale histogram boundary, Left is the left boundary point, and Point is a preset marker point; or If the left boundary point Left is smaller than the preset mark point Point, then determine the inflection point of the gray histogram boundary according to the formula TH=Point, where TH is the inflection point of the gray histogram boundary, and Point is the preset mark point; The determining module is also used for: Determine the peak position of the grayscale histogram distribution according to the grayscale histogram distribution feature; Search in the grayscale histogram distribution in the order of decreasing pixel number from the peak positions of the grayscale histogram distribution; The pixel numbers of the searched positions are added in turn, until the ratio of the sum of the pixel numbers to the sum of all the pixel numbers reaches a preset pixel value threshold ratio; Determining the left position in the distribution of the grayscale histogram as the left boundary point; The determining module is also used for: obtaining the change slope at the left boundary point; If the change slope is less than 1, move the search to the right from the left boundary point in the grayscale histogram distribution, until the change slope at the searched position is greater than or equal to 1; The position point found in the grayscale histogram distribution is used as the preset marker point; or, If the change slope is greater than 1, move the search to the left from the left boundary point in the gray histogram distribution, until the change slope at the searched position is less than or equal to 1; The position point found in the grayscale histogram distribution is used as the preset marker point. 10. The apparatus according to claim 9, wherein the determining module is further configured to: determining a position set consisting of preset positions in the grayscale histogram distribution, wherein the height of the preset position is greater than a preset height threshold; Obtain the number of pixels at each position in the set of positions; The position with the largest number of pixels in the position set is determined as the peak position of the grayscale histogram distribution. 11. The apparatus according to claim 9, wherein the determining module is further configured to: Obtain the pixel ratio value of the peak position relative to the distribution of the grayscale histogram, the number of left pixels corresponding to the fifth point located to the left of the left boundary point, and the number of pixels located to the right of the left boundary point. The number of pixels on the right side corresponding to the five points; According to the pixel scale value, the number of pixels on the left and the number of pixels on the right, the change slope is determined by the following formula:

Among them, k is the change slope at the left boundary point, H(l-5) is the number of left pixels corresponding to the fifth point to the left of the left boundary point, and H(l+5) is the left boundary The number of pixels on the right corresponding to the fifth point to the right of the point, ratio is the pixel ratio value of the peak position relative to the distribution of the grayscale histogram. 12. The apparatus according to claim 11, wherein the determining module is further configured to: Obtain the number of pixels at the peak position; The ratio of the number of pixels at the peak position to 256 is used as the pixel ratio value of the peak position relative to the grayscale histogram distribution. 13. The device according to any one of claims 9-12, wherein the expansion corrosion module is used for: Perform a convolution operation on the image area of the grayscale image and a preset kernel, wherein the kernel includes any one of a square, a rectangle, a diamond, or a hollow circle. 14. The apparatus of claim 13, wherein the inner core has a radius of 2. 15 . The apparatus according to claim 9 , wherein the width and height of the sub-image blocks are both 256. 16 . 16 . The device according to claim 10 , wherein the value range of the pixel value threshold ratio is between 85% and 95%. 17 . 17. An image preprocessing terminal, comprising: memory; processor; and Computer program; Wherein, the computer program is stored in the memory and configured to be executed by the processor to implement an image preprocessing method according to any one of claims 1-8. 18. A computer-readable storage medium, characterized in that a computer program is stored thereon; The computer program is executed by the processor to implement an image preprocessing method according to any one of claims 1-8.

Images