CN113255412B

CN113255412B - Document image processing method, device and medium

Info

Publication number: CN113255412B
Application number: CN202010091495.9A
Authority: CN
Inventors: 彭鑫; 刘坚强; 吴鹏杰
Original assignee: Beijing Xiaomi Pinecone Electronic Co Ltd
Current assignee: Beijing Xiaomi Pinecone Electronic Co Ltd
Priority date: 2020-02-13
Filing date: 2020-02-13
Publication date: 2023-11-24
Anticipated expiration: 2040-02-13
Also published as: CN113255412A

Abstract

The disclosure relates to a document image processing method, device and medium. The processing method comprises the following steps: dividing a document image to be processed into a plurality of blocks according to a preset pixel size, and calculating an incident light color value of each block according to brightness values of pixel points in the blocks; determining a foreground color region, determining color values of pixel points corresponding to the foreground color region, and correcting the incident light color values of corresponding blocks according to the color values of the pixel points corresponding to the foreground color region; calculating the average brightness value of each pixel point corresponding to the foreground color area and the average brightness value of each pixel point corresponding to the background color area based on the foreground color area, and determining whether the background of the document image to be processed is a pure white background or not based on the average brightness value of each pixel point corresponding to the foreground color area and the average brightness value of each pixel point corresponding to the background color area; and processing the document image to be processed based on a determination result of whether the background of the document image to be processed is a plain white background. The processing method of the document image can accurately position the foreground color area and effectively avoid the phenomenon of background color loss.

Description

Document image processing method, device and medium

Technical Field

The present disclosure relates to the field of computer vision, and in particular, to a method, an apparatus, and a medium for processing document images.

Background

In the related art, with the rapid development of portable photographing devices, such as smart phones, people can obtain digitized text of documents by photographing the documents for recording and sharing related information. Compared with the traditional scanner, the digital document is easy to use and easy to operate by shooting through a mobile phone, but the quality of the digital document obtained is not high due to the complex and changeable shooting environment, so that the processing of high-quality document images is a problem to be solved.

Disclosure of Invention

In order to overcome the problems in the related art, the present disclosure provides a method, apparatus, and medium for processing a document image.

According to a first aspect of an embodiment of the present disclosure, there is provided a method for processing a document image, including:

dividing a document image to be processed into a plurality of blocks according to a preset pixel size, and calculating an incident light color value of each block according to brightness values of all pixel points in the blocks;

determining a foreground color region, determining color values of pixel points corresponding to the foreground color region, and correcting the incident light color values of corresponding blocks according to the color values of the pixel points corresponding to the foreground color region;

Calculating the average brightness value of each pixel point corresponding to the foreground color area and the average brightness value of each pixel point corresponding to the background color area based on the foreground color area, and determining whether the background of the document image to be processed is a pure white background or not based on the average brightness value of each pixel point corresponding to the foreground color area and the average brightness value of each pixel point corresponding to the background color area;

and processing the document image to be processed based on a determination result of whether the background of the document image to be processed is a plain white background.

Wherein, calculating the incident light color value of each block according to the color value of each pixel point in the block comprises:

and selecting an average value of the color values of the pixel points with the brightness values of the previous preset percentage as the incident light color value of each block in the descending order.

The determining the foreground color area, determining the color value of each pixel point corresponding to the foreground color area, and correcting the incident light color value of the corresponding block according to the color value of each pixel point corresponding to the foreground color area includes:

performing binarization processing on the document image to be processed to generate a binarized image;

obtaining the background color area based on pixel points corresponding to the white area in the binarized image; determining a foreground color area of the document image to be processed based on the background color area; the method comprises the steps of carrying out a first treatment on the surface of the

And generating a mask area according to the determined foreground color area, correcting the color value of the pixel point corresponding to the mask area in each block by using a preset algorithm, and recalculating the incident light color value of each block.

The obtaining the background color region based on the pixel points corresponding to the white region in the binarized image includes:

processing pixel points corresponding to a white region in the binary image in the following manner to obtain a background color region:

mode one: performing plane fitting and straight line fitting on pixel points corresponding to a white area in the binarized image in an LAB color space to obtain non-outliers; clustering the non-outlier points to obtain a plurality of pixel point clusters; taking the area corresponding to the pixel point in the pixel point cluster with the maximum average brightness value as a background color area;

mode two: clustering pixel points corresponding to a white area in the binarized image to obtain a plurality of pixel point clusters; performing plane fitting and straight line fitting on pixel points in the pixel point cluster with the maximum average brightness value in an LAB color space, and taking a region corresponding to the non-outlier pixel points obtained by fitting as a background color region;

mode three: performing plane fitting and straight line fitting on pixel points corresponding to a white area in a binarized image in an LAB color space to obtain non-outlier points, and taking the area corresponding to the obtained non-outlier points as a background color area;

Mode four: clustering pixel points corresponding to a white area in the binarized image to obtain a plurality of pixel point clusters; and taking the area corresponding to the pixel point in the pixel point cluster with the maximum average brightness value as a background color area.

Wherein, still include:

before the mask region is generated by using the determined foreground color region, the method further comprises:

and performing morphological closing operation processing on the determined foreground color region.

Wherein the determining whether the background of the document image to be processed is a pure white background based on the average brightness value of each pixel point corresponding to the foreground color region and the average brightness value of each pixel point corresponding to the background color region comprises:

and determining whether the background of the document image to be processed is a pure white background or not based on the average brightness value of the pixel points corresponding to the background color area and the difference value between the average brightness value of the pixel points corresponding to the foreground color area and the average brightness value of the pixel points corresponding to the background color area.

Wherein the determining whether the background of the document image to be processed is a pure white background based on the average brightness value of the pixel corresponding to the background color region and the difference value between the average brightness value of the pixel corresponding to the foreground color region and the average brightness value of the pixel corresponding to the background color region comprises:

And when the average brightness value of the pixel points corresponding to the background color area is larger than a first threshold value and the difference value between the average brightness value of the pixel points corresponding to the foreground color area and the average brightness value of the pixel points corresponding to the background color area is larger than a second threshold value, determining that the background of the document image to be processed is a pure white background.

The method for processing the document image to be processed based on the determination result of whether the background of the document image to be processed is a pure white background comprises the following steps:

when the background of the document image to be processed is a pure white background, performing white balance processing on the document image to be processed based on the color values of all pixel points of the document image to be processed and the corrected incident light color values of the corresponding blocks;

acquiring a gray level histogram of the document image to be processed after the white balance processing, and performing brightness contrast balance processing on the image to be processed after the white balance processing based on brightness values of all pixel points in the gray level histogram;

when the background of the document image to be processed is a non-pure white background, acquiring a gray level histogram of the document image to be processed, and carrying out brightness contrast balance processing on the image to be processed based on brightness values of all pixel points in the gray level histogram.

The white balance processing of the document image to be processed based on the color values of the pixels of the document image to be processed and the corrected incident light color values of the corresponding blocks comprises the following steps:

when the ratio of the color value of the pixel point to the corrected incident light color value of the block where the pixel point is located is greater than 1, 255 is used as the color value of the pixel point after white balance processing; when the ratio of the color value of the pixel point to the corrected incident light color value of the block where the pixel point is located is smaller than 1, taking the product of the ratio and 255 as the color value of the pixel point after white balance processing.

The performing brightness contrast balance processing on the image to be processed after the white balance processing based on the brightness value of each pixel point in the gray histogram or performing brightness contrast balance processing on the image to be processed based on the brightness value of each pixel point in the gray histogram includes:

determining the minimum non-0 brightness value and the maximum non-255 brightness value in the gray level histogram;

and adjusting the color value of each pixel point in the image to be processed based on the adjustment parameter according to the minimum non-0 brightness value and the maximum non-255 brightness value.

Wherein the adjustment parameters comprise a first adjustment parameter alpha and a second adjustment parameter beta, wherein,

β＝-V _min *α

Wherein V is _max Represents the maximum non-255 luminance value, V _min Representing a minimum non-0 luminance value;

the adjusting the color value of each pixel point in the image to be processed based on the adjustment parameter according to the minimum non-0 brightness value and the maximum non-255 brightness value comprises:

the color value of each pixel point in the image to be processed is adjusted based on the following algorithm:

C _out ＝α*C _input +β

wherein C is _out Representing the adjusted color values of each pixel point in the image to be processed, C _input Representing the color value before adjustment of each pixel point in the image to be processed.

Wherein the processing method further comprises:

filtering the document image to be processed.

According to a second aspect of the embodiments of the present disclosure, there is provided a processing apparatus of a document image, including:

the computing module is configured to divide the document image to be processed into a plurality of blocks according to a preset pixel size, and compute the incident light color value of each block according to the brightness value of each pixel point in the block;

the correction module is configured to determine a foreground color region, determine color values of pixel points corresponding to the foreground color region, and correct the incident light color value of a corresponding block according to the color values of the pixel points corresponding to the foreground color region;

a determining module configured to calculate, based on the foreground color region, an average luminance value of each pixel point corresponding to the foreground color region and an average luminance value of each pixel point corresponding to the background color region, and determine, based on the average luminance value of each pixel point corresponding to the foreground color region and the average luminance value of each pixel point corresponding to the background color region, whether a background of a document image to be processed is a pure white background;

And a processing module configured to process the document image to be processed based on a result of the determination of whether the background of the document image to be processed is a plain white background.

Wherein the computing module is configured to:

Wherein the correction module is configured to:

Wherein the correction module is further configured to:

Wherein the determination module is configured to:

Wherein the processing module is configured to:

β＝-V _min *α

the processing module is configured to:

C _out ＝α*C _input +β

Wherein the processing device further comprises:

and the filtering module is configured to filter the document image to be processed.

According to a third aspect of the embodiments of the present disclosure, there is provided a processing apparatus of a document image, including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

dividing a document image to be processed into a plurality of blocks according to a preset pixel size, and calculating an incident light color value of each block according to the color value of each pixel point in the block;

According to a fourth aspect of embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium, which when executed by a processor of a mobile terminal, causes the mobile terminal to perform a method of processing a document image, the method comprising:

The technical scheme provided by the embodiment of the disclosure can comprise the following beneficial effects: the processing method of the document image can accurately position the foreground color area and effectively avoid the phenomenon of background color loss.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is an exemplary diagram of a document image, shown according to an exemplary embodiment;

FIG. 2 is a flowchart illustrating a document image processing method according to an exemplary embodiment;

fig. 3 is a flowchart of the method shown in fig. 1 for calculating the color value of the incident light of each tile according to the color values of the pixel points in the tile in step S11;

fig. 4 is a flowchart of a method for determining a foreground color region and determining color values of pixels corresponding to the foreground color region and correcting the color values of incident light of a corresponding block according to the color values of pixels corresponding to the foreground color region in step S12 shown in fig. 1;

FIG. 5 is a flowchart illustrating a document image processing method according to an exemplary embodiment;

FIG. 6 is a schematic diagram of a document image to be processed, according to an example embodiment;

fig. 7 is a flowchart of a method for determining whether the background of the document image to be processed is a pure white background based on the average luminance value of each pixel corresponding to the foreground region and the average luminance value of each pixel corresponding to the background region in step S13 shown in fig. 1 based on the foreground region;

FIG. 8 is a flowchart of a method for determining whether the background of the document image to be processed is a pure white background in step 131 of FIG. 7 based on the average luminance value of the pixels corresponding to the background color region and the difference between the average luminance value of the pixels corresponding to the foreground color region and the average luminance value of the pixels corresponding to the background color region;

FIG. 9 is a flowchart of a method of processing the document image to be processed based on the determination result of step 14 in FIG. 1 as to whether the background of the document image to be processed is a solid white background;

FIG. 10 is a flowchart of a method for performing white balance processing on a document image to be processed based on the color values of each pixel point of the document image to be processed and the corrected incident light color values of the corresponding block when the background is a solid white background in step 141 of FIG. 9;

Fig. 11 is a gray-scale histogram of the document image to be processed after the white balance processing is obtained in step 141 in fig. 9, and luminance-contrast balance processing is performed on the image to be processed after the white balance processing based on the luminance values of the respective pixel points in the gray-scale histogram. A flow chart of a method for performing brightness contrast balance processing on the image to be processed after the white balance processing based on the brightness value of each pixel point in the gray level histogram;

FIG. 12 is a flowchart illustrating a document image processing method according to an exemplary embodiment;

FIG. 13 is a block diagram of a document image processing apparatus according to an exemplary embodiment;

fig. 14 is a block diagram (general structure of a mobile terminal) of a document image processing apparatus according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the invention. Rather, they are merely examples of apparatus and methods consistent with aspects of the invention as detailed in the accompanying claims.

The document itself includes a background color and a foreground color, both of which have a fixed color. For any one color, different color values and different brightness values can be displayed under the action of incident light with different brightness and different colors. Therefore, after the document is photographed and a document image is formed, the color values of the background color and the foreground color in the document image are different from the color values of the foreground color and the background color of the document itself. In the same document image, the same background or foreground can also present different colors and have different color values due to the influence of illumination or shadow.

When a document image is shot, even if a document with a pure white background is shot, in the shot image, the white background part is not pure white, and the black character part is not pure black, so that the background of the document image is uneven, and even the contrast between characters serving as a foreground and the background is not high, and the visual effect is affected. The main reason is the influence of the color value of the incident light, and therefore, the color value of each pixel point in the document image is related to the background color value, the foreground color value, and the incident light color value. Fig. 1 is an exemplary view of a document image according to an exemplary embodiment, and referring to fig. 1, taking a pixel point in a rectangular frame 101 in the drawing as an example, the color of a paper is pure white, and under ideal illumination conditions, the color value of the paper should be 255, expressed as color_paper, where color_paper=255; the incident light color value is assumed to be 200, expressed as color_light, color_light=200; within the rectangular frame 101, the color value of the pixel point= [ (color_paper/255) ] = [ (255/255) ((200/255) ] = (255/255) ] = 200); similarly, for the pixel corresponding to the text 102 of the foreground, the color_foregap is expressed, and it is assumed that the color value of the ink is 40, color_foregap=40, color_light=200, and the color value of the pixel of the foreground of the document image= [ (color_foregap/255) ] = [ (40/255) = [ (200/255) ] = (31.3); from the above calculation, the document image is darkened as a whole due to the influence of the color value of the incident light, affecting the visual effect. Meanwhile, it can be known that the color value of the background of a document of a solid background in a document image can be regarded as an incident light color value.

Similarly, in the same document image, the color value of the background can be used as a constant in the document image, and an enhanced image with uniform background and clear prospect can be obtained by only calculating the color value of the incident light corresponding to each pixel point and eliminating the influence of the incident light. The present disclosure thus provides a method of processing a document image.

Fig. 2 is a flowchart illustrating a document image processing method according to an exemplary embodiment, which is used in a terminal as shown in fig. 2, including the following steps.

In step S11, dividing the document image to be processed into a plurality of blocks according to a preset pixel size, and calculating an incident light color value of each block according to the color value of each pixel point in the block;

in step S12, a foreground color area is determined, color values of pixels corresponding to the foreground color area are determined, and the color values of incident light of corresponding blocks are corrected according to the color values of the pixels;

in step S13, based on the foreground color region, calculating an average luminance value of each pixel point corresponding to the foreground color region and an average luminance value of each pixel point corresponding to the background color region, and determining whether the background of the document image to be processed is a pure white background based on the average luminance value of each pixel point corresponding to the foreground color region and the average luminance value of each pixel point corresponding to the background color region;

In step S14, the document image to be processed is processed based on the determination result of whether the background of the document image to be processed is a plain white background.

According to the processing method of the document image, the foreground color area is accurately positioned through calculation and correction of the incident light color value of the document image, and the processing mode of the document image with the non-pure white background is different from that of the document image with the pure white background through judgment of whether the document image with the non-pure white background is the pure white background, so that the phenomenon of losing the background color can be effectively avoided.

In step S11, the document image is partitioned into blocks, and the color values of the incident light of the blocks are used as the color values of the incident light of the pixel points in the blocks, so that the light and shadow in the blocks can be balanced, and the processed image background is more uniform. The size of the block is determined according to a preset pixel size, and the preset pixel size can be a fixed pixel size, for example, 50x50 pixels, or can be determined according to the size of characters in the document image, for example, each character occupies 15x15 pixels, and then the document image to be processed can be partitioned by using the pixel size of 15x 15. In some cases, one pixel may also be used as one block.

The embodiment of the disclosure provides a processing method of a document image. As shown in fig. 3, in step S11, calculating the incident light color value of each tile from the color values of the pixel points within the tile includes:

in step S111, an average value of the color values of the pixel points with the previous predetermined percentage of the color values in each block is selected as the incident light color value of the block.

In one block, the entire area may be the background of the document image, and the entire area may be the foreground of the document image, i.e., the text and the picture in the document picture, or one block may include both the foreground and the background of the document image. In this embodiment, the luminance values of all the pixels in the block are counted, and arranged in descending order, and the luminance value is taken as a predetermined percentage, for example, the average value of the color values of the pixels with the luminance value of 25% before the luminance value can be taken as the incident light color value of the current block. For the blocks with all areas being the background of the document image, the incident light color value of each block calculated by the method is close to the incident light color value under the real condition, but for the blocks with all areas being the foreground of the document image or the area occupied by the background color area in the blocks is less than 25%, the incident light color value calculated by the method is not the real incident light color value. Therefore, the color value of the incident light of the part of the block needs to be corrected.

The embodiment of the disclosure provides a processing method of a document image. As shown in fig. 4, step S12 of determining a foreground color region, determining color values of each pixel point corresponding to the foreground color region, and correcting the incident light color value of the corresponding block according to the color values of each pixel point corresponding to the foreground color region includes:

step S121, binarizing the document image to be processed to generate a binarized image.

In the binarized image, pixels corresponding to the foreground will be displayed as black and pixels corresponding to the background will be displayed as white. In this step, a preliminary distinction is made between the foreground color region and the background color region. In order to accurately distinguish between the foreground color region and the background color region. And further processing the primarily distinguished background color areas.

Step S122, obtaining a background color area based on pixel points corresponding to a white area in the binarized image; and determining a foreground color area of the document image to be processed based on the background color area.

As described above, in order to accurately distinguish between the foreground color region and the background color region of the document image to be processed, the preliminarily distinguished background color region is further processed, for example, by fitting and/or clustering, to obtain the background color region. Thus, for the document image to be processed, the rest is the foreground color area except the obtained background color area, namely, the white area except the background color area in the binary image, and the rest area plus the part corresponding to the black area in the binary image are all the foreground color area of the image to be processed.

Step S123, generating a mask area according to the determined foreground color area, correcting the color value of the pixel point corresponding to the mask area in each block by using a preset algorithm, and recalculating the incident light color value of each block.

Through the generated mask region, the mask region covers the whole foreground color region, when the color correction of the pixel points of the foreground color region is carried out, the pixel points in the region corresponding to the mask region can be repaired, and if a certain block is a block corresponding to the background color region, the block cannot be selected in the process, so that the repetitive work is avoided. Here, the preset algorithm may be any algorithm for image correction, which is not limited herein.

In the present disclosure, a pixel point corresponding to a white region in a binary image may be processed in the following manner to obtain a background color region:

mode one: performing plane fitting and straight line fitting on pixel points corresponding to a white area in a binarized image in an LAB Color Space (LAB Color Space) to obtain non-outliers; clustering the non-outlier points to obtain a plurality of pixel point clusters; taking the area corresponding to the pixel point in the pixel point cluster with the maximum average brightness value as a background color area;

In the disclosure, a given algorithm, such as a ransac algorithm, is used to perform straight line and plane fitting on pixel points corresponding to a white region in a binary image in an LAB color space, so as to obtain outlier pixel points and non-outlier pixel points. The area corresponding to the outlier pixel points is a foreground color area, and the area corresponding to the non-outlier pixel points is a background color area. In order to further accurately locate the background color area, further processing is carried out on the non-outlier pixel points, namely, clustering is carried out on the non-outlier pixel points according to a preset algorithm, such as calculation of Euclidean distance, so as to obtain a plurality of pixel point clusters; and taking the area corresponding to the pixel point in the pixel point cluster with the maximum average brightness value as a background color area. By the method, the background color area of the document image to be processed can be accurately positioned, and the area outside the background color area is the foreground color area of the document image to be processed.

In the present disclosure, according to a predetermined algorithm, for example, calculating a euclidean distance, clustering pixel points corresponding to a white region in a binary image to obtain a plurality of pixel point clusters; the area corresponding to the pixel point in the pixel point cluster with the maximum average brightness value is a background color area. In order to further accurately locate the background color region, a given algorithm, such as a ransac algorithm, is used for pixel points in the pixel point cluster with the maximum average brightness value, straight lines and planes are fitted in the LAB color space, and the region corresponding to the non-outlier pixel points obtained by fitting is used as the background color region. By the method, the background color area of the document image to be processed can be accurately positioned, and the area outside the background color area is the foreground color area of the document image to be processed.

in the disclosure, a given algorithm, such as a ransac algorithm, is used to perform straight line and plane fitting on pixel points corresponding to a white region in a binary image in an LAB color space, so as to obtain outlier pixel points and non-outlier pixel points. The area corresponding to the outlier pixel points is a foreground color area, and the area corresponding to the non-outlier pixel points is a background color area. In this way, the background color area of the document image to be processed can be accurately positioned, and the area outside the background color area is the foreground color area of the document image to be processed.

In the present disclosure, according to a predetermined algorithm, for example, calculating a euclidean distance, clustering pixel points corresponding to a white region in a binary image to obtain a plurality of pixel point clusters; the area corresponding to the pixel point in the pixel point cluster with the maximum average brightness value is a background color area. In this way, the background color area of the document image to be processed can be accurately positioned, and the area outside the background color area is the foreground color area of the document image to be processed.

The embodiment of the disclosure provides a processing method of a document image. As shown in fig. 5, before generating the mask region using the determined foreground color region, the method further includes step S125 of performing morphological closing operation on the determined foreground color region. In practical applications, the foreground color region includes not only a text region but also a picture region, if the foreground color region includes a picture region, the picture region may include colors with multiple brightness, and even may include a pure white region, so that in order to ensure that a portion of the picture, which is close to the color of the background color region, is correctly marked as the foreground color, morphological closing operation needs to be performed on the foreground color region before a mask region is generated. For example, in the document image to be processed shown in fig. 6, the area framed by the rectangular frame 100 is determined as the foreground color area, but some small areas 125 in the rectangular frame 100 are not determined as the foreground color area, and in order to correctly label these small areas 125 as the foreground color area, morphological closing operation may be used to perform processing, and label these small areas 125 as the foreground color area. The morphological closing operation can be selected from the traditional morphological closing operation image processing method, and the detailed description is omitted here.

The embodiment of the disclosure provides a processing method of a document image. As shown in fig. 7, in step S13, calculating an average luminance value of each pixel point corresponding to the foreground color region and an average luminance value of each pixel point corresponding to the background color region based on the foreground color region, and determining whether the background of the document image to be processed is a pure white background based on the average luminance value of each pixel point corresponding to the foreground color region and the average luminance value of each pixel point corresponding to the background color region includes:

step 131, determining whether the background of the document image to be processed is a pure white background based on the average brightness value of the pixels corresponding to the background color region and the difference between the average brightness value of the pixels corresponding to the foreground color region and the average brightness value of the pixels corresponding to the background color region.

The embodiment of the disclosure provides a processing method of a document image. As shown in fig. 8, step 131, determining whether the background of the document image to be processed is a pure white background based on the average luminance value of the pixels corresponding to the background color region and the difference between the average luminance value of the pixels corresponding to the foreground color region and the average luminance value of the pixels corresponding to the background color region includes:

in step 1311, when the average brightness value of the pixel points corresponding to the background color area is greater than the first threshold value, and the difference between the average brightness value of the pixel points corresponding to the foreground color area and the average brightness value of the pixel points corresponding to the background color area is greater than the second threshold value, it is determined that the background of the document image to be processed is a pure white background.

For example, in the present embodiment, the first threshold is set to 100, the second threshold is set to 5, and when the above two conditions are simultaneously satisfied, it is determined that the document image to be processed is a solid white background.

The embodiment of the disclosure provides a processing method of a document image. As shown in fig. 9, in step S14, based on a determination result of whether or not the background of the document image to be processed is a solid white background, processing the document image to be processed includes:

step 141, when the background of the document image to be processed is a pure white background, performing white balance processing on the document image to be processed based on the color values of the pixels of the document image to be processed and the corrected incident light color values of the corresponding blocks;

and 142, when the background of the document image to be processed is a non-pure white background, acquiring a gray level histogram of the document image to be processed, and carrying out brightness contrast balance processing on the document image to be processed based on the brightness value of each pixel point in the gray level histogram.

The embodiment of the disclosure provides a processing method for determining a document image. As shown in fig. 10, in step 141, when the background is a solid white background, performing white balance processing on the document image to be processed based on the color values of the pixels of the document image to be processed and the corrected incident light color values of the corresponding patches includes:

step 1411, when the ratio of the color value of the pixel point to the corrected incident light color value of the block where the pixel point is located is greater than 1, using 255 as the color value of the pixel point after the white balance processing; when the ratio of the color value of the pixel point to the corrected incident light color value of the block where the pixel point is located is smaller than 1, taking the product of the ratio and 255 as the color value of the pixel point after white balance processing.

The calculation can be performed with the following formula:

wherein C is _out Representing the color value of the pixel point after the white balance processing, C _input Representing the current color value of the pixel point, C _light The color value of the incident light of the block where the pixel point is located is represented.

The embodiment of the disclosure provides a processing method of a document image. As shown in fig. 11, in step 141, a gray-scale histogram of the document image to be processed after the white balance processing is obtained, and luminance-contrast balance processing is performed on the image to be processed after the white balance processing based on the luminance values of the respective pixel points in the gray-scale histogram. The method for performing brightness contrast balance processing on the image to be processed after the white balance processing based on the brightness value of each pixel point in the gray level histogram comprises the following steps:

Step 1412, determining a minimum non-0 luminance value and a maximum non-255 luminance value in the gray histogram;

in step 1413, the color value of each pixel in the image to be processed is adjusted based on the adjustment parameter according to the minimum non-0 luminance value and the maximum non-255 luminance value.

The adjustment parameters include a first adjustment parameter α and a second adjustment parameter β. Wherein,

β＝-V _min *α

according to the minimum non-0 brightness value and the maximum non-255 brightness value, adjusting the color value of each pixel point in the image to be processed based on the adjustment parameter comprises:

C _out ＝α*C _input +β

The embodiment of the disclosure provides a processing method for determining a document image. In step 142, when the background of the document image to be processed is a non-pure white background, a gray histogram of the document image to be processed is obtained, and the brightness contrast balance processing is performed on the document image to be processed based on the brightness value of each pixel point in the gray histogram. The specific processing steps may be performed according to steps 1412 to 1414.

According to the method for processing the document image, the processing mode of the document image with the non-pure white background is different from that of the document image with the pure white background by judging whether the document image with the pure white background is processed, so that the phenomenon of losing the background color can be effectively avoided.

The embodiment of the disclosure provides a processing method of a document image. As shown in fig. 12, before step S11, step S10 is further included to filter the document image to be processed. The filtering is carried out on the image to be processed, so that noise points in the document image to be processed can be filtered, and the processed document image has a better visual effect. The filtering may be in any manner that achieves the above-described effects, for example, a gaussian filtering method may be used. When filtering is performed by a gaussian filtering method, a gaussian convolution kernel may be used to filter the document image to be processed.

In order to compare the method with the method of the existing products in the market, 186 and Zhang Shi pieces of document images are subjected to comparison analysis. The 186 Zhang Shi document image includes handwritten notes, treatise contracts, invoices, business cards, screens, posters, books, and the like. And compared with the existing products in the market from four dimensions, and the comparison results are shown in table 1. The comparison scores given in table 1 are given in terms of a single full score of 5. As can be seen from Table 1, the document image processed by the document image processing method provided by the disclosure has a strong enhancement effect, improves the visual effect of the enhanced document image, and improves the user experience.

Table 1: compared with the existing products in the market

	Market existing product 1	Methods provided by the present disclosure
			Color preservation	3.92	4.41
Detail preservation	4.18	4.55
			Character definition	4.12	4.77
Noise/color block	3.87	4.11

The embodiment of the disclosure provides a processing device for document images. Fig. 13 is a block diagram showing a processing apparatus of a document image according to an exemplary embodiment. Referring to fig. 13, the apparatus includes a calculation module 101, a correction module 102, a determination module 103, and a processing module 104.

A calculating module 101 configured to divide a document image to be processed into a plurality of blocks according to a preset pixel size, and calculate an incident light color value of each block according to a brightness value of each pixel point in the block;

a correction module 102 configured to determine a foreground color region, determine color values of each pixel point corresponding to the foreground color region, and correct the incident light color value of the corresponding block according to the color values of each pixel point corresponding to the foreground color region;

a determining module 103 configured to calculate, based on the foreground color area, an average luminance value of each pixel point corresponding to the foreground color area and an average luminance value of each pixel point corresponding to the background color area, and determine, based on the average luminance value of each pixel point corresponding to the foreground color area and the average luminance value of each pixel point corresponding to the background color area, whether the background of the document image to be processed is a pure white background;

The processing module 104 is configured to process the document image to be processed based on a determination result of whether the background of the document image to be processed is a solid white background.

The computing module 101 is configured to:

The correction module 102 is configured to:

obtaining a background color region based on pixel points corresponding to a white region in the binarized image; determining a foreground color area of the document image to be processed based on the background color area;

The correction module is configured to:

The correction module 102 is further configured to:

before generating the mask region using the determined foreground color region, further comprising:

The determination module 103 is configured to:

and determining whether the background of the document image to be processed is a pure white background or not based on the average brightness value of the pixel points corresponding to the background color region and the difference value of the average brightness value of the pixel points corresponding to the foreground color region and the average brightness value of the pixel points corresponding to the background color region.

The determination module 103 is configured to:

and when the average brightness value of the pixel points corresponding to the background color area is larger than a first threshold value and the difference value of the average brightness value of the pixel points corresponding to the foreground color area and the average brightness value of the pixel points corresponding to the background color area is larger than a second threshold value, determining that the background of the document image to be processed is a pure white background.

The processing module 104 is configured to:

when the background of the document image to be processed is a non-pure white background, acquiring a gray histogram of the document image to be processed, and carrying out brightness contrast balance processing on the image to be processed based on brightness values of all pixel points in the gray histogram.

The processing module 104 is configured to:

The adjustment parameters include a first adjustment parameter a and a second adjustment parameter β, wherein,

β＝-V _min *α

the processing module 104 is configured to:

C _out ＝α*C _input +β

As shown in fig. 13, the document image processing apparatus provided by the present disclosure further includes:

the filtering module 105 is configured to filter the document image to be processed.

The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

Fig. 14 is a block diagram illustrating a processing apparatus 1400 for document images according to an example embodiment. For example, apparatus 1400 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or the like.

Referring to fig. 14, the apparatus 1400 may include one or more of the following components: processing component 1402, memory 1404, power component 1406, multimedia component 1408, audio component 1410, input/output (I/O) interface 1412, sensor component 1414, and communication component 1416.

The processing component 1402 generally controls overall operation of the device 1400, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 1402 may include one or more processors 1420 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 1402 can include one or more modules that facilitate interaction between the processing component 1402 and other components. For example, the processing component 1402 can include a multimedia module to facilitate interaction between the multimedia component 1408 and the processing component 1402.

The memory 1404 is configured to store various types of data to support operations at the device 1400. Examples of such data include instructions for any application or method operating on the device 1400, contact data, phonebook data, messages, pictures, videos, and the like. The memory 1404 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power component 1406 provides power to the various components of the device 1400. The power components 1406 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device 1400.

The multimedia component 1408 includes a screen between the device 1400 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1408 includes a front camera and/or a rear camera. The front camera and/or the rear camera may receive external multimedia data when the device 1400 is in an operational mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 1410 is configured to output and/or input audio signals. For example, audio component 1410 includes a Microphone (MIC) configured to receive external audio signals when device 1400 is in an operational mode, such as a call mode, a recording mode, and a speech recognition mode. The received audio signals may be further stored in the memory 1404 or transmitted via the communication component 1416. In some embodiments, audio component 1410 also includes a speaker for outputting audio signals.

The I/O interface 1412 provides an interface between the processing component 1402 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 1414 includes one or more sensors for providing status assessment of various aspects of the apparatus 1400. For example, the sensor assembly 1414 may detect the on/off state of the device 1400, the relative positioning of the components, such as the display and keypad of the device 1400, the sensor assembly 1414 may also detect a change in position of the device 1400 or a component of the device 1400, the presence or absence of user contact with the device 1400, the orientation or acceleration/deceleration of the device 1400, and a change in temperature of the device 1400. The sensor assembly 1414 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 1414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1414 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 1416 is configured to facilitate communication between the apparatus 1400 and other devices in a wired or wireless manner. The device 1400 may access a wireless network based on a communication standard, such as WiFi,2G, or 3G, or a combination thereof. In one exemplary embodiment, the communication component 1416 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 1416 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 1400 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.

In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as a memory 1404 including instructions executable by the processor 1420 of the apparatus 1400 to perform the above-described method. For example, the non-transitory computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

A non-transitory computer readable storage medium, which when executed by a processor of a mobile terminal, causes the mobile terminal to perform a method of processing a document image, the method comprising:

dividing a document image to be processed into a plurality of blocks according to a preset pixel size, and calculating an incident light color value of each block according to brightness values of pixel points in the blocks;

Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method of processing a document image, the method comprising:

2. The method according to claim 1, wherein calculating the incident light color value of each block from the color values of the pixel points within the block comprises:

3. The method according to claim 1, wherein the determining the foreground color region, determining the color value of each pixel corresponding to the foreground color region, and correcting the incident light color value of the corresponding block according to the color value of each pixel corresponding to the foreground color region comprises:

obtaining the background color area based on pixel points corresponding to the white area in the binarized image; determining a foreground color area of the document image to be processed based on the background color area;

4. The method according to claim 3, wherein the obtaining the background color area based on the pixel points corresponding to the white area in the binarized image includes:

5. A document image processing method according to claim 3, further comprising:

6. The method according to claim 1, wherein the determining whether the background of the document image to be processed is a solid white background based on the average luminance value of each pixel point corresponding to the foreground color region and the average luminance value of each pixel point corresponding to the background color region comprises:

7. The method according to claim 6, wherein determining whether the background of the document image to be processed is a solid white background based on the average luminance value of the background-color region corresponding pixel points and the difference between the average luminance value of the foreground-color region corresponding pixel points and the average luminance value of the background-color region corresponding pixel points comprises:

8. The method according to claim 1, wherein the processing the document image to be processed based on the determination result of whether the background of the document image to be processed is a solid white background, comprises:

9. The method according to claim 8, wherein performing white balance processing on the document image to be processed based on the color value of each pixel point of the document image to be processed and the corrected incident light color value of the corresponding patch comprises:

10. The method according to claim 8, wherein performing the luminance-contrast equalization processing on the image to be processed after the white balance processing based on the luminance value of each pixel in the gradation histogram or performing the luminance-contrast equalization processing on the image to be processed based on the luminance value of each pixel in the gradation histogram comprises:

11. The method for processing a document image according to claim 10, wherein the adjustment parameters include a first adjustment parameter α and a second adjustment parameter β, wherein,

β＝-V _min *α

C _out ＝α*C _input +β

12. The method for processing a document image according to claim 1, further comprising:

filtering the document image to be processed.

13. A processing apparatus of a document image, characterized by comprising:

14. The document image processing apparatus according to claim 13, wherein the calculation module is configured to:

15. The apparatus according to claim 13, wherein the correction module is configured to:

16. The apparatus according to claim 15, wherein the correction module is configured to:

17. The document image processing apparatus according to claim 15, wherein the correction module is further configured to:

18. The apparatus for processing a document image according to claim 13, wherein the determination module is configured to:

19. The document image processing apparatus according to claim 18, wherein the determination module is configured to:

20. The apparatus for processing a document image according to claim 13, wherein the processing module is configured to:

21. The document image processing apparatus according to claim 20, wherein the processing module is configured to:

22. The document image processing apparatus according to claim 20, wherein the processing module is configured to:

23. The apparatus according to claim 22, wherein the adjustment parameters include a first adjustment parameter α and a second adjustment parameter β, wherein,

β＝-V _min *α

the processing module is configured to:

C _out ＝α*C _input +β

24. The apparatus for processing a document image according to claim 13, wherein the apparatus further comprises:

25. A document image processing apparatus, comprising:

A processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

26. A non-transitory computer readable storage medium, which when executed by a processor of a mobile terminal, causes the mobile terminal to perform a method of processing a document image, the method comprising: