JP4346620B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program for binarizing an image such as a document image.

  Various image binarization methods have been proposed in accordance with the binarization target image shooting / imaging target and shooting conditions. For document images and drawings captured by an imaging device such as a scanner, Otsu It is widely known that the single threshold method such as the above method is effective (see Non-Patent Document 1).

  On the other hand, in an image taken with an imaging device such as a digital camera, if the lighting conditions and sensitivity are not constant in the image, binarization with a single threshold is not appropriate. A dynamic threshold method is used in which the binarization threshold value is changed according to the position within.

  Various methods have been proposed as the dynamic threshold method, and a partial image division method is known as a typical method (see Non-Patent Document 2).

In the partial image division method, an image is divided into small regions, threshold values are obtained for each divided region by the method of Otsu, etc., and the obtained threshold values are adopted as threshold values at the center point of each small region. . Then, the threshold value at the center point is complemented smoothly to determine the binarization threshold value at each pixel.
Otsu, "Automatic threshold selection method based on discriminant and least-square criterion", IEICE Transactions, 1980, Vol. J63-D, No. 4, pp.349-356 Mikio Takagi, Yoshihisa Shimoda, "Image Analysis Handbook", The University of Tokyo Press, 1991, pp.502-505

  However, many of the conventional dynamic threshold methods are intended for two-color images corresponding to the figure and the ground, and are based on the premise that this is true even in a small area.

  In addition, even if the image is a two-color image such as a document, a shadow is applied when the image is taken with a digital camera, and the three colors of figure, ground, and shadow may be reflected in a small area. Binarization errors sometimes occurred in the binarization process.

  In addition, for small areas where the interior is either the figure or the ground, there is a device that depends on the processing target, for example, adjustment of the size of the small area, so that no error occurs during the binarization process. Was necessary.

  The present invention has been made to solve such a problem, and is capable of performing stable binarization processing on a shadowed image, an image including a wide figure region, or a wide ground region. It is an object to provide an apparatus, an image processing method, and an image processing program.

To achieve the above object, the image processing apparatus of the present invention, the original image is divided into multiple partial regions, the brightness of the background in the partial region each said partial area by detecting a representative value of brightness for the A plurality of candidates for the background lightness candidate detection unit and a plurality of background lightness candidates detected by the background lightness candidate detection unit, one for each partial region, and based on the combination of the selected candidates wherein the background brightness distribution estimating unit for determining the distribution of the brightness of the background of the original image, the threshold for the binarization processing of the background lightness distribution the original image from the distribution of the brightness of calculated et background by the estimation unit in And a binarization unit that binarizes the original image using the threshold calculated by the threshold calculation unit.

The image processing program of the present invention is an image processing program for executing the image processing in a computer, the computer, the original image is divided into several partial areas, detecting a representative value of brightness for each of the partial region And selecting a background lightness candidate detection unit as a plurality of background lightness candidates in the partial region and a plurality of background lightness candidates detected by the background lightness candidate detection unit one by one for each partial region and, of the original image from the and the background brightness distribution estimating unit for determining the distribution of the brightness of the background of the original image, the background brightness distribution estimating unit by seeking et lightness distribution of the background based on the combination of the selected candidate two A threshold value calculation unit that calculates a threshold value for the binarization process; and a binarization unit that binarizes the original image using the threshold value calculated by the threshold value calculation unit. And features.

The image processing method of the present invention is an image processing method by the image processing apparatus performs the image processing, the image processing apparatus divides the original image into multiple subregions, each representative of the brightness for the partial region Detecting a value as a plurality of candidates for background brightness in the partial area, selecting a plurality of background brightness candidates for each partial area, and based on the combination of the selected candidates , Obtaining a distribution of lightness of the background, calculating a threshold value for binarization processing of the original image from the calculated distribution of lightness of the background , and binarizing the original image by the calculated threshold value. Features.

  In the present invention, the original image is divided into a plurality of partial areas, a lightness value candidate used as the lightness of the background of the original image is detected for each divided partial area, and the background lightness distribution of the original image is estimated using the candidates. . In other words, the background brightness is estimated so that the fluctuation in the entire image area of the original image is minimized, and the binarization threshold value is calculated based on the estimated background brightness, so it is stable against local fluctuations due to shadows, etc. Binarization processing can be performed.

  As described above, according to the present invention, it is possible to perform stable binarization processing on a shadowed image, an image including a wide figure region, or an image including a wide ground region.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention, and FIGS. 2 to 5 are diagrams illustrating exemplary divisions of an original image.

  As shown in FIG. 1, the image processing apparatus includes an imaging apparatus such as a digital camera 1 and a computer 2 (hereinafter referred to as a PC 2) connected to the digital camera 1.

  In this image processing apparatus, the digital camera 1 functions as an image acquisition unit that captures an image of the surface of a character recognition target form and reads and acquires the image. It is assumed that characters and the like are entered in the form as predetermined information, and the form image obtained by the digital camera 1 has a gray scale in which white, black, and other colors, that is, intermediate colors are mixed.

  Hereinafter, “entry” includes both the meanings when characters and the like are handwritten and printed.

  The image information is not limited to the digital camera 1 but may be read by a scanner or the like, or may be acquired by conversion from a document file. Further, the PC 2 may acquire image information stored in another computer through a network such as a LAN.

  The functions of the PC 2 include CPU, RAM, ROM and other memories, an auxiliary storage device such as a hard disk device, an input device such as a keyboard and a pointing device such as a mouse, a display device such as a monitor, and a hardware such as an interface board with the digital camera 1. Hardware and programs such as an operating system (hereinafter referred to as OS) and image processing software.

  The PC 2 includes a storage unit 21, a background brightness candidate detection unit 22, a background brightness candidate supplement unit 25, a background brightness distribution estimation unit 26, a threshold value calculation unit 29, and a binarization unit 33.

  The background lightness candidate detection unit 22 includes a histogram correction unit 23 and an interframe histogram addition unit 24.

  The histogram correction unit 23 generates a histogram obtained by adding the histogram in the vicinity of the partial region to the histogram of the partial region, and uses the generated histogram instead of the histogram of the frame.

  When the original image is one frame of a moving image, the inter-frame histogram adding unit 24 adds the lightness histogram of a frame before the frame obtained by performing background lightness candidate detection to the histogram of the frame. A histogram is generated, and the generated histogram is used instead of the histogram of the frame.

  The histogram correction unit 23 and the interframe histogram addition unit 24 are not essential components. These may be combined into one.

  The background lightness candidate detection unit 22 divides the original image into partial areas according to a predetermined procedure, and detects background lightness value candidates of the original image for each partial area.

  The background lightness candidate supplement unit 25 corrects (supplements) the background lightness value candidates of the original image detected by the background lightness candidate detection unit 22 using a predetermined calculation formula (Equation 1), and corrects (supplements) the correction. The value is stored in the storage unit 21 as a background brightness candidate. When the original image is one frame of a moving image, the background lightness candidate supplement unit 25 supplements the background lightness candidate of the partial area with a set of background lightness candidates in a frame before the frame.

  The background lightness distribution estimation unit 26 reads the background lightness value candidates stored in the storage unit 21, performs a process of selecting candidates from the candidates using an evaluation function (see FIG. 14), and the selected candidates. Is used to estimate the lightness distribution of the background of the original image. That is, the background lightness distribution estimation unit 26 calculates the lightness distribution of the background of the original image using the lightness value selected from the candidates.

  The background lightness distribution estimation unit 26 includes a background lightness distribution candidate enumeration unit 27 and a background lightness distribution selection unit 28.

  The background lightness distribution candidate enumeration unit 27 stores candidates used as the lightness value of the background of the original image for each partial region in the storage unit 21 or a memory such as a buffer, and sets a group of partial regions with each partial region as an initial state. Repetitively merge to generate one group, update the corresponding candidate in the memory every time the generated group is merged, and change the background brightness distribution based on the candidate for each partial area stored in the memory when the fusion is completed List the candidates.

  The background lightness distribution selection unit 28 determines a candidate number S according to a predetermined procedure, and outputs the number candidate as a background lightness estimation result for the entire image, that is, stores it in the storage unit 21.

  The threshold calculation unit 29 binarizes the original image based on the background lightness distribution estimated (calculated) by the background lightness distribution estimation unit 26 (background lightness estimation result in the storage unit 21). Calculate the threshold.

  The threshold calculation unit 29 includes a brightness correction unit 30, a pre-correction threshold calculation unit 31, and a threshold reverse correction unit 32.

  The brightness correction unit 30 corrects the original image by substituting the background brightness distribution read from the storage unit 21 into a predetermined calculation formula (such as (Equation 29) or (Equation 30)), that is, corrects the original image by shading. Thus, a corrected image is generated.

  The pre-correction threshold value calculation unit 31 calculates a pre-correction threshold value for the corrected image generated by the lightness correction unit 30 using a predetermined calculation formula (Otsu's method or the like).

  The threshold reverse correction unit 32 calculates a binarized threshold value by substituting the pre-correction threshold value calculated by the pre-correction threshold value calculation unit 31 into a predetermined calculation formula.

  The binarization unit 33 binarizes the original image read from the storage unit 21 using the threshold value calculated by the threshold value calculation unit 29 (binarization threshold value of the storage unit 21).

  The storage unit 21 is realized by a memory, a hard disk device, or the like. The storage unit 21 stores formulas and selection conditions (evaluation function, etc.) described below. The storage unit 21 stores image data input from the digital camera 1. The storage unit 21 functions as a temporary storage area (working area) for image data when each unit performs image processing and data obtained by each unit using an expression.

  Since this image processing apparatus can be configured by incorporating software that implements the present invention into a computer, the following description will be made assuming such a configuration. However, in the present invention, each unit can be configured as dedicated hardware, an aggregate thereof, or a computer network for distributed processing, and may be implemented with any configuration regardless of the configuration listed here.

  In addition, the configuration shown in the above embodiment is only an example, and each configuration in the embodiment is not necessarily an essential configuration, and some configurations may be selected and used. Other configurations may be added.

  In this image processing apparatus, the abscissa of a pixel on the original image is x, the ordinate is y, the brightness of the coordinate (x, y) on the original image is f (x, y), and the binary image The pixel value at is defined as o (x, y).

  The background lightness candidate detection unit 22 divides the original image by a predetermined procedure, that is, the following several methods, and detects background lightness candidates in the divided partial areas.

で表し、分割された部分領域の個数をＫで表す。 As shown in FIG. 2, the background lightness candidate detection unit 22 divides the original image into a grid and generates a partial region. The position in the vertical direction of the partial area divided into a lattice is represented by i, the position in the horizontal direction is represented by j, the number of divisions in the vertical direction is represented by m, the number of divisions in the horizontal direction is represented by n, and the position of the area Is represented by (i, j), and the region number is

And the number of the divided partial areas is represented by K.

  In the case where the background lightness candidate detection unit 22 divides the original image into partial regions, the image dividing method may be divided in the horizontal direction as shown in FIG. The original image may be divided in the vertical direction as shown in FIG. Further, as shown in FIG. 5, the original image may be divided into partial regions having a non-uniform shape, and may be divided by any division method as long as the number of divisions is limited.

  When detecting the background lightness candidate, the background lightness candidate detecting unit 22 generates a lightness histogram for each of the partial regions obtained by dividing the original image by any one of the following methods a, b, and c. A background lightness candidate is detected from the lightness histogram for each partial region.

FIG. 6 is an example of a brightness histogram generated by the background brightness candidate detection unit 22, and in this example, extreme values for each peak are detected as background brightness candidates p ₁ , p ₂ , p ₃ , and p ₄ . This method is referred to as method a.

  That is, in the case of method a, the background lightness candidate detection unit 22 obtains the lightness value of the background of the original image by detecting the extreme value of the lightness at the peak of the lightness histogram for each partial region obtained by dividing the original image. Candidate for.

  In addition to this, for example, a background brightness candidate may be detected as a representative value of a class up to the upper N class in terms of frequency among the classes of the generated histogram by generating a brightness histogram for each of the regions obtained by dividing the original image. N is a predetermined constant.

FIG. 7 is an example of background lightness candidate detection in the case where the representative values of the classes up to the upper 4 classes are used as background lightness candidates. In this example, background lightness candidates p ₁ , p ₂ , p ₃ , and p ₄ are detected. Yes. This method is referred to as method b.

  The background lightness candidate may be obtained as a cluster representative value by applying a k-means method (existing representative value calculation method) or the like to the lightness sample value for each partial region obtained by dividing the original image. . This method is referred to as method c.

  In addition, as a method for detecting a background lightness candidate, any method may be used as long as it can be expected that a background lightness candidate can be detected.

  As the brightness histogram in the method a and the method b, a histogram obtained independently in each partial region may be used as it is, or a histogram corrected by the histogram correction unit 23 described below may be used.

  Here, the use of the histogram corrected by the histogram correction unit 23 will be described.

  The histogram correction unit 23 adds a histogram in the vicinity of the target divided region to the histogram of the target divided region. The class number of the histogram before correction is l, the histogram of the divided area (i, j) is h (l; i, j), and the corrected histogram of the divided area (i, j) is h ′ (l; i, j, j), the processing by the histogram correction unit 23 is expressed by the following (formula 1).

但し、Ｕ，Ｖは非負の整数であり、ｗ（ｕ，ｖ）は、実関数でｗ（ｕ，ｖ）≧０かつ

とする。

However, U and V are non-negative integers, and w (u, v) is a real function w (u, v) ≧ 0 and

And

If i + u <0, i + u ≧ m, j + v <0 or j + v ≧ n,
It is assumed that h (l; i + u, j + v) = 0.

  Further, when the original image is one frame of the moving image, the processing by the inter-frame histogram adding unit 24 that adds the brightness histogram in the frame before the frame can be performed.

The frame number is t, the histogram of the divided region (i, j) is h (l; i, j, t), and the corrected histogram of the divided region (i, j) is h ′ (l; i, j, When represented by t), the processing by the interframe histogram adding unit 24 is represented by the following (formula 2).

If t−θ is smaller than the first frame number, h ′ (l; i, j, t−θ) = 0.
And

Further, the processing by the histogram correction unit 23 and the processing by the inter-frame histogram addition unit 24 may be combined and the histogram may be corrected by the following (Equation 3).

  The background lightness candidate detected by the background lightness candidate detection unit 22 may be used as it is, or may be corrected by the background lightness candidate supplementing unit 25 described below and the correction value may be used.

  The background lightness candidate supplement unit 25 supplements the background lightness candidates of the target divided region with the background lightness candidates of the neighboring divided regions.

The number of background lightness candidates before supplementation of the divided area (i, j) is represented by N (i, j), and a set of background lightness candidates is represented by P (i, j) = {p ₁ ,..., P _{N (i, j )} expressed _in}, to represent the set of background lightness candidate after supplemented with P '(i, j), treatment with the background brightness candidate supplemental part 25 is represented by the following (equation 4).

但し、Ｕ，Ｖは、非負の整数であり、ｉ+ｕ＜０、ｉ+ｕ≧ｍ、ｊ+ｖ＜０またはｊ+ｖ≧ｎの場合は、Ｐ（ｉ+ｕ，ｊ+ｖ）＝φ（空集合）とする。

However, U and V are non-negative integers. When i + u <0, i + u ≧ m, j + v <0 or j + v ≧ n, P (i + u, j + v) = Φ (empty set).

  When the original image is one frame of a moving image, the background brightness candidate supplementing unit 25 may supplement the background brightness candidates of the frame with a set of background brightness candidates in a frame before the frame.

The number of background lightness candidates before supplementation of the divided region (i, j) at frame number t is represented by N (i, j, t), and a set of background lightness candidates is P (i, j, t) = {p ₁ ,..., P _{N (i, j, t)} }, and a set of background lightness candidates after supplementation is represented by P ′ (i, j, t), the processing by the background lightness candidate supplementing unit 25 is as follows: It is represented by Formula 5).

また、（ｔ−θ）が最初のフレーム番号より小さい場合、ｉ+ｕ＜０、ｉ+ｕ≧ｍ、ｊ+ｖ＜０またはｊ+ｖ≧ｎの場合はＰ（ｉ+ｕ，ｊ+ｖ，ｔ−θ）＝φ（空集合）とする。

If (t−θ) is smaller than the first frame number, if i + u <0, i + u ≧ m, j + v <0 or j + v ≧ n, P (i + u, j + v, t−θ) = φ (empty set).

  The processing result of the background lightness candidate supplement unit 25 may be used as a background lightness candidate as it is, or may be used after being reduced by replacement with a representative value obtained by the k-means method or the like.

  The background lightness distribution estimation unit 26 estimates the background lightness distribution using the background lightness candidates detected by the background lightness candidate detection unit 22.

  The background brightness distribution estimation unit 26 estimates the background brightness for each pixel when estimating the background brightness distribution. The background brightness estimation result at the pixel coordinates (x, y) is b (x, y).

  The background lightness distribution estimation unit 26 obtains a background lightness estimation result b (x, y) for each pixel by the following two steps.

  1) Based on the background lightness candidates detected by the background lightness candidate detection unit 22, a background lightness distribution candidate enumeration unit 27 (to be described later) uses b (x, y) candidates c (x, y; 1) and c (x , Y; 2),... C (x, y; γ). Where γ is the number of candidates for the background brightness distribution.

  2) A candidate number S is set by a background brightness distribution selection unit 28 to be described later, and a candidate for the number is set as a background brightness estimation result for the entire image.

    b (x, y) = c (x, y; S) (Formula 6)

  The background lightness distribution candidate enumeration unit 27 generates a background lightness candidate C (k; s) for each partial region divided by the background lightness candidate detection unit 22 and uses it as a background lightness candidate for the pixels in the region. .

c (x, y; s) = C (k (x, y); s) (Expression 7)
Here, s is a candidate number, and k (x, y) is a number of a divided region including a pixel at coordinates (x, y).

  Hereinafter, with reference to the flowchart of FIG. 8, the concept of “group” is introduced for a partial region, and a calculation procedure for enumerating background lightness candidates C (k; s) for each partial region (the background lightness distribution candidate enumeration unit 27 Operation) will be described.

  A group is a set of one or more partial areas. In the initial state, one area in each partial area is one group, and calculation of the background brightness candidate C (k; s) is completed for each divided area. Sometimes all partial areas form one group.

  In the initial state, the background lightness distribution candidate enumeration unit 27 uses the sth background lightness candidate detected in the region by the background lightness candidate detection unit 22 using the sth candidate C (k; s) of each partial region. Initialization is performed (step 101 in FIG. 8). Step is abbreviated as S.

  The background lightness candidate C (k; s) for each partial region stored in the storage unit 21 is updated by repeating the following four steps.

The number of background brightness candidates in each partial region is a constant value Γ _G within the group G. In the q-th iteration, R _q groups G _{q, 1} , G _{q, 2} ,..., G _{q, Rq} are merged into one group. The iterative process is repeated until all the partial areas form one group.

1． Each group _{G q, 1, G q,} 2, ..., G q, in _Rq, the combination determined by selecting the candidate number of background lightness one by _{1 (s 1, s 2,} ..., s Rq) enumerate all (S102).

2. For each combination (s ₁ , s ₂ ,..., S _Rq ) and area number k, a new background brightness candidate D (k; s ₁ , s ₂ ,..., S _Rq ) is defined by the following (formula 8).

但し、

はk番目の部分領域がグループＧ_gに含まれていることを示している。

However,

Indicates that the k-th partial region is included in the group G _g .

The background lightness distribution candidate enumeration unit 27 applies the new background lightness candidate D (k; s ₁ , s ₂ ,..., S _Rq ) to the following (Equation 9) to apply the cost function E (k; s ₁ , s ₂ , .., S _Rq ) are calculated and obtained (S103).

For example, as shown in FIG. 9, when six non-uniform partial areas are provided in the original image, Ω is a set as follows. The numbers in the figure indicate the numbers of the partial areas.
Ω = {(1, 2), (1, 3), (1, 4), (2, 3), (2, 5), (3,4), (3, 5), (3, 6) , (4, 6), (5, 6) (Equation 10)
However (Equation 9), the cost function _{E (k; s 1, s} 2, ..., s Rq) is an example of a formula for the cost function _{E (k; s 1, s} 2, ..., s Rq) of As a calculation formula, any formula may be used as long as it is a formula for evaluating the magnitude of the variation of the background lightness candidate C (k; s).

3. From the combinations listed in the first stage, the lower ΓMAX combinations are selected in the cost function E (k; s ₁ , s ₂ ,..., S _Rq ) obtained in the second stage.

In the first stage, when the number of listed combinations is less than ΓMAX, all combinations are selected. Assume that the number of selected combinations is Z, the combination number is z, and the z-th combination having the smallest cost function E (k; s ₁ , s ₂ ,..., S _Rq ) is ζz (S104).

4). With the constant value Γ _G = Z, the Z candidates are updated by the following updating formula (Formula 11) (S105).

  In this way, the background lightness distribution candidate enumeration unit 27 repeats the update (repetition process), determines whether or not all partial areas form one group (S106), and all the partial areas include one group. If it is configured (true in S106), the process is terminated.

In the third stage of the above iterative process, the background brightness distribution candidate enumeration unit 27 enumerates the lower-level candidates by the cost function E (k; s ₁ , s ₂ ,..., S _Rq ). The background brightness distribution candidates are listed so that the fluctuation of the background brightness distribution is reduced.

  By using the candidates listed in this way, the background brightness distribution estimation unit 26 can estimate the background brightness so that the variation in brightness is reduced.

The number R _{q of} groups to be merged in each iteration and the combinations G _{q, 1} , G _{q, 2} ,... G _{q, R} are arbitrary, and combinations (s ₁ , s ₂₎ listed in the first stage of the above iterative processing. , ..., if the range number is acceptable for s _Rq), the number of groups R _q and combining _{G q, 1, G q,} 2, ... G q, may be determined how the _R.

  However, the allowable range here is a range in which hardware resources and calculation time required for performing image processing according to the present invention fall within an available range.

  FIG. 10 is a diagram showing that the original image is divided into a grid pattern and subdivided into four vertical and horizontal partial areas (16 partial areas in total), and FIG. 11 shows 16 pieces as shown in FIG. It is a figure which shows an example of the number of groups and a combination which are united by each repetition when a partial area | region is set.

In the combination shown in FIG. 10, four groups are fused in every iteration, and a group fused in the first iteration to the fourth iteration is fused again in the fifth iteration. If the first iteration to the fourth iteration are regarded as the first stage, the fifth iteration is regarded as the second stage, and the stage number is represented by λ, each group is composed of 4 groups of 4 ^λ partial regions. .

Generalizing this fusion method, if the number of stages is Λ and R groups are recursively merged at each stage, the number of partial areas in the group fused at each stage is R ^λ and the total number of partial areas is K = R ^Λ , and the number of iterations per stage is R ^Λ−λ .

For example, partial areas are set in a lattice shape, the number of groups to be merged at each stage is R = r ² , the number of divisions in the vertical and horizontal directions is m = n = r ^Λ, and the number of iterations in the stage is ρ , The repetition serial number is q, the group to be fused in each repetition is G _{q, 1} ,..., G _{q, R,} and the number of the υth partial region belonging to the group G _{q, g} is k _{q, p, g} (υ), q and k _{q, p, g} (υ) are defined by the following (formula 12) to (formula 21), and the groups are merged.

FIG. 11 is a diagram showing an example in the case of r = 2, Λ = 4, and λ = 2, and FIG. 12 is a diagram showing how groups are merged at each stage of r = 2 and Λ = 4.

  The background lightness distribution selection unit 28 determines a candidate number S according to the following procedure, and uses the number candidate as the background lightness estimation result for the entire image. (See (Formula 6))

That is, as shown in FIG. 14, the background lightness distribution selection unit 28, for each candidate number s (1 ≦ s ≦ γ), from the background density distribution c (x, y; s), the evaluation function L (s) shown below ) Are calculated and the candidate number s and evaluation function pairs (s, L (s)) are listed (S111).

  The background lightness distribution selection unit 28 then arranges the listed evaluation function pairs (s, L (s)) in ascending order of the evaluation function L (s) (S112), and sets the candidate number s of the σ-th pair as ( The candidate number S in equation 6) is set (S113).

  However, σ = λ. In some cases, σ = 1, σ = min (2, λ), σ = max (λ-1, 1), and the like are more suitable depending on an image to be processed and conditions. Any method other than the above method may be used.

  The background lightness estimation method in the background lightness distribution estimation unit 26 has been described above. However, any method can be used as long as the method can be expected to reduce the fluctuation of the background lightness candidate b (x, y). May be used.

For example, when the total number of divided areas is small, all candidate number combinations (s ₁ , s ₂ ,..., S _k ) in each divided area are listed, and the following (formula 23) to (formula 25) are used. _{E global (s 1, s 2} , ..., s k) defining Te becomes the minimum _{(s 1, s 2, ...} , s k) for the background brightness estimation result for each pixel by the following equation (26) b ( x, y) may be calculated and obtained.

  Further, post-processing may be performed on the background lightness estimation result b (x, y) estimated by the background lightness distribution estimation unit 26.

For example, b (x, y) is obtained as a result of convolution integral b ′ (x, y) with a Gaussian function H (σx, σy) (see (Equation 28)) having variances in the horizontal and vertical directions of σx and σy It may be replaced. (See (Equation 27))

  When post-processing is performed on the background lightness estimation result b (x, y), the change in b (x, y) becomes gentle, and the distortion of the binarization result at the boundary of the partial region is alleviated.

  The threshold value calculation unit 29 calculates a binarization threshold value based on the background lightness distribution estimated by the background lightness distribution estimation unit 26.

Hereinafter, the binarization threshold value at the coordinates (x, y) is τ (x, y).
The threshold value calculation unit 29 calculates a binarized threshold value by the following three steps.

  1) The corrected image is generated by correcting the original image read out from the storage unit 21 by the brightness correction unit 30 according to the following function formula.

  2) The pre-correction threshold value calculation unit 31 calculates a pre-correction threshold value for the corrected image generated by the brightness correction unit 30.

  3) The threshold reverse correction unit 32 calculates a binarized threshold value using the pre-correction threshold value calculated by the pre-correction threshold value calculation unit 31.

The lightness correction unit 30 corrects light and darkness due to shading of the original image f (x, y).
The lightness correction unit 30 uses the background lightness b (x, y) estimated by the background lightness candidate detection unit 22 to obtain the lightness value g (x, y) of the corrected image by the following (Equation 29).
g (x, y) = f (x, y) −b (x, y) (Equation 29)

Further, the lightness correction unit 30 may perform correction processing based on the following (Equation 30), and may be corrected by any method as long as it can correct shading of the original image.

  Where ε is a positive real number. The pre-correction threshold value calculation unit 31 calculates a single binarization threshold value β for the lightness value g (x, y) of the corrected image obtained by the lightness correction unit 30. The threshold value β calculated by the pre-correction threshold value calculation unit 31 is referred to as a pre-correction threshold value.

  The pre-correction threshold value β is obtained by applying an automatic threshold selection method (existing threshold value calculation method) based on the least square criterion to the lightness value g (x, y) of the corrected image. .

  As a method for calculating the threshold value β before correction, any method may be used as long as it is a method for obtaining a single binarization threshold value for the lightness value g (x, y) of the corrected image.

  The threshold reverse correction unit 32 calculates a binarization threshold τ (x, y) for the original image f (x, y) from the pre-correction threshold β calculated by the pre-correction threshold calculation unit 31. calculate.

  The threshold value reverse correction unit 32 binarizes the brightness value g (x, y) of the corrected image with the threshold value β and the binarized threshold value τ (x , Y), the binarization threshold τ (x, y) is calculated so that the binarization result matches.

When the lightness correction unit 30 performs lightness correction using the above (formula 29), the binarization threshold τ (x, y) is calculated by the following (formula 31).
τ (x, y) = β + b (x, y) (Equation 31)

When the lightness correction unit 30 performs lightness correction using the above (Expression 30), the binarization threshold τ (x, y) is calculated using the following (Expression 32).

  The binarization threshold τ (x, y) is obtained by binarizing the corrected image g (x, y) with the threshold β and the original image f (x, y). You may calculate so that the result binarized by (tau) (x, y) may be similar.

  For example, when the brightness correction unit 30 performs brightness correction using the above (Equation 30), the binarization threshold τ (x, y) may be calculated by the following (Equation 33).

  τ (x, y) = βb (x, y) (Expression 33)

  Further, the threshold value calculation unit 29 may calculate the binarization threshold value as a constant α times the background lightness estimated by the background lightness distribution estimation unit 26.

  τ (x, y) = αb (x, y) (Formula 34)

  As the threshold value calculation unit 29, any means may be used as long as it has a function of adaptively calculating a binarization threshold value according to the estimated background lightness.

  The binarization unit 33 binarizes the original image using the threshold value calculated by the threshold value calculation unit 29.

但し、Ａ，Ｂは、二値化後の画素値であり、Ａ≠Ｂである限り、いかなる組合せでも良い。また、二値化部３３による二値化処理は、等号の位置を変えて下記（式３６）を用いて行っても良い。

The binarization unit 33 performs threshold processing on the brightness f (x, y) with the binarization threshold τ (x, y) for each pixel of the original image, and the pixel value o (x, y) of the binary image. Is calculated.

However, A and B are pixel values after binarization, and any combination may be used as long as A ≠ B. Also, the binarization processing by the binarization unit 33 may be performed using the following (formula 36) by changing the position of the equal sign.

  As described above, according to the image processing apparatus of this embodiment, the original image is divided into a plurality of partial areas according to a predetermined procedure, and lightness value candidates used as the lightness of the background of the original image are detected for each partial area. A background lightness candidate detection unit 22, a background lightness distribution estimation unit 26 that estimates the background lightness distribution of the original image using the detected candidates, and a binarization process for the original image based on the estimated background lightness distribution. By including the threshold value calculation unit 29 for calculating the threshold value and the binarization unit 33 for binarizing the original image using the calculated threshold value, the fluctuation of the entire image area of the original image is minimized. Thus, the background lightness is estimated, and the binarization threshold value is calculated based on the estimated background lightness, so that stable binarization processing can be performed against local fluctuations due to shadows and the like.

1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. It is a figure which shows an example of the partial area | region produced | generated by dividing | segmenting an original image into a grid | lattice form. It is a figure which shows an example of the partial area | region produced | generated by dividing | segmenting an original image vertically. It is a figure which shows an example of the partial area which divided | segmented the original image and produced | generated. It is a figure which shows an example of the partial area | region produced | generated by dividing | segmenting an original image unevenly. It is a figure which shows an example of the candidate detection result in a background lightness candidate detection part. It is a figure which shows an example of the candidate detection result in a background lightness candidate detection part. It is a flowchart which shows operation | movement of a background brightness distribution candidate enumeration part. It is a figure which shows the example of a partial region division used in order to illustrate the combination of the adjacent partial region in a background brightness distribution estimation part. It is an example of area division used for explanation of repetitive processing, and is a diagram in which an original image is subdivided into four vertical and horizontal partial areas. It is a figure which shows an example of the number of groups and a combination which unite the partial area | region of FIG. 10 by each iterative process. It is a figure for demonstrating the number calculation process of the fusion object partial area in a background brightness distribution estimation part. It is a figure which shows the mode of the fusion of the group in a background lightness distribution estimation part. It is a flowchart which shows operation | movement of a background brightness distribution selection part.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Digital camera, 2 ... Computer, 21 ... Memory | storage part, 22 ... Background lightness candidate detection part, 23 ... Histogram correction part, 24 ... Inter-frame histogram addition part, 25 ... Background lightness candidate supplement part, 26 ... Background lightness distribution estimation 27: Background brightness distribution candidate enumeration section, 28 ... Background brightness distribution selection section, 29 ... Threshold calculation section, 30 ... Brightness correction section, 31 ... Pre-correction threshold calculation section, 32 ... Threshold reverse correction Part, 33... Binarization part.

Claims (10)

The original image is divided into multiple partial regions, and the background brightness candidate detection unit, wherein for each of the partial areas by detecting a representative value of brightness and a plurality of candidates of the brightness of the background in the partial region,
The background brightness detected by the candidate detection unit the brightness of the background plural candidates selected one by one for each of the partial regions, the background to determine the distribution of the brightness of the background of the original image based on the combination of the selected candidate A brightness distribution estimation unit;
A threshold value calculation unit for calculating a threshold value for binarization processing of the original image from the distribution of lightness of the background estimated by the background lightness distribution estimation unit;
An image processing apparatus comprising: a binarization unit that binarizes the original image based on the threshold value calculated by the threshold value calculation unit. The background lightness distribution estimation unit,
Obtaining a plurality of combinations of candidates selected one by one from a plurality of candidates for the background brightness for each partial region, and comprising a background brightness distribution candidate enumeration unit for obtaining a background brightness distribution candidate from each of the combinations ,
The background lightness distribution estimation unit performs background lightness candidate selection so that variation in the entire background lightness candidate image selected in each of the combinations is reduced . Image processing device. The background lightness distribution candidate enumeration unit
Candidates used as lightness values of the background of the original image for each partial area are stored in a memory, and a group of partial areas with each partial area as an initial state is repeatedly fused to generate one group,
Update the corresponding candidate in the memory every time the group is merged,
The image processing apparatus according to claim 2, wherein candidates for background lightness distribution are listed based on candidates for each partial area stored in the memory when fusion is completed. The background lightness candidate detection unit,
Finding the brightness of each partial area obtained by dividing the original image, generating a brightness histogram showing the relationship between the brightness of each area and the appearance frequency ,
The image processing apparatus according to claim 1 , wherein a peak pixel value on the generated brightness histogram is detected as a background brightness candidate of the original image . The background lightness candidate detection unit,
The image processing apparatus according to claim 4, further comprising a histogram correction unit that uses a histogram obtained by adding a histogram in the vicinity of the partial area to the histogram of the partial area, instead of the histogram of the frame. When the original image is one frame of a moving image, a histogram obtained by adding the brightness histogram of a frame before the frame obtained by the background brightness candidate detection unit to the histogram of the frame is used instead of the histogram of the frame. image processing equipment according to claim 4, characterized by including the histogram addition unit inter-frame used for.   The image processing apparatus according to claim 1, further comprising a supplementing unit that supplements the background brightness candidates of the divided areas with background brightness candidates of neighboring divided areas.   The supplementary part which supplements the background lightness candidate of the said partial area with the set of background lightness candidates in the frame before the said frame when the said original image is one frame of a moving image is provided. Or the image processing device according to any one of 7; An image processing program for causing a computer to execute image processing,
The computer,
The original image is divided into multiple partial regions, and the background brightness candidate detection unit, wherein for each of the partial areas by detecting a representative value of brightness and a plurality of candidates of the brightness of the background in the partial region,
The background brightness detected by the candidate detection unit the brightness of the background plural candidates selected one by one for each of the partial regions, the background to determine the distribution of the brightness of the background of the original image based on the combination of the selected candidate A brightness distribution estimation unit;
And a threshold calculation unit that calculates a threshold value for binarization of the original image from the distribution of the brightness of the background brightness distribution estimating unit by seeking et background,
An image processing program that functions as a binarization unit that binarizes the original image based on the threshold value calculated by the threshold value calculation unit. An image processing method in which an image processing apparatus performs image processing,
The image processing apparatus is
The original image is divided into multiple partial regions, wherein for each of the partial areas by detecting a representative value of brightness and a plurality of candidates of the brightness of the background in the partial region,
A plurality of candidates for the background brightness are selected one by one for each of the partial regions, and a distribution of the brightness of the background of the original image is obtained based on the combination of the selected candidates .
It said calculating a threshold value for binarization of the original image from the distribution of the brightness of the obtained background,
An image processing method, wherein the original image is binarized by a calculated threshold value.

Images