JP2003333331A - Image processing method and image processing apparatus - Google Patents

Abstract

(57) [Summary] [Problem] To automatically adjust the grayscale value of an image obtained by a digital camera or the like to obtain a desired image. SOLUTION: A signal is corrected so that nonlinear distortion of input image data is reduced, and a relative ratio between a pixel value of a pixel of interest and a gray value of a peripheral distribution area of the corrected signal is calculated. . From this relative ratio, the pixel value of the processing target pixel corresponding to the target pixel is determined and output. Further, the relative ratio between the pixel value of the pixel of interest and the gray value of the surrounding distribution area is calculated by making the target area different, and the gain coefficient is calculated according to the size of the target surrounding distribution area. , And each of the obtained relative ratios is multiplied by a predetermined weight coefficient and the gain coefficient to calculate a composite value. The pixel value of the processing target pixel corresponding to the target pixel is determined and output from the composite value.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method and apparatus capable of automatically adjusting the shading of a digital image obtained by a digital camera or the like to obtain a desired contrast.

[0002]

2. Description of the Related Art A color image photographed by a digital camera is affected by the SN level representing the noise ratio in the analog value obtained by a CCD element which is an image pickup element and the conversion accuracy when converting the analog value into a digital value. However, since it is limited to a range narrower than the dynamic range of the pixel density of a natural image actually photographed, there is a tendency that a phenomenon that information in a shaded detail is lost. This tendency is particularly large when an image of a sample in which a bright area and a dark area are mixed in an image is to be captured.

As an improvement, a method of performing contrast enhancement so as to expand the range of the brightness of a digital image from an image part having higher brightness to an image part having lower brightness can be first considered. The conventional method of contrast enhancement is to create a histogram showing the distribution of the brightness values of all pixels that make up the original image, and use the cumulative curve of the histogram as the brightness conversion curve to set the brightness values of the pixels in the original image to the new brightness. There is a histogram equalization method that converts into values and emphasizes the contrast of the image. In this method, since the luminance of pixels in the entire area of the original image is converted to new luminance by the same luminance conversion curve, there may be a part where the contrast is rather lowered.

Further, a method for improving the lightness expression in a dark part by utilizing human perception characteristics (for example, International Publication No. WO
97/45809, Japanese Patent Publication No. 2000-511315) has been proposed. The structure is shown in FIG. Although a grayscale image is described here as an example, it can be extended to a color image. The pixel value I (x, y) at (x, y) of the image is the processor 301 and the filter 302.
And the filtering process is performed.

For each pixel, the processor 301 calculates an adjusted pixel value I '(x, y) as in (Equation 1). Where F
(x, y) is a peripheral visual field function that represents the peripheral visual field, and “*” indicates convolution calculation processing.

Then, the normalization coefficient K is determined so that F (x, y) satisfies the condition of (Equation 2), whereby the second term of (Equation 1) is the pixel value in the peripheral visual field. Corresponding to the average value of. That is, (Equation 1) corresponds to the ratio of the pixel value of each pixel to the average pixel value in a large area, which is logarithmically converted. The peripheral visual field function F (x, y) is designed to have a higher contribution rate as it approaches the target pixel from the correspondence with the human visual model, and a Gaussian function as in (Equation 3) is applied. Here, c is a constant for controlling the adjusted pixel value I ′ (x, y) of each pixel value I (x, y), and *
Represents convolution.

[0007]

[Equation 1]

[0008]

[Equation 2]

[0009]

[Equation 3]

As described above, according to the conventional invention, the target pixel value with respect to the average pixel value in the peripheral visual field is calculated as the adjusted pixel value I '(x, y), and the filter 3 is used for this value.
02 performs filter processing to generate the Retinex output R (x, y) used by the display 303. The filter 302 displays I ′ (x, y) from the logarithmic domain on the display 30.
The pixel values are converted into the R (x, y) pixel value area handled in Section 3, and the processing of applying the same offset and gain conversion function to all pixels is used for the sake of simplicity of the processing.

[0011]

However, when the input characteristics of the image pickup device have nonlinear distortion, the adjustment effect to be processed changes depending on the brightness. As a result, there is a problem of selecting a shooting target. Furthermore, the logarithmic calculation of (Equation 1) has a problem that it strongly depends on the brightness in the image.

As a method for improving the brightness dependence by the logarithmic calculation, H. Kotera et al. Use a method using a linear Retinex model (for example, Color Forum Japan p.151-15).
3 (2001)) has been announced. However, in order to automatically adjust the gradation value of the image, it is necessary to reduce the image dependency due to the brightness, including the input characteristic of the image pickup device and the γ value of the image data.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing method and apparatus capable of automatically adjusting the gray value of an image without depending on the object of the captured image.

[0014]

SUMMARY OF THE INVENTION In an image processing apparatus for inputting image data, in order to solve the above-mentioned problems, a first image processing apparatus of the present invention reduces nonlinear distortion of the input image data. Signal correction means for correcting, relative ratio calculation means for calculating the relative ratio between the pixel value of the pixel of interest and the gray value of the peripheral distribution area of the corrected signal, and the pixel of interest from the relative ratio An output unit that determines and outputs the pixel value of the pixel to be processed.

In order to solve the above-mentioned problems, an image processing apparatus for adjusting the gray value of an input image by using the relative ratio of the pixel value of the pixel of interest and the gray value of its peripheral distribution area is provided. The image processing device of No. 2 is a relative ratio calculation unit that calculates a relative ratio between a pixel value of a pixel of interest and a gray value of a peripheral distribution region thereof by changing a target peripheral visual field region, and a gain that calculates a gain coefficient. Coefficient calculating means, relative ratio combining means for calculating a combined value by multiplying the relative ratio by a predetermined weighting coefficient and the gain coefficient, and determining the pixel value of the processing target pixel corresponding to the pixel of interest from the combined value And output means for outputting. The gain coefficient is calculated according to the size of the target peripheral distribution area.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) First, the first embodiment of the present invention
The image processing apparatus according to the embodiment will be described. FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention. 2A is an explanatory diagram of the input device 100 of the image processing apparatus according to the first embodiment of the present invention,
2B is an explanatory diagram of the distortion correction circuit 1 of the image processing apparatus according to the first embodiment of the present invention, and FIG. 3 is an output conversion circuit of the image processing apparatus according to the first embodiment of the present invention. 3 is an explanatory diagram of the peripheral luminance calculation circuit 4 of the image processing apparatus according to the first embodiment of the present invention. In each figure of the configuration diagram, the same numbers are given to the same parts. In addition, hereinafter, all pixel units are used as the unit of pixel position (x, y).

In FIG. 1, as the image input device 100, for example, a digital still camera (hereinafter referred to as DSC) is used.
Will be used). The signal distortion of the output G (x, y) with respect to the brightness of the image is corrected by the distortion correction circuit 1, and then the relative ratio R of the brightness between the gazing point I (x, y) and its peripheral distribution is R.
Calculate (x, y). The brightness of the peripheral distribution is calculated by the peripheral luminance calculation circuit 4. The relative ratio R (x, y) is converted into a final signal level (for example, a signal level of 0 to 255 of 8 bits) by the output conversion circuit 3.

Gamma correction is added to the sRGB compliant image data and the DSC image data in order to compensate for the non-linearity of the light emission characteristics of the CRT display in advance. Figure 2
When the density adjustment is performed on the non-linear image data such as the curve A shown in (A), the processing effect of the image processing differs with respect to the brightness of the captured image. This is the input (X1)
Degree of change with respect to brightness in the vicinity (Δy1 / ΔX) and degree of change with respect to brightness at input (X2) (Δy2 / ΔX)
Is different. Due to this difference in the degree of change, the dependence on the image increases. In order to automatically adjust the density of a captured image such as DSC, the smaller the degree of dependence on the image, the better, and stable image quality can be obtained.

Therefore, as shown in FIG. 2B, if the distortion of the curve B is corrected so as to cancel the distortion of the curve A, a more linear output can be obtained. For example, sR
If γ = 1 / 2.2 is set in GB, DSC, etc.,
By performing correction of output = (Vi ^{1 / 2.2} ) ^2.2 = Vi and inverse γ = 2.2 on the image data by the distortion correction circuit 1, a linear original image signal Vi is obtained as an output.
Here, the signal is converted into a linear signal by the inverse γ conversion, but it is sufficient to reduce the nonlinear distortion of the input signal, and the same effect can be obtained. Therefore, the converted value is not limited to linear.

The peripheral brightness calculation circuit 4 calculates the brightness B [x, y] of the peripheral distribution of the gazing point from the distortion-corrected output I (x, y) according to (Expression 4) (Expression 5) (Expression 6). To calculate. Here, * represents a convolution.

[0022]

[Equation 4]

[0023]

[Equation 5]

[0024]

[Equation 6]

The image processing circuit 2 calculates the linear Retinex output R (x, y) using (Equation 7).

[0026]

[Equation 7]

The output conversion circuit 3, as shown in FIG.
A predetermined upper and lower limit range is set from the output of R (x, y), and the range is set to a predetermined output signal level (for example, 0 of 8 bits).
Signal level of ~ 255), display,
Output to a printer, etc.

In this linear Retinex model, lo
Since g conversion is not included, high speed processing is possible. Further, since the output change with respect to the brightness in the image is constant regardless of the brightness, the image dependency is small.

As the peripheral visual field function F (x, y), a Gaussian function that contributes to the target pixel at a higher proportion is used from the correspondence with the human visual model, but originally the brightness of the blurred image is calculated. Therefore, a rectangular filter having a flat characteristic may be used instead. The processing speed can be improved by using the rectangular filter.

In addition, the linear model has an effect of converging the brightness of the output image near the reference brightness (A value), and in a white or near-white area, the brightness is reduced and perceived as gray, I feel unnatural. Therefore, as shown in FIG. 4, by clipping and rounding a certain luminance value SL or more of the curve 500 (curve 501), the luminance reduction in the highlight region is prevented. As a result, R (x, y) is (Equation 8)
To be improved.

[0031]

[Equation 8]

The rounding process of this brightness value is performed by the BAR (Blu
r After Round) process. Note that the rounding process is performed once as in (Equation 9), once the input image data I (x,
It is also possible to perform convolutional integration between y) and F (x, y), and clip the output. This brightness value rounding process is called a BBR (Blur Before Round) process.

[0033]

[Equation 9]

FIG. 8 shows a flowchart of processing of the image processing apparatus according to the first embodiment of the present invention. In FIG. 8, in step S101, the distortion of the input image data is corrected. The DSC image data distortion is, for example, distortion of the display γ value, distortion of the CCD characteristic, etc. based on sRGB. In step S103, the relative ratio of brightness between the gazing point (Center) and the surrounding area (Surround) is calculated for the distortion-corrected image data. In step S105,
The calculated relative ratio is limited by the upper limit and the lower limit, and conversion is performed so that the final signal level is assigned to the clipping range. For example, it is assigned to a signal level of 0 to 255 of 8 bits and is output to a display, a printer or the like. When considering the luminance characteristic of the display, γ = 1 / 2.2.
Multiply the value and output. As a result, a visually linear luminance characteristic can be obtained. In step S107, it is determined whether or not the processing of all image data has been completed, and if completed, the processing ends. If not completed, step S10
It returns to 1 and repeats the process.

As described above, according to the image processing method and apparatus of the first embodiment of the present invention, the processing system does not include log conversion, and further correction is performed so that the nonlinear distortion of image data is reduced. By doing so, similar processing results can be obtained at any brightness level. As a result, there is little image dependence, and the image quality adjustment of the dark portion of the image can be automatically adjusted. Furthermore, by clipping and rounding a certain luminance value or more of the input image data, it is possible to prevent the luminance of the highlight area from decreasing.

(Second Embodiment) Next, the second embodiment of the present invention will be described.
The image processing apparatus according to the embodiment will be described. FIG. 5 is a block diagram of an image processing apparatus according to the second embodiment of the present invention. FIG. 6 is an explanatory diagram of a peripheral brightness calculation circuit 9 of the image processing apparatus according to the second embodiment of the present invention,
FIG. 7 is an explanatory diagram of the gain coefficient calculation circuit 7 of the image processing apparatus according to the second embodiment of the present invention. In each figure of the configuration diagram, the same numbers are given to the same parts. In addition, hereinafter, all pixel units are used as the unit of pixel position (x, y).

In FIG. 5, as the image input device 100, for example, a digital still camera (hereinafter referred to as DSC) is used.
Will be used). The signal distortion of the output G (x, y) with respect to the brightness of the image is corrected by the distortion correction circuit 1, and then the relative ratio R of the brightness between the gazing point I (x, y) and its peripheral distribution is R.
Calculate (x, y). In consideration of the color balance, the monochromatic processing circuit 5 monochromaticizes the color signal as shown in (Equation 10), and the peripheral luminance calculation circuit 9 calculates the brightness of the peripheral distribution from the monochromatic signal Y (x, y). To be done. In the second image processing circuit, Ii (x, y) / {Fs (x, y) * Y (x, y)} is multiplied by the weighting coefficient Ws and the gain coefficient A (σs) to synthesize (Equation 11). Output the relative ratio R (x, y) of. The combined relative ratio R (x, y) is converted by the output conversion circuit 3 into a final signal level (for example, an 8-bit signal level of 0 to 255). Where Ws
= 1 / M, σs = 2 ^m , and A (σs) = σs / σ _M = 2 ^{s -m} .

[0038]

[Equation 10]

[0039]

[Equation 11]

The peripheral brightness calculation circuit 9 uses a plurality of visual field functions.
The Fs (x, y) output is calculated, and the second image processing circuit 8 synthesizes a relative ratio of brightness from a plurality of different field-of-view function Fs (x, y) outputs, so that R ( x, y) can be calculated.

As the visual field function Fs (x, y), a Gaussian function that contributes to the target pixel in a higher proportion from the correspondence with the human visual model may be used. Since the brightness of the blurred image is originally calculated, A rectangular filter with flat characteristics may be used instead. As rectangular filters, filters 200 to 200 having different sizes of target image areas as shown in FIG.
The filter 203 is used. Alternatively, Gaussian functions having different σ values may be used. FIG. 7 shows the relationship between the σs value and the gain coefficient A. The gain coefficient calculation circuit 7 calculates the gain coefficient A according to the σs value.

FIG. 9 shows a flowchart of the processing of the image processing apparatus according to the second embodiment of the present invention. In FIG. 9, steps S201 to S211 are improvements of the part for calculating the relative ratio of brightness in step S103 of the flowchart of the image processing apparatus according to the first embodiment of the present invention shown in FIG. It is also expanded to color. In step S201, the distortion-corrected color image data is subjected to monochromatic processing in consideration of color balance. Monochromatic signal Y (x, y) is Y (x, y) = Kr × Ir (x, y) + K
g × Ig (x, y) + Kb × Ib (x, y), Kr = 0.299, Kg = 0.587, Kb =
It becomes 0.114. In step S203, the monochrome signal Y (x, y)
From this, a plurality of different field functions Fs (x, y) are calculated. In step S205, the gain coefficient A (σs) is calculated according to the size of the area of the visual field function Fs (x, y). For example, A (σs) = σ
s / σ _M = 2 ^sm .

In step S207, the weighting coefficient Ws value (Ws = 1 / M) equally divided by the number M of the visual field function Fs (x, y) is calculated. In step S209, Ii (x, y) / {Fs (x, y) *
The relative ratio of brightness is calculated by multiplying Y (x, y)} by the weighting coefficient Ws and the gain coefficient A (σs). In step S211,
A value obtained by combining the relative ratios of brightness is calculated by (Equation 11). Upon completion of the above processing, the process proceeds to step S105 in the flowchart of the image processing apparatus according to the first embodiment of the present invention shown in FIG. 8, and the combined relative ratio is converted into the final signal level. For example, 8-bit 0 to 2
The signal level of 55 is output to a display, a printer or the like. When considering the brightness characteristics of the display, γ =
Multiply by 1/2 value and output. As a result, a visually linear luminance characteristic can be obtained. In step S107,
It is determined whether or not the processing of all image data has been completed, and if completed, the processing ends. If not completed, the process returns to step S101 to repeat the process. When step S103 is executed, the process proceeds to step S201.

The gain coefficient calculation circuit 7 calculates the gain coefficient A (σs) corresponding to the σs value as the blurred image Ss (x, y, σs) =
Maximum value Smax in the image when {Fs (x, y) * Y (x, y)}
It may be calculated from (s) and the minimum value Smin (s). For example, A (σ
s) = Smin (s) / Smin (M), A (σs) = 1-0.5 × {Smin (s)
+ Smax (s)}, A (σs) = 1-Smax (s) and so on. Here, M is the number of field functions Fs.

As described above, according to the image processing method and apparatus of the second embodiment of the present invention, a plurality of visual field functions, a plurality of visual field functions Fs (x, y) outputs are calculated, and a plurality of different visual fields are obtained. Since the relative ratio of brightness obtained from the output of the function Fs (x, y) is combined, the relative ratio R (x, y) with less image dependency can be calculated.

To calculate the relative ratio, the color image is once converted into a single color and the blurred image is used.
Color balance can be maintained by calculating the relative ratio of G and B independently. As a result, it is possible to automatically adjust the density value in the image without depending on the image while improving the appearance of the dark portion of the image.

In the first and second embodiments described above, all the image data input from the image input means are corrected by the signal correction means (distortion correction circuit) so that the nonlinearity is reduced. However, the present invention is not limited to this.

That is, the distortion correction may be applied to a nonlinear distortion, a device having a comma value for brightness, or an image file, and need not be applied to a linear device or an image file that does not require gamma correction. Therefore, in advance, whether the device has a non-linear distortion or whether the image file has a gamma value is judged by the distortion judging means, and for the input image having a distortion,
The distortion correction circuit may be configured to selectively apply the distortion correction. As the determination information, for example, sRB described in the camera image file standard
There is color space information such as G and gamma information. Thereby, the device distortion and the gamma value of the image file can be automatically corrected.

The processing of the present invention is performed by a central processing unit (CP) used in a computer or the like according to the image processing method.
U) and software processing using a digital signal processor (DSP) or the like can be similarly realized.

Further, the software medium can be downloaded from the outside via a communication line, and the software processing in a local computer can be similarly realized.

[0051]

As described above, according to the present invention, the gray value of an image can be automatically adjusted without depending on the object of the picked-up image.

In particular, according to the image processing method and apparatus according to the first embodiment of the present invention, the processing system does not include log transformation, and further correction is made so that the non-linear distortion of the image data is reduced. , The same processing result can be obtained at any brightness level. As a result, it is possible to automatically adjust the image quality of the dark portion of the image with little image dependency. Furthermore, by clipping and rounding a certain luminance value or more of the input image data, it is possible to prevent the luminance of the highlight area from decreasing.

Further, according to the image processing method and apparatus of the second embodiment of the present invention, a plurality of visual field functions Fs (x, y) outputs are calculated, and a plurality of different visual field functions Fs ( x,
y) Since the relative ratios of brightness obtained from the output are combined, the relative ratio R (x, y) with little image dependence can be calculated.

To calculate the relative ratio, the color image is once converted into a single color and the blurred image is used.
Color balance can be maintained by calculating the relative ratio of G and B independently. As a result, it is possible to automatically adjust the density value in the image without depending on the image while improving the appearance of the dark portion of the image.

[Brief description of drawings]

FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention.

2A is an explanatory diagram of an input device 100 of the image processing apparatus according to the first embodiment of the present invention, and FIG. 2B is a distortion correction of the image processing apparatus according to the first embodiment of the present invention. Explanatory drawing of circuit 1

FIG. 3 is an explanatory diagram of an output conversion circuit 3 of the image processing apparatus according to the first embodiment of the present invention.

FIG. 4 is an explanatory diagram of a peripheral luminance calculation circuit 4 of the image processing apparatus according to the first embodiment of the present invention.

FIG. 5 is a block diagram of an image processing apparatus according to a second embodiment of the present invention.

FIG. 6 is an explanatory diagram of a target area of the image processing apparatus according to the second embodiment of the present invention.

FIG. 7 is an explanatory diagram of parameters of the image processing apparatus according to the second embodiment of the present invention.

FIG. 8 is a flowchart of processing of the image processing apparatus according to the first embodiment of the present invention.

FIG. 9 is a flowchart of processing of the image processing apparatus according to the second embodiment of the present invention.

FIG. 10 is a block diagram showing a configuration of an image processing apparatus in a conventional example.

[Explanation of symbols]

1 Distortion correction circuit 2 Image processing circuit 3 output conversion circuit 4 Ambient luminance calculation circuit 5 Monochromatic circuit 6 Weighting factor 7 Gain coefficient calculation circuit 8 Second image processing circuit 300 Digital Imaging Device 301 processor 302 Filter 303 display

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Tatsumi Watanabe             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. (72) Inventor Yasuhiro Kuwahara             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. (72) Inventor Yusuke Monobe             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. (72) Inventor Hirotaka Oku             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. (72) Inventor Hiroshi Kodera             3-20-20 Oyumino Chuo, Midori Ward, Chiba City, Chiba Prefecture F term (reference) 5B057 BA11 CA08 CA16 CB08 CB16                       CC02 CE06 CE11 CH01 DC22                 5C021 PA77 XA03 XA35                 5C077 MM03 MP01 MP08 PP01 PP15                       PP32 PP37 PP54 PQ03 PQ12                       PQ18 TT09

Claims (6)

[Claims] 1. An image processing apparatus for inputting image data, wherein a signal correcting means for correcting the input image data so as to reduce non-linear distortion, and a pixel value of a pixel of interest and its value for the corrected signal. A relative ratio calculating means for calculating a relative ratio to the gray value of the peripheral distribution area; and an output means for determining and outputting a pixel value of a processing target pixel corresponding to the pixel of interest from the relative ratio. Image processing device. 2. An image processing apparatus for adjusting a gray value of an input image using a relative ratio between a pixel value of a pixel of interest and a gray value of a peripheral distribution region thereof, wherein the pixel value of the pixel of interest and its peripheral distribution region are adjusted. Relative ratio calculation means for calculating the relative ratio with the gray value by changing the target peripheral visual field region, gain coefficient calculation means for calculating the gain coefficient, and a predetermined weighting coefficient and gain coefficient for the relative ratio. And an output unit that determines the pixel value of the pixel to be processed corresponding to the pixel of interest from the composite value and outputs the composite value. . 3. The image processing apparatus according to claim 2, wherein the gain coefficient is calculated according to the size of the peripheral distribution area of interest. 4. An image processing method for inputting image data, the step of correcting so that the nonlinear distortion of the input image data is reduced, and the pixel value of the pixel of interest and its peripheral distribution with respect to the corrected signal. An image processing method comprising: a step of calculating a relative ratio to a gray value of a region; and a step of determining a pixel value of a processing target pixel corresponding to a target pixel from the relative ratio and outputting the pixel value. 5. An image processing method for adjusting a gray value of an input image using a relative ratio between a pixel value of a pixel of interest and a gray value of a peripheral distribution region thereof, wherein the pixel value of the pixel of interest and its peripheral distribution region are adjusted. A step of calculating the relative ratio with the gray value by changing the target peripheral visual field region, a step of calculating the gain coefficient, and a step of calculating the relative ratio by a predetermined weighting coefficient and the gain coefficient, respectively, to obtain a composite value. And a step of determining a pixel value of a pixel to be processed corresponding to the pixel of interest from the combined value and outputting the pixel value. 6. The image processing method according to claim 5, wherein the gain coefficient is calculated according to the size of the target peripheral distribution area.