JP4274426B2 - Image processing method, apparatus, and program - Google Patents

Description

  The present invention relates to image processing, specifically, processing for obtaining a noise level in an image, an image processing method and apparatus for performing noise suppression processing on an image using the obtained noise level, and a program therefor.

  In recent years, digital still cameras (hereinafter referred to as digital cameras) are rapidly spreading. Instead of imaging light information on a silver salt film, a digital camera forms an image on a digital device (CCD, photomultiplier tube, etc.) and stores the image on a recording medium as digital data (hereinafter referred to as a digital image). Since it can be used directly on a computer, it is convenient for image processing and storage.

  Also, an image obtained by imaging light information on a silver salt film by a conventional imaging method, or an image obtained by printing on a printing medium such as photographic paper or paper is read by a reading device such as a scanner. Digital data is stored in the same way as a digital image acquired by a digital camera, or subjected to image processing.

  Such digital images are reproduced by being displayed on a display device such as a monitor or printed on photographic paper. However, since a high-quality image is expected for the reproduced image, the digital image It is necessary to perform various image processing on the image.

  Among these image processes, a process for suppressing noise in an image greatly affects the image quality of the processed image, and various methods have been proposed. Normally, a low-pass filter (LPF) can be applied to suppress noise, but the LPF can suppress noise components from the image, but deteriorates the edge portion in the image signal and blurs the entire image. Therefore, it is desirable to grasp the noise level in the image and suppress noise by adjusting the degree of noise suppression processing according to this noise level, and to prevent deterioration of the edge portion. .

  Conventionally, as a scale for evaluating the noise level of an image, RMS granularity using a standard deviation of density change or a Wiener spectrum obtained by Fourier transform of the density change is used. A method for performing noise suppression processing by evaluating noise using these scales is, for example, taking a chart having no shading change using a digital camera, and obtaining RMS granularity or winner spectrum from the obtained image. When acquiring noise characteristics of a digital camera and performing noise suppression processing on an image acquired later by this digital camera, the noise level in the image is predicted based on the noise characteristics of the digital camera acquired in advance. Processing is done.

  In addition, a method has been proposed in which a flat portion is extracted from an image itself to be processed, and the noise level of the flat portion is obtained to obtain the noise level of the image.

    On the other hand, in the image, the luminance component that constitutes the edge component of the image greatly contributes to human vision, and when performing noise suppression processing on the image, the noise level of the luminance component is obtained and the noise level of the image is determined. It is desired to perform noise suppression processing after evaluation. In Patent Document 1, a directional differentiation process is performed for each pixel in an image to extract an edge component at the pixel position, and an edge or noise is determined according to the degree of the edge to determine the degree of noise suppression processing. It has been proposed to perform noise suppression processing after adjustment.

In addition, as a method of noise suppression processing, for example, by utilizing the fact that most of noise components exist as small amplitude signals in high frequency components of an image, high frequency noise components of small amplitude in the image are separated and suppressed. There is a method using a two-dimensional ε-filter. As described above, the LPF that is usually used for noise suppression processing not only suppresses noise components but also degrades the edges of the signal. Therefore, when the image is targeted, the entire image is blurred. However, the ε-filter has a characteristic of flattening only a small amplitude level change in a signal waveform, and an edge with a steep level change is preserved even when applied to an image. It has the characteristic that the overall sharpness is hardly impaired. The ε-filter basically operates so as to subtract a value obtained by applying a nonlinear transformation function to the amount of change in the amplitude level of the high-frequency component of the image from the original image signal. This nonlinear conversion function is a function that sets the output to 0 when the amplitude of the signal is larger than a predetermined threshold value. That is, when the ε-filter is applied, the output of the non-linear transformation function is 0 in a part of the image where the amplitude is larger than the threshold value, and the original signal of the part is retained in the processed image. On the other hand, in a portion where the amplitude is equal to or less than the above-described threshold, in the processed image, the signal value of the portion is a value obtained by subtracting the output of the nonlinear transformation function (the absolute value is greater than 0) from the original signal value. By doing so, a portion with noise is smoothed and an edge portion with a large amplitude can be held. The threshold value used in this method is a parameter for determining edge or noise.
Japanese Patent Laid-Open No. 10-3539

  However, the noise level of an image obtained by imaging a chart with an image acquisition device such as a digital camera is only the noise characteristic of the image acquisition device, and is natural such as a snapshot image acquired under various image acquisition conditions. The noise level of the image cannot be reflected. For example, it is not possible to correctly evaluate the noise level of an underexposed image using the noise level obtained from an image obtained by imaging a chart such as a sample image under standard exposure conditions. It is difficult to obtain the effect of the suppression process.

  In addition, the noise level of the flat part in the image is only the noise level of the flat part. In the method of obtaining the noise level of the flat part for each image and obtaining the noise level of the entire image, other than the flat part such as a part with a pattern. The noise level cannot be evaluated correctly.

  In the method described in Patent Document 1, an edge component is extracted and processing is performed by determining whether the edge is noise or noise. However, the luminance component is an edge component because the edge component and the noise component are mixed. It is difficult to determine whether it is a noise component. Also, with this method, since the processing is performed locally for each pixel, the noise level of the entire image cannot be determined.

  In addition, in the noise suppression processing method using the ε-filter, it is necessary to determine whether the edge is noise or noise from the threshold, and if this threshold is not appropriate, the degree of noise suppression is insufficient and a large amount of noise components remain, Since noise suppression is so strong that it has an adverse effect such as smoothing edges, it is important to determine the noise level of the entire image and set an appropriate threshold.

  Although many noise components exist in the high frequency band, they exist over each frequency band from the high frequency band to the low frequency band. For example, a noise suppression processing method such as a method using an ε-filter extracts a noise component only in one frequency band and subtracts the component extracted from the original image, and thus eliminates the noise component. I can't. In addition, in order to improve the noise suppression effect, the method of extracting the noise component in one frequency band has only to increase the filtering in this one frequency band, that is, to increase the above-described threshold value, so that an artifact is generated. There is a problem that the image quality of the processed image is not good.

  The present invention has been made in view of the above circumstances, and can appropriately evaluate the noise level of the entire image, and can effectively remove noise and obtain an image with good image quality. It is an object to provide a method and apparatus and a program therefor.

In the first image processing method according to the present invention, when obtaining the noise level of a digital image, the color difference noise level in the color difference component of the digital image is calculated as the noise level , and a band-limited image to be described later is then created. Then, the noise level is obtained from the band limited image .

  Here, the “digital image” means not only a digital image obtained by imaging a subject with a digital camera or the like but also an image on a silver halide photographic film or printed matter (for example, a print) by a reading device such as a scanner. The obtained digital image is also included. Hereinafter, for convenience of explanation, an image simply means a digital image.

  As described above, the luminance component of the image includes the edge component and the noise component, and it is difficult to distinguish the edge component from the noise component. On the other hand, there is a problem in that the noise level of the entire image cannot be evaluated by the method of evaluating the noise level of the image using an image of a flat portion with few edge components in the image. On the other hand, an image, particularly a natural image such as a snapshot image, has a characteristic that there are few edge components although noise exists in the color difference component. The image processing method of the present invention pays attention to this point, and calculates the noise level in the color difference component in the image, that is, the color difference noise level to obtain the noise level of the image.

  Further, as described above, the luminance component greatly contributes to human vision, and a large image quality improvement effect can be obtained by performing noise suppression processing on the luminance component. Therefore, it is desirable that the noise level in the luminance component can be directly grasped. In the image processing method of the present invention, it is preferable that a luminance noise level of the digital image is estimated from the color difference noise level, and the luminance noise level is set as a noise level of the digital image.

  Since images generated by an imaging device such as a digital camera or a reading device such as a scanner have the same cause of noise in luminance and color difference in principle, the relationship between luminance and color difference is used to make color difference noise. The luminance noise level can be estimated from the level. Specifically, for example, a YCrCb space as a space for luminance and color difference is defined by the following equation (1).

Y = 0.299 × R + 0.587 × G + 0.114 × B
Cr = 0.500 × R−0.419 × G−0.081 × B (1)
Cb = −0.169 × R−0.331 × G + 0.500 × B
Y: luminance
Cr, Cb: Color difference
R, G, B: R value, G value, B value

Here, for example, if the noise level of each component is the RMS value, that is, the standard deviation σ of that component, the relationship shown in the following equation (2) is established.

(ΣY) ² = (0.299 × σR) ² + (0.587 × σG) ²
+ (0.114 × σB) ²
(ΣCr) ² = (0.500 × σR) ² + (0.419 × σG) ² (2)
+ (0.081 × σB) ²
(ΣCb) ² = (0.169 × σR) ² + (0.331 × σG) ²
+ (0.500 × σB) ²
Where σ: standard deviation

On the other hand, in an apparatus that acquires an image, it is unlikely that the amount of noise generated in each color pixel of the image sensor is greatly different. That is, if the noise levels in R, G, and B colors are σR, σG, and σB, As shown in Equation (3), it is considered that the noise amount of each color is substantially the same.

σR≈σG≈σB (3)
Where σR, σG, σB: Noise amount

The following equation (4) can be further deduced from the equations (2) and (3).

σY≈1.02σCr≈1.07σCb (4)
Where σY: Luminance noise level
σCr, σCb: Color difference noise level

That is, if the color difference noise level σCr or σCb is known, the luminance noise level σY can be estimated according to the equation (4).

  Further, for example, when the LC1C2 space defined by the following equation (5) is used as the luminance color difference space, the relationship between the luminance noise level and the color difference noise level as in the equation (6) as in the above estimation. Can be obtained.

L = 0.299 × R + 0.587 × G + 0.114 × B
C1 = −0.299 × R−0.587 × G + 0.886 × B (5)
C2 = 0.701 × R−0.587 × G−0.114 × B
Where L: Luminance
C1, C2: Color difference

σL≈σC1 / 1.65≈σC2 / 1.38 (6)
Where σL: Luminance noise level
σC1, σC2: Color difference noise level

Further, the above equations (4) and (6) for estimating the luminance noise level from the color difference noise level are based on the assumption that the components of each color of R, G, and B receive no gain or the same gain. R, G, and B differ in the case where white balance processing is performed or the sensitivity setting of each color pixel is different in an image acquisition device. Since there is a possibility of receiving a gain, when estimating the luminance noise level from the chrominance noise level, rather than estimating the luminance noise level from one chrominance noise level, as shown in equations (7) and (8) It is preferable to estimate the luminance noise level using the average value of the two color difference noise levels.

σY≈ (1.02 × σCr + 1.07 × σCb) / 2 (7)
Where σY: Luminance noise level
σCr, σCb: Color difference noise level


σL≈ (σC1 / 1.65 + σC2 / 1.38) / 2 (8)
Where σL: Luminance noise level
σC1, σC2: Color difference noise level

Further, for example, if the gain received by each of the R, G, and B colors of the image is determined as shown in Expression (9) from the transmittance ratio of the color filter of the apparatus that acquires the image and the image processing conditions such as white balance processing, More preferably, the luminance noise level is estimated from the color difference noise level as shown in the following formulas (10) and (11) according to the gain.

R: G: B = g _r : 1.0: g _b (9)
Where g _r , g _b : gain

Here, the YCrCb space and the LC1C2 space are taken as examples of the luminance / color difference space, but the luminance / chrominance space such as the L * a * b space similarly has a predetermined relationship between the color difference noise level and the luminance noise level. The luminance noise level can be estimated from the color difference noise level even for the luminance color difference space.

Further, noise in an image has different influences depending on the frequency band so that the S / N ratio is lower as the frequency band is higher, and the S / N ratio is higher as the frequency band is lower. When the S / N ratio is high, the degree of mixing of the original signal and noise of the image becomes high and it becomes difficult to evaluate the noise level. Therefore, it is desirable to acquire the noise level in the high frequency band, but the image is acquired. Since most of the images are output after being subjected to image processing such as JPEG compression processing and sharpness correction processing, the information for acquiring the noise level in the highest frequency band is lost. The noise level acquired in this frequency band is not always appropriate. Therefore, when acquiring the noise level from the image, it is considered appropriate to acquire the noise level in a high frequency band other than the highest frequency band, but the image output from the apparatus that acquires the image is enlarged. There is a case where a reduction process, that is, a scaling process is performed. In the image subjected to the scaling process, the frequency band suitable for obtaining the noise level also changes. Therefore, the first image processing method according to the present invention is an image changing process. in accordance with the magnification, determines the frequency band to determine the noise level, based on the image, to create a band-limited image representing the component of the frequency band of said image, the determination of the said noise level from band-limited image It is a feature .

The second image processing method according to the present invention has to perform noise suppression processing on the digital image based on the noise level, this time, the degree of the noise suppression processing in response to the noise level The process is performed so as to adjust.

That is, the second image processing method according to the present invention is more specifically described as follows:
As the noise suppression processing, and the processing on the basis of the digital image, creating a plurality of band-limited image representing a component of each of the plurality of frequency bands of the digital image,
For an input value in which the absolute value of the output value is less than or equal to the absolute value of the input value and the absolute value is less than or equal to a predetermined threshold value, the absolute value of the output value increases as the absolute value of the input value increases. On the other hand, for an input value whose absolute value is larger than the predetermined threshold value, nonlinear conversion processing in which the absolute value of the output value is equal to or smaller than the absolute value of the output value corresponding to the predetermined threshold value is performed for each band limitation. Processing for each pixel value of the image to obtain a plurality of converted images;
A process of multiplying a pixel value of the plurality of converted images by a predetermined subtraction coefficient;
Wherein the pixel values of the digital image, performs the subtraction coefficient by subtracting the pixel values of the multiplied plurality of converted images to obtain a pixel value of the processed image processing ing from the processing,
On top of that, as an output value corresponding to the predetermined threshold value and / or the predetermined threshold value, and wherein the use of what was determined according to the noise level.

  In this case, the noise level may be obtained for each frequency band corresponding to the band limited image, or the same noise level may be applied to each band limited image.

A first image processing apparatus according to the present invention is an image processing apparatus having noise level acquisition means for obtaining a noise level of a digital image,
The noise level acquisition means includes color difference noise level calculation means for calculating a color difference noise level in a color difference component of the digital image, and the color difference noise level is configured to be a noise level of the digital image ,
In addition, for an image that has been scaled at a predetermined scaling ratio as the digital image,
A frequency band determining means for determining a frequency band for obtaining the noise level according to the scaling factor;
Band-limited image creation means for creating a band-limited image representing the frequency band component of the digital image based on the digital image;
It said noise level acquisition means is characterized in der Rukoto seeks the noise level using band-limited image.

  In addition, the noise level acquisition unit further includes a luminance noise level estimation unit that estimates a luminance noise level of the digital image from the color difference noise level calculated by the color difference noise level calculation unit, and the luminance noise level estimation It is preferable that the luminance noise level estimated by the means is the noise level of the digital image.

Second image processing apparatus according to the present invention, which further comprises a noise suppression processing means for performing noise suppression processing on the digital image, the noise suppression processing means, said noise according to the noise level Ru is intended to adjust the degree of suppression processing.

That is, in the second image processing apparatus according to the present invention, more specifically,
The noise suppression processing means, on the basis of the digital image, the band-limited image generating means for generating a plurality of band-limited image representing a component of each of the plurality of frequency bands of the digital image,
For an input value in which the absolute value of the output value is less than or equal to the absolute value of the input value and the absolute value is less than or equal to a predetermined threshold value, the absolute value of the output value increases as the absolute value of the input value increases. On the other hand, for an input value whose absolute value is larger than the predetermined threshold value, nonlinear conversion processing in which the absolute value of the output value is equal to or smaller than the absolute value of the output value corresponding to the predetermined threshold value is performed for each band limitation. Non-linear conversion means for applying to each pixel value of the image to obtain a plurality of converted images;
A pixel of the processed image by multiplying a pixel value of the plurality of converted images by a predetermined subtraction coefficient and subtracting a pixel value of the plurality of converted images multiplied by the subtraction coefficient from a pixel value of the digital image is a shall such a frequency suppression means for obtaining a value,
It is those output values corresponding to the predetermined threshold value and / or the predetermined threshold value is determined according to the noise level.

  You may provide the image processing method of this invention as a program which makes a computer perform.

  According to the image processing method and apparatus of the present invention, a color difference noise level is acquired from an image and used as an image noise level. The color difference component in the image includes a noise component, but since the edge component is small, the noise level of the entire image can be appropriately evaluated based on the color difference noise level of the image.

  In addition, by estimating the luminance noise level from the color difference noise level of the image, it is possible to know the noise level in the luminance component of the image, and based on this, appropriate noise suppression processing of the luminance component that greatly contributes to human vision It is possible to do.

    Further, the image processing method and apparatus of the present invention may not only use the obtained noise level for image quality evaluation of the image, but also perform noise suppression processing on the image based on the noise level. At this time, as a desirable noise suppression process, first, a nonlinear conversion process is performed on a plurality of band limited images representing components for a plurality of different frequency bands of the image created based on the image to be processed. Get converted image. This non-linear conversion process is a process of making the absolute value of the output value less than or equal to the absolute value of the input value, and for an input value whose absolute value is less than or equal to a predetermined threshold value, the larger the absolute value of the input value, While the absolute value of the value is increased, for an input value whose absolute value is greater than a predetermined threshold, the absolute value of the output value is equal to or less than the absolute value of the output value corresponding to the predetermined threshold. Since the threshold value and / or the output value corresponding to the threshold value are determined in accordance with the noise level of the image, the converted image obtained by the nonlinear conversion process includes noise in the frequency band to which the converted image corresponds. Appropriate representation of ingredients. Then, the pixel values of the plurality of converted images are multiplied by a predetermined subtraction coefficient, and the pixel values of the processed image are obtained by subtracting from the pixel values of the original photographic image. The component can be effectively removed.

  In addition, in the conventional technique for extracting and suppressing noise components in one frequency band, it is necessary to enhance nonlinear processing in this one frequency band in order to increase the noise suppression effect, and artifacts are generated in the processed image. However, there was a problem of image quality degradation. On the other hand, in the present invention, when noise suppression processing is performed, if noise components are removed from a plurality of frequency bands, a noise removal effect that does not require so much enhancement of nonlinear processing in each frequency band is obtained. As a result, artifacts can be prevented and processed images with good image quality can be obtained.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus of the present invention performs correction processing for suppressing noise in an image on an image, and a correction processing program read into an auxiliary storage device is stored in a computer (for example, a personal computer). Realized by executing above. The correction processing program is stored in an information storage medium such as a CD-ROM or distributed via a network such as the Internet and installed in a computer.

  Further, since the image data represents an image, the following description will be given without particularly distinguishing the image from the image data.

  As shown in FIG. 1, the image processing apparatus according to the present embodiment has an image input unit 1 that inputs an image D0 (R0, G0, B0) that has been enlarged / reduced at a predetermined enlargement / reduction rate α (including α = 1). YCC conversion means 5 for performing YCC conversion on image D0 to obtain image S0 (Y0, Cb0, Cr0), and a plurality of blurred images S1, S2,... A blurred image creating means 10 for creating Sn (n: integer of 3 or more) for each luminance Y, color difference Cb, and color difference Cr, and a plurality of band limited images using the image S0 and the blurred images S1, S2,. Band-limited image creating means 20 for creating T1, T2,... Tn for each luminance / color difference component, and band-limited images Tpb and Tpr for color difference components for analyzing noise from each band-limited image are determined and determined. Band-limited image Tpb , Tpr to analyze the noise level in the image S0, and noise correction is performed on the image S0 based on the noise level obtained by the noise analysis unit 30 to obtain a noise-corrected image S′0. Noise correction means 40 for obtaining (Y′0, C′r0, C′b0). Hereinafter, details of each component of the present embodiment will be described.

  The image input means 1 inputs the image D0 (R0, G0, B0), and also acquires the enlargement / reduction ratio α of the enlargement / reduction process performed on the image D0 from the attached information of the image D0, etc., and the noise analysis means Output to 30.

  The YCC conversion means 5 converts the R, G, and B values of the image data D0 into the luminance value Y and the color difference values Cb and Cr according to the above-described equation (1).

Y = 0.299 × R + 0.587 × G + 0.114 × B
Cr = 0.500 × R−0.419 × G−0.081 × B (1)
Cb = −0.169 × R−0.331 × G + 0.500 × B
Y: luminance
Cr, Cb: Color difference
R, G, B: R value, G value, B value

The blurred image creating unit 10 creates a plurality of blurred images for each luminance color difference component of the image S0 using the image S0 obtained by the YCC converting unit 5. Note that the blurred image creating unit 10 creates a blurred image for each component by the same method. Here, the operation of the blurred image creating unit 10 will be described by taking a luminance component as an example. FIG. 2 is a block diagram illustrating a configuration of the blurred image creation unit 10. As shown in the figure, the blurred image creation means 10 includes a filtering means 12 that performs filtering processing to obtain filtered images B1, B2,... Bn, an interpolation means 14 that performs interpolation processing on each filtering image, and filtering. The control means 16 which controls the means 12 and the interpolation means 14 is provided. The filtering means 12 performs a filtering process using a low-pass filter. As the low-pass filter, for example, a filter F substantially corresponding to a 5 × 1 grid-shaped one-dimensional Gaussian distribution as shown in FIG. 3 is used. be able to. This filter F has σ = 1 in the following equation (12).

  The filtering unit 12 performs a filtering process on the entire image to be processed by performing a filtering process on the image to be processed in the x direction and the y direction of the pixel using such a filter F.

FIG. 4 shows details of processing that the control unit 16 of the blurred image creation unit 10 causes the filtering unit 12 and the interpolation unit 14 to perform. As illustrated, the filtering unit 12 first performs a filtering process on the luminance component Y0 of the image S0 every other pixel using the filter F shown in FIG. A filtering image B1 (Y1) is obtained by this filtering process. The size of the filtered image B1 is 1/4 of the size of the image S0 (1/2 in the x direction and y direction, respectively). Next, the filtering unit 12 performs the filtering process by the filter F every other pixel on the filtering image B1 (Y1) to obtain the filtering image B2 (Y2). The filtering means 12 repeats the filtering process by such a filter F, and obtains n filtered images Bk (k = 1 to n). The size of the filtering image Bk has become a ^{1/2 2K} of the size of the image S0. FIG. 5 shows frequency characteristics of each filtered image Bk obtained by the filtering unit 12 when n = 3 as an example. As shown in the figure, the response of the filtered image Bk is such that the higher the k, the higher the frequency component is removed.

  In the image processing apparatus according to the present embodiment, the filtering unit 12 performs the filtering process on the x direction and the y direction of the image by the filter F shown in FIG. A filtering process may be performed on the image S0 and the filtered image Bk at a time using a × 5 two-dimensional filter.

The interpolation unit 14 performs an interpolation process on each filtering image Bk obtained by the filtering unit 12 so that the size of each filtering image Bk is the same as that of the image S0. There are various interpolation processing methods such as a B-spline method. In this embodiment, the filtering unit 12 uses a filter F based on a Gaussian signal as a low-pass filter. A Gaussian signal is also used as an interpolation coefficient for performing the interpolation calculation, and this interpolation coefficient is approximated to σ = 2 ^K−1 in the following equation (13).

When interpolating the filtering image B1, since k = 1, σ = 1. The filter for performing interpolation when σ = 1 in the above equation (13) is a 5 × 1 one-dimensional filter F1 as shown in FIG. The interpolating unit 14 first expands the filtering image B1 to the same size as the image S0 by interpolating the pixels having a value of 0 every other pixel one by one with respect to the filtering image B1, and then the enlarged image A blurred image S1 is obtained by performing a filtering process using the filter F1 shown in FIG. The blurred image S1 has the same number of pixels as the image S0, that is, the same size as the image S0.

  Here, the filter F1 shown in FIG. 7 is a 5 × 1 filter. However, before applying the filter F1, pixels having a value of 0 are interpolated with respect to the filtered image B1 every other pixel. The interpolation processing by the means 14 is substantially performed by two types of filters: a 2 × 1 filter (0.5, 0.5) and a 3 × 1 filter (0.1, 0.8, 0.1). Equivalent to filtering.

  When the interpolation means 14 performs interpolation on the filtered image B2, k = 2, so σ = 2. In the above equation (13), the filter corresponding to σ = 2 is an 11 × 1 one-dimensional filter F2 shown in FIG. The interpolation means 14 first expands the filtered image B2 to the same size as the image S0 by interpolating three pixels each having a value of 0 every other pixel with respect to the filtered image B2, and then expands the image into an enlarged image. On the other hand, a filtering process is performed by the filter F2 shown in FIG. 8 to obtain a blurred image S2. The blurred image S2 has the same number of pixels as the image S0, that is, the same size as the image S0.

  Similarly, the filter F2 shown in FIG. 8 is an 11 × 1 filter, but before applying the filter F2, three pixels each having a value of 0 are interpolated with respect to the filtered image B2 by three. Therefore, the interpolation processing by the interpolation means 14 is substantially performed by a 2 × 1 filter (0.5, 0.5) and a 3 × 1 filter (0.3, 0.65, 0.05), ( This is equivalent to filter processing using four types of filters (0.3, 0.74, 0.13) and (0.05, 0.65, 0.3).

In this way, the interpolation means 14 interpolates (2 ^K −1) pixels each having a value of 0 every other pixel for each filtering image Bk, so that the filtering image Bk has the same size as the image S0. And filtering by a filter having a length of (3 × 2 ^K −1) created based on the above equation (13) with respect to the filtered image Bk interpolated with pixels having a value of 0 Processing is performed to obtain a blurred image Sk.

  FIG. 9 shows the frequency characteristics of each blurred image Sk obtained by the blurred image creating means 10 when n = 3 as an example. As shown in the figure, the blurred image Sk is such that the higher the k, the higher the frequency component of the image S0 is removed.

  The band limited image creating means 20 uses the blurred images S1, S2,... Sn obtained by the blurred image creating means 10 to calculate components for a plurality of frequency bands of the image S0 according to the following equation (14). Band-limited images T1, T2,.

Tm = S (m−1) −Sm (14)
Where m is an integer between 1 and n

FIG. 10 shows the frequency characteristics of each band limited image Tm obtained by the band limited image creating means 20 when n = 3 as an example. As illustrated, the band limited image Tm represents a component in the low frequency region of the image S0 as m increases.

  In this way, the blurred image creating unit 10 and the band limited image creating unit 20 create band limited images T1, T2,... Tn for each luminance Y, color difference Cr, and color difference Cb from the image S0.

  The noise analysis means 30 obtains the noise level in the image S0. In the present embodiment, the noise analysis unit 30 obtains the luminance noise level σY and the color difference noise levels σCr, σCb as the noise levels of the image S0. FIG. 11 is a block diagram showing the configuration of the noise analysis means 30. As shown in FIG.

  As shown in FIG. 11, the noise analyzing unit 30 is a band limited image for analyzing the noise level according to the enlargement / reduction ratio α of the image D0 obtained from the image input unit 1 (here, the band limitation of the color difference component). Image) Obtained by band determining means 31 for determining Tpb, Tpr, color difference noise level calculating means 35 for calculating color difference noise levels σCr, σCb using the determined band limited image, and color difference noise level calculating means 35 Luminance noise level estimation means 39 for estimating the luminance noise level σY using the color difference noise level.

  The band determining means 31 determines a band-limited image for calculating the noise level so that the frequency band in the selected frequency region is lowered as the expansion / contraction ratio α increases as much as possible. FIG. 12 shows an example of a mode in which the band determining unit 31 in the present embodiment determines a band limited image. In the illustrated example, the band determination means 31 selects the noise levels so as to select T1, T2, and T3 for each scaling factor α that is 0.75 or less and greater than 0.75 to 1.25 and 1.25. Band-limited images Tpb and Tpr for calculating the above are determined. The band determining unit 31 outputs the band limited images Tpb and Tpr to the color difference noise level calculating unit 35, and calculates the color difference noise level for the band limited image TpY of the luminance component in the frequency band corresponding to the band limited images Tpb and Tpr. It outputs to the means 35.

  The color difference noise level calculation means 35 obtains the color difference noise levels σCb and σCr using the band limited images Tpb and Tpr determined by the band determination means 31. FIG. 13 is a block diagram showing the configuration of the color difference noise level calculation means 35. As shown in the figure, the color difference noise level calculation means 35 includes a pixel removal means 32 that seems to be a luminance edge, a component removal means 34 that seems to be a color difference edge, and a standard deviation calculation means 36. The pixel removal means 32 that seems to be a luminance edge is a pixel having a luminance value equal to or higher than a predetermined threshold based on the luminance value of each pixel of the luminance component band limited image TpY output together with the color difference component band limited images Tpb and Tpr. Is a pixel that seems to be a luminance edge, and pixels corresponding to the pixel that seems to be a luminance edge are removed from the band-limited images Tpb and Tpr of the color difference component.

  Next, the component removing unit 34 that seems to be a color difference edge removes the component that seems to be a color edge from the band limited images Tpb and Tpr after the pixels that are likely to be luminance edges are removed. Each unit of the color difference noise level calculation unit 35 performs the same operation with respect to Tpb and Tpr. Therefore, the band-limited images Tpb and Tpr are not particularly distinguished below and will be described as Tp.

  The component removal means 34 that seems to be a color difference edge creates a histogram of absolute values of color differences for the band limited image Tp from which pixels that are likely to be luminance edges have been removed, and the color difference values that have a frequency greater than a predetermined threshold value in this histogram. Remove as a color-like edge-like component.

  The standard deviation calculation means 36 calculates the standard deviation of the color difference using the color difference value after the color difference edge-like component is removed by the color difference edge-like component removal means 34 in the color difference absolute value histogram and calculates the color difference noise level σCr. , ΣCb is obtained. Here, a value obtained by correcting the calculated standard deviation of the color difference according to the threshold value used in the component removing unit 34 that seems to be a color difference edge may be used as the color difference noise level. This makes it possible to obtain an appropriate noise level even if the color difference value removed as a color difference edge-like component is a noise color difference value. Specifically, assuming that the histogram has a complete normal distribution, the standard deviation calculated using color difference values having a frequency of 0% to 90% is an abbreviation of the standard deviation calculated using all color difference values. For example, when a color difference value having a frequency of 90% or more is removed as a component that seems to be a color difference edge, the standard deviation calculated using a color difference value other than the component that seems to be a color difference edge is 0. 9 times. What is necessary is just to correct | amend so that 1.27 which becomes the reciprocal number of 79 may be multiplied. Similarly, when 80% or 70% is used as the threshold value, the standard deviation may be multiplied by 1.51 or 1.78, respectively.

  The luminance noise level estimation unit 39 estimates the luminance noise level from the color difference noise levels σCr and σCb obtained by the color difference noise level calculation unit 35. In the present embodiment, since YCrCb is used as the luminance color difference space, the luminance noise level estimation means 39 estimates the luminance noise level σY from the color difference noise levels σCr, σCb according to the above-described equation (7).

σY≈ (1.02 × σCr + 1.07 × σCb) / 2 (7)
Where σY: Luminance noise level
σCr, σCb: Color difference noise level

The noise analyzing unit 30 outputs the luminance noise level σY and the color difference noise levels σCr, σCb thus obtained to the noise correcting unit 40.

  In the image processing apparatus of the present embodiment, the noise correction unit 40 performs noise correction by suppressing noise in a plurality of frequency bands of the image S0. As shown in FIG. The correction execution means 46 is provided. The noise correction unit 40 corrects both the luminance and the color difference. Here, the noise correction with respect to the luminance will be described as an example.

  The nonlinear conversion means 42 of the noise correction means 40 performs nonlinear conversion on each band limited image Tm (m = 1 to n) obtained by the band limited image creating means 20 to correspond to each band limited image Tm. The noise components Q1, Q2,..., Qn in the frequency band are extracted. This non-linear transformation is a process of making the output value equal to or less than the input value, and for an input value less than or equal to a predetermined threshold value, the larger the input value, the larger the output value, but greater than the predetermined threshold value. For the input value, the output value is equal to or less than the output value corresponding to the predetermined threshold value. In this embodiment, the process is performed by a function f as shown in FIG. The broken line in the figure indicates a function whose slope is γ (0 <γ ≦ 1). That is, the nonlinear transformation function f used in the nonlinear transformation means 42 of the present embodiment has a slope γ when the absolute value of the input value is smaller than the first threshold Th1, and the absolute value of the input value is the first value. When the threshold value is equal to or greater than the threshold value Th1 and equal to or less than the second threshold value Th2, the slope is smaller than γ. When the absolute value of the input value is greater than the second threshold value Th2, the absolute value of the output value is greater than the absolute value of the input value. This is a function having a small constant value M (hereinafter referred to as an upper limit value).

  Here, the upper limit value M reflects the strength of the noise suppression processing, and is set so as to increase as the noise level increases in accordance with the noise level obtained by the noise analysis means 30. In the present embodiment, if the noise level is σ, the upper limit value M is within the range of 2σ to 3σ with respect to the band limited image T1 that is most affected by noise (that is, the band limited image corresponding to the highest frequency band). Is set to a predetermined value of 2σ, for example, 2σ, and the intensity of noise suppression processing is weakened for other band limited images, and the upper limit value M is set to the band limited image T1. 1/2, that is, σ.

  In addition, the slope γ in the function f shown in FIG. 15 represents the noise suppression ratio of each band limited image, and the S / N ratio becomes higher toward the lower frequency band side. The band-limited image of the band is set to have a lower suppression ratio, that is, γ is set smaller.

  The non-linear conversion means 42 is constituted by the luminance value of the output value by taking the luminance value of each band limited image as an input value, performing non-linear conversion on each band limited image using the non-linear conversion function f shown in FIG. Further, the noise component Qm (m = 1 to n) in the frequency band corresponding to each band limited image is extracted.

The correction execution unit 46 multiplies each noise component Qm extracted by the non-linear conversion unit 42 by the subtraction coefficient β, and subtracts the luminance noise component Qm multiplied by the subtraction coefficient from the luminance component Y0 of the image S0. Thus, the luminance component Y′0 of the noise-corrected image S′0 is obtained. The following equation (15) shows the process performed by the correction execution means 46.

Here, the subtraction coefficient β is β (Y0), that is, determined in accordance with the luminance value Y0 of the pixel of the image S0. Specifically, the pixel having the larger luminance value Y0 is the pixel. The value of the subtraction coefficient β used when obtaining the pixel value Y′0 is small.

  In this way, the noise correcting unit 40 corrects the luminance component Y0 of the image S0 to obtain Y′0. Further, the same noise correction processing as that performed on the luminance component using the band-limited image and the color difference noise levels σCr and σCb to which the color difference components Cr0 and Cb0 respectively correspond is performed, and C′r0, C'b0 is obtained. Thus, a noise-corrected image S′0 composed of Y′0, C′r0, and C′b0 is obtained.

  As described above, according to the image processing apparatus of the present embodiment, when determining the noise level of an image, the color difference component in the image has a noise component, but pays attention to the fact that the edge component is small. An appropriate noise level for the entire image can be obtained by obtaining the color difference noise level and estimating the luminance noise from the color difference noise level. Thereby, appropriate noise suppression processing can be achieved.

  Further, when obtaining the color difference noise level, the frequency band for obtaining the color difference noise level is determined according to the enlargement / reduction ratio of the enlargement / reduction process performed on the image. it can.

  In addition, since noise suppression is performed in a plurality of frequency bands when performing noise suppression processing, in order to enhance the noise suppression effect as in the prior art, the suppression of noise components in one frequency band should be strengthened. Generation of artifacts due to the image quality can be prevented, and the image quality of the processed image can be improved.

  As mentioned above, although preferred embodiment of this invention was described, this invention is not limited to embodiment mentioned above, Unless it deviates from the main point of this invention, various increase / decrease and change can be performed.

  For example, in this embodiment, noise suppression processing is performed on all of the luminance and chrominance components. However, since the luminance component contributes most to human vision, noise suppression processing is performed only on the luminance component. May be performed.

  Further, the specific method of noise suppression processing is not limited to the method of the present embodiment, and various conventional methods may be used.

  In this embodiment, the noise level is obtained from an image in one frequency band (band limited image) of the image. However, the noise level is obtained for each frequency band of the band limited image, and the noise of the band limited image is obtained. You may make it use for a suppression process.

  Also, the method for creating a blurred image and the method for creating a band limited image are not limited to the methods used in the present embodiment, and various conventional methods may be applied.

  In this embodiment, the noise level is obtained for the image and the noise suppression process is performed. However, the obtained noise level may be used for the image quality evaluation of the image. At that time, only one of the color difference noise level and the luminance noise level may be used, or all of them may be used.

  Also, in consideration of the fact that the noise level is lower in the image, the brighter the part, the image is divided into a plurality of divided images having stepwise luminance value ranges according to the luminance value of each pixel of the image (each divided image). The image is composed of pixels having luminance values within the luminance value range corresponding to the stage), and a noise level is obtained for each divided image and used for image quality evaluation or noise suppression processing. May be.

1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. 1 is a block diagram showing the configuration of a blurred image creation unit 10 in the image processing apparatus shown in FIG. The figure which shows the example of the one-dimensional filter F which the filtering means 12 in the blur image creation means 10 shown in FIG. 2 uses The figure which shows the process performed in the blurred image preparation means 10 shown in FIG. The figure which shows the frequency characteristic of the filtering image Bk produced by the filtering means 12 in the blur image creation means 10 shown in FIG. The figure which shows the example of the two-dimensional filter which the filtering means 12 in the blur image creation means 10 shown in FIG. 2 uses The figure which shows the example of the filter F1 which the interpolation means 14 in the blur image creation means 10 shown in FIG. 2 uses for interpolation of the filtering image B1 The figure which shows the example of the filter F2 which the interpolation means 14 in the blur image creation means 10 shown in FIG. 2 uses for interpolation of filtering image B2 The figure which shows the frequency characteristic of the blur image Sk created by the blur image creation means 10 shown in FIG. The figure which shows the frequency characteristic of the band limited image Tk produced by the band limited image preparation means 20 in the image processing apparatus of embodiment of this invention shown in FIG. 1 is a block diagram showing the configuration of noise analysis means 30 in the image processing apparatus according to the embodiment of the present invention shown in FIG. The figure for demonstrating the band determination means 31 in the noise analysis means 30 shown in FIG. 11 is a block diagram showing the configuration of the color difference noise level calculation means 35 in the noise analysis means 30 shown in FIG. 1 is a block diagram showing the configuration of noise correction means 40 in the image processing apparatus according to the embodiment of the present invention shown in FIG. The figure which shows the example of the nonlinear transformation function f which the nonlinear transformation means 42 in the noise correction means 40 shown in FIG. 14 uses

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image input means 5 YCC conversion means 10 Blurred image creation means 12 Filtering means 14 Interpolation means 16 Control means 20 Band-limited image creation means 30 Noise analysis means 31 Band decision means 32 Pixel removal means that seem to be luminance edges 34 Component removal means that seem to be color difference edges 35 Color difference noise level calculation means 36 Standard deviation calculation means 39 Luminance noise level estimation means 40 Noise correction means 42 Non-linear conversion means 46 Correction execution means

Claims (9)

An image processing method for obtaining a noise level of a digital image,
In the image processing method in which the color difference noise level in the color difference component of the digital image is calculated as the noise level ,
The digital image has been subjected to a scaling process at a predetermined scaling ratio,
Determine the frequency band for obtaining the noise level according to the scaling factor,
Based on the digital image, create a band limited image representing the frequency band component of the digital image,
An image processing method comprising obtaining the noise level from the band-limited image . An image processing method for obtaining a noise level of a digital image,
In the image processing method in which the color difference noise level in the color difference component of the digital image is calculated as the noise level,
It performs noise suppression processing for the previous SL digital image,
Adjust the degree of the noise suppression process according to the noise level,
As the noise suppression processing, and the processing on the basis of the digital image, creating a plurality of band-limited image representing a component of each of the plurality of frequency bands of the digital image,
For an input value in which the absolute value of the output value is less than or equal to the absolute value of the input value and the absolute value is less than or equal to a predetermined threshold value, the absolute value of the output value increases as the absolute value of the input value increases. On the other hand, for an input value whose absolute value is larger than the predetermined threshold value, nonlinear conversion processing in which the absolute value of the output value is equal to or smaller than the absolute value of the output value corresponding to the predetermined threshold value is performed for each band limitation. Processing for each pixel value of the image to obtain a plurality of converted images;
A process of multiplying a pixel value of the plurality of converted images by a predetermined subtraction coefficient;
Wherein the pixel values of the digital image, performs the subtraction coefficient by subtracting the pixel values of the multiplied plurality of converted images to obtain a pixel value of the processed image processing ing from the processing,
Wherein as the predetermined threshold value and / or the output value corresponding to the predetermined threshold, images processing how to said Rukoto with those determined according to the noise level. 3. The image processing method according to claim 1, wherein a luminance noise level of the digital image is estimated from the color difference noise level, and the luminance noise level is set as a noise level of the digital image. An image processing apparatus having a noise level acquisition means for obtaining a noise level of a digital image,
The noise level acquisition means includes color difference noise level calculation means for calculating a color difference noise level in a color difference component of the digital image;
In the image processing apparatus configured to set the color difference noise level as the noise level of the digital image ,
The digital image has been subjected to a scaling process at a predetermined scaling ratio,
A frequency band determining means for determining a frequency band for obtaining the noise level according to the scaling factor;
Band-limited image creation means for creating a band-limited image representing the frequency band component of the digital image based on the digital image;
The image processing apparatus, wherein the noise level acquisition means obtains the noise level using the band limited image . An image processing apparatus having a noise level acquisition means for obtaining a noise level of a digital image,
The noise level acquisition means includes color difference noise level calculation means for calculating a color difference noise level in a color difference component of the digital image;
In the image processing apparatus configured to set the color difference noise level as the noise level of the digital image,
Further comprising noise suppression processing means for performing noise suppression processing on the digital image,
The noise level acquisition means adjusts the degree of the noise suppression processing according to the noise level,
The noise suppression processing means, based on the digital image, a band-limited image creating means for creating a plurality of band-limited images representing components for a plurality of frequency bands of the digital image;
For an input value in which the absolute value of the output value is less than or equal to the absolute value of the input value and the absolute value is less than or equal to a predetermined threshold value, the absolute value of the output value increases as the absolute value of the input value increases. On the other hand, for an input value whose absolute value is larger than the predetermined threshold value, nonlinear conversion processing in which the absolute value of the output value is equal to or smaller than the absolute value of the output value corresponding to the predetermined second threshold value is performed. Non-linear conversion means for obtaining a plurality of converted images by applying to each pixel value of the band limited image;
A pixel of the processed image by multiplying a pixel value of the plurality of converted images by a predetermined subtraction coefficient and subtracting a pixel value of the plurality of converted images multiplied by the subtraction coefficient from a pixel value of the digital image Frequency suppression means for obtaining a value,
Images processor you wherein the output value corresponding to the predetermined threshold value and / or the predetermined threshold value is one determined according to the noise level. The noise level acquisition means further comprises luminance noise level estimation means for estimating a luminance noise level of the digital image from the color difference noise level calculated by the color difference noise level calculation means,
6. The image processing apparatus according to claim 4 , wherein the luminance noise level estimated by the luminance noise level estimation means is used as a noise level of the digital image. A program for causing a computer to execute a noise level acquisition process for obtaining a noise level of a digital image,
The noise level acquisition process, in the process der pulp program for calculating a color difference noise level in the chrominance components of the digital image as a noise level of the digital image,
The digital image has been subjected to a scaling process at a predetermined scaling ratio,
A process of determining a frequency band for obtaining the noise level according to the scaling factor;
Based on the digital image, causing the computer to further execute a process of creating a band limited image representing the frequency band component of the digital image,
The noise level process is a process for obtaining the noise level from the band-limited image . A program for causing a computer to execute a noise level acquisition process for obtaining a noise level of a digital image,
In the program in which the noise level acquisition process is a process of calculating a color difference noise level in a color difference component of the digital image as a noise level of the digital image,
Further causing the computer to perform noise suppression processing on the digital image,
Adjust the degree of the noise suppression process according to the noise level,
As the noise suppression processing, and the processing on the basis of the digital image, creating a plurality of band-limited image representing a component of each of the plurality of frequency bands of the digital image,
For an input value in which the absolute value of the output value is less than or equal to the absolute value of the input value and the absolute value is less than or equal to a predetermined threshold value, the absolute value of the output value increases as the absolute value of the input value increases. On the other hand, for an input value whose absolute value is larger than the predetermined threshold value, nonlinear conversion processing in which the absolute value of the output value is equal to or smaller than the absolute value of the output value corresponding to the second threshold value is performed for each band. A process of obtaining a plurality of converted images by applying to each pixel value of the restricted image;
A process of multiplying a pixel value of the plurality of converted images by a predetermined subtraction coefficient;
Wherein the pixel values of the digital image, performs the subtraction coefficient by subtracting the pixel values of the multiplied plurality of converted images to obtain a pixel value of the processed image processing ing from the processing,
It said predetermined as a threshold and / or the output value corresponding to the predetermined threshold value, features and to pulp programs that use those determined according to the noise level. The noise level acquisition process further includes a luminance noise level estimation process for estimating a luminance noise level of the digital image from the color difference noise level obtained by the color difference noise level process, and the luminance noise level is obtained from the digital image. The program according to claim 7 or 8 , characterized in that the processing is performed with a noise level of.

Images