JP2001008038A - Image processing method, image processor and multi-band color photographing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image processing method, an image processing unit and a multi-band color photographing system by which the noise component of a multi-band image can be suppressed or eliminated without deteriorating image quality and the sharpness of the image can be enhanced by sharpening the edge of a photographed object. SOLUTION: Processing such as sharpness emphasis, smoothing and edge detection is applied to original image data of a multi-band image (steps 100, 102, 108), edge area weighted data are obtained from the detected edge, and edge/noise mixed component image data are obtained from the image data with emphasized sharpness and image data subject to smoothing processing (step 104). The correlation between original images is obtained by using the image data, an edge storage/noise suppression coefficient is obtained from the edge area weighted data and the correlation value, the coefficient is multiplied with the edge/noise mixed component image data and the product is added to the smoothing image data to obtain processed image data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multi-band image obtained by photographing the same subject in at least four different wavelength regions, and to reduce noise (noise) components on the image caused by graininess and electrical noise. The present invention belongs to the technical field of an image processing method, an image processing apparatus, and a multi-band color photographic system of a multi-band image that suppresses or removes and enhances sharpness.

[0002]

2. Description of the Related Art Today, with the advance of digital image processing, there is a high demand for high quality and high image quality required for digital images. Even in a photographed image taken with a film, an image in which a subject is sharply captured and which has good color reproduction without noise is required. When such a digital image is a color image, it is generally formed by three images of three primary colors of red (R), green (G) and blue (B).

However, in the field of arts and crafts such as paintings,
In order to faithfully reproduce colors, images are synthesized from a plurality of single-color images obtained by photographing the same subject in at least four different wavelength regions, that is, images are organized and stored using a multiband image. I have. Also, when designing the spectral sensitivity of a photographic film and the spectral sensitivity of an image sensor of a digital camera, etc., in order to obtain images with better color reproduction, it is necessary to quantitatively determine the spectral reflectance of the subject. Multi-band images are used.

In order to obtain such a multi-band image, for example, a silver halide photographic material or C
A solid-state image pickup device such as a CD (charge coupled device) or an image pickup tube such as a photomultiplier is used. Images taken by these means contain noise in any case. Is causing a decrease. For example, when a silver halide photographic light-sensitive material is used, the noise contained in the photographed image is a granular shape of the light-sensitive material itself. Is caused by

In particular, in the case of a multi-band image, when obtaining a multi-band image, it is necessary to calculate and estimate the spectral reflectance of the photographed subject. Therefore, image data of a photographed monochromatic image is photographed in another wavelength range. The image data must be calculated together with the image data of the single-color image, and when obtaining a color image or the like using the image data of a single-color image having a different wavelength range, the image data must be calculated. Is also calculated together. As a result, the noise component is emphasized, a large noise component is included in the spectral reflectance of the subject, and the noise component is emphasized in a multiband image or a color image or the like obtained using image data of a single color image forming the multiband image. This causes a problem that the image quality is deteriorated as compared with a conventional color image obtained by three primary colors of RGB.

Further, since a multi-band image is obtained by synthesizing a plurality of original images, a blurred image in which the edge of the photographing subject is not clear is formed, and a conventional color image obtained by three primary colors of RGB is used. Deterioration of sharpness easily occurs.

In order to improve the image quality against such deterioration of the sharpness of a multi-band image, sharpness enhancement is performed by a Laplacian filter or an unsharp mask as in the case of a conventional image obtained by three primary colors of RGB. Can be. However, when sharpness enhancement is performed using a Laplacian filter or an unsharp mask, noise components included in the image are also enhanced, and a noise problem such that the image quality of the multi-band image is further deteriorated occurs.

To deal with such a problem of noise, there are many image processing methods for suppressing or removing noise components, such as a processing method for processing a noise component of a monochrome image such as a monochrome image and a noise component removing method for a color image. While several methods have been proposed, some image processing methods have been proposed to enhance the sharpness while removing noise components of the captured image. However, since these methods all use a method of clipping noise components, a method of averaging image data, or a method of blurring, the blurred noise pattern should not be visually uncomfortable or removed. There is a problem that a minute subject structure is removed together with a noise component, resulting in an unnatural image or an artifact, which is not suitable for an aesthetic image such as a photograph.

There are also many image processing methods for improving sharpness. However, simply enhancing sharpness simultaneously enhances noise and graininess together with the image, so that even if the sharpness is improved, an image having a deteriorated noise component is obtained. There was such a problem. Some image processing methods for removing noise components while enhancing sharpness have also been proposed. However, as described above, an unnatural image is obtained,
There was a problem that artifacts occurred.

For example, JP-T-57-500131 and JP-T-57-500354 and "Digital Enhancement of Unsharp and Granular Photographic Images", International Conference on Electronic Image Processing, July 1982, No. 179. ~ 18
Page 3 (PG Powell and BEBayer, '' A Method for the
Digital Enhancement of Unsharp, Grainy Photographi
c Images '', Proceedings of the International Confe
rence on ElectronicImage Processing, Jul. 26-28,198
2, pp. 179-183), a processing method using a smoothing processing method (low-pass filter) as a graininess suppression method and an unsharp mask (high-pass filter) as a sharpness enhancement method. Is used. For example, taking the case of a silver halide photographic material as an example, the smoothing process is performed on the signal value of n × n pixels.
This is a process of suppressing graininess by smoothing the signal by multiplying it by a Gaussian weight or the like. In the sharpness enhancement processing, first, a differential value in the direction from the center pixel to the surrounding pixels is obtained using an image signal of m × m pixels, and if the value is smaller than a set threshold, it is regarded as granular or noise and coring is performed. The difference is removed by processing, and the sum of the differential values larger than the remaining threshold value is calculated, multiplied by a constant of 1.0 or more, and added to the smoothed signal to sharpen the sharpness.

In this processing method, since the granular pattern is blurred, the contrast of the density of the granular pattern is reduced, but a large uneven pattern composed of a settled (mottle) of particles constituting the granular pattern becomes visually noticeable. Or it has the disadvantage of appearing as unpleasant granules. In addition, since the image is discriminated from the image with the set threshold value (coring processing), an image signal having a low contrast is erroneously recognized as a particle, and is suppressed or removed together with the particle, or an image signal emphasized as the removed signal. There is a disadvantage that discontinuity occurs at the boundary between the image and the image and unnatural artifacts are seen in the image. In particular, fine images of grass and carpets,
This drawback appears in textures and in images depicting textures such as fabrics, which are visually very unnatural and undesirable artifacts.

This problem is caused not only by the granularity of the silver halide photographic light-sensitive material, but also by noise components mixed in image data caused by shot noise or electric noise when a solid-state image pickup device or an image pickup tube is used. Occurs similarly.

[0013]

SUMMARY OF THE INVENTION Accordingly, the present invention is directed to a multi-band image obtained by photographing the same subject in at least four or more different wavelength regions by overcoming the above problems and including the multi-band image in the image. A noise component of a multi-band image caused by electrical noise caused by an image capturing means, for example, a grain inherent in a silver halide photographic material, a solid-state image sensor such as a CCD, or an image pickup tube such as a photomultiplier, is used to reduce image quality. While it can be suppressed and removed without causing any problems, it is possible to sharpen the edge of the photographed object and improve the sharpness of the image, and without any noise-eliminating patterns that appear visually unpleasant in the image , The fine structure that should not be removed is not removed together with the noise component, and unnatural artifacts occur in the image An image that captures general subjects, including aesthetic subjects such as paintings, without any problems, and obtains aesthetic photographic images with faithful color and tone reproduction and low image structure (sharpness and noise) degradation It is an object of the present invention to provide a processing method, an image processing apparatus, and a multi-band color photographic system using the same.

[0014]

In order to achieve the above object, the present invention has been made by the present inventor in consideration of the following points. That is, the spectral distribution of a subject has a spread including various wavelength components in a general subject, and a color filter for separating a wavelength range when a multi-band photograph is taken generally has some spread. ,
They overlap each other. Therefore, in a multi-band photograph, information of a subject image is recorded over a plurality of original images. For example, when photographing with a silver halide film, the noise component or the grain in the multi-band photograph is photographed on a separate film, so that the grain peculiar to the film has no correlation. In the case of an electronic imaging method such as an image sensor, there is a time lag when shooting using the same image sensor, and the image sensors are separate when shooting with multiple image sensors simultaneously. Therefore, electrical noise generated by the image sensor and the electronic circuit is uncorrelated. In a multi-band photographic image, the image data of the subject has a correlation between the original images, while the noise component is unique to each of the original images. Is calculated, image data having a stronger correlation is determined to be image data of a subject, and image data having a weak correlation is determined to be image data of a noise component (or a granular shape), whereby image data of a subject and a noise component can be identified. . By using this, the inventor can, for example, be inversely proportional to the degree of correlation (or
(Using a non-linear relational expression), that is, by removing more image data having a weak correlation, it is possible to remove noise components and obtain image data having no noise components or few noise components. It was.

That is, according to the present invention, a plurality of original image digital data (original image data) for each wavelength region obtained by photographing the same subject in at least four or more different wavelength regions are converted into the original image data. Perform sharpness enhancement processing to create sharpness enhanced image data, perform smoothing processing on the original image data to create smoothed image data, and convert the smoothed image data from the sharpness enhanced image data corresponding to the wavelength region. Edge / noise mixed image data in which an edge emphasis component and a noise component are mixed is obtained for each of the plurality of original images by subtraction, while the edge intensity data obtained by performing edge detection of the object from the original image data is used to calculate the object. Calculate edge area weighting data for discriminating the edge area and the noise area numerically,
Further, for each pixel position of the original image from the mixed edge / noise image data, a correlation value representing the correlation of the mixed edge / noise image data between the original images is calculated, and the edge area weighted data and the correlation value are calculated. From the values, the edge enhancement component is stored in the edge area of the subject, and the edge storage is suppressed in the area where the noise component is dominant.
A noise suppression coefficient is obtained, the edge / noise mixed image data is multiplied by the edge preservation / noise suppression coefficient to calculate edge enhancement / noise suppression image data, and the edge enhanced / noise suppression image data is added to the smoothed image data. Is added to obtain processed image data in which noise components are suppressed and sharpness is enhanced for each original image.

At this time, it is preferable to perform image processing on a plurality of original image digital data for each wavelength region obtained by photographing the same subject in at least six or more different wavelength regions. Further, the edge intensity data is obtained for each original image for each pixel, the local standard deviation of the image data within a certain range area around each pixel of the original image,
Preferably, the local standard deviation is obtained by taking an average value in the original image, and the edge area weighting data is obtained by normalizing the edge intensity data to 0.
It is preferable to obtain the value by reducing the numerical value of 1 or more and 1 or less to a value that sets the upper limit value to 1 and sets the lower limit value to 0 or more and 1 or less. Further, it is preferable that the correlation value is obtained by taking an absolute value of an average value of the edge / noise-mixed image data in the plurality of original images. It is preferably obtained by multiplying the edge area weighting data. Further, it is preferable that the processed image data is obtained by selectively suppressing only a noise component while preserving an edge emphasis component in accordance with the edge storage / noise suppression coefficient.

Also, the present invention provides an image processing method for multi-band original image data composed of a plurality of original images for each wavelength region obtained by photographing the same subject in at least four or more different wavelength regions. An image processing device for performing a sharpness enhancement process on the original image data to create sharpness enhanced image data; and a smoothing process from the original image data to create smoothed image data. A smoothing processing unit that performs edge detection of the subject from the original image data to obtain edge strength data; and an edge area and a noise area of the subject based on the edge strength data obtained by the edge detection unit. An edge area weighting data calculation unit for obtaining edge area weighting data to be identified by numerical value, and The smoothed image data obtained by the smoothing processing unit is subtracted from the sharpness-enhanced image data obtained by the sharpness-enhancing processing unit to convert the edge-noise mixed image data in which the edge-emphasized component and the noise component are mixed into the original image. An edge / noise-mixed component extraction unit that extracts each of them; and the edge / noise-mixed image data obtained by the edge / noise-mixed component extraction unit for each pixel position of the original image from the edge / noise-mixed image data An image-to-image correlation operation unit obtained by calculating a correlation value indicating a correlation between the original images, and the edge region weighting data obtained by the edge region weighting data operation unit and the inter-image correlation operation unit obtained. From the correlation value, the edge emphasis component is stored in the edge region of the subject, and the noise component is suppressed in the region where the noise component is dominant. An edge preserving / noise suppressing coefficient calculating unit for obtaining an edge preserving / noise suppressing coefficient, and the edge / noise mixed image data and the edge preserving / noise suppressing coefficient calculating unit obtained by the edge / noise mixed component extracting unit. An edge emphasis / noise suppression image data processing unit for obtaining edge emphasis / noise suppression image data using the obtained edge preservation / noise suppression coefficient, and the smoothed image data obtained by the smoothing processing unit; A noise suppression / sharpness enhancement processing operation unit that adds the edge enhancement / noise suppression image data obtained by the edge enhancement / noise suppression image data processing unit and obtains processed image data in which noise components are suppressed and sharpness is enhanced. The present invention provides an image processing apparatus having the following.

At this time, it is preferable that the noise suppression / sharpness enhancement processing operation unit selectively suppresses only the noise component while storing the edge enhancement component according to the edge storage / noise suppression coefficient. Further, the present invention is a multi-band color photographic system for obtaining a multi-band image composed of a plurality of original images for each wavelength region obtained by photographing the same subject in at least four or more different wavelength regions, An object of the present invention is to provide a multi-band color photographic system including an image processing device for suppressing noise of a multi-band image and enhancing sharpness, and further including an operation unit for obtaining a spectral reflectance distribution of a subject.

[0019]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an image processing method, an image processing apparatus, and a multi-band color photographic system according to the present invention will be described in detail based on preferred embodiments shown in the accompanying drawings.

FIG. 1 shows an image processing method according to the present invention.
An embodiment of the image processing apparatus according to the present invention,
1 shows a multi-band photographic system 10 incorporating a sharpness enhanced image processing operation unit 14. The multi-band photographic system 10 performs image processing and obtains a spectral reflectance distribution from original image data I _0i of an original image constituting a multi-band image obtained from the multi-band photographic image capturing input device 12 for photographing a subject. An image display device such as a color display, an image data storage device, or a device that outputs image data obtained from the spectral reflectance distribution to a computer or the like. And a subject spectral reflectance distribution calculating unit 16 for calculating and estimating the spectral reflectance distribution of the subject, and a subject spectral reflectance distribution image data output device 18 for outputting as image data of the spectral reflectance distribution of the subject. Is done.

Multi-band photographic image capturing input device 12
Is an image photographing means for photographing a subject for obtaining a multi-band image, and is a device for sending original image data of the multi-band image to the multi-band photographic system 10. For example, a photographic lens and a CCD sensor for photoelectrically reading an image formed are provided. A bandpass filter including a plurality of color filters and at least four color filters is disposed in front of the photographic lens. There is a digital camera for photographing a multi-band image in which the same subject is photographed while the pass filters are sequentially replaced. Further, the multi-band image photographing apparatus without the band-pass filter includes at least four kinds of CCD sensors capable of photoelectrically reading an image formed in at least four kinds of wavelength regions having different spectral sensitivity characteristics from the photographing lens. Digital camera. In this case, the CCD sensor may include a bandpass filter. Moreover,
The multi-band photographic image capturing input device 12 includes a silver halide photographing camera that forms a plurality of original images as black and white images on a silver halide photographic light-sensitive material sequentially through a band-pass filter including at least four color filters. And a scanner for obtaining the plurality of original images photographed by the camera as digital image data.

The noise removal / sharpness-enhanced image processing operation unit 14 of the multi-band photographic system 10 implements the image processing method of the present invention to suppress noise and enhance sharpness of the multi-band image, and also executes the image processing of the present invention. shows one embodiment of a processing apparatus performs image processing on original image data I _0i, a portion to obtain an enhanced processed image data I _1i of sharpness is noise suppressed from the original image data I _0i, the The configuration will be described later.

The subject spectral reflectance distribution calculating unit 16, the spectral reflectance of the subject from the processed image data I _1i distribution r (x,
y, λ). For example, in the case of a silver halide photograph, the processed image data I _1i is converted into an exposure amount using a characteristic curve of the exposure amount of the silver halide photographic material, and then photographed. It is configured to estimate the spectral reflectance distribution r (x, y, λ) of the subject in combination with the illumination light of the subject, the color filter, and the spectral distribution of the photographic film as a photographing sensor. Note that x and y are the positions of the pixel of interest in the image, and λ is the wavelength of light.

The subject spectral reflectance distribution image data output device 18 converts the subject spectral reflectance distribution r (x, y, λ) obtained by the subject spectral reflectance distribution calculating section 16 into the subject spectral reflectance distribution image data. The output device 18 outputs image data to a computer or the like connected to the output device 18. Subject spectral reflectance distribution image data input to the computer
Each band image constituting the multi-band image is displayed as a black and white image on an image display device such as a CRT monitor,
Image processing such as color reproduction / tone reproduction conversion can be performed to convert the data to RGB color image data, displayed as a color image, or output as a print by a color printer, for various uses.

The noise removal / sharpness-enhanced image processing operation unit 14 generates an original image of a multi-band image composed of a plurality of original images obtained by photographing the same subject in n kinds (n is at least 4 or more) of different wavelength regions. An image processing method according to the present invention is applied to original image data I _{0i of an} image (i is the number of an original image constituting a multiband image, and i is an integer of 1 or more and n or less). As shown,
A sharpness enhancement processing unit 14a, a smoothing processing unit 14b,
Edge detector 14c, edge area weighted data calculator 14d, edge / noise mixed component extractor 14e, inter-image correlation value calculator 14f, edge storage / noise suppression coefficient calculator 14g, edge enhancement / noise suppressed image data processor 1
4h and noise suppression / sharpness enhancement processing operation unit 1
4i.

[0026] In the sharpness enhancement processing unit 14a, sharpness enhancement by the process according to Gaussian unsharp mask (Gaussian USM) and Laplacian (Laplacian) or the like is performed, the original image data I _0i perform sharpness enhancement on the image input system A processing unit that performs blur recovery and creates sharpness-enhanced image data I _Si . If the sharpness degradation of the image is mild, the sharpness of the image can be enhanced.

For example, the unsharp mask is given by
Sea urchin image data I_0i(X, y) (x, y are
(Indicating the position of the pixel of interest)_0iSmoothing (x, y)
Image data <I_0i(X, y)>
Di-emphasis component I_0i(X, y)-<I_0i(X, y)>
multiply the original image data I_0iAdd to (x, y)
As a result, the sharpness enhanced image data I_Si(X, y)
It is a method of seeking. I_Si (X, y) = I_0i(X, y) + a [I_0i(X, y)-<I_0i(X, y)>] (1) where a is a constant for adjusting the degree of sharpness enhancement.
is there. Laplacian is the original image data I_0i(X, y)
Second derivative (Laplacian) ▽ ^TwoI_0i(X, y) is the original image
Those who emphasize sharpness by subtracting from data
And expressed by the following equation: I_Si(X, y) = I_0i(X, y)-▽^TwoI_0i(X, y) (2) As a specific example of sharpness enhancement by Laplacian
The following 3 × 3 coefficient array is often used.
You. 0 −1 0 −1 −1 −1 1 −2 1 −1 5-1 −1 9-1 −1 −2 5 −2 (3) 0 −1 0 −1 −1 −1 1 −2 1

In this coefficient array, an unnatural contour is likely to occur at the edge of an image when particularly strong sharpness enhancement is applied. Therefore, in order to reduce such a drawback, in the present invention, the normal distribution type (Gau
It is preferable to use an unsharp mask using an ssian) blur function. G (x, y) = (1 / 2πσ ² ) exp [− (x ² + y ² ) / 2σ ² ] (4) Here, σ ² is a parameter representing the spread of the normal distribution function, and the ratio of the values at the center x = 0 of the value and the mask in the _{x = x 1, G (x} 1, 0) / G (0,0) = exp [-x 1 2 / 2σ 2] (5) 0.1 By adjusting to be ~ 1.0,
The sharpness of the 3 × 3 unsharp mask can be made desired. When the value of the expression (5) is set to a value close to 1.0, a mask substantially the same as the central Laplacian filter of the expression (8) can be created. In order to change the sharpness of the mask, there is another method of changing the size of the mask. For example, by using a mask of 5 × 5, 7 × 7, 9 × 9, etc., the spatial frequency range of sharpness enhancement is used. Can be changed significantly.

As the function form of the mask, a mask other than the normal distribution type, for example, an exponential function type mask as shown in the following equation (11) can be used. E (x, y) = exp [− (x ² + y ² ) ^1/2 / a] (6) Here, a is a parameter representing the spread of the unsharp mask as in σ ^{2 in} equation (5). And the ratio of the edge value of the mask to the center value of the mask, E (x ₁ , 0) / E (0,0) = exp [−x ₁ / a] (7) is 0.1 to 1.0. By adjusting to become
The sharpness of the 3 × 3 unsharp mask can be made desired. In equation (8), E (x ₁ , 0) / E (0,0) =
A numerical example of a mask of the exponential function of Expression (6) when 0.3 is set is shown. 0.18 0.30 0.18 0.30 1.00 0.30 (8) 0.18 0.30 0.18 When one example of an unsharp mask is calculated from this mask, the following equation (9) is obtained. −0.12 −0.22 −0.12 −0.21 2.32 −0.21 (9) −0.12 −0.21 −0.12

Using such an unsharp mask,
Sharpness-enhanced image data I _Si (x, y) is obtained from the original image data I _0i (x, y), and the obtained sharpness-enhanced image data I _Si (x, y) is sent to the edge / noise mixed component extraction unit 14 e. Sent. It should be noted that the sharpness enhancement method used in the present invention is not limited to the one described above, and other conventionally known sharpness enhancement methods such as spatial frequency filtering (Spatial fiteri).
ng) can also be applied.

Next, the smoothing processing unit 14b is an arithmetic unit that performs smoothing processing from the original image data I _0i (x, y) to create smoothed image data <I _0i (x, y)>. . Examples of the method of the smoothing processing include processing in the real space area and processing in the spatial frequency domain. There are various methods as described below, but the smoothing processing unit 14b is not particularly limited. For example, in the real space area processing, a method of calculating the average value of the sum of all adjacent pixels and replacing the average value with the calculated value,
There are various methods such as a method of multiplying each pixel by a weight coefficient, for example, a normal distribution type function to obtain an average value, and a method of performing a non-linear process such as a median filter. On the other hand, in the processing in the spatial frequency domain, there is a method of applying a low-pass filter. For example, the following formula (10) can be given as an averaging method using a weight coefficient.

[0032]

(Equation 1)

Here, N is a mask size for averaging, and w is a weight coefficient. If w = 1.0, a simple average is obtained. At this time, the following N ×
It is preferable to use a mask of N pixels. Specifically 3 ×
It is preferable to use one having about 3 to 5 × 5, 7 × 7, and 9 × 9. w ₁₁ w ₁₂ w ₁₃・ ・ ・ ・ ・ ・ ・ w _1N w ₂₁ w ₂₂ w ₂₃・ ・ ・ ・ ・ ・ ・ w _2N w ₃₁ w ₃₂ w ₃₃・ ・ ・ ・ ・ w _3N・ ・ ・ ・ (11) ・ ・ ・ ・w _N1 w _N2 w _N3 ... w _NN Each pixel adjacent thereto is multiplied by a weighting coefficient, for example, a normal distribution type function to obtain an average value, and the smoothed image data <I _0i (x,
y)> is obtained. Obtained smoothed image data <I _0i (x,
y)> is sent to the edge / noise mixed component extraction unit 14e.

The edge detector 14c outputs the original image data I _0i
This is a part for detecting the edge of the subject in the image from (x, y). While moving the array of N _E × N _E pixels in the original image,
The variation of the value of the original image data I _0i (x, y) in the array is calculated by using Expression (12), and the local standard deviation σ _{i at} each position is calculated.
By sequentially calculating (x, y) as a local variance, a subject edge in an image is detected. The size (N _E × N _E ) of the pixel array may be appropriately determined in consideration of the detection accuracy and the calculation load. For example, it is preferable to use a size of about 3 × 3, 5 × 5, or 7 × 7. .

[0035]

(Equation 2) <I _0i (x, y)> is the average value of the array,

(Equation 3) It is.

The edge detection performed by the edge detector 14c is not limited to the local dispersion method in which the edge is detected using the local standard deviation of the image data I _0i (x, y). , A method of locally differentiating the array of N _E × N _E pixels while moving to obtain the absolute value thereof, a method of performing edge detection using a Sobel operator, a Roberts operator, or the like, which is a differential type edge detection operator, Ro using a 3 × 3 template corresponding to an edge pattern in eight directions as a template type edge detection operator
A method using a binson operator, a Kirsh operator, or the like may be used. Alternatively, a method using a Laplacian operator may be used.

Edge area weighting data calculation section 14d
Is the local standard deviation σ obtained by the edge detector 14c.
From _i (x, y), obtaining the edge intensity data E (x, y), is a portion to obtain the edge intensity data E (x, y) from the edge region weight data W _E (x, y). In the edge area weighting data W _E (x, y), the larger the value is, the higher the probability of being the subject edge at the pixel position at (x, y), and the smaller the value is, the smaller the value of the pixel at (x, y) is. This is data for each pixel that is numerically determined to contain a large amount of noise components in the image data.

The edge strength data E (x, y) is calculated by the following equation (1) with respect to n local standard deviations σ _i (x, y) obtained for each of the original images constituting the multiband image.
The edge intensity data E (x, y) is obtained by calculating the average value in the original image as in 4).

[0039]

(Equation 4)

The edge intensity data E (x, y) is expressed by the following formula (15).
_The normalized edge strength data E _N (x, y) normalized by _Max and equal to or larger than 0 and equal to or smaller than 1 is obtained. E _N (x, y) = E (x, y) / E _Max (15) where E _max is the maximum value of the edge strength data E (x, y), and normalizes E (x, y). Is a constant for
The method of determining E _Max is to obtain the maximum value from the edge intensity data E (x, y) of the entire image obtained by Expression (15) as in Expression (16) below. E _Max = Max {E (x, y)} (16)

Also, instead of using the maximum value from the entire image, a part of the image data, for example, a specific range of a central portion where an important subject of the image has a high probability, or image data thinned out from the entire image (1 of the original image data). / 4 to 1/10)
The maximum value E _Max may be calculated using the above equations (14) and (16). In this case, the image data in a specific range in the central portion of the image or the thinned image data is obtained by adjusting various processing condition adjustment original image data (prescan image data) which can be obtained in advance at a coarse pixel density before obtaining the original image data of the multiband image. Image data) or may be extracted from the image data after photographing. More preferably, E _Max is an average value of values located within the upper 5% to 10% when the edge strength data E (x, y) is arranged in descending order, for example, a value located within the upper 10%. <E (x, y)>
_{Max 10%} is _defined as E _Max , and when E (x, y) exceeds this E _Max , all are replaced with E _Max . In this case, the average value may be an average value of the entire image, an average value of a predetermined range in a central portion where an important subject is often photographed, or an average value of thinned image data.

Using the normalized edge strength data E _N (x, y) obtained in this way, the following equation (17)
, Edge area weighting data W _E (x, y) is obtained. W _E (x, y) = 1−α _E + α _E E _N (x, y) (17) Here, α _E is a constant for setting the weight of the edge area and the noise area. Any smaller value can be set. The edge area weighting data W _E (x, y) has a value of 1-α _E or more and 1 or less, and it is determined that the greater the value of W _E (x, y), the higher the probability that the pixel position is the edge area of the subject. _WE
It is determined that the smaller the value of (x, y), the higher the probability that the pixel position is a noise region. In a region where the normalized edge strength data E _N (x, y) is close to 1.0, the edge region weighting data W is independent of the value of α _E.
E (x, y) becomes a value close to 1.0, and the edge enhancement component is stored as described later. On the other hand, as the normalized edge strength data E _N (x, y) becomes smaller than 1.0, that is, as the noise component increases, the edge area weighting data W _E (x, y) becomes 1−α _E. Approaching W
According to the value of _E (x, y), noise components are suppressed and removed.

Thus, the constant α_EWill be described later.
Image data ΔI _ENI’(X, y)
Edge area weighting data W serving as a coefficient_E(X, y)
This is an important constant that can be set when determining the value. Edge strength
Degree data E_NIf the value of (x, y) is close to 1.0,
That is, in the region where the edge component of the subject is dominant, the constant α_E
Regardless of the value of the edge area weighting data W_E(X,
y) also shows a value close to 1.0, but the edge intensity data E_N
As the value of (x, y) decreases from 1.0,
As the edge component decreases, the edge area weight
Data W_E(X, y) also becomes smaller and 1-α_EApproach.
At this time, α_EIf a value close to 1.0 is adopted as
Area weighting data W _E(X, y) is the edge strength data
Data E_NIt is almost equal to (x, y) and the edge of the subject
In the region where the component is dominant, the following equations (18) and (2)
As shown in 2), the image data with the edge enhanced
Saves but no noise at the edge of the subject
In the dominant part of the component, the edge area weighting data W
_E(X, y) approaches 0, and the edge noise mixed
Present image data ΔI_ENi(X, y) almost completely removed
Smoothed image data <I_0i(X, y)>, that is, ΔI
_ENiThe fine structure of the subject slightly included in (x, y)
It is not preferable because only removed image data is obtained. one
One, α_EIf a value close to 0.0 is adopted as the
Only image data that is not suppressed or removed at all.
Not preferred. Constant α_EThe optimal value of included in the original image
Information such as the edge of the subject
Varies depending on the balance of the noise components
Should be greater than 0.1 and less than 0.9
Preferred, more preferably greater than 0.3 and greater than 0.7
It is a small value.

The edge / noise mixed component extraction unit 14e
The sharpness enhanced image data I _Si (x, y) sent from the sharpness enhancement processing unit 14a and the smoothed image data <I sent from the smoothing processing operation unit 14b
_0i (x, y) by using the>, a portion for extracting the edge noise mixed image data [Delta] I _ENi (x, y) according to the following formula (18), extracted edge noise mixed image data [Delta] I _ENi (x , Y) are sent to the inter-image correlation value calculation unit 14f and the edge enhancement / noise suppression image data processing unit 14h. ΔI _ENi (x, y) = I _si (x, y) − <I _0i (x, y)> (18)

The image correlation value calculating section 14f calculates a correlation value representing the correlation between the original images of the mixed image data ΔI _ENi (x, y) extracted for each original image from the mixed edge / noise component extracting section 14e. C (x, y) is calculated by the following equation (19).
Is an arithmetic unit that obtains the position of each pixel of the original image. C (x, y) = | (1 / n) _{ΣΔI ENi} (x, y) | (19) That is, for all original images constituting the multi-channel image, image data ΔI _{ENi with} mixed edge and noise
, And the absolute value thereof is obtained to obtain a correlation value C (x, y) for each pixel position. Note that the correlation value C (x, y) obtained by calculating for each pixel position is not limited to the one obtained by the above equation (19), and a correlation value obtained by another method may be used.

Further, the image correlation value calculator 14f obtains a normalized correlation value C _N (x, y) from the obtained correlation value C (x, y) according to the following equation (20), and obtains the normalized correlation value C _N (x, y). The obtained correlation value C _N (x, y) is sent to the edge preservation / noise suppression coefficient calculation unit 14g. C _N (x, y) = C (x, y) / C _Max (20) Here, C _Max is a maximum value selected from the entire image range of the correlation value C (x, y). Not limited,
A correlation value C selected from a part of the image data, for example, a specific range in the central part of the image where an important subject is often photographed.
The maximum value of (x, y) may be used. Alternatively, edge / noise mixed image data is created from the original image data for thinned image data obtained by thinning the image data to, for example, about 1/4 to about 1/10, and a correlation value is obtained. The maximum value selected from the range of the entire image may be C _Max . In this case, the image data and the thinned image data in a specific range in the central portion where an important subject is often photographed are adjusted in various processing conditions that can be obtained in advance at a coarse pixel density before obtaining the original image data of the multiband image. Original image data (pre-scan image data) may be used. For example, the average value of the values located within the top 10% <C
(X, y)> _Max10% is _defined as C _Max , and when C (x, y) exceeds this C _Max , all are replaced with C _Max . In this case, the average value may be an average value of the entire image, an average value of a predetermined range in a central portion where an important subject is often photographed, or an average value of thinned image data.

Edge preservation / noise suppression coefficient calculator 14g
Is the correlation value C obtained by the image correlation value calculator 14f.
From (x, y) and the edge area weighting data W _E (x, y) obtained by the edge area weighting data calculation unit 14d, an edge enhancement component is stored in the edge area, and a noise component is suppressed in the noise area. This is a part for calculating the edge preservation / noise suppression coefficient P _N (x, y) of the following equation (21). P _N (x, y) = C _N (x, y) W _E (x, y) (21) The edge preservation / noise suppression coefficient P _N (x, y) is calculated as edge area weighting data W _E (x, y). ) Is large, the mixed edge / noise image data ΔI generated for each original image
_{It is determined that} the correlation value C _N (x, y) between _ENi (x, y) is large and the probability that the position is an edge of the subject is high at that position, and the mixed edge / noise image data ΔI _ENi (x, y) is used as it is. Save and conversely, edge area weighting data W _E (x, y)
When both the correlation value C _N (x, y) are small, the image data at that pixel position is a coefficient for judging that the probability of being a noise component is high and performing suppression or removal.

The edge-enhanced / noise-suppressed image data processing section 14h _{outputs the} mixed edge / noise image data ΔI _ENi (x,
y) is multiplied by the edge preservation / noise suppression coefficient P _N (x, y) obtained by the edge preservation / noise suppression coefficient calculation unit 14g as in the following equation (22) to selectively select only noise components. Edge-enhanced / noise-suppressed image data ΔI
Calculate _ENI '(x, y). ΔI _ENI '(x, y) = P _N (x, y) ΔI _ENI (x, y) (22) The obtained edge emphasis / noise suppression image data ΔI _ENI '
(X, y) is sent to the noise suppression / sharpness enhancement processing operation unit 14i.

The noise suppression / sharpness enhancement processing operation section 14i is obtained by the edge enhancement / noise suppression image data processing section 14h on the smoothed image data <I _0i (x, y)> obtained by the smoothing processing section 14b. By adding the edge-enhanced / noise-suppressed image data ΔI _ENI ′ (x, y) as in the following equation (23), the processed image data I _1i (x, y) in which noise is suppressed and sharpness is enhanced Get. I _1i (x, y) = <I _0i (x, y)> + ΔI _ENI ′ (x, y) (23) The obtained processed image data I _1i (x, y) is subjected to noise removal and sharpness enhanced image processing. The output data of the calculation unit 14 is sent to the subject spectral reflectance distribution calculation unit 16.

As described above, the noise removal / sharpness emphasized image processing operation unit 14 is configured.

Next, the image processing method of the present invention for suppressing noise and enhancing sharpness of a multi-band image is performed, and a multi-processing apparatus incorporating the image processing apparatus of the present invention for suppressing noise and enhancing sharpness of a multi-band image is implemented. The operation of the band photograph system 10 will be described. First,
Original image data I _0i (x, y) (i is an integer of 1 or more and n or less, and is the number of the original image) in which a subject is photographed in n kinds (n is 4 or more) of wavelength regions by the multi-band photographic image photographing input device 12. Is shown). For example, a multi-band image photographing digital camera in which a band-pass filter including at least four color filters is arranged in front of a photographing lens, and images formed sequentially through the band-pass filters are photoelectrically read by a CCD sensor. , Original image data I _0i corresponding to each band-pass filter of the digital image
(X, y) is obtained and sent to the noise removal / sharpness emphasized image processing operation unit 14.

In the case of the above digital camera for photographing a multi-band image, when photoelectrically read by a CCD sensor, shot noise and electric noise are included in each of a plurality of original images constituting the multi-band image as noise components in image data. Although mixed, the noise component mixed in the image data is a random noise, and thus has a feature that the noise component has no correlation between the original images. Even in the case of silver halide photographic light-sensitive materials, the graininess of the film is a unique grain pattern for each film, and therefore, there is a feature that the graininess is not correlated between the original images.

Next, as shown in FIG.
_0i (x, y) is subjected to sharpness enhancement in the sharpness enhancement processor 14a (step 100). For example, in the case of a method using a Gaussian-type unsharp mask, I _0i (x, y) is weighted from the original image data I _0i (x, y) by a normal distribution function by the equation (11) as shown in the equation (1). The edge enhancement component I _0i (x, y) − <I _0i (x, y)> obtained by subtracting the smoothed image data <I _0i (x, y)> obtained by multiplying the coefficient and taking the average value By multiplying by the coefficient a and adding to the original image data I _0i (x, y), the sharpness enhanced image data I _Si (x, y) is obtained. The method of the sharpness enhancement processing is not limited to the method using the unsharp mask, and the Laplacian (Laplaci
an), etc., and can be performed by a conventionally known method. Here, the sharpness enhanced image data I
_{In Si} (x, y), not only the edge component of the subject is emphasized, but also the noise component is sharpened.

The original image data I _0i (x, y) is subjected to sharpness enhancement (step 100), and at the same time, smoothed by the smoothing processing unit 14b (step 102). As described above, the method of the smoothing processing includes the processing in the real space domain and the processing in the spatial frequency domain, and there are various methods.
4b is not particularly limited. For example, in the real space region processing, as shown in Expression (10), each pixel is weighted with the average value of all adjacent pixels, and weighted using, for example, a coefficient by a normal distribution type function to calculate the average value. Ask.

From the sharpness emphasized image data I _Si (x, y) obtained in step 100 and the smoothed image data <I _0i (x, y)> obtained in step 102, equation (1)
8), the mixed edge / noise image data ΔI _ENi (x,
y) is obtained (step 104). As a result, mixed image data including an edge enhancement component and a noise component of the subject is obtained.

Next, edge / noise mixed image data ΔI _ENi (x, y) for each original image obtained in step 104.
Then, in accordance with equation (19), the image correlation value calculation unit 14f calculates a correlation value C (x, y) representing the correlation between the original images for each pixel position (step 106). Correlation value C
(X, y) is the average value of the edge noise mixed image data ΔI _ENi of four or more original images constituting the multiband image for each pixel position (x, y), as shown in Expression (19). The obtained correlation value C
(X, y) is stable. In particular, the correlation value C (x,
Since y) can be further stabilized, it is preferable that the multiband image is composed of six or more images.

Since the correlation value C (x, y) is obtained by taking the average value of the mixed image data ΔI _ENi of the edge and noise, for example, when the noise component is large and the edge component is small, it is included in each original image. Since there is no correlation between the noises, the noises cancel each other out, and the correlation value C (x, y) becomes a value close to 0. On the other hand, when the noise component is small and the edge component is large, the edge component between the original images has a strong correlation, and thus the correlation value C (x, y) has a large value. As described above, for each pixel position of an image, by using the correlation of the image data between the images, the dominant pixel of the edge component that must not be removed by image processing from the pixel position that is dominant of the noise component The numerical value can be continuously converted to the position using the correlation value C (x, y). Such a correlation value C
(X, y) is a normalized correlation value C _{N of} 0 or more and 1 or less.
(X, y).

The original image data I _0i (x, y) is sharpened (step 100) or smoothed (step 1).
02), and at the same time, is also used for edge detection processing in the other edge detector 14c (step 10).
8). The edge detection is performed by moving the array of N _E × N _E pixels in the original image and calculating the fluctuation of the value of the image data I _0i (x, y) in the array using Expression (12) for each position. The local standard deviation σ _i (x, y) is sequentially calculated as a local variance, obtained as a numerical value, and the edge of the subject in the image is detected.

The edge detection by the edge detection unit 14c is not limited to the method using the standard deviation, but is performed by Sobel operator, Roberts operator, Robinson operator, Ki operator.
A method using an rsh operator or a Laplacian operator may be used.

Next, the edge area weighting data calculation unit 1
In step 4d, the edge area weighting data is processed using the local standard deviation σ _i (x, y) obtained in step 108 to obtain edge strength data E (x, y). Edge area weighting data W _E (x, y) is calculated and obtained from the data _E (x, y) (step 110). The edge intensity data E (x is calculated by calculating the average value of the original image with respect to the n local standard deviations σ _i (x, y) constituting the multiband image as shown in Expression (14). , Y). Further, the edge strength data E (x, y) is normalized by the maximum edge strength data E _Max as shown in Expression (15), and normalized edge strength data E _N (x, y) of 0 or more and 1 or less. y) is obtained. The normalized edge strength data E _N (x, y) further obtains edge area weighting data W _E (x, y) having a value equal to or greater than 1−α _E and equal to or less than 1 according to Expression (17).

As described above, the edge is determined using the local standard deviation.
Area weighting data W_EOriginal picture to obtain (x, y)
Image data I_0iEdge of subject with large fluctuation of (x, y)
In the part, the local standard deviation becomes large, and W_E(X,
y) indicates a value close to 1. On the other hand, the edge component of the subject
Low and where the image data contains many noise components
Is W_E(X, y) is the minimum value 1-α_EBecome closer to here
And α_ESets the value greater than 0 and less than 1
It is a constant that can be set and has a fine structure
Even if the image has a fine structure,
The constant α_E
Can be set optimally. Constant α _EIs from 0.1
It is preferable to set the value to a large value smaller than 0.9.
Preferably a value greater than 0.3 and less than 0.7
You.

Next, the correlation value C obtained in step 106
(X, y) and the edge area weight obtained in step 110
Attached data W_EFrom (x, y), according to equation (21)
The edge preservation / noise suppression coefficient calculation unit 14g
And the edge preservation / noise suppression coefficient P_NGet (x, y)
(Step 112). Edge preservation / noise suppression coefficient P_N
(X, y) is the edge area weighting data W _E(X,
If y) is large, the edge and noise generated for each original image
Image data ΔI_ENiCorrelation value C between (x, y)
_N(X, y) is large and at that position the edge of the subject
Is judged to be high, the image data
Data ΔI_ENi(X, y) is saved, and the edge area weight is
Attached data W_E(X, y) and correlation value C_N(X, y)
If both are small, the edge noise at that pixel position
Image data ΔI_ENiHas a high probability of being a noise component
Is a coefficient to suppress and further remove
You. Thus, the edge preservation / noise suppression coefficient P
_N(X, y) indicates that the noise component is dominant in the image data
Is close to 0 when the edge component of the subject is dominant.
The normalized correlation value C close to 1_N(X, y) and
1-α when the noise component is dominant_EBecome closer to
When the edge component of the map is dominant, the edge near 1
Area weighting data W_EMultiplying by (x, y)
, Which is clearly a noise component
Image data is suppressed and removed, while the edge
The image data in the area is not suppressed or deleted.
Even in image data of an image portion having a fine structure
Noise components without suppressing or removing image data
It is possible to perform image processing for selectively suppressing and removing only the image data.

Next, in the edge emphasis / noise suppression image data processing unit 14h, the edge preservation / noise suppression coefficient P _N (x, y) obtained in step 112 is converted to the value in step 1
Edge / noise mixed image data Δ obtained by 04
Equation (22) multiplies I _ENi (x, y) to preserve the edge enhancement component and selectively remove only the noise component to remove edge enhancement / noise suppressed image data ΔI _ENI ′ (x, y
y) is calculated (step 114).

Finally, the edge-enhanced / noise-suppressed image data ΔI _ENI 'obtained at step 114 is added to the smoothed image data <I _0i (x, y)> obtained at step 102.
By adding (x, y), the processed image data I _1i (x, y) with noise suppression and sharpness enhanced is obtained as shown in Expression (23). Thereafter, the processed image data I from the noise removal / sharpness emphasized image processing operation unit 14
_1i (x, y) and sent to the subject spectral reflectance distribution calculation unit 16. In the portion of the original image data I _0i (x, y) where the noise component is dominant, the edge preservation / noise suppression coefficient P _N (x, y) is close to 0, so
The noise-suppressed image data ΔI _ENI ′ (x, y) is close to 0. As a result, the processed image data I _1i (x, y) has smoothed image data <I
_0i (x, y)> is substantially the same as the image data.
Since the edge preservation / noise suppression coefficient P _N (x, y) is close to 1 in a portion where the subject has many edge components, the edge-noise mixed image data ΔI _ENi (x, y)
y) is stored as it is, and the smoothed image data <I
_0i (x, y)>. In a portion where the noise component and the edge component coexist, the edge / noise mixed image data ΔI _ENi according to the value of the edge preservation / noise suppression coefficient P _N (x, y), that is, the ratio corresponding to the noise component. (X, y) is selectively suppressed and removed.

The processed image data I _1i (x, y) sent to the subject spectral reflectance distribution estimation calculating section 16 is represented by the subject spectral reflectance distribution for each pixel in the subject spectral reflectance distribution calculating section 16. Estimation is performed on the spectral reflectance distribution r (x, y, λ) of the subject, which is image data. The spectral reflectance distribution r (x, y, λ) of the subject is estimated by processing the image data I _1i (x, y), the spectral radiance E (λ) of the subject illumination light used at the time of photographing, and the The spectral transmittance t (λ) of a color filter used to obtain an image, the spectral sensitivity (including the spectral transmittance of a camera lens) of a CCD image sensor or a silver halide photographic film as an image sensor of a camera, and the sensor for light. Using the existing response method. Next, the subject's spectral reflectance distribution r (x, y,
[lambda]) using the subject spectral reflectance distribution image data output device 18 and the subject spectral reflectance distribution image data output device 18
Is output as image data to a device such as a computer connected to the computer. Image data of the spectral reflectance distribution r (x, y, λ) of the subject input to the computer is obtained by converting a band image for each wavelength region constituting a multiband image into a black and white image on an image display device such as a CRT monitor. Display,
Image processing such as color reproduction / tone reproduction conversion can be performed to convert the image data into RGB color image data, displayed as a color image, and output as a print by a color printer.

As described above, the image processing method of the present invention and the image processing apparatus of the present invention for suppressing noise and enhancing sharpness of a multi-band image are described in the multi-band photographic system 1.
However, the present invention is not limited to the above example. The multi-band photographic image capturing input device 42 and the image output device 50 are incorporated as shown in FIG. 4, and the original image data I _0i (x , Y) to implement the image processing method of the present invention for noise suppression and sharpness enhancement of a multi-band image to process image data I
_1i (x, y), from which the spectral reflectance distribution r of the subject is obtained.
A multi-band image noise elimination / sharpness emphasis image processing unit 44 for obtaining (x, y, λ), an image monitor / image processing parameter setting unit 46, and a color image reproduction (color / tone) processing unit 48 The multi-band type color photographic system 40 for obtaining a color image may be used.

Multi-band photographic image capturing input device 42
Has the same configuration and the same operation as the multi-band photographic image capturing input device 12 connected to the multi-band photographic system 10 shown in FIG. I _0i (x, y) is sent to the multi-band image noise removal / sharpness enhanced image processing unit 44 and also sent to the image monitor / image processing parameter setting unit 46.

The multi-band image noise elimination / sharpness enhanced image processing unit 44 combines the noise elimination / sharpness enhanced image processing operation unit 14 and the subject spectral reflectance distribution operation unit 16 of the multi-band photographic system 10 shown in FIG. Since the processing unit is an integrated processing unit and has the same configuration and operation, a description thereof will be omitted, but the estimated spectral reflectance distribution r (x, y, λ) of the subject is subjected to a color image reproduction (color / tone) processing. It is sent to the unit 48.

Image monitor / image processing parameter setting unit 4
6 displays images of a plurality of original images constituting the multi-band image, while the operator looks at the display screen,
Settings for various processes performed by the multi-band image noise removal / sharpness-enhanced image processing unit 44, for example, select a sharpness processing method, a smoothing method, or an edge detection method, or select a constant α _E or E _Max and it can be performed by C _Max settings, etc., color conversion correction, gradation correction, density correction, a mouse or a keyboard while watching the display screen settings of coefficients and parameters such as, those set condition, Multiband image noise removal / sharpness enhanced image processing section 44 and color image reproduction (color / tone) processing section 4
8 Color image reproduction (color / tone) processing unit 48
Is the obtained spectral reflectance distribution r (x, y, λ) of the subject.
Then, the image data is converted into RGB color photographic image data when photographed by a camera using a silver halide photographic film or a digital still camera. The spectral reflectance distribution r (x,
y, λ) is converted into RGB image data I ₁ (x, y) when photographed by the camera, the spectral radiance E (λ) of the subject illumination light at the time of photographing, and the CCD which is a photographing sensor of the camera.
An existing method is used using the image sensor (including the spectral transmittance distribution of the color filter) or the spectral sensitivity distribution of the silver halide photographic film (including the spectral transmittance of the camera lens) and the response characteristics of the sensor to light. The obtained RGB image data I ₁ (x, y) is output to the image output device 50.

The image output device 50 displays an image of RGB image data I ₁ (x, y) obtained from a multi-band image of a subject, and a color display such as a normal CRT monitor or a liquid crystal monitor for performing print output, or a digital display. a photo color printer or the like, from the image data I ₁ it is possible to obtain a color image.

As described above, the image processing apparatus of the present invention for implementing the image processing method of the present invention for suppressing noise and enhancing sharpness of a multi-band image can be used only when a multi-band photograph is to be incorporated into a multi-band photographic system. And the spectral reflectance distribution r (x, y,
When estimating λ), for example, the spectral reflectance distribution r (x, y, λ) of the subject for performing color reproduction simulation
Can also be used in the case where is obtained without including a noise component.

The image processing method and the image processing apparatus of the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiments, and various improvements and designs of the present invention can be made without departing from the gist of the present invention. Of course, changes may be made.

[0073]

As described above in detail, according to the present invention, it is possible to reduce the image quality of noise inherent to the silver halide photographic light-sensitive material, a solid-state image pickup device such as a CCD device, or an image pickup tube. By removing / suppressing the image, high-quality subject spectral reflectance image data and color image data with less noise components can be obtained.
The sharpness deteriorated by an optical system such as a photographic film or a scanner for digitizing the film can be improved without deteriorating noise components and graininess, and a multiband photographic image or a color image with good image quality can be obtained. Moreover, by using an edge preservation / noise suppression coefficient obtained by multiplying a noise component or a correlation value numerically representing an edge component of a subject by edge area weighting data, image data of a portion that is clearly a noise component is suppressed. , While removing the image data of the edge part of the subject without suppressing and deleting
Even in the image data of the image portion of the fine structure such as cloth, the image data of the fine structure can be suppressed and the image processing without removing the image data can be performed. Photographs aesthetic subjects such as paintings, without causing microscopic structures that should not be removed, along with noise components, and without causing artifacts in the images. To obtain an aesthetic image faithfully.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating the concept of a multi-band photographic system incorporating an image processing apparatus according to an embodiment of the present invention for suppressing noise and enhancing sharpness of a multi-band image.

FIG. 2 is a block diagram illustrating a configuration of a noise removal / sharpness emphasized image processing operation unit illustrated in FIG. 1;

FIG. 3 is a flowchart illustrating an embodiment of the image processing method of the present invention.

FIG. 4 is a block diagram showing the concept of a multi-band type color photograph system incorporating an embodiment of the image processing apparatus of the present invention for suppressing noise and enhancing sharpness of a multi-band image.

[Explanation of symbols]

Reference Signs List 10 Multi-band photographic system 12, 42 Multi-band photographic image capturing input device 14 Noise removal / sharpness-enhanced image processing operation unit 14a Sharpness-enhancement processing unit 14b Smoothing processing unit 14c Edge detection unit 14d Edge area weighting data operation unit 14e Edge noise Mixed component extraction section 14f Image correlation value calculation section 14g Edge preservation / noise suppression coefficient calculation section 14h Edge enhancement / noise suppression image data processing section 14i Noise suppression / sharpness enhancement processing calculation section 16 Subject spectral reflectance distribution calculation section 18 Subject spectrum Reflectance distribution image data output device 40 Multi-band type color photographic system 44 Multi-band image noise elimination / sharpness enhanced image processing 46 Image monitor / image processing parameter setting unit 48 Color image reproduction (color / tone) processing unit 0 image output device

Claims (10)

[Claims] A plurality of original image digital data (original image data) for each wavelength region obtained by photographing the same subject in at least four or more different wavelength regions are subjected to sharpness enhancement processing on the original image data. Line to create sharpness-enhanced image data, perform smoothing processing on the original image data to create smoothed image data, and subtract the smoothed image data from the sharpness-enhanced image data corresponding to the wavelength region to obtain an edge. Edge / noise mixed image data in which an emphasis component and a noise component are mixed is obtained for each of the plurality of original images. On the other hand, the edge area and noise of the object are obtained from edge intensity data obtained by performing edge detection of the object from the original image data. Calculating edge area weighting data for identifying the area by digitizing the area; Calculating, for each pixel position of the original image from the data, a correlation value representing the correlation of the edge / noise mixed image data between the original images, and calculating the edge of the subject from the edge area weighting data and the correlation value. In the region, the edge enhancement component is stored, and in the region where the noise component is dominant, the edge storage / noise suppression coefficient for suppressing the noise component is obtained.
Multiplying a noise suppression coefficient to calculate edge-enhanced / noise-suppressed image data; and adding the edge-enhanced / noise-suppressed image data to the smoothed image data to suppress noise components and enhance sharpness of the processed image data. Is obtained for each original image. 2. The image processing method according to claim 1, wherein image processing is performed on a plurality of original image digital data for each wavelength region obtained by photographing the same subject in at least six or more different wavelength regions. 3. The edge intensity data is obtained by calculating a local standard deviation of image data within a certain range around each pixel of the original image for each original image for each pixel. The image processing method according to claim 1, wherein the image processing method is obtained by taking an average value of the standard deviation in the original image. 4. The edge area weighting data is obtained by reducing and converting a numerical value from 0 to 1 obtained by normalizing the edge strength data to a value from 0 to 1 while setting an upper limit value to 1. The image processing method according to claim 3, which is obtained. 5. The image processing method according to claim 1, wherein the correlation value is obtained by taking an absolute value of an average value of the edge / noise mixed image data in the plurality of original images. . 6. The image processing method according to claim 1, wherein said edge preserving / noise suppressing coefficient is obtained by multiplying said correlation value by said edge area weighting data. 7. The image processing apparatus according to claim 1, wherein the processed image data includes
The image processing method according to any one of claims 1 to 6, wherein the image processing method is obtained by selectively suppressing only a noise component while preserving an edge enhancement component according to a noise suppression coefficient. 8. An image processing apparatus for performing image processing on a plurality of original image digital data (original image data) for each wavelength region obtained by photographing the same subject in at least four or more different wavelength regions. A sharpness enhancement processing section that performs sharpness enhancement processing on the original image data to create sharpness enhanced image data; a smoothing processing section that performs smoothing processing from the original image data to create smoothed image data; An edge detector for detecting edge of the subject from the original image data to obtain edge intensity data; and, based on the edge intensity data obtained by the edge detector,
An edge area weighting data calculating unit for obtaining edge area weighting data for discriminating an edge area and a noise area of the subject by numerically identifying; and the smoothing from the sharpness enhanced image data obtained by the sharpness enhancement processing unit corresponding to the wavelength area. An edge / noise mixed component extraction unit for subtracting the smoothed image data obtained by the processing unit and extracting edge / noise mixed image data in which an edge enhancement component and a noise component are mixed for each original image; An image obtained by calculating a correlation value indicating a correlation between the original images of the edge / noise mixed image data obtained by the edge / noise mixed component extraction unit for each pixel position of the original image from the mixed image data An inter-correlation calculation unit, and the edge region weighting data obtained by the edge region weighting data calculation unit From the correlation value obtained by the inter-image correlation calculation unit, the edge emphasizing component is stored in the edge region of the subject, and the edge storing / noise suppressing coefficient for suppressing the noise component is suppressed in the region where the noise component is dominant. A storage / noise suppression coefficient calculation unit; and the edge / noise mixture image data obtained by the edge / noise mixture component extraction unit and the edge storage / noise suppression coefficient obtained by the edge storage / noise suppression coefficient calculation unit An edge emphasis / noise suppression image data processing unit that obtains edge emphasis / noise suppression image data by using the smoothing image data and the edge emphasis / noise suppression image data processing unit obtained by the smoothing processing unit The obtained edge-enhanced / noise-suppressed image data is added to obtain processed image data in which noise components are suppressed and sharpness is enhanced. An image processing apparatus comprising: a noise suppression / sharpness enhancement processing operation unit that obtains the following. 9. The noise suppression / sharpness enhancement processing operation unit according to claim 8, wherein only the noise component is selectively suppressed while storing the edge enhancement component according to the edge storage / noise suppression coefficient. Image processing device. 10. A multi-band color photographic system for obtaining a plurality of original images obtained by photographing the same subject in at least four or more different wavelength regions, wherein the multi-band color photographic system according to claim 8 or 9 is used. A multi-band color photographic system including an image processing device for noise suppression and sharpness enhancement and an arithmetic unit for obtaining a subject spectral reflectance distribution.