JP4438363B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus that removes the influence of dust or the like in image data taken with an electronic camera or the like.

  Conventionally, in order to correct the influence of dust mixed in an optical system during the manufacture of a video camera, a technique for photographing a white pattern for each aperture value and recording correction information in advance is disclosed in Japanese Patent Laid-Open No. 9-51459. Is disclosed. Further, as a countermeasure against dust that may change constantly in the field of copying machines, a technique for detecting dust by taking in white reference data having a uniform reflecting surface before reading a document is disclosed in Japanese Patent Laid-Open No. 10-294870 and other special techniques. This is disclosed in Japanese Utility Model Publication No. 11-27475. Furthermore, in the scanner field, as a substitute for this white reference data, there is a method of providing an infrared light sensor, obtaining transmittance data simultaneously with visible light data, and obtaining a transmittance attenuation signal due to film defects. U.S. Pat. No. 6,195,161.

JP-A-9-51459 Japanese Patent Laid-Open No. 10-294870 Japanese Patent Laid-Open No. 11-27475 USP 6,195,161

  However, conventional cameras are only intended for fixed dust adhered to optical components during manufacturing, and dust that changes with the frequency of use or the passage of time has not been taken into account. In an interchangeable lens type single-lens reflex camera that has begun to spread today, the reflection of dust that changes with time tends to be a big problem, especially because the optical components at the front of the image sensor are exposed.

  On the other hand, in a copier or a scanner, dust data is acquired before or at the same time as the main scan to deal with dust having a temporal change. However, the structure is different from the camera, and it has a uniform illumination means for the original or film surface at a fixed distance, and it has a completely uniform reflection surface or a new infrared illumination means. By doing so, it is relatively easy to obtain transmittance data. However, it is usually difficult for electronic cameras to obtain such complete uniform surface transmittance data except for inspection during manufacturing.

  Further, the copying machine and the scanner are basically fixed optical systems, and there is no need to consider that dust changes due to changes in the optical system. On the other hand, conventional video cameras do not cope with changes in optical conditions other than the aperture value.

According to the first aspect of the present invention, there is provided image input means for inputting an image photographed by an image sensor through an optical system and subjected to nonlinear gradation processing different from the square root characteristic, and a value of a pixel of interest in the input image. Defect information creating means for creating defect information relating to a projected image of a defect in the middle of an optical path generated in an image based on an average value of a plurality of pixels within a predetermined range including the target pixel; and defect information The creation means specifies the gamma processing method applied to the input image using the tag information in the image file, and uses the inverse gamma curve generated based on this to temporarily linearize the gradation of the input image. After returning to the gradation, it is further converted into a non-linear gradation with a square root characteristic, an edge part in the image is detected in the non-linear gradation space with the square root characteristic, and a flat part region is extracted based on the detected edge part, Extracted flat Based on the information of the area, the area where the flatness is assured in the image, characterized by creating a defect information by calculating the average value.
According to a second aspect of the present invention, in the image processing device according to the first aspect, in the gradation space returned to the linear gradation, the value of the target pixel and the values of a plurality of pixels within a predetermined range including the target pixel are obtained. Defect information is created by calculating a relative ratio with an average value.
According to a third aspect of the present invention, in the image processing apparatus according to the second aspect, the inverse value of the relative ratio created by the defect information creating means is multiplied by the value of the corresponding pixel in the gradation space returned to the linear gradation. And a defect correcting means for correcting the defect.
According to a fourth aspect of the present invention, in the image processing apparatus according to any one of the first to third aspects, the image input means applies the compression processing to a compressed image that has been compressed into JPEG, JPEG2000, or TIFF. It is characterized by inputting after decompressing the compression.
The invention of claim 5 is characterized in that to perform the functions of the image processing apparatus according to claim 1 to the computer.

  According to the present invention, even if an image photographed with an electronic camera or the like is subjected to nonlinear gradation processing for final output, defects such as dust can be removed neatly without being influenced by the state of the optical system. .

-First embodiment-
(Configuration of electronic camera and personal computer)
FIG. 1 is a diagram illustrating a configuration of an interchangeable lens type single-lens reflex electronic still camera (hereinafter referred to as an electronic camera). The electronic camera 1 has a variable optical system 3 including a camera body 2 and a mount type interchangeable lens. The variable optical system 3 has a lens 4 and a diaphragm 5 inside. Although the lens 4 is composed of a plurality of optical lens groups, in the figure, it is represented by a single lens, and the position of the lens 4 is referred to as a main pupil position (hereinafter simply referred to as the pupil position). The variable optical system 3 may be a zoom lens. The pupil position is a value determined by the lens type and the zoom position of the zoom lens. May vary depending on focal length.

  The camera body 2 includes a shutter 6, an optical component 7 such as an optical filter or a cover glass, and an image sensor 8. The variable optical system 3 can be attached to and detached from the mount portion 9 of the camera body 2. Further, the variable optical system 3 transmits optical parameters such as information on the pupil position and information on the aperture value to the control unit 17 (FIG. 2) of the electronic camera 1 via the mount unit 9. The aperture value changes from F2.8 to F22, for example.

Reference numeral 10 denotes dust adhering to the surface of the optical component 7 in front of the image sensor 8. As a result of an experiment for changing the aperture value and pupil position of the variable optical system 3 and evaluating the change of dust shadows reflected in the photographed image, the following two facts were found.
(1) The size of dust shadow and the light transmittance change depending on the aperture value.
(2) The dust position shifts depending on the pupil position of the lens.
From these two experimental facts, it can be seen that even if the dust is attached to the fixed position, the way the dust is reflected changes every time the photographing condition (aperture value and pupil position) of the lens changes. A technique for removing the influence of dust on such a variable optical system will be described below.

  FIG. 2 is a block diagram of the electronic camera 1, a PC (personal computer) 31, and a peripheral device. The PC 31 functions as an image processing apparatus, acquires image data from the electronic camera 1, and performs dust influence removal processing described later.

  The electronic camera 1 includes a variable optical system 3, an optical component 7, a shutter 6 (not shown in FIG. 2), an image sensor 8, an analog signal processing unit 12, an A / D conversion unit 13, a timing control unit 14, and an image processing unit 15. , An operation unit 16, a control unit 17, a memory 18, a compression / decompression unit 19, a display image generation unit 20, a monitor 21, a memory card interface unit 22, and an external interface unit 23.

  The imaging element 8 images a subject through the variable optical system 3 and outputs an image signal (imaging signal) corresponding to the captured subject image. The imaging element 8 has a rectangular imaging region composed of a plurality of pixels, and sequentially outputs image signals, which are analog signals corresponding to the charges accumulated in the pixels, to the analog signal processing unit 12 in units of pixels. Output. The image sensor 8 is composed of, for example, a single-plate color CCD. The analog signal processing unit 12 includes a CDS (correlated double sampling) circuit, an AGC (auto gain control) circuit, and the like, and performs predetermined analog processing on the input image signal. The A / D conversion unit 13 converts the analog signal processed by the analog signal processing unit 12 into a digital signal. The timing control unit 14 is controlled by the control unit 17 and controls the timing of each operation of the image sensor 8, the analog signal processing unit 12, the A / D conversion unit 13, and the image processing unit 15.

  The memory card interface unit 22 interfaces with a memory card (card-like removable memory) 30. The external interface unit 23 interfaces with an external device such as the PC 31 via a predetermined cable or wireless transmission path. The operation unit 16 corresponds to a release button, a mode switching selection button, or the like. The monitor 21 displays various menus and displays a subject image captured by the image sensor 8 and a reproduced image based on image data stored in a memory card. The output of the operation unit 16 is connected to the control unit 17, and the output of the display image generation unit 20 is connected to the monitor 21. The image processing unit 15 is composed of, for example, a one-chip microprocessor dedicated to image processing.

  The A / D conversion unit 13, the image processing unit 15, the control unit 17, the memory 18, the compression / decompression unit 19, the display image generation unit 20, the memory card interface unit 22, and the external interface unit 23 are mutually connected via a bus 24. It is connected to the.

  A monitor 32, a printer 33, and the like are connected to the PC 31, and an application program recorded on the CD-ROM 34 is installed in advance. In addition to a CPU, a memory, and a hard disk (not shown), the PC 31 includes a memory card interface unit (not shown) that interfaces with the memory card 30, an electronic camera 1, etc. via a predetermined cable or wireless transmission path. An external interface unit (not shown) that interfaces with an external device is provided.

  In the electronic camera 1 configured as shown in FIG. 2, when the shooting mode is selected by the operator via the operation unit 16 and the release button is pressed, the control unit 17 causes the image sensor 8 to operate via the timing control unit 14. The timing control is performed on the analog signal processing unit 12 and the A / D conversion unit 13. The imaging element 8 generates an image signal corresponding to the optical image formed in the imaging area by the variable optical system 3. The image signal is subjected to predetermined analog signal processing in the analog signal processing unit 12, and is output to the A / D conversion unit 13 as an image signal after analog processing. The A / D conversion unit 13 digitizes the analog-processed image signal and supplies it to the image processing unit 15 as image data.

  In the electronic camera 1 of the present embodiment, the image sensor 8 has a case where the most representative R (red), G (green), and B (blue) color filters of a single-plate color image sensor are arranged in a Bayer array. For example, it is assumed that the image data supplied to the image processing unit 15 is represented in the RGB color system. Each pixel constituting the image data has color information of any one of RGB color components. Here, one photoelectric conversion element constituting the image sensor 8 is referred to as a pixel, and one unit of image data corresponding to this pixel is also referred to as a pixel. An image is also a concept composed of a plurality of pixels.

  The image processing unit 15 performs image processing such as interpolation, gradation conversion, and contour enhancement on such image data. The image data that has undergone such image processing is subjected to predetermined compression processing by the compression / decompression unit 19 as necessary, and is recorded in the memory card 30 via the memory card interface unit 22. Image data for which image processing has been completed may be recorded on the memory card 30 without being subjected to compression processing.

  The image data for which image processing has been completed is provided to the PC 31 via the memory card 30. It may be provided to the PC 31 via the external interface 23 and a predetermined cable or wireless transmission path. It is assumed that the image data for which image processing has been completed has undergone interpolation processing, and color information of all RGB color components is present in each pixel.

(Dust removal process)
Next, processing for removing the influence of dust in each captured image data will be described. In the first embodiment, a method for detecting and removing dust in a correction target image will be described. The dust contained in the middle of the optical path is generally blurred, and includes many spots that are not covered by a normal edge detection filter. There are various types of garbage ranging from large garbage to small garbage, and the F value dependency is large. Small garbage is in such a blurred state even when it is narrowed down, and disappears on the open side. On the other hand, large garbage is in such a blurred state on the open side. Using this property, the first embodiment employs a method of self-extracting dust from a correction target image itself and performing self-correction without a reference image with uniform illumination.

  In the self-extraction of dust, transmittance data is obtained using a kernel described later. In order to obtain an accurate dust transmittance signal, it is necessary to use a kernel in the flat portion. First, in order to detect the flat portion, the image structure of the correction target image is examined using an edge filter. Here, the flat portion refers to a partially uniform region in the image. In the present embodiment, a completely non-uniform blue sky including a gradation and a peripheral darkening of the lens, a nearly uniform wall surface, and the like are also detected as a uniform region. Even if the flat portion image detected as a uniform region includes gradation, shading, etc., the calculation is performed so as not to be affected by gradation, shading, etc. when calculating the transmittance.

(Operation on the electronic camera side)
FIG. 3 is a diagram for describing a photographing procedure on the electronic camera 1 side in the first embodiment. 1) Normal photographing 301 is performed at the pupil position P1 and the aperture value A1, and the correction target image data 1 is output. 2) Normal photographing 302 is performed at the pupil position P2 and the aperture value A2, and the correction target image data 2 is output. 3) Normal photographing 303 is performed at the pupil position P3 and the aperture value A3, and the correction target image data 3 is output. 4) The normal photographing 304 is performed at the pupil position P4 and the aperture value A4, and the correction target image data 4 is output. In other words, the electronic camera 1 performs photographing (normal photographing) toward a subject to be photographed to obtain a correction target image. Here, outputting image data means recording on the memory card 30 or directly outputting to the PC 31 via the external interface 23.

(Image processing device side operation)
Image data captured by the electronic camera 1 is provided to the PC 31 after predetermined image processing. The PC 31 performs dust removal processing using the correction target image data. The PC 31 may be said to be an image processing apparatus that performs dust removal processing. The correction target image data is input to the PC 31 after the Bayer array RGB interpolation processing is completed.

  FIG. 4 is a diagram illustrating a flowchart of processing performed by the PC 31 that is the image processing apparatus. In step S29, the compressed image is decompressed. In step S30, inverse gamma correction is performed. In step S31, a luminance plane is generated. In step S32, an edge map is generated by gamma correction of the luminance plane, edge extraction filter processing, and threshold determination. In step S33, a dark portion is added to the edge map. In step S34, the edge map is enlarged. In step S35, conversion to a flat map is performed. In step S36, a self gain extraction process is performed. In step S37, self-gain correction processing is performed. Details of each step will be described below.

Compressed Image Decompression Process In step S29, since the inverse gamma correction described later and the dust effect removal process of the main subject cannot be performed in the compressed state, the decompression process of the compressed image is performed according to the image format to be handled. For example, decompression processing is required for an image that has been irreversibly compressed, such as JPEG or JPEG 2000, and an image that has been reversibly compressed with an optional function such as TIFF.

Inverse gamma correction In step S30, inverse gamma correction is performed. Usually, a JPEG image or a TIFF image output as 8 bits is subjected to gradation gamma processing. These are slightly different from the square root type characteristics described later, and are not optimal for the purpose of separating dust and in-image edges. At this time, the gamma processing method performed in the camera or external application software is embedded in a tag in each file such as JPEG or TIFF, and a reverse gamma correction curve is generated using this tag information. It shall be. The input signal of the nonlinear gradation is x ((0 ≦ x ≦ xmax), the output signal of the linear gradation is y (0 ≦ y ≦ ymax), and the conversion to the linear gradation is performed using the following equation (1). This conversion is performed for each of the R, G, and B components.

Here, gamma correction is a method that modifies the density of the image by changing the relationship between the input and output of the gamma curve, that is, the brightness, tone, color balance, etc. Can be adjusted. Each camera maker prepares multiple gamma curves to create various pictures for each image, and sets the mode such as high contrast, normal contrast, low contrast, etc., and allows the user to select the camera according to the image Is automatically selected. These gamma curves are defined for each section of the input signal y or are defined by a look-up table and cannot be uniquely expressed by one expression over the entire section. When the role is represented by one formula, it may be represented by formula (33). This process is important because the gamma curve prepared for making a picture often deviates greatly from this. For ease of explanation, the case of equation (33) is taken as an example, and when the value of γ is 1 in equation (33), it represents that the input signal and the output signal are directly proportional.

  The inverse gamma correction processed in the present embodiment means that the processing of equation (1) is performed by matching various gamma curves prepared in advance with the curve used for each image. For example, the nonlinear input / output relationship when γ = 0.3 shown in FIG. 5 is returned to the linear input / output relationship shown in FIG. 6 where γ = 1.

3) Generation of luminance plane In step S31, a luminance plane is generated. For each pixel [i, j] of the correction target image data, a linear gradation luminance signal Y is generated from the linear gradation RGB signal using the following equation (2). [i, j] indicates the position of the pixel.
Y [i, j] = (R [i, j] + 2 * G [i, j] + B [i, j]) / 4 ... (2)
Although it is possible to individually analyze each of the R, G, and B surfaces, basically, the influence of dust shadows only causes signal attenuation and is not related to color components. Therefore, here, conversion to a luminance component that can reduce the influence of random noise while effectively using all information is performed. In addition, by doing so, it is only necessary to analyze the luminance component single surface from the RGB3 surface, and the speed can be increased. The luminance component generation ratio is not limited to the above, and may be R: G: B = 0.3: 0.6: 0.1 or the like.

4) Generation of edge map In step S32, the edge extraction filter is applied to the luminance plane to separate the flat portion and the edge portion in the correction target image. Reflection of dust contained in the optical path in the image appears as dust shadow with very low contrast, and is often not detected by an edge extraction filter as in the prior art. If this fact is used in reverse, it can be assumed in many places that the edge portion extracted by the edge extraction filter is basically an edge in the image, not dust. In order to further distinguish between edges and dust in the image, first, gradation correction processing is performed on the luminance plane.

ここで、図７はγ＝０．５としたときの非線形階調の入力信号と出力信号の関係を示す図である。なお、上記式（３）はべき乗関数になっている。また、γ＝０．５は、べき乗関数が平方根関数となっている。このように式（３）が平方根となるような特性により補正を行うことを、ここでは平方根型のガンマ補正と呼ぶ。 4-1) Gamma correction of luminance plane It is assumed that the correction target image is input with linear gradation (γ = 1) and the above-described luminance plane is generated. At this time, the input signal is Y (0 ≦ Y ≦ Ymax), the output signal is Y ′ (0 ≦ Y ′ ≦ Y′max), and γ = 0.5. For example, gradation conversion as in the following equation (3) Apply.

Here, FIG. 7 is a diagram illustrating the relationship between the input signal and the output signal of the non-linear gradation when γ = 0.5. The above equation (3) is a power function. Further, when γ = 0.5, the power function is a square root function. In this way, the correction based on the characteristic that Equation (3) becomes the square root is referred to as square root type gamma correction.

  This conversion is a process of increasing the halftone contrast on the low luminance side and decreasing the contrast on the high luminance side. In other words, dust shadows are not noticeable in dark places and are conspicuous in bright places, so the contrast of dust shadows in bright places is reduced by this conversion. Yes. Accordingly, the degree of separation between the dust shadow and the normal edge contrast is improved. Further, in order to convert shot noise caused by the quantum fluctuation of Y ′ so that it can be handled uniformly over all gradations, it is best to set γ = 0.5 in view of the error propagation law. Here, the reverse gamma processing of 2) and the square root type gamma correction are equivalent to one tone conversion processing.

4-2) Edge Extraction Filter Processing Next, the edge extraction filter according to FIG. 8 and the following equation (4) is applied to the luminance surface Y ′ subjected to gamma correction. Let YH [i, j] be the edge extraction component of each pixel.
YH [i, j] = {| Y '[i-1, j] -Y' [i, j] | + | Y '[i + 1, j] -Y' [i, j] |
+ | Y '[i, j-1] -Y' [i, j] | + | Y '[i, j + 1] -Y' [i, j] |
+ | Y '[i-1, j-1] -Y' [i, j] | + | Y '[i + 1, j + 1] -Y' [i, j] |
+ | Y '[i-1, j + 1] -Y' [i, j] | + | Y '[i + 1, j-1] -Y' [i, j] |
+ | Y '[i-2, j-1] -Y' [i, j] | + | Y '[i + 2, j + 1] -Y' [i, j] |
+ | Y '[i-2, j + 1] -Y' [i, j] | + | Y '[i + 2, j-1] -Y' [i, j] |
+ | Y '[i-1, j-2] -Y' [i, j] | + | Y '[i + 1, j + 2] -Y' [i, j] |
+ | Y '[i-1, j + 2] -Y' [i, j] | + | Y '[i + 1, j-2] -Y' [i, j] |
+ | Y '[i-3, j] -Y' [i, j] | + | Y '[i + 3, j] -Y' [i, j] |
+ | Y '[i, j-3] -Y' [i, j] | + | Y '[i, j + 3] -Y' [i, j] |
+ | Y '[i-3, j-3] -Y' [i, j] | + | Y '[i + 3, j + 3] -Y' [i, j] |
+ | Y '[i-3, j + 3] -Y' [i, j] | + | Y '[i + 3, j-3] -Y' [i, j] |} / 24 ... (Four)
Here, the filter is designed to collect absolute value differences having a plurality of correlation distances from all directions without omission so that all the original edge portions of the image are extracted as much as possible.

4-3) Threshold Determination The edge extraction component YH is determined as a threshold by the following equations (5) and (6), the edge portion or the flat portion is classified, and the result is output to the edge map EDGE [i, j]. . The threshold value Th1 takes a value of about 1 to 5 for 255 gradations. The dust shadow existing on the edge portion is an area that does not require dust removal because the dust shadow is basically buried in the vibration signal of the edge portion and is not noticeable.
if YH [i, j]> Th1 EDGE [i, j] = 1 (edge part) ... (5)
else EDGE [i, j] = 0 (flat part) ... (6)
As described above, a region not detected by edge detection is called a flat portion.

  As described above, gradation conversion is performed in 4-1) to change the weighting between gradations, and threshold determination is performed with a constant threshold Th1 over all gradations in 4-3). However, almost the same effect can be obtained by performing edge extraction while maintaining the linear gradation, and setting a threshold value according to the luminance level and performing threshold determination.

5) Adding a dark portion to the edge map The edge map represents a region where gain map extraction described later should not be performed. In addition to the edge region, a region that is dangerous to extract a gain map is a dark region (dark portion). In the dark area, the S / N is poor, so the reliability is low even if the relative gain is extracted. Further, dust shadows existing on the dark part are hardly noticeable, so there is no need to remove dust. Therefore, in step S33, a dark region is also added to the edge map by the following equation (7). The threshold value Th2 is set to a value of about 20 or less for 255 linear gradations. This operation can be easily understood by describing this operation as “EDGE '= EDGE + DARK”.
if Y [i, j] ≦ Th2 EDGE [i, j] = 1 ... (7)

6) Enlargement processing of edge map In 8-1) to be described later, the relative ratio between the center value of the range (2a + 1) x (2b + 1) pixels in which the local average is taken in the flat portion and the average value is transmitted. Generate a rate map. Accordingly, in step S34, (2a + 1) x (2b + 1) pixel enlargement processing of the edge portion is performed by the following equation (8) so that the edge portion does not enter the kernel in advance. m = 1,2, ..., a, n = 1,2, ..., b.
if EDGE [i, j] = 1 EDGE [i ± m, j ± n] = 1 ... (8)

7) Conversion to flat map In step S35, the edge map EDGE [i, j] is converted to the flat map FLAT [i, j] by the following equations (9) and (10). Achieved by bit inversion. The flat region indicated by the flat map represents a region that may be self-extracted in the correction target image by the gain map extraction kernel formed of (2a + 1) x (2b + 1) pixels.
if EDGE [i, j] = 0 FLAT [i, j] = 1 (flat part) ... (9)
else FLAT [i, j] = 0 (edge part) ... (10)

8) Self-Gain Extraction In step S36, a transmittance map is generated (gain map extraction) for the area of FLAT [i, j] = 1, which consists of the following processing.

8-1) Local normalization processing (self gain extraction processing)
The flat area indicated by the flat map is not necessarily completely uniform. For such flat regions, local pixel value normalization (normalization) processing is performed, and the transmittance signal T [i, j] of each pixel in the region of FLAT [i, j] = 1 is obtained. , Using the following equation (11). That is, the relative ratio between the value of the pixel of interest [i, j] and the average pixel value in the local range including this pixel is determined for each pixel. And all the areas of FLAT [i, j] = 0 are set to T [i, j] = 1. As a result, nonuniformities such as gradation and shading included in the flat area data are eliminated without any problem in terms of algorithm, and only a decrease in transmittance due to dust shadows can be extracted. The transmittance of the entire image obtained in this way is called a transmittance map (gain map). The transmittance map indicates defect information on the flat area of the correction target image. The pixel value is a value of a color signal (color information) or a luminance signal (luminance information) of a color component in each pixel. For example, when represented by 1 byte, it takes a value from 0 to 255.

  Here, the range (2a + 1) × (2b + 1) pixels for which the local average is taken is set to be larger than the maximum assumed dust diameter. Ideally, if the area is about 3 times larger than the dust shadow, accurate transmittance data can be obtained. a represents the number of pixels extending left and right around the pixel of interest [i, j], and b represents the number of pixels extending up and down around the pixel of interest [i, j]. For example, if the pixel pitch of the image sensor 8 is 12 um and the distance between the imaging surface and the dust adhesion surface is 1.5 mm, when the aperture value is F22, the diameter of the huge dust is about 15 pixels and the aperture value is F4. The diameter of huge garbage is about 40 pixels. Therefore, it is preferable to set a = 40, b = 40, and set the local average range as an 81 × 81 pixel range. Further, if the sensor is in the millions of pixels class, it may be fixed at 101 × 101 pixels. These are only examples, and may be pixel ranges based on other numbers of pixels.

  The dust shadow greatly depends on the aperture value, and the small dust disappears as soon as the aperture is opened. However, the large dust may occupy a large area even though the shadow is thin even when the aperture is opened. Depending on the pixel pitch width of the image sensor, there may be a case where a round dust shadow is formed over several tens of pixels on the open side, and in that case, it becomes necessary to take a local average over a very wide range. Therefore, in order to increase the processing speed, there is no problem even if the processing is representatively performed with thinned pixels.

  The processing for calculating the relative ratio in the range of (2a + 1) x (2b + 1) pixels is called local normalization processing (self gain extraction processing). A filter that performs a relativization operation in the range of (2a + 1) x (2b + 1) pixels may be called a gain extraction kernel. FIG. 9 is a diagram illustrating a state in which the local normalization process is performed on the luminance surface. FIG. 9A is a diagram illustrating luminance signals of pixels arranged in a horizontal direction in the luminance plane. Reference numerals 41 and 42 indicate that the luminance signal is reduced due to dust. FIG. 9B is a diagram obtained by performing the above-described local normalization processing on the luminance signal of FIG. 9A. That is, pixel value normalization processing is performed in a local range. Reference numerals 43 and 44 correspond to the reference numerals 41 and 42 in FIG. 9A, and indicate the transmittance of the places where dust exists. In this way, nonuniformities such as gradation and shading included in the uniform surface data are eliminated, and only a decrease in transmittance due to dust shadows can be extracted. Thereby, the position of the dust and the degree of transmittance can be known at the same time.

8-2) Statistical analysis of transmittance map Average M for the entire image of the transmittance map obtained by the above-mentioned local normalization processing in order to distinguish dust information and random noise included in the aforementioned transmittance map. Is obtained from the following equation (12), and the standard deviation σ is obtained from the following equation (13). Nx and Ny represent the total number of pixels in the x and y directions.

8-3) Threshold determination Basically, the area ratio of the dust signal in the transmittance map is sufficiently small, and the statistical analysis result in 8-2) shows that the random noise associated with the quantum fluctuation of the transmittance signal ( It is thought that shot noise) is being evaluated. Reference numeral 46 obtained by enlarging the reference numeral 45 in FIG. 9 shows a state in which there is this fine random noise. When a histogram of the transmittance map is taken, a normal distribution with a standard deviation σ centering on an average value M (M is a value close to 1) is obtained. FIG. 10 is a diagram showing a histogram of the transmittance map. Since it is considered that the fluctuation range is not subjected to a change in transmittance due to dust shadows, the transmittance may be forcibly set to 1. That is, threshold determination is performed under the following conditions (14) and (15).
if | T [i, j] -M | ≦ 3σ then T [i, j] = 1 ... (14)
else T [i, j] = T [i, j] ... (15)

  Since the normally distributed random data is 99.7% if the range of ± 3σ is collected, the influence of random noise can be excluded almost accurately. The transmittance deviating from ± 3σ is an abnormal signal that can hardly be explained by statistical errors, and is considered to represent a phenomenon caused by a decrease in transmittance due to dust shadows. This abnormal portion is usually a value smaller than 1 in the case of dust shadows.

  However, there are some which show a value larger than 1 although the ratio is small. This is not the effect of dust shadows, but is a phenomenon seen when defects caused by striae (non-uniform refractive index) of optical low-pass filters cause interference fringes that make incident light stronger or weaker. is there. Thereby, this method can also be used for detecting defects of optical members other than dust contained in the optical path. Further, the influence of pixel defects in the image sensor can also be determined by this method. Although dust is more likely to remain near the image sensor 8 without being blurred, it can be accurately determined even when dust on the photographing lens is reflected with considerable blur.

When only the influence of dust shadows is removed, the threshold value may be determined according to the following conditions (16), (17), and (18).
if | T [i, j] -M | ≦ 3σ then T [i, j] = 1 ... (16)
else if T [i, j]> 1 T [i, j] = 1 ... (17)
else T [i, j] = T [i, j] ... (18)
Since the average value M used for determination always takes a value close to 1, it may be replaced with 1.

  In this way, two types of defect information, map information (determined based on whether T = 1) or not, and transmittance information representing the degree of defect, can be obtained at the same time. Note that the transmittance map described above may be referred to as a gain map because it represents a local relative gain.

  Usually, a defect such as dust is detected by a differential filter for edge detection. However, when dust in the middle of the optical path is targeted, it appears as a dust shadow with very low contrast with the surroundings due to optical blurring. In such a case, the differential filter is very insensitive and can hardly be detected. However, as described above, when a determination method using the statistical property of transmittance is used, it is possible to detect dust with very high sensitivity, and it is possible to correct the influence of foreign matter in the target optical path.

9) Self-Gain Correction In step S37, self-gain correction is performed. The correction target image data is corrected using the transmittance map obtained as described above. As shown in equations (19), (20), and (21), gain correction is performed by multiplying the R, G, and B values of the correction target image data by the reciprocal of the transmittance signal.
R [i, j] = R [ij] / T [i, j] ... (19)
G [i, j] = G [ij] / T [i, j] ... (20)
B [i, j] = B [ij] / T [i, j] ... (21)

  As a result, a decrease in luminance due to dust shadows can be neatly corrected. In addition, it is possible to prevent unnecessary correction since the transmittance map is subjected to threshold determination in places where there is no need for correction. That is, since the influence of random noise is removed from the transmittance T of the dust-free portion, there is no concern that the noise of the RGB signal is amplified.

10) Gamma correction For the R, G, and B components that have been gain-corrected for the dust transmittance in the linear gradation as described above, the original nonlinear gradation characteristics are restored so that the user can immediately confirm the image in step S38. The conversion to be returned is performed by the following equation (34).
x '= γ (y) ... (34)

  As described above, in the first embodiment, even with an ordinary electronic camera that does not have a special mechanism for preventing dust, an image that has been subjected to gamma processing for final output taken at an arbitrary time is appropriately corrected. can do. Further, even if there is no uniform plane reference image data, dust in the correction target image itself can be self-extracted and corrected. That is, a region that satisfies a predetermined condition that can guarantee flatness as described above is extracted from one captured image. The extracted same area is set as a reference image and a correction target image. Further, there is no need to take into account the influence of the variable optical system. In particular, it is effective in removing small dust shadows that occupy a huge number.

-Second Embodiment-
In the second embodiment, a reference image for obtaining dust information is photographed only once, and the transmittance map is not based on the reference image but using information on the dust position, as in the first embodiment. A method of self-extraction from the correction target image itself is adopted. In the first embodiment, in the edge map extraction, large dust is extracted as an edge, and correction may not be performed. However, the second embodiment is a defect of the first embodiment. It will also deal with. That is, while using the method for generating a highly reliable transmittance map according to the first embodiment, it is slightly burdened on the user, but a standard image that is uniform once a day to one month for dust reference. This is a method for further improving the dust removal performance by using an image obtained by shooting.

  The reference image is not a completely uniform white reference data, but a blue sky, a nearly uniform wall surface, a gray chart, a plain paper surface, or the like is photographed and used instead. The reference data in this case may include lens peripheral light reduction, subject gradation, image sensor shading, and the like. It is assumed that the reference data can be acquired in a situation where it can be easily photographed at a location that is actually familiar, and does not require strict uniformity, and is converted into a uniform one by an algorithm on the image processing side. Differences in optical conditions (pupil position, F value) between the reference image and the correction target image are dealt with on the image processing side.

  Note that the configurations of the electronic camera 1 and the PC 31 as the image processing apparatus are the same as those in the first embodiment, and a description thereof will be omitted.

(Operation on the electronic camera side)
FIG. 11 is a diagram for explaining a photographing procedure on the electronic camera 1 side in the second embodiment. 1) Uniform plane photography 201 is performed at the pupil position P0 and the aperture value A0, and the reference image data 0 is output. 2) The normal photographing 202 is performed at the pupil position P1 and the aperture value A1, and the correction target image data 1 is output. 3) The normal photographing 203 is performed at the pupil position P2 and the aperture value A2, and the correction target image data 2 is output. 4) The normal photographing 204 is performed at the pupil position P3 and the aperture value A3, and the correction target image data 3 is output. That is, first, the electronic camera 1 is photographed with a uniform surface facing the sky or a wall surface (uniform surface photographing), and then the electronic camera 1 is photographed as needed toward the subject to be photographed (normal photographing).

  Here, it is assumed that the aperture value A0 of the reference image is taken in a state in which the aperture is most narrowed in a variable range prepared in the variable optical system 3. The most narrowed aperture value is, for example, about F22 for a standard lens. On the other hand, it is assumed that the aperture value of the correction target image is the same as that of the reference image or closer to the open side.

  Uniform surface photography can be omitted as long as the state of dust adhesion does not change. Although the number of insertions of uniform plane photography has never been too many, data that is usually only once a day can be effective garbage data. The determination as to whether or not to perform uniform plane imaging is left to the photographer. However, if the previously performed uniform plane imaging is too far in time, the reference data obtained by the uniform plane imaging may be unreliable. Therefore, only the reference image data for uniform plane imaging within a predetermined time from normal imaging may be used. Further, it is not always necessary to perform uniform plane imaging first. You may use the reference image data of the uniform surface photography performed later. When there are a plurality of uniform plane imaging before and after the normal imaging, the reference image data of the uniform plane imaging that is closest in time may be used. Alternatively, if you are concerned about the possibility of newly attached dust, you may select one from the second closest one before and after photographing.

(Image processing device side operation)
FIG. 12 is a diagram illustrating a flowchart of processing performed by the PC 31 that is the image processing apparatus.

<Processing for reference image>
1) Compressed image decompression process When the standard image file is compressed, the decompression process is performed in the compressed image decompression process in step S39 in the same manner as the method performed on the correction target image in 1) of the first embodiment. I do.

2) Inverse gamma correction When the gradation of the reference image is not a linear gradation but a non-linear gradation subjected to a gamma correction, the correction is performed on the image to be corrected in step S40 in 2) of the first embodiment. Reverse gamma correction is performed in the same way as the method.

3) Generation of Luminance Surface The generation of the luminance surface in step S41 is the same as the method performed on the correction target image in 3) of the first embodiment.

4) Generation of transmittance map In step S42, a transmittance map is generated according to the following procedure.
4-1) Local normalization processing (gain extraction processing)
As described above, the reference image data is not necessarily completely uniform. Therefore, the generated luminance surface is not completely uniform. For such a luminance surface, the processing of Expression (11) performed for only the flat region in 8-1) of the first embodiment is performed for all the pixels here. Thereby, the transmittance signal T [i, j] of each pixel is calculated.

4-2) Low-pass processing of transmittance map The low-pass processing of the transmittance map may be selectable, but it is preferable to include this processing because it is most effective. Since the transmittance signal T [i, j] includes random noise associated with the quantum fluctuation of the luminance signal, there is a region where the influence of dust shadows remains slightly at a level where the transmittance is close to 1. Because of the randomness, if the threshold determination in 4-4) is performed, dust shadows may be extracted from the spots. In order to prevent this, when the dust shadows are grouped by the low-pass filter according to the following equation (22), the appearance is slightly improved.
T [i, j] = {4 * T [i, j]
+ 2 * (T [i-1, j] + T [i + 1, j] + T [i, j-1] + T [i, j + 1])
+ 1 * (T [i-1, j-1] + T [i-1, j + 1] + T [i + 1, j-1] + T [i + 1, j + 1])} / 16 ... (22)

4-3) Statistical analysis of transmittance map Statistical analysis of the transmittance map is the same as the method performed in 8-2) of the first embodiment.

4-4) Threshold Determination The threshold determination is the same as the method performed in 8-3) of the first embodiment.

5) Pupil position conversion of transmittance map In step S43, the pupil position of the transmittance map is converted. When the pupil positions of the reference image and the corrected image are different from each other, the pupil position of the reference image is converted into a dust position predicted to appear when viewed from the pupil position of the correction target image. FIG. 13 is a diagram illustrating how the dust shadow position changes as the pupil position changes. FIG. 13A is a diagram illustrating the relationship between the pupil position, dust, and the imaging surface of the imaging device 8. FIG. 13B is a diagram illustrating a state in which dust shadows are moving on the imaging surface with changes in the pupil position.

ただし、距離lは光学部品の厚みを空気中の光路長に換算した値である。 As apparent from FIG. 13, when the pupil position is different, the position of dust reflected in the image is shifted in the radial direction from the optical axis 51, that is, the center of the image. Here, the amount Δr of dust deviated in the radial direction from the optical axis 51 in the image is estimated. Assuming that the pupil position of the reference image is P0, the pupil position of the correction target image is P0 ′, and dust is attached at a distance l from the imaging surface, Δr can be calculated by the following equation (23).

However, the distance l is a value obtained by converting the thickness of the optical component into an optical path length in the air.

ずれ量Δrは、光軸５１から距離が遠くになるに従い大きくなる。実際の画像の周辺部では、瞳位置の値によっては数十画素に及ぶこともある。 The transmittance map T [i, j] of the reference image is displaced to [r ′, θ] by the following equation (24) on the polar coordinates [r, θ], and the transmittance map T on the coordinates [i, j] 'Convert to [i, j].

The shift amount Δr increases as the distance from the optical axis 51 increases. In the periphery of an actual image, it may reach several tens of pixels depending on the value of the pupil position.

6) F value conversion of transmittance map In step S44, F value conversion of the transmittance map is performed. When the aperture values of the reference image and the corrected image are different from each other, the dust diameter and the transmittance of the reference image are converted into an F value into the dust diameter and the transmittance of the correction target image at the more open aperture value. FIG. 14 is a diagram illustrating how the size of the dust shadow changes when the F value, which is the aperture value, changes. FIG. 14A shows a case where the F value is large, and FIG. 14B shows a case where the F value is small. As is clear from FIG. 14, when the definition formula of the F value (F = focal length / effective aperture of the lens) is applied to the distance l from the imaging surface having a similar relationship to the dust adhesion position and the dust spread Γ, the following formula ( 25) is established.

  When l is divided by the pixel pitch a [mm / pixel] of the image sensor, the dust diameter can be expressed by the number of pixels. In this way, it can be predicted that the dust in the point image spreads to the size of the width Γ when the stop has the F value.

  On the other hand, the distribution function of the point image is considered to spread the dust shadow by applying light to the point image dust evenly from each incident angle of the lens opened within the aperture value. You can assume that it has a function. Therefore, the F value conversion can be performed to accurately predict the dust diameter and the transmittance by applying a uniform low-pass filter process represented by a filter width Γ pixel. As the low-pass filter, a circular non-separable filter having a diameter of Γ is considered to be normal, but there is no problem with a vertical Γ and horizontal Γ square separated filter in consideration of speeding up the processing.

  For example, when l = 0.5 mm and a = 5 μm / pixel, the transmission map of F22 is converted to F16, F11, F8, F5.6, and F4. The original filter coefficient is represented in a format as shown in FIG. Using the one-dimensional filter coefficient of FIG. Note that the one-dimensional filter coefficient of the aperture value F16 has a coefficient of 0.5 at both ends and seven coefficients. This is because the even-width spread is filtered in an odd-width range that spreads evenly in the horizontal and vertical directions around the pixel of interest. FIG. 16 is a diagram illustrating the aperture value F16 filter as a two-dimensional filter.

  By performing the above conversion process, the transmittance map of the reference image is converted into a transmittance map in the pupil position and F value states of the correction target image. That is, the transmittance map equivalent to the transmittance map generated under the optical condition where the correction target image is captured is generated as the transmittance map of the reference image.

7) Determination of threshold value of transmittance map In step S45, the threshold value of the transmittance map is determined. When the F value conversion of the transmittance map is performed, a number of pixels close to the transmittance 1 in which dust shadows are almost disappearing are generated by the low-pass filter process. In order to distinguish these from clear dust shadows, threshold determination is performed again using the following equations (26) and (27). Here, the standard deviation value σ calculated in the “transmission map generation” process of 4) is used again. The pupil position and the transmittance map after F-number conversion are represented by T ′ [i, j].
if | T '[i, j] -1 | ≦ 3σ then T' [i, j] = 1 ... (26)
else T '[i, j] = T' [i, j] ... (27)

8) Conversion to dust map In step S46, the transmittance map is binarized by the following equations (28) and (29) and converted to a dust map dmap [i, j].
if T '[i, j] <1 dmap [i, j] = 1 ... (28)
else dmap [i, j] = 0 ... (29)
Here, the determination of Expression (28) may be made such that T ′ [i, j] <0.95 with a little more margin.

9) Dust Map Enlarging Process In step S47, the dust map is enlarged by the error assumed in the pupil position conversion by the following equation (30), so that the area within the allowable error includes dust. Make a garbage map. Here, for example, an error of ± 3 pixels is expected. m = 1,2,3 and n = 1,2,3.
if dmap [i, j] = 1 dmap [i ± m, j ± n] = 1 ... (30)

<Processing for correction target image>
1) Compressed image decompression process The decompressed image decompression process in step S49 is the same as 1) of the first embodiment.

2) Inverse gamma correction The inverse gamma correction in step S50 is the same as 2) in the first embodiment.

3) Generation of Luminance Surface The generation of the luminance surface in step S51 is the same as 3) of the first embodiment.

4) Edge map generation The generation of the edge map in step S52 is the same as 4) of the first embodiment.

5) Add dark part to edge map The process of adding the dark part to the edge map in step S53 is the same as 5) of the first embodiment. This operation can be easily understood by describing this operation as “EDGE '= EDGE + DARK”.

6) Excluding the dust region from the edge map In step S54, the dust region is excluded from the edge map. Most of the dust shadows are not extracted because the contrast is low. However, some of the dust shadows are large dust and have high contrast, and the edges may be extracted. In particular, it occurs in some dust shadows in the correction target image photographed with narrowing down. In order to prevent these dust shadows from deviating from the designation of the gain extraction region as an edge region, the dust position is forcibly excluded from the edge portion by the following equation (31) using the dust map information obtained in step S46. Process. Here, in order to prevent the edge region from being cut off too much, the dust map before the dust map enlargement process in step S47 is used. This operation can be easily understood by describing this operation as “EDGE '' = EDGE + DARK-DUST”.
if dmap [i, j] = 1 EDGE [i, j] = 0 ... (31)

7) Edge Map Enlargement Process The edge map enlargement process in step S55 is the same as step S34 in the first embodiment.

8) Conversion to flat map The conversion to the flat map in step S56 is the same as step S35 in the first embodiment.

9) Identification of self gain extraction region In step S57, the self gain extraction region is identified. It is most reasonable from the viewpoint of preventing erroneous correction of the correction target image to perform dust removal only on a flat part and an area specified as a dust area. Therefore, area information that satisfies both conditions is obtained by the following equation (32) and substituted into the flat map. That is, FLAT = 1 only when 1 is set in both FLAT and dmap, and FLAT = 0 otherwise.
FLAT [i, j] = FLAT [i, j] * dmap [i, j] ... (32)

10) Self-Gain Extraction Unlike the first embodiment, the self-gain extraction in step S58 is performed only for the local normalization process (gain extraction process). Subsequent statistical analysis of the transmittance map and dust region limitation processing by threshold processing are unnecessary because they are already limited to the vicinity of dust by the processing of “9) Identification of self-gain extraction region” described above. The local normalization process (gain extraction process) is the same as that in the first embodiment. As described above, by performing the dust search by expanding the gain extraction area around the dust by the error of the pupil position conversion accuracy by the process of step S47, dust can be extracted without omission.

  Here, low-pass processing is performed on the self-extracted transmittance map. Low-pass processing similar to the processing performed on the reference image in the present embodiment is performed on all self-extracted regions of T [i, j], and the pixels [i, j included in T [i, j] ] Of the fluctuation component. In the present embodiment, the low-pass process is an important process because the self-gain correction is performed by extracting the self-gain without limiting the threshold value process by statistical analysis only to the local region of the dust position. That is, the pixel value and the transmittance value T [i, j] fluctuate in the same direction before the low-pass processing, so if self-gain correction described later is performed without performing the low-pass processing, the image is entirely covered only in the local region. Cheap. Therefore, if the fluctuation component of T [i, j] is removed, the fluctuation component of the pixel value is not lost, and the continuity of graininess with the peripheral portion can be maintained. This is particularly effective when there is a lot of randomness such as a high-sensitivity image. Here, the low-pass filter may be designed to be a little stronger than that for the reference image, and a large dust portion that is easily affected by the low-pass filter (where T [i, j] is considerably smaller than 1). However, the low-pass filter process may be removed.

11) Self-gain correction The self-gain correction in step S59 is the same as step S37 in the first embodiment. Even if the pupil position conversion accuracy of the transmittance map of the reference image is poor, the dust transmittance information is extracted from the correction target image itself, so that clean correction without any deviation can be performed. The self-gain extraction is performed only in the self-gain extraction area specified in step S57. Therefore, the correction process is performed only within this range, and the processing load is reduced.

12) Gamma correction The gamma correction in step S60 is the same as step S38 in the first embodiment.

  As described above, in the second embodiment, by effectively using the dust map information of the reference image, it is possible to remove all dust from large dust to small dust in the correction target image that has been subjected to gamma processing for final output. Self-extraction is possible without leakage. In addition, even when the pupil position conversion accuracy of the transmittance map of the reference image is poor, dust can be self-extracted without any problem. Further, the burden on the photographer for the reference image photographing can be very small.

  In the image processing apparatuses according to the first embodiment and the second embodiment described above, an image is generated in an image that has been subjected to gamma processing for final output that has been taken with an electronic camera at an arbitrary use time and an arbitrary use condition. It is possible to appropriately correct a defect such as a black stain due to the influence of dust or the like and reproduce a high-quality image.

  In the second embodiment, in order to create the transmittance map, a reference image that the photographer thinks is almost uniform is photographed, and the photographed reference image is subjected to local standard processing or the like. A transmission map was created. However, there may be a small pattern or the like in the reference image that the photographer thinks is almost uniform. This can be dealt with by basically shooting the subject with a blur. For example, the paper may be photographed at a position closer than the shortest photographing distance of the lens. Even if there is a small pattern, it is almost uniform enough to achieve the purpose if it blurs to an image that changes slowly over a wide range than the gain extraction kernel of (2a + 1) x (2b + 1) size Can be a reference image.

  In the second embodiment, the self-gain extraction region is specified in step S57, and correction is performed in a limited range in step S58. Thus, the method of limiting the correction range to the peripheral range (neighboring region) including the dust region can also be applied in the first embodiment. In the first embodiment, dust may be identified from the obtained transmittance map and its peripheral area may be obtained.

  In the above-described embodiment, the JPEG, JPEG2000, and TIFF optional functions have been described as examples of the image compression format. However, the present invention can be applied to any other image compression format such as ZIP. In the above embodiment, an example of an 8-bit output JPEG or TIFF image converted into a non-linear gradation by gamma correction has been described as an example, but non-linear gradation data such as 8-bit BMP, PICT, or PNG, The present invention is applicable to any other non-linear gradation data such as non-linear gradation data in 16-bit TIFF as long as the gamma characteristic can be specified.

  In the above embodiment, an example of the RGB color system of the Bayer array has been described, but it goes without saying that it does not depend on the color filter arrangement method at all as long as interpolation processing is finally performed. The same applies to other color systems (for example, complementary color systems).

  In the above embodiment, an example of an interchangeable lens type single-lens reflex electronic still camera has been described. However, the present invention is not necessarily limited to this content. The present invention can also be applied to a non-interchangeable lens type camera. What is necessary is just to acquire a pupil position and an aperture value by a well-known method suitably.

  In the above-described embodiment, an example in which image data captured by the electronic still camera 1 is processed has been described. However, the present invention is not necessarily limited to this content. The present invention can also be applied to image data taken by a video camera that handles moving images. It can also be applied to image data taken with a camera-equipped mobile phone. Furthermore, it can be applied to a copying machine, a scanner, and the like. That is, the present invention can be applied to any image data captured using an image sensor.

  In the above-described embodiment, an example in which image data captured by the electronic camera 1 is processed by the PC (personal computer) 31 to remove the influence of dust has been described. However, the present invention is not necessarily limited to this content. Such a program may be provided on the electronic camera 1. Further, such a program may be provided in a printer, a projection device, or the like. That is, the present invention can be applied to any apparatus that handles image data.

  The program executed on the PC 31 can be provided through a recording medium such as a CD-ROM or a data signal such as the Internet. FIG. 17 is a diagram showing this state. The PC 31 receives a program via the CD-ROM 34. Further, the PC 31 has a connection function with the communication line 401. A computer 402 is a server computer that provides the program, and stores the program in a recording medium such as a hard disk 403. The communication line 401 is a communication line such as the Internet or personal computer communication, or a dedicated communication line. The computer 402 reads the program using the hard disk 403 and transmits the program to the PC 31 via the communication line 401. That is, the program is transmitted as a data signal on a carrier wave via the communication line 401. Thus, the program can be supplied as a computer-readable computer program product in various forms such as a recording medium and a carrier wave.

It is a figure which shows the structure of the electronic camera of an interchangeable lens system. 1 is a block diagram of an electronic camera, a personal computer (PC), and a peripheral device. It is a figure explaining the imaging | photography procedure on the electronic camera side in 1st Embodiment. It is a figure which shows the flowchart of the process performed by PC in 1st Embodiment. Gamma curve when γ = 0.3 Gamma curve when γ = 1 Gamma curve when γ = 0.5 It is a figure which shows an edge extraction filter. It is a figure which shows a mode that the local normalization process was performed with respect to the luminance surface. It is a figure which shows the histogram of a transmittance | permeability map. It is a figure explaining the imaging | photography procedure on the electronic camera side in 1st Embodiment. It is a figure which shows the flowchart of the process performed by PC in 2nd Embodiment. It is a figure which shows a mode that the position of a dust shadow changes when a pupil position changes. It is a figure which shows a mode that the magnitude | size of a dust shadow changes when F value which is an aperture value changes. It is a figure which shows the one-dimensional filter coefficient with respect to each aperture value. It is the figure which represented the filter converted into the transmittance | permeability map in aperture value F16 with the two-dimensional filter. It is a figure which shows the flowchart of the process performed by PC in 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Electronic camera 2 Camera main body 3 Variable optical system 4 Lens 5 Aperture 6 Shutter 7 Optical component 8 Imaging element 9 Mount part 12 Analog signal processing part 13 A / D conversion part 14 Timing control part 15 Image processing part 16 Operation part 17 Control part DESCRIPTION OF SYMBOLS 18 Memory 19 Compression / decompression part 20 Display image generation part 21, 32 Monitor 22 Memory card interface part 23 External interface part 24 Bus 30 Memory card 31 PC (personal computer)
33 Printer 34 CD-ROM

Claims (5)

Image input means for inputting an image captured by an image sensor through an optical system and subjected to nonlinear gradation processing different from the square root characteristic;
In the input image, based on the value of the target pixel and the average value of the values of a plurality of pixels within a predetermined range including the target pixel, a defect related to the projected image of the defect in the middle of the optical path generated in the image A defect information creating means for creating information,
The defect information creating means specifies a gamma processing method applied to the input image using tag information in an image file, and uses an inverse gamma curve generated based on the gamma processing method to identify the input image. After the gradation is once returned to the linear gradation, it is further converted into a non-linear gradation having a square root characteristic, and an edge portion in the image is detected in the non-linear gradation space having the square root characteristic, and based on the detected edge portion The defect information is created by calculating a mean value for a region in which flatness is guaranteed in the image based on the extracted information on the flat portion region. An image processing apparatus. The image processing apparatus according to claim 1.
The defect information creating means calculates a relative ratio between the value of the pixel of interest and the average value of a plurality of pixels within a predetermined range including the pixel of interest in the gradation space returned to the linear gradation. An image processing apparatus that creates the defect information. The image processing apparatus according to claim 2,
It further comprises defect correction means for correcting the defect by multiplying the inverse value of the relative ratio created by the defect information creation means by the value of the corresponding pixel in the gradation space returned to the linear gradation. An image processing apparatus. In the image processing device according to any one of claims 1 to 3,
An image processing apparatus, wherein the image input means inputs a compressed image that has been compressed into JPEG, JPEG2000, or TIFF after decompressing the compressed image. An image processing program for causing a computer to execute the function of the image processing apparatus according to claim 1.

Images