JP2015179951A - Image processing apparatus, imaging apparatus, image processing method, program and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of high-performance noise removal processing.SOLUTION: The image processing apparatus includes: judgment means for judging whether the absolute value of a luminance difference between a first pixel and a second pixel is a first threshold or smaller, and whether the absolute value of a chroma difference between the first pixel and the second pixel is a second threshold or smaller; processing means for performing smoothing processing on the color difference signal between the first pixel and the second pixel, when the absolute value of the luminance difference is the first threshold or smaller and the absolute value of the chroma difference is the second threshold or smaller; and set means for variably setting the first threshold according to the change of a luminance value between the first pixel and a neighboring pixel of the first pixel.

Description

  The present invention relates to an image processing apparatus that removes noise components.

  In an imaging apparatus such as a digital camera, it is required to remove noise components with high accuracy due to demands for higher sensitivity and higher pixels. Patent Document 1 acquires a luminance difference signal that represents a difference between luminance signals of a pixel of interest and other pixels, and a color difference signal that represents a difference between saturation signals of the pixel of interest and other pixels, and these difference signals. An imaging apparatus that determines a removal amount of a noise component by comparing a predetermined threshold value with a predetermined threshold value is disclosed.

JP 2006-109416 A

  In the imaging apparatus disclosed in Patent Document 1, in order to remove the coloring generated at the high luminance edge, it is necessary to increase the threshold value for the luminance difference signal to obtain the effect of noise processing. However, at this time, the resolution of the color edge tends to disappear at the edge portion of the colored subject.

  Therefore, the present invention provides an image processing device, an imaging device, an image processing method, a program, and a storage medium that can perform high-performance noise removal processing.

  An image processing apparatus according to one aspect of the present invention has an absolute value of a luminance difference between a first pixel and a second pixel that is equal to or less than a first threshold value, and saturation between the first pixel and the second pixel. Determining means for determining whether or not the absolute value of the difference is equal to or smaller than a second threshold; and the absolute value of the luminance difference is equal to or smaller than the first threshold and the absolute value of the saturation difference is equal to the second threshold. When the threshold value is less than or equal to the threshold value, a change in luminance value between the processing means for performing a smoothing process on the color difference signals of the first pixel and the second pixel, and the first pixel and a neighboring pixel of the first pixel And setting means for setting the first threshold value to be changeable according to

  An imaging device as another aspect of the present invention includes an imaging unit and the image processing device configured to perform image processing based on an output signal from the imaging unit.

  According to another aspect of the present invention, there is provided an image processing method, comprising: setting a first threshold value according to a change in luminance value between a first pixel and a neighboring pixel of the first pixel; It is determined whether the absolute value of the luminance difference between two pixels is equal to or less than the first threshold value, and the absolute value of the saturation difference between the first pixel and the second pixel is equal to or less than the second threshold value. And a color difference between the first pixel and the second pixel when the absolute value of the luminance difference is equal to or smaller than the first threshold value and the absolute value of the saturation difference is equal to or smaller than the second threshold value. Performing a smoothing process on the signal.

  A program according to another aspect of the present invention is configured to cause a computer to execute the image processing method.

  A storage medium according to another aspect of the present invention stores the program.

  Other objects and features of the present invention are illustrated in the following examples.

  According to the present invention, it is possible to provide an image processing device, an imaging device, an image processing method, a program, and a storage medium capable of performing high-performance noise removal processing.

It is a block diagram of the imaging device in a 1st embodiment. It is a flowchart which shows the setting operation | movement of the brightness | luminance smoothing process threshold value in 1st Embodiment. 6 is a flowchart illustrating an operation of a color NR unit in the first embodiment. It is explanatory drawing of the smoothing process in 1st Embodiment. It is explanatory drawing of the brightness | luminance smoothing process threshold value in 1st Embodiment. It is explanatory drawing of the brightness | luminance smoothing process threshold value in 1st Embodiment. In 1st Embodiment, it is a related figure of the inclination of a brightness | luminance and a brightness | luminance smoothing process threshold value. It is a block diagram of the imaging device in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
First, the configuration of the imaging apparatus according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram of an imaging apparatus 100 according to this embodiment. The imaging apparatus 100 in FIG. 1 includes an imaging unit 1000 and other elements that constitute the image processing apparatus. The image processing apparatus is configured to perform image processing based on an output signal from the imaging unit 1000.

  The imaging unit 1000 includes a photographic lens (imaging optical system) and an imaging element, and photoelectrically converts an optical image (subject image) to output an image signal (analog signal). The analog signal output from the imaging unit 1000 is input to the A / D conversion unit 1001. The A / D conversion unit 1001 converts an analog signal into a digital signal. An image signal (digital image signal) converted into a digital signal by the A / D conversion unit 1001 is input to the WB unit 1002 (white balance unit). The WB unit 1002 performs known white balance adjustment on the digital image signal.

  The image signal (luminance signal) output from the WB unit 1002 is input to the luminance NR unit 1003 (luminance noise reduction unit). The luminance NR unit 1003 performs luminance NR processing (luminance noise reduction processing) on the image signal output from the WB unit 1002. As the luminance NR process, a known method of detecting the direction of the edge component included in the image and reducing noise by a smoothing process using a low-pass filter along the direction of the detected edge component can be used. . However, the present embodiment is not limited to this, and other methods may be used. The luminance signal corrected by the luminance NR unit 1003 (the luminance NR process is performed) is input to the luminance signal processing unit 1004. The luminance signal processing unit 1004 performs luminance signal processing such as known gamma correction on the luminance signal.

  The image signal output from the WB unit 1002 is input to the color interpolation unit 1005. The color interpolation unit 1005 performs a known color interpolation process on the image signal, and converts the image signal into a color difference signal V (RY), U (BY), or the like. The color difference signal output from the color interpolation unit 1005 is input to the color NR unit 1006 (color noise reduction unit). The color NR unit 1006 performs color NR processing (correction processing) on the color difference signal. Details of this will be described later. The color NR unit 1006 includes a V direction NR processing unit 1007 (vertical direction NR processing unit) and an H direction NR processing unit 1008 (horizontal direction NR processing unit). The V direction NR processing unit 1007 performs color NR processing in the vertical direction, and then the H direction NR processing unit 1008 performs color NR processing in the horizontal direction. In this embodiment, the V-direction NR processing unit 1007 includes a determination unit 1071, a processing unit 1072, and a setting unit 1073. Preferably, the H-direction NR processing unit 1008 is also provided with similar means. The operation of each of these means will be described later.

  A color signal obtained by being processed (corrected) by the color NR unit 1006 is input to the color signal processing unit 1009. The color signal processing unit 1009 performs color correction processing such as known color conversion matrix processing and color suppression processing on the color signal (color difference signal). In the present embodiment, the color NR unit 1006 removes a signal having a small amplitude as noise as an edge while preserving the edge signal included in the color difference signal. Next, an example of the operation of the color NR unit 1006 will be described with reference to FIG.

  FIG. 3 is a flowchart showing the operation of the color NR unit 1006. Here, the operation of the V direction NR processing unit 1007 will be described assuming that the luminance component signal (luminance signal) is Y and the color component signal (color signal, color difference signal) is UV. First, in step S101, the V-direction NR processing unit 1007 sets a color smoothing processing threshold Th_c that is a threshold for a color component signal in the pixel of interest. In step S102, the V-direction NR processing unit 1007 (setting unit 1073) sets a luminance smoothing processing threshold Th_y that is a threshold for a luminance component signal in the pixel of interest.

  Subsequently, in step S103, the V-direction NR processing unit 1007 generates a saturation signal C from the color component signal (color difference signal) for each pixel based on the following equation (1).

  Next, in step S104, the V-direction NR processing unit 1007 calculates a difference absolute value DIFF_C (ABS (Cc−Ci)) between the saturation signal Cc of the pixel of interest and the saturation signal Ci of the surrounding pixels. Subsequently, in step S105, the V-direction NR processing unit 1007 calculates a difference absolute value DIFF_Y (ABS (Yc−Yi)) between the luminance signal Yc of the target pixel and the luminance signal Yi of the peripheral pixels. In order to simplify the processing, the description will be made assuming that the value of the G pixel is used instead of the luminance signal Y (that is, Y = G).

  Next, in step S106, the V-direction NR processing unit 1007 (determination unit 1071) determines that the difference absolute value DIFF_C is equal to or smaller than the color smoothing threshold value Th_c and the difference absolute value DIFF_Y is equal to or smaller than the luminance smoothing threshold value Th_y. It is determined whether or not. When DIFF_C ≦ Th_c and DIFF_Y ≦ Th_y are satisfied, the process proceeds to step S107, and a flag for setting the surrounding pixels as the smoothing target pixels is set. On the other hand, if the determination condition (DIFF_C ≦ Th_c and DIFF_Y ≦ Th_y) in step S106 is not satisfied, the process proceeds to step S108.

  In step S <b> 108, the V direction NR processing unit 1007 determines whether there is a peripheral pixel to be calculated. If there are peripheral pixels to be calculated, the process returns to step S103, and steps S103 to S108 are repeated. On the other hand, if there is no peripheral pixel to be calculated in step S108, the process proceeds to step S109.

  In step S109, the V-direction NR processing unit 1007 (processing unit 1072) performs a smoothing process on pixels (peripheral pixels) for which a flag to be smoothed is set. FIG. 4 is an explanatory diagram of the smoothing process. As illustrated in FIG. 4, the V-direction NR processing unit 1007 performs a smoothing process on five pixels centered on the target pixel. The V-direction NR processing unit 1007, for example, when the pixel of interest and the surrounding pixels of i = 0 and 3 are subjected to the smoothing process, the color difference signals U_out and V_out after the smoothing process are expressed by the following equation (2), Each is expressed as (3).

U_out = (U0 + Uc + U3) / 3 (2)
V_out = (V0 + Vc + V3) / 3 (3)
Here, with reference to FIG. 5 and FIG. 6, the setting of the luminance smoothing process threshold Th_y in step S102 will be described in detail. FIG. 5 shows an area where the coloring generated in the black and white edge area should be removed, and FIG. 6 shows a green edge where the edge color should remain.

  FIG. 5A and FIG. 6A are relationship diagrams between pixel positions and pixel values, where the horizontal axis indicates the pixel position and the vertical axis indicates the pixel value. In FIG. 5A, the value of the R pixel at the pixel position x (x = 0, 1, c, 2, 3) is R [x], and the value of the G pixel is G [x]. At this time, G [0] = G [1] = R [0] = R [1] = S1, G [c] = S2, R [c] = S3, R [2] = G [2] = R [3] = G [3] = S4 The value of the color difference signal at the pixel position x is R−Y [x]. At this time, RY [c] = S3-S2 at the pixel position x = c, and RY [x] = 0 at each of the other pixel positions x = 0, 1, 2, and 3, and at the pixel position c. Coloring occurs. In this way, coloring is likely to occur at the edge of the achromatic luminance.

  FIGS. 5B and 6B are relationship diagrams between the pixel position and the difference absolute value DIFF_Y (the luminance Yc of the pixel of interest c and the luminance Yi of the peripheral pixel i), and the horizontal axis represents the pixel position. The vertical axis indicates the absolute difference value DIFF_Y (| G [c] −G [x] |). In FIG. 5B, the absolute difference value DIFF_Y increases at peripheral pixels i = 0, 1, 2, and 3, and the peripheral pixels at these positions are not subject to smoothing processing. Therefore, smoothing processing is not performed on these peripheral pixels, and the coloring of the edge at the pixel position c remains as it is. When the threshold value of the absolute difference value DIFF_Y is increased so as to be the target pixel of the smoothing process, the pixel position 0 becomes the target pixel of the smoothing process near the green edge shown in FIG. The edge is blurred.

  Therefore, in this embodiment, it is determined whether or not the luminance Yc at the target position c is a sharp edge, that is, the luminance gradient (change in luminance) of the target pixel (between the target pixel and neighboring pixels). Specifically, for example, the calculation is performed using the luminance gradient GradY = G [2] −G [1] obtained from the values of the pixels located on both sides of the target pixel. In the luminance edge in FIG. 5A, GradY = S4-S1, and in the vicinity of the green edge in FIG. 6A, GradY = S3-S2. For this reason, the threshold value of the absolute difference value DIFF_Y that is the target of the smoothing process is set to be large only when it is determined that the luminance gradient GradY of the target pixel is large. Thereby, it is possible to reduce the coloration of the edge while reducing the blur of the color edge.

  Next, a method for setting the luminance smoothing process threshold Th_y will be described with reference to FIG. FIG. 2 is a flowchart showing the setting operation (step S102) of the luminance smoothing process threshold Th_y. First, in step S201, the color NR unit 1006 (V direction NR processing unit 1007) calculates the luminance gradient GradY of the pixel of interest. Subsequently, in step S202, the V-direction NR processing unit 1007 compares the luminance gradient GradY of the target pixel with the threshold value MIN. If the luminance gradient GradY of the pixel of interest is smaller than the threshold MIN, the process proceeds to step S206. In step S206, the V direction NR processing unit 1007 sets the smoothing processing threshold Th_y to Th_y1.

  On the other hand, if it is determined in step S202 that the luminance gradient GradY of the target pixel is greater than or equal to the threshold value MIN, the process proceeds to step S203. In step S203, the V direction NR processing unit 1007 compares the luminance gradient GradY of the pixel of interest with a threshold MAX (threshold MAX> threshold MIN). When the luminance gradient GradY of the target pixel is larger than the threshold value MAX, the V-direction NR processing unit 1007 sets the luminance smoothing processing threshold value Th_y to Th_y2. On the other hand, if the luminance gradient GradY of the target pixel is equal to or smaller than the threshold value MAX in step S203, the process proceeds to step S204. In step S204, the V-direction NR processing unit 1007 sets the brightness smoothing processing threshold Th_y to a value between Th_y1 and Th_y2 using linear interpolation.

  FIG. 7 is a relationship diagram between the luminance gradient GradY of the pixel of interest and the luminance smoothing threshold value Th_y. The horizontal axis represents the luminance gradient GradY of the pixel of interest, and the vertical axis represents the luminance smoothing threshold value Th_y. As illustrated in FIG. 7, when the luminance gradient GradY of the pixel of interest is smaller than the threshold MIN, the luminance smoothing processing threshold Th_y is set to Th_y1. Further, when the luminance gradient GradY of the target pixel is larger than the threshold value MAX, the luminance smoothing processing threshold value Th_y is set to Th_y2. When the luminance gradient GradY of the pixel of interest is a value between the threshold value MIN and the threshold value MAX, it is set as represented by the following equation (4).

  As described above, in this embodiment, the V-direction NR processing unit 1007 detects a luminance edge near the target pixel. When the luminance edge is large, the V direction NR processing unit 1007 sets the luminance smoothing processing threshold Th_y to be large. When the luminance smoothing process threshold Th_y is large, the number of pixels that are the targets of the smoothing process increases. For this reason, the effect of the smoothing process can be increased only for the edge pixels. With such a configuration, it is possible to reduce the influence of coloring of the pixels on the luminance edge while reducing the deterioration of the resolution of the color around the luminance edge. In the present embodiment, the operation of the V-direction NR processing unit 1007 has been described. However, the present invention is not limited to this, and the operation of the H-direction NR processing unit 1008 is V V except that the target pixel is different. This is equivalent to the direction NR processing unit 1007.

  In the present embodiment, the G signal (green signal) is used as the luminance signal Y. However, the present invention is not limited to this. For example, the luminance signal Y may be generated using a signal such as a known luminance composition ratio Y = 0.3R + 0.6G + 0.1B.

  In the present embodiment, the luminance smoothing processing threshold Th_y is changed according to the luminance gradient GradY of the adjacent pixel adjacent to the pixel of interest, but is not limited to this. For example, the luminance gradient GradY may be calculated using a pixel near the target pixel instead of the adjacent pixel. Further, the threshold value Th_c of the color smoothing process may be changed according to the luminance gradient GradY of the adjacent pixels.

  Further, in the present embodiment, the luminance gradient GradY is calculated based on GradY = G [2] −G [1], but is not limited to this. For example, two gradients G [2] -G [c] and G [c] -G [1] are compared, and the larger or smaller of these gradients is selected as the luminance gradient GradY. Good.

  As described above, in this embodiment, the determination unit 1071 determines that the absolute value of the luminance difference between the first pixel and the second pixel is equal to or less than the first threshold value and the saturation difference between the first pixel and the second pixel. It is determined whether the absolute value of is less than or equal to the second threshold value. The processing unit 1072 smoothes the color difference signals of the first pixel and the second pixel when the absolute value of the luminance difference is equal to or smaller than the first threshold value and the absolute value of the saturation difference is equal to or smaller than the second threshold value. Process. The setting unit 1073 sets the first threshold value so as to be changeable according to a change in luminance value (luminance gradient) between the first pixel and a neighboring pixel of the first pixel. Preferably, the first pixel is the target pixel c, and the second pixel is a plurality of peripheral pixels i (for example, i = 0 to 3) arranged around the target pixel c, but is not limited thereto. ) At least one pixel.

  Preferably, the setting unit 1073 sets the first threshold (brightness smoothing processing threshold Th_y) to be larger as the change in the luminance value between the first pixel and the neighboring pixel is larger. Preferably, the setting unit 1073 sets the second threshold value (color smoothing process threshold Th_c) to be larger as the change in the luminance value between the first pixel and the neighboring pixel is larger. More preferably, the neighboring pixel is a pixel adjacent to the first pixel.

  Preferably, the setting unit 1073 sets the first threshold (brightness smoothing) when the amount of change in luminance value between the first pixel and the neighboring pixel (brightness gradient GradY) is smaller than the first amount of change (threshold MIN). A first value (Th_y1) is set as the processing threshold value Th_y) (S206). In addition, when the change amount of the luminance value between the first pixel and the neighboring pixel is larger than the second change amount (threshold value MAX), the setting unit 1073 sets the second value that is larger than the first value as the first threshold value. (Th_y2) is set (S205). More preferably, when the change amount of the luminance value between the first pixel and the neighboring pixel is the third change amount between the first change amount and the second change amount, the setting unit 1073 sets the first threshold value as the first threshold value. A third value (Th1_y1 <Th_y <Th_y2) between the first value and the second value is set (S204).

(Second Embodiment)
Next, a second embodiment of the present invention will be described with reference to FIG. FIG. 8 is a block diagram of the imaging apparatus 200 in the present embodiment. The imaging apparatus 200 of the present embodiment is different from the imaging apparatus 100 of the first embodiment in that NR processing is performed using a normal image (first image data) and a reduced image (second image data). Hereinafter, description of points common to the first embodiment will be omitted, and different points will be mainly described.

  In the present embodiment, the imaging unit 2000, the A / D conversion unit 2001, and the WB unit 2002 are the imaging unit 1000, the A / D conversion unit 1001, and the first embodiment described with reference to FIG. Each is the same as the WB unit 1002. For this reason, those descriptions are omitted. Data (image signal) from the WB unit 2002 is output to the luminance NR unit 2003 and the color interpolation unit 2004. Note that the luminance NR unit 2003, the color interpolation unit 2004, and the color NR unit 2005 are the same as the luminance NR unit 1003, the color interpolation unit 1005, and the color NR unit 1006, respectively. Therefore, detailed description thereof is omitted. Data from the WB unit 2002 is input to the edge detection unit 2006. The edge detection unit 2006 includes, for example, a known edge detection filter, applies the edge detection filter to the input signal, and outputs the output as an edge strength output signal.

  The data output from the WB unit 2002 is input to the LPF 2101 (low-pass filter) and the DS unit 2102 (down-sampling unit). The LPF 2101 performs LPF processing, and the DS unit 2102 performs downsampling processing. Thereby, synchronized low-resolution image data is generated. The synchronized data output from the DS unit 2102 is input to the luminance NR unit 2103. The luminance NR unit 2103 performs luminance NR processing. Since the operation of the luminance NR unit 2103 is the same as that of the luminance NR unit 2003, description thereof is omitted.

  The luminance signal corrected (processed) by the luminance NR unit 2103 is input to the US unit 2104 (upsampling unit). The US unit 2104 performs an upsampling process on the luminance signal, and outputs the processed luminance signal to the combining unit 2007 (combining unit). The data output from the DS unit 2102 is input to the color interpolation unit 2105. The color interpolation unit 2105 performs color interpolation processing on the data from the DS unit 2102. The color signal (color difference signal) subjected to color interpolation processing by the color interpolation unit 2105 is output to a color NR unit 2106 described later. The color NR unit 2106 corrects the color signal, and the corrected color signal is output to the US unit 2107. The US unit 2107 performs upsampling processing on the color signal input from the color NR unit 2106 and outputs the processed color signal to the combining unit 2007.

  The combining unit 2007 determines a combining ratio of the luminance signal and the color signal (color difference signal) in accordance with the edge intensity output signal from the edge detection unit 2006. The combining unit 2007 also outputs the luminance signal (high resolution luminance signal) output from the luminance NR unit 2003 and the luminance signal (low resolution luminance signal) output from the US unit 2104 according to the determined luminance signal combining ratio. And synthesize. Similarly, the synthesis unit 2007 synthesizes the color signal (high resolution color signal) from the color NR unit 2005 and the color signal (low resolution color signal) from the US unit 2107 according to the synthesis ratio of the color signals. . The high resolution luminance signal and the high resolution color signal are signals obtained based on the first image data. The low resolution luminance signal and the low resolution color signal are signals obtained based on the second image data having a lower resolution than the first image data. The combining unit 2007 outputs the combined luminance signal to the luminance signal processing unit 2008. The synthesizing unit 2007 outputs the synthesized color signal to the color signal processing unit 2009. Since the operations of the luminance signal processing unit 2008 and the color signal processing unit 2009 are the same as those in the first embodiment, their detailed description is omitted.

  Next, the operation of the color NR unit 2106 in this embodiment will be described. The color NR unit 2106 is the same as the color NR unit 1006 of the first embodiment described with reference to FIG. 1 except that the parameter setting method is different. Therefore, only the parameter setting method in the color NR unit 2106 will be described here.

  When the coloring at the high luminance edge remains in the color NR portion 2106, the remaining coloring in the US portion 2107 is enlarged. For this reason, the threshold value set by the color NR unit 2106 is set to be larger than the threshold value set by the color NR unit 2005. The color NR unit 2106 is set so that the smoothing process is performed more strongly, and is controlled so that coloring at a higher luminance edge does not remain. Specifically, the first threshold value (Th_y) set when the luminance gradient GradY shown in FIG. 5 is smaller than the first change amount (threshold value MIN) is larger than the first value (Th_y1). A value of 3 (Th_y3) is set. Further, a fourth value (Th_y4) larger than the second value (Th_y2) is set as the first threshold set when the luminance gradient GradY is larger than the second change amount (threshold MAX). Alternatively, the threshold value MAX and the threshold value MIN used when determining the threshold value Th_y of the smoothing process in accordance with the luminance gradient GradY shown in FIG. Is also controlled to be a small value. As described above, in the present embodiment, when the smoothing process is performed from the reduced image, the control is performed such that the smoothing process is more easily performed than the image before the reduction.

  As described above, in this embodiment, the image processing apparatus includes a generation unit (WB unit 2002, LPF 2101, and DS unit 2102) and a synthesis unit (synthesis unit 2007). The generating unit generates the first image data and the second image data having a lower resolution than the first image data. The synthesizing unit synthesizes the first image data and the second image data. The setting means (color NR units 2005 and 2106) increases the first threshold value for each of the first image data and the second image data as the change in the luminance value between the first pixel and the neighboring pixel increases. Set.

  Preferably, when the change amount of the luminance value between the first pixel and the neighboring pixels is smaller than the first change amount (threshold value MIN) with respect to the first image data, the setting unit sets the first threshold value (Th_y) as the first threshold value (Th_y). A value of 1 (Th_y1) is set. Further, the setting means sets the third value (Th_y3) as the first threshold value when the change amount of the luminance value between the first pixel and the neighboring pixel is smaller than the third change amount with respect to the second image data. . Here, it is preferable to satisfy at least one of the third change amount (threshold value) being smaller than the first change amount and the third value being larger than the first value.

  Further preferably, when the change amount of the luminance value between the first pixel and the neighboring pixels is larger than the second change amount (threshold value MAX) with respect to the first image data, the setting unit preferably sets the first threshold value as the first threshold value. A second value (Th_y2) larger than the value is set. The setting unit sets the fourth value (Th_y4) as the first threshold value when the change amount of the luminance value between the first pixel and the neighboring pixel is larger than the fourth change amount with respect to the second image data. . Here, it is preferable to satisfy at least one of the fourth change amount (threshold value) being smaller than the second change amount and the fourth value being larger than the second value.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. To be executed. In this case, a computer-executable program describing the procedure of the imaging apparatus control method and a storage medium storing the program constitute the present invention.

  According to each embodiment, it is possible to perform noise removal processing in which coloring that occurs in a high-luminance edge portion is reduced while maintaining color resolution. Therefore, according to each embodiment, it is possible to provide an image processing device, an imaging device, an image processing method, a program, and a storage medium that can execute high-performance noise removal processing.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

1071 Determination means 1072 Processing means 1073 Setting means

Claims (14)

Whether the absolute value of the luminance difference between the first pixel and the second pixel is less than or equal to the first threshold value, and the absolute value of the saturation difference between the first pixel and the second pixel is less than or equal to the second threshold value Determining means for determining whether or not;
When the absolute value of the luminance difference is equal to or smaller than the first threshold value and the absolute value of the saturation difference is equal to or smaller than the second threshold value, the color difference signals of the first pixel and the second pixel are Processing means for performing a smoothing process;
An image processing apparatus comprising: a setting unit configured to change the first threshold value according to a change in luminance value between the first pixel and a neighboring pixel of the first pixel.   The image processing apparatus according to claim 1, wherein the setting unit sets the first threshold value to be larger as a change in the luminance value between the first pixel and the neighboring pixel is larger. . The setting means includes
When the change amount of the luminance value between the first pixel and the neighboring pixel is smaller than the first change amount, a first value is set as the first threshold value,
When the change amount of the luminance value between the first pixel and the neighboring pixel is larger than the second change amount, a second value larger than the first value is set as the first threshold value; The image processing apparatus according to claim 1, wherein:   When the change amount of the luminance value between the first pixel and the neighboring pixel is a third change amount between the first change amount and the second change amount, the setting unit may change the first change amount. The image processing apparatus according to claim 3, wherein a third value between the first value and the second value is set as a threshold value. The first pixel is a pixel of interest;
5. The image processing apparatus according to claim 1, wherein the second pixel is at least one of a plurality of peripheral pixels of the pixel of interest.   The image processing apparatus according to claim 1, wherein the neighboring pixel is a pixel adjacent to the first pixel.   7. The setting unit according to claim 1, wherein the setting unit sets the second threshold value to be larger as a change in the luminance value between the first pixel and the neighboring pixel is larger. The image processing apparatus according to item. Generating means for generating first image data and second image data having a lower resolution than the first image data;
A synthesis unit that synthesizes the first image data and the second image data;
The setting means sets the first threshold value to be larger as the change in the luminance value between the first pixel and the neighboring pixel is larger for each of the first image data and the second image data. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The setting means includes
For the first image data, when the amount of change in the luminance value between the first pixel and the neighboring pixel is smaller than the first amount of change, a first value is set as the first threshold value,
Regarding the second image data, when the change amount of the luminance value between the first pixel and the neighboring pixels is smaller than the third change amount, the third value is set as the first threshold value.
9. The device according to claim 8, wherein the third change amount satisfies at least one of a smaller value than the first change amount and a third value greater than the first value. Image processing device. The setting means includes
For the first image data, when the change amount of the luminance value between the first pixel and the neighboring pixel is larger than the second change amount, the second threshold value is larger than the first value as the first threshold value. Set the value of
For the second image data, when the change amount of the luminance value between the first pixel and the neighboring pixel is larger than a fourth change amount, a fourth value that is larger than the third value as the first threshold value. Set the value of
The fourth change amount satisfies at least one of the second change amount being smaller than the second change amount and the fourth value being larger than the second value. The image processing apparatus described. An imaging unit;
The image processing apparatus according to claim 1, wherein the image processing apparatus is configured to perform image processing based on an output signal from the image capturing unit. Setting a first threshold in response to a change in luminance value between the first pixel and a neighboring pixel of the first pixel;
The absolute value of the luminance difference between the first pixel and the second pixel is less than or equal to the first threshold value, and the absolute value of the saturation difference between the first pixel and the second pixel is less than or equal to the second threshold value. Determining whether there is,
When the absolute value of the luminance difference is equal to or smaller than the first threshold value and the absolute value of the saturation difference is equal to or smaller than the second threshold value, the color difference signals of the first pixel and the second pixel are And a step of performing a smoothing process.   A program configured to cause a computer to execute the image processing method according to claim 12.   A storage medium storing the program according to claim 13.