JP4471748B2 - Image signal processing device - Google Patents

Description

  The present invention relates to an image signal processing apparatus that performs processing for suppressing noise such as dark current included in an image signal generated by an image sensor.

  In an imaging apparatus such as a digital camera, an image signal with a suitable signal level can be obtained by adjusting the exposure time and the gain of the amplifier circuit according to the brightness of the subject. However, when the exposure time is lengthened, dark current noise included in the image signal increases, and when the amplification gain is increased, even the noise component is amplified.

  Therefore, conventionally, noise is removed from an image signal using a filter circuit.

  FIG. 11 is a block diagram illustrating a configuration of a conventional image signal processing apparatus that processes an image signal generated by an image sensor. An image signal output from the image pickup device 2 such as a CCD (Charge Coupled Device) image sensor is processed by the analog signal processing circuit 4 and then converted into digital data by the A / D conversion circuit 6. The signal is input to the signal processing circuit 8. In the digital signal processing circuit 8, two-dimensional filter processing is performed by a line memory 10 that holds image data for a plurality of lines and a pixel correction filter circuit 12. FIG. 11 shows a configuration in which the line memory 10 is provided for three lines.

  The output of the pixel correction filter circuit 12 is input to a low-pass filter (LPF) 14. When the image pickup device 2 includes a color filter having a general Bayer arrangement, the LPF 14 traps a frequency component that is ½ of the sampling frequency in each of the vertical and horizontal directions in order to remove the output level difference for each color filter. . The output of the LPF 14 is subjected to gradation correction processing by the gamma correction circuit 16.

  The output of the pixel correction filter circuit 12 is also input to the contour correction signal generation circuit 18. The contour correction signal generation circuit 18 obtains a second derivative waveform of the image signal that greatly fluctuates at the edge portion in the image, and generates a contour correction signal by multiplying it by an appropriate gain. The generated contour correction signal is combined with the image signal by the adder 20, thereby realizing a contour enhancement process which is one of image quality adjustments.

  Conventionally, the pixel correction filter circuit 12 constitutes a median filter or a mean filter and is used for noise removal.

  The pixel correction filter circuit 12 acquires pixel values of a plurality of reference pixels including a target pixel and pixels at predetermined positions around the target pixel from the image data held in the line memory 10.

  The mean filter calculates an average value of pixel values of a plurality of reference pixels and replaces the average value with the pixel value of the target pixel. The mean filter has an effect of smoothing pixel values within the filter size, and thus is expected to be effective in removing noise that occurs sparsely within the filter size. Here, the dark current is likely to occur continuously due to pixel defects or the like, and the pixel range affected by the dark current is expanded as the exposure time is increased. Further, the dark current does not have both positive and negative polarities, and acts only in one direction, for example, increasing the pixel value. Because of these properties, the mean filter has a problem that it is difficult to suitably remove dark current noise. For example, if only one pixel among the five reference pixels is affected by the dark current component, the dark current noise is reduced to some extent by the mean filter. As the proportion of pixels affected by increases, the noise reduction effect decreases.

  On the other hand, the median filter calculates a median value among the pixel values of a plurality of reference pixels and replaces the median value with the pixel value of the target pixel. The median filter is easier to remove dark current noise than the mean filter, but when the proportion of pixels that are affected by dark current among the reference pixels becomes higher, the pixel selected as the median is affected by dark current. Is likely to have been received. Further, if the number of reference pixels constituting the filter is increased, it becomes difficult to be influenced by the regional spread of dark current, but there is a problem that the scale of a circuit for detecting the median becomes large.

  The present invention has been made to solve the above problems, and provides an image signal processing apparatus that processes an image signal generated by an image sensor, and that suppresses noise in which a region where a dark current or the like is easily generated is suppressed. For the purpose.

  The image signal processing apparatus according to the present invention associates a target point with the target point based on n reference pixels (n is an integer of 3 or more) having a predetermined positional relationship with respect to the target point arranged in the image. A filter circuit for generating a converted pixel value, wherein the upper α pixel and the lower β pixel (where α and β are α + β ≧ 1 and n−α−β ≧ 2) among the pixel values of the n reference pixels. And a pixel correction filter circuit that calculates an average value of the pixel values of (n−α−β) remaining reference pixels excluding (which is an integer of 0 or more to satisfy) as the converted pixel value.

  According to the present invention, when α ≧ 1 and β ≧ 1, the upper α pixel and the lower β pixel of the pixel value are removed, whereas when β = 0, only the upper α pixel is set to α = 0. In some cases, only the lower β pixels are removed and the remaining reference pixels are defined. The pixel values affected by noise are expected to be located at the upper and lower positions. By excluding them, the percentage of pixels that are not affected by noise or have relatively little influence on the plurality of residual reference pixels Is expected to increase. Which of the upper and lower levels is excluded and the number of pixels is set according to the nature of the noise. Furthermore, by averaging the pixel values of a plurality of residual reference pixels, if some of the residual reference pixels are affected by noise, the influence can be reduced.

  Another image signal processing apparatus according to the present invention relates to the upper α pixel and the lower β pixel that are removed by the pixel correction filter circuit when the pixel value is increased by a dark current generated in the image sensor. , Α> β.

  The influence of dark current acts to shift the pixel value in a certain direction. The direction of the shift depends on the definition of the image signal and the pixel value, but it is usually determined that the pixel value increases as the signal charge accumulated in the pixel increases, while the dark current reduces the signal charge. In this case, the pixel value shifts to the upper side when affected by the dark current. Therefore, by placing weight on pixel removal above the lower side, the influence of dark current on the remaining reference pixels can be reduced.

  In another image signal processing device according to the present invention, the center of gravity of the coordinates of the representative points of each of the plurality of remaining reference pixels connects the representative points of each of the plurality of reference pixels located in the vicinity of the target point. The number of the remaining reference pixels is set so as to exist in the polygonal region formed in (1).

  The representative point is a predetermined coordinate in a pixel having a predetermined spread, and is associated with a sampling point of the image. The reference pixels may be arranged so as to surround the target point several times. In this case, the reference pixels around the target point mean a reference pixel group that surrounds the target pixel on the innermost side. When the number of residual reference pixels is increased, the average position of the center of gravity approaches the target point. This centroid corresponds to the sampling point of the converted pixel value. According to the present invention, the deviation between the center of gravity and the target point can be made, for example, equal to or less than the distance to the representative point of the reference pixel close to the target point, and jaggy noise or the like is suppressed to improve the image quality. A preferred aspect of the present invention is the image signal processing device in which the number of the remaining reference pixels is set so that the center of gravity is located in an inner portion excluding the circumference of the polygonal region.

  Another image signal processing apparatus according to the present invention includes the pixel correction filter circuit, and includes a first signal process including a pixel correction process by the pixel correction filter circuit, and a second signal process not including the pixel correction process. And a control unit that controls switching between the first signal processing and the second signal processing in the processing unit in accordance with exposure control in the imaging device when acquiring an image signal. Part.

  The amount of noise such as dark current varies according to exposure control by exposure time and amplification gain. According to the present invention, for example, the first signal processing is performed under an exposure control condition where a lot of noise is expected, and the noise is reduced by the pixel correction processing by the pixel correction filter circuit, but the noise is expected to be small. Under the exposure control conditions, the second signal processing is performed, thereby avoiding resolution reduction that is a side effect of the pixel correction processing.

  Still another image signal processing apparatus according to the present invention includes the pixel correction filter circuit, and includes a first signal process including a pixel correction process by the pixel correction filter circuit, and a second signal process not including the pixel correction process. And a processing unit that combines the output by the first signal processing and the output by the second signal processing at a variable combining ratio, and according to exposure control in the imaging device when acquiring an image signal And a control unit for controlling the synthesis ratio.

  According to the present invention, for example, based on the exposure control conditions, the output ratio of the first signal processing is increased continuously or stepwise when there is a lot of noise, and conversely the second signal processing when the noise is low. As the output ratio is increased, noise suppression and resolution are ensured.

  In another image signal processing apparatus according to the present invention, the processing unit generates and outputs a noise-suppressed image signal in which noise is suppressed, and the first signal processing is performed on the image signal after the pixel correction processing. The low-pass filter processing based on the first low-pass characteristic is performed to generate the noise-suppressed image signal, and the second signal processing is performed on the image signal not subjected to the pixel correction processing. The noise suppression image signal is generated by performing low-pass filter processing based on a second low-pass characteristic different from the pass characteristic. A preferred aspect of the present invention is an image signal processing device in which the first low-pass characteristic has a cut-off frequency lower than that of the second low-pass characteristic.

  In still another image signal processing device according to the present invention, the processing unit generates and outputs a contour correction signal corresponding to a contour of an image based on the image signal, and the first signal processing includes the first signal processing, The contour correction signal based on the first band pass characteristic is generated using the image signal after the pixel correction processing, and the second signal processing uses the image signal not subjected to the pixel correction processing to generate the first band The contour correction signal based on a second band pass characteristic different from the pass characteristic is generated. A preferred aspect of the present invention is an image signal processing device in which the first band pass characteristic has a lower pass band frequency than the second band pass characteristic.

  In the image signal processing method according to the present invention, the upper α among pixel values of n reference pixels (n is an integer of 3 or more) having a predetermined positional relationship with a target point arranged in an image. The pixels of (n−α−β) remaining reference pixels excluding pixels and lower β pixels (where α and β are integers of 0 or more that satisfy α + β ≧ 1 and n−α−β ≧ 2) A step of obtaining a value, a step of calculating an average value of pixel values of (n−α−β) remaining reference pixels, and a step of associating the average value with the target point as a converted pixel value A new image signal is generated.

  According to the present invention, it is possible to suitably suppress noise in which a generation region of dark current or the like tends to expand.

  Hereinafter, embodiments of the present invention (hereinafter referred to as embodiments) will be described with reference to the drawings.

  FIG. 1 is a schematic block diagram illustrating a configuration of an image signal processing apparatus according to an embodiment of the present invention, in which tone correction, contour correction, and noise suppression are performed based on an image signal output from an image sensor 30. Generate image data. Here, the image sensor 30 is a CCD image sensor, and the image signal Y 0 (t) output from the image sensor 30 is input to the analog signal processing circuit 32. The analog signal processing circuit 32 performs processing such as sample hold and automatic gain control (AGC) on the image signal Y0 (t) to generate an image signal Y1 (t) according to a predetermined format. The A / D conversion circuit 34 converts the image signal Y1 (t) output from the analog signal processing circuit 32 into digital data, and outputs image data D0 (n). The digital signal processing circuit 36 takes in the image data D0 (n) from the A / D conversion circuit 34 and performs various processes.

  The digital signal processing circuit 36 includes a subtractor 40, a line memory group 50, a pixel correction filter circuit 52, a first signal generation unit 54, a second signal generation unit 56, multipliers 58 and 60, an adder 62, and a YC processing unit 64. And a control unit 66.

  The subtractor 40 removes the optical black signal component OPB from the image data D0 (n).

  The line memory group 50 is composed of a line memory of 7 lines, holds image data for 7 lines continuous in the vertical direction, and provides the image data for processing in the pixel correction filter circuit 52 and the second signal generator 56. When the processing of the pixel correction filter circuit 52 and the second signal generation unit 56 for the currently held 7 lines of data is completed, the subsequent line is replaced with the line 1 that has been read first, and the line memory group 50 Is read.

  The pixel correction filter circuit 52 performs a later-described two-dimensional filter process based on image data held in the line memory group 50. In this two-dimensional filter process, a process to be described later is performed based on a reference pixel including a target pixel and a plurality of pixels arranged around the target pixel, and an obtained result value is output as a converted pixel value of the target pixel. The pixel correction filter circuit 52 sets a target pixel for each line held in the line memory group 50, performs a two-dimensional filter process on the seven target pixels in parallel, and is configured with converted pixel values. Seven lines of image signals are output to the first signal generator 54 in parallel.

  The first signal generation unit 54 generates a luminance signal Y that has been subjected to LPF processing in both the vertical and horizontal directions and a contour correction signal A based on the seven lines of image signals output in parallel from the pixel correction filter circuit 52. And output. The second signal generator 56 generates and outputs a luminance signal Y ′ and a contour correction signal A ′ that have been similarly subjected to LPF processing based on the image signals of 7 lines read out in parallel from the line memory group 50. .

  FIG. 2 is a block diagram showing a schematic configuration of the first signal generation unit 54 and the second signal generation unit 56, and includes an LPF 80 and a contour correction signal generation circuit 82. The LPF 80 includes a VLPF 84 that performs LPF processing in the vertical direction and an HLPF 86 that performs LPF processing in the horizontal direction, and generates a luminance signal Y or Y ′. Here, the VLPF 84 receives 7 lines from the pixel correction filter circuit 52 or the line memory group 50 as taps, and outputs an image signal of 1 line as the LPF processing result. The HLPF 86 performs LPF processing on the image signal based on the set tap coefficient.

  The contour correction signal generation circuit 82 generates, as the contour correction signal A, a contour correction signal Av for the vertical direction and a contour correction signal Ah for the horizontal direction. The signal Av is generated by sequentially processing the image signal obtained from the pixel correction filter circuit 52 or the line memory group 50 by the VBPF 88 and the HLPF 90 that perform band pass filter (BPF) processing in the vertical direction. The signal Ah is generated by sequentially processing the image signal obtained from the pixel correction filter circuit 52 or the line memory group 50 with the VLPF 94 and the HBPF 96 that performs the BPF processing in the horizontal direction. Note that the second derivative waveform of the image signal that greatly fluctuates at the edge portion in the image is obtained by the BPF processing. Further, the contour correction signal generation circuit 82 can be configured to multiply A and A ′ by an appropriate gain for output.

  The first signal generation unit 54 and the second signal generation unit 56 have the common block configuration shown in FIG. 2, but the tap coefficients set in the LPFs 84, 86, 90, 94, the VBPF 88, and the HBPF 96 are different as will be described later. Can be made.

  The multiplier 58 multiplies the output signal of the first signal generation unit 54 by a coefficient R, while the multiplier 60 multiplies the output signal of the second signal generation unit 56 by a coefficient (1-R). Here, the value of R is controlled by the control unit 66 based on the exposure control condition for the image sensor 30.

  The adder 62 adds and synthesizes the outputs of the multipliers 58 and 60 and outputs the result to the YC processing unit 64. Specifically, the luminance signal Ys synthesized based on the luminance signals Y and Y ′ and the contour correction signal As synthesized based on the contour correction signals A and A ′ are output from the adder 62 to the YC processing unit 64. Is done.

  The YC processing unit 64 performs gradation correction processing and contour correction processing using the luminance signal Ys and the contour correction signal As. FIG. 3 shows a schematic block configuration of the YC processing unit 64, which includes a gamma correction circuit 100 and an adder 102. The gamma correction circuit 100 performs a process of converting the signal level on the image signal Ys based on the nonlinear conversion characteristic. The luminance signal output from the gamma correction circuit 100 is added and combined with the contour correction signal As by the adder 102, thereby generating a luminance signal with contour correction. The YC processing unit 64 can further perform other signal processing such as color separation to generate a color signal, but a description thereof is omitted here.

  Next, the two-dimensional filter process performed by the pixel correction filter circuit 52 will be described. FIG. 4 is a schematic diagram showing an example of the arrangement of image data and the position of reference pixels in an image region (hereinafter referred to as a window) used for the processing of the two-dimensional filter. Here, the window is 7 vertical pixels × 5 horizontal pixels. Incidentally, the seven pixels in the vertical direction correspond to the number of lines held in the line memory group 50. Further, R, G, and B in the figure represent colors associated with each pixel, and mean red, green, and blue, respectively. Pixels in which the symbol G is circled indicate reference pixels. Incidentally, the color arrangement here is a Bayer arrangement. Five or four reference pixels are set in the window. FIG. 4A shows the arrangement of the reference pixels when the target point whose value is determined by the filtering process corresponds to the center pixel of the window. The reference pixels in this case are a total of five pixels, one pixel corresponding to the target point and four pixels positioned in the vertical and horizontal directions. These reference pixels are set to pixels of the same color. For example, the central reference pixel corresponding to the target point shown in FIG. 4 is a G pixel, and the surrounding four reference pixels are set at positions shifted by two pixels vertically and horizontally from the central reference pixel. The five reference pixels in this arrangement can be set in the central three lines of the seven lines constituting the window. That is, in the example of FIG. 4, when the target pixel is the B pixel that is the horizontal center pixel of the third and fifth lines, similarly to the case where the G pixel of the horizontal center pixel of the fourth line is the target point. A total of five reference pixels located at the target point and its top, bottom, left, and right are set.

  On the other hand, four reference pixels are set for each of the upper and lower two lines of the seven lines constituting the window. FIG. 4B shows an example of this case, and shows the arrangement of reference pixels when the target point is the G pixel of the horizontal center pixel of the second line from the top. The reference pixels in this case are a total of four pixels, one pixel corresponding to the target point, two pixels located in the left-right direction of the target point, and one pixel located either above or below the target point. Also in this case, the reference pixel is set to a pixel of the same color.

  The pixel correction filter circuit 52 ignores the upper three pixels (or two pixels) in order of increasing pixel values among the five (or four) reference pixels set for each target point, and the remaining two pixels An average value of the pixel values of the reference pixels is obtained, and the average value is assigned to the target point as a converted pixel value. The processing in the pixel correction filter circuit 52 can be realized by using a program, but here, an example in which it is realized as a hardware circuit capable of higher speed processing will be described. 5 and 6 are circuit diagrams showing the schematic configuration. FIG. 5 shows a circuit configuration for performing the process when there are five reference pixels, and FIG. 6 shows a circuit configuration for performing the process when there are four reference pixels. In order to perform in parallel the processing for the target points set on the three central lines of the window and the processing for the target points set on the two upper and lower lines of the window, the pixel correction filter circuit 52 Includes three circuits shown in FIG. 5 and four circuits shown in FIG.

  The circuit of FIG. 5 receives pixel values V1 to V5 of five reference pixels. The block 110 receives V1 to V4, obtains the minimum value M1 among them, and outputs it to the averaging circuit 112. In addition, the block 110 generates a selector signal S according to the pixel having the minimum value and supplies the selector signal S to the block 114. Based on the selector signal S, the block 114 removes the minimum value pixel from V1 to V4, takes in the value of four pixels added with V5 instead, and obtains the minimum value M2 of them to the addition averaging circuit 112. Output. The addition averaging circuit 112 calculates an average value of M1 and M2, and outputs this as a pixel value W corresponding to the target point.

  The block 110 includes three comparators 120 to 124, three selectors 126 to 130, and a decoder 132. The comparator 120 compares V1 and V2, and inputs a signal corresponding to the comparison result to the selector 126 and the decoder 132. Based on the comparison result, the selector 126 selects the smaller one of V1 and V2 and outputs it to the comparator 124 and the selector 130. The comparator 122 compares the magnitudes of V3 and V4, and inputs a signal corresponding to the comparison result to the selector 128 and the decoder 132. Based on the comparison result, the selector 128 selects the smaller one of V3 and V4 and outputs it to the comparator 124 and the selector 130. The comparator 124 compares the two pixel values input from the selectors 126 and 128 and inputs a signal corresponding to the comparison result to the selector 130 and the decoder 132. Based on the comparison result, the selector 130 selects and outputs the smaller of the two input data. This output value is M1. The decoder 132 generates a signal S for selecting one of the four selectors 140 to 146 provided at the input of the block 114 based on the comparison results input from the comparators 120 to 124.

  The block 114 includes three comparators 150 to 154 and three selectors 156 to 160 in addition to the four selectors 140 to 146 provided on the input side. In the selectors 140 to 146, V1 to V4 are input to one input terminal, and V5 is input to the other input terminal in common. The selector signal S causes one of the selectors 140 to 146 that receives the minimum value of V1 to V4 as one input data to output the other input data V5, and the remaining three selectors In contrast, the input data on the side other than V5 is output. That is, as described above, the selectors 140 to 146 output three pixel values excluding the minimum value of V1 to V4 and V5. The processing of the comparators 150 to 154 and the selectors 156 to 160 for the four pixel values output from the selectors 140 to 146 is the same as the processing of the comparators 120 to 124 and the selectors 126 to 130 in the block 110 described above. From 160, the minimum value of the four pixel values, that is, M2 is output.

  The circuit of FIG. 6 receives pixel values V6 to V9 of four reference pixels. The block 200 to which V6 to V8 are input can be easily understood if the configuration of the block 110 is simplified because the number of pixels to be processed is one less. The block 200 calculates the minimum value M1 from V6 to V8 and outputs it to the averaging circuit 202. On the other hand, the configuration and operation of the block 204 can be easily understood if the configuration of the block 114 is simplified by one pixel to be processed. Based on the selector signal S generated by the block 200, the block 204 removes the minimum value pixel from V6 to V8, takes in the value of three pixels added with V9 instead, and obtains the minimum value M2 of them. The result is output to the averaging circuit 202. The addition averaging circuit 202 calculates an average value of M1 and M2, and outputs this as a pixel value W corresponding to the target point.

  The pixel correction filter circuit 52 processes the seven lines of image data held in the line memory group 50 while sequentially moving a vertical 7 pixel × horizontal 5 pixel window one pixel at a time in the horizontal direction. The pixel correction filter circuit 52 includes a plurality of the circuits shown in FIGS. 5 and 6 as described above, and performs parallel processing for seven pixels arranged vertically at the center in the horizontal direction of the window at each position of the window. Thus, converted pixel values corresponding to these seven pixels are generated.

  The first signal generation unit 54 captures the converted pixel values of the seven pixels arranged in the vertical direction output from the pixel correction filter circuit 52 into the VLPFs 84 and 94 and VBPF 88 that are filters in the vertical direction. These vertical filters are successively inputted with a set of seven vertical pixels, and each filter generates one pixel data for each set. As the window moves in the horizontal direction, the subsequent filters HLPF 86 and 90 and HBPF 96 in the horizontal direction perform processing on a plurality of consecutive image data sequentially output from the filter in the vertical direction.

  The second signal generator 56 directly takes out the image data of the seven pixels arranged in the vertical direction as the target point by the pixel correction filter circuit 52 from the line memory group 50, performs the processing in the VLPF 84, 94 and the VBPF 88, and then the HLPF 86. 90 and HBPF 96. The processing of the second signal generation unit 56 is performed in conjunction with the processing of the first signal generation unit 54, and the processing results for the pixels at the same position are output in synchronization.

  As described above, according to the tap coefficients set in the LPFs 84, 86, 90, 94, the VBPF 88, and the HBPF 96, the first signal generation unit 54 and the second signal generation unit 56 set the pass characteristics of these filters. Can be different. The schematic transmission characteristic diagram shown in FIG. 7 is an example. 7A and 7B are examples of LPF characteristics. For example, it is assumed that the characteristics of the LPFs of the second signal generator 56 have relatively high cutoff frequencies as shown in FIG. On the other hand, the characteristic of each LPF of the first signal generation unit 54 is assumed to have a relatively low cutoff frequency as shown in FIG. FIGS. 7C and 7D are examples of BPF characteristics. For example, the characteristics of each BPF of the second signal generation unit 56 are relatively wide as shown in FIG. 7C. On the other hand, the characteristics of each BPF of the first signal generation unit 54 can be relatively low as shown in FIG.

  By providing such a characteristic difference, it is possible to suitably cope with the change in shot noise caused by dark current according to the exposure time. That is, the first signal generator 54 is set to have a characteristic that the attenuation capability with respect to the noise component is relatively high as shown in FIGS. An image signal in which image quality deterioration is suitably suppressed is obtained. On the other hand, the second signal generator 56 is set to have a characteristic with a relatively low attenuation capability with respect to the noise component as shown in FIGS. An image signal in which a suitable resolution is maintained under photographing conditions can be obtained.

  As described above, in the present apparatus, the first signal generation unit 54 and the second signal generation unit 56 have different characteristics depending on the presence / absence of the pixel correction filter circuit 52 and the characteristic difference between the first signal generation unit 54 and the second signal generation unit 56. Image signals having different noise suppression effects and different resolutions can be obtained. Then, the control unit 66 mixes these image signals by controlling the mixing ratio in conjunction with the exposure conditions, so that a suitable image quality according to the exposure conditions can be realized.

  In the exposure control for the image pickup device 30, in order to brighten the image, for example, the exposure time is first extended, and when the exposure time reaches the upper limit, the analog gain is then increased by automatic gain control (AGC). When the gain reaches the upper limit, control for increasing the gain of automatic digital gain control (ADG) is performed. Based on the exposure time and each gain, the control unit 66 changes a parameter R that determines a multiplication coefficient set in each of the multipliers 58 and 60 according to which state exposure control is currently in. For example, since the dark current is likely to increase as the exposure control shifts to the brighter side of the image, R is increased continuously or stepwise. Thereby, the specific gravity of the image signal on the first signal generation unit 54 side having a high noise removal effect is increased, and a suitable noise suppression effect can be expected.

  Further, the signal on the first signal generation unit 54 side and the signal on the second signal generation unit 56 side can be switched and output with a predetermined reference point of the exposure control condition as a boundary. Further, 100% of the image signal on the second signal generator 56 side is output up to a predetermined reference point of the exposure control condition, and when the reference point is exceeded, the above-described continuous or stepwise adjustment control of the mixing ratio is started. It can also be configured as follows.

  In the above configuration, for example, the reference pixels are a total of five pixels arranged vertically and horizontally with the target point as the center, but the arrangement of the reference pixels is not limited to this. Further, in the above-described configuration, an example has been shown in which, in response to the increase in pixel value in accordance with the dark current, several higher-order pixels are removed from the pixel value order, and the average of the remaining two pixels is taken. However, for example, when the pixel value is defined as a negative value, the pixel value may decrease as the dark current increases. In this case, the lower pixel in the rank order of the pixel value is selected. Can be configured to remove. It is also possible to remove both the upper and lower pixels. At this time, for example, when the pixel value increases due to an increase in dark current, the number of removed pixels higher than the lower order is increased, and conversely, when the pixel value decreases due to an increase in dark current, Increasing the number of pixels to be removed is suitable for removing dark current noise. If both the upper and lower sides are removed, pixels with a large influence of dark current are removed by removal on one side, and defective pixels that do not generate signal charges are removed by removal on the other side. It is possible to improve.

  In the line memory group 50, as described above, an average value of reference pixels (residual reference pixels) to be left among the reference pixels is obtained and assigned to the target point. The average value substantially corresponds to a sampling value at the center of gravity of the coordinates of the remaining reference pixels. Hereinafter, the center of gravity is referred to as an average value sampling point. Here, a predetermined representative point, for example, the position of the sampling point corresponding to the pixel can be used as the coordinates of each residual reference pixel.

  The number of residual reference pixels is assumed to be two or more pixels in order to obtain an average value, but can be determined in consideration that the average value sampling point is located within a predetermined distance from the target point. . With such consideration, it is possible to suppress a decrease in resolution caused by regarding the sampling value at the average sampling point as the sampling value of the target point.

  8 to 10 are schematic diagrams illustrating some examples of the arrangement of reference pixels and average value sampling points. In the figure, ◯ indicates the position of the sampling point of the reference pixel, while the average value sampling point is located within the point indicated by the x mark or the hatched area. FIG. 8A corresponds to the arrangement of FIG. 4A and the circuit configuration of FIG. 5 described above, and shows a case where there are 5 reference pixels and 2 residual reference pixels. FIG. 8B corresponds to the arrangement of FIG. 4B and the circuit configuration of FIG. 6 described above, and shows a case where there are four reference pixels and two remaining reference pixels. In these cases, the average value sampling point is located on the side of a quadrangle or triangle connecting the sampling points of the reference pixel P surrounding the target point T or inside these polygons. Here, the pixel interval in the vertical direction and the pixel interval in the horizontal direction of the image sensor 30 are basically set substantially equal. The point H that is the average sampling point farthest from the target point T is an intermediate point between the sides connecting the points P. This average value sampling point H is located closer to the target point T than the reference pixel P nearest to the target point. Incidentally, when the median filter which is the conventional technique is applied to the above reference pixel, the target pixel T is replaced with a pixel whose signal level order is the center of the reference pixels. That is, the target pixel T may be replaced with any of the surrounding reference pixels P. On the other hand, in the present apparatus, as described above, the farthest point that can be replaced with the target pixel T is the average value sampling point H that is closer to the target pixel T than the reference pixel P. That is, according to the present apparatus, the target pixel T can be replaced with a point closer to the target pixel T than when the median filter is used.

  FIG. 9 shows a case where there are four reference pixels, and two of them are the remaining reference pixels. The target point T in this case is the center of gravity of the four reference pixels P. Also in this case, the average value sampling point is located on the side of a rectangle connecting the sampling points of the reference pixel P surrounding the target point T or inside the polygon.

  FIGS. 10A and 10B show the case where the reference pixel is a pixel group of 3 rows and 3 columns. FIG. 6A shows a case where the remaining reference pixels are three pixels. In this case, the region where the average sampling point is present is in contact with a rectangle connecting the reference pixels P surrounding the target point T. On the other hand, FIG. 4B shows a case where the remaining reference pixels are four pixels. In this case, the region where the average sampling point is present is included inside the rectangle connecting the reference pixels P surrounding the target point T.

1 is a schematic block diagram illustrating a configuration of an image signal processing apparatus according to an embodiment of the present invention. It is a block diagram which shows the schematic structure of a 1st signal generation part and a 2nd signal generation part. 3 is a schematic block configuration of a YC processing unit. It is a schematic diagram which shows an example of arrangement | positioning of the image data and the position of a reference pixel in the image area | region used for the process of a two-dimensional filter. FIG. 5 is a schematic circuit diagram of a portion that performs two-dimensional filter processing on five reference pixels in a pixel correction filter circuit. FIG. 4 is a schematic circuit diagram of a portion that performs two-dimensional filter processing on four reference pixels in a pixel correction filter circuit. It is a typical pass characteristic figure of LPF and BPF which constitute the 1st signal generation part and the 2nd signal generation part. It is a schematic diagram which shows arrangement | positioning of a reference pixel and an average value sampling point. It is a schematic diagram which shows the example of other arrangement | positioning of a reference pixel and an average value sampling point. It is a schematic diagram which shows the example of other arrangement | positioning of a reference pixel and an average value sampling point. It is a block diagram which shows the structure of the conventional image signal processing apparatus which processes the image signal produced | generated by the image pick-up element.

Explanation of symbols

  Reference Signs List 30 imaging device, 32 analog signal processing circuit, 34 A / D conversion circuit, 36 digital signal processing circuit, 40 subtractor, 50 line memory group, 52 pixel correction filter circuit, 54 first signal generation unit, 56 second signal generation Unit, 58, 60 multiplier, 62, 102 adder, 64 YC processing unit, 66 control unit, 80 LPF, 82 contour correction signal generation circuit, 84, 94 VLPF, 86, 90 HLPF, 88 VBPF, 96 HBPF, 100 Gamma correction circuit, 110, 114, 200, 204 blocks, 112 addition averaging circuit, 120, 122, 124, 150, 152, 154 comparator, 126, 128, 130, 140, 142, 144, 146, 156, 158, 160 selector, 132 decoder.

Claims (10)

In an image signal processing apparatus that processes an image signal generated by an image sensor,
This is a filter circuit that generates a converted pixel value associated with a target point based on n reference pixels (n is an integer of 3 or more) having a predetermined positional relationship with the target point arranged in the image. Of the pixel values of the n reference pixels, the upper α pixel and the lower β pixel (where α and β are integers of 0 or more that satisfy α + β ≧ 1 and n−α−β ≧ 2) are excluded. A pixel correction filter circuit that calculates an average value of the pixel values of (n−α−β) remaining reference pixels as the converted pixel value;
When the pixel value increases due to dark current generated in the image sensor,
The pixel correction filter circuit has a relationship of α> β with respect to the upper α pixel and the lower β pixel to be removed. The image signal processing apparatus according to claim 1,
The residual reference pixel is a polygonal region formed by connecting the representative points of each of the plurality of reference pixels in which the center of gravity of the coordinates of the representative points of each of the plurality of residual reference pixels is located around the vicinity of the target point. Be set to exist within,
An image signal processing apparatus. The image signal processing apparatus according to claim 2,
The residual reference pixel is set so that the center of gravity is located in an inner portion excluding the circumference of the polygonal region;
An image signal processing apparatus. In the image signal processing device according to any one of claims 1 to 3,
A processing unit that includes the pixel correction filter circuit, and selectively performs a first signal process involving pixel correction processing by the pixel correction filter circuit and a second signal process not involving the pixel correction process;
A control unit that controls switching between the first signal processing and the second signal processing in the processing unit in accordance with exposure control in the imaging element when acquiring the image signal;
An image signal processing apparatus comprising: In the image signal processing device according to any one of claims 1 to 3,
A first signal processing that includes the pixel correction filter circuit and that includes a pixel correction process performed by the pixel correction filter circuit; and a second signal process that does not include the pixel correction process. A processing unit that combines the output of the second signal processing with a variable combining ratio;
A control unit that controls the synthesis ratio in accordance with exposure control in the imaging device when acquiring the image signal;
An image signal processing apparatus comprising: In the image signal processing device according to claim 4 or 5,
The processing unit generates and outputs a noise-suppressed image signal in which noise is suppressed,
The first signal processing generates a noise-suppressed image signal by performing low-pass filter processing based on a first low-pass characteristic on the image signal after the pixel correction processing,
The second signal processing performs low-pass filter processing based on a second low-pass characteristic different from the first low-pass characteristic on the image signal that is not subjected to the pixel correction process, and thereby the noise-suppressed image signal Generating,
An image signal processing apparatus. The image signal processing device according to any one of claims 4 to 6,
The processing unit generates and outputs a contour correction signal corresponding to the contour of the image based on the image signal,
The first signal processing uses the image signal after the pixel correction processing to generate the contour correction signal based on a first band pass characteristic,
The second signal processing uses the image signal not subjected to the pixel correction processing to generate the contour correction signal based on a second band pass characteristic different from the first band pass characteristic;
An image signal processing apparatus. The image signal processing device according to claim 6,
The image signal processing apparatus according to claim 1, wherein the first low-pass characteristic has a cut-off frequency lower than that of the second low-pass characteristic. The image signal processing apparatus according to claim 7,
The image signal processing apparatus according to claim 1, wherein the first band pass characteristic has a pass band frequency lower than that of the second band pass characteristic. In an image signal processing method for processing an image signal generated by an image sensor,
Of the pixel values of n reference pixels (n is an integer of 3 or more) having a predetermined positional relationship with respect to a target point arranged in the image, an upper α pixel and a lower β pixel (where α, obtaining the pixel values of (n−α−β) remaining reference pixels excluding (β is an integer greater than or equal to 0 satisfying α + β ≧ 1 and n−α−β ≧ 2);
Calculating an average value of pixel values of the (n−α−β) remaining reference pixels;
Associating the average value with the target point as a converted pixel value;
And generating a new image signal ,
An image signal processing method having a relationship of α> β with respect to the upper α pixel and the lower β pixel to be removed when the pixel value is increased by a dark current generated in the image sensor .

Images