JP4517548B2 - Imaging device and noise removal method for imaging signal - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging apparatus suitable for, for example, a digital still camera and a noise removal method for an imaging signal.
[0002]
[Prior art]
Recently, with the spread of personal computers, digital still cameras have also been widely used.
[0003]
FIG. 6 exemplifies a pixel configuration of a CCD (Charge Coupled Device) as an image sensor that is often used in this type of digital still camera.
[0004]
In the drawing, each square-shaped rectangle indicates a pixel C formed of one light receiving element, and each of the pixels C, C,... Is arranged in a matrix to form an imaging surface of the CCD.
[0005]
However, by an optical lens system (not shown) combined with the CCD, each pixel C inside the range indicated by the frame line FL in the figure becomes a light receiving portion RO effective for imaging, while the range indicated by the frame line FL Each pixel C on the outside becomes an optical dark portion OB that is shielded from light.
[0006]
Normally, the output from the pixel C of the optical dark portion OB is used as a zero level corresponding to pure black in the output signal of the pixel C of the light receiving portion RO where no light strikes.
[0007]
Specifically, this is a so-called OB clamp technique in which the average value of the signal levels of the all-optical dark portion OB is subtracted from the output data of each pixel C of the light receiving portion RO. Therefore, an accurate output of image data can be obtained even with a CCD having individual differences in the level of the output signal.
[0008]
FIG. 7 shows a circuit configuration of an imaging system of such a digital camera. In the figure, analog value image signals constituting the entire range including the light receiving portion RO and the optical dark portion OB obtained by imaging with the CCD 11 are set by the sample hold circuit (S / H) 12 at that time. After being sampled and held in a time width corresponding to the pixel reading speed from the CCD 11, it is converted into digital data by the A / D converter 13.
[0009]
Based on the data of each pixel of the digital value thus obtained, the OB clamp unit 14 first calculates an average value per pixel from the sum of the pixels in the optical dark portion OB, and then uses the average value to receive the light receiving unit. The above-described OB clamping is performed by subtracting the average value from each pixel in the RO section, and the image data after the OB clamping is sent to the color processing circuit 15.
[0010]
In the color processing circuit 15, after performing correction processing such as gamma correction and white balance correction, interpolation processing is performed to compensate for the color components, and then the primary color system or complementary color system image data is recorded for recording the luminance color difference system. Convert to image data YUV.
[0011]
Then, the luminance / color-difference image data is compressed in accordance with a predetermined method, for example, JPEG (Joint Photographic Coding Experts Group), and is output to a recording system and a display circuit in the next stage (not shown).
[0012]
Accordingly, various timing signals are supplied from the timing generation circuit (TG) 16 to all of the CCD 11, the sample hold circuit 12, the A / D converter 13, the OB clamp unit 14, and the color processing circuit 15. The timing generation circuit 16 is composed of a digital circuit.
[0013]
That is, the driving and reading of the CCD 11 are performed by the timing signal from the timing generating circuit 16, and the reading is performed in units of one pixel by raster scanning similar to the television screen. The output signal is analog. On the other hand, the timing generation circuit 16 sends the timing signal by digital operation as described above. Therefore, the image signal output from the CCD 11 is inevitably mixed with noise accompanying the digital circuit operation. It will end up.
[0014]
[Problems to be solved by the invention]
The above point will be described in more detail.
[0015]
FIG. 8 shows a primary color filter array generally called a Bayer array generally employed in the CCD 11, and G (green) × 2 pixels, R, as indicated by a broken line range D in the figure. (4) A pixel interpolation process or the like is performed by the subsequent color processing circuit 15 using a total of four pixels in a rectangular shape of 2 pixels in the horizontal direction and 2 pixels in the vertical direction, each consisting of (red) and B (blue) × 1 pixel. Become.
[0016]
That is, the timing generation circuit 16 sends to the color processing circuit 15 a timing signal obtained by dividing the timing signal for sampling for one pixel period read from the CCD 11 by half. The noise associated with the timing signal and the noise associated with the operation of the color processing circuit 15 due to the timing signal are mixed in the image signal read from the CCD 11.
[0017]
Therefore, noise accompanying the processing in the color processing circuit 15 is mixed in a period of two horizontal pixels when reading a signal from the CCD 11, and thereby color reproducibility particularly in a dark part where the signal level in the light receiving part RO is low. Will be adversely affected.
[0018]
A timing signal obtained by dividing the timing signal for each pixel by 1/4 is also output from the timing generation circuit 16. The timing signal divided by 1/2 is applied to reading the signal from the CCD 11. The effect of noise is the greatest.
[0019]
In addition, the timing signal for each pixel itself is a noise mixed in the signal from the CCD 11, but in this case, since the noise is mixed uniformly in all the pixels, it is not so conspicuous in the entire image.
[0020]
As one of methods for eliminating the influence of noise on the readout of the signal from the CCD 11 by the timing signal obtained by dividing the timing signal for each pixel by 1/2 as described above, the one shown in FIG. It has been.
[0021]
FIG. 9 (1) illustrates the timing of signals read in pixel units from the odd lines of the CCD 11 having the Bayer array color filters shown in FIG. 8, and FIG. 9 (2) shows the pixel unit. The basic frequency signal of the transfer clock synchronized with reading is shown.
[0022]
Based on the fundamental frequency signal shown in FIG. 9 (2), a signal as shown in FIG. 9 (3) is generated by dividing the signal by 1/2, and a signal is read from the CCD 11 using this signal. It is considered that the offset amount is changed and set at a cycle of two pixels.
[0023]
However, such a method for dealing with noise assuming a predetermined period cannot cope with noise generated at other periods and timings. A digital camera using a CCD uses a lot of digital circuits in addition to driving the CCD, such as normal image signal processing, and its operation state changes from moment to moment, so that the noise generation state also changes accordingly. .
[0024]
Therefore, even if the noise is canceled assuming a predetermined period as described above, it is difficult to deal with all the noise, and as a result, deterioration of the image quality of the image data obtained by imaging cannot be avoided. There was a bug.
[0025]
The present invention has been made in view of the above circumstances, and the object of the present invention is to reliably remove noise components superimposed on the signal when reading the image signal from the solid-state imaging device, and to improve the image quality. An object of the present invention is to provide an imaging apparatus capable of preventing deterioration and a noise removal method for an imaging signal.
[0026]
[Means for Solving the Problems]
An imaging apparatus according to a first aspect of the present invention includes a solid-state imaging device, a generating unit that generates a noise cancellation signal for removing noise included in an image signal obtained by the solid-state imaging device, and the generating unit Superimposing means for superimposing the noise canceling signal generated by the above on the image signal obtained by the solid-state imaging device, state determining means for determining the operating state of the peripheral circuit, and operation of the peripheral circuit determined by this state determining means Control means for controlling the superimposition of the noise cancellation signal on the image signal by the superimposing means according to the state, the control means is a noise generated by the generating means according to the operating state of the peripheral circuit The canceling signal pattern is switched to a different pattern.
[0028]
Imaging device according to a second aspect of the present invention includes a generation step of generating a noise cancellation signal for removing noise included in image signals obtained by the solid-state image pickup element, noise canceling is generated in this generation step A superimposition step for superimposing the signal on the image signal obtained by the solid-state imaging device, a state determination step for determining the operation state of the peripheral circuit, and the superposition according to the operation state of the peripheral circuit determined in the state determination step. A control step for controlling the superimposition of the noise cancellation signal on the image signal in the step, and the control step includes a pattern of the noise cancellation signal generated in the generation step according to the operating state of the peripheral circuit. Are switched to different patterns.
[0044]
DETAILED DESCRIPTION OF THE INVENTION
(First embodiment)
Hereinafter, a case where the present invention is applied to a digital still camera will be described as a first embodiment with reference to the drawings.
[0045]
FIG. 1 shows a circuit configuration of an imaging system of the digital still camera according to the embodiment. In the figure, an optical image of a subject is formed on the CCD 22 by the optical lens system 21, and as a result, an analog image signal is obtained.
[0046]
The CCD 22 is formed, for example, by forming a primary color Bayer color filter. The image signal obtained by the CCD 22 is sampled and held by a CDS / ADC circuit 23 and then A / D converted to digitize. Is done.
[0047]
The OB clamp circuit 24 executes OB clamping for each pixel value constituting the digital image signal thus obtained. At this time, an OB clamp is appropriately changed by superimposing a noise cancellation pattern signal sent from a noise pattern ROM 25 described later, and the clamped image signal is output to the color signal processing circuit 26.
[0048]
The color signal processing circuit 26 performs various correction processes and interpolation processes on the image signal composed of the pixel values of the Bayer array, converts the image signal into a luminance / chrominance image signal, further compresses the data, and then performs a next step (not shown). Output to stage recording and display circuits.
[0049]
In addition, the operating state of the color signal processing circuit 26 is output to the noise pattern ROM 25. The noise pattern ROM 25 stores in advance a plurality of pattern signals for noise cancellation, selects and reads out an optimum pattern signal corresponding to the operation state input from the color signal processing circuit 26, and reads the OB. It is sent to the clamp circuit 24.
[0050]
Accordingly, various timing signals are supplied from the timing generation circuit (TG) 27 to all of the CCD 22, the CDS / ADC circuit 23, the OB clamp circuit 24, the color signal processing circuit 26, and the noise pattern ROM 25, and the operation timings are determined. Control is executed in an integrated manner.
[0051]
FIG. 2 mainly shows a detailed circuit configuration in the color signal processing circuit 26. In the figure, the image signal composed of the Bayer array pixel values sent from the OB clamp circuit 24 is sequentially stored in the buffer memory 31 and then read out to the correction processing unit 32.
[0052]
The correction processing unit 32 adjusts each color component by performing gamma correction and white balance correction. The image signal after the correction processing is stored in the buffer memory 33 and then sent to the interpolation / YUV conversion unit 34. It is done.
[0053]
This interpolation / YUV conversion unit 34 first interpolates R and B color components to make the G component and the data amount equal to the image signal consisting of the pixel values of the Bayer array, and then performs luminance color difference by a predetermined matrix calculation. This is converted into a system image signal (YUV data), and the obtained image signal is sent to the JPEG processing unit 35.
[0054]
The JPEG processing unit 35 compresses the amount of data obtained by performing processing such as ADCT (Adaptive Discrete Cosine Transform) and Huffman coding on the transmitted image signal. The image signal is sent as an output of the color signal processing circuit 26 to the recording and display circuits in the next stage.
[0055]
Thus, the control unit 36 performs overall control of the operations of the buffer memory 31, the correction processing unit 32, the buffer memory 33, the interpolation / YUV conversion unit 34, and the JPEG processing unit 35. It operates based on the timing signal from the generation circuit 27 and also outputs a signal corresponding to the operation state of each circuit to the noise pattern ROM 25.
[0056]
Next, the operation of the above embodiment will be described.
[0057]
FIG. 3 shows the processing contents executed by the control unit 36 of the color signal processing circuit 26 in the imaging mode, and initially waits for the timing signal from the timing generation circuit 27 to be input ( Step S01) When it is determined that the input has been made, it is next determined whether or not the interpolation / YUV conversion unit 34 is operating at that time (Step S02).
[0058]
This is because, while the interpolation / YUV conversion unit 34 performs the interpolation process using the image signals for a plurality of predetermined lines stored in the buffer memory 33, the noise of the two pixel period described above is generated. While the image signal for a predetermined plurality of lines is newly accumulated in 33 for the next interpolation process, the interpolation / YUV conversion unit 34 stops the interpolation process, so that no noise is generated.
[0059]
Therefore, the control unit 36 appropriately selects a pattern signal for noise cancellation in the noise pattern ROM 25 by determining whether the interpolation / YUV conversion unit 34 is performing an interpolation process. Read and output to the OB clamp circuit 24.
[0060]
If it is determined in step S02 that the interpolation process by the interpolation / YUV conversion unit 34 is being performed, the control unit 36 sends a pattern signal for canceling noise of a two-pixel period to the noise pattern ROM 25. After outputting a signal to be read and output to the OB clamp circuit 24 (step S03), it waits for the next timing signal to be sent from the timing generation circuit 27 again.
[0061]
Further, the interpolation / YUV conversion unit 34 is not in the process of interpolation processing, and the image signals for a predetermined number of lines are accumulated in the buffer memory 33. Therefore, the interpolation / YUV conversion unit 34 temporarily stops the interpolation processing. If it is determined that the signal is present, the control unit 36 outputs a signal for stopping reading of the pattern signal for noise cancellation to the noise pattern ROM 25 (step S04), and the next timing signal is generated again. It waits for transmission from the circuit 27.
[0062]
Therefore, in the OB clamp circuit 24, when a pattern signal is read from the noise pattern ROM 25, the level of the OB clamp is appropriately changed each time according to the content of the pattern signal, thereby eliminating the influence of noise as much as possible. Will be able to.
[0063]
As described above, when the image signal is read from the CCD 22, the noise pattern corresponding to the state of the interpolation operation of the interpolation / YUV conversion unit 34 of the color signal processing circuit 26 which is considered to be particularly affected by noise on the image signal. Since the noise removal is performed by switching the pattern signal generated in the ROM 25, it is possible to more reliably eliminate the influence of noise and prevent the image quality from deteriorating.
[0064]
In addition, since the operation is executed only at the timing when the operation for removing the noise is necessary, the pattern signal for noise cancellation is added to the image signal not affected by the noise. In addition to preventing noise from being superimposed, it is possible to reduce processing steps for image signals by avoiding useless circuit operations, and to promptly send them to a subsequent display system or recording system circuit. .
[0065]
In particular, regarding the execution of the interpolation operation in the interpolation / YUV conversion unit 34, the control unit 36 constantly monitors this and switches the signal pattern for noise cancellation at an accurate timing. Can be eliminated.
[0066]
(Second Embodiment)
A case where the present invention is applied to a digital still camera will be described below as a second embodiment with reference to the drawings.
[0067]
FIG. 4 shows a circuit configuration of an imaging system of the digital still camera according to the embodiment. In the figure, an optical image of a subject is formed on a CCD 42 by an optical lens system 41, and as a result, an analog image signal is obtained.
[0068]
The CCD 42 is formed, for example, by forming a color filter of a primary color Bayer array, and is driven in synchronization with a timing signal from a timing generation circuit (TG) 47. An image signal obtained by the CCD 42 is a CDS. The signal is also sampled and held by the / ADC circuit 43 in synchronization with the sampling clock from the timing generation circuit 47 and digitized by A / D conversion.
[0069]
The pixel values constituting the digital image signal thus obtained are sequentially sent to the adder 44 and the OB data accumulation circuit 45.
[0070]
The OB data accumulation circuit 45 accumulates and stores the pixel values of the entire optical dark portion OB in the image signal in accordance with the pixel timing signal of the optical dark portion OB from the timing generation circuit 47, calculates the average value thereof, and adds the adder. 46.
[0071]
The timing generation circuit 47 outputs not only the CCD 42, the CDS / ADC circuit 43, and the OB data accumulation circuit 45 but also the noise position ROM 48 as an operation clock and a timing signal output from the CDS / ADC circuit 43 at that time. The pixel position (row digit position) of the image signal is given.
[0072]
In response to an input from the timing generation circuit 47, the noise position ROM 48 outputs information representing a pixel region in the corresponding image to the noise pattern RAM 49 based on a pre-stored table pattern.
[0073]
This noise pattern RAM 49 is sent from the timing generation circuit 47, and the pixel of which color pixel (R, Gr, B, Gb (where Gr, Gb is located next to R, B, respectively) in the color filter. A correction signal for each color pixel of the optical dark portion OB corresponding to the information on the pixel area from the noise position ROM 48 based on the timing signal indicating whether it corresponds to G))). To the adder 46.
[0074]
Thus, in the adder 46, the correction value for each color pixel of the optical dark portion OB corresponding to the pixel area information from the noise pattern RAM 49 and the average value of the pixel values of the entire optical dark portion OB from the OB data accumulating circuit 45. Is added to obtain a zero (black) level signal as the sum, and the obtained zero level signal is sent to the adder 44.
[0075]
The adder 44 adds the zero level signal and the pixel value from the CDS / ADC circuit 43 and outputs the sum to the color signal processing circuit 50.
[0076]
The color signal processing circuit 50 has a circuit configuration similar to that of the color signal processing circuit 26 in FIG. 2, and based on the timing signal from the timing generation circuit 47, the pixel value of the Bayer array that is the output of the adder 44 is used. The image signal is stored in the buffer memory as appropriate, and after various correction processing and interpolation processing, it is converted into a luminance / chrominance image signal and further compressed, and then the recording system and display in the next stage (not shown) Output to the system circuit.
[0077]
Next, the operation of the above embodiment will be described.
[0078]
FIG. 5 illustrates pixel area division in an image captured by the CCD 42, and the image is divided into rectangular areas (1) to (3) according to the number of lines from the upper end of the image. Shall.
[0079]
That is, in the process in which the image signal obtained by the CCD 42 is digitized by the CDS / ADC circuit 43 and transferred to the color signal processing circuit 50 via the adder 44, the OB data accumulation circuit 45, and the adder 46, When the signal processing circuit 50 is affected by the noise of the period corresponding to the color signal processing, the region in the image and the actual color signal processing content in the color signal processing circuit 50 are linked. If this is the case, the influence of noise applied from the color signal processing circuit 50 can be completely eliminated by switching the noise cancellation pattern signal for each region in the image.
[0080]
Therefore, the code signal corresponding to the regions (1) to (3) in the image is read from the noise position ROM 48 as information representing the pixel region to the noise pattern RAM 49, and the noise pattern RAM 49 receiving the code signal at that time If the correction value corresponding to the noise pattern for each color pixel (R, Gr, B, Gb) of the color filter is sent to the adder 46 to perform the zero level correction of the optical dark portion OB, at that time Then, the zero level data including the pattern signal for more appropriate noise cancellation in accordance with the content of the color signal processing executed by the color signal processing circuit 50 is transferred from the CDS / ADC circuit 43 by the adder 44. It can be superimposed on the pixel values that have been made.
[0081]
As described above, since the noise pattern given from the color signal processing circuit 50 can be predicted by the segmented region in the image to which the pixel belongs, switching to a more appropriate pattern signal ensures the influence of noise from the image signal. It can be removed and deterioration of image quality can be prevented.
[0082]
Therefore, in both the first and second embodiments, the noise included in the image signal is canceled by changing the signal level of the optical dark portion OB that is the reference of the zero level of the image signal. Therefore, it is possible to facilitate the pattern signal superimposing process on the image signal.
[0083]
In the first and second embodiments, the CCD is used as the solid-state image sensor. However, the present invention is not limited to this, and another solid-state image sensor such as a CMOS area sensor may be used.
[0084]
In addition, the first and second embodiments are both shown when applied to a digital still camera. However, the present invention is not limited thereto, and imaging is performed using a solid-state imaging device. As long as it is a simple imaging device, it may be a digital video camera, a CCD camera incorporated in a mobile phone, a personal computer, or the like.
[0085]
In addition, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention.
[0086]
Further, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiment, at least one of the problems described in the column of the problem to be solved by the invention can be solved, and described in the column of the effect of the invention. In a case where at least one of the obtained effects can be obtained, a configuration in which this configuration requirement is deleted can be extracted as an invention.
[0087]
【The invention's effect】
According to the present invention, it is possible to reliably remove the influence of noise due to the operating state of the peripheral circuit and prevent the deterioration of the image quality.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a circuit configuration of an imaging system of a digital still camera according to a first embodiment of the present invention.
2 is a block diagram showing a detailed circuit configuration mainly of a color signal processing circuit of FIG. 1; FIG.
FIG. 3 is a flowchart showing processing details of noise removal in the imaging mode according to the embodiment;
FIG. 4 is a block diagram showing a circuit configuration of an imaging system of a digital still camera according to a second embodiment of the present invention.
FIG. 5 is a view illustrating a segmented region of pixel positions in the image according to the embodiment;
FIG. 6 is a diagram illustrating a pixel configuration of a CCD as an image sensor.
FIG. 7 is a block diagram illustrating a circuit configuration of an imaging system of a general digital camera.
FIG. 8 is a diagram showing a configuration of a color filter having a primary color Bayer arrangement.
FIG. 9 is a diagram illustrating a noise generation state corresponding to a calling cycle of CCD pixels.
[Explanation of symbols]
11 ... CCD
12 ... Sample hold circuit (S / H)
13 ... A / D converter 14 ... OB clamp unit 15 ... color processing circuit 16 ... timing generation circuit (TG)
21 ... Optical lens system 22 ... CCD
23 ... CDS / ADC circuit 24 ... OB clamp circuit 25 ... noise pattern ROM
26 ... Color signal processing circuit 27 ... Timing generation circuit (TG)
31 ... Buffer memory 32 ... Correction processing unit 33 ... Buffer memory 34 ... Interpolation / YUV conversion unit 35 ... JPEG processing unit 36 ... Control unit 41 ... Optical lens system 42 ... CCD
43 ... CDS / ADC circuit 44 ... adder 45 ... OB data accumulating circuit 46 ... adder 47 ... timing generation circuit (TG)
48 ... Noise position ROM
49 ... Noise pattern RAM
50 ... color signal processing circuit C ... pixel D ... broken line range RO ... light receiving part OB ... optical dark part

Claims (5)

A solid-state image sensor;
Generating means for generating a noise cancellation signal for removing noise included in the image signal obtained by the solid-state imaging device;
Superimposing means for superimposing the noise cancellation signal generated by the generating means on the image signal obtained by the solid-state imaging device;
State discriminating means for discriminating the operating state of the peripheral circuit;
Control means for controlling the superimposition of the noise canceling signal on the image signal by the superimposing means according to the operation state of the peripheral circuit determined by the state determining means,
The image pickup apparatus according to claim 1, wherein the control means switches the pattern of the noise cancellation signal generated by the generation means to a different pattern according to the operating state of the peripheral circuit. Further comprising storage means for storing a plurality of noise cancellation signals of different patterns,
The generating means generates a noise canceling signal by reading the noise canceling signal from the storage means,
2. The image pickup apparatus according to claim 1, wherein the control means switches the pattern of the noise cancellation signal read from the storage means to a different pattern in accordance with the operating state of the peripheral circuit. Further comprising detection means for detecting a portion of an image signal obtained by the solid-state imaging device,
The state discriminating unit discriminates the operation state of the peripheral circuit from the part of the image signal detected by the detecting unit,
3. The image pickup apparatus according to claim 1, wherein the control unit controls superimposition of a noise canceling signal on the image signal by the superimposing unit according to a detection result of the detecting unit.   The superimposing means generates a zero level signal by changing the signal level of the optical dark pixel in the image signal obtained by the solid-state imaging device using the noise cancellation signal generated by the generating means, The imaging apparatus according to claim 1, wherein a zero level signal is superimposed on an image signal. A generation step of generating a noise cancellation signal for removing noise included in the image signal obtained by the solid-state imaging device;
A superimposition step of superimposing the noise cancellation signal generated in this generation step on the image signal obtained by the solid-state imaging device;
A state determination step of determining the operation state of the peripheral circuit;
A control step of controlling the superimposition of the noise cancellation signal on the image signal in the superimposition step according to the operation state of the peripheral circuit determined in the state determination step,
The method for removing noise of an imaging signal, wherein the control step switches and controls the pattern of the noise cancellation signal generated in the generation step according to an operating state of the peripheral circuit to a different pattern.

Images