JP2007251611A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus which corrects a flicker occurring in photographing of a still picture while photographing an animation by a small-scale circuit constitution. <P>SOLUTION: In the case of photographing the moving picture simultaneously while photographing the animation, photographing is performed after changing an exposure time to that for photographing the still picture. First, with a picture signal obtained by photographing (FL2) before changing the exposure time as reference, a correction gain to the picture signal of a frame (FL3) just after changing is calculated by each horizontal line, and the picture signal of the frame (FL3) is flicker-corrected. After the frame, with the flicker-corrected newest picture signal as reference, flicker correction is performed one by one. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus such as a digital still camera or a digital video camera, and more particularly to an imaging apparatus capable of simultaneously capturing still images during moving image shooting.

  When moving image shooting is performed using an image pickup apparatus including an XY address type image pickup device under illumination of a fluorescent lamp that is directly turned on by a commercial AC power supply, so-called fluorescent lamp flicker may occur. In order to eliminate this, an imaging apparatus capable of shooting a moving image normally determines an exposure time during moving image shooting by measuring the period of change in luminance of the illumination when the power is turned on.

  For example, when the frequency of a commercial AC power supply is 50 Hz (Hertz), consider a case where moving image shooting is performed under illumination of a non-inverter fluorescent lamp. In this case, the period of the luminance change of illumination is 1/100 second. Then, consider a case where one frame period of a moving image is 1/60 second in accordance with the NTSC (National Television Standards Committee) method (vertical synchronization frequency is 60 Hz).

  As shown in FIG. 12, if the exposure time of each pixel is less than 1/100 second, the exposure amount for the pixels on the same horizontal line varies between frames, and even between different horizontal lines in the same frame. The exposure amount is different. As a result, an image including band-like flicker (line flicker) as shown in the lowermost stage of FIG. 12 is obtained.

  Therefore, the exposure time of each pixel is set to an integral multiple of the period of the luminance change of illumination, for example, 1/100 second as shown in FIG. In the XY address type image pickup device, the exposure timing differs for each horizontal line, but by setting the exposure time of each pixel to 1/100 seconds, for example, the exposure amount becomes constant regardless of the exposure timing. Flicker does not occur.

  Even in the same horizontal line, the exposure timing of each pixel is sequentially shifted, but since the timing difference is sufficiently short compared to the period of the luminance change of the fluorescent lamp, the exposure timing of adjacent pixels on the same horizontal line. Can be considered the same.

  In moving image shooting, a relatively long exposure time (for example, 1/100 second) is set in order to smoothly express the movement of a moving object. As an example of an image taken with such a relatively long exposure time, an image 60 in FIG. 14 is shown. If the exposure time is lengthened, the exposure time becomes longer with respect to the movement of the subject and the subject blur (moving subject blur) becomes larger, so that the motion can be expressed smoothly. At this time, the fluorescent lamp flicker can be eliminated from the image obtained by moving image shooting by appropriately setting the exposure time according to the cycle of the luminance change of the illumination.

  By the way, digital video cameras and the like are often equipped with a function of taking not only moving images but also still images. When shooting a still image, by setting the exposure time to be relatively short (for example, 1/250 seconds), a clear image representing the instantaneous state of the moving subject is acquired. As an example of an image taken with such a relatively short exposure time, an image 61 in FIG. 14 is shown.

  There is no problem when shooting moving images and still images completely separated in time, but in recent years, digital video cameras, etc. have a function to simultaneously record still images according to user operations during moving image shooting. Is required.

  When shooting a still image without stopping the movie shooting (when shooting simultaneously) during movie shooting, the relatively long exposure time set for movie shooting is, for example, temporarily to obtain a clear still image. The exposure time is changed to a relatively short exposure time for still image shooting. In this case, flicker appears in an image of a frame shot with an exposure time shorter than the exposure time for moving image shooting (for example, 1/100 second), as shown in FIG.

  As a conventional technique, there is a technique of detecting a flicker generation period and correcting the flicker based on the detected generation period. However, when such a method is used, it is necessary to acquire several frames of images in order to obtain a correction effect, and in order to correct flicker of all the frames, all of those images (image signals) must be acquired. It must be stored in a frame memory or the like. For this reason, there was a problem that the circuit scale would increase.

JP 2004-222228 A JP 2005-33616 A

  As described above, flicker may occur when a still image is shot during moving image shooting. A method for realizing the flicker correction with a small circuit configuration is desired.

  Accordingly, an object of the present invention is to provide an imaging apparatus capable of correcting flicker that may occur in still image shooting during moving image shooting with a small circuit configuration.

In order to achieve the above object, an image pickup apparatus according to the present invention processes an image pickup element for taking an image and an image signal obtained by taking an image using the image pickup element while varying exposure timing between different horizontal lines. A signal processing unit for capturing a moving image and a still image. When capturing a still image during moving image shooting, a moving image is captured between one or more frames for capturing a still image. Taking an exposure time different from the exposure time for
The signal processing unit includes a reference image signal which is an image signal obtained by photographing using the exposure time for moving image photographing before photographing the still image, and an image signal obtained by photographing the still image photographing frame. Based on a comparison with a certain correction target image signal, according to the flicker component that is included in the correction target image signal due to the adoption of the exposure time different from the exposure time for moving image shooting Flicker correction value calculating means for calculating a flicker correction value, and flicker correction means for correcting the correction target image signal in the direction of reducing the flicker component using the flicker correction value.
It is characterized by

  By comparing the reference image signal and the correction target image signal, the flicker component included in the correction target image signal can be detected, and by correcting the correction target image signal according to the flicker component, the flicker component can be detected. Can be reduced. A circuit for correcting flicker is used in order to reduce flicker through a process of comparing a reference image signal and a correction target image signal, instead of using a technique that increases the circuit scale such as detecting a flicker occurrence period. Can be expected to be smaller.

  Specifically, for example, the still image shooting frame includes first, second,..., And nth frames (n is an integer equal to or greater than 2), and the correction target image signal includes first, second, and second frames. .., And the nth correction target image signal corresponding to the nth frame, the flicker correction value calculating means includes the reference image signal and the first image signal. A first flicker correction value is calculated as the flicker correction value from the comparison with the correction target image signal, and the flicker correction means corrects the first correction target image signal using the first flicker correction value, and the flicker. The correction value calculation means calculates a second flicker correction value as the flicker correction value from the comparison between the corrected first image signal and the second correction target image signal, and the flicker correction means 2 Use flicker correction value The flicker correction value calculation means is configured to compare the corrected (n-1) th correction target image signal with the nth correction target image signal. An nth flicker correction value is calculated as the flicker correction value, and the flicker correction means corrects the nth correction target image signal using the nth flicker correction value.

  First, the first correction target signal is corrected based on a comparison between “a reference image signal using an exposure time for moving image shooting with sufficiently small flicker components included” and “first correction target image signal including flicker components”. To do. Next, the second correction target signal is corrected based on a comparison between the “first corrected target image signal after the flicker component is reduced” and the “second corrected target image signal including the flicker component”. In this way, by sequentially changing the image signal serving as a correction reference, it is possible to keep the memory area for flicker correction very small.

  More specifically, for example, a plurality of detection areas are provided in the imaging area of the imaging device, each detection area includes pixels on different horizontal lines, and each of the first flicker correction values is 1 or 2 or more The second flicker correction value is composed of a plurality of second element correction values each corresponding to one or more horizontal lines,..., The nth flicker correction value is composed of a plurality of nth element correction values each corresponding to one or more horizontal lines, and the flicker correction value calculation means is a signal corresponding to each detection area in the reference image signal. And a signal corresponding to each detection region in the first correction target image signal, by detecting the flicker component included in the first correction target image signal for each detection region, each first element correction The value From the comparison between the signal corresponding to each detection region in the first correction target image signal after being output and corrected and the signal corresponding to each detection region in the second correction target image signal, the second signal is detected for each detection region. By detecting the flicker component included in the correction target image signal, each second element correction value is calculated,... In each detection region in the corrected (n-1) correction target image signal. By comparing the corresponding signal and the signal corresponding to each detection area in the n-th correction target image signal, the flicker component included in the n-th correction target image signal is detected for each detection area, thereby An n element correction value is calculated.

  More specifically, for example, a recording unit that records information necessary for calculating the first, second,..., And nth flicker correction values is provided, and the recording unit includes the first flicker. Information for calculating the correction value, information for calculating the second flicker correction value,..., And information for calculating the nth flicker correction value are recorded while being sequentially updated.

  More specifically, for example, it comprises recording means for recording information necessary for calculating the first, second,..., And nth flicker correction values, and the flicker correction value calculating means includes the reference A first flicker correction value is calculated from a comparison between the comparison target value calculated from the image signal and the comparison target value calculated from the first correction target image signal, and is calculated from the corrected first correction target image signal. The second flicker correction value is calculated from the comparison between the comparison target value and the comparison target value calculated from the second correction target image signal, and the corrected (n-1) th correction target image signal. The n th flicker correction value is calculated from a comparison between the comparison target value calculated from the above and the comparison target value calculated from the n th correction target image signal, and the recording means calculates from the reference image signal. Comparison target value, corrected first The comparison target value calculated from the correction target image signal, and the comparison target value calculated from the corrected (n-1) th correction target image signal are recorded while being sequentially updated as the information. The

  As a result, the memory area required for flicker correction can be greatly reduced. In the embodiment described later, AVE_PRE [m: n] corresponds to the comparison target value.

  Further, for example, the still image shooting frame includes first, second,..., And nth frames (n is an integer of 2 or more), and the correction target image signal is the first, second,. When the image signal includes first, second,..., And nth correction target image signals corresponding to the nth frame, the flicker correction value calculation means is configured to output the reference image signal and the first and second image signals. ,..., And the nth correction target image signal, the first, second,..., And nth flicker correction values are calculated as the flicker correction values, respectively. The first, second,..., And nth flicker correction values are corrected using the first, second,..., And nth flicker correction values, respectively.

  Further, for example, when the still image shooting frame is composed of one frame, the flicker correction value calculation means calculates the flicker correction value from a comparison between the reference image signal and the correction target image signal.

  For example, a plurality of detection areas are provided in the image pickup area of the image pickup element, each detection area includes pixels on different horizontal lines, and the flicker correction value is a plurality corresponding to one or more horizontal lines. The flicker correction value calculation means comprises a detection region based on a comparison between a signal corresponding to each detection region in the reference image signal and a signal corresponding to each detection region in the correction target image signal. Each element correction value is calculated by detecting the flicker component included in the correction target image signal every time.

  More specifically, for example, the image sensor is an XY address scanning type image sensor.

  As described above, according to the present invention, flicker that may occur in still image shooting during moving image shooting can be corrected with a small circuit configuration. For example, it is possible to reduce the amount of frame memory used for flicker correction, or it is possible to perform flicker correction without using a frame memory.

  Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. In the drawings to be referred to, the same components are denoted by the same reference numerals. FIG. 1 is an overall block diagram of an imaging apparatus 1 according to an embodiment of the present invention. The imaging device 1 is, for example, a digital still camera or a digital video camera.

  The imaging apparatus 1 includes an optical system 11, an aperture 12, an imaging device 13, a preprocessing unit 14, a driver 17, a timing generator 18, a main control unit 19, a signal processing unit 20, and a frame memory buffer 40. And an external recording medium 41 and an operation unit 42. The imaging device 1 is an imaging device that can simultaneously capture both moving images and still images.

  The optical system 11 is composed of, for example, a plurality of lenses, and incident light from an imaging target is incident on the image sensor 13 via the lens and the diaphragm 12. The driver 17 controls the optical system 11 based on a control signal from the main control unit 19 and controls the zoom magnification and focal length of the optical system 11. The driver 17 controls the opening amount (size of the opening) of the diaphragm 12 based on a control signal from the main control unit 19. When the optical images incident on the optical system 11 are the same, the amount of incident light per unit time on the image sensor 13 increases as the aperture of the diaphragm 12 increases.

  The imaging device 13 is an XY address scanning type imaging device, and is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor. By forming a plurality of pixels arranged in a two-dimensional matrix, a vertical scanning circuit, a horizontal scanning circuit, and a video signal output circuit on a semiconductor substrate capable of forming a CMOS, an example of the image sensor 13 is provided. A CMOS image sensor is formed. Each pixel includes a photodiode, a transfer gate, a selection gate, an amplification transistor, a reset gate, and the like.

  The image sensor 13 photoelectrically converts an optical image incident through the optical system 11 and the diaphragm 12 and outputs an electrical signal obtained by the photoelectric conversion to the preprocessing unit 14. More specifically, in each shooting, each pixel stores a signal charge having a charge amount corresponding to the exposure time. Each pixel sequentially outputs an electrical signal having a magnitude proportional to the amount of stored signal charge to the pre-processing unit 14 at the subsequent stage. When the optical images incident on the optical system 11 are the same and the aperture amount of the diaphragm 12 is the same, the magnitude (intensity) of the electrical signal from the image sensor 13 (each pixel) increases in proportion to the exposure time. To do.

  The image sensor 13 is a single-plate image sensor capable of color photography. Each pixel constituting the image sensor 13 is provided with, for example, one of red (R), green (G), and blue (B) color filters (not shown). Note that a three-plate image sensor can be adopted as the image sensor 13.

  The image sensor 13 has an electronic shutter function, and the exposure time is controlled by an electronic shutter control signal from the timing generator 18. Since the main control unit 19 controls the operation of the timing generator 18, the main control unit 19 and the timing generator 18 function as an exposure time control unit that controls the exposure time. The shutter speed in the electronic shutter control is specified by the exposure time. In the same frame, the exposure time (length of exposure time) for each pixel constituting the image sensor 13 is the same.

  The preprocessing unit 14 includes an amplifier circuit 15 and an A / D converter (analog-digital converter) 16. The amplifier circuit 15 amplifies an analog signal, which is an output signal of the image sensor 13, with a specified amplification level and outputs the amplified signal. The A / D converter 16 converts the output signal of the amplifier circuit 15 into a digital signal according to the signal from the timing generator 18. The preprocessing unit 14 performs further appropriate processing on the digital signal as necessary, and outputs an image signal representing the captured image. The image signal output from the preprocessing unit 14 is given to the signal processing unit 20.

  The timing generator 18 controls the timing of each operation of the image sensor 13 and the preprocessing unit 14 in accordance with the shooting timing. Specifically, for example, the timing for reading the output signal from the image sensor 13 (the timing for giving the output signal from the image sensor 13 to the preprocessing unit 14) and the output timing of the signal from the preprocessing unit 14 are controlled.

  The signal processing unit 20 includes a pixel number conversion unit 21, a motion detection unit 22, a moving image processing unit 23, a moving image compression unit 24, a still image processing unit 25, a still image compression unit 26, and a flicker detection unit (flicker detection unit). A correction value calculation unit) 27 and a flicker correction unit 28. The image signal output from the preprocessing unit 14 is supplied to the flicker detection unit 27 and the flicker correction unit 28, and is also supplied to the pixel number conversion unit 21, the motion detection unit 22, and the still image processing unit 25 via the flicker correction unit 28. It is done. Hereinafter, the image signal (digital image data) output from the preprocessing unit 14 is particularly referred to as an original image signal.

  The flicker detection unit 27 and the flicker correction unit 28 are parts that function effectively when a still image is simultaneously shot during moving image shooting, and the flicker correction unit 28 outputs an image signal obtained by correcting the original image signal. To the pixel number conversion unit 21, the motion detection unit 22, and the still image processing unit 25. The input / output signals of the flicker correction unit 28 are the same during continuous movie shooting or when shooting still images only.

  The pixel number conversion unit 21 reduces the image size of the image signal by subjecting the image signal sent from the flicker correction unit 28 to cutout processing, filter processing, thinning-out processing, and the like. The image signal obtained by this reduction is given to the moving image processing unit 23. The moving image processing unit 23 performs necessary video processing on the image signal from the pixel number conversion unit 21. This video processing includes, for example, generation of luminance signals and color difference signals, gamma correction, contour enhancement processing, and the like. The moving image processing unit 23 outputs the image signal obtained through this video processing to the moving image compression unit 24. The moving image compression unit 24 compresses the image signal from the moving image processing unit 23 using a predetermined compression method such as MPEG (Moving Picture Experts Group) and outputs the compressed image signal to the external recording medium 41.

  The still image processing unit 25 performs necessary video processing on the image signal sent from the flicker correction unit 28. This video processing includes, for example, generation of luminance signals and color difference signals, gamma correction, contour enhancement processing, and the like. The still image processing unit 25 outputs an image signal obtained through this video processing to the still image compression unit 26. The still image compression unit 26 compresses the image signal from the still image processing unit 25 using a predetermined compression method such as JPEG (Joint Photographic Experts Group), and outputs the compressed image signal to the external recording medium 41. To do.

  The motion detection unit 22 detects a motion vector of the subject using a known representative point matching method or the like based on image signals sequentially transmitted during moving image shooting. During moving image shooting, the pixel number conversion unit 21 and the moving image processing unit 23 refer to the motion vector detected by the motion detection unit 22 and perform necessary processing (for example, so-called camera shake correction).

  The frame memory buffer 40 is composed of a semiconductor memory such as a DRAM (Dynamic Random Access Memory) and can read and write data. Each part constituting the signal processing unit 20 writes data to be temporarily recorded in the frame memory buffer 40 and reads data to be referred to from the frame memory buffer 40. The external recording medium 41 is composed of one or a plurality of recording media, and each recording medium is, for example, a semiconductor memory, a memory card, a magnetic tape, or the like.

  The main control unit 19 includes a CPU (Central Processing Unit) and the like, and comprehensively controls the operation of each part in the imaging apparatus 1. When the user operates the operation unit 42 composed of input keys (not shown) or the like, a signal representing the operation content is transmitted to the main control unit 19. The main control unit 19 controls the driver 17, the timing generator 18, the signal processing unit 20, and the like in accordance with the signal, so that still image shooting and recording and moving image shooting and recording are performed.

  When performing moving image shooting in response to an operation on the operation unit 42, the main control unit 19 controls the driver 17 and the timing generator 18 to set the exposure setting of the image sensor 13 (exposure setting of the imaging device 1) for moving image shooting. Make it. The exposure setting for moving image shooting is an exposure setting suitable for moving image shooting. During moving image shooting, original image signals are output from the preprocessing unit 14 one after another at a predetermined frame rate (that is, at predetermined intervals). These original image signals representing the captured moving image are sequentially sent to the external recording medium 41 through the predetermined processing of the pixel number conversion unit 21, the moving image processing unit 23, and the moving image compression unit 24, and are sent to the external recording medium 41. To be recorded. As a result, the moving image is shot and the shot moving image is recorded.

  When taking a still image in response to an operation on the operation unit 42, the main control unit 19 controls the driver 17 and the timing generator 18 to set the exposure setting of the image sensor 13 (exposure setting of the imaging device 1). Make it for. The exposure setting for still image shooting is an exposure setting suitable for still image shooting. The original image signal obtained by shooting the still image is sent to the external recording medium 41 through predetermined processing of the still image processing unit 25 and the still image compression unit 26 and is recorded on the external recording medium 41. As a result, a still image is shot and a recorded still image is recorded.

  The “exposure setting” includes settings such as the aperture amount of the diaphragm 12 and the exposure time (length of exposure time).

  Consider a case where moving image shooting is performed under illumination of a non-inverter type fluorescent lamp when the frequency of a commercial AC power supply is 50 Hz (Hertz). In this case, the period of the luminance change of illumination is 1/100 second. Then, consider a case where one frame period of a moving image is 1/60 second in accordance with the NTSC (National Television Standards Committee) method (vertical synchronization frequency is 60 Hz).

  Since the image sensor 13 is an XY address scanning type image sensor, the exposure timing between different horizontal lines is different. Therefore, as described with reference to FIG. 12, when the exposure time of each pixel is less than 1/100 second, the exposure amount for pixels on the same horizontal line varies between frames, and even in the same frame. The amount of exposure differs between different horizontal lines.

  For this reason, the exposure time for moving image shooting is set to an integral multiple of the cycle of the luminance change of illumination. This eliminates the fluorescent flicker during continuous moving image shooting. Even in the same horizontal line, the exposure timing of each pixel is sequentially shifted, but since the timing difference is sufficiently short compared to the period of the luminance change of the fluorescent lamp, the exposure timing of adjacent pixels on the same horizontal line. Can be considered the same (ie, ignoring fluorescent flicker on the same horizontal line of the same frame). Further, the period of change in luminance of illumination is measured using a known technique when the power of the imaging apparatus 1 is turned on, and the exposure time for moving image shooting is determined in advance based on the measurement result. And

  In moving image shooting, it is desirable to perform shooting with a relatively long exposure time in order to smoothly reproduce the movement of the subject. On the other hand, the exposure time for still image shooting is set to be relatively short so as to obtain a clear still image. When the exposure time is sufficiently short relative to the movement of the subject, a clear image without so-called “subject blur” (moving subject blur) can be obtained as shown in an image 61 of FIG.

Hereinafter, the exposure time for the moving image photographing and the exposure time for still image shooting, respectively, indicated by "T _A" and "T _B", the exposure time T _A and T _B, respectively, 1/100 seconds and 1 / 250 seconds.

  With reference to FIG. 2, an outline operation in the case where a still image is simultaneously shot and recorded during moving image shooting and recording will be described. Movie shooting is performed in the order of frames FL1, FL2, FL3, FL4, FL5, FL6,.

Assume that during moving image shooting, one still image is shot during a period corresponding to the frame FL4 in response to a still image shooting operation on the operation unit 42. Then, assume that the exposure times in the three frames FL3, FL4, and FL5 are different from T _A for photographing this still image. Before and after the frame FL1, FL2 and FL6, exposure time in ... is the T _A.

When performing still image shooting, while a relatively short T _B of exposure time, the opening amount of the order diaphragm 12 to compensate for the lack of exposure by shortening the exposure time will be greater than that for the moving image, the diaphragm Since the change in the opening amount of 12 is accompanied by a mechanical operation, a certain amount of time is required. Therefore, different from the T _A also exposure time in the frame FL3 and FL5.

Specifically, the opening amounts of the frames FL1, FL2, and FL6 are D _A for moving image shooting, the opening amount of the frame FL4 is D _B for still image shooting, and the opening amounts of the frames FL3 and FL5 are D _C1 and When D _C2 is satisfied, D _A <D _C1 <D _B and D _A <D _C2 <D _B are satisfied. The identity of D _C1 and D _C2 does not matter. The exposure time for the frame FL4 is T _B. When the exposure times of the frames FL3 and FL5 are T _C1 and T _C2 , T _A > T _C1 > T _B and T _A > T _C2 > T _B are satisfied. The identity of T _C1 and T _C2 does not matter.

  By adjusting the exposure time and the opening amount of the diaphragm 12 as described above, the brightness of the images in FL1 to FL6 (for example, the average luminance of the entire screen) is made uniform. Instead of adjusting the aperture amount of the diaphragm 12, the amplification degree of the amplifier circuit 15 may be adjusted, thereby making the image brightness in the FL1 to FL6 uniform. Further, the exposure time, the opening amount of the diaphragm 12 and the amplification degree of the amplifier circuit 15 may all be adjusted, thereby making the image brightness in the frames FL1 to FL6 uniform.

Each original image signal obtained by photographing in the frames FL1 to FL6 is sequentially sent to the flicker detection unit 27 and the flicker correction unit 28. The original image signals obtained by photographing in the frames FL1, FL2, FL3, FL4, FL5 and FL6 are assumed to be I _O1 , I _O2 , I _O3 , I _O4 , I _O5 and I _O6 , respectively. Flicker does not appear in the images represented by the original image signals I _O1 , I _O2 and I _O6 , but due to the adoption of an exposure time shorter than T _A , the original image signals I _O3 , I _O4 and I _O5 Flicker appears in the displayed image.

The flicker detection unit 27 detects flicker components included in the original image signals I _O3 , I _O4 and I _O5 , and calculates a flicker correction value for canceling the flicker components. The flicker correction unit 28 corrects the original image signals I _O3 , I _O4, and I _O5 using the calculated flicker correction values to generate and output corrected image signals I _R3 , I _R4, and I _R5 , respectively. To do. The flicker of the image represented by the corrected image signals I _R3 , I _R4 and I _R5 is reduced (ideally completely eliminated). Note that the original image signals I _O1 , I _O2, and I _O6 are output to the _subsequent stage (the pixel number conversion unit 21 and the like) as they are.

The original image signals I _O1 , I _O2 and I _O6 corresponding to the frames FL1, FL2 and FL6 and the corrected image signals I _R3 , I _R4 and I _R5 corresponding to the frames FL3, FL4 and FL5 The image is sent to the still image processing unit 25 (and the motion detection unit 22). These (I _O1 , I _O2 and I _O6 and I _R3 , I _R4 and I _R5 ) are subjected to predetermined processing by the moving image processing unit 23 and the moving image compression unit 24 and are externally recorded as image signals (image data) representing moving images. Recorded on the medium 41. On the other hand, the corrected image signal I _R4 is recorded on the external recording medium 41 as an image signal (image data) representing a still image through predetermined processing by the still image processing unit 25 and the still image compression unit 26.

  Next, a method for calculating a flicker correction value by the flicker detection unit 27 and flicker correction by the flicker correction unit 28 using the same will be described in detail.

  FIG. 3 is a schematic diagram of an imaging region of the imaging device 13. As shown in FIG. 3, the imaging region of the imaging device 13 is considered to be divided into M in the vertical direction and N in the horizontal direction. Thereby, the imaging area of the imaging device 13 is divided into M × N areas. M and N are each an integer of 2 or more.

The region divided into M × N is regarded as a matrix of M rows and N columns. When considering the origin X of the imaging region of the imaging device 13 as a reference,
The first line includes regions AR [1: 1], AR [1: 2],..., AR [1: N−1] and AR [1: N]. The areas AR [2: 1], AR [2: 2],..., AR [2: N-1] and AR [2: N] are included, and the (M-1) th line. Includes areas AR [M-1: 1], AR [M-1: 2], ..., AR [M-1: N-1] and AR [M-1: N], and M rows The eye includes regions AR [M: 1], AR [M: 2],..., AR [M: N−1] and AR [M: N].

  The first column includes areas AR [1: 1], AR [2: 1],..., AR [M−1: 1] and AR [M: 1]. Includes areas AR [1: 2], AR [2: 2], ..., AR [M-1: 2] and AR [M: 2], ..., (N-1) columns The eye includes regions AR [1: N-1], AR [2: N-1], ..., AR [M-1: N-1] and AR [M: N-1], The Nth column includes areas AR [1: N], AR [2: N],..., AR [M−1: N], and AR [M: N].

  Regions belonging to the same row include pixels on the same horizontal line. Regions belonging to different rows include pixels on different horizontal lines. Regions belonging to the same column include pixels on the same vertical line. Regions belonging to different columns include pixels on different vertical lines.

  For example, the area AR [1: 1] is composed of pixels (in this case, 30 pixels in total) on the first to fifth horizontal lines and on the first to sixth vertical lines, and the area AR [ 1: 2] is composed of pixels (in this case, 30 pixels in total) on the first to fifth horizontal lines and on the seventh to twelfth vertical lines, and the area AR [2: 1] , Pixels on the sixth to tenth horizontal lines and on the first to sixth vertical lines (in this case, 30 pixels in total).

  In addition, you may make it an adjacent area | region contain the same pixel. For example, the area AR [1: 1] includes pixels (in this case, a total of 30 pixels) on the first to fifth horizontal lines and on the first to sixth vertical lines, and the area AR [ 1: 2] is composed of pixels (in this case, 30 pixels in total) on the first to fifth horizontal lines and on the fifth to ninth vertical lines, and the area AR [2: 1] The pixels may be composed of pixels on the fourth to eighth horizontal lines and on the first to sixth vertical lines (in this case, 30 pixels in total).

The flicker detection unit 27 and the flicker correction unit 28 perform flicker correction under the following assumptions.
First, between adjacent frames, the movement of the subject and the accompanying luminance change are slight (or can be ignored).
Second, the image signal after correction by the flicker correction unit 28 (almost) contains no flicker component.

Under the above assumption, as shown in FIG. 4, the flicker detection unit 27 first uses the original image signal I _O2 that does not include the flicker component as the first image signal that is the correction reference, and includes the flicker component. the original image signal I _O3 which have a second image signal which is the target of correction, flicker correction on the second image signal from the comparison of the first image signal (I _O2) and the second image signal (I _O3) (I _O3) The value α is calculated. Then, the flicker correction unit 28 corrects the original image signal I _O3 as the second image signal using the flicker correction value α, and generates a corrected image signal I _R3 .

Next, the flicker detection unit 27 uses the corrected image signal I _{R3 from} which the flicker component is removed as the first image signal, the original image signal I _O4 including the flicker component as the second image signal, and the first image signal. A flicker correction value β for the second image signal (I _O4 ) is calculated from a comparison between (I _R3 ) and the second image signal (I _O4 ). Then, the flicker correction unit 28 corrects the original image signal I _O4 as the second image signal using the flicker correction value β, and generates a corrected image signal I _R4 .

Next, the flicker detection unit 27 uses the corrected image signal I _{R4 from} which the flicker component is removed as the first image signal, the original image signal I _O5 including the flicker component as the second image signal, and the first image. A flicker correction value γ for the second image signal (I _O5 ) is calculated from a comparison between the signal (I _R4 ) and the second image signal (I _O5 ). Then, the flicker correction unit 28 corrects the original image signal I _O5 as the second image signal using the flicker correction value γ to generate a corrected image signal I _R5 .

  As an example of the flicker correction method along the above flow, a first correction method and a second correction method are illustrated.

[First correction method]
First, the first correction method will be described. FIG. 5 is a flowchart for explaining the first correction method.

In step S1, first, the flicker detection unit 27 refers to the first image signal and the second image signal. The first and second image signals are specified as described above. For the sake of concreteness of explanation, assume that the first and second image signals are the original image signals I _O2 and I _O3 , respectively, as an example.

And the average value of the signal level in each area | region in the 1st image signal which is an image signal of the previous frame (previous frame) is calculated. Specifically, focusing on the first image signal (I _O2 ), the average value of the signal levels of the pixels constituting the area AR [m: n] is calculated for each area AR [m: n] (step S1). ). A total of M × N average values is calculated. The average value calculated for the area AR [m: n] of the first image signal is denoted as AVE_PRE [m: n]. m is an arbitrary integer of 1 to M, and n is an arbitrary integer of 1 to N.

  For example, the signal level is a value of a luminance signal representing luminance. In this case, the original image signal includes a luminance signal representing the luminance of each pixel. For example, when the area AR [1: 1] is composed of 30 pixels, AVE_PRE [1: 1] is an average value of 30 luminance signal values, which is within the area AR [1: 1]. Represents the average luminance of a pixel. As is well known, for example, the luminance signal for a certain pixel is calculated in consideration of pixel values around that pixel.

Similarly, the average value of the signal levels in each region in the second image signal that is the image signal of the current frame (current frame) is calculated. Specifically, focusing on the second image signal (I _O3 ), the average value of the signal levels of the pixels constituting the area AR [m: n] is calculated for each area AR [m: n] (step S1). ). A total of M × N average values is calculated. The average value calculated for the area AR [m: n] of the second image signal is denoted as AVE_NOW [m: n].

In step S2 following step S1, the ratio of the average value obtained in step S1 (expressed as RATIO [m: n]) is calculated for each region between the previous frame and the current frame. Specifically, each ratio RATIO [m: n] is calculated based on the following formula (1). A total of M × N ratios are calculated.
RATIO [m: n] = AVE_PRE [m: n] / AVE_NOW [m: n]
... (1)

  In step S3 following step S2, an average value for each row of the ratio obtained in step S2 is obtained. The average value of the ratios (RATIO [m: 1] to RATIO [m: n]) for the m-th row is expressed as AVE_RATIO [m]. Specifically, each average value AVE_RATIO [m] is calculated based on the following formula (2). A total of M average values is calculated.

Assuming that the optical image incident on the image sensor 13 is not changed between the previous frame and the current frame, for example, AVE_RATIO [1] is a horizontal line (in the center of the area [1: n] belonging to the first row ( FIG. 6 shows a signal level ratio between frames due to the flicker component with respect to 70) in FIG. In the second image signal (I _O3 ) containing the flicker component, the flicker component is obtained by multiplying the signal level of the pixel on the horizontal line (reference numeral 70 in FIG. 6) located at the center by AVE_RATIO [1]. Can be countered.

  In step S4 following step S3, a correction gain (signal level correction gain) is calculated for each horizontal line based on AVE_RATIO [m] obtained in step S3.

  FIG. 7 shows a plot of AVE_RATIO [m]. Consider a case where each area AR [m: n] includes pixels on a plurality of horizontal lines. In this case, in each region, the correction gain to be adopted for the horizontal line located at the center is different from the correction gain to be adopted for the horizontal line not located at the center.

  Therefore, for example, the correction gain (element correction value) for each horizontal line of the image sensor 13 is calculated by linearly interpolating AVE_RATIO [m] in the vertical line direction.

  Specifically, first, AVE_RATIO [m] is set as a correction gain for a horizontal line located at the center of the area AR [m: n] belonging to the m-th row. For example, AVE_RATIO [1] is a correction gain (hereinafter referred to as correction gain G1) for a horizontal line (hereinafter referred to as horizontal line L1) located in the center of the area AR [1: n] belonging to the first row. Let AVE_RATIO [2] be a correction gain (hereinafter referred to as correction gain G2) for a horizontal line (hereinafter referred to as horizontal line L2) located in the center of the area AR [2: n] belonging to the second row.

Then, each correction gain for the horizontal line between the horizontal lines L1 and L2 is calculated by linearly interpolating the correction gains G1 and G2 in the vertical line direction. For example, horizontal lines between the horizontal lines L1 and L2, adjacent to the horizontal line L1 _A and L2 A two _A with and horizontal lines L1 _A horizontal line L1, the horizontal line L2 _A adjacent to the horizontal line L2 If you, the correction gain in the horizontal lines L1 _a is a "(G2-G1) / 3 + G1 ", the correction gain in the horizontal line L2 _a is set to "2 (G2-G1) / 3 + G1 ".

  Correction gains for other horizontal lines are calculated in the same manner. In FIG. 7, the curve denoted by reference numeral 71 is a curve representing each correction gain calculated using this linear interpolation.

  In addition, although the example which calculates a correction gain for every horizontal line by linearly interpolating AVE_RATIO [m] was shown, it is also possible to employ | adopt various interpolation methods other than linear interpolation as an interpolation method. For example, when calculating a correction gain for a horizontal line between a horizontal line located at the center of the area AR [m: n] and a horizontal line located at the center of the area AR [m + 1: n] by interpolation, AVE_RATIO [ In addition to m] and AVE_RATIO [m + 1], AVE_RATIO [m-1], AVE_RATIO [m + 2], and the like may be used, and interpolation using an approximate curve may be used.

The processing in steps S1 to S4 is performed by the flicker detection unit 27. As described above, the total k correction gains (element correction values) calculated for each horizontal line constitute a flicker correction value (α) for the second image signal (I _O3 ). However, k represents the number of horizontal lines of the image sensor 13.

In step S5 following step S4, the flicker correction unit 28 corrects the second image signal (I _O3 ) using each correction gain calculated in step S4. Specifically, in the second image signal (I _O3 ), each signal level (for example, luminance) for the pixels on the first, second,..., (K−1) th and kth horizontal lines. Signal value) is multiplied by correction gains corresponding to the first, second,..., (K−1) th, and kth horizontal lines, respectively, calculated in step S4. As a result, a corrected image signal (I _R3 ) in which the flicker component is excluded from the second image signal (I _O3 ) is generated.

As described with reference to FIG. 4, the corrected image signals I _R4 and I _R5 are also generated by repeating the processes of steps S1 to S5 while sequentially exchanging the first and second image signals.

  Although an example in which correction gains (a total of k correction gains) are calculated for each horizontal line has been shown, one correction gain may be assigned to a plurality of horizontal lines. For example, the same correction gain is assigned to every two horizontal lines. In this case, the total number of correction gains calculated in step S4 is k / 2, and the flicker correction value (α) is configured by k / 2 correction gains (element correction values).

In this case, in step S5, the same signal level (for example, the value of the luminance signal) for the pixels on the first and second horizontal lines corresponds to the first and second horizontal lines. Multiply by the correction gain. The same applies to the other horizontal lines, and the corrected image signal (I _R3 ) is generated.

  Further, although the contents are included in the technique of “assigning one correction gain to a plurality of horizontal lines”, if the number of horizontal lines included in each area AR [m: n] is sufficiently small, the step is performed. It is also possible to omit the interpolation process in S4. In this case, AVE_RATIO [m] is set as a correction gain for all horizontal lines included in the area AR [m: n].

  Further, although the case where each area AR [m: n] includes pixels on a plurality of horizontal lines has been described above, the case where each area AR [m: n] includes only pixels on one horizontal line, that is, imaging. When the number of horizontal lines of the element 13 is M, AVE_RATIO [m] itself can be a correction gain corresponding to the mth horizontal line (the area AR [m: n] is the mth). Only consist of pixels on the horizontal line). In this case, no interpolation process is required when calculating each correction gain. However, in order to be easily affected by noise or the like, it is preferable to finally adopt a numerical value group composed of AVE_RATIO [m] spatially filtered in the vertical line direction as each correction gain.

  The above-described flicker correction method is proposed under the assumption that the movement of the subject and the accompanying luminance change are slight between adjacent frames, and the actual image is not within the assumption. There is also. Considering this, for example, when determining the average value AVE_RATIO [m] for each row of RATIO [m: n] obtained in step S2 in step S3, RATIO [m: n] is determined according to a predetermined rule. RATIO [m: n] determined to be invalid may be ignored in calculating the average value.

  For example, only the row with m = 1 will be noted and described. Assume N = 8. In this case, eight ratios (RATIO [1: 1] to RATIO [1: 8]) are calculated in step S2. These eight ratios are R1, R1, R3, R4, R5, R6, R7 and R8, respectively.

  Then, for example, considering the variance of R1 to R8, a ratio that deviates more than a predetermined threshold from the average value of the eight ratios (R1 to R8) is determined to be largely influenced by noise and is invalidated. For example, when it is determined that R1 and R2 are invalid, AVE_RATIO [1] = (RATIO [1: 3] + RATIO [1: 4] + RATIO [1: 5] + RATIO [1: 6] + RATIO [1: 7] + RATIO [1: 7] 1: 8]) / 6 to calculate AVE_RATIO [1].

  It is also possible to set N = 1. In this case, the imaging area of the imaging element 13 is divided into a total of M areas in the horizontal line direction, as shown in FIG. Further, as shown in FIG. 9, the imaging area of the imaging device 13 may be divided into two in the vertical direction, and one imaging area obtained by the division may be divided into a total of M areas in the horizontal line direction. . However, in consideration of the advantages of the above-described processing based on the validity / invalidity determination of RATIO [m: n], it is desirable to set N to 2 or more.

[Second correction method]
Next, the second correction method of the flicker correction method will be described. Note that the items described in the first correction method can be applied to the second correction method as appropriate. FIG. 10 is a flowchart for explaining the second correction method. The processing of steps S11 to S14 in FIG. 10 is performed by the flicker detection unit 27, and the processing of step S15 is performed by the flicker correction unit 28.

  In the second correction method, the same processing as in the first correction method is performed for each color of the color filter provided in the pixel. As an example, a case where any one of red (R), green (G), and blue (B) color filters (not shown) is arranged in a so-called Bayer array in each pixel constituting the image sensor 13 is considered. . FIG. 11 shows the arrangement of color filters in a certain area AR [m: n]. Each area AR [m: n] includes a plurality of pixels provided with a red color filter (hereinafter referred to as R pixels), and a plurality of pixels provided with a green color filter (hereinafter referred to as G pixels). And a plurality of pixels (hereinafter referred to as B pixels) provided with a blue color filter.

  The original image signal supplied to the signal processing unit 20 includes an R signal value (R signal level) proportional to the signal charge stored in the R pixel and a G signal value proportional to the signal charge stored in the G pixel. (G signal level) and B signal value (B signal level) proportional to the signal charge stored in the B pixel are included.

First, in step S11 (see FIG. 10), the flicker detection unit 27 refers to the first image signal and the second image signal. For the sake of concreteness of explanation, assume that the first and second image signals are the original image signals I _O2 and I _O3 , respectively, as an example.

Then, in the first image signal, an average value of R signal values, an average value of G signal values, and an average value of B signal values are calculated for each region AR [m: n] (step S11). Specifically, focusing on the first image signal (I _O2 ), for each area AR [m: n], the average value AVE_PRE_R [m: R signal value of R pixels constituting the area AR [m: n]. n], average value AVE_PRE_G [m: n] of G signal values of G pixels constituting the area AR [m: n], and average value of B signal values of the B pixels constituting the area AR [m: n] AVE_PRE_B [m: n] is calculated.

Similarly, in the second image signal, an average value of R signal values, an average value of G signal values, and an average value of B signal values are calculated for each area AR [m: n] (step S11). Specifically, focusing on the second image signal (I _O3 ), an average value AVE_NOW_R [m: R signal value of R pixels constituting the area AR [m: n] for each area AR [m: n]. n], average value AVE_NOW_G [m: n] of G signal values of G pixels constituting the area AR [m: n], and average value of B signal values of B pixels constituting the area AR [m: n] AVE_NOW_B [m: n] is calculated.

In step S12 following step S11, the ratio of the average values obtained in step S11 is calculated for each region and for each color. Specifically, RATIO_R [m: n], RATIO_G [m: n], and RATIO_B [m: n] are calculated based on the following formulas (3R), (3G), and (3B).
RATIO_R [m: n]
= AVE_PRE_R [m: n] / AVE_NOW_R [m: n] (3R)
RATIO_G [m: n]
= AVE_PRE_G [m: n] / AVE_NOW_G [m: n] (3G)
RATIO_B [m: n]
= AVE_PRE_B [m: n] / AVE_NOW_B [m: n] (3B)

  In step S13 following step S12, the average values AVE_RATIO_R [m], AVE_RATIO_G [m], and AVE_RATIO_B [m] are calculated based on the following formulas (4R), (4G), and (4B) using the ratio obtained in step S12. Is calculated.

  In step S14 following step S13, processing similar to that in step S4 in FIG. 5 is performed. That is, in step S14, a correction gain (signal level correction gain) is calculated for each horizontal line and for each color using linear interpolation based on the average value AVE_RATIO_R [m] obtained in step S13.

  AVE_RATIO_R [m] is an R correction gain for a horizontal line located at the center of the area AR [m: n] belonging to the m-th row. Then, R correction gain (element correction value) for each horizontal line of the image sensor 13 is calculated by linearly interpolating AVE_RATIO_R [m] in the vertical line direction. This linear interpolation method is the same as that in step S4 of FIG.

  Similarly, AVE_RATIO_G [m] is a G correction gain for a horizontal line located at the center of the area AR [m: n] belonging to the m-th row. Then, G correction gain (element correction value) for each horizontal line of the image sensor 13 is calculated by linearly interpolating AVE_RATIO_G [m] in the vertical line direction.

  Similarly, AVE_RATIO_B [m] is a B correction gain for the horizontal line located at the center of the area AR [m: n] belonging to the m-th row. Then, B correction gain (element correction value) for each horizontal line of the image sensor 13 is calculated by linearly interpolating AVE_RATIO_G [m] in the vertical line direction.

  However, in the case of the Bayer array, since only two color filters exist in one horizontal line, the calculation of the correction gain for the horizontal line in which no corresponding color pixel exists is omitted. For example, the calculation of the B correction gain for a horizontal line in which only R and G pixels exist is omitted. It is assumed that the R pixel exists only on the odd-numbered horizontal line and the B pixel exists only on the even-numbered horizontal line. The number k of horizontal lines is an even number.

In step S15 subsequent to step S14, the flicker correction unit 28 corrects the second image signal (I _O3 ) for each color using each correction gain calculated in step S14.

Specifically, each R for R pixels on the first, third,..., (K-3), (k-1) th horizontal lines in the second image signal (I _O3 ). The signal value is multiplied by the R correction gain corresponding to the first, third,..., (K-3) th, and (k-1) th horizontal lines calculated in step S14.
In the second image signal (I _O3 ), the G signal values for the G pixels on the first, second,..., (K−1) th and kth horizontal lines are respectively set in step S14. The calculated G correction gains corresponding to the first, second,..., (K−1) th, and kth horizontal lines are multiplied.
In the second image signal (I _O3 ), the B signal values for the B pixels on the second, fourth,..., (K−2), and k-th horizontal lines are respectively set in step S14. The calculated B correction gains corresponding to the second, fourth,..., (K−2), and k-th horizontal lines are multiplied.

As a result, a corrected image signal (I _R3 ) in which the flicker component is excluded from the second image signal (I _O3 ) is generated. As described with reference to FIG. 4, the corrected image signals I _R4 and I _R5 are also generated by repeating the processes of steps S11 to S15 while sequentially exchanging the first and second image signals.

  Usually, the composition ratio of R, G, and B in the illumination light of the fluorescent lamp varies somewhat depending on the luminance of the illumination. For this reason, the hue change (color flicker) of the image may occur in the frames FL3 to FL5, but this hue change can also be corrected by adopting the second correction method.

  As described in the description of the first correction method, one R correction gain, one G correction gain, and one B correction gain may be assigned to a plurality of horizontal lines. At this time, the interpolation processing in step S14 can be omitted.

  Further, as described in the description of the first correction method, each area AR [m: n] can include only pixels on one horizontal line. In this case, calculation of various values for a horizontal line for which no corresponding color pixel exists is omitted. For example, calculation of AVE_PRE_B [m: n], AVE_NOW_B [m: n], RATIO_B [m: n], AVE_RATIO_B [m], and B correction value corresponding to a region (or row) where only R pixels and G pixels exist Is omitted.

  Further, as described in the description of the first correction method, when the average value (AVE_RATIO_R [m] or the like) is calculated in step S13, RATIO_R [m: n] or the like according to a predetermined rule. RATIO_R [m: n] or the like determined to be invalid may be ignored in calculating the average value.

[Memory usage]
Next, a method of using a memory necessary for performing flicker correction using the first or second correction method described above will be described. The first and second memory usage methods that can be employed by the imaging apparatus 1 will be exemplified.

  First, the first memory usage method will be described. In the first memory usage method, a memory area having a predetermined size in the frame memory 40 is used. This memory area is called a first correction memory area (corresponding to the recording means). The predetermined size is, for example, a memory size that can store an image signal of one frame without excess or deficiency.

When the first memory usage method is adopted, the “original image signal I _O2 , corrected image signals I _R3 , I _R4 (, I _R5 )” corresponding to the lowermost stage in FIG. It is stored in the area while being updated sequentially.

That is, first, the original image signal I _O2 necessary for calculating the flicker correction value α is stored in the first correction memory area. The flicker correction value α is calculated and the corrected image signal I _R3 is generated using the original image signal I _O2 stored in the first correction memory area and the original image signal I _O3 sent from the preprocessing unit 14. I do. The generated corrected image signal I _R3 is sent to the pixel number conversion unit 21 and the like and stored in the first correction memory area (the original image signal I _O2 in the first correction memory area is overwritten).

In the next frame, the flicker correction value β is calculated and the corrected image signal using the corrected image signal I _R3 stored in the first correction memory area and the original image signal I _O4 sent from the preprocessing unit 14. I _R4 is generated. The generated corrected image signal I _R4 is sent to the pixel number conversion unit 21 and the like and is stored in the first correction memory area (the corrected image signal I _R3 in the first correction memory area is overwritten).

  The update of the image signal in the first correction memory area as described above is continued during the still image shooting frame (FL3 to FL5 in the above example). By adopting this method, the memory area required for flicker correction can be kept small.

  Next, the second memory usage method will be described. In the second memory usage method, the second correction memory area (corresponding to the recording means) is used. The second correction memory area is provided in a memory other than the frame memory 40 (for example, an internal register of the signal processing unit 20; not shown). However, it may be provided in the frame memory 40.

  When the second memory usage method is employed, when the first correction method shown in FIG. 5 is used, AVE_PRE [m: n] (corresponding to the comparison target value) calculated in step S1 of FIG. The correction memory area is sequentially updated and stored. When employing the second memory usage method, when the second correction method shown in FIG. 10 is used, AVE_PRE_R [m: n], AVE_PRE_G [m: n], and AVE_PRE_B [m] calculated in step S11 of FIG. : N] are sequentially updated and stored in the second correction memory area. This will be described more specifically by taking the case of using the first correction method shown in FIG. 5 as an example.

First, AVE_PRE [m: n] necessary for calculating the flicker correction value α is calculated in a state where the original image signal I _O2 is regarded as the first image signal that is a reference for correction, and the calculated AVE_PRE [m : N] is stored in the second correction memory area. The flicker correction value α is calculated and the corrected image signal I _R3 is calculated using AVE_PRE [m: n] stored in the second correction memory area and the original image signal I _O3 sent from the preprocessing unit 14. Generate.

After generating the corrected image signals I _R3, now in a state where the corrected image signal I _R3 captures a first image signal, AVE_PRE needed to calculate the flicker correction value β [m: n] a new And the newly calculated AVE_PRE [m: n] is stored in the second correction memory area. At this time, AVE_PRE [m: n] in the second correction memory area calculated in a state where the original image signal I _O2 is regarded as the first image signal is overwritten and updated. Then, flicker correction value β is calculated and corrected image signal I _R4 is calculated using AVE_PRE [m: n] stored in the second correction memory area and original image signal I _O4 sent from preprocessing unit 14. Generate.

After generating the corrected image signals I _R4, while now that captures the correction image signal I _R4 and the first image signal, AVE_PRE needed to calculate the flicker correction value γ [m: n] a new And the newly calculated AVE_PRE [m: n] is stored in the second correction memory area. At this time, AVE_PRE [m: n] in the second correction memory area calculated in a state where the corrected image signal I _R3 is regarded as the first image signal is overwritten and updated.

  The update of AVE_PRE [m: n] in the second correction memory area as described above is continued during the still image shooting frame (FL3 to FL5 in the above example). By adopting this method, the memory area required for flicker correction can be reduced. Further, the necessary memory area is very small as compared with the first memory usage method, and the frame memory 40 need not be used.

[Deformation etc.]
Although the example in which the first image signal serving as the correction reference is sequentially changed (I _O2 → I _R3 → I _R4 ) has been shown, the first image signal serving as the correction reference is fixed to the original image signal I _O2 . It is also possible to adopt the technique of doing. Also in this case, the first or second correction method described above can be employed.

In this case, the second image signal captured the original image signal I _O3, from comparison between the first image signal (I _O2) and the second image signal (I _O3), canceling the flicker component contained in the original image signal I _O3 Flicker correction value (same as the above flicker correction value α) is calculated, and the original image signal I _O3 is corrected using the flicker correction value. Also, the second image signal captured the original image signal I _O4, from comparison between the first image signal (I _O2) and the second image signal (I _O4), for canceling the flicker component contained in the original image signal I _O4 Flicker correction value (corresponding to the flicker correction value β) is calculated, and the original image signal I _O4 is corrected using the flicker correction value. Also, the second image signal captured the original image signal I _O5, from comparison between the first image signal (I _O2) and the second image signal (I _O5), for canceling the flicker component contained in the original image signal I _O5 Flicker correction value (corresponding to the flicker correction value γ) is calculated, and the original image signal I _O5 is corrected using the flicker correction value.

However, the case of fixing the first image signal as a reference of correction to the original image signal I _O2, the accuracy of the consideration of flicker correction that object moves deteriorated. For this reason, as described above, it is desirable to sequentially change the first image signal serving as a correction reference.

The case where the three frames FL3 to FL5 are used to shoot one still image is illustrated. However, when the change of the aperture amount of the diaphragm 12 is completed in a sufficiently short time, the still image shooting frame is set to 1 It is possible to have two frames. For example, only the frame FL3 is a still image shooting frame. In this case, the opening amount of the exposure time and the aperture 12 in the frame FL1, FL2 and FL4~FL6 are the T _A and D _A for moving image shooting, the opening amount of the exposure time and the aperture 12 in the frame FL3 is for still image shooting T _B and D _B (see FIG. 2).

The operation when the still image shooting frame is composed of one frame FL3 is the same as the above-described operation. It is merely unnecessary to perform flicker correction of the original image signals I _O4 and I _O5 corresponding to the frames FL4 and FL5. Of course, when a plurality of still images are continuously photographed, a plurality of consecutive frames can be used as a still image photographing frame regardless of the changing speed of the aperture amount of the diaphragm 12.

  In FIG. 3, the areas AR [1: 1] to AR [1: N] belonging to the first row of the M rows and N columns matrix constitute one detection region. Similarly, the areas AR [m: 1] to AR [m: N] belonging to the mth row also constitute one detection area (where m is an integer in the range of 2 to M). Each detection area includes pixels on different horizontal lines. In FIG. 8 or FIG. 9, the area AR [1: 1] belonging to the first row constitutes one detection area. Similarly, the area AR [m: 1] belonging to the m-th row also constitutes one detection area (here, m is any integer in the range of 2 to M).

  The “average value” described above may of course be an “integrated value”.

1 is an overall block diagram of an imaging apparatus according to an embodiment of the present invention. It is a figure for demonstrating the operation | movement of the still image simultaneous imaging | photography during video recording performed using the imaging device of FIG. It is a figure showing a mode that the imaging area of the imaging device of FIG. 1 was divided | segmented into the area | region of M rows and N columns. FIG. 2 is a diagram for explaining a flicker correction operation by a flicker detection unit and a flicker correction unit in FIG. 1. 3 is a flowchart for explaining a first correction method of flicker correction by a flicker detection unit and a flicker correction unit in FIG. 1. It is a figure for demonstrating the 1st correction method of the flicker correction by the flicker detection part of FIG. 1, and a flicker correction | amendment part. It is a figure which represents typically the correction gain calculated in the flicker detection part of FIG. FIG. 10 is a diagram for explaining a modification of the flicker correction method by the flicker detection unit and the flicker correction unit of FIG. 1. FIG. 10 is a diagram for explaining a modification of the flicker correction method by the flicker detection unit and the flicker correction unit of FIG. 1. 6 is a flowchart for explaining a second correction method of flicker correction by the flicker detection unit and the flicker correction unit of FIG. 1. It is a figure which shows the arrangement | sequence of the color filter in each area | region shown in FIG. It is a figure for demonstrating the generation | occurrence | production principle of a fluorescent lamp flicker. It is a figure for demonstrating the generation | occurrence | production principle of a fluorescent lamp flicker. It is a figure for demonstrating the relationship between exposure time and a picked-up image. It is a figure for demonstrating the generation | occurrence | production principle of a fluorescent lamp flicker.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 11 Optical system 12 Aperture 13 Imaging element 14 Preprocessing part 17 Driver 18 Timing generator 19 Main control part 20 Signal processing part 21 Pixel number conversion part 22 Motion detection part 23 Movie processing part 24 Movie compression part 25 Still picture processing part 25 26 still image compression unit 27 flicker detection unit 28 flicker correction unit 40 frame memory buffer 41 external recording medium 42 operation unit

Claims (9)

An image sensor for taking a picture and a signal processing unit for processing an image signal obtained by taking a picture using the image sensor while taking different exposure timings between different horizontal lines, and taking a video and a still picture In a possible imaging device,
When shooting a still image during movie shooting, an exposure time different from the exposure time for movie shooting is taken between still image shooting frames consisting of one or more frames,
The signal processing unit
A reference image signal that is an image signal obtained by photographing using the exposure time for moving image photographing before photographing the still image, and a correction target image signal that is an image signal obtained by photographing the still image photographing frame; Based on the comparison, the flicker correction value corresponding to the flicker component that is included in the correction target image signal due to the use of the exposure time different from the exposure time for moving image shooting is calculated. Flicker correction value calculation means;
An image pickup apparatus comprising: flicker correction means for correcting the correction target image signal in a direction of reducing the flicker component using the flicker correction value. The still image shooting frame is composed of first, second,..., And nth frames (n is an integer of 2 or more), and the correction target image signal is first, second,. And the first, second,..., And nth correction target image signals corresponding to the nth frame,
The flicker correction value calculation means calculates a first flicker correction value as the flicker correction value from a comparison between the reference image signal and the first correction target image signal, and the flicker correction means calculates the first flicker correction value. The first correction target image signal is corrected using
The flicker correction value calculating means calculates a second flicker correction value as the flicker correction value from a comparison between the corrected first image signal and the second correction target image signal, and the flicker correction means Correcting the second correction target image signal using the second flicker correction value;
The flicker correction value calculating means calculates an nth flicker correction value as the flicker correction value from a comparison between the corrected (n−1) th correction target image signal and the nth correction target image signal, and the flicker The imaging apparatus according to claim 1, wherein the correction unit corrects the n-th correction target image signal using the n-th flicker correction value. A plurality of detection areas are provided in the imaging area of the imaging element, each detection area includes pixels on different horizontal lines,
The first flicker correction value includes a plurality of first element correction values each corresponding to one or more horizontal lines, and the second flicker correction value corresponds to one or more horizontal lines. The nth flicker correction value is composed of a plurality of nth element correction values each corresponding to one or more horizontal lines,
The flicker correction value calculating means compares the first signal for each detection region based on a comparison between a signal corresponding to each detection region in the reference image signal and a signal corresponding to each detection region in the first correction target image signal. By detecting the flicker component included in the correction target image signal, each first element correction value is calculated,
From the comparison between the signal corresponding to each detection region in the first correction target image signal after correction and the signal corresponding to each detection region in the second correction target image signal, the second correction target is detected for each detection region. By detecting the flicker component included in the image signal, each second element correction value is calculated,
From the comparison between the signal corresponding to each detection region in the (n−1) th correction target image signal after correction and the signal corresponding to each detection region in the nth correction target image signal, The imaging apparatus according to claim 2, wherein each n-th element correction value is calculated by detecting the flicker component included in the n-th correction target image signal. Recording means for recording information necessary to calculate the first, second,..., And nth flicker correction values;
The recording means includes information for calculating the first flicker correction value, information for calculating the second flicker correction value,..., And information for calculating the nth flicker correction value. The image pickup apparatus according to claim 2, wherein the image pickup apparatus is recorded while being sequentially updated. Recording means for recording information necessary to calculate the first, second,..., And nth flicker correction values;
The flicker correction value calculating means includes
Calculating a first flicker correction value from a comparison between a comparison target value calculated from the reference image signal and a comparison target value calculated from the first correction target image signal;
A second flicker correction value is calculated from a comparison between the comparison target value calculated from the first correction target image signal after correction and the comparison target value calculated from the second correction target image signal;
Calculating the nth flicker correction value from a comparison between the comparison target value calculated from the corrected (n-1) th correction target image signal and the comparison target value calculated from the nth correction target image signal;
The recording means includes a comparison target value calculated from the reference image signal, a comparison target value calculated from the corrected first correction target image signal, and the corrected (n− 1) The imaging apparatus according to claim 2, wherein the comparison target value calculated from the correction target image signal is recorded while being sequentially updated as the information. The still image shooting frame is composed of first, second,..., And nth frames (n is an integer of 2 or more), and the correction target image signal is first, second,. And the first, second,..., And nth correction target image signals corresponding to the nth frame,
The flicker correction value calculating means compares the reference image signal with the first, second,..., And nth correction target image signals, respectively, as the flicker correction value. And the nth flicker correction value is calculated, and the flicker correction means uses the first, second,..., And nth flicker correction values, respectively, for the first, second,. The imaging apparatus according to claim 1, wherein the n-th correction target image signal is corrected. When the still image shooting frame consists of one frame,
The imaging apparatus according to claim 1, wherein the flicker correction value calculating unit calculates the flicker correction value from a comparison between the reference image signal and the correction target image signal. A plurality of detection areas are provided in the imaging area of the imaging element, each detection area includes pixels on different horizontal lines,
The flicker correction value includes a plurality of element correction values each corresponding to one or more horizontal lines,
The flicker correction value calculation means calculates the correction target image signal for each detection region based on a comparison between a signal corresponding to each detection region in the reference image signal and a signal corresponding to each detection region in the correction target image signal. The image pickup apparatus according to claim 7, wherein each element correction value is calculated by detecting the flicker component included in the image pickup apparatus. The image pickup apparatus according to claim 1, wherein the image pickup element is an XY address scanning type image pickup element.