JP2005175864A - Image pickup device and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique of an image pickup device, for simplifying a noise elimination processing in moving image photographing at the time of low illuminance. <P>SOLUTION: The image pickup device 1A is provided with a circulation filter part 26 for eliminating noise in the animation photographing at the time of the low illuminance. In the circulation filter part 26, signals for which a gain coefficient is multiplied in a feedback amplifier part 264 with an image of one frame before outputted from a frame delay part 265 and signals for which the gain coefficient (feedback coefficient) is multiplied in an amplifier part 262 with the present frame image are added in an addition part 263. In this case, the gain coefficient of the amplifier part 262 and the gain coefficient of the feedback amplifier part 264 are changed corresponding to the correlation of the present frame image and the image of the previous frame. Then, in the circulation filter part 26, since RAW image data (Bayer data) before pixel interpolation in a pixel interpolation part 31 are processed, a data amount is reduced and the noise elimination processing is simplified. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique of an imaging apparatus that captures an image of a subject and sequentially generates frames constituting a moving image.

  In moving image shooting with a digital camera (imaging device), the sensitivity tends to be insufficient with the increase in the number of pixels, and an increase in noise at low illumination is inevitable.

  Thus, for example, Patent Document 1 discloses a technique for performing noise reduction based on a correlation between frames in order to improve moving image shooting at low illuminance.

JP-A-6-38098

  The technique of Patent Document 1 does not mention the data format of an image to be subjected to feedback filter processing. However, when performing feedback filter processing on a frame image after pixel interpolation, the amount of data to be processed Therefore, the process becomes complicated and it becomes difficult to perform a quick process. This problem will be described with reference to FIG.

  FIG. 12 is a block diagram for explaining feedback filter processing according to the prior art.

  Raw image data (Bayer data) generated by the image sensor is subjected to pixel interpolation by the pixel interpolation unit 81, and the image data subjected to pixel interpolation is subjected to cyclic filter processing by the two circulation filter units 82. In this case, since the pixel-interpolated image data is input to the circulation filter unit 82 as color difference data, the amount of data to be processed becomes large and the processing becomes complicated.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for an imaging apparatus that can simplify noise removal processing in moving image shooting at low illuminance.

  In order to solve the above problems, the invention of claim 1 is an imaging apparatus, wherein (a) a moving image capturing unit that captures an image of a subject and sequentially generates frames constituting the moving image as RAW image data; When a predetermined shooting condition is satisfied, the difference between the RAW image data of the current frame generated by the moving image shooting means and the RAW image data of the previous frame is multiplied by a feedback coefficient to generate the RAW image data of the current frame. Filter means for performing a filter process subtracting from (1), and (b-1) setting means for setting the feedback coefficient based on control information relating to imaging.

  According to a second aspect of the present invention, in the image pickup apparatus according to the first aspect of the present invention, (c) the RAW image data of the frame generated by the moving image photographing means before the filter processing by the filter means is applied. First correction means for performing black level correction and / or pixel defect correction is further provided.

  According to a third aspect of the present invention, in the imaging apparatus according to the first or second aspect of the present invention, (d) shading correction and RAW image data of a frame subjected to the filtering process by the filtering means are performed. And / or second correction means for correcting white balance.

  Further, the invention of claim 4 is an imaging apparatus, wherein (a) a moving image capturing step of capturing an image of a subject and sequentially generating frames constituting the moving image as RAW image data; and (b) the moving image capturing step. A first correction step of performing black level correction and / or pixel defect correction on the RAW image data of the generated frame; and (c) when a predetermined condition is satisfied, correction is performed in the first correction step. A filtering step of performing a filtering process for multiplying the difference between the RAW image data of the current frame and the RAW image data of the previous frame by a feedback coefficient set based on the control information related to shooting and subtracting from the RAW image data of the current frame; (D) Shading correction and / or white balance correction is performed on the RAW image data of the frame subjected to the filtering process in the filtering step. Cormorant comprises means for performing each of the second correction step.

  Further, the invention of claim 5 is an image processing method, wherein (a) a moving image capturing step of capturing an image of a subject and sequentially generating frames constituting a moving image as RAW image data; and (b) the moving image capturing step. A first correction step of performing black level correction and / or pixel defect correction on the RAW image data of the frame generated in step (c), and (c) when a predetermined condition is satisfied, correction is performed in the first correction step. A filtering step of performing a filtering process for multiplying the difference between the RAW image data of the current frame and the RAW image data of the previous frame by a feedback coefficient set based on the control information related to shooting and subtracting from the RAW image data of the current frame (D) shading correction and / or white balance correction for the RAW image data of the frame subjected to the filtering process in the filtering step. And a second correction step of performing.

  According to the first to third aspects of the present invention, when a predetermined photographing condition is satisfied, photographing is performed on the difference between the RAW image data of the current frame generated by the moving image photographing unit and the RAW image data of the previous frame. Since the filtering process for subtracting from the RAW image data of the current frame by multiplying the feedback coefficient set based on the control information on the current frame is performed, the noise removal process can be simplified in moving image shooting at low illuminance.

  In particular, in the invention of claim 2, since black level correction and / or pixel defect correction is performed on the RAW image data of the frame generated by the moving image photographing means before the filter processing, black level correction and pixel defect correction are performed. Can be simplified.

  In the invention of claim 3, since shading correction and / or white balance correction is performed on the RAW image data of the frame subjected to the filter processing by the filter means, the shading correction and white balance correction processing is simplified. it can.

  According to the fourth and fifth aspects of the present invention, a predetermined condition is satisfied after black level correction and / or pixel defect correction is performed on the RAW image data of the frame generated in the moving image shooting process. Is subtracted from the RAW image data of the current frame by multiplying the difference between the RAW image data of the current frame subjected to the above correction and the RAW image data of the previous frame by a feedback coefficient set on the basis of the control information related to shooting. After performing the filtering process, shading correction and / or white balance correction is performed on the RAW image data of the frame subjected to the filtering process. As a result, noise removal processing can be simplified in moving image shooting at low illumination, and black level correction, pixel defect correction, shading correction, and white balance correction processing can be simplified.

<First Embodiment>
<Principal configuration of imaging device>
FIG. 1 is a diagram illustrating a main configuration of an imaging apparatus 1A according to the first embodiment of the present invention. Here, FIGS. 1A to 1C correspond to a front view, a rear view, and a top view of the imaging apparatus 1A, respectively.

  The imaging device 1 </ b> A is configured as a digital camera and includes a photographing lens 10. In addition, a photometric sensor 11 is provided on the front surface of the image pickup apparatus 1A to perform photometry on the subject and generate a luminance signal.

  The image pickup apparatus 1A is provided with a mode switch 12 and a shutter button 13 on the upper surface thereof.

  The mode switch 12 is a shooting mode (REC mode) for capturing an image of a subject and recording the still image, a moving image mode (MOVE mode) for capturing a moving image, and an image recorded in the memory card 9 (see FIG. 2). Is a switch for switching between a playback mode (PLAY mode) for playing back.

  The shutter button 13 is a two-stage switch that can detect a half-pressed state (S1 on) and a fully pressed state (S2 on). When the shutter button 13 is half-pressed in the above photographing mode, the zoom / focus motor driver 47 (see FIG. 2) is driven, and the operation of moving the photographing lens 10 to the in-focus position is performed. On the other hand, when the shutter button 13 is fully pressed in the shooting mode, a main shooting operation, that is, a recording shooting operation is performed.

  An LCD (Liquid Crystal Display) monitor 42, an electronic viewfinder (EVF) 43, a camera shake correction switch 14, and a frame advance / zoom switch 15 are displayed on the back of the image pickup apparatus 1A. Is provided.

  The camera shake correction switch 14 drives the camera shake correction actuator driver 48 (FIG. 2) so that camera shake detected by the camera shake sensor 49 (FIG. 2) does not adversely affect shooting. This is a switch for setting the correction mode. Here, the imaging sensor 16 can move two-dimensionally in a direction orthogonal to the optical axis of the photographing lens 10 by, for example, two actuators that perform linear driving.

  The frame advance / zoom switch 15 is composed of four buttons, and is a switch for instructing frame advance of a recorded image in playback mode and zooming at the time of shooting.

  FIG. 2 is a diagram illustrating functional blocks of the imaging apparatus 1A.

  The imaging apparatus 1A includes an imaging sensor 16, a signal processing unit 2 connected to the imaging sensor 16 so as to be able to transmit data, an image processing unit 3 connected to the signal processing unit 2, and a camera control unit 40A connected to the image processing unit 3. And.

  The imaging sensor 16 is configured as an area sensor (imaging device) in which R (red), G (green), and B (blue) primary color transmission filters are arranged in a checkered pattern (Bayer array) in units of pixels. This is a pixel readout type. The imaging sensor 16 functions as a moving image capturing unit that captures an image of a subject and sequentially generates frames constituting the moving image as RAW image data (Bayer data).

  The signal processing unit 2 includes a CDS 21, an AGC 22, and an A / D conversion unit 23.

  The analog image signal acquired and output by the imaging sensor 16 is sampled by the CDS 21 and noise is removed, and then the AGC 22 multiplies the analog gain corresponding to the imaging sensitivity to perform sensitivity correction.

  The A / D conversion unit 23 is configured as a 14-bit converter, and digitizes the analog signal normalized by the AGC 22. The digitally converted image signal is subjected to predetermined image processing by the image processing unit 3 to generate an image file.

  The image processing unit 3 includes a CPU and a memory, and includes a RAW addition unit 30, a digital processing unit 3p, and an image compression unit 35. Further, the image processing unit 3 includes a ranging calculation / motion detection unit 36, an OSD unit 37, a video encoder 38, and a memory card driver 39.

  The digital processing unit 3p includes a pixel interpolation unit 31, a resolution conversion unit 32, a white balance control unit 33, and a gamma correction unit 34.

  The image data input to the image processing unit 3 is written into the image memory 41 in synchronization with the reading of the image sensor 16. Thereafter, the image data stored in the image memory 41 is accessed, and various processes are performed in the image processing unit 3.

  In the image data in the image memory 41, first, the white balance control unit 33 performs gain correction of RGB pixels independently, and RGB white balance correction is performed. In this white balance correction, a portion that is originally white from a photographic subject is estimated from brightness, saturation data, and the like, and an average value, a G / R ratio, and a G / B ratio for each of R, G, and B are obtained. Based on these pieces of information, the R and B correction gains are controlled.

  The white balance-corrected image data is obtained by masking each RGB pixel with the respective filter pattern by the RAW adder 30 and the pixel interpolator 31, and then, for G pixels having pixel values up to a high band, a median (intermediate value) filter To replace the average value of the intermediate binary values of the surrounding four pixels. For the R pixel and the B pixel, average interpolation is performed for the same color in the surrounding nine pixels.

  The pixel-interpolated image data is subjected to nonlinear conversion suitable for each output device by the gamma correction unit 34, specifically, gamma correction and offset adjustment, and stored in the image memory 41.

  The image data stored in the image memory 41 is horizontally or vertically reduced or thinned out to the number of pixels set by the resolution conversion unit 32, and after compression processing by the image compression unit 35, the memory card driver 39 Is recorded in the memory card 9 attached to the card. At the time of this image recording, a photographic image with a designated resolution is recorded, and a screen nail image (VGA) for reproduction display is created and recorded linked to the photographic image. At the time of recording and reproduction, a screen nail image is displayed on the LCD monitor 42, thereby enabling high-speed image display.

  The resolution conversion unit 32 also performs pixel thinning for image display, and creates a low-resolution image for display on the LCD monitor 42 or the EVF 43. At the time of preview, a low resolution image of 640 × 240 pixels read out from the image memory 41 is encoded into NTSC / PAL by the video encoder 38, and this is reproduced as a field image on the LCD monitor 42 or the EVF 43.

  The distance measurement calculation / motion detection unit 36 divides the image into a plurality of blocks, performs BFP processing for each block, extracts the frequency components in the focus area, and compares them in units of frames, thereby comparing the focus lens. Is driven. After such video (video) AF is performed and focusing is completed, the movement of the subject is detected based on the amount of change in the evaluation value in each block, and AF control is performed again.

  The AF evaluation value is a low value when the distance measurement target moves during AF driving, and a high value is output when the AF evaluation value shifts to a constant focus state. A correlation between frames, which will be described later, is determined using an AF evaluation value having such characteristics, and a feedback coefficient is set.

  The OSD (on-screen display) unit 37 can generate various characters, symbols, frames, and the like, and can superimpose them on an arbitrary position of the display image. The OSD unit 37 can display various characters, symbols, frames, and the like on the LCD monitor 42 as necessary.

  The camera control unit 40A includes a CPU and a memory, and is a part that comprehensively controls each unit of the imaging device 1. Specifically, the operation input performed by the photographer is processed with respect to the camera operation switch 50 having the mode switch 12 and the shutter button 13. In addition, the camera control unit 40A switches to a shooting mode and a playback mode in which a subject is imaged and the image data is recorded by operating the mode setting switch 12 by the photographer.

  The tripod detector 51 is a part that detects that a tripod is attached to a tripod hole 52 (shown by a broken line in FIGS. 1A and 1B) provided on the lower surface of the imaging apparatus 1A.

  In the imaging device 1A, the optical aperture of the shutter 44 is fixed open by the aperture driver 45 during preview display (live view display) in which the subject is displayed on the LCD monitor 42 in a moving image manner in the shooting preparation state before the main shooting. Regarding the charge accumulation time (exposure time) of the image sensor 16 corresponding to the shutter speed (SS), the camera control unit 40A calculates exposure control data based on the live view image acquired by the image sensor 16. Then, feedback control is performed on the timing generator sensor driver 46 so that the exposure time of the image sensor 16 is appropriate based on a program diagram set in advance based on the calculated exposure control data.

  When the charge accumulation in the image sensor 16 is completed, the photoelectrically converted signal is shifted to the light-shielded transfer path in the image sensor 16 and read out from the transfer path via the buffer.

  In the imaging apparatus 1A having the above-described configuration, noise removal can be performed in moving image shooting at low illuminance. This noise processing will be described in detail below.

<Noise removal during movie shooting>
FIG. 3 is a diagram for explaining noise removal during moving image shooting in the imaging apparatus 1A.

  In the imaging apparatus 1A, the image processing unit 3 includes a black correction unit 24, a flaw correction unit 25, a circulation filter unit 26, a shading correction unit 27, and a white balance (WB) amplifier unit 28. In each of these units, image processing is performed on the RAW image data of the digital signal generated by the imaging sensor 16 and output from the signal processing unit 2. The RAW image data is image data output from the image sensor 16 and is image data before pixel interpolation.

  In the imaging apparatus 1A, in addition to the pixel interpolation unit 31 and the gamma correction unit 34 described above, the image processing unit 3 includes a matrix unit 61 and a color difference matrix unit that perform filter processing using, for example, a 3 × 3 matrix. 62 and a contour correcting unit 63 are provided.

  The black correction unit 24 is a part that performs black level correction on the RAW image data of the frame generated by the imaging sensor 16.

  The flaw correction unit 25 is a part that corrects a pixel defect when a flaw, that is, a pixel defect exists in the RAW image data of the frame generated by the imaging sensor 16.

  The cyclic filter unit 26 includes a correlation detection unit 261, an amplifier unit 262, an addition unit 263, a feedback amplifier unit 264, and a frame delay unit 265.

  In this circular filter unit 26, a signal obtained by multiplying an image one frame before output from a frame delay unit 265 having a frame memory and holding image data by a feedback amplifier unit 264 by a gain coefficient (feedback coefficient), A signal obtained by multiplying the frame image by a gain coefficient set by the amplifier unit 262 is added by the adding unit 263. That is, the cyclic filter unit 26 performs a filtering process for multiplying the difference between the RAW image data of the current frame generated by the image sensor 16 and the RAW image data of the previous frame by the feedback coefficient and subtracting from the RAW image data of the current frame. Done.

  The feedback coefficient of the feedback amplifier unit 264 is determined according to the correlation between the current frame image and the previous frame image. Specifically, when the correlation between frames is large, the feedback coefficient of the feedback amplifier unit 264 is increased to increase the feedback amount and suppress uncorrelated noise. On the other hand, when the feedback coefficient is increased when the correlation between frames is small, image blurring occurs. Therefore, when the frame correlation is small, the feedback coefficient is decreased to reduce the amount of feedback and minimize image degradation. .

  Regarding the control of the feedback coefficient, the correlation detection unit 261 obtains a correlation between frames based on the AF evaluation value, and is set to a feedback coefficient corresponding to this correlation.

  The shading correction unit 27 is configured such that the amount of light differs between the central portion and the peripheral portion of the image by the photographing lens 10 or the like with respect to the RAW image data of the frame subjected to the filtering process by the circulation filter unit 26, specifically, It is a part which corrects that a part becomes dark. In this shading correction unit 27, processing for amplifying at different amplification factors is performed in each part of the image.

  The WB amplifier unit 28 performs RGB white balance correction on the RAW image data of the frame subjected to the filter processing by the circulation filter unit 26 at a portion corresponding to the white balance control unit 33 described above.

  The RAW image data of the frame for which the white balance correction has been performed by the WB amplifier unit 28 is subjected to pixel interpolation by the pixel interpolation unit 31, and matrix calculation is performed by the matrix unit 61. The gamma correction unit 33 performs gamma correction, the color difference matrix unit 62 converts the luminance signal (Y) and the color difference signal (C), and then the contour correction unit 63 performs contour correction on the image.

<Operation of Imaging Device 1A>
FIG. 4 is a flowchart showing the basic operation of the imaging apparatus 1A. This operation is executed by the camera control unit 40A.

  In step S1, the feedback coefficient is set to a default value (for example, 0.2). That is, the gain coefficient in the feedback amplifier unit 264 of the circulation filter unit 26 is set to an initial value.

  In step S2, the VGA image is read as a moving image. That is, a frame image is read from the image sensor 16.

  In step S3, the exposure amount is calculated based on the image data read in step S2, and the optimum shutter speed, aperture value (Fno), and AGC gain of the AGC unit 22 are calculated.

  In step S4, the optical aperture of the shutter 44 is driven by the aperture driver 45 based on the aperture value calculated in step S3.

  In step S5, the gain coefficient of AGC 22 is set based on the gain value calculated in step S3.

  In step S6, it is determined whether the moving image NR (noise reduction) is automatically set. Here, for example, it is determined whether or not the automatic setting of the moving image NR is selected on the menu screen displayed on the LCD monitor 42. If the automatic setting has been performed, the process proceeds to step S7. If the automatic setting has not been performed, the process proceeds to step S14.

  In step S7, it is determined whether the set gain in the AGC 22 is greater than a predetermined value Ref1. Here, when the AGC gain corresponding to the photographing sensitivity is larger than the predetermined value Ref1, it is determined that the illumination is low, and the process proceeds to step S8. When the AGC gain is equal to or less than the predetermined value Ref1, the process proceeds to step S14.

  In step S8, it is determined whether the AF evaluation value calculated by the distance measurement calculation / motion detection unit 36 is greater than a predetermined value Ref2. This is an operation for positively performing the filter processing in the circulation filter unit 26 because it can be determined that the subject is stationary to some extent when the AF evaluation value is a certain value or more. If the AF evaluation value is larger than the predetermined value Ref2, the process proceeds to step S9. If the AF evaluation value is equal to or smaller than the predetermined value Ref2, the process proceeds to step S14.

  In step S9, it is determined whether or not the correlation degree of the AF evaluation value is high. Here, when the correlation degree of the AF evaluation value, which is control information related to photographing, is high, the process proceeds to step S10, and when the correlation degree is low, the process proceeds to step S11.

  In step S10, the feedback coefficient of the feedback amplifier unit 264 is increased and set. Here, for example, 0.1 is added to the feedback coefficient.

  In step S11, the feedback coefficient of the feedback amplifier unit 264 is reduced and set. Here, for example, 0.1 is subtracted from the feedback coefficient.

  In step S12, a feedback coefficient limiter process is performed. That is, when the feedback coefficient is larger than the upper limit value (for example, 0.8), the feedback coefficient is set to be limited to this upper limit value. Thereby, excessive feedback is suppressed, and appropriate noise removal can be performed.

  In step S13, the circulation filter unit 264 performs circulation filter processing.

  In step S14, the feedback coefficient of the feedback amplifier unit 264 is set to a default value (for example, 0.2).

  In step S15, the image processing unit 3 performs imaging processing such as pixel interpolation on the frame image.

  In step S16, a video image (moving image) is output.

  In step S17, the video image output in step S16 is displayed on a display unit such as an LCD monitor 42.

  In step S18, it is determined whether or not the moving image shooting has been completed. Specifically, it is determined whether the user has operated the shutter button 13 for ending the moving image shooting. Here, when the moving image shooting is not finished, the flow returns to step S2 and the moving image shooting is continued.

  By the above operation of the image pickup apparatus 1A, the circulatory filter processing is performed on the RAW image data, so that noise removal at low illuminance can be simplified. In addition, since noise is reduced at an early stage before pixel interpolation processing, it is possible to reduce errors generated in each processing due to noise images.

Second Embodiment
The imaging device 1B according to the second embodiment of the present invention has a configuration similar to that of the imaging device 1A according to the first embodiment, as shown in FIGS. Is different.

  That is, in the circulation filter unit 26 of the image pickup apparatus 1B, the correlation detection unit 261 shown in FIG. 3 determines a feedback coefficient according to the control value of the camera shake correction calculated based on the measurement value of the camera shake sensor 49.

<Operation of Imaging Device 1B>
FIG. 5 is a flowchart showing the basic operation of the imaging apparatus 1B. This operation is executed by the camera control unit 40B.

  In steps S21 to S28, operations similar to those in steps S1 to S8 shown in the flowchart of FIG. 4 are performed.

  In step S29, it is determined whether camera shake correction is on. Specifically, it is determined whether the camera shake correction switch 14 is operated and the camera shake correction mode is set. If camera shake correction is on, the process proceeds to step S30. If not, the process proceeds to step S31 and the feedback coefficient is set to zero.

  In step S30, it is determined whether or not the camera shake detection value detected by the camera shake sensor 49 is smaller than a predetermined value Ref3. Here, since the AF evaluation value exceeds the predetermined value Ref2 in step S28, it can be determined that the subject is stationary to some extent. Therefore, when the detected value of camera shake is smaller than the predetermined value Ref3, it can be determined that the frame correlation is high, and thus the feedback coefficient is increased in step S32. On the other hand, if the camera shake detection value is equal to or greater than the predetermined value Ref3, an error in feedback processing due to camera shake occurs, so the feedback coefficient is decreased in step S33.

  In steps S32 to S40, operations similar to those in steps S10 to S18 shown in the flowchart of FIG. 4 are performed.

  The operation of the imaging apparatus 1B as described above also exhibits the same effect as that of the imaging apparatus 1A of the first embodiment.

<Third Embodiment>
The imaging device 1C according to the third embodiment of the present invention has a configuration similar to that of the imaging device 1A according to the first embodiment, as shown in FIGS. Is different.

  That is, in the circulation filter unit 26 of the image pickup apparatus 1C, the correlation detection unit 261 shown in FIG. 3 determines a feedback coefficient according to the calculated focal length of the photographing lens 10.

  Since the occurrence level of camera shake varies depending on the focal length (angle of view) of the photographic lens 10, it is assumed that in the case of a long focal length, a slight blur of the subject due to camera shake occurs. Therefore, as described later, in the imaging apparatus 1C, the correlation detection unit 261 sets a feedback coefficient according to the focal length at the time of shooting, specifically, less than 30 mm, 30 to 50 mm, and 50 to 100 mm in terms of 135 mm. Under the condition of the focal length, the feedback amplifier section 264 is set to have a feedback coefficient of 0.8, 0.5, and 0.2, and the cyclic filter process is performed.

<Operation of Imaging Device 1C>
FIG. 6 is a flowchart showing the basic operation of the imaging apparatus 1C. This operation is executed by the camera control unit 40C.

  In steps S41 to S46, operations similar to those in steps S2 to S7 shown in the flowchart of FIG. 4 are performed.

  In step S47, it is determined whether the focal length of the taking lens 10 is greater than 100 mm. If the focal length is greater than 100 mm, the process proceeds to step S48, and if it is 100 mm or less, the process proceeds to step S54.

  In step S48, it is determined whether the focal length of the taking lens 10 is greater than 50 mm. If the focal length is greater than 50 mm, the process proceeds to step S50, and if it is 50 mm or less, the process proceeds to step S49.

  In step S49, it is determined whether the focal length of the taking lens 10 is greater than 30 mm. If the focal length is greater than 30 mm, the process proceeds to step S51, and if it is 30 mm or less, the process proceeds to step S52.

  In step S50, the feedback coefficient of the feedback amplifier unit 264 is set to 0.2.

  In step S51, the feedback coefficient of the feedback amplifier unit 264 is set to 0.5.

  In step S52, the feedback coefficient of the feedback amplifier unit 264 is set to 0.8.

  In step S53, the same operation as step S13 shown in the flowchart of FIG. 4 is performed.

  In step S54, the feedback coefficient of the feedback amplifier unit 264 is set to zero. That is, the circulation filter process is prohibited.

  In steps S55 to S58, operations similar to those in steps S15 to S18 shown in the flowchart of FIG. 4 are performed.

  Also by the operation of the imaging apparatus 1C as described above, the same effect as that of the imaging apparatus 1A of the first embodiment is exhibited.

<Fourth embodiment>
The imaging device 1D according to the fourth embodiment of the present invention has a configuration similar to that of the imaging device 1A according to the first embodiment, as shown in FIGS. Is different.

  That is, in the circulation filter unit 26 of the imaging apparatus 1D, the correlation detection unit 261 shown in FIG. 3 determines a feedback coefficient corresponding to the calculated shutter speed.

  When the AF evaluation value exceeds a certain value, it can be determined that the subject is stationary to some extent, and therefore the feedback coefficient is controlled in conjunction with the calculated value (set value) of the calculated SS. . For example, SS is less than 1 / 250s, 1 / 250-1 / 100s, and 1 / 100-1 / 30s under each SS setting condition, the feedback amplifier unit 264 has a feedback coefficient of 0.2, 0.5, and It is set to 0.8 and appropriate cyclic filter processing is performed. Regarding the setting of the feedback coefficient, since the correlation between frame images becomes extremely small when SS is short, that is, at high shutter speed, an afterimage effect occurs when the feedback coefficient is increased, and an unnatural image is generated by cyclic filter processing. Therefore, the feedback coefficient is set smaller as SS is shorter. On the other hand, for SS (exposure time) of 1/30 s or more, priority is given to multi-frame shooting (detailed in the fifth embodiment), and circulation filter processing is prohibited because there is a high possibility of causing camera shake.

<Operation of Imaging Device 1D>
FIG. 7 is a flowchart showing the basic operation of the imaging apparatus 1D. This operation is executed by the camera control unit 40D.

  In steps S61 to S67, operations similar to those in steps S2 to S7 shown in the flowchart of FIG. 4 are performed.

  In step S68, it is determined whether the calculated value of SS (shutter speed) is smaller than 1/30 s. If it is smaller than 1/30 s, the process proceeds to step S69, and if it is 1/30 s or longer, the process proceeds to step S75.

  In step S69, it is determined whether the calculated value of SS is smaller than 1 / 250s. If it is smaller than 1 / 250s, the process proceeds to step S71, and if it is 1 / 250s or more, the process proceeds to step S70.

  In step S70, it is determined whether the calculated value of SS is smaller than 1/100 s. If it is smaller than 1/100 s, the process proceeds to step S72, and if it is 1/100 s or longer, the process proceeds to step S73.

  In step S71, the feedback coefficient of the feedback amplifier unit 264 is set to 0.2.

  In step S72, the feedback coefficient of the feedback amplifier unit 264 is set to 0.5.

  In step S73, the feedback coefficient of the feedback amplifier unit 264 is set to 0.8.

  In step S74, the same operation as step S13 shown in the flowchart of FIG. 4 is performed.

  In step S75, the feedback coefficient of the feedback amplifier unit 264 is set to zero. That is, priority is given to multi-frame exposure in which the SS set value is 1/30 s or more, and the circulation filter processing is prohibited.

  In steps S76 to S79, operations similar to those in steps S15 to S18 shown in the flowchart of FIG. 4 are performed.

  The operation of the imaging apparatus 1D as described above also exhibits the same effect as that of the imaging apparatus 1A of the first embodiment.

<Fifth Embodiment>
An imaging apparatus 1E according to the fifth embodiment of the present invention has a configuration similar to that of the imaging apparatus 1A according to the first embodiment as shown in FIGS. 1 and 2, but the camera control unit is different. Yes.

  That is, in the camera control unit 40E of the imaging apparatus 1E, a program for selecting two noise reduction processes described below is stored in the memory.

  For these two noise reduction processes, the cyclic filter process based on the AF evaluation value described in the first embodiment and the noise reduction process by multi-frame exposure can be selected.

  For the above multi-frame exposure, the sensitivity setting at the time of shooting is lowered. That is, the noise is reduced by setting the amplifier gain of the AGC 22 of the signal processing unit 2 low. In such a case, it is necessary to secure a large exposure amount of the image sensor 16 in order to achieve proper exposure. That is, at low illuminance, the shutter (aperture) 44 is set to the open side or the charge accumulation time is set longer, but the shutter (aperture) 44 is opened at the time of moving image shooting in a low illuminance scene where noise occurs. In this state, the charge accumulation time is set to a time (multiframe exposure) over a plurality of frames.

  However, moving image shooting by multi-frame exposure has the disadvantage that the blurring of an image of one frame is enlarged and the frame rate of the moving image is also slowed, so that it becomes weak against moving subjects and camera shake during shooting.

  Therefore, in order to compensate for the disadvantages of such multi-frame exposure, the imaging apparatus 1E selects multi-frame exposure and cyclic filter processing according to conditions to reduce noise during moving image shooting. For this selection, either “multi-frame exposure” or “circular filter processing” can be designated on the menu screen displayed on the LCD monitor 42, and “automatic setting” for automatically setting these two processes is set. It is also possible to specify.

  It should be noted that the above circulation filter processing is not limited to the case where the AF evaluation value is used, but the camera shake detection value described in the second embodiment, the focal length of the photographing lens described in the third embodiment, and the fourth embodiment. You may perform the cyclic | annular filter process using the calculated value of SS demonstrated by the form.

<Operation of Imaging Device 1E>
FIG. 8 is a flowchart showing the basic operation of the imaging apparatus 1E. This operation is executed by the camera control unit 40E.

  In steps S81 to S84, operations similar to those in steps S2 to S5 shown in the flowchart of FIG. 4 are performed.

  In step S85, it is determined whether or not moving image shooting is set. Specifically, it is determined whether or not the mode changeover switch 12 is selected in the moving image mode. If the moving image shooting is set, the process proceeds to step S86. If the moving image shooting is not set, the process proceeds to step S92.

  In steps S86 to S87, operations similar to those in steps S6 to S7 shown in the flowchart of FIG. 4 are performed.

  In step S88, it is determined whether the focal length of the taking lens 10 is set to telephoto. This telephoto setting means that it is set to 85 mm or more, preferably 100 mm or more in terms of 135 mm, for example.

  In the case of the circular filter processing, the blur amount (flow amount) of the image can be controlled by the feedback coefficient. On the other hand, when performing multi-frame exposure at a telephoto position, the exposure time of each frame becomes long, so that image blurring is likely to occur, and the frame rate is also slowed down, so that image distortion due to blurring is also noticeable.

  Conversely, in the case of a wide angle of view that is not telephoto, subject blurring is less likely to occur, so multi-frame exposure that lowers the effective sensitivity increases the quality of each frame and improves still image quality.

  For the reasons described above, in step S88, it is determined whether or not the telephoto setting has been made.

  In step S88, if the telephoto is set, the process proceeds to step S90. If the telephoto is not set, the process proceeds to step S91.

  In step S89, it is determined whether the circulation filter process is manually selected as the noise process during moving image shooting. If cyclic filter processing is selected, the process proceeds to step S90. If multiframe exposure is selected instead of cyclic filter processing, the process proceeds to step S91.

  In step S90, the circulation filter process based on the AF evaluation value described in the first embodiment is performed.

  In step S91, the above-described multiframe exposure is performed.

  In steps S92 to S95, operations similar to those in steps S15 to S18 shown in the flowchart of FIG. 4 are performed.

  As described above, in the imaging apparatus 1E, the noise processing at the time of shooting two moving images is selected in consideration of the presence / absence of the telephoto setting, so that the optimum noise reduction processing can be performed while preventing the occurrence of errors.

  Note that the imaging apparatus 1E according to the fifth embodiment may perform the operation shown in the flowchart of FIG. 9 described below.

  In the flowchart shown in FIG. 9, the operation in step S96 is added to the flowchart shown in FIG.

  In this step S96, it is determined whether the fixing of the tripod has been detected. Specifically, it is determined whether or not the tripod detector 51 detects that the tripod is attached to the tripod hole 52. Here, when fixing of the tripod is detected, the process proceeds to step S90, and the circulation filter process is performed. On the other hand, when the fixing of the tripod is not detected, the process proceeds to step S91 and multiframe exposure is performed.

  As described above, whether or not the tripod is attached is determined, and when the fixing of the tripod is detected, the cause of the camera shake is eliminated, so that appropriate noise removal can be performed by multi-frame exposure.

<Sixth Embodiment>
An imaging apparatus 1F according to the sixth embodiment of the present invention has a configuration similar to that of the imaging apparatus 1A of the first embodiment as shown in FIGS. 1 and 2, but the camera control unit is different. Yes.

  That is, in the camera control unit 40F of the imaging apparatus 1F, a program for selecting two noise reduction processes described below is stored in the memory.

  For the two noise reduction processes, the cyclic filter process based on the AF evaluation value described in the first embodiment and the noise reduction process by multi-frame exposure described in the fifth embodiment can be selected. The imaging apparatus 1F can automatically switch between the two noise reduction processes according to the movement of the subject.

  In the above-described circulation filter processing, not only the AF evaluation value is used, but the camera shake detection value described in the second embodiment, the focal length of the photographing lens described in the third embodiment, and the fourth embodiment are described. You may perform the cyclic | annular filter process using the calculated value of SS demonstrated by the form.

<Operation of Imaging Device 1E>
FIG. 10 is a flowchart showing the basic operation of the imaging apparatus 1F. This operation is executed by the camera control unit 40F.

  In steps S101 to S107, operations similar to those in steps S81 to S87 shown in the flowchart of FIG. 8 are performed.

  In step S108, it is determined whether a moving object has been detected. Specifically, it is determined whether or not the subject movement is detected in the distance measurement calculation / motion detection unit 36.

  The reason for determining the detection of a moving object is to select a circulation filter process that is strong against the movement of the subject in the case of a moving object, and to select multi-frame exposure if it is not a moving object, that is, a stationary subject. As a result, blurring due to movement of the subject during moving image shooting can be reduced.

  In this step S108, when a moving body is detected, it progresses to step S110, and when a moving body is not detected, it progresses to step S111.

  In steps S109 to S115, operations similar to those in steps S89 to S95 shown in the flowchart of FIG. 8 are performed.

  As described above, in the imaging apparatus 1F, noise processing at the time of shooting two moving images is selected in consideration of detection of moving objects, so that it is possible to perform optimal noise reduction processing while preventing occurrence of errors.

<Seventh embodiment>
An imaging apparatus 1G according to the seventh embodiment of the present invention has a configuration similar to that of the imaging apparatus 1A of the first embodiment as shown in FIGS. 1 and 2, but the camera control unit is different. Yes.

  That is, in the camera control unit 40G of the imaging apparatus 1G, a program for selecting two noise reduction processes described below is stored in the memory.

  For the two noise reduction processes, the cyclic filter process based on the AF evaluation value described in the first embodiment and the noise reduction process by multi-frame exposure described in the fifth embodiment can be selected. The imaging device 1G can switch between the two noise reduction processes according to the setting state of the shooting mode (sport mode, night view mode, macro mode).

  With regard to the above shooting modes, for example, on the menu screen displayed on the LCD monitor 42, each of the shooting modes “sport mode”, “night view mode” and “macro mode” can be selected by operating the frame advance / zoom switch 15. It can be done.

  It should be noted that the above circulation filter processing is not limited to the case where the AF evaluation value is used, but the camera shake detection value described in the second embodiment, the focal length of the photographing lens described in the third embodiment, and the fourth embodiment. You may perform the cyclic | annular filter process using the calculated value of SS demonstrated by the form.

<Operation of Imaging Device 1G>
FIG. 11 is a flowchart showing the basic operation of the imaging apparatus 1G. This operation is executed by the camera control unit 40G.

  In steps S121 to S127, operations similar to those in steps S81 to S87 shown in the flowchart of FIG. 8 are performed.

  In step S128, it is determined whether the sport mode is set. Specifically, it is determined whether or not “sport mode” is selected on the menu screen displayed on the LCD monitor 42.

  Here, the setting of the sport mode is determined because the high-speed shutter is prioritized when the sport mode is selected, so that the exposure time is determined by the cyclic filter process without performing multi-frame exposure over a plurality of frames. This is for noise removal.

  If the sport mode is set in step S128, the process proceeds to step S129. If the sport mode is not set, the process proceeds to step S133.

  In step S129, it is determined whether the night view mode is set. Specifically, it is determined whether or not “night scene mode” is selected on the menu screen displayed on the LCD monitor 42.

  Here, the setting of the night view mode is generally determined when a night view mode in which camera shake is likely to occur is selected, since a tripod is assumed to be used, noise removal is performed by multi-frame exposure instead of cyclic filter processing. This is because it is preferable.

  In step S129, if the night view mode is set, the process proceeds to step S133. If the night view mode is not set, the process proceeds to step S130.

  In step S130, it is determined whether the macro mode is set. Specifically, it is determined whether or not “macro mode” is selected on the menu screen displayed on the LCD monitor 42.

  Here, the setting of the macro mode is generally determined when a tripod is used when a macro mode that is likely to cause camera shake is selected. Therefore, noise removal is performed by multi-frame exposure instead of cyclic filter processing. This is because it is preferable.

  In step S130, if the macro mode is set, the process proceeds to step S133. If the macro mode is not set, the process proceeds to step S132.

  In steps S131 to S137, operations similar to those in steps S89 to S95 shown in the flowchart of FIG. 8 are performed.

  As described above, in the imaging apparatus 1G, noise processing at the time of shooting two moving images is selected in consideration of the set shooting mode, and thus it is possible to perform optimal noise reduction processing while preventing occurrence of errors.

<Modification>
In each of the above-described embodiments, it is not essential to perform the cyclic filter process when the AGC setting gain is a predetermined value (Ref1) or more, and it is not necessary to perform the cyclic filter process based on the subject brightness. You may judge. In this case, the process moves to the circulation filter process in the case of a shooting condition where the subject brightness is a predetermined value or less.

It is a figure which shows the principal part structure of 1 A of imaging devices which concern on 1st Embodiment of this invention. It is a figure which shows the functional block of 1 A of imaging devices. It is a figure for demonstrating the noise removal at the time of the video recording in 1 A of imaging devices. It is a flowchart which shows basic operation | movement of 1 A of imaging devices. It is a flowchart which shows the basic operation | movement of the imaging device 1B which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the fundamental operation | movement of 1 C of imaging devices which concern on 3rd Embodiment of this invention. It is a flowchart which shows the basic operation | movement of imaging device 1D which concerns on 4th Embodiment of this invention. It is a flowchart which shows the basic operation | movement of the imaging device 1E which concerns on 5th Embodiment of this invention. It is a flowchart which shows other operation | movement of the imaging device 1E. It is a flowchart which shows the basic operation | movement of the imaging device 1F which concerns on 6th Embodiment of this invention. It is a flowchart which shows the basic operation | movement of the imaging device 1G which concerns on 7th Embodiment of this invention. It is a block diagram for demonstrating the feedback filter process which concerns on a prior art.

Explanation of symbols

1A to 1G Imaging device 2 Signal processing unit 3 Image processing unit 10 Shooting lens 14 Camera shake correction switch 24 Black correction unit 25 Scratch correction unit 26, 82 Circulation filter unit 27 Shading correction unit 28 White balance (WB) amplifier units 31, 81 Pixel interpolation unit 36 Distance calculation / motion detection unit 40 Camera control unit 42 LCD monitor 48 Camera shake correction actuator driver 49 Camera shake sensor 51 Tripod detection unit 261 Correlation detection unit

Claims (5)

An imaging device,
(a) moving image capturing means for capturing an image of a subject and sequentially generating frames constituting the moving image as RAW image data;
(b) When a predetermined shooting condition is satisfied, the difference between the RAW image data of the current frame generated by the moving image shooting means and the RAW image data of the previous frame is multiplied by a feedback coefficient to multiply the RAW of the current frame. Filter means for performing filter processing to be subtracted from the image data;
With
The filter means includes
(b-1) a setting means for setting the feedback coefficient based on the control information related to shooting,
An imaging device comprising: The imaging device according to claim 1,
(c) first correction means for performing black level correction and / or pixel defect correction on the RAW image data of the frame generated by the moving image photographing means before the filter processing by the filter means;
An image pickup apparatus further comprising: In the imaging device according to claim 1 or 2,
(d) second correction means for performing shading correction and / or white balance correction on the RAW image data of the frame subjected to the filter processing by the filter means;
An image pickup apparatus further comprising: An imaging device,
(a) a moving image shooting step of capturing a subject and sequentially generating frames constituting the moving image as RAW image data;
(b) a first correction step of performing black level correction and / or pixel defect correction on the RAW image data of the frame generated in the moving image shooting step;
(c) When a predetermined condition is satisfied, the difference between the RAW image data of the current frame that has been corrected in the first correction step and the RAW image data of the previous frame is set based on control information related to shooting. A filtering step of performing a filtering process that multiplies the feedback coefficient to subtract from the raw image data of the current frame;
(d) a second correction step of performing shading correction and / or white balance correction on the RAW image data of the frame subjected to the filtering process in the filtering step;
An imaging apparatus comprising means for executing each of the above. An image processing method comprising:
(a) a moving image shooting step of capturing a subject and sequentially generating frames constituting the moving image as RAW image data;
(b) a first correction step of performing black level correction and / or pixel defect correction on the RAW image data of the frame generated in the moving image shooting step;
(c) When a predetermined condition is satisfied, the difference between the RAW image data of the current frame that has been corrected in the first correction step and the RAW image data of the previous frame is set based on control information related to shooting. A filtering step of performing a filtering process that multiplies the feedback coefficient to subtract from the RAW image data of the current frame;
(d) a second correction step of performing shading correction and / or white balance correction on the RAW image data of the frame subjected to the filtering process in the filtering step;
An image processing method comprising:

Images