JP2021026180A - Image shake correction device and imaging apparatus - Google Patents

Abstract

To improve the accuracy in detecting parallel shake.SOLUTION: An image shake correction device comprises: first parallel shake detection means that detects parallel shake based on a motion vector acquired based on the difference between a plurality of images, and an angular shake signal indicating the angular shake of the apparatus; second parallel shake detection means that detects parallel shake based on information different from that for the first parallel shake detection means; acquisition means that acquires information on a subject in an image; and control means that controls image shake correction of the image. The control means selects, from the first and second parallel shake detection means, shake detection means used for acquisition of a parallel shake correction amount based on the information on the subject acquired by the acquisition means. The control means then acquires the parallel shake correction amount based on a parallel shake amount detected by the selected parallel shake detection means, and controls the image shake correction based on the parallel shake correction amount.SELECTED DRAWING: Figure 2

Description

The present invention relates to an anti-vibration control technique for correcting image shake caused by camera shake or the like.

There are known image pickup devices and interchangeable lens devices including an image shake correction device having an optical image shake correction function, which corrects image shake caused by shake of the image pickup device by using a movable lens or an image sensor. In addition, imaging with an image shake correction device having an electronic image shake correction function that corrects image shake caused by shake of the image pickup device by changing the cutout position of each frame of the moving image or performing image processing. The device is also known.

The image shake correction device detects angular shake caused by rotational shake of the image pickup device due to camera shake and parallel shake (also called shift shake) caused by shake of the image pickup device in the shift direction, and shakes according to the detected shake amount. Perform the correction.

In general shooting scenes, the influence of angle shake is dominant, and techniques for reducing the influence of angle shake have advanced. In particular, in recent years, improvements in the performance of angular velocity sensors have made it possible to detect angular velocity in a wider band than before, especially in a low frequency band. By utilizing this, the correction performance of the angle shake of the camera has been improved, and it has become possible to shoot in a longer time. On the other hand, when shooting at longer seconds, parallel runout, which was not a problem in the past, has become noticeable in some cases. As a method for correcting parallel runout, Patent Document 1 discloses a technique for determining the amount of parallel runout from the ratio of the output of the accelerometer and the output of the angular velocity meter and controlling the runout correction unit.

JP-A-2007-171786

With the ability to shoot in longer seconds, camera shake in the lower frequency band has become noticeable in some cases. However, in general, accelerometers are often unsuitable for detecting camera shake in the low frequency band. Therefore, as in Patent Document 1, when the amount of parallel runout is obtained from the ratio of the output of the accelerometer and the output of the angular velocity meter, the detection accuracy of the parallel runout is lower in the low frequency band than in the high frequency band, and the low frequency In some cases, parallel runout was noticeable.

An object of the present invention is to improve the detection accuracy of low-frequency parallel runout.

The image shake correction device according to the present invention is a first parallel runout detection device that detects parallel runout based on a motion vector acquired based on the difference between a plurality of images and an angle runout signal indicating the angular runout of the device. Control means, a second parallel runout detecting means for detecting parallel runout based on information different from that of the first parallel runout detecting means, an acquisition means for acquiring information on a subject of an image, and image shake correction of an image. The control means includes a control means, and the control means is a runout detection means used for acquiring a parallel runout correction amount from the first and second parallel runout detection means based on the information of the subject acquired by the acquisition means. Is selected, the parallel runout correction amount is acquired based on the parallel runout amount detected by the selected parallel runout detection means, and the image runout correction is controlled based on the parallel runout correction amount.

Other aspects of the present invention will be described in the form of carrying out the invention.

According to the present invention, it is possible to improve the detection accuracy of parallel runout in the image runout correction device.

System diagram of the imaging system 2000 Block diagram showing the calculation method of parallel runout The figure which shows the composition of the image plane runout Flow chart of correction coefficient circle calculation performed by camera system control unit 5 Block diagram showing the calculation method of parallel runout Block diagram showing the calculation method of parallel runout

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate description is omitted.

An imaging system 2000 including an image shake correction device, which is an embodiment of the present invention, will be described with reference to FIG. FIG. 1 is a diagram showing a configuration of an imaging system 2000 of this embodiment. FIG. 1A is a central sectional view of the imaging system 2000. FIG. 1B is a block diagram showing an electrical configuration of the imaging system 2000. Those having the same code number in FIGS. 1 (a) and 1 (b) correspond to each other.

In FIG. 1A, the imaging system 2000 includes an interchangeable lens 2 and a camera 1, and the interchangeable lens 2 and the camera 1 are electrically and mechanically detachably detachable via a mount (not shown) provided with an electrical contact 11. It is connected to the. The interchangeable lens 2 includes a photographing optical system 3 including a plurality of optical elements. The interchangeable lens 2 includes a lens driving means 13 so that the light beam from the subject transmitted through the photographing optical system 3 of the interchangeable lens 2 is satisfactorily imaged on the image pickup element 6 of the camera 1. The lens driving means 13 drives the photographing optical system 3 by receiving a control signal from the lens system driving unit 12, and various actuators can be used as the lens driving means 13. The photographing optical system 3 includes a focus lens for adjusting the focus, an aperture, an image shake correction lens 9 for correcting image shake by moving the optical axis 4, and the like.

Further, in the camera body 1, the shutter mechanism 17 is driven and controlled by the shutter driving means 18 so that the exposure is performed at the shooting seconds set by the photographer or determined by the camera system control unit 5. The image sensor 6 is an image sensor that photoelectrically converts an image formed on an image plane and outputs an image signal.

In FIG. 1A, the z-axis is an axis parallel to the optical axis 4. The x-axis and y-axis are orthogonal to the z-axis and are parallel to each side of the image sensor 6. Further, for the sake of clarity, the origin of the xyz space is written outside the image pickup system 2000 in FIGS. 1 to 4, but the origin is actually located at the center of the image pickup device 6.

The camera system control unit 5 determines the amount of shake correction for reducing the influence of shake based on the signals output from the release detection unit 7, the operation detection unit 8, the acceleration detection means 16, the angular velocity detection means 15, and the image sensor 6. Calculate. Then, the shake correction amount is output to the shake correction means 14 and the lens system control unit 12. The operation detection unit 8 detects the photographer's operations such as setting the shutter speed, F value, and height.

The angular velocity detecting means 15 detects an angular velocity around the x-axis, an angular velocity around the y-axis, and a rotational component around the z-axis. As the angular velocity detecting means 15, for example, a gyro sensor can be used.

The acceleration detecting means 16 detects acceleration, which is a translational component in the x-axis direction, the y-axis direction, and the z-axis direction. The detection method is not particularly limited, and an acceleration sensor such as a capacitance type or a piezoresistive type can be used.

The shake correction means 14 moves the image pickup element 6 so that the position of the image pickup element 6 moves in a plane perpendicular to the optical axis 4 based on the shake correction amount acquired from the camera system control unit 5. The camera system control unit 5 controls the image shake correction by moving the image sensor 6 by outputting the shake correction amount to the shake correction means 14.

The camera 1 is provided with a release detection unit 7, and the release detection unit 7 detects a release open / close signal (not shown) and sends the detected open / close signal to the camera system control unit 5. There are two types of open / close signals detected by the release detection unit 7. The release detection unit 7 can detect a two-stage switch, a switch SW1 that is turned on by half-pressing the release button and a switch SW2 that is turned on by fully pressing the release button. The lens system control unit 12 outputs a command based on the runout correction amount to the lens driving means 13. The lens driving means 13 moves the image shake correction lens 9 in the x direction and the y direction of FIG. 1A, and performs runout correction in consideration of both angular runout and parallel runout. The shake correction means 14 moves the image sensor 6 in the x-direction and the y-direction of FIG. 1, and performs run-out correction in consideration of both angular run-out and parallel run-out. Further, the runout correction means 14 rotates the image sensor 6 around the z-axis to perform runout correction in consideration of both the angular runout and the parallel runout caused by the rotational movement around the z-axis.

The correction method based on the runout correction amount received from the camera system control unit 5 is not limited to this method, and other forms may be used. For example, there is a method of using so-called electronic image shake correction, which reduces the influence of shake by changing the cutout position of each shooting frame generated based on the output of the image sensor 6. Moreover, you may use the correction method which combined them appropriately.

When the image pickup system 2000 is shaken, the positions of the image pickup element 6 and the subject image are displaced, and image plane shake occurs. Imaging caused by translation of the subject image on the image sensor 6 due to translational movement with respect to the xy plane is parallel to the image plane, and imaging is caused by angular movement due to rotational movement with the x, y, and z axes as the rotation axes. The deflection of the subject image on the element 6 is referred to as an image plane angle deflection.

FIG. 2 is a block diagram showing a configuration of a parallel runout detection unit 5a in the camera system control unit 5 that detects parallel runout. The angular velocity detecting means 15 of FIG. 2 is the angular velocity detecting unit 15 of FIG. 1 (b). The acceleration detecting means 16 of FIG. 2 is the acceleration detecting means 16 shown in FIG. 1 (b). The detection results of the angular velocity detecting means 15 and the acceleration detecting means 16 are input to the parallel runout detecting unit 5a of the camera system control unit 5.

In this embodiment, the parallel runout detection unit 5a includes a first parallel runout detection means, a second parallel runout detection means, and a determination unit 218 for selecting a runout detection detection means used for acquiring the parallel runout correction amount. Be prepared. The first parallel runout detecting means detects the parallel runout based on the runout angle based on the detection result of the angular velocity detecting means 15 and the runout speed based on the detection result of the motion vector detecting means 201. The detection result of the first parallel runout detecting means may be referred to as a first parallel runout amount. On the other hand, the second parallel runout detecting means detects the parallel runout based on the runout angle based on the detection result of the angular velocity detecting means 15 and the runout speed based on the detection result of the acceleration detecting means 16. The detection result of the second parallel runout detecting means may be referred to as a second parallel runout amount.

The first parallel runout detecting means will be described. The first parallel runout detecting means has a runout angle based on the detection result of the angular velocity detecting means 15 and a runout speed based on the detection result of the motion vector detecting means 201, and the parallel runout with respect to the detection result of the angular velocity detecting means. Get the ratio (called the correction coefficient) of. Then, the first parallel runout amount is acquired by multiplying the acquired correction coefficient by the detection result of the angular velocity detecting means 15. The correction coefficient is the sum of the angular runout image plane velocity corrected by the lens driving means 13 and the runout correction means 14, the uncorrected angular runaway image plane velocity, and the uncorrected parallel runout image plane velocity. Obtained based on the ratio of the image plane runout velocity to the parallel runout image plane velocity.

Each functional block constituting the first parallel runout detecting means will be described.

The motion vector detecting means 201 detects the velocity of the image plane runout in the x, y, and z-axis directions of the image plane runout. Specifically, the motion vector detecting means 201 detects the motion vector from the difference between a plurality of images (the amount of movement of the feature points between frames) based on the image signal output from the image sensor 6, thereby detecting the feature points. Detect the movement speed. Therefore, when the subject included in the image signal has a repetitive pattern, or the contrast or brightness is low, the feature points cannot be accurately captured, and accurate detection may be difficult. Further, even when a completely different subject is moving between frames, the amount of movement of the feature point cannot be calculated, which makes accurate detection difficult. When the subject is stationary, detecting the moving speed of the feature point is to detect the speed of the image plane shake generated by the shake of the image pickup apparatus. When the subject is moving, it is possible to detect the moving speed of the feature points. The moving speed of the feature points is the speed of the image plane shake caused by the shake of the imaging device and the speed of the image plane shake caused by the movement of the subject. It will be.

The image plane runout speed detected by the motion vector detecting means 201 is a mixture of the angular shake image plane speed and the parallel runout image plane speed. The angular runout image plane velocity is the velocity of the runout generated on the image plane due to the angular runout, and the parallel runout image plane velocity is the runout speed generated on the image plane due to the parallel runout.

The angular velocity image plane velocity calculator 204 calculates the angular velocity image plane velocity by multiplying the output signal of the angular velocity detecting means 15 by the focal length and outputs it to the parallel runout extractor 207. The runout correction amount detecting means 208 detects the angular runout image plane velocity corrected by the lens driving means 13 and the runout correction means 14, and outputs the speed to the parallel runout extractor 207.

The parallel runout extractor 207 extracts the parallel runout image plane velocity from the output of the motion vector detecting means 201. Specifically, the parallel runout extractor 207 is the output of the runout correction amount detecting means 208 from the output of the angle runout image plane velocity calculator 204, which is the angle runout image plane velocity corrected by the runout correction function. Is subtracted. As a result, the uncorrected angular deflection image plane velocity, that is, the image plane velocity of the angular runout reflected in the motion vector is calculated. Then, the parallel runout image plane velocity is extracted by subtracting the uncorrected angular runout image plane velocity from the output of the motion vector detecting means 201.

FIG. 3 is an image diagram of the calculation performed by the parallel runout image plane velocity extractor 207. The sum of the angular runout image plane velocity (corrected) 301, the angular runaway image plane velocity (uncorrected) 302, and the parallel runout image plane velocity (uncorrected) 303 in FIG. 3 did not perform any runout correction. It is the speed of image plane deflection that sometimes appears on the image plane of the image sensor 6. The angular runout image plane velocity (corrected) 301 is the output of the runout correction amount detecting means 208. The output of the angular deflection image plane velocity calculator 204 is equal to the sum of the angular deflection image plane velocity (corrected) 301 and the angular deflection image plane velocity (uncorrected) 302. The angular deflection image plane velocity (uncorrected) can be calculated from the difference between the outputs of the angular deflection image plane velocity calculator 204 and the shake correction amount detecting means 208 in FIG. The output of the motion vector detecting means 201 of FIG. 2 is equal to the sum of the image plane angular runout (uncorrected) 302 and the image plane parallel runout (uncorrected) 303 of FIG. From the sum of the angular runout image plane velocity (uncorrected) 302 and the parallel runout image plane velocity (uncorrected) 303, which are the outputs of the motion vector detecting means 201, the angular runout image plane velocity calculator 204 and the runout correction amount detecting means The angular deflection image plane velocity (uncorrected) 302, which is the difference between the outputs of 208, is subtracted. As a result, the angular runout component is removed from the motion vector, and the parallel runout image plane velocity (uncorrected) 303, which is the image plane velocity of the parallel runout component, is calculated.

Returning to the description of FIG. The first LPF211 applies a low-pass filter to the output signal of the parallel runout image plane velocity extractor 207 to extract a specific band and output it. The band to be extracted is a band of 10 Hz or less including camera shake. When shooting with a long exposure, the dominant band is higher than the low frequency, so the filter band is changed according to the conditions, such as extracting the band of 1 Hz or less. The second LPF 212 applies a low-pass filter to the output signal of the angular velocity detecting means 202 to extract a specific band and output it. The band extracted by the second LPF212 is basically the same as that of the first LPF211. The comparator 216 calculates and outputs the first correction coefficient from the ratio of the output signals of the first LPF 211 and the second LPF 212 and outputs it. In other words, the first correction coefficient is a signal obtained by LPF-processing the image plane velocity of parallel runout acquired based on the outputs of the motion vector detecting means and the angular velocity detecting means, and a signal obtained by LPF-processing the angular velocity. Obtained based on the ratio. The parallel runout calculation unit 220 can acquire the first parallel runout amount by multiplying the first correction coefficient by the output of the angular velocity detecting means 15. As described above, the motion vector detecting means 201, the parallel runout image plane velocity extractor 207, the angular runout image plane velocity calculator 204, the runout correction amount detecting means 208, the first and second LPF211 and 212, and the first comparator. 216, the parallel runout calculation unit 220 functions as the first parallel runout detection means.

Next, each functional block constituting the second parallel runout detecting means will be described.

In this embodiment, the second parallel runout detecting means acquires the second correction coefficient, which is the ratio of the parallel runout to the detection result of the angular velocity detecting means, based on the outputs of the angular velocity detecting means 15 and the acceleration detecting means 16. To do.

The output of the acceleration detecting means 16 is input to the first BPF 213. The first BPF 213 applies a bandpass filter to the output signal of the acceleration detection means 16 to extract a signal in a specific band, and outputs the extracted signal to the integrator 205. The integrator 205 integrates the output signal of the first BPF 213 to calculate a velocity signal, and outputs the velocity signal to the image magnification multiplier 401. The image magnification multiplier 401 converts the parallel runout speed into the image plane parallel runout speed, which is the parallel runout speed on the image plane, by multiplying the output result of the integrator 205 by the image magnification, and outputs the parallel runout speed to the second comparator 217. To do. The location of the image magnification multiplier 401 is not particularly limited, and this conversion may be performed at any processing timing of the first BPF 213 to the comparator 217. Further, the image magnification is a focal length / subject distance, and the subject distance can be estimated from, for example, the position of the focus lens (and the position of the zoom lens if any).

On the other hand, the output of the angular velocity detecting means 15 is input to the second BPF 209. The second BPF 209 applies a bandpass filter to the output signal of the angular velocity detecting means 15 to extract a specific band, and outputs the extracted signal to the HPF 210. The band of the filter of the second BPF 209 is basically the same as that of the first BPF 213. The HPF 210 applies a high-pass filter to the output signal of the second BPF 209, so that the HPF 210 is in phase with the output signal of the integrator 205, which is a velocity signal based on acceleration, and is output to the second comparator 217. When performing complete integration with the integrator 205, the processing of HPF210 becomes unnecessary. The second comparator 217 calculates and outputs the second correction coefficient from the ratio of the output signal of the HPF 210 and the output signal of the image magnification multiplier 401, which is the speed based on the angular velocity signal. In other words, the second correction coefficient is acquired based on the ratio of the angular velocity based on the angular velocity signal to the image plane (translation) velocity based on the acceleration signal. The parallel runout calculation unit 220 can acquire the second parallel runout amount by multiplying the second correction coefficient by the output of the angular velocity detecting means 15. As described above, the angular velocity detecting means 15, the acceleration detecting means 16, the first and second BPF 213, 209, the HPF 210, the integrator 205, the image magnification multiplier 401, the second comparator 217, and so on. The parallel runout calculation unit 220 functions as a second parallel runout detection means.

The parallel runout detection unit 5a of the present embodiment determines which of the first and second correction coefficients is to be output to the parallel runout calculation unit 220. As a result, it is possible to select which of the first and second parallel runout detection means is used to output the parallel runout amount acquired to the correction amount calculation means (not shown) in the camera system control unit. To do. This selection is performed by the determination unit 218 based on the subject information acquired by the subject information detecting means 214 and the shake information detected by the shake information detecting means 215.

The subject information detecting means 214 detects and outputs the state of the subject to be photographed by the imaging device 200. The subject information includes the brightness of the subject, the contrast of the subject, the pattern of the subject, the moving speed of the main subject whose focus is adjusted, and the moving speed of the background (subject other than the main subject).

The runout information detecting means 215 receives the angular runout image plane velocity (uncorrected) 302 and the parallel runout image plane velocity (uncorrected) 303 as inputs from the parallel runout image plane velocity extractor 207, and outputs them as they are.

The determination unit 218 receives the output signal of the subject information detecting means 214, the output signal of the runout information detecting means 215, the output signal of the comparator 216, and the output signal of the comparator 217 as inputs. As described above, the determination unit 218 determines which output signal of the comparator 216 or the comparator 217 is used. The determination unit 218 basically selects the first correction coefficient acquired based on the motion vector detection means 201. This is because the signal detected by the motion vector detecting means 201 has a wider measurable band than the signal detected by the acceleration detecting means 16. The band referred to here refers to a camera shake band, which is generally a band of 10 Hz or less. However, there may be a situation where the detection accuracy of the motion vector detecting means 201 is lowered. Therefore, in the present embodiment, when the motion vector detection accuracy is low, the second parallel runout detection means is selected by selecting the second correction coefficient acquired based on the output signal of the acceleration detection means 16. To do.

The determination unit 218 determines the reliability of the motion vector based on the output signal of the subject information detecting means 214. If it is determined that the reliability is higher than the first threshold value, the first correction coefficient is selected, and if it is determined that the reliability is equal to or lower than the first threshold value, the second correction coefficient is selected. The reliability of the motion vector is acquired based on the information detected by the subject information detecting means 214 described above. It is determined that the reliability of the motion vector is lowered when the brightness of the subject is low, the contrast of the subject is low, the moving speed of the subject is fast, or the subject has a repeating pattern. Specifically, if the brightness of the subject is less than or equal to the second threshold value, the contrast of the subject is less than or equal to the third threshold value, or the moving speed of the subject is less than or equal to the fourth threshold value, the reliability Is determined to be equal to or less than the first threshold value. Further, even when it is determined that the subject has a repeating pattern, it is determined that the reliability is equal to or less than the first threshold value.

The method of acquiring the moving speed of the subject is not particularly limited, but for example, by estimating the parallel runout image plane velocity (uncorrected) from the angular shake image plane velocity (corrected) + the angular shake image plane velocity (uncorrected), It is possible to estimate whether or not the moving speed of the subject is faster than the fourth threshold value. The output of the motion vector detecting means 201 is larger than the value obtained by adding the angular deflection image plane velocity (uncorrected) and the parallel deflection image plane velocity (uncorrected) estimated from the angular deflection image plane velocity (corrected + uncorrected). In that case, the main subject may be moving. The estimation of whether or not the main subject is moving is effective when the moving subject is dominant in each shooting frame output by the image sensor 6. Therefore, the subject information detecting means 214 acquires the information of the angular deflection image plane velocity (corrected) 301 + the angular deflection image plane velocity (uncorrected) 302 from the angular deflection image plane velocity calculator 204. Then, by multiplying the acquired angular runout image plane velocity (corrected + uncorrected) by the average ratio of the angular runout image plane velocity and the parallel runout image plane velocity, the parallel runout image plane velocity (uncorrected). ) Is estimated. Then, when the image plane deflection velocity based on the output of the motion vector detecting means 201 is larger than the addition result of the angular deflection image plane velocity (uncorrected) and the estimated value of the parallel runout image plane velocity (uncorrected), It is determined that the moving speed of the subject is faster than the fourth threshold value. Then, information indicating that the moving speed of the subject is faster than the fourth threshold value is output to the determination unit 218. .. The moving speed of the subject is the fourth when the image plane runout speed is larger than the predetermined value or more than the addition result, or the image plane runout speed is higher than the value obtained by multiplying the addition result by the predetermined value. It may be judged that it is faster than the threshold value. The average ratio of the angular shake image plane velocity (corrected + uncorrected) and the parallel shake image plane velocity (uncorrected) differs depending on the shooting seconds (exposure time) at the time of shooting. Therefore, for example, the ratio of the parallel runout image plane velocity / the angular runout image plane velocity is stored in the storage means in the camera body 1 in association with the shooting seconds. Then, the subject information detecting means 214 reads the ratio corresponding to the shooting seconds set by the user or the exposure control unit in the camera system control unit 5 from the storage means, and obtains an estimated value of the parallel runout image plane velocity (uncorrected). Can be obtained. Further, the determination unit 218 has the first and second correction coefficients according to the ratio of the angular runout image plane velocity (uncorrected) and the parallel runout image plane velocity (uncorrected), which is the output of the runout information detecting means 215. Determine which correction coefficient to use. In the determination unit 218, when the parallel runout image plane velocity (uncorrected) is small enough to be buried in the detection error as compared with the angular runout image plane velocity (uncorrected), it is not the first correction coefficient but the second. It is determined to output the correction coefficient to the parallel runout calculation unit 220. As shown in FIG. 3, the parallel runout image plane velocity (uncorrected) is obtained by subtracting the angular runout image plane velocity (uncorrected) from the output of the motion vector detecting means 201. Therefore, when the parallel runout image plane velocity (uncorrected) is small, it may be buried in the detection errors of the motion vector detecting means 201, the angular velocity detecting means 15, and the acceleration detecting means 16. Therefore, the ratio of the angular runout image plane velocity (uncorrected) to the parallel runout image plane velocity (uncorrected) (parallel runout image plane velocity (uncorrected) / angular runout image plane velocity (uncorrected)) is equal to or less than the sixth threshold value. In the case of, it is determined to output the second correction coefficient to the parallel runout calculation unit 220.

When comparing the angular deflection image plane velocity (uncorrected) and the parallel runout image plane velocity (uncorrected), an estimated value can be used instead of the detected value. As described above, the parallel runout image plane velocity (uncorrected) is used using the average ratio of the angular runout image plane velocity (corrected) + the angular runout image plane velocity (uncorrected) and the parallel runout image plane velocity (uncorrected). ) Is estimated. Compare the estimated value of the parallel runout image plane velocity (uncorrected) obtained by the average ratio with the angular runout image plane velocity (uncorrected), and if the ratio is less than or equal to the sixth threshold, the parallel runout image plane velocity It is determined whether (uncorrected) is buried in the detection error, and the second correction coefficient is output.

In addition, it is considered that the reliability of the motion vector decreases even when the image pickup device has a large deflection. Therefore, if the sum of the angular velocity image plane velocity (uncorrected) and the parallel runout image plane velocity (uncorrected) detected by the runout information detecting means 215 or the output of the angular velocity detecting means is equal to or greater than the threshold value, the reliability of the motion vector May be determined to be equal to or less than the first threshold value.

The second correction coefficient acquired based on the acceleration detecting means 16 greatly deviates from the value of the first correction coefficient acquired based on the motion vector detecting means 201, except in situations where the SH shock is large, such as during continuous shooting. It can also be used to check if it is not. For example, when the values of the first correction coefficient and the second correction coefficient are close to each other, it can be said that the reliability of the correction coefficient is high. If the two values are significantly different, it can be said that the correction coefficient is unreliable. The reliability of the correction coefficient can be confirmed by comparing the correction coefficients of both. When the reliability of the correction coefficient is high, the first correction coefficient acquired based on the motion vector detecting means 201, which is said to have a wider band and better detection accuracy, is used. If the correction factor is unreliable, the previously calculated correction factor is used without updating the correction factor. If there is no previously calculated correction coefficient, the correction coefficient can be set to 0 and parallel runout correction can be performed.

The correction coefficient output as a result of determination by the determination unit 218 and the output of the angular velocity detecting means 15 are input to the parallel runout calculation unit 220. As described above, the parallel runout calculation unit 220 calculates the parallel runout by multiplying the input correction coefficient by the angular velocity which is the output of the angular velocity detecting means.

In the processing of the parallel runout calculation unit 220, a signal obtained by applying a high-pass filter or a band-pass filter to the output of the angular velocity detecting means 15 may be used as the angular velocity signal. The extraction band at this time is the same as that of the first LPF211 and the second BPF209. By filtering the output of the angular velocity detecting means 15, it is possible to prevent the non-major band from being amplified.

FIG. 4 is a flow chart of parallel runout acquisition performed by the parallel runout detection unit 5a. In this flow, the first and second correction coefficients are periodically updated, and when a shooting instruction (SW2) is received, a parallel runout image is obtained based on the correction coefficient selected from the first and second correction coefficients. The surface velocity is calculated. The flow of FIG. 4 starts when an aiming signal is input. The aiming signal is determined and input based on a finder signal (not shown), an angular velocity detecting means signal, and a SW1 signal.

At the start of the flow, the angular velocity detecting means 15 detects the angular velocity, the camera control unit 5 acquires a correction amount for correcting the angular velocity based on the detected angular velocity, and the shake correcting means 14 and the lens driving means 13 acquire the correction amount. Runout correction is performed based on the corrected amount. At this stage, the parallel runout has not been corrected yet. When the lens device includes the angular velocity detecting means, the lens system control unit 12 acquires a correction amount for correcting the angular velocity based on the output of the angular velocity detecting means of the lens device, and the lens driving means 13 is acquired. The image shake correction lens 9 may be moved based on the correction amount. Since a known technique can be used for the angle runout correction based on the angular velocity, the details will be omitted.

In step S501, the camera system control unit 5 controls the motion vector detecting means 201, the angular velocity detecting means 15, and the acceleration detecting means 16 to detect the motion vector, the angular velocity, and the acceleration. This process corresponds to the processes of the motion vector detecting means 201, the angular velocity detecting means 15, and the runout detecting means 16 described with reference to FIG.

Next, in step S502, the camera system control unit 5 calculates the first and second correction coefficients. This process corresponds to the process from the input of the detection results of the motion vector, the angular velocity, and the acceleration described with reference to FIG. 2 to the output of the first comparator 216 and the second comparator 217. When the first and second correction coefficients are acquired by the first and second comparators 216 and 217, the process proceeds to step S503.

In step S503, subject information is detected, runout correction information is detected, and the process proceeds to step S504. This processing corresponds to the processing of the shake correction amount detecting means 208 and the subject information detecting means 214 described with reference to FIG.

In step S504, based on the information acquired in step S503, it is determined which of the first and second correction coefficients acquired in step S502 is used to acquire the parallel runout amount. This process corresponds to the process of the determination unit 218 described with reference to FIG. By selecting the correction coefficient in this step, the first and second parallel runout detection means can be selected. When the correction coefficient is selected, the process proceeds to step S504.

In step S504, the release detection unit 7 determines whether or not ON of SW2 is detected. If the ON of SW2 is not detected, the process returns to step S501, and the first and second correction coefficients are acquired and selected based on the newly detected motion vector, angular velocity, and acceleration. On the other hand, when the ON of SW2 is detected, the process proceeds to step S506 to acquire the image plane velocity of parallel runout. This process corresponds to the process of the parallel runout calculation unit 220 described with reference to FIG. This flow ends when the image plane velocity of parallel runout is acquired.

The acquired image plane velocity of the parallel runout is output from the parallel runout detection unit 5a and used for the acquisition of the parallel runout correction amount by the camera system control unit 5. As a method for acquiring the parallel runout correction amount based on the parallel runout by the camera system control unit 5, a known method can be used, and thus details will be omitted. For example, when the parallel runout is corrected only on the camera side, the parallel runout amount obtained by integrating the acquired parallel runout speed in the shooting seconds and the value obtained by filtering the parallel runout amount are used as the parallel runout correction amount. be able to. Further, when the camera and the interchangeable lens share the correction, the value obtained by multiplying the parallel runout amount by the camera share ratio can be used as the parallel runout correction amount on the camera side. When the camera and the interchangeable lens share the correction of parallel runout, the camera system control unit 5 acquires the amount of parallel runout correction for each of the camera and the interchangeable lens, and the amount of parallel runout correction on the interchangeable lens side is passed through the electrical contact 11. It may be transmitted to the lens system control unit 12. Further, the parallel runout speed acquired by the parallel runout detection unit 5a is transmitted to the lens system control unit 12 via the electrical contact 11, and the lens system control unit 12 acquires the parallel runout correction amount based on the parallel runout amount. May be good.

In the second embodiment, an example in which the second parallel runout detecting means has a configuration different from that of the first embodiment will be described. In Example 1, the acceleration detecting means 16 and the angular velocity detecting means 15 were used to acquire the second correction coefficient, and the second parallel runout amount was acquired. However, the method of calculating the parallel runout from the output signal of the acceleration detecting means 16 is not limited to this, and it does not have to be the method of obtaining the correction coefficient. A case where the correction coefficient is not obtained will be described with reference to the block diagram of FIG. Blocks that have the same function as in FIG. 2 are indicated by the same reference numerals. A part of the same processing as in FIG. 2 will be omitted.

The configuration of the first parallel runout detecting means is the same as that of the first embodiment. The parallel runout image plane velocity is extracted from the output signal of the motion vector detection means 201 by processing such as the parallel runout image plane velocity extractor 207, and the first correction coefficient is based on the ratio with the angular runout image plane velocity by the comparator 216. Is calculated. From the first correction coefficient and the output signal of the angular velocity detecting means 15, the first parallel runout calculation unit 230 calculates the first parallel runout amount. The first parallel runout calculation unit 230 is different from the parallel runout calculation unit 220 of the first embodiment in that the second parallel runout amount is not calculated, but the parallel runout amount based on the correction coefficient and the output of the angular velocity detecting means 15 The calculation method is the same as that of the parallel runout calculation unit 220.

On the other hand, the second parallel runout detecting means acquires the parallel runout speed by the strap-down method. First, the output signals of the angular velocity detecting means 15 and the acceleration detecting means 16 are input to the second parallel runout calculation unit 232.

The second parallel runout calculation unit 232 calculates the speed of parallel runout by integrating the output signal of the acceleration detecting means 16, that is, the detected acceleration in the second order. Further, in the second parallel runout calculation unit 232, when integrating the output signal of the acceleration detecting means 16, the output of the angular velocity detecting means 15 is used to cancel the noise due to the influence of gravity or the like. Since the attitude change of the acceleration detecting means 16 can be known by using the output signal of the angular velocity detecting means 15, the gravity influence can be calculated, and the gravity influence is subtracted from the output signal of the acceleration detecting means 16 and then the second-order integration is performed. Outputs the parallel runout speed. If the acceleration detecting means 16 is an acceleration detecting means that does not cause noise due to the influence of gravity, the second parallel runout 232 may simply integrate the input acceleration signal in the second order. Further, the acceleration detecting means 16 is a translation detecting means using inertia. In addition, instead of the acceleration detecting means 16, a translation detecting means for detecting the velocity can also be used. In that case, the parallel runout velocity can be obtained by first-order integrating the velocity signal which is the output of the translational detection means. The output signal of the second parallel runout calculation unit 232 is output to the image magnification multiplier 401. In the image magnification multiplier 401, the parallel runout speed is converted into the second parallel runout image plane speed, which is the parallel runout speed on the image plane, by multiplying the output signal of the second parallel runout calculation unit 232 by the image magnification, and the judgment is made. Output to unit 231.

In the determination unit 231, the parallelism acquired by either the first parallel runout calculation unit 230 or the second parallel runout calculation unit 232 based on the output signal of the subject information detection means 214 and the output signal of the runout information detection means 215. Determine whether to output the runout amount as the final parallel runout amount. The determination method of the determination unit 231 is the same as that of the determination unit 218 of the twelfth embodiment.

In this way, the determination unit 231 of the present embodiment selects which of the first and second parallel runouts is to be the parallel runout amount, so that the first and second parallel runouts From the detecting means, one parallel runout detecting means is selected as the detecting means used for detecting the amount of parallel runout. The parallel runout amount selected by the judgment unit 231 and output from the judgment unit 231 is input to the correction amount acquisition unit (not shown) in the camera system control unit, and the correction amount acquisition unit performs parallel runout based on the parallel runout amount. The correction amount is acquired.

In Examples 1 and 2, one of the first and second parallel runout detecting means was selected to obtain the parallel runout amount. In the third embodiment, an example of acquiring the parallel runout amount by using the first parallel runout detecting means and the second parallel runout detecting means in combination will be described. In this embodiment, among the parallel runouts, the low frequency band is detected by the first parallel runout detecting means and the high frequency band is detected by the second parallel runout detecting means, and the outputs of the two parallel runout detecting means are added. , Get the amount of parallel runout. The accelerometer functioning as the acceleration detecting means 16 has a property of being relatively suitable for detecting high-frequency runout, and the motion vector being relatively suitable for detecting low-frequency runout. Therefore, as described in Examples 1 and 2, not only the output of either the motion vector detecting means 201 or the acceleration detecting means 16 is used, but the parallel runout amount used to acquire the correction amount based on both outputs. The method of obtaining the above will be described with reference to FIG.

The parallel runout detection unit 5c of FIG. 6 acquires the first correction coefficient based on the output of the motion vector detection means 201 and the output of the angular velocity detection means 15, and outputs the angular velocity detection means 15 and the acceleration detection means 16. The point of acquiring the second correction coefficient based on the same is the same as that of the first embodiment. However, the filter characteristics of the LPF and BPF used are different from those of Example 1.

Further, in this embodiment, the output signal of the angular velocity detecting means 15 is also output to the third LPF 243 and the third BPF 244, which is also different from the first embodiment. In the third LPF243, the output signal of the angular velocity detecting means 15 is subjected to a low-pass filter to extract and output a signal in the low frequency band. The cutoff frequency of the third LPF243 is approximately 2 Hz, although it depends on the detection accuracy of the motion vector detecting means 201. The third BPF 244 extracts and outputs a signal in a specific frequency band by applying a bandpass filter to the output signal of the angular velocity detecting means 15. The pass band of the bandpass filter of the third BPF 244 is a band having a higher frequency than the pass band of the third LPF 243 and does not overlap with the pass band of the third LPF 243. When the cutoff frequency of the third LPF243 is set to 2 Hz as described above, the pass band of the third BPF 244 is set to a band higher than 2 Hz. Since the frequency of camera shake is generally 10 Hz or less, the pass band of the third LPF243 is, for example, larger than 2 Hz and 10 Hz or less. If the cutoff frequency of the third LPF243 is 3 Hz, the pass band of the third BPF 244 is set to be larger than 3 Hz and 10 Hz or less. The cutoff frequencies of the fourth LPF247 and the fifth LPF248 are set to the same cutoff frequencies as the third LPF243. The pass band of the bandpass filter of the fourth BPF 245 and the fifth BPF 246 is set to the same band as that of the third BPF 244.

The third parallel runout calculation unit 240 multiplies the first correction coefficient, which is the output signal of the comparator 216, and the angular velocity of the low frequency band, which is the output signal of the third LPF243, to parallelize the low frequency band. The runout image plane velocity is acquired and output to the addition unit 241. As described above, the motion vector detecting means 201, the parallel runout image plane velocity extractor 207, the angular runout image plane velocity calculator 204, the runout correction amount detecting means 208, the third to fifth LPF243, 247, 248, and the first The comparator 216 and the third parallel runout calculation unit 240 function as the third parallel runout detection means.

On the other hand, in the fourth parallel runout calculation unit 242, the second correction coefficient, which is the output signal of the comparator 217, is multiplied by the angular velocity in the high frequency band, which is the output signal of the third BPF 244, and the image magnification to obtain a high frequency. The parallel runout image plane velocity of the band is acquired and output to the addition unit 241. In this way, the third to fifth BPF 244, F245, 246, HPF210, integrator 205, second comparator 217, and the second parallel runout calculation unit 242 detect the second parallel runout. Functions as a means. The function of the image magnification multiplier 401 of the first embodiment is incorporated in the fourth parallel runout calculation unit in this embodiment.

The addition unit 241 calculates the parallel runout image plane velocity used for the correction of the parallel runout by adding the output of the third parallel runout calculation unit 240 and the output of the fourth parallel runout calculation unit 242. Further, the addition unit 241 uses only the output of either the third parallel runout calculation unit 240 or the fourth parallel runout calculation unit 242 based on the output results of the runout information detection means 215 and the subject information detection means 214. In some cases, the judgment is made. The addition unit 241 determines, for example, whether or not to use the first correction coefficient based on the same criteria as the determination unit 218 of the first embodiment. When it is determined that the first correction coefficient is used, the output of the third parallel runout calculation unit 240 and the output of the fourth parallel runout calculation unit 242 are added. On the other hand, when it is determined that the first correction coefficient is not used (corresponding to the case where it is determined that the second correction coefficient is used in Example 1), the fourth correction coefficient is used because only the second correction coefficient is used. The output of the parallel runout calculation unit is taken as the output of the parallel runout detection unit 5c. At this time, since the output of the fourth parallel runout calculation unit does not include runout information in the low frequency band, the lower limit of the pass band of the third to fifth BPFs may be lowered. For example, when it is determined whether or not to use the first correction coefficient is periodically determined based on the outputs of the subject information detecting means 214 and the runout information producing means 215, and it is determined that the first correction coefficient is not used. It is assumed that the first correction coefficient cannot be used next time. Then, from the following calculation, the lower limit value of the pass band of the third to fifth BPFs may be lowered. In this case, if it is determined that the first correction coefficient is used, the lower limit of the pass band of the third to fifth BPFs shall be restored.

[Modification example]
In Examples 1 to 3, the image plane runout velocity was acquired by integrating the detection results by the acceleration detecting means, but the method for acquiring the image plane runout velocity is not limited to this. For example, instead of the acceleration detecting means 16, the image plane runout speed may be acquired by using a means for detecting the speed of the camera, such as a GPS, a TOF sensor, a geomagnetic sensor, or an external camera. As a method of acquiring the image plane runout speed using an external camera, for example, there is a method of detecting parallel runout using triangulation based on information obtained by capturing a subject with a plurality of external cameras. When the acceleration detecting means 16 of the first embodiment is replaced with a sensor that detects the speed of a camera such as a TOF sensor, the output of the speed sensor is multiplied by the first BPF 213 and then multiplied by the image magnification multiplier 401. By doing so, the image plane runout speed can be obtained.

Further, in the first embodiment, the first and second correction coefficients are acquired by the first and second comparators, and then the determination unit 218 determines which correction coefficient is to be used. It was determined which of the second parallel runout detecting means was used to calculate the parallel runout correction amount. However, the method of selecting the parallel runout detecting means used for calculating the parallel runout correction amount is not limited to this. For example, instead of selecting which correction coefficient to use at the stage of acquiring the first and second correction coefficients, the first and second parallel runout amounts are calculated, respectively, as in the second embodiment. , The determination unit 218 may select which parallel runout amount is used to acquire the parallel runout correction amount. Further, before acquiring the first and second correction coefficients, it is selected and selected which parallel runout detecting means is used to acquire the correction amount based on the information detected by the subject information detecting means 214. Only the correction coefficient of the other side may be acquired. In this case, the processing load of the parallel runout detection unit 5a can be reduced, although the selection cannot be made according to the ratio of the angular runout image plane velocity (uncorrected) and the parallel runout image plane velocity (uncorrected).

The same applies to the second embodiment, and the determination unit 231 corrects using any of the parallel runout detecting means based on the information detected by the subject information detecting means 214 before acquiring the first and second parallel runout amounts. You may select whether to acquire the amount and acquire only the selected parallel runout amount.

Further, in Examples 1 to 3, since the parallel runout is corrected only during the exposure for shooting, the parallel runout amount is not acquired before the shooting instruction (SW2) is received, but the shooting instruction is received and the parallel runout amount is acquired. However, the amount of parallel runout may be obtained before the shooting instruction is received. By acquiring the amount of parallel runout before receiving a shooting instruction, it is possible to correct the parallel runout during aiming before shooting.

Further, in Examples 1 to 3, the parallel runout detection units 5a, 5b, and 5c output the parallel runout image plane velocity, and the correction amount acquisition unit in which the parallel runout image plane velocity is input is the parallel runout image plane velocity. The amount of parallel runout was obtained by integrating. However, each of the parallel runout detection units 5a, 5b, and 5c may output the amount of parallel runout instead of the image plane velocity of the parallel runout. You may simply add an integrator that integrates the outputs from the parallel runout calculation unit 220, the determination unit 231 and the addition unit 241. Further, instead of acquiring the correction coefficient based on the parallel runout image plane velocity / angular runout image plane velocity, the correction coefficient is acquired based on the parallel runout amount / angular runout amount, which is the ratio obtained by integrating these speeds, and the angular velocity detection means. The output of may be integrated and then multiplied by the correction factor. Further, in the third embodiment, the parallel runout image plane velocities are added as the outputs of the first parallel runout detection means and the second parallel runout detection means, but the respective parallel runout image plane velocities are integrated and acquired. , The amount of parallel runout may be added.

Further, in Examples 1 to 3, the parallel runout detection units 5a, 5b, and 5c are assumed to be functional blocks in the camera control unit 5. However, the parallel runout detection units 5a, 5b, 5c may be inside the interchangeable lens 2. In this case, since the image pickup signal is acquired by the camera body 1, for example, the motion vector detecting means 201 may be arranged in the camera system control unit 5, and the other functional blocks may be arranged in the lens system control unit 12 of the interchangeable lens 2. Good. Further, the camera body 1 and the interchangeable lens 2 may each include parallel runout detection units 5a, 5b, and 5c. In this case, for example, the camera body 1 and the interchangeable lens 2 each include an angular velocity detecting means 15 and an acceleration detecting means 16, and the motion vector detecting means 201 in the camera system control unit 5 causes a motion vector and a runout based on the motion vector. The image plane velocity is transmitted to the interchangeable lens 2. The interchangeable lens 2 has parallel runout as in Examples 1 to 3 based on the received motion vector and the image plane velocity of the runout based on the received motion vector and the output of the angular velocity detecting means and the acceleration detecting means included in the interchangeable lens 2. You can get the quantity.

1 Camera 2 Interchangeable lens 5 Camera system control unit 6 Imaging element 9 Image shake correction lens 12 Lens system control unit 13 Lens drive means 14 Shake correction means 15 Angle speed detection means 16 Shake detection means 201 Motion vector detection means 15 Angle speed detection means 16 Acceleration Detection means 214 Subject information detection means 215 Shake information detection means 218 Judgment unit

Claims (17)

A motion vector obtained based on the difference between multiple images,
An angular runout signal indicating an angular runout of the device, a first parallel runout detecting means for detecting parallel runout based on the angular runout signal, and
A second parallel runout detecting means that detects parallel runout based on information different from that of the first parallel runout detecting means,
An acquisition method for acquiring information on the subject of an image,
It is equipped with a control means for controlling image shake correction of an image.
The control means selects the runout detection means used for acquiring the parallel runout correction amount from the first and second parallel runout detection means based on the information of the subject acquired by the acquisition means.
The parallel runout correction amount is acquired based on the parallel runout amount detected by the selected parallel runout detection means, and the parallel runout correction amount is acquired.
An image shake correction device characterized in that image shake correction is controlled based on the parallel runout correction amount. The control means
When the reliability of the motion vector is determined based on the information of the subject and it is determined that the reliability is higher than the first threshold value, the first type of vibration detecting means used for acquiring the parallel runout correction amount is used. The image shake correction device according to claim 1, wherein the parallel runout detecting means is selected. The control means
It is determined whether or not the reliability of the motion vector is higher than the first threshold value by determining whether or not the brightness of the subject is higher than the second threshold value based on the information of the subject. 2. The image shake correction device according to claim 2. The control means
It is determined whether or not the reliability of the motion vector is higher than the first threshold value by determining whether or not the contrast of the subject is higher than the third threshold value based on the information of the subject. The image shake correction device according to claim 2 or 3. The control means
It is determined whether or not the reliability of the motion vector is higher than the first threshold value by determining whether or not the moving speed of the subject is higher than the fourth threshold value based on the information of the subject. The image shake correction device according to any one of claims 2 to 4, wherein the image shake correction device is characterized. The control means
The claim is characterized in that it is determined whether or not the reliability of the motion vector is higher than the first threshold value by determining whether or not a repetitive pattern is detected from the image based on the information of the subject. The image shake correction device according to any one of 2 to 5. The control means
When it is determined that the ratio of the parallel runout component of the device to the angular runout component of the device is equal to or less than the threshold value, the second parallel runout detection means is selected as the runout detection means used for acquiring the parallel runout correction amount. The image shake correction device according to any one of claims 1 to 6, wherein the image shake correction device is characterized. One of claims 1 to 7, wherein the second runout detecting means detects parallel runout based on a signal indicating the runout speed of the device and an angular runout signal indicating the angular runout of the device. The image shake correction device according to item 1. The image shake correction device according to claim 8, wherein the signal indicating the runout speed of the device is based on an output from an acceleration sensor that detects the acceleration of the device. The image shake correction device according to claim 8, wherein the signal indicating the runout speed of the device is acquired by using at least one of a GPS, a TOF sensor, a geomagnetic sensor, and an external camera. The second parallel runout detecting means is
The image shake correction device according to any one of claims 1 to 10, wherein the parallel runout is detected based on a signal obtained by second-order integration of the output of an acceleration sensor that detects the acceleration of the device. The first parallel runout detecting means is
The first correction coefficient is acquired based on the angle runout signal and the motion vector, and
Claims 1 to 1, wherein the parallel runout is acquired based on the first correction coefficient and the angle runout signal indicating the angle runout after the angle runout signal used for acquiring the first correction coefficient. The image shake correction device according to any one of No. 11. The first correction coefficient is
The image according to claim 12, wherein the parallel runout component obtained by removing the angle runout component based on the angle runout signal from the motion vector is acquired based on the ratio of the angle runout component. Runout correction device. The control means
The first parallel runout detecting means and the second parallel runout detecting means are selected as the runout detecting means used for acquiring the parallel runout correction amount.
The parallel runout correction amount is acquired based on the addition result of the parallel runout detected by the first parallel runout detecting means and the parallel runout detected by the second parallel runout detecting means. The image shake correction device according to any one of claims 1 to 13. The control means
As a runout detecting means used for acquiring the parallel runout correction amount,
Whether to select the first parallel runout detecting means and the second parallel runout detecting means, or
The image shake correction device according to claim 14, wherein one of the first parallel runout detecting means and the second parallel runout detecting means is selected based on the information of the subject. .. Imaging means and
A motion vector detecting means that detects the motion vector based on a plurality of images acquired by the imaging means, and
An imaging device including the image shake correction device according to any one of claims 1 to 15. The imaging device according to claim 16, wherein the acquisition of the first correction coefficient and the second correction coefficient is executed before the acceptance of the shooting instruction.

Images