JP2019161539A - Imaging device and control method thereof - Google Patents

Abstract

To reduce cost and power consumption at startup and also reduce startup time by estimating an absolute angle value of optical means on the basis of shake information.SOLUTION: A lens barrel 102 of an imaging device includes an imaging optical system and an imaging unit 205. The lens barrel 102 is tilted by a rotating unit 104 in a pitch direction, and the lens barrel 102 is panned by a rotating unit 105 in a yaw direction. The control unit 213 estimates an absolute angle value of the rotating unit 105 based on a mounting angle of an angular velocity meter 106 that is mounted to a fixing part 103 of the imaging device, on the basis of output of the angular velocity meter 106 and a motion vector of an imaged image obtained from an image processing unit 206. The control unit 213 calculates a correction amount on the basis of an estimated absolute angle value and the output of the angular velocity meter 106, thereby driving the rotating units 104, 105 and controlling image shake correction by an image deformation unit 208.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for detecting an absolute angle value of a rotation mechanism unit based on shake information.

  An image blur correction device mounted on an imaging apparatus detects camera shake or the like and suppresses image blur of an image on an image plane using an image blur correction unit. Image blur correction methods include a method of moving a lens and an image sensor on a plane perpendicular to the optical axis, and a method of rotationally driving a lens barrel including the image pickup optical system and the image sensor. In moving image shooting, as a method of correcting various image blurs that occur in the image of each shooting frame output by the image sensor, an electronic method that calculates the amount of image deformation of the shot image and deforms the image so as to cancel it. There is a correction method.

  In order to detect vibration such as camera shake, an angular velocity sensor (gyro sensor) is generally used, and the image blur correction unit is controlled based on the detected angular velocity. There is also a technique for analyzing motion blur between frames and calculating a motion vector. An image region is divided into small blocks, and a global motion vector representing the entire amount of motion is calculated from local motion vectors calculated for each block. Elements other than camera shake due to subject blur can be removed.

  An apparatus disclosed in Patent Literature 1 includes a rotating unit that holds a lens barrel including an imaging optical system and an imaging element, and a main body unit that holds the rotating unit so that the lens barrel can rotate in at least two axial directions. And a vibration detection unit that detects rotational vibrations in the three-axis directions disposed in the main body unit. A target value for shake correction is calculated from the vibration detection result by the vibration detection unit and the relative angle between the main body part and the movable part, and the rotating part is driven according to the target value to perform image blur correction.

JP 2008-1116836 A

  When image blur correction is performed using a rotation mechanism that can change a large angle as in Patent Document 1, it is necessary to detect an accurate relative angle between the main body and the movable part. In other words, based on the relative angle value between the main body and the movable part (the absolute angle value of the rotating part), the vibration output of each axis at the attachment position of the vibration detecting part is output to the drive axis direction of the rotating mechanism part. A so-called rotation axis conversion to be converted is necessary.

  In a configuration in which the rotating unit cannot be touched from the outside, there is a method in which the rotational position when the power is turned off is stored in a memory, and the rotational position is read immediately after the power is turned on. However, in the case where the rotating unit is configured to be touched from the outside, the rotation angle may change during the power-off period. In this case, it is necessary to detect the absolute angle value of the rotating part immediately after the power is turned on.

The method using a high-resolution encoder for detecting the absolute angle value has problems such as an increase in the size of the apparatus and an increase in power consumption and cost. In addition, a method of providing a sensor for detecting the reference position and driving the rotation mechanism after the power is turned on to search for the reference position by the initialization operation increases power consumption due to the initialization operation and takes a startup time.
An object of the present invention is to estimate the absolute angle value of the optical means based on the shake information, thereby reducing cost and power consumption at startup and shortening startup time.

  An apparatus according to an embodiment of the present invention includes an optical unit having an image sensor that images a subject through an imaging optical system, a rotation driving unit that rotates the optical unit with respect to a fixed unit, and an angular velocity attached to the fixed unit. The first detection means using the outputs of the first detection means, the second detection means for detecting the amount of motion of the image from the output of the imaging device, and the outputs of the first and second detection means. Control means for estimating the absolute angle value of the optical means with reference to the mounting angle of the means, and control for controlling the rotation drive means using the output of the first detection means and the absolute angle value by the estimation means Means.

  According to the present invention, by estimating the absolute angle value of the optical means based on the shake information, it is possible to reduce costs and power consumption at startup, and to shorten the startup time.

It is a figure which shows typically the imaging device which concerns on this embodiment. It is a figure which shows the structure of the imaging device which concerns on this embodiment. It is a flowchart which shows the absolute angle estimation of this embodiment, and an image blurring correction process. It is a block diagram of an image blur correction control unit in the present embodiment. It is a block diagram of the image deformation amount calculation part in this embodiment. It is a flowchart which shows the image blurring correction process in this embodiment. It is a flowchart which shows the process following FIG. It is explanatory drawing of the rotating shaft conversion in this embodiment. It is a block diagram of absolute angle estimation in this embodiment. It is a figure which shows the detection range of the motion vector of the image in this embodiment.

  Embodiments of the present invention will be described in detail with reference to the drawings. The present invention is applicable not only to digital cameras and digital video cameras but also to imaging devices such as mobile phones, surveillance cameras, and web cameras.

  FIG. 1 is a schematic diagram illustrating a configuration example of an imaging apparatus according to the present embodiment. FIG. 1A illustrates main components of the imaging device 101. The imaging apparatus 101 includes an operation member such as a power switch operated by a user. The lens barrel 102 is an optical unit including a photographic lens group, an aperture, and an image sensor that constitute an image pickup optical system, and is attached to the image pickup apparatus 101. The imaging apparatus 101 is provided with a rotation mechanism unit for driving the lens barrel 102 to rotate with respect to the fixed unit 103. The angular velocity meter 106 constitutes a shake detection unit that detects the shake of the imaging device 101 and is mounted on the fixed unit 103 of the imaging device 101. Although FIG. 1A shows an example in which the angular velocity meter 106 is attached to the side surface of the fixed portion 103, the angular velocity meter 106 may be attached to a circuit board inside the fixed portion 103. The angular velocity meter 106 outputs a shake angular velocity detection signal based on the angle and position of attachment to the fixed portion 103.

  FIG. 1B is a diagram for explaining the definition of the coordinate axes and directions at the reference position of the fixing unit 103. In 3D Cartesian coordinates consisting of the X, Y, and Z axes, the rotation direction about the X axis is the pitch direction, the rotation direction about the Y axis is the yaw direction, and the rotation direction is about the Z axis Is defined as the roll direction. Assume that the angular velocities in the respective axial directions defined in FIG. 1B are detected at the mounting position of the angular velocity meter 106, that is, the reference position in the fixed portion 103. For the sake of brevity, the panning direction related to drive control is abbreviated as the P direction and the tilting direction is abbreviated as the T direction.

  The imaging device 101 includes a plurality of motor drive mechanisms. The first motor drive mechanism unit 104 is a unit (hereinafter referred to as a tilt rotation unit) that rotates the lens barrel 102 in the pitch direction. The second motor drive mechanism unit 105 is a unit that rotates the lens barrel 102 in the yaw direction (hereinafter referred to as a pan rotation unit). The lens barrel 102 is rotated by the tilt rotation unit 104 and the pan rotation unit 105, and as shown in FIGS. 1C and 1D, the imaging direction of the imaging apparatus 101, that is, the direction of the optical axis 108 of the imaging optical system is changed. Be changed.

  FIG. 2 is a block diagram illustrating a configuration of an imaging apparatus having an image blur correction function according to the present embodiment. The lens barrel 102 includes a zoom unit 201 that performs zooming, a focus unit 203 that performs focus adjustment, and an imaging unit 205.

  The zoom drive control unit 202 controls the drive of the zoom unit 201. The focus drive control unit 204 controls the drive of the focus unit 203. Each control unit 202, 204 performs drive control of the movable lens in accordance with a control command from the control unit 213.

  The imaging unit 205 includes an imaging device, receives light incident through the imaging optical system, and outputs an electrical signal by photoelectric conversion. The imaging unit 205 outputs charge information corresponding to the amount of light received by the imaging element to the image processing unit 206 as an analog image signal.

  The image processing unit 206 acquires an output signal of the imaging unit 205, performs A / D (analog / digital) conversion, performs predetermined image processing on the digital image data, and outputs the processed digital image data. The predetermined image processing includes distortion correction, white balance adjustment, color interpolation processing, and the like. The image processing unit 206 calculates a motion vector representing the amount of motion of the subject by comparing a plurality of captured image data having different imaging times, and outputs the motion vector to the control unit 213. The image processing unit 206 converts the signal into a video signal (video signal) conforming to a predetermined format and stores it in the image memory 207. The predetermined format is a JPEG (Joint Photographic Experts Group) format, a format such as the NTSC format or the PAL format.

  The image deformation unit 208 acquires image data from the image memory 207 and performs image deformation processing based on the image deformation amount calculated by the control unit 213. The image data after the image transformation processing is output to the image recording unit 209 and recorded on a recording medium such as a nonvolatile memory.

  The control unit 213 includes a CPU (Central Processing Unit) and controls the entire imaging system. The lens barrel rotation drive unit 212 drives the tilt rotation unit 104 and the pan rotation unit 105 in accordance with a control command from the control unit 213. Thereby, the lens barrel 102 is driven in the T direction and the P direction.

  The apparatus shake detection unit 210 detects the shake of the imaging apparatus 101 and outputs a detection signal to the control unit 213. The imaging apparatus 101 is equipped with an angular velocity meter (gyro sensor) 106 that detects angular velocity in three axial directions. The operation unit 211 includes operation members such as a power button and a button for changing settings of the imaging apparatus 101 in order for the user to operate the imaging system. When the user turns on the power button, power is supplied to the entire imaging system and the imaging apparatus 101 is activated.

  With reference to FIG. 3, the process of this embodiment is demonstrated. FIG. 3 is a flowchart for explaining the absolute angle estimation process and the image blur correction process of the imaging apparatus. The following processing is realized by the CPU included in the control unit 213 executing a predetermined program. When the power button of the operation unit 211 is turned on, the imaging apparatus 101 is activated and the following processing is started.

  First, in step S <b> 301, the control unit 213 acquires a triaxial angular velocity output from the angular velocity meter 106 installed in the fixed unit 103 and acquires a motion vector calculated by the image processing unit 206. In next step S302, the control unit 213 determines whether or not the absolute angle estimation is incomplete. When the absolute angle estimation is not completed, the process proceeds to S303, and the absolute angle estimation process is performed in the processes from S303 to S310. When the absolute angle estimation is completed in S302, the process proceeds to S311 and the image blur correction process is repeatedly executed at a predetermined sampling period.

  Hereinafter, after describing the image blur correction process, the absolute angle estimation process will be described. FIG. 4 is a block diagram showing a configuration for calculating rotation correction amounts in the P direction and T direction and electronic shake correction amounts in image blur correction. The angular velocity meter 106 outputs the angular velocity detection signal to the shake correction angle calculation unit 502, the conversion unit 503, and the image deformation amount calculation unit 509.

  The tilt rotation unit 104 and the pan rotation unit 105 are respectively provided with encoders, and the rotation angle is acquired. Current position information 501 of each unit used for panning and tilting control is sent to the conversion unit 503 and the image deformation amount calculation unit 509.

  The P-direction shake correction angle calculation unit 502 acquires an angular velocity detection signal from the angular velocity meter 106, calculates the P-direction shake angle, and outputs it to the adder 505. The adder 505 adds the target angle corresponding to the angle change instruction in the P direction manually performed by the operation unit 211 and the output of the shake correction angle calculation unit 502, and adds the addition result to the lens barrel rotation driving unit 212. To the pan rotation unit 507.

  The conversion unit 503 acquires the angular velocity detection signal from the angular velocity meter 106 and the current position information 501 and executes a conversion process described later. The shake correction angle calculation unit 504 in the T direction acquires the output of the conversion unit 503, calculates the shake angle in the T direction, and outputs it to the adder 506. The adder 506 adds the target angle corresponding to the angle change instruction in the T direction that is manually operated by the operation unit 211 and the output of the shake correction angle calculation unit 504, and adds the addition result to the lens barrel rotation driving unit 212. To the tilt rotation unit 508.

  The image deformation amount calculation unit 509 acquires the angular velocity detection signal from the angular velocity meter 106 and the current position information 501, calculates the image deformation amount, and outputs the image deformation amount to the image deformation unit 208. FIG. 5 is a block diagram illustrating a configuration of the image deformation amount calculation unit 509. The image deformation amount calculation unit 509 includes differentiators 601 and 602 for the current position information 501, rotation axis conversion units 603 and 604, and subtractors 605 to 607. The image deformation amount calculation unit 509 includes correction amount calculation units 608 to 612 that calculate a translation correction amount, a tilt correction amount, and a rotation correction amount from the output of each subtractor, and an image deformation amount synthesis unit 613. The image deformation amount synthesizing unit 613 acquires the output of each correction amount calculation unit and outputs the combined image deformation amount to the image deformation unit 208. Details of processing performed by each unit shown in FIGS. 4 and 5 will be described later.

  6 and 7 are flowcharts for explaining the image blur correction process shown in S311 of FIG. When the image blur correction process is started, first, in S401 of FIG. 6, a process of taking the triaxial angular velocity output from the angular velocity meter 106 installed in the fixed unit 103 is performed, and the process proceeds to S402. In S <b> 402, the lens barrel rotation driving unit 212 acquires current position information 501 in panning and tilting control from encoders installed in the tilt rotation unit 104 and the pan rotation unit 105, respectively. As the current position information 501, current angle values of the tilt rotation unit 104 and the pan rotation unit 105 are acquired.

  In step S403, the shake correction angle calculation unit 502 calculates a target angle for shake correction in the P direction from the angular velocity output in the yaw direction. The angular velocity meter 106 (see FIG. 1) is mounted on the fixed portion 103, and the rotation axis of the pan rotation unit 105 coincides with the rotation axis of the angular velocity meter 106 in the yaw direction. Therefore, the shake correction angle calculation unit 502 performs integration processing after converting a low frequency component with respect to the yaw angular velocity output from the angular velocity meter 106 with a high-pass filter (hereinafter referred to as HPF), and converts the yaw angular velocity into an angle. Thereby, the shake correction angle in the P direction is calculated, and the shake correction in the P direction is performed by the rotational drive of the pan rotation unit 105. The pan rotation unit 507 of the lens barrel rotation drive unit 212 is a drive unit that rotates the pan rotation unit 105 about the rotation axis. In order to calculate the target angle of the pan rotation unit 507, in S403, the shake correction target angle is calculated only from the yaw angular velocity output of the angular velocity meter 106, and then the process proceeds to S404.

  In S404, based on the angular velocity output in the pitch direction and the angular velocity output in the roll direction of the angular velocity meter 106, and the current position information 501 (angle information in the P direction) acquired in S402, the shake correction angle calculation unit 504 performs the T direction correction. A target angle for shake correction is calculated. Since the angular velocity meter 106 is mounted on the fixed unit 103, the correction amount for performing shake correction in the T direction varies depending on the rotation angle of the pan rotation unit 105.

  FIG. 1A shows a case where the rotational position in the P direction is the positive position, that is, the X axis is always positioned perpendicular to the optical axis 108. In this case, since the axis of the tilt rotation unit 104 capable of performing the shake correction in the T direction coincides with the pitch axis in FIG. 1B, the shake correction target angle in the T direction is determined from only the angular velocity output in the pitch direction. Calculated. Further, it is assumed that the rotation angle in the P direction is rotated by 90 degrees from the normal position, that is, the Z axis is always located in the vertical direction with respect to the optical axis 108. In this case, since the axis of the tilt rotation unit 104 coincides with the roll axis of FIG. 1B, the shake correction target angle in the T direction is calculated only from the angular velocity output in the roll direction. When the position corresponding to the rotation angle of the pan rotation unit 105 is between the normal position shown in FIG. 1A and the rotation position of 90 degrees, the conversion unit 503 in FIG. 4 determines the pitch direction based on the rotation angle. Are combined with the angular velocity output in the roll direction. The conversion unit 503 performs rotation direction conversion to the axis in the tilt rotation unit 104 and calculates an angular velocity in the T direction. Specifically, the calculation formula used in the conversion unit 503 is expressed by Formula (1).

式（１）中の記号の意味は、以下のとおりである。
Ｗｔｌ：Ｔ方向の角速度
Ｗｘ：角速度計１０６のピッチ方向の角速度
Ｗｚ：角速度計１０６のロール方向の角速度
θpa：Ｐ方向の回転角度。

The meanings of the symbols in the formula (1) are as follows.
Wtl: Angular velocity in the T direction Wx: Angular velocity in the pitch direction of the angular velocity meter 106 Wz: Angular velocity in the roll direction of the angular velocity meter 106 θpa: Rotational angle in the P direction.

  The shake correction angle calculation unit 504 acquires the angular velocity Wtl in the T direction calculated as described above, and calculates the shake correction target angle in the T direction. That is, the angular velocity Wtl in the T direction is converted into an angle by an integration process after the low frequency component is cut by the HPF. Based on the shake correction angle in the T direction, image blur correction is performed by rotational driving of the tilt rotation unit 508. The tilt rotation unit 508 of the lens barrel rotation drive unit 212 is a drive unit that rotates the tilt rotation unit 104 about the rotation axis.

  After S404, the process proceeds to S405. The processing from S405 to S412 is processing performed by the image deformation amount calculation unit 509 in order to perform electronic image blur correction by image deformation. The image deformation amount calculation unit 509 calculates the image deformation amount using the remaining shake angle on the imaging surface and the focal length of the imaging optical system. The image deforming unit 208 performs electronic image blur correction by performing image deformation processing using geometric transformation such as projective transformation.

  In S405, the angular velocity obtained by rotationally converting the three-axis angular velocity output from the angular velocity meter 106 attached to the fixed unit 103 to the axis on the lens barrel 102 including the image sensor is calculated. This will be specifically described with reference to FIG. FIG. 8A is a diagram illustrating the definition of the axis for the fixed unit 103 of the imaging apparatus. The angular velocity in the pitch direction around the X axis is denoted as Wx, and the angular velocity in the yaw direction around the Y axis is denoted as Wy. The angular velocity in the roll direction around the Z axis is denoted as Wz.

  FIG. 8B is a diagram for explaining the definition of the axis with respect to the lens barrel 102 which is a rotating part. The axes rotated with respect to the X axis, the Y axis, and the Z axis are denoted as X ′, Y ′, and Z ′, respectively. The angular velocity in the rotational direction with respect to the X ′ axis is denoted as Wx ′, the angular velocity in the rotational direction with respect to the Y ′ axis is denoted as Wy ′, and the angular velocity in the rotational direction with respect to the Z ′ axis is denoted as Wz ′. Also, the angle of the Y ′ axis when the Y axis is tilted toward the Z axis is θx, the angle of the Z ′ axis when the Z axis is tilted toward the X axis is θy, and the angle of the X ′ axis when the X axis is tilted toward the Y axis Is expressed as θz. θx, θy, and θz each represent an angle of difference between the fixed portion 103 and the rotating portion (lens barrel 102). In this embodiment, since driving is performed with two axes in the T direction and the P direction, θz is always zero.

Here, the rotation matrix around the X-axis, Y-axis, and Z-axis in the three-dimensional space can be expressed by the following equations (2) to (4).

The rotation axis conversion unit 604 converts the output of the angular velocity meter 106 disposed in the fixed unit 103 into an angular velocity based on the imaging surface reference axis definition according to the rotation angle of the lens barrel 102. The angular velocity meter 106 arranged in the fixed portion 103 can detect an angular velocity Wx around the X axis, an angular velocity Wy around the Y axis, and an angular velocity Wz around the Z axis. The angular velocity W of the fixed portion 103 is represented by a column vector of the following equation (5), and the angular velocity W ′ of the lens barrel 102 after rotation is represented by a column vector of the following equation (6). In order to convert W to W ′, the following equation (7) is used.

  With the above method, the rotation axis conversion unit 604 determines the three axis angular velocities (PitchIm, YawIm, RollIm).

  Next, in S406, differentiation processing of the current position information 501 in the P direction and the T direction is executed. The differentiator 601 calculates a tilt angular velocity (TiltSpd) by differentiating the current tilt angle. The differentiator 602 calculates a pan angular velocity (PanSpd) by differentiating the current pan angle. In step S407, the rotation axis conversion unit 603 converts the rotation axis in order to convert the pan angular velocity and the tilt angle into a mechanical correction angular velocity in the axis of the lens barrel 102.

The pan angular velocity (PanSpd) is expressed as Wpa, and the tilt rotation angle is expressed as θtl. The Yaw angular velocity (YawPn) based on the imaging surface reference axis definition is expressed as Wya, and the angular velocity in the roll direction (RollPn) is expressed as Wrl. In consideration of the tilt rotation angle θtl, processing for converting Wpa to Wya and Wrl is performed. At this time, since the tilt rotation axis coincides with the pitch rotation axis on the imaging surface, the pitch rotation axis may be ignored. Therefore, the conversion formula is the following formula (8).

  Next, in S408, the drive angular velocity (PitchTlt) of the tilt rotation unit 104 is calculated. Since the influence of the direction on the image plane with respect to the driving in the T direction is not changed (always affecting the vertical blur), the tilt angular speed (TiltSpd) is set as it is as PitchTlt which is the mechanical correction pitch angular speed, and the process proceeds to S409. .

  In S409, each output of the differentiator 601 and the rotation axis conversion unit 603 is subtracted from the output (PitchIm, YawIm, RollIm) of the rotation axis conversion unit 604. The subtractor 605 subtracts the mechanical correction angular velocity PitchTlt output from the differentiator 601 from PitchIm output from the rotation axis converter 604. The subtractor 606 subtracts the mechanical correction angular velocity YawPn output from the rotation axis conversion unit 603 from YawIm output from the rotation axis conversion unit 604. The subtractor 607 subtracts the mechanical correction angular velocity RollPn output from the rotation axis conversion unit 603 from RollIm output from the rotation axis conversion unit 604. Subtractors 605, 606, and 607 calculate mechanical correction remaining angular velocities (denoted as PitchErr, YawErr, and RollErr), respectively.

  In S410 of FIG. 7, the vertical translation correction amount and the vertical tilt correction amount are calculated based on the pitch correction remaining angular velocity (PitchErr) and the focal length. The vertical translation correction amount calculation unit 608 acquires PitchErr and calculates the vertical translation correction amount. PitchErr outputs a signal in a high frequency band by blocking low frequency components included in angular velocity data by HPF. The focal length f of the imaging optical system is calculated from the output of the encoder of the zoom unit 201. The output of the HPF is converted into an angle by an integrator, and the longitudinal translation correction amount is calculated by further multiplying by the focal length f. The vertical translation correction amount calculation unit 608 outputs the calculated vertical translation correction amount to the image deformation amount synthesis unit 613.

  The vertical tilt correction amount calculation unit 609 acquires PitchErr and calculates the vertical tilt correction amount. PitchErr outputs a signal in a high frequency band by blocking low frequency components included in angular velocity data by HPF. The HPF output is converted into an angle by an integrator, and further divided by the focal length f to calculate a vertical tilt correction amount. The vertical tilt correction amount calculation unit 609 outputs the calculated vertical tilt correction amount to the image deformation amount synthesis unit 613.

  After calculating the vertical translation correction amount and the vertical tilt correction amount, the process proceeds to S411. In S411, the lateral translation correction amount and the lateral tilt correction amount are calculated based on the yaw correction remaining angular velocity (YawErr) and the focal length. The lateral translation correction amount calculation unit 610 acquires YawErr and calculates the lateral translation correction amount. The lateral tilt correction amount calculation unit 611 acquires YawErr and calculates the lateral tilt correction amount. Since the calculation method of the horizontal translation correction amount and the horizontal tilt correction amount is the same as the calculation method of the vertical translation correction amount and the vertical tilt correction amount except that the input is YawErr, detailed description thereof will be omitted.

  After calculating the lateral translation correction amount and the lateral tilt correction amount, the process proceeds to S412. In S412, the rotation correction amount calculation unit 612 calculates the rotation correction amount from the roll correction remaining angular velocity (RollErr). RollErr is converted into an angle by an integrator after a low frequency component included in the angular velocity data is cut off by HPF. The rotation correction amount calculation unit 612 outputs the calculated rotation correction amount to the image deformation amount synthesis unit 613.

  In step S413, the image deformation amount combining unit 613 acquires the outputs of the correction amount calculation units 608 to 612 and combines the image deformation amounts. An operation for synthesizing the translation correction amount, the tilt correction amount, and the rotation correction amount is performed. The image deformation amount combining unit 613 outputs the value of each element of the calculated projective transformation matrix to the image deformation unit 208. In step S414, the image deformation unit 208 performs image deformation processing based on the output of the image deformation amount combining unit 613. Thereby, electronic image blur correction is performed, and the process proceeds to S415.

  In step S415, the control unit 213 determines whether the user has used the operation unit 211 to instruct an angle change in the P direction or the T direction by manual operation. For example, the user can set the target angle using a dedicated operation member provided in the imaging apparatus 101. As the operation member, an operation button capable of rotating right and left related to the P direction and rotating upward and downward related to the T direction is used. The target angle is set according to the operation time during which the operation button is pressed. For example, there is a setting method in which α degrees is added to the target angle every time a predetermined threshold time T elapses, but other setting methods may be used. Alternatively, the target angle is set by notifying the imaging device 101 of the target angle by an operation instruction in an external device that can communicate with the imaging device 101.

  If it is determined in S415 that an angle change instruction has been issued, the process proceeds to S416. If it is determined in S415 that no angle change instruction has been issued, the previous value is held as the target angle, and the process proceeds to S417. In step S416, the control unit 213 sets the target angle set by manual operation, and then proceeds to step S417.

  In S417, the control unit 213 calculates a final target angle related to panning control and tilting control. The final target angle is calculated from the pan shake correction target angle and tilt shake correction target angle calculated in S403 and S404, respectively, and the manual target angle set in S416. The final target angle is a target angle in the P direction for instructing the pan rotation unit 507 of the lens barrel rotation driving unit 212, and a target in the T direction for instructing the tilt rotation unit 508 of the lens barrel rotation driving unit 212. Is an angle. The target angle in the P direction is calculated by the adder 505 adding the output of the shake correction angle calculation unit 502 and the manual target angle in the P direction from the operation unit 211. The target angle in the T direction is calculated by adding the output from the shake correction angle calculation unit 504 and the manual target angle in the T direction from the operation unit 211 by the adder 506.

  In S418, panning and tilting are driven. The target angle in the P direction and the target angle in the T direction calculated in S417 are output to the pan rotation unit 507 and the tilt rotation unit 508, respectively. The shake correction in each direction is performed, and the subroutine ends. This process is executed at a predetermined sampling period.

  Based on the output of the angular velocity meter 106 provided in the fixed unit 103, mechanical image blur correction (mechanical correction) by the pan rotation unit 507 and tilt rotation unit 508, and electronic image blur based on the blur amount on the imaging surface. Correction is performed. In order to perform each correction, accurate tilt absolute angle and pan absolute angle detection information (current position information 501) is required.

For example, the absolute angle detection method includes the following methods.
(A) A method using an encoder capable of high-resolution detection.
This method has a problem that it is expensive in addition to an increase in size and power consumption.
(B) A method in which a photo interrupter or the like is provided at a reference position, and after the power is turned on, panning and tilting are driven to search for the reference position.
The search for the reference position is performed as an initialization operation, and the control unit acquires a detection signal from a photo interrupter or the like, and sets the searched position as the reference position. However, in this method, there are problems such as an increase in power consumption due to the initialization operation and an operation time for the reset motion.

  In the present embodiment, processing for estimating the absolute angle value is performed without adding an absolute angle detection sensor and without requiring a large reset motion operation when the power is turned on. FIG. 9 is a block diagram illustrating a configuration example for performing the process of estimating the absolute angle value in the P direction. The estimation calculation unit 803 shown in FIG. 9 estimates the absolute angle value of the lens barrel 102 based on the mounting angle of the angular velocity meter 106. Since the lens barrel 102 is rotationally driven by the pan rotation unit 105, the absolute angle value of the pan rotation unit 105 may be estimated. The estimation calculation unit 803 obtains the output of the angular velocity meter 106, the image motion vector 801, the focal length 802 of the imaging optical system, and the value of the absolute angle 807 in the T direction, and the value of the absolute angle 806 in the P direction. presume. The estimation calculation unit 803 includes a calculation unit 804 that calculates a blur angular velocity on the imaging surface, and a Kalman filter 805. The Kalman filter 805 calculates an estimated value of the absolute angle 806 in the P direction based on the output of the calculation unit 804 and the value of the absolute angle 807 in the T direction.

  Here, the angle in the T direction has a configuration capable of detecting an absolute angle value. For example, the method (B) is used, and the reference position is searched by performing reset driving in the T direction at the time of activation. In many cases, a configuration is adopted in which the movable range of the tilt angle is not larger than the movable range of the pan angle. For example, while the tilt angle is in the range of −20 degrees to +90 degrees, the movable range of the pan angle is in the range of −180 degrees to +180 degrees. Since the movable range in the T direction is narrow, it is not necessary to perform a reset operation with a large angle change, so that the power consumption of the initialization operation can be reduced. Further, the movable part driven in the T direction has an advantage that the power consumption during driving is small because the mass is not large compared to the movable part driven in the P direction.

  About the movable part driven to a T direction, it is set as the structure covered with cover members, such as glass, so that a user cannot touch from the outside. In the detection of the absolute angle value, the absolute angle value when the power is shut off is written and held in a storage area of a nonvolatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory). When the power is turned on, the absolute angle value is read from the storage area and used, so that the absolute angle value can be detected immediately after startup without performing a reset operation.

  On the other hand, when the movable range of the movable portion driven in the P direction is increased to, for example, −180 degrees to +180 degrees, it is difficult to make a mechanical configuration that the user cannot touch. In addition, when the power is turned off, the user may manually rotate the pan angle, or the angle may be shifted because the user unintentionally touched the movable part. For example, it is assumed that the method (B) is adopted in driving in the P direction. In this case, if the movable range is large, a large angle change is required during the reset operation, and the initialization operation takes time. In addition to the increase in power consumption during start-up and driving, there is a concern about the effect on appearance due to a large reset motion being performed at each start-up. In the following, a method for estimating the absolute angle value in the P direction in which the absolute angle value in the T direction can be detected immediately will be described in detail.

  The relationship between the triaxial angular velocity that is the output of the angular velocity meter 106 provided in the fixed portion 103 and the triaxial angular velocity in the lens barrel (on the imaging surface) that is the movable portion is expressed by the above equation (7). Is done. A method of calculating the angular velocity W ′ of the lens barrel 102 corresponding to the blur amount on the imaging surface will be described. The imaging unit 205 acquires an image signal by photoelectrically converting reflected light from the subject into an electric signal, and the image signal is converted into a digital signal. The image processing unit 206 compares the past image data (for example, one frame before) stored in advance with the current image data, that is, two consecutive image data, thereby calculating the motion vector from the relative displacement information of the image. Is calculated.

式（９）中のarctan()は逆正接関数である。なお、回転方向におけるＷｚ’のベクトルを検出することも可能であるが、本実施形態では演算負荷を低減するため、並進方向の動きベクトルの検出値のみを用いて、Ｐ方向の角度値が推定される。 The calculation unit 804 in FIG. 9 converts the image plane velocity, which is the output of the motion vector, into a blur angular velocity on the imaging surface by the following equation (9). Regarding the motion vector in the translation direction, the vertical blur speed on the imaging surface is denoted as Vx, and the horizontal blur speed is denoted as Vy. Conversion to the blur angular velocity on the imaging surface can be performed by Vx, Vy, and the focal length f of the imaging optical system.

Arctan () in equation (9) is an arctangent function. Although it is possible to detect the vector of Wz ′ in the rotation direction, in this embodiment, in order to reduce the calculation load, the angle value in the P direction is estimated using only the detected motion vector value in the translation direction. Is done.

  FIG. 10 is a diagram illustrating a detection range when detecting a motion vector in a captured image. Of the captured image 901, a range 902 indicates a motion vector detection range during the absolute angle value estimation process. Thus, the detection range 902 is limited to the vicinity of the center of the image, and a motion vector at a location away from the center of the image is not used. The reason is to suppress as much as possible the influence of the blur in the roll direction on the imaging surface, and the influence of the output error on the vertical and horizontal translational shake output due to the occurrence of the shake can be reduced.

From Expression (7), Wx ′ and Wy ′ are as follows.

As described above, since the absolute angle value can be detected for the angle in the T direction, sin θx and cos θx are known. From this, the expressions (10) and (11) can be expressed by the expression (12).

式（１３）乃至（１６）中のkは離散時間を表す。εは推定パラメータの変動成分を表すシステムノイズパラメータであり、ωは観測ノイズパラメータである。 From equation (12), the following equation of state is derived.

K in the equations (13) to (16) represents discrete time. ε is a system noise parameter representing a fluctuation component of the estimation parameter, and ω is an observation noise parameter.

  From equations (13) to (16), cos θy and sin θy, which are estimation parameters represented as state variables, are motion vector output data Wx ′ and Wy ′ that are observation quantities, a tilt angle θx, an angular velocity output Wx, It can be estimated from Wz. From the estimated cos θy and sin θy, θy that is an absolute angle in the P direction is derived.

For equation (16), the filtering step for estimating x (k) using the Kalman filter 805 (FIG. 9) is as follows.
・ Step 1: Calculation of Kalman gain

Step 2: Calculation of estimated parameters

Step 3: Calculation of estimated error variance

In the above equations (17) to (19), the discrete time k corresponds to the number of filtering steps. K is the Kalman gain, and P is the estimated error covariance matrix. σ _ω is the observation noise variance (scalar amount), and R _ε is a system parameter that takes into account fluctuations in the estimated parameter. The estimated error covariance matrix P is assumed to have an initial value given as an appropriate design value. If an excessively large value is set as the initial value, the estimation result diverges and tuning is necessary according to the observation noise.

  On the other hand, in the Kalman filter 805, the estimated error variance value is also calculated at the same time in the equation (19) in step 3. This is a value serving as an index indicating how much the estimation result x (j) at a certain time j varies from k = 0 to k = j, and corresponds to the reliability of the estimation parameter x at that time j. Value. Therefore, when the estimated error variance value has settled to a predetermined value after activation, it is determined that the estimation of the absolute angle value in the P direction has been completed. After the absolute angle value for image blur correction is set, processing for updating the driving angle of the panning operation is performed with reference to the absolute angle value obtained by the estimation processing.

  In addition to the absolute angle value estimation method using a Kalman filter, there is an estimation method using a sequential least square method. However, since the successive least squares method does not consider observation noise or system noise (estimated parameter fluctuation component), the robustness of filtering is low and it cannot be adapted to parameter fluctuation. Therefore, it is desirable to use a Kalman filter in actual design.

  In addition, during the absolute angle value estimation period, that is, until the time point when the estimated error variance value is equal to or less than a predetermined threshold value and it is determined that the estimation of the absolute angle value in the P direction is completed, the driving of panning and tilting is performed. Is stopped. The reason is to prevent the calculation of the Kalman filter 805 from becoming unstable due to the fluctuation of θy during the estimation period. When it is determined that the estimation of the absolute angle value in the P direction is completed when the estimated error variance value is equal to or less than a predetermined threshold, a reference value for absolute angle detection is set, and then panning or tilting is performed. Driving is allowed.

  As described above, the value of the absolute angle 806 in the P direction can be estimated using the Kalman filter 805. However, since this estimation uses each output of the motion vector and angular velocity meter, it is correct if the output is zero or if the actual blur amount is less than each sensor noise of the motion vector and angular velocity meter output. The estimation result cannot be obtained. Therefore, only when the output of the angular velocity meter is equal to or greater than a predetermined threshold, the estimation process of the absolute angle value in the P direction by the Kalman filter 805 is performed.

  Next, the processing shown in S303 to S310 in FIG. 3 will be described. As described above, the absolute angle value cannot be estimated with high accuracy unless the blur amount of the imaging apparatus is large to some extent. In S303, the control unit 213 compares the angular velocity value output from the angular velocity meter 106 with the first threshold value. The first threshold is denoted as Gs1. It is determined whether or not the angular velocity value is equal to or greater than Gs1, and if it is smaller than Gs1, the process returns to S301, and the absolute angle value estimation process is not performed. When the shake of the image pickup apparatus is very small, it is not necessary to perform image blur correction, and thus it is not necessary to estimate the absolute angle value.

  If the angular velocity value is greater than or equal to the first threshold (Gs1 or greater) in S303, the process proceeds to S304. In step S304, the control unit 213 compares the angular velocity value output from the angular velocity meter 106 with a second threshold value. The second threshold is denoted as Gs2. Gs2 is a value larger than Gs1, and is set based on the amount of blur in which the motion vector can be detected. Since the motion vector is detected as an image plane blurring amount from the difference in image data between frames, the maximum value that can be detected is determined. In addition, since the sampling for motion vector detection also depends on the frame rate, it cannot be made dark and fast. In the case of a large amount of shake or high-frequency vibration, the motion vector detection accuracy may be reduced. Therefore, Gs2 is set as a threshold value that ensures the detection accuracy of the image plane blur amount based on the motion vector. When the angular velocity value is equal to or smaller than Gs2, the absolute angle value estimation process is executed. When the angular velocity value is larger than Gs2, the absolute angle value estimation process is not performed.

  If the angular velocity value is less than or equal to the second threshold (Gs2 or less) in S304, the process proceeds to S305, and if the angular velocity value is greater than the second threshold, the process proceeds to S308. In step S305, the control unit 213 stops panning and tilting. In step S306, as described using the equations (9) to (19), an absolute angle value estimation process is executed.

  When the motion vector cannot be acquired in S306, the absolute angle value estimation process is not executed. For example, when a subject with low contrast is captured, or when a subject with gradation in one direction is captured, the motion vector detection accuracy may not be sufficient. For this reason, the control unit 213 determines that the motion vector has low reliability, and does not perform the absolute angle value estimation process.

  In step S307, the control unit 213 determines whether the absolute angle value has been estimated. When it is determined that the estimated error variance value is equal to or smaller than the predetermined threshold value and the estimation of the absolute angle value is completed, the process proceeds to S310. If it is determined that the estimated error variance value is larger than the predetermined threshold value and the absolute angle value estimation is incomplete, the process proceeds to S308. For example, if the state where the reliability of the motion vector is low continues for a long time, the time during which the absolute angle value estimation process is not performed becomes long. At that time, since it is determined that the estimated error variance value is not less than the predetermined threshold value and the estimation of the absolute angle value is not completed, the process proceeds to S308.

  In step S308, the control unit 213 determines whether the count value of the predetermined counter has reached a threshold value. This count value corresponds to a time during which the absolute angle value cannot be estimated. The number of times of entering the processing of S308 is counted, and when the counted value reaches a predetermined threshold, the process proceeds to S309. If the count value is less than the predetermined threshold value, the process returns to S301 and continues. In S309, after the initialization operation for searching for the reference position of the angle in the P direction is performed, the process proceeds to S310.

  In S310, the control unit 213 sets the flag or the like that the estimation of the absolute angle value is completed, and then returns the process to S301. In the next S302, since the estimation of the absolute angle value has been completed, the process proceeds from S302 to S311 and thereafter the image blur correction process is repeatedly executed.

  In the present embodiment, in an imaging apparatus including a rotation mechanism unit that drives panning and tilting, the absolute angle value in the P direction is calculated using the output of the angular velocity meter, the motion vector, and the absolute angle value in the T direction after the power is turned on. presume. Image blur correction is performed based on the estimated absolute angle value. This eliminates the need to always perform an initialization operation with a large reset motion when the power is turned on. An absolute angle value can be calculated while suppressing cost, size, and power consumption, and image blur correction can be performed immediately after the power is turned on.

  Further, in the present embodiment, it is determined whether or not the shake amount detected by the angular velocity meter is within a predetermined range, and when the shake amount is within the predetermined range, an absolute angle value estimation process is executed. If the absolute angle value estimation process is not completed within a predetermined time, the absolute angle is detected by the initialization operation for searching for the reference position. When it is difficult to detect the motion vector and it is difficult to estimate the absolute angle value, image blur correction is performed after the reset position is detected by driving the rotation mechanism unit. An appropriate absolute position detection method is automatically selected according to the state of the imaging device, the shooting situation, and the like.

  In an imaging device capable of changing the angle of the lens barrel, the absolute angle value can be estimated without adding an absolute angle value detection component or performing a large motion reset operation. According to the present embodiment, a cost reduction effect, a power consumption reduction effect at startup, and a startup time reduction effect can be obtained.

DESCRIPTION OF SYMBOLS 102 Lens barrel 104,105 Rotation unit 106 Angular velocity meter 206 Image processing part 208 Image deformation part 213 Control part

Claims (11)

Optical means having an image sensor for imaging a subject through an imaging optical system;
Rotation driving means for rotating the optical means with respect to the fixed portion;
First detection means attached to the fixed portion for detecting angular velocity;
Second detection means for detecting the amount of motion of the image from the output of the image sensor;
Estimating means for estimating an absolute angle value of the optical means with reference to an attachment angle of the first detecting means, using outputs of the first and second detecting means;
An imaging apparatus comprising: control means for controlling the rotation driving means using an output of the first detection means and an absolute angle value by the estimation means. The control unit performs image blur correction by calculating a correction amount of image blur correction from an output of the first detection unit and an absolute angle value by the estimation unit and controlling the rotation driving unit. The imaging device according to claim 1. 2. The image blur correction according to claim 1, wherein the control unit calculates an image blur correction amount from an output of the first detection unit and an absolute angle value by the estimation unit, and performs image blur correction by image processing. Imaging device. 4. The imaging apparatus according to claim 1, wherein the control unit stops the control of the rotation driving unit while the estimation unit estimates the absolute angle value. 5. The rotation driving means comprises a plurality of driving means for rotating the movable part of the optical means around a plurality of rotation axes,
The imaging device according to any one of claims 1 to 4, wherein the estimation unit estimates an absolute angle value centered on one of the plurality of rotation axes. Further comprising third detection means for detecting a reference position of the drive means;
If the estimation of the absolute angle value does not end even after a predetermined time has elapsed from the time when the estimation unit starts estimating the absolute angle value, the control unit searches for a reference position of the drive unit. The imaging apparatus according to claim 5, wherein an operation is controlled, and the third detection unit detects the reference position in the operation and outputs a detection signal to the control unit. The second detection unit detects a motion amount in a detection range including a center portion of a captured image while the estimation unit performs an absolute angle value estimation process. The imaging device according to any one of 6. 8. The method according to claim 1, wherein the estimation unit performs an estimation process using a Kalman filter or a sequential least square method, and terminates the estimation process when a variance value of an estimation error is equal to or less than a threshold value. The imaging device described. The third detecting means detects a reference position of the first driving means among the plurality of driving means,
The control unit controls an operation of searching for a reference position of the first driving unit, and the third detection unit detects the reference position in the operation,
The imaging apparatus according to claim 6, wherein the estimation unit estimates an absolute angle value centered on a rotation axis of the second drive unit among the plurality of drive units. The imaging apparatus according to claim 9, wherein a movable range of the movable part driven by the first driving unit is narrower than a movable range of the movable part driven by the second driving unit. A control method executed by an image pickup apparatus including an optical unit having an image pickup element that picks up an image of a subject through an image pickup optical system, and a rotation driving unit that rotates the optical unit with respect to a fixed unit,
Detecting an angular velocity by a first detection means attached to the fixed portion, and detecting a motion amount of an image from an output of the image sensor by a second detection means;
Using the outputs of the first and second detection means to estimate an absolute angle value of the optical means relative to the mounting angle of the first detection means;
And a step of controlling the rotation driving means using the output of the first detection means and the absolute angle value.

Images