JP6824710B2 - Zoom control device and zoom control method, imaging device - Google Patents

Description

The present invention relates to a zoom control device, a zoom control method, and a technique of an imaging device.

Imaging devices include optical zoom that drives a zoom lens, electronic zoom that magnifies a part of the captured image, or both, and the magnification has been increasing in recent years even in popular machines that are not for business use. I'm out. When a photographer who is unfamiliar with imaging performs telephoto imaging using a high-magnification model, it is difficult to capture the subject, and it often happens that the captured subject is immediately framed out.

Patent Document 1 discloses a technique in which a camera automatically zooms out the imaging angle of view to the wide-angle side when it detects the amount of movement of the imaging device and determines that the user is performing a search operation for a subject. There is. Hereinafter, the zooming support control function that assists the user in framing by zooming out the imaging angle of view according to the movement of the imaging device is referred to as a framing assist zoom and is abbreviated as "FA zoom". It should be noted that returning the zoom position to the original position (zooming in) from the state in which the imaging angle of view is zoomed out by FA zoom is also included in the FA zoom function.

JP 2015-102853

In such a zoom control device that automatically zooms out based on the detected movement of the image pickup device, it is required to perform zoom control according to the user's intention.

The present invention provides a zoom control device, a zoom control method, and an imaging device capable of performing zoom control more in line with the user's intention.

The zoom control device as one aspect of the present invention controls the angle of view. The zoom control device includes an acquisition means for acquiring the amount of movement of the imaging optical system used for imaging the main subject, an image blur correction control means for controlling image blur correction means for correcting image blur, and an image blur correction means. based on the difference between the amount of motion obtained by the correction amount and the acquisition means by a calculation means for exiting calculate the movement amount of the main object image in the captured image, the movement amount of the main subject image is smaller than the first threshold value A determination means for determining whether or not the image is used, and when the movement amount of the main subject image is equal to or greater than the first threshold value, the angle of view is wider than the angle of view when the movement amount of the main subject image is smaller than the first threshold value. It is characterized by having a control means for controlling the operation.

Other aspects of the invention will be described in embodiments of the invention.

According to the present invention, it is possible to perform zoom control more in line with the user's intention.

Block diagram showing a configuration example of the image pickup apparatus according to the first embodiment Schematic diagram illustrating FA zoom A flowchart for explaining the process of calculating the movement amount of the subject image in the movement amount calculation unit according to the first embodiment. The figure explaining the moving amount calculation process of the subject image in 1st Example The figure explaining the moving amount calculation process of the subject image at the time of image blur correction in 1st Example. The figure explaining the movement amount calculation process of the subject image by the motion vector in 1st Example. The figure explaining the motion vector detection and the histogram calculation process in 1st Example The figure explaining the moving amount calculation process of the subject image when the electronic zoom is enabled in 1st Example. Block diagram showing a configuration example of an imaging device according to a second embodiment The figure explaining the threshold value calculation process at the time of enabling electronic zoom in 2nd Example Block diagram showing a configuration example of the image pickup apparatus according to the third embodiment 1. The figure which shows the change of the angular velocity and the angular displacement amount in the comparative example. The figure which shows the change of the angular velocity and the angular displacement amount in 3rd Example The flowchart which shows the process in 3rd Example. The figure which shows the method of calculating the reference value of the angular displacement amount in the modification of 3rd Example.

Hereinafter, examples of the present invention will be described in detail with reference to the accompanying drawings. Each of the embodiments described below relates to a zoom control device capable of performing zoom control more in line with the user's intention by setting the conditions for zooming out in detail. In the first embodiment and the second embodiment, an imaging device including a zoom control device that performs zoom control based on the amount of movement of the subject image in the captured image will be described. In the third embodiment, an imaging device including a zoom control device that zooms out when the imaging device makes a large movement will be described.

In Example 1 of the present invention, an imaging device including a zoom control device that performs zoom control based on the amount of movement of a subject image in the captured image will be described. The FA zoom function disclosed in Japanese Patent Application Laid-Open No. 2015-102853 appropriately determines whether or not to zoom out when the amount of movement of the imaging device and the amount of movement of the subject on the captured image do not match. It's difficult. Therefore, there is a possibility that the zoom-out will not be performed at the timing desired by the photographer. For example, during electronic zooming, the movement of the subject position on the captured image is large with respect to a slight movement of the imaging device, and the subject may be lost. Therefore, it is desirable to make it easier to activate the FA zoom function when the electronic zoom is performed as compared with when the electronic zoom is not performed. On the contrary, since it is relatively easy for the photographer to capture a subject moving in a fixed direction at a constant speed, when the photographer captures the subject well, the position of the subject on the captured image with respect to the amount of movement of the image pickup device. The amount of movement of is small. That is, even if the amount of movement of the image pickup device is large, it is desirable not to activate the FA zoom when the amount of movement of the subject position on the captured image is small.

In this embodiment, zoom control is facilitated according to the photographer's intention by determining whether or not to perform zoom out based on the amount of movement of the subject image in the captured image.

In this embodiment, each component and its operation when the zoom control device is applied to the image pickup device will be specifically described. FIG. 1 is a diagram showing a configuration example of an image pickup apparatus 100 having an FA zoom function. The image pickup apparatus 100 has an image blur correction function for correcting image blur due to camera shake or the like. The image pickup apparatus 100 calculates the amount of movement of the image pickup apparatus for FA zoom by using the shake information of the image pickup apparatus detected for image blur correction. In this case, the vibration information detecting means is also used, but since the motion amount detection sensor dedicated to the FA zoom may be provided, it is not essential to provide the image blur correction function in order to realize the FA zoom function.

Before the explanation of the FA zoom function, the configuration related to the image blur correction function will be described. The runout detection sensor 101 detects the runout applied to the image pickup optical system of the image pickup apparatus 100. The runout sensor 101 is composed of an angular velocity sensor such as a vibration gyro, and detects an angular velocity (movement velocity) as a first value regarding movement in the yaw direction and the pitch direction. The blur correction amount calculation unit 103 in the microcomputer 102 obtains the angular displacement amount in the yaw direction and the pitch direction by integrating the angular velocities in the yaw direction and the pitch direction, respectively, and acquires the blur correction amount from each of the displacement amounts. To do. The runout detection sensor 101 outputs a detection signal to the microcomputer (μCOM) 102. The microcomputer 102 acquires the runout detection signal and performs signal processing. FIG. 1 shows a functional block diagram of the processing executed by the microcomputer 102.

The blur correction amount corresponds to an amount for driving the blur correction mechanism unit 108 in order to cancel the blur of the captured image. Specifically, the blur correction amount calculation unit 103 includes an integrator, integrates the angular velocity signal output by the runout detection sensor 101, converts it into an angular signal, and outputs it to the subtractor 104. At this time, by integrating the angular velocity in the yaw direction and the angular velocity in the pitch direction, respectively, the angular signal in the yaw direction and the angular signal in the pitch direction can be obtained.

The subtractor 104 subtracts the position data of the blur correction mechanism unit 108, which will be described later, from the output of the blur correction amount calculation unit 103, acquires the deviation data, and outputs the deviation data to the control filter 105. The control filter 105 performs signal processing by an amplifier and a phase compensation filter on the deviation data acquired from the subtractor 104. The control filter 105 outputs the processed signal to the shift lens control unit 106.

The shift lens control unit 106 drives and controls the shift lens (correction lens for image blur correction) of the blur correction mechanism unit 108 by controlling the motor 107 according to the output of the control filter 105. Specifically, the shift lens control unit 106 modulates the motor control amount into a PWM (pulse width modulation) waveform that changes the duty ratio of the pulse wave to drive the motor 107. The motor 107 is, for example, a voice coil type motor. By driving the motor 107, the shift lens of the blur correction mechanism unit 108 moves in a direction different from the optical axis direction (for example, a direction orthogonal to the optical axis). The shift lens control unit 106 performs feedback control so that the deviation data from the subtractor 104 approaches 0 as the shift lens moves.

In this embodiment, as an example of the blur correction mechanism unit 108, a lens unit that can move in a direction different from the optical axis direction is illustrated. In FIG. 1, the blur correction mechanism unit 108 and the image sensor 110 are configured as separate units. .. Not limited to such an embodiment, for example, the blur correction mechanism unit 108 may have a structure in which the image sensor 110 is arranged on the movable unit and can move in a direction different from the optical axis direction. In this case, image blur correction is performed by controlling the movement of the movable unit including the image sensor 110. Further, the blur correction mechanism unit 108 may perform so-called electronic vibration isolation. Image blur correction can be performed by changing the position of the cropping range in the image acquired by the image sensor 110.

The position detection sensor 109 includes a magnet and a magnetic detection element (Hall element) provided at a position facing the magnet. The position detection sensor 109 detects the amount of movement of the correction lens of the blur correction mechanism unit 108 (the amount of movement in the direction perpendicular to the optical axis), and outputs the detection signal to the subtractor 104. As a result, a feedback control system is configured in which the correction lens is moved in a direction different from the optical axis direction to follow the target position with respect to the output of the blur correction amount calculation unit 103. That is, the correction lens moves based on the blur correction amount, and the blur of the image caused by the shake of the image pickup apparatus 100 is corrected. A subject image corrected for vertical or horizontal blur is formed on the image pickup surface of the image sensor 110.

The image sensor 110 photoelectrically converts the optical image of the subject imaged by the image pickup optical system including the blur correction mechanism unit 108 and the zoom lens 120, and outputs an image pickup signal (image signal). The imaging optical system may be provided integrally with the imaging device, or may be provided interchangeably with the imaging device (lens interchangeable type). The signal processing unit 111 processes the signal acquired from the image sensor 110. The signal processing unit 111 generates, for example, a video signal (video signal) conforming to the NTSC (National Television System Committee) format and outputs it to the image memory 112 and the motion vector detection unit 125. The signal processing unit 111 performs predetermined processing such as CDS (correlated double sampling), AGC (automatic gain adjustment), A / D conversion, gamma correction, and white balance on the image pickup signal from the image pickup element 110. May be good.

The memory read control unit 113 determines the read position and read range of the image data stored in the image memory 112. For example, when executing the electronic zoom, the memory read control unit 113 sets the image read range to be large or small.

The recording control unit 114 controls the recording of the data read from the image memory 112 by the memory read control unit 113. When the user gives a video signal recording operation instruction by the operation unit 130, the recording control unit 114 controls to output the data read from the image memory 112 to the recording medium 115 and record the data. The recording medium 115 is, for example, an information recording medium such as a semiconductor memory or a magnetic recording medium such as a hard disk.

Next, the configuration related to the FA zoom function will be described. The zoom control device of this embodiment can be composed of a runout detection sensor 101 and a microcomputer 102. In this embodiment, the microcomputer 102 includes a movement amount calculation unit 116, a capture determination unit 117, a zoom control unit 118, an electronic zoom control unit 122, a blur correction amount calculation unit 103, a subtractor 104, a control filter 105, and a shift lens control. Includes part 106.

The shake detection sensor 101 detects the shake of the imaging device 100, and outputs the shake detection signal to the subject movement amount calculation unit (hereinafter referred to as the movement amount calculation unit) 116. The movement amount calculation unit 116 calculates the movement amount of the subject image. The details of the processing will be described later.

The subject capture determination unit (hereinafter referred to as capture determination unit) 117 is a threshold value used for determining the movement amount of the subject image from the movement amount calculation unit 116 and determining whether or not to perform zoom-out, and is the first threshold value. Compare with the threshold). The capture determination unit 117 determines the capture state of the subject depending on whether or not the movement amount of the subject image is equal to or greater than the threshold value. The capture state of the subject is a state in which the determination result is either one of the photographer capturing the subject with the image pickup apparatus or the state of searching for the subject. The movement amount of the subject image calculated by the movement amount calculation unit 116 is, for example, a value indicating the position change of the subject image on the image sensor 110 in pixel units. Further, the threshold value is a limit value of the imaging resolution.

The zoom control unit 118 acquires the determination result of the capture determination unit 117, and outputs a drive signal for zooming out or zooming in to the zoom motor 119 according to the determination result. When the determination result that the image pickup apparatus 100 is in the zoom-out state and the capture determination unit 117 is in the state of capturing the subject by the image pickup apparatus is input, the zoom control unit 118 outputs a drive signal for zooming in to the zoom motor. Output to 119. On the other hand, when the determination result that the image pickup device is in the zoomed state (the state in which the FA zoom is not functioning) and the capture determination unit 117 is in the state of searching for the subject is input, the zoom control unit 118 will perform the zoom control unit 118. The drive signal for zooming out is output to the zoom motor 119.

When the determination result that the image pickup apparatus 100 is in the zoom-out state and the capture determination unit 117 is searching for the subject is input, the image pickup angle of view is not changed. Similarly, even when the image pickup device is in the zoomed-in state and the determination result that the image pickup device is capturing the subject is input from the capture determination unit 117, the image pickup angle of view is not changed.

The zoom motor 119 is, for example, a stepping motor, and the feed screw is rotated by the rotation of the rotor, and the zoom lens 120 is moved in the optical axis direction. The zoom control unit 118 calculates a drive signal of the number of pulses required to move the zoom lens 120 to the target position and supplies the drive signal to the zoom motor 119. The zoom control unit 118 may input a drive signal to the zoom motor 119 until it is detected by the position detection sensor 121 that detects the position of the zoom lens 120 that the zoom lens 120 has reached the target position. In the following description, the zoom-out performed by the zoom control unit 118 by activating the FA function is referred to as FA zoom-out, and the zoom-in performed to return from the zoomed-out state by the FA zoom-out to the telephoto side is referred to as FA zoom-in. In this embodiment, the FA zoom-out is a zoom-out performed in response to the determination result of the capture determination unit 117.

A position scale is fixed to the holding frame of the zoom lens 120 in order to detect the position. A position detection sensor 121 is fixed to a lens barrel portion (not shown) at a position facing the position scale. The position scale has a scale pattern such as a magnetic pattern or a light reflection pattern formed in the optical axis direction. The position detection sensor 121 magnetically or optically reads the position scale and outputs a position detection signal indicating the position of the zoom lens 120 in the optical axis direction. The detection signal of the position detection sensor 121 is input to the zoom control unit 118 and used for the position control of the zoom lens 120. On the other hand, the position detection sensor 109 also supplies a detection signal to the movement amount calculation unit 116. In FIG. 1, the A terminals 123 and 124, which are indicated by A in a circular frame, are shown to be electrically connected to each other.

With respect to the movement of the image pickup apparatus 100, the shift lens control unit 106 controls the shift lens to suppress the movement of the image. The amount of movement of the subject image is calculated by subtracting the amount of movement of the imaging range due to the movement of the shift lens from the amount of movement of the imaging range due to the movement of the imaging device.

The image pickup apparatus 100 has an electronic zoom function, and the electronic zoom control unit 122 is in charge of zoom control by image processing. When the electronic zoom function is enabled, the zoom control unit 118 calculates the electronic zoom magnification and outputs it to the electronic zoom control unit 122. The electronic zoom control unit 122 instructs the memory read-out control unit 113 of the image read-out range based on the electronic zoom magnification. Further, the electronic zoom control unit 122 outputs the electronic zoom magnification to the movement amount calculation unit 116.

In the electronic zoom control, a process of changing the cropping range of the image is executed. When the image cut out by this process is enlarged when it is displayed, the amount of movement of the subject image on the captured image is larger than that when the electronic zoom control is disabled even if the movement of the imaging device 100 is the same. Become. Therefore, the movement amount calculation unit 116 corrects the movement amount of the imaging range accompanying the movement of the imaging device with the electronic zoom magnification, and calculates the corrected value as the movement amount of the subject image. The photographer can select whether to enable or disable the electronic zoom control by setting menus on the display screen.

The motion vector detection unit 125 acquires the current video signal generated by the signal processing unit 111 and the video signal one frame before stored in the image memory 112, and in the image based on the luminance signal included in the video signal. Detects the motion vector of the subject in. For example, a block matching method is used as a motion vector detection method. The block matching method is a method in which an captured image is divided into a region called a block, and for example, a similar portion between an image one frame before and a current image is detected in block units. In an arbitrary range in the image one frame before, the process of detecting the position having the largest correlation value with the arbitrary block in the current image as a similar block position is executed. A motion vector, which is motion information between frame images, is detected based on the amount of displacement between the position of an arbitrary block in the current image and the position of a similar block in the image one frame before.

The motion vector detection unit 125 may use a motion vector detection method other than the block matching method.

The motion vector detection unit 125 outputs the detected motion vector to the movement amount calculation unit 116. The B terminals 126 and 127, respectively, which are indicated by B in the circular frame in FIG. 1, are shown to be electrically connected to each other. When FA zoom is enabled, zoom out is automatically activated when the amount of movement of the subject image is large. On the other hand, when the photographer selects an imaging state in which FA zoom is not effective, zooming out is not automatically performed even if the amount of movement of the subject image is large. The setting for switching between enabling and disabling the FA zoom can be selected by the photographer operating the operation unit 130. The operation unit 130 can be composed of an operation switch, an operation button, a touch panel, and the like, and accepts a photographer's operation on the camera, such as a menu operation, a zoom key operation, a moving image imaging start operation, and a still image imaging instruction operation. The operation unit 130 also accepts an operation of interrupting the FA zoom out. As an operation for interrupting the zoom-out, a zoom key operation for instructing the zoom-in and zoom-out when the FA zoom function is not enabled may be used.

A specific example of the FA zoom will be described with reference to FIGS. 2 (A) to 2 (C). FIG. 2A illustrates a state in which the image pickup device 100 is moved from the left direction to the right direction in the figure in order to capture the ball 201 which is the main subject moving by the photographer. The image pickup device 100 captures the subjects 202 and 203 in addition to the ball 201. Further, in FIG. 2A, it is assumed that the ball 201 is moving, the current ball 201a is indicated by a black circle, and the past ball 201b is indicated by a shaded circle. FIG. 2B exemplifies an image (captured image) captured at this time. FIG. 2C shows a temporal change in the amount of movement of the subject image. The times t1 to t3 shown in FIGS. 2 (B) and 2 (C) have a relationship of "t1 <t2 <t3".

Until the time t1, the photographer hardly moves the image pickup apparatus 100, and the amount of movement of the subject image is close to zero. When the ball 201 is released at time t1, the photographer moves the image pickup device 100 in an attempt to catch the ball 201. At this time, as shown in FIG. 2C, the amount of movement of the subject image begins to increase. At time t2, the amount of movement of the subject image exceeds the threshold value. As for the relationship between the threshold value and the movement amount of the subject image, as shown in FIG. 2 (B), when the moving main subject cannot be captured by the imaging device and the main subject is about to deviate from the imaging angle of view, the movement amount Is set to be greater than or equal to the threshold value. The capture determination unit 117 determines that the photographer is searching for the subject at the time t2 when the movement amount of the subject image becomes equal to or greater than the threshold value. The zoom control unit 118 performs zoom-out control in response to this determination result. Therefore, the imaging angle of view becomes wide at time t3 after time t2. That is, the movement of the captured image with respect to the movement of the imaging device 100 becomes smaller as the imaging angle of view becomes wider, and the movement of the imaging device 100 itself also becomes smaller. Therefore, as shown in FIG. 2C, the amount of movement of the subject image gradually decreases after reaching the peak value after the time t2.

The calculation process of the movement amount of the subject image performed by the movement amount calculation unit 116 will be described in detail with reference to the flowchart of FIG. 3 and FIGS. 4 to 8. The subject movement amount calculation unit 116 as a microcomputer performs the processing according to the subject movement amount calculation program which is a computer program. In FIG. 3 and the following description, the step is abbreviated as S.
In S100 of FIG. 3, the movement amount calculation unit 116 calculates the movement amount of the subject image from the movement amount of the image pickup apparatus 100 and temporarily stores it. Specific examples will be described with reference to FIGS. 4 (A) and 4 (B) and FIG.

FIG. 4A is a schematic view showing the relationship between the imaging angle of view θ, the imaging range d0, and the subject 202 in a state where the optical axis X of the imaging device 100 coincides with the horizontal axis H. In FIG. 4, the imaging optical system of the imaging apparatus is schematically shown by a single lens 210. The imaging range d0 corresponds to the vertical range of the captured image. The angular velocity output of the runout detection sensor 101 is converted into an angle by integration. The value obtained by integrating the angular velocity is the amount of displacement of the angle caused by the runout (hereinafter referred to as the runout angle). The deflection angle may be acquired by starting integration from the timing when the FA zoom function is enabled by the photographer, or the angular velocity in a predetermined period (for example, 1 second) may be integrated and sequentially updated. The output (shake angle) of the runout detection sensor 101 converted into an angle is expressed as α, the current focal length of the imaging optical system is expressed as f, and the amount of movement of the subject image is expressed as d. FIG. 4B is a schematic view showing the amount of movement d of the subject image when the image pickup apparatus 100 swings at a swing angle α. The optical axis X of the image pickup apparatus is tilted with respect to the horizontal axis H at a deflection angle α. At this time, the movement amount d of the subject image is the position P0 of the subject image before the shake of the imaging device (state of FIG. 4 (A)) and after the shake occurs (state of FIG. 4 (B)). This is the difference from the position P1 of the subject image.
The movement amount d is calculated using the following formula (1).
d = f × tan α ・ ・ ・ Equation (1)
tan α is a tangent function of the swing angle α. Since the movement of the correction lens of the blur correction mechanism unit 108 suppresses the movement of the position of the subject image in the captured image, the amount of movement of the subject image in the captured image is smaller than that of the image pickup device 100. .. In FIG. 5, the correction amount due to the movement amount of the correction lens is expressed as β (unit: angle), and the movement amount of the subject image at this time is expressed as d ^* . The movement amount d ^* of the subject image is the position P1 of the subject image after the shake occurs (the state of FIG. 4B), and the lens 210 is shifted to correct the generated shake, and the blur is corrected. It is a difference from the position P2 of the subject image in the state (the state of FIG. 5). In the case of the runout angle α and the correction amount β, the movement amount d ^* of the subject image is calculated by the following equation (2).
d ^* = f × tan (α-β) ・ ・ ・ Equation (2)
The length per pixel is expressed as u, and the unit thereof is, for example, mm / pixel. If the movement amount d ^* of the subject image is divided by u and d ^* / u is calculated, it can be converted from the length unit to the pixel unit. The unit regarding the movement amount of the subject image is not particularly limited, but when the unit of the motion vector is a pixel unit, the unit of the movement amount of the subject image is referred to as a pixel unit below. The image pickup apparatus 100 can be set to a state in which image blur correction is not performed. In this case, the value of the correction amount β is only zero. That is, since it can be expressed by the above equation (2), a state in which image blur correction is not performed will be described.

S101 of FIG. 3 is a process of determining whether or not the motion vector can be detected by the motion vector detection unit 125. Since the FA zoom is a function for preventing the main subject desired by the photographer from being out of frame, in this embodiment, the motion vector is detected and the subject is determined by the amount of movement of the main subject image in the actual captured image. Judge the movement of. That is, when the motion vector can be detected, the determination is basically made based on the motion vector, so the process proceeds to S102. If the motion vector cannot be detected, the process proceeds to S104. As an example in which the motion vector cannot be detected, there is a case where the correct motion vector cannot be detected because the subject has low contrast. In addition, the motion vector is detected when the motion vector related to a specific subject cannot be detected because the number of moving subjects in one screen is large, or because the amount of vibration of the imaging device between one frame is too large. It may not be possible to take an image in a possible state.

The motion vector calculation process of the main subject 204 (a, b) in S102 will be described with reference to FIGS. 6 and 7. FIG. 6A shows the movement of the image pickup apparatus and the movement of the main subject 204. The main subject at time t1 is indicated by reference numeral 204a, and the main subject at time t2 is indicated by reference numeral 204b. It is illustrated that the main subject 204 moves from the time t1 to the time t2, and the photographer moves the image pickup apparatus 100 at a deflection angle α in order to capture the main subject 204. FIG. 6B illustrates an image captured at time t1, and FIG. 6C illustrates an image captured at time t2. FIG. 7A illustrates motion vectors V01 to V05 in a plurality of detection blocks set in the captured image. FIG. 7B illustrates a histogram of the detected motion vectors. The horizontal axis represents the motion vector value, and the vertical axis represents the number of detected motion vectors (number of detected vectors).

Since the motion vector of the main subject 204 and the motion vector of the background are mixed in the output of the motion vector detection unit 125, in this embodiment, the motion vectors are classified using the histogram illustrated in FIG. 7B. The motion vector detection unit 125 calculates motion vectors V01 to V05 for each detection block based on the data of the captured image at time t1 and the captured image at time t2. A process of generating a histogram shown in FIG. 7B is executed from the plurality of detected motion vectors.

FIG. 7B exemplifies the first motion vector 205 having a larger number of detected vectors and the second motion vector 206 having a smaller number of detected vectors. It can be determined that the motion vector having a value similar to the motion amount d ^* of the subject image calculated from the motion amount of the image pickup apparatus 100 is the motion of the image due to the camera motion, that is, the motion vector of the background. In the example of FIG. 7B, since the motion vector value of the first motion vector 205 having the larger number of detected vectors has a value close to the motion vector d ^* of the motion subject image, it is a background motion vector. Consider it as. Therefore, the motion vector of the main subject 204 excludes the motion vector determined to be the background motion vector (first motion vector 205) from all the motion vectors, and the number of detected motion vectors is the largest among the remaining motion vectors. It is determined as a motion vector with many motions. In the example of FIG. 7B, the motion vector value of the second motion vector 206 is regarded as the motion vector value of the main subject 204. The magnitude of the motion vector of the detected main subject 204 is expressed as v.

In S103 of FIG. 3, the movement amount calculation unit 116 replaces the magnitude v of the motion vector of the main subject calculated in S102 with the movement amount of the subject image, and ends the process. That is, when the motion vector can be detected, the subject image movement amount stored in S100 is updated to the motion vector of the main subject. On the other hand, if it is determined in S101 that the motion vector cannot be detected (No), the movement amount of the subject image cannot be updated by the motion vector v of the main subject. Therefore, the movement amount of the subject image acquired in S100 or the subject The correction value obtained by correcting the image movement amount with the electronic zoom magnification is used as the final movement amount of the subject image. Therefore, in S104, the movement amount calculation unit 116 determines whether or not the electronic zoom control is being executed. If the electronic zoom control is executed, the process proceeds to S105, and if the electronic zoom control is not executed, the movement amount d ^* of the subject image calculated in S100 is the final subject image. After being determined as the amount of movement, the process ends.

In S105, the movement amount calculation unit 116 corrects the movement amount d ^* of the subject image calculated in S100 with the electronic zoom magnification. The corrected movement amount (denoted as e) is calculated by the following formula (3).
e = d ^* × Electronic zoom magnification ・ ・ ・ Equation (3)
The process of S105 will be described with reference to FIGS. 8A to 8C. FIG. 8A illustrates a state in which the main subject 207 moves from time t1 to time t2, and the photographer moves the image pickup apparatus 100 at a deflection angle α in order to capture the main subject 207. Let d be the amount of movement of the subject image on the captured image with respect to the amount of movement of the image pickup device (when β = 0, and when β is not zero, it is d ^* ). The movement amount d of the subject image is the difference between the subject position at time t1 and the subject position at time t2. FIG. 8B illustrates an image captured when the electronic zoom is not controlled, and FIG. 8C illustrates an image captured when the electronic zoom is controlled.

In the imaging situation shown in FIG. 8A, when the electronic zoom is not controlled, the amount of movement of the subject image in the captured image shown in FIG. 8B is d. On the other hand, when the electronic zoom is controlled, the process of enlarging or reducing the image by the electronic zoom is executed. Therefore, as shown in FIG. 8C, the amount of movement of the subject image in the captured image is e. That is, when the electronic zoom is controlled, the amount of change in the position of the subject image in the captured image changes according to the electronic zoom magnification even if the movement of the imaging device 100 is the same. In S105, the movement amount calculation unit 116 determines the movement amount e of the subject image corrected by using the electronic zoom magnification as the movement amount of the subject image, and ends the process. Even if the motion vector cannot be obtained by the correction used in the above equation (3), the movement amount of the subject image can be calculated from the movement amount of the image pickup apparatus 100 in consideration of the electronic zoom.

According to this embodiment, zooming support control can be performed at an appropriate timing according to the imaging situation by determining based on the amount of movement of the subject image in the captured image.

Next, Example 2 of the present invention will be described. Similar to the first embodiment, the present embodiment also describes an imaging device including a zoom control device that performs zoom control based on the amount of movement of the subject image in the captured image. In the zoom control device of this embodiment, the microcomputer 202 includes a threshold value acquisition unit 228, and the subject capture determination unit 217 determines whether or not the subject is captured by using the threshold value acquired by the threshold value acquisition unit 228. Is different from Example 1.

FIG. 9 illustrates the image pickup apparatus 200 having the FA zoom function of this embodiment. In the image pickup apparatus 200 according to the present embodiment, by using the reference numerals already used for the same configuration as the image pickup apparatus 100 according to the first embodiment, detailed description thereof will be omitted, and the differences will be mainly described. To do.

As described above, in the imaging device 200, the microcomputer (μCOM) 202 includes a threshold value acquisition unit 228. The movement amount calculation unit 216 calculates the movement amount of the subject image based on the output of the runout detection sensor 101. Specifically, by subtracting the value obtained by integrating the angular velocity output of the shake detection sensor 101 and converting the amount of blur correction related to the shake correction mechanism unit 108 into an angle, the subject is set in units of angle. The amount of movement of the image (denoted as d2) is calculated. The unit of the movement amount d ^* of the subject image in the first embodiment was a pixel, but the basic idea of the calculation method is the same. This embodiment differs from the first embodiment in that the amount of movement of the subject image is acquired by an angle instead of the length (in pixels). However, also in this embodiment, as in the first embodiment, the movement amount of the subject image may be acquired in units of pixels, and in the first embodiment, the movement amount of the subject image may be acquired in units of angles. May be good. The movement amount d2 of the subject image is input to the subject capture determination unit 217.

The subject capture determination unit 217 determines the capture state of the subject based on the movement amount d2 of the subject image and the motion vector (see B terminals 126 and 127) detected by the motion vector detection unit 125. In the determination process using the motion vector as in the case of the first embodiment, the capture state is determined based on the motion vector of the main subject, but the threshold value is calculated by the threshold value acquisition unit 228 based on the information from the zoom control unit 218. The point is different from Example 1. The threshold value acquired by the threshold value acquisition unit 228 is a threshold value for determining the capture state of the subject with respect to the movement amount d2 of the subject image. A specific example will be described with reference to FIG.

FIG. 10A illustrates a threshold value for the focal length depending on the presence or absence of electronic zoom control. The horizontal axis represents the focal length of the imaging optical system, and the vertical axis represents the threshold value. In this example, the focal length is shown on the horizontal axis, but it may be replaced with the distance from the camera to the subject. The threshold value when the electronic zoom control is not performed by the electronic zoom control unit 222 (OFF) is expressed as TH1 and is shown by a dotted line graph. Further, the threshold value when the electronic zoom control is performed by the electronic zoom control unit 222 (ON) is expressed as TH2 and is shown by a solid line graph. On the telephoto side, a process of changing the determination threshold value according to the focal length is performed so that the zoom-out is activated by the amount of movement of the image pickup device smaller than that on the wide-angle side.

Further, in FIG. 10A, “TH2 <TH1” is obtained for the same focal length. When the electronic zoom control is executed (when the electronic zoom magnification is larger than 1), compared with the case where the electronic zoom control is not executed even with the same amount of movement of the image pickup device (when the electronic zoom magnification is 1). , The amount of movement of the subject image on the captured image becomes large. Therefore, when the electronic zoom magnification is larger than 1, the threshold value is set smaller than that when the electronic zoom magnification is 1, so that the zoom out can be easily activated. Specifically, as shown in the following equation (4), the value obtained by dividing the threshold value TH1 by the electronic zoom magnification is set as the threshold value TH2.
TH2 = TH1 / electronic zoom magnification ... Equation (4)
TH2 may be set for each electronic zoom magnification and acquired by reading out the information stored in the storage unit constituting the threshold value acquisition unit 228, or the TH1 and electronic zoom stored in the storage unit. It may be calculated from the magnification.

When the electronic zoom control is performed, the zoom out is activated when the movement amount d2 of the subject image becomes the threshold value TH2 or more. The threshold values TH1 and TH2 are first threshold values used for determining whether or not to perform FA zoom out. After that, when the movement amount d2 of the subject image falls below the threshold value TH3 for capture, the zoom-in control is started, and the process of returning to the original focal length on the telephoto side is performed. The capture threshold is a threshold for determining that the subject image is within the captured image and captures the subject, and when the movement amount d2 of the subject image is less than the capture threshold, the subject image is It is determined that the subject is captured in the captured image.

In other words, the threshold value for capture is a threshold value for determining whether or not to perform FA zoom-in, and is also referred to as a second threshold value. FIG. 10B is a diagram for explaining the limitation of the threshold value when the electronic zoom control is performed. The horizontal axis represents the focal length of the imaging optical system (or the distance from the camera to the subject), and the vertical axis represents the threshold value.

At the same focal length, "TH3 <TH2", and diff indicates the difference between TH2 and TH3. The value of TH3 also changes depending on whether the electronic zoom control is performed or not. The description of the capture determination threshold value when the electronic zoom control is not executed will be omitted.

When TH2 is reduced by the electronic zoom magnification, the difference diff with TH3 becomes small. When the difference diff is too small, there are the following problems regarding the operation of the FA zoom. When the movement amount d2 of the subject image slightly exceeds TH2 and the zoom-out is activated, immediately after that, the movement amount d2 becomes slightly smaller than TH3 and the zoom-in starts. That is, the hunting phenomenon in which zooming out and zooming in frequently occur may make it difficult to see the image. Therefore, in this embodiment, the TH2 is limited so that the difference diff does not become smaller than the predetermined value (lower limit value) even if the focal length changes. Specifically, the predetermined value is a value corresponding to the amount of movement when the photographer performs fixed-point imaging so as not to move the imaging device 100. The reason is that the movement amount d2 does not become equal to or higher than the first threshold value TH2 or lower than the second threshold value TH3 only by the influence of camera shake.

In this embodiment, a process of changing the determination threshold value for determining the capture state of the subject is performed. According to this embodiment, zooming support control can be performed based on the comparison result between the movement amount of the subject image in the captured image and the determination threshold value after the change.

In this embodiment, the threshold value is changed according to the electronic zoom, but the same effect can be obtained even if the capture determination unit 217 corrects the magnitude v of the main subject motion vector with the electronic zoom magnification and then compares it with the threshold value. Obtainable. Further, in this embodiment, the threshold value is changed according to the focal length (optical zoom magnification) and the electronic zoom magnification, but the threshold value may be changed only according to the focal length or the electronic zoom magnification. It is considered that changing the threshold value according to both the focal length and the electronic zoom magnification enables zoom control according to the photographer's intention rather than changing the threshold value according to only one of them. .. On the other hand, if the change is made according to only one of them, the amount of data to be stored and the amount of calculation can be reduced.

In Example 3 of the present invention, an image pickup device including a zoom control device that performs FA zoom-out when the image pickup device makes a large movement will be described. When performing FA zoom-out based on the detected movement of the image pickup device, the movement of the image pickup device due to the photographer's search for the subject (panning) is distinguished from the movement of the image pickup device due to camera shake, and the image pickup device moves due to panning. It is preferable to zoom out in some cases.

In this embodiment, the first value (velocity, acceleration) related to the movement of the image pickup apparatus and the second value, which is the amount of movement, are used to set a position as a reference for acquiring the runout angle using acceleration. By doing so, it is possible to distinguish between a small movement of the image pickup device due to camera shake and a large movement of the image pickup device due to panning. It should be noted that the velocity includes the angular velocity, the acceleration includes the angular acceleration, and the movement amount includes the deflection angle. In this embodiment, the angular velocity is acquired as the first value by using the runout detection sensor that detects the angular velocity, and the runout angle is acquired as the second value by integrating the angular velocity.

FIG. 11 illustrates the image pickup apparatus 300 having the FA zoom function of this embodiment. In the image pickup apparatus 300 according to the present embodiment, by using the reference numerals already used for the same configuration as the image pickup apparatus 100 according to the first embodiment, detailed description thereof will be omitted, and the differences will be mainly described. To do.

The microcomputer 102 of the image pickup apparatus 300 of this embodiment functions as a motion determination unit 128 and a threshold value acquisition unit 129. The motion determination unit 128 and the threshold value acquisition unit 129 determine whether or not to perform zoom-out by the FA zoom function. The motion determination unit 128 is an operation of changing the direction of the image pickup device 300 so that the photographer searches for a subject by using the angular velocity obtained from the output of the runout detection sensor 101 and the runout angle α obtained by integrating the angular velocities. Determine whether or not (panning) is being performed. Panning is the operation of moving the camera so that it changes its orientation in the horizontal direction. However, not only horizontal movement but also vertical movement (tilting) is often included in panning in a broad sense, and in the present invention and the present specification, when simply referring to panning, tilting is also included. ..

Generally, a photographer who loses sight of a subject moves the camera greatly to search for the subject. Therefore, when the angular velocity and the amount of angular displacement of the camera are equal to or greater than a predetermined threshold value (at least one of the angular velocity threshold and the angular displacement threshold described later), the motion determination unit 128 moves the camera to search for the subject. Judge as something. On the contrary, if the photographer can capture the subject stably, the movement of the camera becomes small. Therefore, when either the angular velocity or the angular displacement amount of the camera is smaller than the motion determination threshold value, the motion determination unit 128 determines that the photographer is not searching for the subject, that is, the subject is captured in the imaging screen. The threshold value is calculated by the threshold value acquisition unit 129. The relationship between the angular velocity and the swing angle α and the threshold value will be described in detail later. The motion detection means is configured by the runout sensor 101 that detects the angular velocity and the motion determination unit 128 that acquires the threshold value from the threshold value acquisition unit 129.

The zoom control unit 118 gives a drive signal for performing FA zoom-out or FA zoom-in to the zoom motor 119 in response to the determination result of the motion determination unit 128. The zoom control unit 116 and the motion determination unit 128 constitute an angle of view changing means. A zoom control device is composed of a runout detection sensor 101, a motion determination unit 128, and a zoom control unit 118.

The threshold value acquisition unit 129 acquires a threshold value based on the current focal length (zoom position) obtained from the zoom control unit 118. Similar to the second embodiment, the threshold value on the telephoto side is set smaller than the threshold value on the wide-angle side. In this example, since the electronic zoom is not considered, the threshold value is determined according to the focal length (TH1 of Example 2 is used). However, in the case of a zoom control device in which FA zoom out is activated even during electronic zoom control, it is preferable to use a threshold value (TH2) set according to the electronic zoom magnification and the focal length as in the second embodiment.

Next, the relationship between the angular velocity and the deflection angle of the image pickup apparatus 300 and the threshold value for determining whether or not to activate the FA zoom out will be described. 12 and 13 (A) and 13 (B) show the angular velocity and deflection of the camera while panning is being performed while following the moving main subject from the state in which the subject is captured in the imaging screen (within the angle of view). An example of a horn is shown. In each figure, the threshold value for the angular velocity (hereinafter, also referred to as the third threshold value or the angular velocity threshold value) and the detected angular velocity and the threshold value for the deflection angle (hereinafter, also referred to as the fourth threshold value or the deflection angle threshold value) and the acquired runout. Shows the relationship with the horn.

FIG. 12 shows an example of the change between the angular velocity and the swing angle α in the comparative example in which the swing angle is acquired by integrating the angular velocity from the time when the subject is captured on the imaging screen (the time at the left end in the figure). It is a figure.

In FIG. 12, while the subject is captured in the imaging screen before panning, the change in the deflection angle is small, the angular velocity temporarily increases, and the detected angular velocity exceeds the angular velocity threshold (in FIG. 12). , At the time of A), the runout angle is less than the runout angle threshold value, and FA zoom out is not performed. After that, when slow panning is performed so as to follow the subject, the runout angle increases in one direction (positive direction in the case of FIG. 12), and the runout angle α eventually becomes equal to or higher than the runout angle threshold value. In this state, when the angular velocity temporarily increases and exceeds the angular velocity threshold value (at the time point B in FIG. 12), FA zoom-out is performed even though the subject is captured in the imaging screen.

On the other hand, in this embodiment, as shown in FIG. 13 (A), when the angular velocity is slower than the angular velocity threshold value while capturing the subject in the imaging screen before panning, the deflection angle is reset to 0 and the angular velocity is set. The integration of the angular velocity is started from the time when is equal to or higher than the angular velocity threshold value (the time point C in FIG. 13), and the swing angle α is acquired. Then, the runout angle is reset to 0 as the angular velocity becomes smaller than the angular velocity threshold value again. This is the same during the subsequent slow panning. Since the time during which the angular velocity becomes equal to or higher than the angular velocity threshold value is short during slow panning, the deflection angle is reset to 0 before the deflection angle becomes larger than the angular velocity threshold value. Therefore, FA zoom-out can be prevented when the subject can be followed by slow panning.

On the other hand, in FIG. 13B showing the present embodiment, as in FIG. 13A, sudden panning is performed in order to search for the lost subject from the state where the subject is captured in the imaging screen before panning. As a result, the angular velocity becomes equal to or higher than the angular velocity threshold value, and the state continues. As a result, the integration of the angular velocity is started from the time when the subject is started to be searched for, and the runout angle is acquired. At the time point E in FIG. 13B, the runout angle becomes equal to or higher than the runout angle threshold value. In this case, FA zoom out is performed. As described above, in this embodiment, the timing at which the integration of the angular velocity for acquiring the deflection angle is started is set to the time when the angular velocity becomes equal to or higher than the angular velocity threshold value. As a result, FA zoom-out is not performed when slow panning is performed, and FA zoom-out is performed when sudden panning is performed.

The flowchart of FIG. 14 shows the processing of FA zoom control (angle of view control) performed by the microcomputer 102. The microcomputer 102 performs this process according to the FA zoom control program which is a computer program.

In step S200, the zoom control unit 118 holds the current zoom position as the reference zoom position f0. This reference zoom position f0 is a target zoom position when the FA zoom-in is performed after the FA zoom-out is performed.

Next, in step S201, the zoom control unit 118 calculates a target zoom position (hereinafter referred to as a target zoom-out position) f1 when performing FA zoom-out. For example, when the zoom-out amount in FA zoom-out is set to 1 / n times in terms of focal length, the target zoom-out position f1 is f0 / n. The value n for calculating the target zoom-out position may be a predetermined fixed value or a value that can be set by the photographer through a menu operation or the like. Further, the zoom-out amount may be set by the number of steps. In this case, the FA zoom out causes the zoom lens to move to the wide-angle side by m steps. The value m may be a predetermined fixed value or a value that can be set by the photographer.

Next, in step S202, the motion determination unit 128 determines whether or not the angular velocity is equal to or greater than the angular velocity threshold value. If the angular velocity is equal to or greater than the angular velocity threshold value, the process proceeds to step S204, otherwise the process proceeds to step S203.

In step S203, the runout angle is reset to 0, the current routine is terminated, and the process returns to step S200. In the first embodiment, the shake correction amount calculation unit 103 performed the acquisition of the runout angle (acceleration integration), but in this embodiment, the runout angle reference position used for the shake correction and the runout angle used for the FA zoom function. Since the reference position is different from that of the above, the deflection angle used for the FA zoom function is acquired by the motion determination unit 128, but is not limited to this. In this embodiment, the reference position of the runout angle used for blur correction is the position of the optical axis X when the blur correction function is turned on.

On the other hand, in step S204, the zoom control unit 118 calculates the runout angle. Specifically, the runout angle is calculated by integrating the angular velocities obtained from the runout detection sensor 101. Further, when the runout angle is calculated in step S204 of the previous routine, the runout angle calculated last time and the runout angle calculated this time are integrated. As a result, it is possible to calculate the runout angle (integral value of the angular velocity starting from the time when the angular velocity becomes equal to or higher than the angular velocity threshold value) with the position of the optical axis as the reference position when the angular velocity becomes equal to or higher than the angular velocity threshold value. it can. If the runout angle was calculated in the previous routine, the integrated value of the previous runout angle and the current runout angle is used. In the previous routine, the steps S202 to S203 are used. If the runout angle is not calculated, the current integration result is used. Let a certain runout angle be the runout angle calculated in this step. Then, the process proceeds to step S205.

In step S205, the motion determination unit 128 causes a sudden panning as if the runout angle (integrated value) is equal to or greater than the runout angle threshold value (equal to or greater than the runout angle threshold value), that is, the photographer searches for the subject. Determine if it is done. The zoom control unit 118 proceeds to step S206 when the runout angle is equal to or greater than the runout angle threshold value, and performs FA zoom out assuming that the subject is captured in the imaging screen when the runout angle is smaller than the runout angle threshold value. Return to step S200 without doing so.

In step S206, the zoom control unit 118 performs FA zoom out toward the target zoom out position f1 calculated in step S201.

Then, in step S207, the zoom control unit 118 determines whether or not the current zoom position has reached the target zoom-out position f1. If the target zoom-out position f1 has not been reached, the process returns to step S206 to continue FA zoom-out. When the target zoom-out position f1 is reached, the process proceeds to step S208.

In step S208, the zoom control unit 118 causes the motion determination unit 128 to determine whether the subject is captured in the imaging screen or whether sudden panning is performed so as to search for the subject. The zoom control unit 118 proceeds to step S209 when the subject is captured in the imaging screen, and repeats this step while holding the zoom position at f1 when panning is performed so as to search for the subject.

In step S209, the zoom control unit 118 performs FA zoom-in toward the reference zoom position f0 held in step S200.

Then, in step S210, the zoom control unit 118 determines whether or not the current zoom position has reached the reference zoom position f0, and if the current zoom position has not reached the reference zoom position, FA zooming is continued. On the other hand, when the reference zoom position f0 is reached, the current routine is terminated and the process returns to step S200.

After the FA zoom-in is completed, the zoom control unit 118 may change the value n (or m) for calculating the target zoom-out position. If the photographer does not change the imaging angle of view through the zoom key operation, the reference zoom position f0 and the target zoom-out position f1 may be held and the next routine may be started from step S202.

As described above, according to the present embodiment, the integral value of the angular velocity from the time when the angular velocity becomes equal to or higher than the angular velocity threshold value is used as the deflection angle, and the deflection angle is reset when the angular velocity becomes less than the angular velocity threshold value. Then, when the runout angle becomes equal to or higher than the runout angle threshold value, FA zoom out is performed. As a result, FA zoom-out is not performed when the camera is slowly moved while capturing the subject in the captured image, and FA zoom-out is performed when the camera is moved quickly and greatly so as to search for a subject deviated from the captured image. Can be done.

In the above-described embodiment, the reference value of the runout angle is set to 0, but as a modification of this embodiment, there is an example in which the runout angle for a certain period when the angular velocity is less than the angular velocity threshold value is used as the reference value.

FIG. 15 shows the runout angle reference value. In this embodiment, the average value of the runout angle is calculated in a predetermined period (reference value calculation period) while the angular velocity is slower than the angular velocity threshold value, and this is used as the runout angle reference value. Then, the runout angle from the runout angle reference value is calculated in a state where the angular velocity is equal to or higher than the angular velocity threshold value, and FA zoom out is performed when the runout angle becomes larger than the runout angle threshold value. The runout angle from the runout angle reference value can also be obtained by taking the difference between the runout angle with the reference value set to 0 and the runout angle reference value as in the above embodiment. The reference value calculation period can be appropriately set, and may be set to, for example, 1 second.

While the subject is captured in the imaging screen without panning (and while the subject is captured in the imaging screen while performing slow panning (not shown)), the time during which the angular velocity exceeds the angular velocity threshold is short. It's time. Therefore, each time the angular velocity becomes slower than the angular velocity threshold value, the runout angle is averaged with the runout angle up to that point, and the averaged runout angle does not exceed the runout angle threshold value. Therefore, it is possible to prevent FA zoom-out from being performed while the subject can be followed by slow panning.

On the other hand, as shown in FIG. 15, by performing a sudden panning to search for the lost subject, the angular velocity becomes faster than the angular velocity threshold value, and the state continues. As a result, the runout angle based on the runout angle reference value becomes equal to or greater than the runout angle threshold value (E'). In this case, FA zoom out is performed. In this way, even if a runout angle whose reference value is the average value of the runout angle during the runout angle reference value calculation period is used, FA zoom-out is not performed when slow panning is performed, and sudden panning is performed. At that time, FA zoom out can be performed.

In addition, if the average value of the runout angle in the reference value calculation period is used as the runout angle reference value, the average runout angle in a certain stop period before the panning operation can be used as the reference value. It is possible to determine the displacement of the runout angle due to the above with more accuracy.

In this embodiment, the motion is determined using the angular velocity and the runout angle, but the angular acceleration can also be used instead of the angular velocity. Further, instead of using the runout detection sensor, the motion vector may be detected from a plurality of consecutive frame images obtained by moving image imaging, and the angular velocity (angular acceleration) and the runout angle may be obtained from the motion vector, or from the motion vector. The speed, acceleration, and movement amount of the main subject image may be acquired. Even when using a motion vector, the amount of movement of the main subject image should be reset when the magnitude of the motion vector becomes less than a predetermined value in order to prevent FA zoom-out during slow panning. Is preferable.

The fourth embodiment of the present invention is a combination of the first embodiment and the third embodiment, and is a method of acquiring the amount of movement of the subject image when it is determined that the motion vector cannot be detected in step S101 of the flowchart shown in FIG. Is different from Example 1. In this embodiment, if it is determined in step S101 that the motion vector cannot be detected, the deflection angle is acquired with the time point at which the angular velocity becomes equal to or higher than the angular velocity threshold value as in Example 3, and the acquired deflection angle is acquired. The amount of movement d ^* of the subject image is acquired by setting the angular velocity as α in Equation 2. At this time, similarly to the third embodiment, when the angular velocity becomes less than the angular velocity threshold value, the runout angle is reset to 0 (or the calculated runout angle reference value). While the electronic zoom is being executed (when the electronic zoom magnification is ≠ 1), the point of correction by the electronic zoom magnification is the same as in the first embodiment.

Although the present invention has been described in detail based on the preferred examples thereof, the present invention is not limited to these specific examples, and various forms within the scope of the gist of the present invention are also included in the present invention. included. Some of the above-mentioned examples may be combined as appropriate. That is, the image pickup apparatus does not necessarily have to be composed of one mechanism. For example, in the application to an interchangeable lens type camera, a configuration including a microcomputer, a shake detection sensor, a blur correction mechanism, and a zoom lens in a lens device that can be attached to the camera body is also included in the technical scope of the present invention. .. In addition, a detection example using a motion vector with respect to the amount of movement of the main subject image has been described. Not limited to this, it can also be realized in a form of calculating the movement amount of the main subject image by using, for example, subject recognition, face position detection, or the like.
[Other Examples]
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100,200 Imaging device 101 Shake detection sensor 103 Shake correction amount calculation unit 108 Shake correction mechanism unit 116,216 Movement amount calculation unit 117,217 Capture judgment unit 118,218 Zoom control unit 122,222 Electronic zoom control unit 125 Motion vector detection Part 129, 228 Threshold acquisition part

Claims (16)

A zoom control device that controls the angle of view.
An acquisition means for acquiring the amount of movement of the imaging optical system used for imaging the main subject, and
Image blur correction control means that controls image blur correction means that corrects image blur,
Based on the difference between the correction amount and the motion amount obtained by the obtaining means according to the image blur correcting means, calculation means for exiting calculate the movement amount of the main object image in the captured image,
A determination means for determining whether or not the movement amount of the main subject image is equal to or greater than the first threshold value.
Control to control the angle of view to be wider than the angle of view when the movement amount of the main subject image is smaller than the first threshold value when the movement amount of the main subject image is equal to or greater than the first threshold value. A zoom control device characterized by having means. It has an electronic zoom control means that changes the electronic zoom magnification by image processing.
When the electronic zoom magnification is changed by the electronic zoom control means, the calculation means corrects the movement amount of the main subject image by using the electronic zoom magnification, and calculates the corrected value as the movement amount. The zoom control device according to claim 1, wherein the zoom control device is characterized. It has a detection means that detects motion vectors from images captured at different times.
When the motion vector of the main subject image is detected by the detection means, the calculation means calculates the movement amount of the main subject image based on the motion vector, and the motion vector is not detected by the detection means. In this case, the zoom control device according to claim 1 or 2 , wherein the movement amount of the main subject image is calculated based on the difference between the correction amount and the movement amount. The zoom control according to any one of claims 1 to 3 , wherein the first threshold value differs depending on at least one of the focal length of the imaging optical system and the distance to the main subject. apparatus. The first threshold value when the focal length of the imaging optical system or the distance to the subject is large is smaller than the first threshold value when the focal length of the imaging optical system or the distance to the subject is small. The zoom control device according to claim 4 . It has an electronic zoom control means that changes the electronic zoom magnification by image processing.
The zoom control device according to claim 4 , wherein the first threshold value differs depending on the electronic zoom magnification. The control means performs zoom-out control for widening the angle of view when the movement amount of the subject image is equal to or greater than the first threshold value, and the movement amount of the subject image is performed while the zoom-out control is being performed. The zoom control device according to any one of claims 1 to 6 , wherein zooming control for narrowing the angle of view is performed when is smaller than the second threshold value. A threshold value acquisition means for acquiring the first and second threshold values that differ depending on at least one of the focal length of the imaging optical system and the distance to the main subject.
It has an electronic zoom control means that changes the electronic zoom magnification by image processing.
Wherein when the electronic zoom magnification is greater than 1, claim, wherein the difference between the first threshold and the second threshold value that is acquired by the threshold acquisition means is greater than or equal to a preset value 7 The zoom control device described in. The acquisition means acquires at least one of the speed and acceleration of the movement of the imaging optical system as a first value regarding the movement of the imaging optical system.
The calculation means acquires a second value, which is a value relating to the amount of movement of the motion of the imaging optical system from the time when the first value becomes equal to or higher than the third threshold value.
The zoom control device according to any one of claims 1 to 8, wherein the movement amount of the main subject image is acquired based on the difference between the second value and the correction amount . The calculation means resets the second value when the first value becomes less than the third threshold value, and again determines the time when the first value becomes equal to or higher than the third threshold value. The zoom control device according to claim 9 , wherein the second value is reacquired as a reference. It has a detection means that detects motion vectors from images captured at different times.
When the motion vector of the main subject is detected by the detection means, the calculation means calculates the movement amount of the main subject image based on the motion vector of the main subject, and the detection means calculates the movement amount of the main subject. The zoom control device according to claim 9 or 10 , wherein when the motion vector is not detected, the movement amount of the main subject image is calculated based on the difference between the correction amount and the second value. .. The control means uses the average value of the second values in a predetermined period during which the first value is equal to or less than the third threshold value as a reference value, and the first value becomes equal to or higher than the third threshold value. The zoom control device according to any one of claims 9 to 11, wherein the second value is acquired from the time when the second value is obtained. The zoom control device according to any one of claims 9 to 12 , wherein the first value is an angular velocity or an angular acceleration, and the second value is a deflection angle. The zoom control device according to any one of claims 1 to 13 .
An image pickup apparatus including an image pickup device that performs image pickup at the angle of view. The imaging device according to claim 14 , further comprising an imaging optical system in which the angle of view is changed by the zoom control device. A zoom control method executed in a zoom control device that controls the angle of view.
The process of acquiring the amount of movement of the imaging optical system used for imaging the main subject, and
The process of correcting image blur and
Based on the difference between the amount of image blur correction by the step of correcting the blur of the image and the amount of movement acquired by the step of acquiring the amount of movement, the amount of movement of the main subject image in the captured image is determined . a step of de San,
A step of determining whether or not the movement amount of the main subject image is larger than the first threshold value, and
A step of controlling the angle of view to be wider than the angle of view when the movement amount of the main subject image is smaller than the first threshold value when the movement amount of the main subject image is larger than the first threshold value. A zoom control method characterized by having and.