JP6833607B2 - Control device, image pickup device, control method, program, and storage medium - Google Patents

Description

The present invention relates to an imaging device capable of panning.

There is panning as a shooting technique that expresses the sense of speed of the subject (moving object). The purpose of this photographing technique is to make a moving subject image stationary and to flow a background image by moving the camera according to the movement of the subject. When the photographer moves the camera according to the movement of the subject in panning, if the speed of moving the camera is not appropriate, there will be a difference between the moving speed of the subject and the speed of moving the camera, resulting in a blurred image of the subject. Often.

Patent Document 1 discloses a method of absorbing a difference between a moving speed of a subject and a moving speed of a camera by moving a shift lens. However, even if the moving speed of the subject changes during exposure for panning, for example, the subject may be blurred.

Patent Document 2 discloses an imaging device that determines the success or failure of panning and changes the recording method according to the contrast value of the subject in the panning image. According to the image pickup apparatus disclosed in Patent Document 2, by determining the success or failure of the panning shot, the information on the success or failure can be utilized in the subsequent processing.

Japanese Unexamined Patent Publication No. 2006-317848 Japanese Unexamined Patent Publication No. 2014-150370

However, although the image pickup apparatus disclosed in Patent Document 2 detects a subject from a captured image, it is difficult to detect the subject when panning fails. This will be described with reference to FIG. FIG. 4 is an explanatory diagram of panning. As shown in FIG. 4, before and after the main shooting of the panning shot, the pre-shooting frames (a-1) and (a-2) and the post-shooting frames (a-3) and (a-4) are acquired. To do. These frames are acquired, for example, as images (through images) for the photographer to confirm the shooting angle of view.

Here, if the panning is successful, it is possible to acquire an image in which the subject is stationary and only the background is flowing, as in the recorded image (b) in FIG. On the other hand, when panning fails, the subject becomes a blurred image as shown in the recorded image (c) in FIG. For this reason, when trying to detect a subject from an image in which panning has failed, it is difficult to distinguish whether the subject is blurred because the blurred area is the background.

Even if the subject information detected from the pre-shooting frames (a-1) and (a-2) is used, the panning shot is taken in a long time so that the background flows, so the shooting conditions of the frames before and after shooting , The shooting conditions of the recorded image by the main shooting are often different from each other. In the example of FIG. 4, the aperture value, the exposure time, and the sensitivity are different from each other. Therefore, the subject areas in the pre-shooting frame (a-1) and the recorded images (b) and (c) are different from each other as in (d), (e), and (f). As a result, it is difficult to determine the criteria for blurring.

Therefore, the present invention provides a control device, an imaging device, a control method, a program, and a storage medium capable of calculating a panning success rate for determining the success or failure of a captured image during panning. With the goal.

The control device as one aspect of the present invention acquires the first information from the first image before the start of the panning shooting of the imaging device, and from the second image after the panning shooting of the imaging device is completed. It has an acquisition means for acquiring the second information, and a calculation means for calculating data regarding whether or not the panning shooting was successful based on the first information and the second information.

An imaging device as another aspect of the present invention includes an imaging element that photoelectrically converts an optical image formed via an imaging optical system, and the control device.

The control method as another aspect of the present invention includes a step of acquiring the first information from the first image before the start of the panning shooting of the imaging device, and a second step after the panning shooting of the imaging device is completed. It has a step of acquiring the second information from the image of the above, and a step of calculating data regarding whether or not the panning shooting was successful based on the first information and the second information.

A program as another aspect of the present invention causes a computer to execute the image processing method.

A storage medium as another aspect of the present invention stores the program.

Other objects and features of the present invention will be described in the following embodiments.

According to the present invention, there is provided a control device, an imaging device, an image processing method, a program, and a storage medium capable of calculating a panning success rate for determining the success or failure of a captured image during panning. can do.

It is a block diagram of the image pickup apparatus in each embodiment. It is a flowchart which shows the calculation process of the panning success rate in 1st Embodiment. It is a flowchart which shows the calculation process of the panning success rate in 2nd Embodiment. It is explanatory drawing of the panning shot in each embodiment. It is explanatory drawing of the pre-exposure information and the post-exposure information in the first embodiment. It is explanatory drawing of the pre-exposure information and the post-exposure information in a third embodiment. It is explanatory drawing of the pre-exposure information and the post-exposure information in a fourth embodiment. It is explanatory drawing of the pre-exposure information and the post-exposure information in a fifth embodiment. It is explanatory drawing of the pre-exposure information and the post-exposure information in the sixth embodiment. It is a figure which shows the use example of the panning success rate in 7th Embodiment. It is a figure which shows the use example of the panning success rate in 8th Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First Embodiment)
First, the image pickup apparatus according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram of the image pickup apparatus 100 according to the present embodiment. The image pickup device 100 may be not only a camera such as a digital camera or a digital video camera, but also an electronic device having a camera function such as a mobile phone with a camera function or a computer with a camera function. The optical system 101 is an imaging optical system (imaging optical system) including a lens group, a shutter, an aperture, and the like. The lens group includes a correction lens and a focus lens that correct image blur due to camera shake and the like.

The optical system 101 forms an image of light from a subject on an image pickup device 102 according to a control signal of a CPU (central processing unit, control device) 103. The image pickup device 102 is an image pickup device such as a CCD (charge-coupled device) image sensor or a CMOS (complementary metal oxide semiconductor) image sensor, and photoelectrically converts an optical image (subject image) formed via the optical system 101. And output the image signal. In the present embodiment, the optical system 101 is integrally configured with the image pickup apparatus main body including the image pickup element 102, but the present invention is not limited to this. This embodiment can also be applied to an image pickup apparatus provided with an image pickup apparatus main body and an optical system (interchangeable lens) that can be attached to and detached from the image pickup apparatus main body. The angular velocity sensor 105 is, for example, a gyro sensor, detects an angular velocity representing the amount of movement of the image pickup apparatus 100, and outputs an angular velocity detection signal as an electric signal to the CPU 103.

The CPU (control device) 103 controls each unit constituting the image pickup device 100 according to an input signal or the like by executing a program stored in advance in a memory (storage means) such as the secondary storage device 108. In the present embodiment, the CPU 103 includes acquisition means 103a, calculation means 103b, drive means 103c, and recording control means 103d. The primary storage device 104 is, for example, a volatile memory such as a RAM (random access memory), stores temporary data, and is used as a work memory of the CPU 103. The data stored in the primary storage device 104 is used by the image processing device 106 and may be recorded on the recording medium 107. The secondary storage device 108 is, for example, a non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory). The secondary storage device 108 stores a program (firmware) for controlling the image pickup device 100 and various setting information. The programs and various information stored in the secondary storage device 108 are used by the CPU 103.

The recording medium (recording means) 107 records captured image data and the like stored in the primary storage device 104 based on the control by the recording control means 103d. The recording medium 107 is removable from the image pickup apparatus 100, for example, like a semiconductor memory card. The data recorded on the recording medium 107 can be attached to an external device such as a personal computer and read out. As described above, the image pickup apparatus 100 has a mechanism for attaching / detaching the recording medium 107 and a reading / writing function. The display unit 109 displays a viewfinder image at the time of shooting, a shot image, a GUI (graphical user interface) image for interactive operation, and the like. The operation unit 110 is a group of input devices that accepts the operation of the photographer and transmits input information to the CPU 103, and includes buttons, levers, a touch panel, and the like. The operation unit 110 may also include an input device that can be operated by using voice, line of sight, or the like.

The image processing device 100 of the present embodiment has a plurality of image processing patterns applied to the captured image by the image processing device 106, and the image processing pattern can be set by the operation unit 110 as the image processing mode. In addition to the so-called development process, the image processing device 106 performs various image processes such as color tone adjustment according to the shooting mode. At least a part of the functions of the image processing device 106 may be realized by software processing by the CPU 103.

Next, the calculation process of the panning success rate in the present embodiment will be described with reference to FIG. FIG. 2A is a flowchart showing a process of calculating the success rate of panning. FIG. 2B is a flowchart showing the details of step S207 of FIG. 2A. Each step of FIGS. 2A and 2B is mainly executed by each part of the CPU 103.

First, in step S201, the CPU 103 acquires a frame (frame image). The CPU 103 acquires, for example, a frame image before the main shooting of the panning shot, as shown as the pre-shooting frame (a-1) in FIG. Since step S201 is executed a plurality of times as described later, a plurality of frames exist before the pre-shooting frame (a-1) in FIG. Subsequently, in step S202, the CPU 103 detects a subject (subject information) from the frame image acquired in step S201. Regarding the detection of the subject, various methods such as a method of extracting the subject area using the motion vector obtained from the frame image, as disclosed in Patent Document 1, are known. The description of is omitted.

Subsequently, in step S203, the CPU 103 determines whether or not to perform the exposure operation for the main shooting of the panning shot. In the present embodiment, the CPU 103 determines whether or not the shutter button included in the operation unit 110 is fully pressed (hereinafter referred to as S2). When the full press S2 is not performed, the CPU 103 repeats the operations of steps S201 and S202 to sequentially acquire images such as the pre-shooting frames (a-1) and (a-2) of FIG. On the other hand, if the full push S2 is performed in step S203, the process proceeds to step S204.

In step S204, the CPU 103 (acquisition means 103a) acquires the pre-exposure information 211. FIG. 5 is an explanatory diagram of information before and after exposure in the present embodiment. In the present embodiment, the CPU 103 divides the screen at equal intervals and sets a plurality of blocks (subject block, background block) as shown by the frame (a) in FIG. The CPU 103 stores the position of the block to which the subject belongs (subject block) among the plurality of blocks as the pre-exposure information 211 based on the subject information obtained in the last step S202. The pre-exposure information 211 is stored in, for example, the primary storage device 104.

Subsequently, in step S205, the CPU 103 starts the exposure operation for panning. The CPU 103 controls, for example, the shutter included in the optical system 101. Subsequently, in step S206, the CPU 103 determines whether or not a predetermined exposure time as the exposure time has elapsed. If the exposure time has not elapsed, step S206 is repeated until the exposure time has elapsed. On the other hand, when the exposure time has elapsed, the process proceeds to step S207. In step S207, the CPU 103 ends the panning exposure operation. Subsequently, in step S208, the CPU 103 (acquisition means 103a) acquires the post-exposure information 212 in the same manner as in step S204.

Here, the method of acquiring the post-exposure information 212 (step S208) will be described in detail with reference to FIG. 2 (b). FIG. 2B is a flowchart of a method for acquiring post-exposure information 212 (step S208).

First, in step S2081, the CPU 103 acquires a frame. The CPU 103 acquires, for example, an image as shown as a post-shooting frame (a-3) and (a-4) in FIG. It should be noted that the image acquired here does not necessarily have to be displayed so as to be visible to the photographer. Subsequently, in step S2082, the CPU 103 detects the subject (subject information). Similar to step S202, the CPU 103 detects the subject by extracting the subject area from the post-shooting frames (a-3) and (a-4) acquired in step S2081. Subsequently, in step S2083, the CPU 103 acquires post-exposure information 212 based on the subject information acquired in step S2082. In the present embodiment, the CPU 103 acquires the post-exposure information 212 in the same manner as in step S204. For example, in the CPU 103, as shown as the frame (b) in FIG. 5, the subject belongs to a plurality of blocks obtained by dividing the screen at equal intervals based on the subject information obtained in step S2082. The position of the block is stored as post-exposure information 212. The post-exposure information 212 is stored in, for example, the primary storage device 104.

Subsequently, in step S209 of FIG. 2A, the CPU 103 (calculation means 103b) calculates the panning success rate. Here, the calculation process of the panning success rate in the present embodiment will be described with reference to FIG. At the time of panning, it is necessary that the image pickup device 100 is able to follow the subject by moving the image pickup device 100 (camera) at an appropriate speed as described above. When the subject can be tracked, the position of the subject on the image pickup surface (on the surface of the image pickup element 102) of the image pickup apparatus 100 is always a constant position. For example, when the subject before exposure is at the position shown in the frame (a) in FIG. 5, the subject after exposure is at the same position as the frame (a) with respect to the screen, as in the frame (b). Must be. On the other hand, when the panning has failed, the image pickup apparatus 100 has not been able to properly follow the subject, so that the subject after exposure has a different subject position from the frame (a) like the frame (c). It is likely that it is in position. Therefore, in the present embodiment, the success rate of panning is calculated by comparing the subject positions before and after exposure. As a method for detecting the subject position, as described above, various methods such as a method for extracting a subject area using a motion vector obtained from a frame image as in Patent Document 1 are known. The description here will be omitted.

In the present embodiment, it is possible to calculate the success rate of panning by, for example, determining how much the subject areas overlap before and after exposure. First, in the frame (a) in FIG. 5, four blocks of subjects shown by solid lines exist among the plurality of blocks obtained by dividing the screen at equal intervals. On the other hand, in the frame (b) of FIG. 5, the number of subject blocks overlapping the subject block of the frame (a) of FIG. 5 is 4. The panning success rate R _{b at} this time is calculated by, for example, the following equation (1).

On the other hand, for example, in the frame (c) of FIG. 5, the number of subject blocks overlapping the subject block of the frame (a) is 2. The panning success rate R _{c at} this time is calculated by, for example, the following equation (2).

As described above, in the present embodiment, the control device (CPU 103) has the acquisition means 103a and the calculation means 103b. The acquisition means 103a acquires the first information (pre-exposure information 211) from the first image (image before the main shooting) before the start of the panning shooting of the imaging device, and after the panning shooting of the imaging device is completed. The second information (post-exposure information 212) is acquired from the second image (image after the main shooting). The calculation means calculates data regarding whether or not the panning is successful based on the first information and the second information.

Preferably, the calculation means 103b calculates the panning success rate (success degree) as data regarding whether or not the panning is successful. Further, preferably, the acquisition means 103a acquires information on the subject area (area including the main subject) from the first image as the first information, and information on the subject area from the second image as the second information. To get. More preferably, the acquisition means 103a acquires information about an area including a moving body (moving subject) as information about the subject area. Further, preferably, the acquisition means 103a acquires information on the position (subject block) of the subject area as information on the subject area.

According to the present embodiment, it is possible to calculate the panning success rate based on the subject information before exposure (pre-exposure information 211) and the subject information after exposure (post-exposure information 212). In addition, by using the subject information before and after panning, the detection of the subject is not affected by the success or failure of panning, and it is based on the image taken under the same conditions in the calculation processing of the panning success rate. It is possible to make a judgment.

(Second Embodiment)
Next, a second embodiment of the present invention will be described with reference to FIG. In the present embodiment, the optical system 101 includes a shift lens capable of moving in a direction orthogonal to the optical axis. The image pickup apparatus 100 of the present embodiment has a function of absorbing the difference between the moving speed of the subject and the moving speed of the imaging apparatus 100 by moving the shift lens to assist the photographer in panning.

FIG. 3 is a flowchart showing the calculation process of the panning success rate in the present embodiment. Each step of FIG. 3 is mainly executed based on a command of the CPU 103. FIG. 3 differs from the first embodiment described with reference to FIG. 2 in that shift lens drive (step S306) and shift lens centering (step S311) are added. Since the other steps S301 to S305 and S307 to S310 in FIG. 3 are the same as the steps S201 to S209 in FIG. 2, their description will be omitted.

In step S306, the CPU 103 drives the shift lens. By driving the shift lens, it is possible to absorb the difference between the moving speed of the subject and the moving speed of the image pickup apparatus 100 to assist the photographer in panning. As for the assistance of panning, various methods such as a method of driving a shift lens by converting a motion vector of a subject into a movement amount of the shift lens, as disclosed in Patent Document 1, are known. Therefore, the description here is omitted.

In this way, the photographer is assisted in panning, but when the shooting is completed, it is necessary to return the position of the shift lens to the initial position for the next shooting. Therefore, in step S311 the CPU 103 centers the shift lens. When the shift lens centering operation is performed, the position of the image to be captured on the imaging surface also moves. At this time, since the position of the subject on the imaging surface also changes, it is difficult to calculate the panning success rate as described above. Therefore, by centering the shift lens after the acquisition of the post-exposure information 212 in step S309, the panning success rate can be calculated easily and accurately.

As described above, in the present embodiment, the driving means 103c drives the shift lens in the direction orthogonal to the optical axis of the imaging optical system so as to assist the panning shot. Preferably, the driving means 103c drives the shift lens to the initial position (centering) after the acquiring means 103a acquires the second information (post-exposure information 212). Therefore, according to the present embodiment, it is possible to calculate the success rate of panning while assisting the panning shot of the photographer by moving the shift lens.

(Third Embodiment)
Next, a third embodiment of the present invention will be described with reference to FIG. In the first embodiment, a method of using information of a block (subject block) in the screen to which the subject belongs as pre-exposure information and post-exposure information has been described, but in the present embodiment, as pre-exposure information and post-exposure information. A method of using the brightness information in the subject area will be described. FIG. 6 is an explanatory diagram of pre-exposure information and post-exposure information in the present embodiment.

In the present embodiment, as the pre-exposure information 211 in step S203, the luminance histogram of the block to which the subject belongs (subject block) among the plurality of blocks obtained by dividing the screen at equal intervals is stored. At this time, for example, from the image data (a-2) in FIG. 6, a histogram (a-3) showing the relationship between the brightness and the power is acquired. Similarly, for the post-exposure information 212, histograms (b-3) and (c-3) are acquired from the image data (b-2) or (c-2), respectively.

Here, for example, a histogram (a-3), (b -3), (c-3) , respectively _{H a,} H b, when the _{H c,} shot success ratio _R b, _{R c,} for example the following formula It is calculated as in (3) and (4), respectively.

In the formulas (3) and (4), Min (a, b) means to select the smaller value of a and b. In the present embodiment, the histogram _{H a,} H b, represents each _{H c} in degrees of pixel values 0 to 255, represents the frequency of a pixel value x as _H a [x].

In this way, the panning success rate can be calculated based on the subject information before exposure (pre-exposure information 211) and the subject information after exposure (post-exposure information 212). In this embodiment, the subject position is acquired before the exposure, and is not reacquired after the exposure. Therefore, there is an advantage that at least one frame is required to be acquired after exposure.

The acquisition unit 103a of the present embodiment acquires the luminance information (pixel value of the subject area) of the subject area as the information regarding the subject area. Then, the calculation means 103b calculates the difference between the pixel values only in the subject area in order to calculate the panning success rate. In panning, it is ideal that the subject is always imaged at the same position on the imaging surface, but since the background is flowing, the position where the background is imaged changes. According to this embodiment, it is possible to calculate the panning success rate by comparing the subject information before and after exposure only in the subject area.

(Fourth Embodiment)
Next, a fourth embodiment of the present invention will be described with reference to FIG. 7. In this embodiment, a method of using the vector information of the subject area as the pre-exposure information and the post-exposure information will be described. FIG. 7 is an explanatory diagram of pre-exposure information and post-exposure information in the present embodiment.

For example, in the subject detection in step S201 of FIG. 2, the CPU 103 calculates the motion vector in each block as in the frame (a-1) in FIG. 7 by the method disclosed in Patent Document 1. .. Then, the CPU 103 extracts a motion vector in the subject area from the calculated motion vector. Subsequently, the CPU 103 can obtain a histogram (a-2) by taking a histogram for each movement amount. Similarly, the histograms (b-2) and (c-2) of FIG. 7 can be obtained for the post-exposure information. For example, a histogram (a-2), (b -2), (c-2) degrees of each _H a _of, H b, when the _{H c,} shot success ratio _R b, _{R c,} for example the following formula ( It is calculated as in 5) and (6), respectively.

In the present embodiment, in the histogram H _a, it represents the frequency of a certain vector quantity x as H _{a [x],} integrating with respect to the total amount of movement of this.

As described above, in the present embodiment, the acquisition means 103a acquires information (motion vector, vector information) regarding the amount of movement of the subject area as information regarding the subject area. Preferably, the acquisition unit 103a has a first movement amount (first vector information) related to the first image and a second movement amount (second vector) related to the second image as information regarding the movement amount of the subject area. Information) and get. Then, the calculation means 103b compares the first movement amount and the second movement amount (based on the similarity between the first movement amount and the second movement amount), and calculates the panning success rate. ..

In this way, the panning success rate can be calculated based on the subject information before exposure (pre-exposure information 211) and the subject information after exposure (post-exposure information 212). According to the method of the present embodiment, it is less affected by the size of the subject and the contrast of the subject than in the first embodiment and the second embodiment.

(Fifth Embodiment)
Next, a fifth embodiment of the present invention will be described with reference to FIG. In this embodiment, a method of using pre-exposure and post-exposure motion vectors as pre-exposure information and post-exposure information will be described. In the fourth embodiment, the motion vector calculated between the pre-shooting frames (a-1) and (a-2) in FIG. 4 and the post-shooting frames (a-3) and (a-4) The similarity of the calculated motion vectors was determined. On the other hand, in the present embodiment, the motion vector is calculated between the pre-shooting frame (a-2) and the post-shooting frame (a-3).

FIG. 8 is an explanatory diagram of pre-exposure information and post-exposure information in the present embodiment. For example, as the pre-exposure information 211 acquired in step S204 of FIG. 2, information necessary for calculating the motion vector is retained. Similarly, the post-exposure information 212 in step S208 holds the information necessary for calculating the motion vector. When calculating the success rate in step S209, the CPU 103 calculates the motion vector in the subject area by using the pre-exposure information 211 and the post-exposure information 212. Here, in the case of panning, since the image pickup apparatus 100 follows the subject in accordance with the moving speed, the motion vector is ideally small.

_{Therefore, assuming that the powers of the motion vectors are H b} and H _c , respectively, as shown in the histogram (b-2) or (c-2) in FIG. 8, the panning success ratios R _b and R _c are expressed by, for example, the following equations ( 7) and (8) are calculated respectively.

(| X | is the absolute value of x)
In the equations (7) and (8), | x | indicates the absolute value of x. Further, in the equations (7) and (8), N is an arbitrary constant, and R _b and R _c each take a value of 0 or more and 1 or less. Further, the frequency of a certain vector quantity x in the histogram H _{b is expressed as H b} [x], and this is integrated with respect to the total vector quantity.

As described above, in the present embodiment, the acquisition means 103a compares the first image (image before the main shooting) with the second image (image after the main shooting) and provides information (vector) regarding the amount of movement of the subject area. Information) is acquired. Therefore, according to the present embodiment, it is possible to calculate the panning success rate based on the subject information before exposure (pre-exposure information 211) and the subject information after exposure (post-exposure information 212). In this embodiment, as in the third embodiment, the size of the subject and the contrast of the subject are not easily affected. Further, as in the second embodiment, there is an advantage that at least one frame is required to be acquired after exposure.

(Sixth Embodiment)
Next, a sixth embodiment of the present invention will be described with reference to FIG. In the first to fifth embodiments, attention was paid to whether or not a panning image could be taken while the subject was stationary, but as in the present embodiment, an evaluation value for the amount of panning in the background is calculated. Is also possible. FIG. 9 is an explanatory diagram of pre-exposure information and post-exposure information in the present embodiment. In the fifth embodiment, the motion vector of the subject area was calculated, but in the present embodiment, the motion vector of the background area is calculated. As a result, it is possible to obtain a histogram of the motion vector of the background as shown in the histograms (b-2) and (c-2) of FIG.

From the amount of movement of the background region obtained in this way, for example, the frequencies of the motion vectors shown as the histograms (b-2) and (c-2) in FIG. 9 are defined as H _b and H _c , respectively. In this case, the following equation (9), can be calculated as expressed by (10), the rate of success panning _R b, by multiplying the _{R c R b ', R c} '.

In equations (9) and (10), N is an arbitrary constant.

As described above, in the present embodiment, the acquisition means 103a acquires the information regarding the movement amount of the background region (the region including the background) from the first image as the first information, and the second information as the second information. Obtain information (vector information) regarding the amount of movement of the background area from the image. Therefore, according to the present embodiment, by calculating the evaluation value (information about the movement amount of the background area) with respect to the background flow amount, the higher the success rate is, the more the subject is stationary and the background flow amount is large. Can be calculated. Although various methods for calculating the success rate of panning shots have been described in the first to sixth embodiments, in each calculation method, for example, a specific method is selected according to the size of the subject and the shooting mode, or a plurality of methods are selected. It is also possible to combine calculation methods.

(7th Embodiment)
Next, a seventh embodiment of the present invention will be described with reference to FIG. In the first to sixth embodiments, the method of calculating the panning success rate has been described, but in the present embodiment, the method of using the calculated panning success rate will be described. FIG. 10 is an explanatory diagram of a method of using the panning success rate in the present embodiment.

For example, the success rate of panning is 0.98, 0.60, 0.50, 0.90 as shown in the frames (a), (b), (c), and (d) in FIG. (The higher the number, the higher the success rate of panning). By saving the panning success rate together with the captured image when recording the image, the saved success rate can be referred to in an image viewing tool or the like. For example, when the photographer selects a panning image, if the sort order is "following success rate order", the frames (e), (f), (g), and (h) in FIG. In addition, sort in descending order of success rate of panning. This allows the photographer to easily select a suitable panning image.

In the present embodiment, the recording control means 103d records the image data by the panning shot and the panning success rate in association with each other on the recording medium 107. That is, by recording (attaching) the panning success rate calculated at the time of shooting to the image, it is possible to reduce the time and effort for the photographer to select the image later.

(8th Embodiment)
Next, an eighth embodiment of the present invention will be described with reference to FIG. In the present embodiment, as in the seventh embodiment, other usage methods of the panning success rate will be described. FIG. 11 is an explanatory diagram of a method of using the panning success rate in the present embodiment.

In the present embodiment, in continuous shooting of panning, the shift lens is moved in a direction orthogonal to the optical axis based on the moving speed of the subject detected from the frames (a-1) and (a-2) in FIG. Consider the case where the photographer is moved to perform optical correction to assist the photographer in panning. First, considering the case where continuous shooting is continued without updating the movement speed of the detected subject, when the movement speed of the subject changes during continuous shooting, in the subsequent shooting, the captured image (d) in FIG. 11 There is a possibility of mass-producing such failed images. On the other hand, when the subject is detected and the moving speed is updated every time, the processing increases, so that the continuous shooting speed may decrease.

Therefore, for example, when the frames (a-3), (a-4), and (a-5) in FIG. 11 are acquired for each panning shot and the panning success rate is calculated for each frame, the captured image in FIG. 11 is obtained. In the shooting of (d), the success rate of panning shots is low. Here, for example, when the success rate of panning shot becomes lower than a certain threshold value, continuous shooting is interrupted once and then re-detection is performed to facilitate acquisition of a successful panning shot image while maintaining the continuous shooting speed. be able to.

As described above, in the present embodiment, the calculation means 103b calculates the panning success rate for each continuous shooting, and the driving means 103c changes the operation according to the panning success rate. Therefore, according to the present embodiment, it is possible to reduce the frequency of occurrence of failed images while maintaining the continuous shooting speed.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

According to each embodiment, a control device, an imaging device, a control method, a program, and a storage medium capable of calculating a panning success rate for determining the success or failure of a captured image at the time of panning are provided. can do.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and modifications can be made within the scope of the gist thereof.

103 CPU (control device)
103a Acquisition means 103b Calculation means

Claims (18)

An acquisition means for acquiring the first information from the first image before the start of the panning shooting of the imaging device and acquiring the second information from the second image after the panning shooting of the imaging device is completed.
A control device comprising: a calculation means for calculating data regarding whether or not the panning shooting was successful based on the first information and the second information. The control device according to claim 1, wherein the calculation means calculates a panning success rate as data regarding whether or not the panning shooting is successful. The acquisition means
As the first information, information about the subject area is acquired from the first image, and the information is obtained.
The control device according to claim 1 or 2, wherein as the second information, information about the subject area is acquired from the second image. The control device according to claim 3, wherein the acquisition means acquires information about an area including a moving body as information about the subject area. The control device according to claim 3 or 4, wherein the acquisition means acquires information regarding the position of the subject area as information regarding the subject area. The control device according to claim 3 or 4, wherein the acquisition means acquires luminance information of the subject area as information about the subject area. The control device according to claim 3 or 4, wherein the acquisition means acquires information on the amount of movement of the subject area as information on the subject area. The acquisition means acquires the first movement amount of the first image and the second movement amount of the second image as information about the movement amount of the subject area.
The calculation means according to claim 7, wherein the calculation means compares the first movement amount with the second movement amount to calculate data regarding whether or not the panning photography is successful. Control device. The control device according to claim 7, wherein the acquisition means acquires information on the movement amount of the subject area by comparing the first image with the second image. The acquisition means
As the first information, information on the amount of movement of the background area is acquired from the first image, and the information is obtained.
The control device according to claim 1 or 2, wherein as the second information, information regarding the amount of movement of the background region is acquired from the second image. Further having a driving means for driving the shift lens in a direction orthogonal to the optical axis of the imaging optical system,
The control device according to any one of claims 1 to 10, wherein the driving means drives the shift lens so as to assist the panning shooting. The control device according to claim 11, wherein the driving means drives the shift lens to an initial position after the acquiring means acquires the second information. The control device according to claim 11 or 12, wherein the driving means changes its operation according to data regarding whether or not the panning shooting is successful. Any one of claims 1 to 12, further comprising a recording control means for associating the image data obtained by the panning photography with the data relating to whether or not the panning photography was successful and recording the data in the recording means. The control device described in. An image sensor that photoelectrically converts an optical image formed via an image pickup optical system,
An acquisition means for acquiring the first information from the first image before the start of the panning shooting of the imaging device and acquiring the second information from the second image after the panning shooting of the imaging device is completed.
An image pickup apparatus comprising: a calculation means for calculating data regarding whether or not the panning shooting is successful based on the first information and the second information. The step of acquiring the first information from the first image before the start of panning photography of the imaging device, and
A step of acquiring the second information from the second image after the panning shooting of the imaging device is completed, and
A control method comprising: a step of calculating data regarding whether or not the panning shooting is successful based on the first information and the second information. The step of acquiring the first information from the first image before the start of panning photography of the imaging device, and
A step of acquiring the second information from the second image after the panning shooting of the imaging device is completed, and
A program characterized in that a computer is made to execute a step of calculating data regarding whether or not the panning shooting is successful based on the first information and the second information. A computer-readable storage medium that stores the program of claim 17.