JP6618324B2 - Zoom control device, imaging device, control method for zoom control device, control program for zoom control device, and storage medium - Google Patents

Description

  The present invention relates to a technique for detecting a subject image from an image and controlling a zoom magnification.

  Conventionally, a zoom lens is driven to perform optical zooming (hereinafter referred to as optical zoom), and a part of a captured image is enlarged to perform electronic zooming (hereinafter referred to as electronic zoom). There is an imaging device having a function. In addition, there is an auto-zoom (automatic scaling) function that automatically changes the zoom magnification according to the detection information of the subject. Hereinafter, the optical zoom operation and the electronic zoom operation are collectively referred to as a zoom operation.

  Patent Document 1 discloses a configuration of a camera equipped with an auto zoom function that keeps the size of a subject constant. Patent Document 1 discloses a method in which a photographer selects a composition from a plurality of options such as “face up”, “bust up”, and “whole body” in an auto zoom function. In the auto zoom control that maintains the size of the subject image, the size of the subject image serving as a reference for zoom control is determined based on the selected composition. For example, when the photographer selects the “whole body” composition, the zoom operation is performed so that the whole body image of the person who is the subject is included in the shooting screen.

JP 2012-95019 A

  However, in Patent Document 1, it is necessary to select in advance the composition that the photographer wants to shoot with the auto zoom function. For this reason, it is necessary for the photographer to perform a setting change operation every time the shooting scene or the state of the subject (movement, number, etc.) changes. In addition, if the composition setting is not appropriate for the scene to be photographed, there is a possibility that the subject cannot be photographed with an appropriate size. For example, when a “face” composition is set in a scene where a moving subject or a distant subject is shot, the subject may be out of frame. In addition, when a “whole body” composition is set in a scene where a still subject is photographed, the subject may be too small.

  In view of the above problems, an object of the present invention is to perform auto zoom control with a composition suitable for a scene determined based on subject detection information without the photographer selecting a composition.

  An apparatus according to the present invention includes a subject detection unit that detects a subject image from an image, a determination unit that acquires information including information detected by the subject detection unit and determines a scene, and a scene determined by the determination unit. Based on the composition selection means for selecting one of the plurality of compositions, the composition selected by the composition selection means, and the size and position of the subject image detected by the subject detection means, A setting means for setting a reference size of a subject image used for zoom magnification control, and a zoom magnification control based on the reference size set by the setting means and the size of the subject image sequentially detected by the subject detection means. Control means.

  According to the present invention, it is possible to perform auto zoom control with a composition suitable for a scene determined based on subject detection information without the photographer selecting a composition.

It is a block diagram which shows the structural example of the digital camera which concerns on embodiment of this invention. It is the figure which illustrated the relationship between a focal distance and a focus lens position for every subject distance. It is explanatory drawing of the process which prevents the frame out of the screen of a to-be-photographed object (object). It is explanatory drawing of the process which prevents the frame out of the screen of a to-be-photographed object (person). It is explanatory drawing of the process which prevents the size change within the screen of a to-be-photographed object (person). It is a figure explaining composition setting when a subject is a person. It is a flowchart explaining the flow of a process of an auto zoom function. It is a flowchart explaining a subject search process. It is a figure explaining the search end field of subject search processing. It is a flowchart explaining a subject designation process. It is a flowchart explaining another example of subject specification processing. It is a flowchart explaining the setting process of a reference | standard size. It is a flowchart explaining the calculation process of a screen size. FIG. 11 is a transition diagram for explaining automatic composition determination in composition setting “auto”. It is a figure which shows the outermost subject position and screen size. 10 is a flowchart for explaining a reference size calculation process in composition setting “auto”. It is a flowchart explaining the calculation process of a horizontal direction subject position ratio. It is a flowchart explaining the calculation process of a vertical direction subject position ratio. It is a table | surface which shows the number of faces which estimates a to-be-photographed object position. It is a flowchart explaining the process of auto zoom control. It is a flowchart explaining a zoom operation. FIG. 12 is a transition diagram for explaining automatic composition determination in composition setting “auto” in the second embodiment.

(First embodiment)
Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram illustrating a configuration example of a digital camera 100 having an auto zoom function as an example of an imaging apparatus including a zoom control device according to the present embodiment.

  The lens barrel 101 holds a lens group therein. The zoom lens 102 adjusts the focal length by moving in the optical axis direction of the lens barrel 101, and optically changes the angle of view (moves the zoom position). The focus lens 103 performs focus adjustment by moving in the optical axis direction of the lens barrel 101. The anti-vibration lens (image blur correction lens) 104 corrects image blur caused by camera shake or the like. A diaphragm and shutter 105 for adjusting the amount of light are used for exposure control. In the present embodiment, the digital camera 100 is an imaging device in which the lens barrel 101 and the camera main body are integrally configured, but is not limited to this. The present embodiment can also be applied to an imaging system including a camera body and an interchangeable lens that can be attached to and detached from the camera body.

  The imaging element 106 receives light that has passed through the lens barrel 101, and generates an imaging signal by converting a subject image into an electrical signal by photoelectric conversion. The image sensor 106 is a CCD (Charge Coupled Device) type or CMOS (Complementary Metal Oxide Semiconductor) type image sensor or the like. An imaging signal from the imaging element 106 is input to the image processing circuit 107, and various processes such as a pixel interpolation process and a color conversion process are performed. The image data after various processing is stored in the image memory 108. The image memory 108 is a storage device such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory).

  The display unit 109 includes a TFT type LCD (Thin Film Transistor Driven Liquid Crystal Display) and displays a captured image (image data) and specific information (for example, shooting information). An electronic viewfinder (EVF) function for the photographer to adjust the angle of view can be provided by displaying information such as a live view related to the photographed image.

  The aperture shutter driver 110 calculates an exposure control value (aperture value and shutter speed) based on the luminance information obtained by the image processing in the image processing circuit 107, and drives the aperture and shutter 105 based on the calculation result. . Thereby, automatic exposure (AE) control is performed. The anti-vibration lens driving unit 111 calculates a shake amount applied to the digital camera 100 based on shake detection information obtained by an angular velocity sensor such as a gyro sensor. According to the calculation result, the image stabilizing lens 104 is driven so as to cancel (reduce) the shake amount applied to the digital camera 100.

  The focus lens driving unit 112 drives the focus lens 103. In the present embodiment, the digital camera 100 performs automatic focus adjustment (AF) control by a contrast method. That is, the focus lens driving unit 112 drives the focus lens 103 so that the subject is focused based on the focus adjustment information (contrast evaluation value) of the photographing optical system obtained by the image processing in the image processing circuit 107. However, the present invention is not limited to this, and as the AF control other than the contrast method, the phase difference AF method may be used, and the AF control is performed by a plurality of methods such as a combination of the contrast method and another method. It may be configured.

  The zoom lens driving unit 113 drives the zoom lens 102 in accordance with a zoom operation instruction. The operation unit 117 includes a zoom lever or a zoom button as a zoom operation member for the photographer to instruct the camera to perform zooming. The system control unit 114 detects the operation amount and operation direction of the zoom operation member used for the zoom instruction operation, calculates the zoom drive speed and the zoom drive direction, and moves the zoom lens 102 along the optical axis according to the calculation result. Take control.

The image data generated by the shooting operation is sent to the recording unit 116 via the interface (I / F) unit 115 and recorded. The image data is recorded on an external recording medium, a nonvolatile memory 118, or both. The external recording medium is a memory card or the like used by being attached to the digital camera 100. The nonvolatile memory 118 is a storage medium built in the digital camera 100. In addition to program data and image data, the memory 118 stores setting information of the digital camera 100 and information such as a zoom-in position in an auto zoom function described later.
In addition to the zoom operation member, the operation unit 117 includes a release switch for instructing start of photographing, an auto zoom operation switch for instructing start and end of an auto zoom function, and the like. A signal from the operation unit 117 is sent to the system control unit 114.

  The system control unit 114 includes an arithmetic device such as a CPU (Central Processing Unit). The system control unit 114 controls the entire digital camera 100 by sending a control command to each unit in accordance with a photographer's operation. The system control unit 114 executes various control programs stored in the memory 118, for example, programs for performing control of the image sensor 106, AE / AF control, zoom control (including auto zoom processing), and the like.

  In order to maintain the in-focus state when the angle of view is changed by the optical zoom, when the lens barrel 101 is a rear focus type, it is necessary to move the focus lens 103 to an appropriate focus position according to the position of the zoom lens 102. . Such control is called computer zoom (CZ) control, and is performed by the CZ control unit 119 in the system control unit 114. FIG. 2 is a diagram illustrating the relationship between the focal length of the zoom lens and the focus position for each subject distance. In FIG. 2, the relationship between the focal length of the zoom lens and the in-focus focus position is shown as a graph as a data table showing each distance to the subject. In the present embodiment, this table is referred to as a focus cam table. In FIG. 2, the horizontal axis indicates the focal length corresponding to the zoom position, and the vertical axis indicates the focus position. The distance from the digital camera 100 to the subject (subject distance) is illustrated beside each graph line.

  The system control unit 114 controls the focus lens driving unit 112 when performing AF control, and performs a scanning operation by moving the focus lens 103 within a predetermined range. A focus position which is a focal point is detected by a known method using a contrast evaluation value or the like obtained during the scanning operation. The subject distance is measured by referring to the focus cam table using the zoom position and the focus position at that time.

  Next, control related to the auto zoom function in the system control unit 114 will be described. As shown in FIG. 1, the system control unit 114 includes a CZ control unit 119, an electronic zoom control unit 120, an auto zoom control unit (hereinafter referred to as AZ control unit) 122, and a subject detection unit 123.

  The digital camera 100 has an optical zoom function and an electronic zoom function, and the CZ control unit 119 and the zoom lens driving unit 113 are in charge of controlling the optical zoom. The CZ control unit 119 detects the zoom position of the zoom lens 102 at every predetermined control period during the zoom operation. The CZ control unit 119 performs control to drive the focus lens 103 so as to follow the focus cam table at the subject distance measured by the AF control according to the detected zoom position. This makes it possible to perform an optical zoom operation while maintaining the in-focus state.

On the other hand, the electronic zoom control unit 120 and the image memory 108 are in charge of controlling the electronic zoom. The electronic zoom control unit 120 implements an electronic zoom function by cutting out data in the target area from the image data transferred to the image memory 108. In addition, the electronic zoom control unit 120 realizes a smooth electronic zoom display by displaying the display unit 109 while gradually increasing the range to be extracted at the frame rate period of the image to be captured by the image sensor 106.
The subject detection unit 123 sequentially detects a desired subject region from the image data stored in the image memory 108. In the present embodiment, a subject detection method (face detection process, color detection process) for detecting a subject (a face or object such as a person) based on face information or color information included in image data will be described.

  The face detection process is a process for detecting a face area existing in image data by a known algorithm. For example, the subject detection unit 123 extracts a feature amount from a square partial region on the image data, and compares the feature amount with a facial feature amount prepared in advance. When the correlation value between the two exceeds a predetermined threshold, the subject detection unit 123 determines that the partial area is a face area. By repeating this determination process while changing the combination of the size of the partial area, the arrangement position, and the arrangement angle, various face areas existing in the image data can be detected.

  In the color detection process, a process of storing color information of a subject area designated according to a subject designation method described later as a characteristic color is executed. The color detection process is performed when the subject to be detected is an object (“thing” other than a person). As the color information, an RGB signal, a luminance signal (Y signal), a color difference (R−Y, B−Y) signal, etc., which are output signals from the image processing circuit 107 are used. When the subject is detected, the subject detection unit 123 divides the image data into a plurality of partial areas, and calculates an average value of luminance and color difference for each partial area. In addition, the subject detection unit 123 compares the feature color information stored in advance with the color information of each region at the time of subject detection, and sets a partial region where the difference in luminance and color difference is equal to or less than a predetermined amount as a subject region candidate. A process is performed in which a group of adjacent partial areas of the subject area candidates is set as the same color area, and an area where the same color area falls within a predetermined size range is set as a final subject area.

  The subject detection unit 123 calculates the size of the subject region on the image data by using the subject distance information measured by the CZ control unit 119 and the focal length information of the zoom lens 102 together with the face information and the color information. . The attitude detection unit 124 detects the attitude of the digital camera 100 (for example, normal position / above grip / under the grip) based on information from the acceleration sensor. The shake detection unit 125 detects the shake state of the digital camera 100 by determination based on angular velocity information or the like by a gyro sensor. The shake detection unit 125 determines that the camera is in a hand-held state when the shake amount (detection value) applied to the gyro sensor or the like is greater than or equal to a predetermined amount (threshold), and is fixed to a tripod or the like if it is less than the predetermined amount. It is determined that the state has been changed. The acceleration sensor and the gyro sensor used for the posture detection and the shake detection may be configured to also serve as the sensor of the detection unit for acquiring the control information of the anti-vibration lens driving unit 111.

Next, the outline of the auto zoom function in the present embodiment and the AZ control unit 122 will be described. In a camera not equipped with an auto zoom function, the following operations are required when the subject moves out of the frame while framing in the telephoto state and waiting for a photo opportunity, etc.
First, the photographer searches for a subject after performing a zoom-out operation by operating the zoom operation member. After searching for the subject, the photographer adjusts the angle of view by zooming until the desired angle of view is reached again. Even when the size of the subject image changes due to the movement of the subject, the photographer needs to adjust the size of the subject image by operating the zoom operation member.

  On the other hand, in a camera equipped with an auto-zoom function, after the photographer sets the auto-zoom function, an operation for designating a subject with a touch panel or the like may be performed to designate the subject to be photographed. When the auto-zoom function is set, the zoom operation is automatically performed so that the designated subject is within a predetermined size near the center of the image. In addition to the touch panel operation, the subject can be specified by specifying the subject near the center of the screen when the photographer operates a specific button, or by automatically selecting the main subject from the subjects detected by the camera. There are methods to select.

  The subject detection unit 123 calculates the position and size of the subject region designated from the image memory 108 on the image data. By performing this process continuously on the sampled image data each time an image is displayed as a live view, the movement of the subject can be tracked. When the image of the tracked subject is detected in a zoom-out area described later, or when the detected image of the subject becomes larger than a predetermined size, the AZ control unit 122 starts a zoom-out operation. That is, the AZ control unit 122 instructs the CZ control unit 119 or the electronic zoom control unit 120 to zoom out in the wide angle direction. When the subject is detected within the zoom-in area and the subject image falls within a predetermined size range, a zoom-in operation is performed toward the telephoto side. With such a process, the photographer need only move the camera so that the image of the desired subject is within the screen without worrying about the zoom operation. Even if the subject is about to be out of frame, the zoom magnification is automatically changed, so that the angle of view can be adjusted more easily.

  Next, the start conditions for the zoom-out operation and the zoom-in operation will be described with reference to FIGS. FIG. 3 is an explanatory diagram of processing for preventing the subject (object) from being out of the frame. FIG. 4 is an explanatory diagram of processing for preventing the subject (person) from going out of the frame.

  3 and 4, a frame 300a and a frame 300b are first tracking frames (hereinafter referred to as mono tracking frames) for tracking a subject (object), and frames 400a to 400f track a subject (person's face). This is a second tracking frame (hereinafter referred to as a face tracking frame). Hereinafter, when the subject is applicable to both a person and an object, the mono tracking frame and the face tracking frame may be collectively referred to as a subject tracking frame. The subject tracking frame is displayed so as to surround the subject image on the EVF screen (electronic viewfinder screen) of the display unit 109 so that the subject designated by the photographer can be seen. The position and size of the subject tracking frame on the screen are sequentially calculated by the subject detection unit 123 based on the face information and the color information, and are updated at a frame rate cycle.

  With reference to FIG. 3, a process for preventing the subject (airplane) from being out of frame outside the screen will be described. In FIG. 3A, a zoom-out area ZO (hereinafter referred to as a ZO area) indicates an area outside a predetermined ratio with respect to the entire angle of view (the entire screen) displayed in EVF. For example, assume a case where the center point of the screen is 0%, the entire screen is 100%, and a position that is 80% of the entire screen is set as the boundary of the ZO region. In this case, 80% to 100% of the entire screen is the ZO region. When a part of the mono tracking frame 300a enters the ZO area, the AZ control unit 122 performs control to start the zoom-out operation. Further, the AZ control unit 122 stores in the memory 118 the zoom magnification before zoom movement (corresponding to the zoom-in angle of view). The target zoom magnification and zoom speed during the zoom-out operation are set in advance according to the size and movement speed of the subject image. Further, the target zoom magnification and zoom speed may be appropriately calculated according to the size and moving speed of the subject image. The zoom-out operation is executed according to the target zoom magnification and zoom speed. Thereby, it is possible to effectively prevent the subject from being out of frame.

  FIG. 3B illustrates the angle of view when a zoom-out operation corresponding to a predetermined zoom change amount is performed from the angle of view of FIG. A zoom-in area ZI (hereinafter referred to as a ZI area) shown in FIG. 3B indicates an area inside a predetermined ratio with respect to the zoom-in angle of view (view angle before zoom-out) 301 in the subject search state. . For example, it is assumed that the center point of the screen is 0%, the zoom-in angle of view 301 is 100%, and a position that is 70% of the zoom-in angle of view 301 is set as the boundary of the ZI region. In this case, an area of 0 to 70% in the entire zoom-in angle of view 301 becomes the ZI area. At this time, for example, when the zoom-out magnification is ½, the zoom-in angle of view 301 is 50% of the entire screen. Therefore, the boundary of the ZI area is 70% × (½) = 35% with respect to the entire screen, and it can be said that the ZI area is an area of 0 to 35% with respect to the entire screen. When the photographer changes the orientation of the camera so that the mono tracking frame 300b fits inside the ZI area, the AZ control unit 122 performs control to start a zoom-in operation.

  Next, a process for preventing the subject (person) from going out of the frame will be described with reference to FIG. When the subject is a person, the zoom-out operation starts when a part of the face tracking frame enters the ZO region, and the zoom-in operation is performed when the face tracking frame falls within the ZI region. When the subject is a person, unlike the case of the object, the moving direction of the subject can be predicted to some extent, so the ZO region and the ZI region are set according to the region of the predicted moving direction. In hand-held shooting, the subject may be out of frame due to the influence of camera shake or the like. However, when the subject is out of frame due to camera shake or the like, it is possible for the photographer to re-frame in by an operation to try to frame the subject. Here, when the ZO area is set at the top of the screen, the face tracking frame may enter the ZO area even when a person is photographed near the center, and zooming out may occur unintentionally. Therefore, when the subject is a person and is in a hand-held state (a state where the photographer is holding the camera), the ZO area is not set at the top of the screen in consideration of the framing operation of the photographer.

  As described above, in the present embodiment, when the face is detected by the subject detection unit 123, the AZ control unit 122 changes the ZO region and the ZI region according to the posture and shaking state of the camera. The posture of the camera is detected by the posture detection unit 124, and the shaking state is determined from the detection result by the shake detection unit 125. The detection result by the shake detection unit 125 is a detection result of whether or not the camera is in a handheld state. Hereinafter, a specific description will be given with reference to FIGS.

  FIG. 4A shows the ZO region and the ZI region that are set when the photographer holds the camera in hand. In such a shooting scene, when the subject moves in the horizontal direction and moves out of the frame, the position of the subject in the screen moves in the horizontal direction (longitudinal direction) with respect to the normal screen. Therefore, both the ZO region and the ZI region are arranged in a vertical band shape in the vertical direction (short direction) with respect to the screen at the normal position. The ZO area is set in a vertical band shape positioned near both ends in the horizontal direction on a horizontally long rectangular screen, and the ZO area is set in a vertical band shape positioned in the center of the screen. In this case, when the face tracking frame 400a enters the ZO region, the AZ control unit 122 determines the start of zoom-out and performs zoom-out operation control corresponding to a predetermined zoom magnification. When the face tracking frame 400b is included in the ZI region, the AZ control unit 122 determines the start of zoom-in, and performs zoom-in operation control corresponding to a predetermined zoom magnification up to the zoom return position. By setting the ZO region and the ZI region in this way, it is possible to effectively prevent the subject (person) from being out of frame.

  FIG. 4B shows the ZO region and the ZI region that are set when the camera posture is changed in the same shooting scene and the photographer holds the camera in a vertical position under the grip or on the grip. . In this case, the ZO region and the ZI region are arranged in a vertical band shape in the vertical direction (longitudinal direction) with respect to the screen at the vertical position. That is, the ZO region is set to a vertical band shape that is positioned near both ends in the horizontal direction on a vertically long rectangular screen, and the ZI region is set to a vertical band shape that is positioned at the center of the screen. In this case, when the face tracking frame 400c enters the ZO region, the AZ control unit 122 determines the start of zoom-out and performs zoom-out operation control corresponding to a predetermined zoom magnification. When the face tracking frame 400d is included in the ZI region, the AZ control unit 122 determines the start of zoom-in, and performs zoom-in operation control corresponding to a predetermined zoom magnification up to the zoom return position. By setting the ZO area and the ZI area in this way, it is possible to detect the movement of the subject in the horizontal direction and effectively prevent frame-out.

  FIG. 4C shows a ZO region and a ZI region that are set when the detection state in the shake detection unit 125 is a fixed state. The fixed state is a state in which the camera is fixed by a tripod or the like, and there is no possibility of frame-out due to camera shake. Furthermore, if the subject zooms in when the subject is not framed in the vicinity of the center of the screen, there is a possibility that the frame is out due to the operation. Therefore, a ZO region is set for the entire peripheral portion of the screen, and a ZI region is set inside the zoom-in angle of view. That is, the ZO area is set to a rectangular band located near the edges in the vertical and horizontal directions of the screen, and the ZI area is set to a rectangular shape located at the center of the screen. In this case, when the face tracking frame 400e enters the ZO region, the AZ control unit 122 determines the start of zoom-out and performs zoom-out operation control corresponding to a predetermined zoom magnification. When the face tracking frame 400f is included in the ZI area, the AZ control unit 122 determines the zoom-in start and performs zoom-in operation control corresponding to a predetermined zoom magnification up to the zoom return position.

  As described above, in the present embodiment, the respective ranges of the ZO region and the ZI region are dynamically changed according to changes in the posture of the camera and the photographing state (hand-held state / fixed state). Accordingly, it is possible to effectively prevent the subject from being out of frame while preventing malfunction due to camera shake or the like. Note that the ZO area or the ZI area may be changed according to either the camera posture or the shooting state (handheld state / fixed state), or only one of the ZO area and the ZI area may be changed. Good.

  Next, a zoom operation for keeping the ratio of the subject image in the screen within a predetermined range will be described. In this embodiment, control is performed so as to keep the size of the subject image within the predetermined range from the reference size by automatically performing a zoom operation when the size of the detected subject image changes more than a predetermined multiple of the reference size. (Size holding control) is performed. FIG. 5 is a diagram for explaining processing for maintaining the size of the subject (person) image in the screen. FIG. 6 is an explanatory diagram of composition setting of a subject (person).

  First, with reference to FIG. 6, the setting of a range (composition) in which a subject to be tracked is included in the screen will be described. FIG. 6A illustrates a screen display when the composition setting is set to “manual”. When “manual” is set, the photographer changes the size of the face to be tracked by performing a manual zoom operation by operating the zoom lever while viewing a human image on the screen. The size of the subject image at that time is stored in the memory 118 as a reference size. FIG. 6B illustrates a screen display when the composition setting is set to “face”. When the composition of “face” is set, the size that fits the face on the screen according to the posture of the camera and the direction of the face is calculated as the reference size and stored in the memory 118. Similarly, FIG. 6C illustrates a screen display when the composition setting is set to “upper body”, and FIG. 6D illustrates a screen display when the composition setting is set to “whole body”. Illustrate. A reference size is calculated and stored in the memory 118 so as to have a size set on the screen.

  FIG. 6E illustrates a screen display when the composition setting is set to “auto”. When “Auto” is set, the AZ control unit 122 determines an appropriate composition based on the position of the subject, the size of the subject, the number of subjects, the posture of the camera, and the like. Further, a reference size is calculated and stored in the memory 118 so as to achieve the determined composition. A method for calculating the reference size will be described later with reference to FIG.

  The photographer can perform an operation of changing the composition setting by operating the left and right buttons of the operation unit 117 or selecting on the setting menu from the shooting screen. When the composition setting change operation is performed, the AZ control unit 122 updates the composition setting information stored in the memory 118. FIG. 6 shows an example in which “manual”, “face”, “upper body”, “whole body”, and “auto” composition are set when the subject is a person, but the composition setting is limited to this. It is not something. Only a part of the five illustrated compositions may be set, or other compositions may be included. When the subject is an object, for example, “manual”, “large”, “medium”, “small”, and “auto” compositions may be set.

  With reference to FIG. 5, a zoom operation for keeping the ratio of the subject image in the screen within a predetermined range will be described as an example in which the composition setting is “manual”. FIGS. 5A to 5C illustrate a zoom operation automatically performed by the camera when a subject (person) approaches the camera. This is a zoom-out operation for keeping the ratio of the subject image on the screen within a predetermined ratio. In FIG. 5, the face tracking frames 500a to 500f are displayed so as to surround the face area as the characteristic area of the person who is the subject. Therefore, here, the size of the face tracking frame will be described as the subject size.

  FIG. 5A shows the angle of view when a subject is designated according to a subject designation method described later. The size of the face tracking frame 500a at the time of subject designation is stored in the memory 118 as a reference subject size (reference size). FIG. 5B shows the angle of view when the subject approaches the camera without changing the zoom magnification from the state of FIG. For example, a size that is 150% of the size of the face tracking frame 500a that is the reference subject size is set as the start size of the zoom-out operation. When the relationship of the subject tracking frame (face tracking frame) is “face tracking frame 500b> face tracking frame 500a × 150%”, that is, the tracking frame has changed beyond a predetermined change amount with respect to the reference size. Sometimes, the AZ control unit 122 determines the start of the zoom-out operation.

  FIG. 5C shows an angle of view zoomed out from the angle of view 501 of FIG. 5B by a predetermined zoom magnification, and a face tracking frame 500c. Here, the predetermined zoom magnification is set to 1 / 1.5 in consideration of the rate of change (150%) of the face tracking frame size with respect to the reference subject size when starting the zoom-out operation. Thereafter, when the subject further approaches the camera, the subject image can be kept within a predetermined ratio by further zooming out to the wide angle side. Therefore, the photographer can concentrate only on the operation of the release switch.

  On the other hand, FIGS. 5D to 5F illustrate a zoom operation automatically performed by the camera when a person who is a subject moves away from the camera. This is a zoom-in operation for keeping the ratio of the subject image on the screen within a predetermined range. FIG. 5D shows the angle of view when a subject is designated according to a subject designation method described later. The size of the face tracking frame 500d at the time of subject designation is stored in the memory 118 as a reference subject size (when the composition setting is “manual”).

  FIG. 5E shows the angle of view when the subject moves away from the camera without changing the zoom magnification from the state of FIG. For example, a size that is 50% of the size of the face tracking frame 500d that is the reference subject size is set as the start size of the zoom-in operation. When the relationship of the face tracking frame becomes “face tracking frame 500e <face tracking frame 500d × 50%”, it is determined that the tracking frame has changed beyond a predetermined change amount with respect to the reference size. When this determination condition is satisfied and the face tracking frame 500e is included in the ZI region, the AZ control unit 122 determines the start of the zoom-in operation. Here, the ZI region is set inside the angle of view 502 zoomed in by a predetermined zoom magnification with respect to the angle of view of FIG.

  FIG. 5F shows an angle of view (corresponding to the angle of view 502) zoomed in by a predetermined zoom magnification from the angle of view of FIG. 5E, and a face tracking frame 500f. Here, the predetermined zoom magnification is set to 1 / 0.5 in consideration of the rate of change (50%) of the face tracking frame size with respect to the reference subject size when starting the zoom-in operation.

  FIG. 3 illustrates the processing for preventing frame-out when the subject is an object, and FIG. 4 illustrates the case where the subject is a person. Further, FIG. 5 illustrates the processing for keeping the ratio of the size of the subject image in the screen within a predetermined range when the subject is a person. Note that when the subject to be tracked is an object, the zoom operation start determination for controlling the size of the subject image shown in FIG. 5 may be performed as in the case where the subject is a person. Even when the composition setting is other than “manual”, the zoom operation is automatically performed in the same manner as in FIG. 5 according to the respective reference sizes.

Next, processing of the auto zoom function will be described with reference to FIGS. FIG. 7 is a flowchart illustrating an example of overall processing of the auto zoom function. The following auto-zoom functions are performed based on commands from the system control unit 114 (AZ control unit 122) unless otherwise specified.
First, in S <b> 700, the AZ control unit 122 determines the operation state of the auto zoom operation switch of the operation unit 117. When the auto zoom operation switch is pressed, the process proceeds to S701, and when the switch is not pressed, the determination process of S700 is repeated. In step S701, subject search processing is executed, and further subject specification processing in step S702, basic size setting processing in step S703, and auto zoom control in step S704 are sequentially executed.

  Immediately after the processing of the auto zoom function is started, or when the subject search button of the operation unit 117 is pressed (Yes in S705), the subject search processing in S701 is executed. With reference to FIG. 8, the subject search process executed in step S701 will be described. When the subject to be photographed is far away from the camera, the subject detection unit 123 may not be able to detect if the subject feature area is too small even if the subject image exists in the screen. In such a scene, the auto zoom function based on the subject information cannot be executed. Therefore, subject search processing is performed in which a zoom-in operation is performed while detecting the subject up to a predetermined focal length.

  In S800, it is determined whether or not the subject search button has been pressed immediately after the start of auto-zoom, or not just after the start of auto-zoom but during auto-zoom. In S800, in the case of the former positive determination result (yes), the process proceeds to S801, and in the case of a negative determination result (no), the process proceeds to S803. In step S801, processing for acquiring a subject search end focal length (hereinafter referred to as a search end focal length) is executed. In this embodiment, immediately after the start of auto-zoom, the search end focal length is 85 mm in terms of 35 mm. This focal length is a focal length of a standard lens that is generally suitable for photographing a person. Even when there is no subject (person) or a face cannot be detected (such as when facing backwards), after the search is completed. It is possible to enter a subject detection waiting state with an appropriate angle of view. However, the focal length is not limited to 85 mm, and may be other focal lengths (for example, 50 mm, 100 mm, etc.). Further, a configuration in which the photographer can change the search end focal length from the menu setting, or a configuration in which the search end focal length can be changed in accordance with the shooting mode or the subject to be shot. When the processing proceeds from S801 to S802, the focal length at the start of the mode is compared with the search end focal length, and it is determined whether the focal length is wider than the search end focal length or on the telephoto side. If the focal length at the start of auto-zoom is closer to the telephoto side than the search end focal length (no in S802), the subject search process is terminated. If the focal length at the start of auto-zoom is on the wide-angle side with respect to the search end focal length (yes in S802), the process proceeds to S805.

  On the other hand, when it is determined in S800 that the subject search is performed by operating the subject search button, in S803, a process of acquiring the search end focal length is executed. In the present embodiment, the search end focal length when the subject search button is operated is set to the focal length on the telephoto side by 2.5 times the focal length when the operation is executed. However, this magnification is an example, and other magnifications may be used. In addition, a configuration in which the photographer can change the search end focal length from the menu setting, or a configuration in which the search end focal length can be changed in accordance with the shooting mode or the subject to be shot. Proceeding from S803 to S804, the search end focal length acquired in S803 is compared with the telephoto end (telephoto end) focal length, and it is determined whether the search end focal length is wider or telephoto than the telephoto end focal length. . If the search end focal length is closer to the telephoto side than the telephoto end focal length, the subject search processing is terminated. If the search end focal length is on the wide-angle side with respect to the telephoto end focal length, the process advances to step S805. In S805, the AZ control unit 122 sets the zoom change amount corresponding to the search end focal length acquired in S801 or S803 in the CZ control unit 119 or the electronic zoom control unit 120, and instructs to perform a zoom-in operation.

  Next, in S806, a determination is made as to whether or not the current focal length has reached the search end focal length. If the current focal length reaches the search end focal length, the process advances to step S809 to stop the zoom-in operation, and then the subject search process ends. If it is determined in S806 that the current focal length has not reached the search end focal length, the process proceeds to S807. S807 is a process for determining whether or not a subject has been detected by the subject detection unit 123. If it is determined in S807 that the subject has been detected, the process proceeds to S808. If it is not determined that the subject has been detected, the process returns to S806. In S808, it is determined whether or not the position where the subject is detected is within a predetermined search end area on the screen. The predetermined search end region is determined by a threshold value (position determination reference value) set in advance according to the subject. As a result of the determination, if the subject detection position detected in S807 is within a predetermined search end area on the screen, the process proceeds to S809, the zoom-in operation is stopped, and the subject search process is ended. If it is determined in S808 that the detection position of the subject is not within the predetermined search end area on the screen, the process returns to S806, and the determination processing from S806 to S808 is continued.

  Here, the predetermined search end region described in S808 will be described with reference to FIG. When photographing a specific subject (person or object), the subject to be photographed is often arranged near the center of the screen. Further, when a subject not to be photographed is detected around the screen and the subject search process is terminated, the subject search process may be terminated before the subject to be photographed is detected. In this case, the auto zoom function as intended by the photographer cannot be executed. Therefore, in the present embodiment, the search end region is set as the determination region so that the subject search process is ended only when the subject is detected in the region near the center of the screen. FIG. 9A illustrates a search end area 900a that is set when the subject is an object (airplane). In the case of an object, a predetermined ratio, for example, a range of 70% from the center of the screen is set as the subject search end region in any of the upper, lower, left and right directions of the screen. On the other hand, FIG. 9B illustrates the search end area 900b set when the subject is a person. In the case of a person, the search ends so that the area at the top of the screen includes a range of 100% from the center of the screen and the other direction (left and right direction, downward) includes a predetermined ratio from the center of the screen, for example, a range of 70%. An area is set. When the subject is a person, there is a possibility that the face tracking frame 902 of the subject to be photographed is also arranged at the top of the screen. Therefore, even when a subject is detected at the top of the screen, the search end area is set so that the subject search process is completed.

  Next, with reference to FIG. 10, the subject designation process shown in step S702 in FIG. 7 will be described. FIG. 10A is a flowchart illustrating a processing example in which the user designates a desired subject using the touch panel configuring the operation unit 117. In this example, it is assumed that the user performs an operation of specifying by touching the subject image displayed on the display unit 109. In S1000, the AZ control unit 122 determines whether the touch panel has been pressed. When the touch panel is pressed, the process proceeds to S1001, and when the touch panel is not pressed, the determination process of S1000 is repeated.

  In S1001, the AZ control unit 122 acquires information on a position touched by the user (touch position). In next step S1002, the AZ control unit 122 notifies the subject detection unit 123 of the touch position, and the subject detection unit 123 performs face detection near the touch position. If a face is detected in the vicinity of the touch position in S1002, it is determined that the main subject is a person, and the process proceeds to S1003. If no face is detected near the touch position in S1002, it is determined that the main subject is an object other than a person, and the process proceeds to S1004.

  In S1003, the AZ control unit 122 performs control to store the face information of the person who is the target of automatic tracking in the memory 118. Specific face information includes information such as the face size, face detection position, face orientation, etc. when the subject is designated. In addition, in a camera having a face authentication function, identification information such as an authentication ID is also stored in the memory 118. In step S <b> 1004, the AZ control unit 122 performs control to store the characteristic color in the vicinity of the touch position in the memory 118 as the color information of the automatic tracking target. Specific color information includes information such as the characteristic color at the time of subject designation and its luminance, the value of the color difference, the size of the same color area, and the position of the center of gravity of the same color area. In the following description, face information and color information are collectively referred to as subject information (including subject size, subject detection position, etc.).

After the subject information is stored in the memory 118 in S1003 or S1004, the process proceeds to S1005 and a subject tracking frame is displayed. In step S1005, the AZ control unit 122 performs control for causing the display unit 109 to display a subject tracking frame (mono tracking frame or face tracking frame) having a size corresponding to the subject size around the subject detection position. Thereafter, the subject designation process ends.
As described above, in the system control unit 114, the subject detection unit 123 detects the subject at the position designated by the photographer on the display unit 109 or in the vicinity of the designated position. The AZ control unit 122 displays the subject tracking frame on the display unit 109. According to the processing example in FIG. 10A, the subject that the photographer wants to track can be easily specified by an intuitive method. However, the method for specifying the subject is not limited to this method. Another method will be described with reference to FIG.

  FIG. 10B is a flowchart showing a processing example when the user designates a subject using a switch (subject designation switch) different from the auto-zoom operation switch as an operation member constituting the operation unit 117. is there. First, in step S1006, the AZ control unit 122 displays a frame serving as a guide for specifying a subject near the center of the screen of the display unit 109. The photographer adjusts the orientation of the camera so that the image of the subject to be tracked fits in the vicinity of the center of the screen using this frame as a guide. In next step S1007, the AZ control unit 122 determines whether or not the subject specifying switch has been pressed. If the subject designation switch is pressed, the process proceeds to S1008. If the switch is not pressed, the determination process of S1007 is repeated in a waiting state.

  In step S1008, the subject detection unit 123 performs face detection near the center of the screen. If a face is detected near the center of the screen, the main subject is determined to be a person, and the process proceeds to S1009. On the other hand, if a face is not detected near the center of the screen, the main subject is determined to be an object other than a person, and the process proceeds to S1010. In S1009, the same processing as S1003 in FIG. 10A is performed, and in S1010, the same processing as S1004 in FIG. 10A is performed. After the subject information is stored in the memory 118 in S1009 or S1010, the process proceeds to S1011 where the subject tracking frame (mono tracking frame or face tracking frame) is displayed, and the subject designation process is terminated.

  10B, the subject detection unit 123 detects the subject at the screen center position of the display unit 109 or in the vicinity of the screen center position. Then, the AZ control unit 122 causes the display unit 109 to display a subject tracking frame indicating the position of the subject. Even in a camera not equipped with an operation member such as a touch panel, the user can easily specify a subject.

  FIG. 11 is a flowchart showing an example of processing for automatically selecting a subject to be tracked from faces detected when an auto zoom operation switch constituting the operation unit 117 is pressed. First, in step S1100, the subject detection unit 123 performs face detection on the entire screen and determines whether or not a human face has been detected. If even one person's face is detected on the entire screen, the main subject is determined to be a person, and the process proceeds to S1101. If no face is detected on the screen, the process proceeds to S1105.

  In step S1101, the subject detection unit 123 selects the face as the main face when there is only one person with the detected face. When there are a plurality of detected faces, the subject detection unit 123 selects a main face as a subject to be tracked from the faces. As a determination criterion for main face selection, for example, there is a method of preferentially selecting a face whose face detection position is located near the center of the screen. In addition, when there are a plurality of faces at the same position, there is a method of selecting a larger face as the main face. In the case of a camera having a face authentication function, if there is a face registered for authentication, there is a method in which the face is given priority as a main face.

  Proceeding from S1101 to S1102, the AZ control unit 122 stores the face information of the selected main face in the memory 118, and then proceeds to S1103 to display the face tracking frame on the screen. In next step S1104, face change determination is performed. That is, when a plurality of faces are detected, the main face automatically selected is not necessarily the face intended by the photographer. Therefore, the photographer can change the main face. At this time, when the photographer presses a switch (such as an auto zoom operation switch) on the operation unit 117 to designate a desired face, another face that is not selected as the main face from the faces in which the face tracking frame is detected. A process of changing the main face is performed. When the main face is changed in S1104, the process returns to S1102 again, and the face information to be stored in the memory 118 is updated. In S1103, the face tracking frame is changed to the size and detection position of the newly selected main face.

  On the other hand, when the process proceeds from S1100 to S1105, it is determined that the main subject is an object other than a person, and the AZ control unit 122 performs control to store the characteristic color near the center of the screen in the memory 118 as color information of an automatic tracking target. I do. In the next step S1106, after the mono tracking frame is displayed on the screen, the subject designation process ends. Basic processes in S1102, S1105, and S1103 (S1106) in FIG. 11 are the same as those in S1003, S1004, and S1005 in FIG.

  As described above, in the processing example of FIG. 11, the subject detection unit 123 performs face detection on the entire screen of the display unit 109 in the system control unit 114. When a plurality of faces are detected, the AZ control unit 122 performs control to display on the display unit 109 a subject tracking frame indicating the position of the first face as a subject from among the plurality of faces. In addition, when the first face is changed to the second face as the subject, the AZ control unit 122 performs control to display a subject tracking frame indicating the position of the second face on the display unit 109. The photographer can easily specify a desired subject with fewer operations.

  For the subject designation processing, in the case of a camera having both the touch panel constituting the operation unit 117 and the subject designation switch, both the method of FIG. 10A and the method of FIG. 10B are applied simultaneously. May be. Also, after the subject change determination in S706 shown in FIG. 7 is made, the subject is designated by the method of FIG. 10A or FIG. The subject can also be specified by the method. The subject designation process to be applied may be changed according to the process flow. When the subject designation process (S702 in FIG. 7) is completed, the process proceeds to S703 in FIG. 7, and a process for setting the size of the subject to be tracked as a reference size is executed.

  With reference to FIG. 12 to FIG. 19, the reference size setting process in S703 shown in FIG. 7 will be described. FIG. 12 is a flowchart illustrating an example of a reference size setting process when the subject is a person. First, in S1200, the AZ control unit 122 determines whether or not the composition setting stored in the memory 118 is the above-described “manual”. If the composition setting is “manual”, the process advances to step S1201. If the composition setting is other than “manual”, the process advances to step S1203.

  In step S1201, the AZ control unit 122 determines subject detection. The AZ control unit 122 determines whether the subject detection unit 123 detects a face when the subject is a person or whether the same characteristic color is detected when the subject is an object. The determination process in S1201 is repeated until the subject is detected, and if a subject is detected, the process proceeds to S1202. In S1202, the AZ control unit 122 sets the size of the detected subject image as a reference size, stores it in the memory 118, and ends the process. Accordingly, when the composition setting is “manual”, the size (detected size) of the subject image detected first when the reference size setting process is started is stored in the memory 118 as the reference size.

  On the other hand, in S <b> 1203, the AZ control unit 122 calculates the screen size based on the camera posture information determined by the posture detection unit 124. This screen size is used to calculate the reference size in S1211, S1704, S1705, S1806, and S1807, which will be described later. The screen size calculation process will be described with reference to FIG. In this embodiment, the case where the screen size of the still image is long side × short side = 640 × 480 and the screen size of the moving image is long side × short side = 640 × 360 is exemplified, but the screen size is limited to this. Is not to be done.

  In S1300, the posture detection unit 124 determines whether the photographer holds the camera at the normal position or the vertical position. If it is determined that the photographer holds the camera in the normal position, the process proceeds to S1301. If the photographer holds the camera in the vertical position, the process proceeds to S1306. In S1301, processing for setting the long side size 640 to the horizontal screen size is executed. In next step S1302, it is determined whether or not the current setting mode is a mode (still image / moving image simultaneous shooting mode) in which a still image and a moving image can be simultaneously captured. In the still image / video simultaneous shooting mode, the photographer can simultaneously perform still image shooting by pressing the release switch of the operation unit 117 while shooting a movie, or a framing operation immediately before shooting a still image as a movie. It is a mode that can be recorded automatically. If it is determined in S1302 that the still image / moving image simultaneous shooting mode is set, the processing compares the short side size of the moving image with the short side size of the still image and sets a smaller size as the vertical screen size. This is to prevent the subject from being framed out of the screen regardless of the angle of view of either a still image or a moving image by performing auto zoom according to the small screen size. Therefore, in this embodiment, in the still image / moving image simultaneous shooting mode, the process proceeds to S1304, and 360, which is the short side size of the moving image, is set as the vertical screen size. Thereafter, the screen size calculation process ends.

  If it is determined in S1302 that the mode is not the still image / moving image simultaneous shooting mode, the process proceeds to S1303. In step S1303, it is determined whether moving image recording is in progress or still image framing is in progress. If it is determined that a moving image is being recorded, the process proceeds to S1304. If it is determined that a moving image is not being recorded, the process proceeds to S1305. In S1305, the short side size 480 of the still image is set as the vertical screen size.

  If it is determined in S1300 that the photographer is in the vertical position, the processing from S1306 to S1310 is executed. The processes of S1306 to S1310 are processes in which the settings of the horizontal screen size and the vertical screen size are reversed with respect to the processes of S1301 to S1305. That is, in the above description, “horizontal” and “vertical” may be interchanged so that the detailed description is omitted. In this embodiment, the still image screen size is 4: 3 (see FIGS. 15A and 15B), and the moving image screen size is 16: 9 (see FIG. 15C). The screen size may be changed according to the aspect ratio setting. Thereby, it is possible to make the ratio of the person image to the screen constant regardless of the aspect ratio. Further, as an example of the still image / moving image simultaneous shooting mode, an example in which the screen size is different between the still image and the moving image has been shown.

  When the screen size is determined in S1203 of FIG. 12, the process proceeds to S1204. When the composition setting stored in the memory 118 is any one of “face”, “upper body”, and “whole body”, in steps S1204 to S1209, the AZ control unit 122 determines the face image occupied on the screen according to the composition setting. Determine the size ratio (called face ratio). The face ratio of the present embodiment indicates the ratio of the length of the face tracking frame in the vertical direction relative to the screen, but may be indicated by the ratio of the length in the horizontal direction, the area ratio, or the like. Specifically, for example, it is determined whether or not the composition setting is “face” in S1204. If the composition setting is “face” (yes in S1204), the face ratio is determined to be 27% (S1207). If the composition setting is not “face”, the process advances to step S1205 to determine whether the composition setting is “upper body”. If the composition setting is “upper body” (yes in step S1205), the face ratio is 16%. It is determined (S1208). If the composition setting is not “upper body”, the process proceeds to S1206 to determine whether or not the composition setting is “whole body”. If the composition setting is “whole body” (yes in S1206), the face ratio is determined to be 7%. (S1209). After the face ratio is determined in steps S1204 to S1209, the process proceeds to S1210.

  In S1210, the AZ control unit 122 calculates the reference size by multiplying the face ratio determined in S1204 to S1209 by the vertical screen size calculated in S1203. This is a method of calculating a reference size from a face ratio with respect to a screen size in the vertical direction on the premise of a subject (person) in a standing posture. When the subject detection unit 123 can determine the face direction, the screen size direction may be determined based on the vertical direction of the face on the screen. Further, after the process of storing the calculated reference size in the memory 118 is executed, the reference size setting process is terminated. Thus, when the composition setting is any one of “face”, “upper body”, and “whole body”, a size that is a predetermined face ratio corresponding to the composition setting with respect to the screen size is set as the reference size.

  If the composition setting is not “whole body” in S1206, that is, if it is determined that the composition setting is “auto”, the process proceeds to S1211, and the AZ control unit 122 moves the shooting state, the number of subjects, and the movement of the subject. An appropriate composition is automatically determined according to the above. With reference to FIG. 14, the automatic composition determination and composition selection processing in the composition setting “auto” in S1211 will be described. The composition setting “auto” is a mode in which the composition can be further easily adjusted by discriminating a shooting scene and automatically selecting an appropriate composition (automatic composition selection).

  FIG. 14 shows a transition diagram of the composition automatically selected in the present embodiment. In the present embodiment, composition selection processing is performed in which an appropriate composition is selected from the five compositions of the upper body 1400, the whole body 1401, the face 1402, the plurality of people 1403, and the manual 1404 according to the shooting scene. In the composition adjustment operation, when shooting a still image, it is necessary to instantaneously set the subject image to an appropriate size before shooting. On the other hand, at the time of shooting a moving image, it is necessary to always set the subject image to an appropriate size while shooting, such as following a moving subject or approaching a stationary subject. In addition, since the image data at the time of zoom-in operation or zoom-out operation by changing the composition is recorded at the time of moving image shooting, an effective video can be taken by changing the composition at an appropriate timing. Thus, the appropriate composition differs between the still image and the moving image. Therefore, processing for changing the composition to be selected depending on the shooting state is executed during still image framing 1405 and during moving image recording 1406. That is, in the present embodiment, a still image shooting preparation state is illustrated as the first operation state, and a moving image recording state is illustrated as the second operation state. As a composition of a selection candidate belonging to the first composition group in the still image framing 1405, any composition is selected from the upper body 1400, the whole body 1401, and a plurality of people 1403. As the composition of the selection candidate belonging to the second composition group in the moving image recording 1406, any one of the upper body 1400, the whole body 1401, and the face 1402 is selected. That is, the composition of the upper body 1400 and the whole body 1401 is common to the first composition group and the second composition group, the composition of the plurality of persons 1403 is included only in the first composition group, and the composition of the face 1402 is the second composition group. It is included only in the composition group. Note that three or more operation states and a plurality of composition groups (including compositions that can be selected in each operation state) can be set in accordance with the specifications of the imaging apparatus.

  Here, the determination conditions of the shooting scene and the transition of each composition will be described. When the automatic composition determination process S1211 of the composition setting “auto” starts, the upper body 1400 is set as the composition in the initial state. The composition of the upper body 1400 is a composition that is selected when it is determined during the still image framing 1405 or the moving image recording 1406 that the scene is a single subject that is still. In such a scene, by composing the upper body of the subject within the screen, it is possible to achieve a composition that is reasonably close to the subject.

  When the movement of the subject to be photographed is detected in the composition of the upper body 1400, or when it is determined that the distance from the camera to the subject is greater than the predetermined distance D1 (first threshold), the composition is the whole body 1401. Changed to When the composition of the whole body 1401 is determined to be a scene for shooting a moving subject or a distant subject, that is, a subject that is likely to be framed out of the screen during the still image framing 1405 or the moving image recording 1406. The composition to be selected. In such a scene, the subject is set so as not to be out of the frame as much as possible by following the subject with a composition that fits the whole body of the subject. Further, when the composition of the whole body 1401 detects that the subject to be photographed is stationary for a predetermined time or more and it is determined that the distance from the camera to the subject is smaller than the predetermined distance D2 (second threshold). The process of returning the composition to the upper body 1400 is executed. The predetermined time is a predetermined reference time for determination.

  Hereinafter, a method for determining the movement and stillness of a subject will be described. When at least one of the amount of change in the position and size of the subject detected by the subject detection unit 123 or the amount of shake detected by the shake detection unit 125 while detecting the subject is larger than a predetermined amount, It is determined that the subject to be photographed is moving. The change in the position and size of the subject detected by the subject detection unit 123 and the amount of shake detected by the shake detection unit 125 while the subject is detected continue for a predetermined time or longer. In this case, it is determined that the subject to be photographed is stationary. The duration of the detection state is measured by a timer for timekeeping.

  Further, a method for calculating the distance from the camera to the subject will be described. First, a reference focal length and a subject distance with respect to a subject size are measured in advance, and measurement data is stored in the memory 118. The subject distance is calculated by performing an operation of “subject distance = (reference subject distance × focal length × reference subject size) / (reference focal length × detection size)” with respect to the reference value stored in the memory 118. . For example, it is assumed that a subject image having a detection size of 40 pix is detected at a focal length of 120 mm when the reference value is a reference subject distance of 2 m, a reference focal length of 24 mm, and a reference subject size of 20 pix (pixels). In this case, the subject distance is calculated as (2 m × 120 mm × 20 pix) / (24 mm × 40 pix) = 5 m.

  Next, a composition when a plurality of subjects are detected in the screen will be described. When a plurality of subjects are detected in the screen in the composition of the upper body 1400 or the whole body 1401 during the still image framing 1405, processing for changing the composition to the plurality of people 1403 is performed. At this time, the number of detected subjects is stored in the memory 118. The composition of the plurality of persons 1403 is a composition that is selected when it is determined that a group photo shooting scene in which all of a plurality of subjects fall within a predetermined angle of view during 1405 during still image framing. In the composition state of the plurality of people 1403, a determination process is performed as to whether or not the number of detected subjects has changed. When the number of detected subjects increases, a new subject has entered the screen, so the number of subjects stored in the memory 118 is immediately updated, and the composition is changed. On the other hand, when the number of detected subjects decreases, for example, there is a possibility that the state may be temporarily undetectable, such as when any subject is turned sideways. For this reason, if the composition is changed immediately, a subject that can no longer be detected may be out of frame. Therefore, when the number of subjects decreases, it is determined whether or not the state has continued for a predetermined time or more. When the state continues for a predetermined time or more, the number of subjects stored in the memory 118 is updated and the composition is changed. Furthermore, when the subject is alone, a process of returning the composition to the upper body 1400 is executed.

  If the composition is such that all the subjects fall within the angle of view during moving image recording 1406, there is a concern that when a subject other than the subject is moving around, a zoom malfunction may occur in response to that subject. Therefore, even when a plurality of subjects are detected in the moving image recording 1406, processing for maintaining the upper body 1400 or the whole body 1401 with the main subject as the subject to be photographed is performed. In the composition of the upper body 1400 during moving image recording 1406, when a face is detected for a predetermined time or more near the center of the screen, a process of changing the composition to the face 1402 is performed. The composition of the face 1402 is a composition that is selected when it is determined during the moving image recording 1406 that the photographer is paying attention to the face of the subject. In such a scene, the composition of the face closer to the face than the upper body allows the face of the subject of interest to be photographed in a larger size. Further, during the zoom-in operation by this composition change, a more effective video can be taken by changing the zoom speed to a significantly lower speed or higher speed than the normal speed. Furthermore, when the face 1402 is detected in the vicinity of the screen for a predetermined time or more in the composition of the face 1402, or when it is determined that the photographer has changed the framing, the composition is returned to the upper body 1400. Processing is performed. In the present embodiment, the peripheral area of the screen is exemplified as the specific area of the image, and the timer for measuring the duration of the state where the position of the face image of the subject detected by the subject detection unit 123 is the peripheral area. Further, as a method for determining the framing change, when the subject detection unit 123 has not detected the subject and the shake amount detected by the shake detection unit 125 is larger than a predetermined amount, the photographer performs framing. It is determined that it has changed.

  During still image framing 1405, the photographer may continue to wait for a photo opportunity with the same composition until the face orientation and facial expression of the person are in a desired state. In this case, if the composition is close to the face, the composition may be different from the photographer's intention. Therefore, even if a face is detected in the vicinity of the center of the screen for a predetermined time or more in the composition of the upper body 1400 during the still image framing 1405, processing for maintaining the composition of the upper body 1400 is executed.

  Next, a composition when the shooting state is changed by starting / ending moving image recording will be described. Each composition of the upper body 1400 and the whole body 1401 is a composition that can be selected in either case of the still image framing 1405 or the moving image recording 1406. Therefore, in those compositions, the original composition is maintained even if the shooting state is changed. On the other hand, the composition of the plural people 1403 and the face 1402 is a composition that is selected only in one of the cases of the still image framing 1405 and the moving image recording 1406. Therefore, when the moving image recording is started with the composition of a plurality of people 1403 during the still image framing 1405, or when the moving image recording is ended with the composition of the face 1402 during the moving image recording 1406, the upper body 1400, which is a common composition, is displayed. It is possible to change. However, in this case, there is a concern that the zoom operation starts at the start and end of moving image recording, so that an image due to the zoom operation is recorded at the beginning of the moving image and that the photographer feels troublesome due to the framing of the still image. . Therefore, when the shooting state is changed in such a state, processing for temporarily changing the composition to the manual 1404 is executed. The composition of the manual 1404 is a composition in which the size of the main subject image detected first after the composition change is used as a reference size. Therefore, the zoom operation does not start immediately after changing the shooting state, and the photographer is not uncomfortable. Even when the subject starts to move, the size of the subject image can be maintained. Further, when a predetermined time elapses in the composition of the manual 1404, processing for returning the composition to the upper body 1400 is executed.

  As described above, in S <b> 1211 of FIG. 12, when the composition setting is “auto”, an appropriate composition is automatically determined according to the shooting state, the number of subjects, the movement of subjects, and the like. Data indicating the composition after the determination is stored in the memory 118, and the process proceeds to S1212.

  In S1212, whether the composition determined in S1211 has been changed with respect to the previous composition (whether the composition selected in FIG. 14 has transitioned or the number of subjects has changed in the state of multiple people 1403). Is determined. If the composition has been changed, the process advances to step S1213 to determine whether the selected composition is the composition of the manual 1404. If the selected composition is the composition of the manual 1404, the process proceeds to S1201, and the process of storing the size of the subject image detected first in the memory 118 as the reference size is executed. If the selected composition is not the composition of the manual 1404, the process proceeds to S1215, and the reference size in the composition setting “auto” is calculated. If it is determined in S1212 that the composition has not been changed, the process proceeds to S1214 to determine whether or not the subject image has moved within the screen. As a method of calculating the amount of movement of the subject image within the screen, the center position of the face tracking frame immediately after the zoom operation is performed to the angle of view that becomes the reference size after the reference size is set in S1215 is stored in the memory 118. Remembered. A distance from the position stored in the memory 118 to the center position of the detected face tracking frame is calculated as a movement amount. Alternatively, the amount of movement in the screen may be calculated by adding the center position of the face tracking frame at every predetermined period. When the calculated movement amount is equal to or greater than a predetermined amount, it is determined that the subject has moved. The predetermined amount is normalized by the size of the detected face tracking frame (set to a predetermined multiple of the size of the face tracking frame), so that the actual amount of movement of the subject regardless of the size of the subject image on the screen Can be comparable. If it is determined in S1214 that the subject is moving within the screen, the process proceeds to S1215, and the reference size in the composition setting “auto” is calculated. If it is determined in S1214 that the subject image has not moved within the screen, the reference size setting process is terminated without updating the reference size.

  Hereinafter, with reference to FIGS. 15 to 19, the reference size calculation process in the composition setting “auto” will be described. When the composition setting is any one of “face”, “upper body”, and “whole body”, the reference size is calculated so that the face has a constant ratio to the screen size in S1210 of FIG. That is, the same reference size is calculated regardless of the position of the subject image on the screen. Therefore, if the composition is changed so that the subject image becomes larger (zoom in) when the subject is in the periphery of the screen, the subject may be out of frame. When the reference size is calculated by this method, the photographer needs to frame the subject near the center of the screen. Therefore, it is not suitable for a case where the subject is shifted from the center of the screen and taken together with the background. Further, when there are a plurality of subjects, it is necessary to calculate the reference size by another method. Therefore, when the composition setting is “auto”, the size of the subject image is determined based on the composition determined in step S1211, and the reference is set so that the subject is within the screen regardless of the position of the detected subject. A process of calculating the size is performed.

FIG. 15 is a diagram showing the relationship between the position of the subject closest to the screen among a plurality of subjects and the screen size. 15A and 15B show examples of still image shooting screens, and FIG. 15C shows an example of moving image shooting screens, illustrating a plurality of subjects. FIG. 16 is a flowchart showing the entire reference size calculation process in composition setting “auto”.
First, in S1600 of FIG. 16, processing for calculating the subject position ratio Rh in the horizontal direction is executed. The subject position ratio Rh in the horizontal direction is the horizontal position near the shoulder of the subject closest to the screen center (hereinafter referred to as the shoulder position, referred to as Xsmax) with respect to a predetermined ratio to the screen size in the horizontal direction. It is the ratio which distance x2 occupies. The predetermined ratio is, for example, 80% or 90% of the screen size in the horizontal direction, and is changed according to the number of people arranged in the horizontal direction in S1703 to be described later. In FIG. 15A, among the subjects 1500a, 1501a, and 1502a, the subject that is closest to the screen center in the horizontal direction is the subject 1500a. With reference to the flowchart of FIG. 17, the calculation processing of the subject position ratio Rh in the horizontal direction will be described.

  In S1700, the shoulder position (denoted as Xs) of the subject is calculated based on the center position and size of the subject's face tracking frame. The center of the screen is the origin (0, 0), the center position of the face tracking frame is (Xc, Yc), and the size of the face tracking frame is S. The shoulder position Xs on the peripheral side of the screen can be calculated by “Xs = | Xc | + S × Ns”, where Ns is the number of face tracking frames occupying the distance from the center position of the face tracking frame to the shoulder position. FIG. 19 shows a setting example of Ns. The value of Ns is changed according to the composition determined in S1211 of FIG. For example, in the case of the composition of the face 1402, Ns = 1 (in this case, the horizontal position near the ear, not the shoulder position) is set. In other cases (“upper body”, “whole body”, “multiple people”), Ns = 2 is set.

  After calculating the shoulder position Xs of the subject, the process proceeds to S1701. In S1701, when the maximum value Xsmax of the shoulder position is not stored in the memory 118, the shoulder position Xs calculated in S1700 is stored as the maximum shoulder position Xsmax. When the maximum shoulder position Xsmax is stored in the memory 118, the shoulder position Xs calculated in S1700 is compared with the maximum shoulder position Xsmax. When the shoulder position Xs is larger than the maximum shoulder position Xsmax, a process of updating the maximum shoulder position Xsmax in the memory 118 with the shoulder position Xs is executed. After updating the maximum shoulder position Xsmax, the process proceeds to S1702. In S1702, it is determined whether or not the process of calculating the shoulder position Xs and updating the maximum shoulder position Xsmax is completed for all subjects. If the calculation of the shoulder position Xs and the update of the maximum shoulder position Xsmax are not completed, the process returns to S1700. When the determination of the maximum shoulder position Xsmax is completed for all the subjects and the update of the maximum shoulder position Xsmax is completed, the process proceeds to S1703.

  In S1703, determination processing is performed for the number of people whose subjects are arranged in the horizontal direction (the number of people arranged in the horizontal direction). The purpose of this is to create a composition that fills the screen when there are many subjects, such as a group photo, and to leave a margin around the screen when there are few subjects. As a method of counting the number of people arranged in the horizontal direction, when the face tracking frames overlap in the vertical direction (up and down direction of the screen), the overlapping subjects are counted as one person. For example, when four subjects are detected in the screen and the face tracking frames of two of them are overlapped on the top and bottom of the screen, they are counted as three. In S1703, the number of people arranged in the horizontal direction is compared with a threshold value (for example, two people). If it is determined that the number of people arranged in the horizontal direction is two or less, the process proceeds to S1704. If it is determined that there are three or more persons, the process proceeds to S1705. In S1704 and S1705, the subject position ratio Rh in the horizontal direction is calculated. The horizontal direction subject position ratio Rh in S1704 is calculated as the ratio of the maximum shoulder position Xsmax × 2 to 80% of the horizontal screen size. The horizontal subject position ratio Rh in S1705 is calculated as the ratio of the maximum shoulder position Xsmax × 2 to 90% of the horizontal screen size. After S1704 or S1705, the subject position ratio Rh calculation process is terminated.

  Next, in S1601 of FIG. 16, processing for calculating the subject position ratio Rv in the vertical direction is executed. The subject position ratio Rv in the vertical direction is the vertical position of the head of the most peripheral subject from the center of the screen (hereinafter referred to as the head position) or the vertical position of the body part with respect to a predetermined ratio of the screen size in the vertical direction. This is the ratio of the distance x2 (hereinafter referred to as body position). The predetermined ratio is 90% of the screen size in the vertical direction, for example. Hereinafter, the head position of the most peripheral subject from the center of the screen is denoted as Yhmax, and the body position of the most peripheral subject from the center of the screen is denoted as Ybmin. In FIG. 15B, of the subjects 1500b, 1501b, and 1502b, the subject with the head at the most periphery from the center of the screen in the vertical direction is the subject 1500b, and the subject at the most peripheral body position from the center of the screen is the subject 1501b. It is. With reference to the flowchart of FIG. 18, the calculation process of the subject position ratio Rv in the vertical direction will be described.

  In S1800 of FIG. 18, the head position of the subject (denoted Yh) is calculated based on the center position and size of the face tracking frame of the subject. The head position Yh can be calculated by “Yh = Yc + S × Nh”. Nh is the number of face tracking frames in the distance from the center position of the face tracking frame to the head position. FIG. 19 shows an example of setting Nh. The value of Nh is set to Nh = 1.5 regardless of the composition determined in S1211 of FIG. After calculating the head position Yh of the subject, the process proceeds to S1801.

  In S1801, when the maximum value Yhmax of the head position is not stored in the memory 118, a process of storing the head position Yh calculated in S1800 as the maximum head position Yhmax is executed. When the maximum head position Yhmax is stored in the memory 118, the head position Yh calculated in S1800 is compared with the maximum head position Yhmax. When the head position Yh is larger than the maximum head position Yhmax, processing for updating the maximum head position Yhmax in the memory 118 with the head position Yh is performed. After the update process, the process proceeds to S1802.

  In next step S1802, the body position (denoted as Yb) of the subject is calculated based on the center position and size of the face tracking frame of the subject. The body position Yb can be calculated by “Yb = Yc−S × Nb”. Nb is the number of face tracking frames in the distance from the center position of the face tracking frame to the body position. FIG. 19 shows a setting example of Nb. The value of Nb is changed according to the composition determined in S1211 of FIG. For example, in the case of the composition of the face 1402, Nb = 1.5 is set. Nb = 5 is set in the case of the composition of the upper body 1400, Nb = 10 is set in the case of the composition of the whole body 1401, and Nb = 3.5 is set in the case of the composition of a plurality of people 1403. This setting is the center of the screen in each composition, the face 1402 is the chest, the upper body 1400 is below the waist, the whole body 1401 is the foot, and the plurality of people 1403 is near the waist. After calculating the body position Yb of the subject, the process proceeds to S1803.

  In S1803, when the minimum value Ybmin of the body position is not stored in the memory 118, a process of storing the body position Yb calculated in S1802 as the minimum body position Ybmin is executed. If the minimum body position Ybmin is stored in the memory 118, the body position Yb calculated in S1802 is compared with the minimum body position Ybmin. When the body position Yb is smaller than the minimum body position Ybmin, a process of updating the minimum body position Ybmin in the memory 118 with the body position Yb is performed. It progresses to S1804 after an update process.

  In S1804, the head position Yh and the body position Yb are calculated for all the subjects, and it is determined whether or not the process of updating the maximum head position Yhmax and the minimum body position Ybmin is completed. If the update of the maximum head position Yhmax and the minimum body position Ybmin has not been completed, the process returns to S1800. When the determination of the maximum head position Yhmax and the minimum body position Ybmin is completed for all subjects and the update of the maximum head position Yhmax and the minimum body position Ybmin is completed, the process proceeds to S1805. In S1805, the absolute value of the maximum head position Yhmax and the absolute value of the minimum body position Ybmin are compared, and a process of determining which is located closer to the screen periphery is performed. If it is determined that the maximum head position Yhmax is located closer to the screen periphery, the process proceeds to S1806. If it is determined that the minimum body position Ybmin is located closer to the periphery of the screen, the process proceeds to S1807. In S1806 and S1807, the subject position ratio Rv in the vertical direction is calculated. The vertical subject position ratio Rv in S1806 is calculated as a ratio of the absolute value of the maximum head position Yhmax × 2 with respect to 90% of the vertical screen size. The vertical subject position ratio Rv in S1807 is calculated as a ratio of the absolute value of the minimum body position Ybmin × 2 with respect to 90% of the vertical screen size. After S1806 or S1807, the vertical subject position ratio Rv calculation process is terminated.

  Subsequently, in S1602 of FIG. 16, the subject position ratio Rh in the horizontal direction is compared with the subject position ratio Rv in the vertical direction. Thereby, it is possible to determine a position having the largest ratio among the distance ratios from the center of the screen to each position of the subject with respect to the predetermined ratio of the screen size in each direction, that is, the position closest to the screen periphery. it can. Each position is a maximum shoulder position Xsmax, a maximum head position Yhmax, and a minimum body position Ybmin. If it is determined that the horizontal subject position ratio Rh is greater than the vertical subject position ratio Rv, the process proceeds to S1603, and it is determined that the horizontal subject position ratio Rh is less than or equal to the vertical subject position ratio Rv. In step S1604, the process proceeds to step S1604. In S1603 and S1604, a process for calculating a reference size is executed based on the detected size of the main subject and the ratio of the subject position closest to the screen determined in S1602. The reference size is calculated by “detected size / horizontal subject position ratio Rh” in S1603, and calculated by “detected size / vertical direction subject position ratio Rv” in S1604. That is, in S1603, the zoom operation is executed so that the main subject image has a reference size obtained by multiplying the size of the main subject image by “1 / horizontal subject position ratio Rh”. In S1604, the zoom operation is executed so that the main subject image has a reference size obtained by multiplying the size of the main subject image by “1 / vertical subject position ratio Rv”. In this way, the angle of view based on the composition determined in S1211 can be set without causing the subject in the screen to be out of frame.

  In S1215 of FIG. 12, when the composition setting is “auto”, the subject position that is the most peripheral in the horizontal and vertical directions is determined based on the composition determined in S1211, and the reference is set so that the position fits on the screen. The size is calculated. As a result, the subject can be held at an appropriate angle of view without being out of the frame no matter where the subject image is in the screen. In addition, whether the number of subjects is singular or plural, the reference size can be calculated by the same process. In the present embodiment, the direction of the screen size for calculating the reference size is determined on the assumption that the subject is in a standing posture. Not limited to this, when the subject detection unit 123 can determine the face direction, the screen size direction may be determined based on the vertical direction of the face on the screen. Furthermore, the composition determination and the reference size calculation example when the subject is a person have been described, but the present invention can also be applied when the subject is an object. However, in this case, the selected composition is “large”, “medium”, “small”, and “plurality” instead of “face”, “upper body”, “whole body”, and “multiple people”. Accordingly, the number of mono tracking frames (corresponding to Ns, Nh, and Nb in the case of a person) when calculating the subject position may be set.

  When the reference size setting process illustrated in FIG. 12 is completed, the process proceeds to S704 in FIG. In step S <b> 704, the AZ control unit 122 performs auto zoom control based on the subject information detected by the subject detection unit 123 and the reference size stored in the memory 118. The auto zoom control will be described later with reference to FIG. After the auto zoom control ends, the process proceeds to S705, where the AZ control unit 122 determines a subject search instruction from the photographer. That is, it is determined whether or not the photographer has pressed the subject search button on the operation unit 117. If the subject search button is pressed, the process returns to S701 and subject search processing is executed. If the subject search button has not been pressed, the process proceeds to S706. In S <b> 706, it is determined whether or not the subject to be subjected to auto zoom has been changed by the operation of the touch panel of the operation unit 117, the subject designation switch, or the like. If the subject has been changed, the process returns to S702 and subject designation processing is executed. If the subject has not been changed, the process proceeds to S707.

  In S707, it is determined whether or not the composition setting has been changed by operating the left and right buttons of the operation unit 117 from the shooting screen. When the composition setting is changed, the composition setting data stored in the memory 118 is updated, and then the process returns to S703 to execute the reference size setting process. If the composition setting has not been changed, the process proceeds to S708. In S708, it is determined whether or not the auto zoom operation switch of the operation unit 117 has been pressed. If it is determined that the auto zoom operation switch has been pressed, the auto zoom function is terminated. If it is determined that the auto zoom operation switch has not been pressed, the process proceeds to S709. S709 is a process for determining whether or not the composition setting stored in the memory 118 is “auto”. If the composition setting is “auto”, the process returns to S703, and a reference size setting process including automatic composition determination is executed. If the composition setting is other than “auto”, the process returns to S704 and the auto zoom control is continued.

  Next, the auto zoom control process of S704 will be described with reference to the flowchart of FIG. First, in S2000, the AZ control unit 122 determines whether or not a subject is detected by the subject detection unit 123. If no subject is detected in S2000, the auto zoom control is terminated. If the subject is detected in S2000, the process proceeds to S2001. The determination shown in each step of S2001 to S2003 is an automatic zoom start determination process for frame-out prevention control. That is, this process is performed to prevent the subject from being out of the screen as described with reference to FIGS. In step S2001, the AZ control unit 122 determines whether the subject tracking frame related to the subject to be tracked has entered the ZO region. This ZO region corresponds to the ZO region described with reference to FIGS. 3A and 4A to 4C. If the subject tracking frame enters the ZO area in S2001, that is, if the possibility that the subject is out of frame is high, the process proceeds to S2004, and the zoom-out operation is started. The zoom-out operation here corresponds to a zoom-out operation for frame-out prevention control. After the zoom-out operation, the auto zoom control is terminated.

  On the other hand, if the subject tracking frame has not entered the ZO area in S2001, that is, if the camera is capturing the subject near the center of the screen, the process proceeds to S2002. In S2002, it is determined whether or not the previous zoom operation is a zoom-out operation due to entering the ZO area in S2001, that is, whether or not it is a zoom-out operation for frame-out prevention control. If it is determined in S2002 that the zoom-out operation for frame-out prevention control has been performed, the process proceeds to S2003. If it is determined in S2002 that the zoom-out operation has not been performed for the frame-out prevention control, the process proceeds to S2006.

  In S2003, the AZ control unit 122 determines whether or not the subject tracking frame related to the subject to be tracked is within the ZI region (whether or not included). The ZI region here corresponds to the ZI region described with reference to FIGS. 3B and 4A to 4C. If it is determined in S2003 that the subject tracking frame does not fit in the ZI area, the auto zoom control process ends. On the other hand, if it is determined that the subject tracking frame is within the ZI region, the process proceeds to S2005. That is, if the camera captures the subject so that the subject size is near the center of the screen and within the angle of view at the zoom return position, a zoom-in operation is started in S2005. The zoom-in operation here corresponds to a zoom-in operation for frame-out prevention control. After zooming in, the auto zoom control is terminated.

  In this embodiment, in order to achieve both frame-out prevention control and size retention control in auto-zoom, first, the subject is captured near the center of the screen by frame-out prevention control, and then size retention control can be executed. For this reason, in the state after the zoom-out operation for the frame-out prevention control, the auto-zoom processing (the following S2006 to S2010) for keeping the subject size constant (the size holding control) described below is not performed. . In other words, when the frame-out prevention control is performed, the execution of the size holding control is limited until the zoom-out operation for the frame-out prevention control is completed.

  Next, each process of S2006 to S2010 will be described. If it is determined in S2002 that the zoom-out operation has not been performed for the frame-out prevention control, the process proceeds to S2006, and the AZ control unit 122 sets the subject detection size and the reference size to a predetermined magnification (N1 and N1> 1). ) And the size multiplied. The subject size of the reference subject information set in S1202, S1210, and S1215 of FIG. 12 is compared with the subject size detected in S2000. If the size of the subject detected in S2000 is larger than N1 times the size of the reference subject, that is, if the ratio of the subject image to the screen exceeds a predetermined value, the process proceeds to S2009. In S2009, the AZ control unit 122 starts a zoom-out operation. The zoom-out operation here corresponds to a zoom-out operation for size retention control. After the zoom-out operation, the auto zoom control is terminated.

  On the other hand, in S2006, when the size of the subject detected in S2000 is N1 times or less than the size of the reference subject, the process proceeds to S2007. In S2007, the AZ control unit 122 compares the subject size of the reference subject information with the size of the subject detected in S2000. When the size of the detected subject is smaller than a predetermined magnification (denoted as N2 and N2 <1) with respect to the size of the reference subject (less than N2 times), that is, the ratio of the subject image to the screen is a predetermined value. If it is less, the process proceeds to S2008. On the other hand, when the size of the detected subject is N2 times or more than the size of the reference subject, the auto zoom control is terminated.

  In S2008, the AZ control unit 122 determines whether or not the subject tracking frame related to the subject to be tracked is within the ZI region (whether or not included). This is to prevent the subject from being out of frame by the zoom-in operation when the subject is in the periphery of the screen. The ZI region here corresponds to the ZI region described with reference to FIG. If it is determined in S2008 that the subject tracking frame is not within the ZI area, the auto zoom control is terminated.

  On the other hand, if it is determined in S2008 that the subject tracking frame is within the ZI area, the process proceeds to S2010. In S2010, the AZ control unit 122 starts a zoom-in operation. The zoom-in operation here corresponds to a zoom-in operation for size retention control. Thus, in this embodiment, in order to prevent the subject from being out of frame even in the zoom-in operation for size retention control, the zoom-in operation is started after the subject image is within the ZI region. After zooming in, the auto zoom control is terminated.

  Next, the zoom operation will be described with reference to FIG. FIG. 21 is a flowchart illustrating the zoom-out operation or the zoom-in operation in S2004, S2005, S2009, and S2010 in FIG. First, in step S <b> 2100, the AZ control unit 122 acquires the zoom change amount (change amount of zoom magnification) from the memory 118. In the case of a zoom-out operation for frame-out prevention control, the zoom-out change amount is set according to the detected subject information. Specifically, in the zoom-out operation for frame-out prevention control (S2004 in FIG. 20), the smaller the subject size, the smaller the zoom-out change amount is set. Thereby, it is possible to prevent the subject from becoming undetectable because the size of the subject becomes too small due to the zoom-out operation. In consideration of the minimum size that can be detected by the subject, the zoom-out operation may not be performed when the size of the subject is smaller than a predetermined size. In the zoom-in operation for frame-out prevention control, the zoom magnification before the start of the zoom-out operation is stored in the memory 118, and the zoom-in change amount is set so as to be the same zoom magnification as that before the start of the zoom-out operation.

  In the zoom-out operation for size retention control (S2009 in FIG. 20), a zoom-out change amount (1 / N1 times) corresponding to a predetermined N1 times used for the determination in S2006 is set. Thereby, even when the subject cannot be detected, a minimum zoom-out operation until the size of the subject reaches the size of the reference subject can be performed. Similarly, in the case of the zoom-in operation for size retention control (S2010 in FIG. 20), the zoom-in change amount (1 / N2 times) corresponding to the predetermined N2 times used for the determination in S2007 is set.

  In step S2101, the AZ control unit 122 sets the zoom change amount acquired in step S2100 in the CZ control unit 119 or the electronic zoom control unit 120, and instructs the zoom processing to be performed. In the next step S2102, the AZ control unit 122 determines which zoom operation is being performed for frame-out prevention control or size retention control. If the current zoom operation is a zoom-out prevention zoom operation (S2004, S2005 in FIG. 20), the process proceeds to S2105. If the current zoom operation is a zoom operation for size maintenance control (S2009, S2010 in FIG. 20), the process proceeds to S2103.

  In step S <b> 2103, the AZ control unit 122 determines whether the subject is detected by the subject detection unit 123. If a subject is detected, the process proceeds to S2104. If a subject is not detected, the process proceeds to S2105. In S2104, the size of the subject indicated by the reference subject information is compared with the size of the subject detected in S2103. As a result of the comparison, if the size of the subject detected in S2103 and the size of the reference subject do not fall within a predetermined ratio range (within a predetermined change amount), the process proceeds to S2102 again to determine the zoom operation. continue. If the size of the subject detected in S2103 and the size of the reference subject are within a predetermined range due to the zoom operation, the process proceeds to S2106. In S2106, the AZ control unit 122 stops the zoom operation, and then ends the zoom operation process.

  If it is determined in S2102 that the zoom-out operation for frame-out prevention control is being performed, or if the subject is not detected in S2103 during the zoom operation for size retention control, the process proceeds to S2105. In S2105, the AZ control unit 122 determines based on the zoom change amount acquired in S2100 whether or not a scaling process corresponding to a predetermined amount of zoom change amount corresponding to each zoom operation has been performed. If the zooming process for the predetermined zoom change amount has not been performed, the process returns to S2102 again to continue the process. If zoom processing of a predetermined zoom change amount has been performed, the process proceeds to S2106, where the AZ control unit 122 stops zooming in operation and ends the zoom operation processing.

  In the present embodiment, the shooting scene is determined based on the movement and number of subjects, the detection time of the subject, the distance from the camera to the subject, and the like, and the camera automatically selects an appropriate composition according to the determined scene. Based on the composition selected by the composition selection process, a process for setting the reference size of the subject image is performed, and the zoom operation is performed so that the detected subject image becomes the reference size. According to the present embodiment, it is possible to perform auto zoom control so that an appropriate composition is obtained according to the shooting scene determined based on the detection information of the camera without the photographer selecting the composition.

(Second Embodiment)
Next, a second embodiment of the present invention will be described with reference to FIG. FIG. 22 shows an example of a transition diagram of a composition different from that in FIG. 14 in the automatic composition determination processing S1211 of the composition setting “auto” in FIG. Note that the same composition as that in FIG. 14 is denoted by the same reference numeral, and detailed description thereof is omitted.

In the first embodiment, an example in which a shooting scene is determined when a person different from a photographer is photographed as a subject has been described. The mode at this time is called a normal mode for convenience. In some cases, one or more persons including the photographer himself / herself are photographed as a subject with the camera facing the photographer himself / herself. A mode suitable for such shooting is called a self-portrait mode. In the self-portrait mode, it is possible to perform effective person shooting by adjusting the angle of view suitable for self-portrait by auto-zoom control and applying a background blur effect, a skin beautifying filter effect, and the like.
In the composition adjustment operation, in the normal mode, the subject image needs to be appropriately sized depending on whether the subject is stationary or moving. On the other hand, in the self-portrait mode, it is necessary to change the size of an object that is still, depending on whether the camera is being shot by hand or is being shot. For example, when shooting by hand, the face is taken so that it is within the angle of view, and when the camera is placed, the whole body is taken within the angle of view. As described above, the appropriate composition differs between the shooting in the normal mode and the shooting in the self-portrait mode.
Therefore, in the present embodiment, processing for changing the composition to be selected is executed in the normal mode 2200 and the self-portrait mode 2201 in FIG. That is, the normal mode is illustrated as the first mode, and the self-portrait mode is illustrated as the second mode. As a composition of selection candidates belonging to the first composition group in the normal mode 2200, any composition is selected from the upper body 1400, the whole body 1401, and a plurality of people 1403. As a composition of selection candidates belonging to the second composition group in the self-portrait mode 2201, any one of the self-portrait face 2202 and the self-portrait whole body 2203 is selected.

Here, the determination conditions of the shooting scene and the transition of each composition will be described. When the automatic composition determination process S1211 of the composition setting “auto” starts, the upper body 1400 is set as the composition in the initial state. When the mode is the normal mode 2200, the transition is the same as that during the still image framing 1405 in FIG.
When the mode is changed from the normal mode 2200 to the self-portrait mode 2201, the shake detection unit 125 determines whether the camera is in a hand-held state based on the shake amount applied to the gyro sensor or the like. When the mode is changed to the self-portrait mode 2201 and the shake amount is a predetermined amount or more and it is determined that the camera is handheld, the composition is changed to the self-portrait face 2202. The composition of the self-portrait face 2202 is a composition in the case of taking a picture with his / her face up by hand. Also, when shooting with a plurality of people including the photographer, the zoom is adjusted so that the faces of all the subjects are as large as possible and within the angle of view.

  In the self-portrait mode 2201, the same composition is selected regardless of whether the subject is only a photographer or a plurality of people. That is, the subject position estimated face number shown in FIG. 19 is set to the same value. As described above, the subject position estimated face number is a face tracking frame for estimating the shoulder position Xs, head position Yh, and body position Yb of the subject in the reference size calculation process S1215 of the composition setting “auto” in FIG. This is the number of tracking frames from the center position (Ns, Nh, Nb). In particular, the reference size is calculated so that the number of tracking frames Ns up to the shoulder position and the number of tracking frames Nb up to the body position differ depending on the selected composition. In the first embodiment, when a subject is photographed with a “face” composition, the number of tracking frames from the center position (Xc, Yc) of the face tracking frame is set to the shoulder position Ns = 1 and the body position Nb = 1. .5, a reference size was calculated in which the subject's ears and the vicinity of the chest were within the angle of view. When a plurality of subjects are photographed with a composition of “multiple persons”, the number of tracking frames from the center position (Xc, Yc) of the face tracking frame is set as the shoulder position Ns = 2 and the body position Nb = 3.5. A reference size was calculated with a range that includes all the subject's shoulders and waists within the angle of view. On the other hand, in the case of the self-portraited face 2202, the shoulder position Ns = 1 and the body position Nb = 1.5, which are the same number of tracking frames as the “face”, regardless of the number of subjects, are set. As a result, a reference size is calculated so that the angle of view covers all the ears and chests of all subjects, and the zoom is adjusted to an angle of view suitable for hand-held self-portrait photography.

  When the self-portrait face 2202 is changed, the number of detected subjects is stored in the memory 118. In the composition state of the self-portraiting face 2202, determination processing is performed to determine whether or not the number of detected subjects has changed. When the number of detected subjects increases, the number of subjects stored in the memory 118 is immediately updated, and the reference size is changed. When the number of subjects decreases, when the state continues for a predetermined time or more, the number of subjects stored in the memory 118 is updated and the composition is changed. This is to prevent the subject from being out of frame by changing the composition immediately when the subject cannot be detected temporarily because it is turned sideways, as in the case of the plurality of people 1403.

  When the mode is changed from the normal mode 2200 to the self-portrait mode 2201, the shake detection unit 125 has the camera placed on a desk or the like (fixed) based on the shake amount applied to the gyro sensor or the like. Determine if it is in a state. When the mode is changed to the self-portrait mode 2201, and it is determined that the camera shake is less than the predetermined amount and the camera is placed, the composition is changed to the self-portrait whole body 2203. The composition of the self-portrait whole body 2203 is a composition when the whole body is photographed by placing a camera. Further, when shooting a plurality of people including the photographer, the zoom is adjusted so that the whole body of all the subjects is within the angle of view. In the case of the self-portrait whole body 2203, the shoulder position Ns = 2 and the body position Nb = 10 which are the same number of tracking frames as the “whole body” regardless of the number of subjects are set. As a result, a reference size is calculated so that the shoulders and feet of all subjects are within the angle of view, and the zoom is adjusted to an angle of view suitable for taking a self-portrait with the camera placed. When the self-portrait whole body 2203 is changed, the number of detected subjects is stored in the memory 118. Similarly to the self-portrait 2202 in the self-portrait whole body 2203, when the number of subjects changes, the number of subjects stored in the memory 118 is updated and the reference size is changed.

In the present embodiment, the photographing scene to be discriminated is changed according to the set mode, and the camera automatically selects an appropriate composition for each mode. In the present embodiment, the case where an appropriate composition is selected from a plurality of composition selection candidates in one mode has been illustrated, but depending on the mode, only one composition may be a selection candidate. Furthermore, although the case of the normal mode and the self-portrait mode has been illustrated as the mode, it is not limited to this mode. For example, in addition to the normal mode and the self-portrait mode, there are embodiments in which the composition of the athlete's “whole body” is selected in the sport mode, and in the baby mode, the composition of the baby's “face” is selected. May be.
According to the present embodiment, the auto zoom control can be performed so that an appropriate composition is obtained according to the shooting scene determined based on the set mode and the detection information of the camera without the photographer selecting the composition. it can.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 digital camera
Reference Signs List 102 Zoom lens 109 Display unit 113 Zoom lens drive unit 117 Operation unit 118 Memory 119 CZ control unit 120 Electronic zoom control unit 122 Auto zoom control unit 123 Subject detection unit

Claims (15)

Subject detection means for detecting a subject image from the image;
Determination means for acquiring information including detection information by the subject detection means and determining a scene;
Composition selecting means for selecting one of a plurality of compositions based on the scene determined by the determining means;
Setting means for setting a reference size of the subject image used for zoom magnification control based on the composition selected by the composition selection means and the size and position of the subject image detected by the subject detection means;
Control means for controlling the zoom magnification based on the reference size set by the setting means and the size of the subject image sequentially detected by the subject detection means;
A zoom control apparatus comprising:   The determining means determines the scene by acquiring the number of subject images detected by the subject detecting means, and the composition selecting means is a composition when the number of the subject images is singular, or of the subject images. The zoom control apparatus according to claim 1, wherein a composition in a case where the number is plural is selected.   The determination unit determines the scene by acquiring information on the position of the subject image detected by the subject detection unit in the image, and the composition selection unit selects a composition for each scene determined by the determination unit. The zoom control device according to claim 1, wherein the zoom control device is a zoom control device.   4. The zoom control apparatus according to claim 1, wherein the determination unit determines the scene from a change amount of a size of a subject image detected by the subject detection unit. 5. Further comprising a detection means for detecting a Re rocking of the zoom control unit,
5. The zoom control apparatus according to claim 1, wherein the determination unit determines a scene by acquiring detection information from the detection unit. A measuring unit that measures a distance from the zoom control device to a subject using information on a size of a subject image detected by the subject detecting unit and information on a zoom magnification controlled by the control unit;
The zoom control apparatus according to claim 1, wherein the determination unit determines a scene by acquiring information on the distance measured by the measurement unit.   The determination unit determines a scene by acquiring information about the distance measured by the measurement unit and compares the information with a threshold value, and the composition selection unit determines that the distance measured by the measurement unit is greater than the threshold value. The zoom control apparatus according to claim 6, wherein the zoom control apparatus changes to the first composition when the distance measured by the measuring unit is smaller than a threshold value. It further has an acquisition means for acquiring the set shooting mode,
The composition selection means changes the composition of the selection candidate when the shooting mode acquired by the acquisition means is the first mode and the composition of the selection candidate when the shooting mode is the second mode. The zoom control apparatus according to any one of claims 1 to 7, wherein the zoom control apparatus is characterized in that:   The composition selection means selects a composition belonging to the first composition group when the composition mode is composed of at least one composition, and is in the second mode. 9. The zoom control apparatus according to claim 8, wherein a composition belonging to the second composition group is selected.   9. The first mode is a mode used for photographing a subject other than a photographer, and the second mode is a mode used for photographing a subject including the photographer itself. Or the zoom control device according to 9;   The control means controls the zoom magnification so that the size of the subject image detected by the subject detection means falls within a predetermined ratio with respect to the reference size. A zoom control device according to claim 1. A zoom control device according to any one of claims 1 to 11,
Imaging means for photoelectrically converting a subject image to generate an image;
Comprising zoom means for changing the zoom magnification by driving the lens or image processing;
The control means controls the zoom means, and controls a zoom magnification according to a change in the size of the subject image detected by the subject detection means based on the reference size set by the setting means. An imaging device. A control method executed by a zoom control device,
A determination step of acquiring information including detection information of a subject image detected from an image and determining a scene;
A composition selection step for selecting one of a plurality of compositions based on the scene determined in the determination step;
A setting step for setting a reference size of a subject image used for zoom magnification control based on the composition selected by the composition selection step and the size and position of the subject image indicated by the detection information;
A control step for controlling a zoom magnification based on the reference size set in the setting step and the size of the subject image indicated by the detection information;
A control method for a zoom control apparatus comprising:   A control program for a zoom control apparatus, which causes a computer to execute each step of the control method for a zoom control apparatus according to claim 13.   15. A storage medium in which a control program for the zoom control device according to claim 14 is stored.

Images