JP4939203B2 - Display control device, imaging device, and control method - Google Patents

Description

  The present invention relates to an image display control apparatus and an imaging apparatus capable of displaying a captured image in an enlarged manner.

Conventionally, a display technique for confirming a focus state as to whether or not an image captured by an imaging apparatus is in focus has been disclosed. For example, the digital camera disclosed in Patent Document 1 displays an enlarged area including an in-focus position designated before shooting at a magnification corresponding to the number of shooting pixels.
Japanese Patent No. 3534101

  As described above, a partial region of the captured image is displayed in order to confirm the focused state of the captured image. The partial area to be displayed is an area when the user as the photographer specifies an area that the user wants to focus on, or when the imaging apparatus automatically specifies an area that the user will desire to focus on. It is. Therefore, the focus of the optical system of the image pickup apparatus is adjusted so as to focus on the designated area. However, there is a time lag between the timing when the area desired to be focused in this way is specified and the subject corresponding to the area is focused to the timing at which shooting is actually performed for recording. is there. When the user changes the angle of view while maintaining the focus during this time lag, a region different from the region corresponding to the focused subject is displayed as a partial region. This is the same when, for example, the user changes the shooting composition while maintaining the focus during the time lag described above.

  In such a case, the displayed partial area is often not an area corresponding to the focused subject, and there is a problem that it is difficult for the user to check the in-focus state.

The present invention has been made to solve the above-described problems, and is a technical idea for controlling the display of a captured image, and includes a position detection unit that detects a subject position in the captured image, and an image capturing for image recording. Extraction means for extracting subject information of the captured image before and after processing, conversion means for converting the subject information extracted by the extraction means in accordance with the zoom state after extraction, and the subject position detected by the position detection means Control means for controlling to display a partial area of the corresponding captured image, and the control means displays the partial area of the captured image when a plurality of subject positions are detected by the position detection means. If, subject extracted after converted object information and the imaging processing by the conversion means but before the imaging processing for the extracted image recorded by the extraction unit Selecting a partial area of a captured image to be displayed from a plurality of object position detected by said position detecting means on the basis of the distribution.

  According to the present invention, an in-focus state of an image to be recorded or recorded by performing display for confirming the in-focus state with a region corresponding to the focused subject as a partial region. It is easier to confirm this in a breakthrough than in the past.

Example 1
The configuration of a camera as an imaging apparatus of an embodiment that well expresses the features of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram showing a configuration of an embodiment of the present invention. In FIG. 1, reference numeral 100 denotes an imaging apparatus such as a camera or a video camera. Reference numeral 10 denotes a photographing lens, and reference numeral 12 denotes a shutter having an aperture function.

  Reference numeral 14 denotes an image sensor such as a CCD or CMOS that converts an optical image into an electric signal, and 16 denotes an A / D converter that converts an analog signal output from the image sensor 14 into a digital signal. A timing generation circuit 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26, and is controlled by the memory control circuit 22 and the system control circuit 50.

  An image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 16 or the data from the memory control circuit 22. Further, the image processing circuit 20 performs predetermined arithmetic processing using the captured image data. The system control circuit 50 controls the exposure control means 40 and the focus control means 42 based on the obtained calculation result. In addition, TTL (through the lens) AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash pre-emission) processing are performed.

  Further, the image processing circuit 20 performs predetermined arithmetic processing using the captured image data, and also performs TTL (through the lens) type AWB (auto white balance) processing based on the obtained arithmetic result. ing. In addition, face detection processing as subject position detection processing is performed on input data from the A / D converter 16 and the memory control circuit 22. Information such as the number of detected faces, the size, position, and eye position of each detected face is also extracted.

  A memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression circuit 32. The data of the A / D converter 16 is written into the image display memory 24 or the memory 30 via the image processing circuit 20 and the memory control circuit 22 or the data of the A / D converter 16 is directly passed through the memory control circuit 22. It is.

  Reference numeral 24 denotes an image display memory, 26 denotes a D / A converter, and 28 denotes an image display unit including a TFT LCD. The display image data written in the image display memory 24 is displayed by the image display unit 28 via the D / A converter 26. If the image data captured using the image display unit 28 is sequentially displayed, the electronic viewfinder function can be realized. Further, the image display unit 28 can arbitrarily turn on / off the display according to an instruction from the system control circuit 50. When the display is turned off, the power consumption of the imaging apparatus 100 can be significantly reduced. I can do it.

  Reference numeral 30 denotes a memory for storing captured still images and moving images. The memory 30 has a storage capacity sufficient to store a predetermined number of still images and a moving image for a predetermined time. Thereby, even in the case of continuous shooting or panoramic shooting in which a plurality of still images are continuously shot, it is possible to write a large amount of images to the memory 30 at high speed. The memory 30 can also be used as a work area for the system control circuit 50.

  A compression / decompression circuit 32 compresses and decompresses image data by adaptive discrete cosine transform (ADCT) or the like. The compression / decompression circuit 32 reads an image stored in the memory 30, performs a compression process or an expansion process, and writes the processed data to the memory 30.

  Reference numeral 40 denotes an exposure control means for controlling the shutter 12 having a diaphragm function, and has a flash light control function in cooperation with the flash 48. Reference numeral 42 denotes focus control means for controlling the focal position of the taking lens 10, reference numeral 44 denotes zoom control means for controlling zooming of the taking lens 10, and reference numeral 46 denotes barrier control means for controlling the operation of the protection means 102 serving as a barrier.

  A flash 48 has an AF auxiliary light projecting function and a flash light control function.

  The exposure control means 40 and the focus control means 42 are controlled using the TTL method, and the system control circuit 50 uses the exposure control means 40 and the focus control means based on the calculation result obtained by calculating the captured image data by the image processing circuit 20. 42 is controlled.

  Reference numeral 50 denotes a system control circuit that controls the entire imaging apparatus 100, and 52 denotes a memory that stores constants, variables, programs, and the like for operation of the system control circuit 50.

  Reference numeral 54 denotes a display unit such as a liquid crystal display device or a speaker that displays an operation state, a message, and the like using characters, images, sounds, and the like in accordance with execution of a program in the system control circuit 50. The display unit 54 is installed at a single or a plurality of locations near the operation unit of the imaging apparatus 100 where it can be easily seen, and is configured by, for example, a combination of an LCD, an LED, a sounding element, and the like. In addition, the display unit 54 is partially installed in the optical viewfinder 104. Among the display contents of the display unit 54, what is displayed in the optical viewfinder 104 includes, for example, a focus display, a shooting preparation completion display, a camera shake warning display, and a recording medium writing operation display. Further, among the display contents of the display unit 54, what is displayed on the LED or the like includes, for example, a focus display, a shooting preparation completion display, a camera shake warning display, a recording medium writing operation display, and the like.

  Reference numeral 56 denotes an electrically erasable / recordable non-volatile memory such as an EEPROM.

  Reference numerals 60, 62, 64, 66, and 70 denote operation means for inputting various operation instructions of the system control circuit 50. The operation means 56 is configured by a single or a combination of a switch, a dial, a touch panel, pointing by eye-gaze detection, a voice recognition device, and the like.

  Here, a specific description of these operating means will be given. Reference numeral 60 denotes a mode dial switch which can switch and set each function photographing mode.

  Reference numeral 62 denotes a shutter switch SW1. The shutter switch SW1 is turned on during the operation of a shutter button (not shown). When in the ON state, the start of operations such as AF processing, AE processing, AWB processing, and EF processing is instructed.

  Reference numeral 64 denotes a shutter switch SW2. The shutter switch SW2 is turned on when the operation of a shutter button (not shown) is completed. In the ON state, an exposure process is performed in which a signal read from the image sensor 14 is written to the memory 30 via the A / D converter 16 and the memory control circuit 22. Further, development processing using computations in the image processing circuit 20 and the memory control circuit 22, image data is read from the memory 30, and compression is performed by the compression / decompression circuit 32. Image data is written to the recording medium 200 or 210 as the compressed signal. The shutter switch SW2 instructs the start of a series of processing such as recording processing.

  Reference numeral 66 denotes a playback mode switch, which can set various function modes such as a playback mode, a multi-screen playback / erase mode, and a PC connection mode.

  Reference numeral 68 denotes an attitude mode switch. The posture mode switch 68 serves as a trigger for detecting the posture of the imaging apparatus 100 by the camera posture detection means 106. The camera attitude detected at the time of imaging is recorded on the recording medium as an image attitude flag together with the captured image. At the time of reproduction, ON / OFF of an attitude mode for performing an operation of switching an image display method according to the read image attitude flag is set.

  An operation unit 70 includes various buttons and a touch panel. The operation unit 70 includes a selection / switching button for setting selection and switching of various functions. Further, there are an image display ON / OFF switch for setting ON / OFF of the image display unit 28, a quick review ON / OFF switch for setting a quick review function for automatically reproducing image data taken immediately after shooting, and the like.

  Reference numeral 72 denotes a power switch, which can switch between power-on and power-off modes. Reference numeral 80 denotes power supply control means. Reference numeral 82 denotes a connector, 84 denotes a connector, and 86 denotes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like.

  Reference numerals 90 and 94 denote interfaces with recording media such as memory cards and hard disks. Interfaces 90 and 94 are connectors 92 and 96 for connecting to a recording medium such as a memory card or a hard disk, and 98 is a recording medium attachment / detachment for detecting whether or not the recording medium 200 or 210 is attached to the connector 92 or 96. It is a detection means.

  Further, the interfaces 90 and 94 and the connectors 92 and 96 comply with standards such as a PCMCIA card and a CF (Compact Flash (registered trademark)) card. Therefore, by connecting various communication cards, image data and management information attached to the image data can be transferred to and from other computers and peripheral devices such as a printer. The various communication cards are communication cards such as a LAN card, a modem card, a USB card, an IEEE 1394 card, a P1284 card, a SCSI card, and a PHS.

  Reference numeral 104 denotes an optical viewfinder.

  Reference numeral 106 denotes camera posture detection means, which can detect either the horizontal position shooting or the vertical position shooting by detecting the posture state of the imaging apparatus 100.

  Reference numeral 108 denotes an acceleration / angular velocity detection unit, and the amount of movement and the rotation angle of the imaging apparatus 100 can be obtained by using the integral output.

  A communication unit 110 has various communication functions. For example, RS is 32C, USB, IEEE1394, P1284, SCSI, modem, LAN, and wireless communication.

  Reference numeral 112 denotes a connector for connecting the imaging apparatus 100 to another device by the communication unit 110 or an antenna in the case of wireless communication.

  Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium 210 includes a recording unit 212 composed of a semiconductor memory, a magnetic disk, and the like, an interface 214 with the imaging device 100, and a connector 216 that connects to the imaging device 100.

  The operation of the first embodiment of the present invention will be described with reference to FIGS.

  2 and 3 show a flowchart of a main routine of the image pickup apparatus 100 of the first embodiment.

  The operation of the imaging apparatus 100 will be described with reference to FIGS.

  First, upon power-on such as battery replacement, the system control circuit 50 initializes flags, control variables, and the like (S101), and initializes the image display of the image display unit 28 to an OFF state (S102). The system control circuit 50 determines the set position of the power switch 66, and if the power switch 66 is set to OFF (S103), the display on each display unit is changed to the end state. After performing a predetermined end process such as shutting off unnecessary power of each part of the imaging apparatus 100 including the image display unit 28 by the power supply control unit 80 (S104), the process returns to S103.

  On the other hand, if the power switch 66 is set to ON (S103), the system control circuit 50 determines the set position of the mode dial 60. If the mode dial 60 is set to the shooting mode (S105), the process proceeds to S107. If the mode dial 60 is set to the reproduction mode (S105), the image reproduction process is executed (S106), and if the process is completed, the process returns to S103. Details of the image reproduction processing S106 will be described later with reference to FIG.

  The system control circuit 50 determines whether there is a problem with the operation of the imaging apparatus 100 and the recording medium 200 or 210 with respect to the remaining capacity and operation status of the power supply 86 by the power supply control means 80 (S107, S108). If there is a problem, a predetermined warning is displayed by image or sound using the display unit 54 (S109), and the process returns to S103.

  The system control circuit 50 determines whether or not face detection is set by the operation unit 70, and updates the face detection ON / OFF state (S110). The face detection ON / OFF setting can be switched by a face button included in the operation unit 70 or can be switched by a camera setting menu. If the setting state of face detection is updated, it will progress to S111.

  The system control circuit 50 determines whether an operation corresponding to AF lock (hereinafter referred to as AFL) release has been performed by the operation unit 70 or the like (S111). If the AFL release operation has not been performed, the process proceeds to S113. If an AFL release operation has been performed (S111), the AFL flag stored in the internal memory of the system control circuit 50 or the memory 52 is turned off (S112), and the process proceeds to S113.

  The system control circuit 50 uses the display unit 54 to display various setting states of the image capturing apparatus 100 using images and sounds (S113). When displaying a live image, if a face is detected by the face detection process (S125), a frame (hereinafter referred to as a face frame) is displayed at a corresponding position.

  The system control circuit 50 determines whether or not face detection is ON (S124). If face detection is OFF (S124), the process proceeds to S127. The setting of face detection ON / OFF is performed in S110. If face detection is ON (S124), face detection processing is performed (S125). The face detection process as the object position detection benefit is performed by the image processing unit 20 on the input data from the A / D converter 16 and the memory control circuit 22. If the face detection process is completed (S125), the process proceeds to S126.

  The system control circuit 50 stores the feature amount of each face extracted in S125 as subject information in the internal memory or the memory 52 of the system control circuit 50 (S126). The feature quantity to be extracted and stored includes the position in the captured image of the face as the subject. In addition, it is the position of the face size in the captured image. In addition, by using these pieces of information, the relative position of the eyes with respect to the face as the subject, the distance between the eyes, and the like can be used as feature amounts.

  If the shutter switch SW1 has not been pressed (S127), the process returns to S103. If the shutter switch SW1 is pressed (S127), the process proceeds to S128.

  The system control circuit 50 performs photometric processing to determine the aperture value and shutter time (S128). In the photometric process, the flash is set if necessary. Also, white balance processing is performed.

  After completing the AE / AWB process S128, the system control circuit 50 determines the state of the AFL flag stored in the internal memory of the system control circuit 50 or the memory 52 (S129). If the AFL flag is set, the process proceeds to S137 as an instruction for imaging processing for image recording. If the AFL flag is not set (S129), the system control circuit 50 refers to the latest face detection result stored in the system control circuit 50 or the internal memory 52, and if a face is detected (S130), S131. Proceed to

  The system control circuit 50 designates a portion corresponding to the detected face area as a focus area. AF operation is performed (S131), and the process proceeds to S132. The AF operation is an operation for extracting the degree of contrast of the image in the designated focus area while shifting the focus position of the photographing lens 10.

  The system control circuit 50 selects one focus area in accordance with a predetermined algorithm among the focus areas for which the focus determination is OK in S131. The selected one focus area is designated as a focusing frame and stored in the internal memory of the system control circuit 50 or the memory 52 (S132). In S132, the internal memory of the system control circuit 50 or the storage of the memory 52 stores the coordinates as information on the focusing frame.

  Here, the part corresponding to the face detected in S131 is designated as the focus area, and the AF operation is performed. Therefore, the in-focus frame stored and stored in the internal memory of the system control circuit 50 or the memory 52 corresponds to a part of the detected faces. That is, an AF operation is performed to focus on a subject that the user desires or is estimated to be desired. Therefore, the timing for storing the feature quantity as subject information may be S132.

  If the focus determination is NG in all focus areas, the focus frame information is cleared.

  In S131, the system control circuit 50 stores the facial feature amount corresponding to the focusing frame in the internal memory or the memory 52 of the system control circuit 50 (S133). In S134, the feature amount of the focused face is acquired from the feature amounts stored in S126, and is separately stored as focused face information. If there is no in-focus face, the in-focus face information is cleared.

  The system control circuit 50 stores information on the current state of the imaging apparatus in the internal memory of the system control circuit 50 or the memory 52 (S134). This timing is a timing in which the AF operation is performed and the subject is desired or is estimated to be desired. The storage here stores, for example, electronic zoom magnification and zoom information (focal length, optical zoom magnification, etc.) acquired from the zoom control unit 44 in a memory.

  If it is determined in S130 that no face is detected (S131), the process proceeds to S135. Note that if the face detection is set to OFF in S110, the process proceeds to S135. The system control circuit 50 performs an AF operation based on a predetermined focus area without using the face detection result (S135). Then, one focus area is selected from the focus areas for which the focus determination is OK according to a predetermined algorithm and designated as a focus frame. Then, the information on the focusing frame is stored in the internal memory of the system control circuit 50 or the memory 52 (S136). If the focus determination is NG in all focus areas, the focus frame information is cleared without being saved.

  If the shutter switch SW2 is not pressed (S137) and the shutter switch SW1 is also released (S138), the process returns to S103. If the shutter switch SW2 is not pressed (S137) and the shutter switch SW1 is pressed (S138), the process proceeds to S139.

  The system control circuit 50 determines whether or not the AFL button included in the operation unit 70 has been pressed (S139), and returns to S137 if the AFL button has not been pressed. If the AFL button has been pressed (S139), the system control circuit 50 validates the AFL flag stored in the internal memory of the system control circuit 50 or the memory 52 (S140), and returns to S137.

  If the shutter switch SW2 is pressed (S137), the system control circuit 50 performs the following processing. That is, the system control circuit 50 performs an exposure process for writing image data obtained by imaging in the memory 30. This processing is performed via the image sensor 14, the A / D converter 16, the image processing circuit 20, the memory control circuit 22, or directly from the A / D converter via the memory control circuit 22. Further, the system control circuit 50 performs a photographing process including a developing process for reading out the image data written in the memory 30 and performing various processes (S141). This processing is performed using the memory control circuit 22 and, if necessary, the image processing circuit 20. Specifically, a charge signal is read from the image sensor 14, and photographed image data is written into the memory 30 via the A / D converter 16, the image processing circuit 20, and the memory control circuit 22. Alternatively, the captured image data is written in the memory 30 directly from the A / D converter 16 via the memory control circuit 22.

  When the photographing process S141 is completed, the system control circuit 50 stores the information related to the current state of the imaging device (timing when photographing for recording is performed) in the internal memory or the memory 52 of the system control circuit 50. (S142). This timing is different from the timing when the AF operation is performed and the user focuses on the subject desired or estimated to be desired. In other words, the composition may change because it differs from the timing at which the AF operation is performed. The information stored here corresponds to the information stored in S134. For example, electronic zoom magnification and zoom information acquired from the zoom control unit 44 (focal length, optical zoom magnification, etc.). In S142, information indicating how much the imaging device has been moved is further acquired from the timing of S134. For example, the acceleration / angular velocity detection sensor 108 is used. By comparing the information stored at the AF operation timing (S134) and the shooting timing (S142) in the later reproduction processing, the feature amount and the focusing frame are converted or corrected.

  If the shooting timing information is stored (S142), a quick review process is performed (S127). Details of the quick review process will be described later with reference to FIG.

  Note that the order of storing information related to the current state of the imaging device (timing at which imaging for imaging for recording) is performed after the imaging processing S141 is completed (S142) is not necessarily in this order. . The process of S141 and the process of S142 may be performed in parallel. Further, in response to pressing of the shutter switch SW2 (S137), the processing of S142 may be started at the timing of pressing SW2.

  When the quick review display S143 is completed, the system control circuit 50 reads the captured image data written in the memory 30. Then, various kinds of image processing are performed using the memory control circuit 22 and, if necessary, the image processing circuit 20, and compression processing is performed to perform image compression processing according to the mode set using the compression / decompression circuit 32. Thereafter, a recording process for writing image data to the recording medium 200 or 210 such as a memory card or a compact flash (registered trademark) card is executed via the interface 90 or 94 and the connector 92 or 96 (S144). In addition to the captured image, the following information is also written to the recording medium 200 or 210. These are the in-focus frame information stored in S132 or S136 in the internal memory or the memory 52 of the system control circuit 50, and the feature amount related to the in-focus face stored in S133. Further, the information regarding the state of the imaging apparatus at the time of the AF operation stored in S134 and the information regarding the state of the imaging apparatus at the time of shooting stored in S142 are also written.

  When the image display unit 28 is ON, a display such as “BUSY” that clearly indicates that the writing operation is being performed while the image data is being written to the recording medium 200 or 210 is displayed on the image display unit 28. To do.

  Further, the display unit 54 also displays a recording medium writing operation display such as blinking the LED.

  If the recording process is completed (S144), the process returns to S103.

  FIG. 4 shows a detailed flowchart of the image reproduction process in S106 of FIG.

  The system control circuit 50 determines whether the single reproduction mode is selected (S401), and if the single reproduction mode is selected, the system control circuit 50 proceeds to S403. Switching between the single reproduction mode / multiple reproduction mode can be performed using the operation unit 70 or the like.

  If it is not in the single reproduction mode (S401), processing corresponding to the reproduction operation is executed (S402), and when the processing is completed, the image reproduction processing routine S106 is terminated.

  In the case of the single reproduction mode (S401), the system control circuit 50 determines whether or not it is in the focus confirmation mode (S403), and executes the focus confirmation reproduction process S404 if it is in the focus confirmation mode. ON / OFF of the focus confirmation mode can be switched by a button included in the operation unit 70 or can be switched by a camera setting menu. Details of the focus confirmation reproduction process S404 will be described later with reference to FIG. When the focus confirmation reproduction process S404 is completed, the image reproduction process routine S106 is terminated.

  If it is not in the focus confirmation mode, processing corresponding to the reproduction operation is executed (S402), and when the processing is completed, the image reproduction processing routine S106 is terminated.

  FIG. 5 shows a detailed flowchart of the quick review process in S143 of FIG. The quick review is a process of displaying an image immediately before being captured for recording so that the photographer can confirm it.

  The system control circuit 50 determines whether or not the mode is the quick review mode (S701). If the mode is not the mode for performing the quick review, the quick review processing routine S143 is terminated. The quick review mode can be turned ON / OFF using a button included in the operation unit 70 or can be switched using a camera setting menu.

  If it is the quick review mode (S701), the system control circuit 50 determines whether or not it is in the focus confirmation mode (S702). If it is the focus confirmation mode, the focus confirmation reproduction process S703 is performed, and if the process is completed, the quick review process routine S143 is terminated. ON / OFF of the focus confirmation mode can be switched by a button included in the operation unit 70 or can be switched by a camera setting menu. Details of the focus confirmation reproduction process S703 will be described later with reference to FIG.

  When the focus confirmation mode is not set (S702), the system control circuit 50 displays the captured image display data written in the image display memory 24 on the image display unit 28 (S704), and proceeds to S705.

  The system control circuit 50 determines whether or not to end the display (S705). If the display is not ended, the system control circuit 50 returns to S705. When determining whether to end the display, the elapse of the quick review time, the presence / absence of a display end instruction from the operation unit 70, and the like are considered. Even when an image forwarding instruction is given by the operation unit 70 or the like, the display is once ended and the reproduced image file is updated.

  If it is determined that the display is to be terminated (S705), the display of the captured image by the image display unit 28 is terminated (S706), and the quick review processing routine S143 is terminated.

  FIG. 6 shows a detailed flowchart of the focus confirmation reproduction process in S404 of FIG. 4 and S703 of FIG. In S404 (reproduction mode) and S703 (rec review), the location where information to be used is stored is different, but the processing flow is the same, and therefore the same flowchart will be used.

  The system control circuit 50 determines whether in-focus face information is stored in the image file in S404 (playback mode) and in the internal memory or the memory 52 of the system control circuit 50 in S703 (rec review) ( S1001). If the in-focus face information is stored, the process proceeds to S1002. The focused face information is a feature quantity of the face as the focused subject information stored in the file in S144 in the internal memory or the memory 52 of the system control circuit 50 in S133. If face detection is set to OFF in S110, or if no face is in focus in S131 even when face detection is ON, the in-focus face information is not stored.

  When the in-focus face information is stored (S1001), the system control circuit 50 inputs the display image data written in the image display memory 24 to the image processing unit 20 via the memory control circuit 22, and Face detection processing is executed (S1002). In S404 (reproduction mode), the image data in the file to be reproduced is developed in advance as image data for display in the image display memory 24, and the above processing is performed. In S703 (quick review), the above processing is performed in a state where display data for the image taken immediately before is developed in the image display memory 24.

  When the face detection process ends (S1002), the system control circuit 50 determines whether or not a face is detected in S1002 (S1003). If a face is detected, the process proceeds to S1004.

  The system control circuit 50 acquires a face feature amount as subject information detected from the face detection result of S1002 (S1004). The obtained feature amount is converted using the state information of the image pickup apparatus at the time of AF operation and the state information of the image pickup apparatus at the time of image pickup for recording (S1005). In S404 (playback mode), the state information at the time of AF operation and the state information at the time of shooting, which are written as additional information in the image file in S144, are used. On the other hand, in S703 (during REC review), state information at the time of AF operation and state information at the time of photographing stored in the internal memory of the system control circuit 50 or the memory 52 are used.

  The feature amount of the face is used for comparison determination whether the face detected in S1002 is the face focused in the AF operation in S131. For example, if the zoom magnification or composition is changed after the AF operation of S131 (S141), the feature amount such as the size of the face changes between the AF operation and the shooting even for the same face. Therefore, it is necessary to convert feature amounts using information about the imaging apparatus 100 such as zoom magnification so that the feature amounts can be compared correctly. The state information at the time of AF operation and the state information at the time of photographing used for the conversion of the feature amount include the output of an acceleration sensor and an angular velocity sensor in addition to the optical zoom magnification and the electronic zoom magnification.

  By using the optical zoom magnification and the electronic zoom magnification, a comparison using a feature amount such as a face size depending on a zoom state of the imaging apparatus 100 can be performed. By using the integrated value of the output of the acceleration sensor and the angular velocity sensor from the time of AF operation to the time of shooting, even when the imaging device 100 moves or rotates, a comparison using the position of the face and eyes as the feature amount is possible. It becomes. In this case, the output from the acceleration / angular velocity detector 108 is used. Note that the conversion processing in S1005 can be omitted when face comparison determination can be performed using feature quantities that do not depend on the state of the imaging apparatus 100.

  When the feature amount conversion is completed (S1005), the system control circuit 50 sorts the faces detected in S1002 in the order in which the feature amount is close to the focused face feature amount (S1006). In S404 (playback mode), the focused face feature amount is written in the image file in S144, and in S703 (quick review), it is stored in the internal memory or the memory 52 of the system control circuit 50. It is conceivable that a plurality of faces are focused in a single process, and there are a plurality of feature amounts of the focused face. In this case, one face selected according to a predetermined algorithm among the focused faces is designated as the focused face. Note that the following processing may be repeated for a plurality of focused faces.

  The system control circuit 50 selects and sets a partial area corresponding to the face whose feature quantity is closest to the in-focus face as a result of sorting in S1006 from the faces detected in S1002 (S1007). .

  The system control circuit 50 determines whether or not the difference between the feature amount of the candidate face to be enlarged and the feature amount of the focused face falls within a certain threshold (S1008).

  If the difference between the feature amounts exceeds the threshold value, the system control circuit 50 determines whether or not a face closest to the focused face is set as a candidate face to be enlarged (S1009). ). If the candidate face to be enlarged is the face closest to the focused face, the process proceeds to S1010. If the candidate face to be enlarged is not the face closest to the focused face, the process returns to S1007.

  The system control circuit 50 displays a warning using the image display unit 28 and the display unit 54 (S1010). The warning display indicates to the user that there is no face that is considered to match the focused face in S1008.

  The system control circuit 50 determines whether or not to end the display based on a timeout and a display end instruction from the operation unit 70 (S1011). If the display is to be terminated, the process proceeds to S1019. If the display is not to be terminated, the process returns to S1011. Even when an image forwarding instruction is given by the operation unit 70 or the like, the display is once ended and the reproduced image file is updated.

  If the difference between the feature amount of the candidate face to be enlarged and the feature amount of the focused face is less than the threshold in S1008, the corresponding partial area is displayed. That is, the system control circuit 50 enlarges the display candidate face using the display unit 28 (S1012), and proceeds to S1013.

  The system control circuit 50 determines whether or not to end the display based on a time-out and a display end instruction from the operation unit 70 (S1013). If the display is to be terminated, the process proceeds to S1019. If the display is not to be terminated, the process proceeds to S1014. Even when an image forwarding instruction is given by the operation unit 70 or the like, the display is once ended and the reproduced image file is updated.

  The system control circuit 50 determines whether or not there is an instruction to display the next face by the operation unit 70 or the like (S1014). If there is no instruction, the process returns to S1013, and if there is an instruction, the process proceeds to S1015.

  The system control circuit 50 determines whether there is still a candidate face to be enlarged (S1015). If there are remaining candidates, the system control circuit 50 proceeds to S1016 and updates the candidate face to the next face in the sort result of S1006. The process returns to S1008, and if there is no candidate, the process returns to S1007.

  If the information on the focused face is not stored in S1001, the process proceeds to S1020. That is, if the face AF process of S131 is not performed and if no face is in focus during the face AF process (S131), the process proceeds to S1020.

  The system control circuit 50 determines whether or not in-focus frame information is stored in the internal information of the system control circuit 50 or in the memory 52 in S404 (during playback mode), and in S703 (during quick review). (S1020). If there is in-focus frame information, a portion corresponding to the in-focus frame position of the image is enlarged and displayed using the image display unit 28 (S1021). If there is no in-focus frame information (S1020), the central portion of the image is enlarged and displayed using the image display unit 28, and the process proceeds to S1024.

  If it is determined in step S1003 that no face has been detected, the process proceeds to step S1023, and a warning is displayed using the image display unit 28 and the display unit 54 that the in-focus face does not exist in the reproduced image (S1023). Proceed to

  The system control circuit 50 determines whether or not the display should be terminated by a timeout and a display termination instruction from the operation unit 70 (S1024). If the display should be terminated, the process proceeds to S1019, and if the display should not be terminated. The process returns to S1024. Even when an image forwarding instruction is given by the operation unit 70 or the like, the display is once ended and the reproduced image file is updated.

  Finally, the system control circuit 50 ends the display by the image display unit 28 and the display unit (S1019), and ends the focus confirmation reproduction processing routines S404 and S703.

  In the above flow, the system control circuit 50 converts the detected face feature amount as the subject information using the state information of the imaging device (S1005). In this respect, the following technical idea may be used. In other words, this is a method that does not use feature quantities as subject information. For example, the system control circuit 50 performs an enlarged display in S1012 based on a comparison between the state information of the image pickup apparatus during the AF operation and the state information of the image pickup apparatus during the image pickup for recording. When the zoom magnification is changed from the imaging apparatus during the AF operation, the size of the subject after imaging is predicted in accordance with the zoom magnification, and the face area to be enlarged is selected. Similarly, when the position of the imaging apparatus during the AF operation changes, the position of the subject after imaging is predicted in accordance with the movement, and the face area to be enlarged is selected. However, in this case, the appropriate accuracy of the enlarged image may be slightly inferior to the technical idea using the feature amount as subject information as in the above flow.

  Next, a screen display example of the first embodiment of the present invention will be described with reference to FIGS.

  First, an example of a screen display when face detection is set to OFF in FIG. 2S110 and normal AF (FIG. 3S135) is performed is shown.

  FIG. 9 shows a screen display example of the image display unit 28 when the AF process (S135 in FIG. 3) is performed. A focusing frame 501 is displayed on the image display unit 28 at a position focused by AF processing. In FIG. 9, a focusing frame is displayed as a result of focusing on the larger face.

  In this state, when shooting is performed by changing the angle of view (composition) of the imaging apparatus 100 and normal playback is performed instead of focus confirmation playback (S402 in FIG. 4), the display on the image display unit 28 is as shown in FIG. Compared to FIG. 9, in FIG. 10, the larger face in focus moves to the left.

  When focus confirmation reproduction is performed here (S404 in FIG. 4), the screen display of the image display unit 28 in FIG. 6S1021 is as shown in FIG. The image 621 displays a captured image recorded in the image file, and is an image obtained by compressing and reducing data in the file. An area 623 is an area on the image display unit 28 corresponding to an area cut out from the image data in the file in order to confirm the focus state, and is generated by the system control circuit 50 based on the focusing frame information added to the image file. Is done. The image 622 is an image obtained by cutting out data corresponding to the region 623 from the image data and enlarging it on the image display unit 28. The display of the image 622 makes it easy for the user to determine whether the imaging data corresponding to the region 623 is in focus. However, since the in-focus frame exists near the center of the image as shown in FIG. 9 at the time of AF processing, if the vicinity of the center of the image is enlarged and displayed as it is during playback (S1021 in FIG. 6), the in-focus frame as shown in FIG. An area that is different from the face is enlarged. An area 623 and an enlarged image 622 in FIG. 12 do not correspond to the portion where the imaging apparatus has performed the focusing process, and are not areas where the user should confirm the focus.

  Next, a screen display example when the face detection is set to ON in FIG. 2S110 and the face AF processing (S131 in FIG. 3) is performed is shown.

  As in the case of performing the normal AF process, the screen display in the image display unit 28 during the face AF process (S131 in FIG. 3) is as shown in FIG. Further, the screen display in the image display unit 28 when normal reproduction is performed instead of focus confirmation reproduction (S402 in FIG. 4) is as shown in FIG.

  When focus confirmation reproduction is performed (S404 in FIG. 4), a screen as shown in FIG. 11 is displayed on the image display unit 28 in FIG. 6S1012. The image 521 displays a captured image recorded in the image file, and is an image obtained by compressing and reducing data in the file. An area 523 is an area on the image display unit 28 corresponding to an area cut out from image data in the file in order to confirm the focus state. Among the faces detected by the face detection process of S1002 in FIG. 6, it is determined that the feature amount (for example, the face size) matches or is sufficiently close to the face focused by the face AF process (the larger face in FIG. 9). Is displayed at the position corresponding to the face. The image 622 is an image obtained by cutting out data corresponding to the region 623 from the image data and enlarging it on the image display unit 28. In this case, the position of the face focused in FIGS. 9 and 10 has moved. Even so, the face detection process (S1002 in FIG. 6) is performed on the image in FIG. In addition, even if there are a plurality of faces in the image, the face to be enlarged is determined using the feature amount such as the size of the face, so that the smaller face in FIG. I can do it.

(Example 2)
In the first embodiment, when the face detection setting is turned OFF, it is not possible to correctly enlarge and display the area corresponding to the in-focus portion of the imaging apparatus 100. In the second embodiment, when the face detection process is not performed, the in-focus portion can be correctly displayed in an enlarged manner. The operation of the second embodiment of the present invention will be described with reference to FIGS. 2 to 5, 8, and 7. FIG.

  The flowcharts of the main routine of the image pickup apparatus 100 of the second embodiment are as shown in FIGS. The flow of FIG. 2 is the same as that of FIG. 2 of the first embodiment. FIG. 7 is a diagram corresponding to FIG. 3 of the first embodiment. Since face detection is not performed, S124, S125, S126, S130, and S131 of FIG. 3 are not provided. Further, the feature of the first embodiment is S150 of FIG. 7 with respect to S133 of FIG.

  That is, the system control circuit 50 extracts the feature amount from the image of the in-focus frame portion stored in S136 and stores it in the internal memory of the system control circuit 50 or the memory 52 (S150). This feature amount corresponds to the facial feature amount of the first embodiment. Color, luminance, etc. can be used as the feature amount.

  FIG. 8 shows a detailed flowchart of the focus confirmation reproduction process in S404 of FIG. 4 and S703 of FIG. 5 in the second embodiment. Although the storage location of information to be used is different between S404 (in playback mode) and S703 (in REC review), the processing flow is the same and will be described with the same flowchart.

  The system control circuit 50 uses the focusing frame information stored in the internal memory of the system control circuit 50 or the memory 52 as the additional information of the image file in S404 (during playback mode) and as the additional information of the image file in S703 (during REC review). (S1101). Further, in S404 (during playback mode), conversion is performed using additional information of the image file, and in S703 (during REC review) using AF time information and shooting time information stored in the internal memory of the system control circuit 50 or the memory 52. (S1101). The process of enlarging and reducing the focusing frame size according to the optical zoom magnification and electronic zoom magnification, the amount of movement of the imaging device 100 from AF to shooting, and the position of the focusing frame in the image according to the rotation angle Process to move. When the focusing frame conversion process is completed (S1101), the process proceeds to S1102.

  The system control circuit 50 extracts a feature amount from the portion corresponding to the in-focus frame after being converted in S1101 of the image data (S1102). Information corresponding to the feature quantity stored in S150, such as color and brightness, is obtained as the feature quantity. When the feature amount of the in-focus frame portion is extracted (S1102), the process proceeds to S1103.

  The system control circuit 50 determines whether or not the difference between the feature amount of the in-focus frame portion and the feature amount extracted in S1102 falls within the threshold (S1103). The feature amount of the in-focus frame portion is stored in the internal memory of the system control circuit 50 or the memory 52 as additional information of the image file in S404 (during playback mode) and in S703 (during REC review). .

  The system control circuit 50 uses the image display unit 28 to enlarge and display a portion corresponding to the in-focus frame after the conversion in S1101 among the image data in the image file (S1104).

  If the difference between the feature amounts is greater than or equal to the threshold value in S1103, a warning display is displayed using the image display unit 28 and the display unit 54 that the in-focus portion during AF processing is not present in the captured image (S1105). ), And proceeds to S1106.

  The system control circuit 50 determines whether or not to end the display based on a timeout and a display end instruction from the operation unit 70 (S1106). If the display is not to end, the process returns to S1106. If the display should be terminated, display termination processing by the image display unit 28 and the display unit 54 is performed (S1107), and the focus confirmation processing routines S404 and S703 are terminated. Even when an image forwarding instruction is given by the operation unit 70 or the like, the display is once ended and the reproduced image file is updated.

  Next, a screen display example of the second embodiment of the present invention will be described with reference to FIGS.

  The screen display on the image display unit 28 at the time of AF processing (FIG. 7 S135) and the screen display on the image display unit 28 when normal playback is performed instead of focus confirmation playback (S402 in FIG. 4) are the same as in the first embodiment. 9 and FIG.

  When focus confirmation playback is performed (S404 in FIG. 4), a screen as shown in FIG. 11 is displayed on the image display unit 28 in FIG. 8S1104. The image 521 displays a captured image recorded in the image file, and is an image obtained by compressing and reducing data in the file. An area 523 is an area on the image display unit 28 corresponding to an area cut out from the image data in the file in order to confirm the focus state, and is displayed at the converted in-focus frame position obtained by the frame conversion processing in S1101 of FIG. Is done. The image 622 is an image obtained by cutting out data corresponding to the region 623 from the image data and enlarging it on the image display unit 28. In this case, the position of the face focused in FIGS. 9 and 10 has moved. Nevertheless, by moving the focusing frame to the left by using the amount of movement of the imaging apparatus 100 from the AF processing to the shooting by the focusing frame conversion processing in FIG. 8S1101, the position focused in FIG. Enlarged display is possible.

  Although the second embodiment has been described as not using face detection, it can also be implemented in combination with the first embodiment using face detection. That is, when the face detection setting is valid, the processing as in the first embodiment is performed, and when the face detection setting is invalid, the processing according to the second embodiment is performed. In this case, different information may be used at the time of face detection and at other times as the feature amount of the in-focus portion and the information at the time of AF / shooting. For example, when the face detection setting is valid, the face size, the face position, the eye position, etc. as the feature value of the focused face are optically used as information about the state of the imaging device at the time of AF and shooting. Use zoom magnification and electronic zoom magnification. On the other hand, when the face detection setting is invalid, color, brightness, or the like may be used as the feature amount of the focused portion. In addition, as information regarding the state of the imaging apparatus at the time of AF and photographing, an optical zoom magnification, an electronic zoom magnification, a movement amount using an acceleration / angular velocity detection unit, and a rotation angle may be used. This is particularly effective when the feature amount such as the size and position of the face obtained only when the face is detected is more useful in specifying the in-focus portion.

  The recording media 200 and 210 may be constituted by a memory card such as a PCMCIA card or a compact flash (registered trademark), a hard disk, or the like. In addition, a micro DAT, a magneto-optical disk, an optical disk such as a CD-R or a CD-RW, a phase change optical disk such as a DVD, or the like may be used.

  Further, the recording media 200 and 210 may be a composite medium in which a memory card and a hard disk are integrated, or may be configured to be partly removable from the composite medium.

  In the above embodiment, the recording media 200 and 210 have been described as being separated from the imaging device 100 and can be arbitrarily connected. However, any or all of the recording media are fixed to the imaging device 100. Also good.

  The recording medium 200 or 210 may be connected to the imaging apparatus 100 in an arbitrary number or a plurality of recording media.

  In the above embodiment, the recording medium 200 and 210 are described as being mounted on the imaging apparatus 100. However, the recording medium may have a single or a plurality of combinations.

  Note that the present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device.

  Another object of the present invention is to read and execute a program code stored in a storage medium by a computer (or CPU or MPU) of a system or apparatus. In the program code, a storage medium (or recording medium) that records software codes for realizing the functions of the above-described embodiments is supplied to a system or apparatus. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. In addition, the functions of the above-described embodiments are not only realized by executing the program code read by the computer. For example, an operating system (OS) running on a computer performs part or all of actual processing based on an instruction of the program code, and the functions of the above-described embodiments may be realized by the processing. included.

  Further, the program code read from the storage medium is written in a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. After the writing, the CPU or the like provided in the function expansion card or function expansion unit performs part or all of the actual processing based on the instruction of the program code. Needless to say, the process includes the case where the functions of the above-described embodiments are realized.

  In addition, you may make it this invention combine the above-mentioned each embodiment or those technical elements as needed.

  Further, the present invention may be applied to all or a part of the structure of the claims or the embodiment to form one device or to be combined with another device. It may be an element constituting the apparatus.

It is a block diagram of a configuration of an embodiment of the present invention. It is a part of flowchart of the main routine of one Example of this invention. It is a part of flowchart of the main routine of 1st Example of this invention. It is a flowchart of the image reproduction process routine of one Example of this invention. It is a flowchart of the quick review process routine of one Example of this invention. It is a flowchart of the focus confirmation reproduction | regeneration processing routine of 1st Example of this invention. It is a part of flowchart of the main routine of 2nd Example of this invention. It is a flowchart of the focus confirmation reproduction | regeneration processing routine of the 2nd Example of this invention. It is an example of a display at the time of AF process of one Example of this invention. It is an example of a display at the time of the image reproduction process of one Example of this invention. It is an example of a display at the time of focus confirmation reproduction | regeneration processing of one Example of this invention. It is an example of a display at the time of focus confirmation reproduction | regeneration processing of one Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Shooting lens 12 Shutter 14 Image pick-up element 16 A / D converter 18 Timing generation circuit 20 Image processing circuit 22 Memory control circuit 24 Image display memory 26 D / A converter 28 Image display part 30 Memory 32 Image compression / decompression circuit 40 Exposure Control means 42 Focus control means 44 Zoom control means 46 Barrier control circuit 48 Flash 50 System control circuit 52 Memory 54 Display unit 56 Non-volatile memory 60 Mode dial switch 62 Shutter switch SW1
64 Shutter switch SW2
66 Playback mode switch 68 Attitude mode switch 70 Operation unit 72 Power switch 80 Power control means 82 Connector 84 Connector 86 Power supply means 90 Interface 92 Connector 94 Interface 96 Connector 98 Recording medium attachment / detachment detection means 100 Imaging device 102 Barrier 104 Optical viewfinder 106 Camera attitude Detection means 108 Acceleration / angular velocity detection means 110 Communication means 112 Connector (or antenna)
DESCRIPTION OF SYMBOLS 200 Recording medium 202 Recording part 204 Interface 206 Connector 210 Recording medium 212 Recording part 214 Interface 216 Connector 501 Focus frame 521 Image data reduced display 522 Focused part enlarged display 523 Focused part 621 Image data reduced display 622 Focused part enlarged display 623 Focusing part

Claims (6)

A display control device that controls display of a captured image,
Position detecting means for detecting a subject position in the captured image;
Extraction means for extracting subject information of a captured image before and after imaging processing for image recording;
Conversion means for converting subject information extracted by the extraction means in accordance with a zoom state after extraction;
Control means for controlling to display a partial region of the captured image corresponding to the subject position detected by the position detection means,
When the control unit displays a partial region of the captured image when a plurality of subject positions are detected by the position detection unit, the control unit is prior to the imaging process for image recording extracted by the extraction unit , and A display comprising: selecting a partial area of a captured image to be displayed from a plurality of subject positions detected by the position detecting unit based on subject information converted by the converting unit and subject information extracted after the imaging process. Control device.   The display control apparatus according to claim 1, wherein the state information of the imaging apparatus includes a movement amount or a rotation angle of the imaging apparatus. The object position, the display control device according to claim 1 or 2, characterized in that the position of the face of the subject in the captured image. Wherein, corresponding when the object position is not present, the display control device according to any one of claims 1 to 3, characterized in that a warning display before and after the imaging for recording. Imaging apparatus characterized by having an imaging device for acquiring an image signal to be displayed by the display control device according to any one of claims 1 to 4. A control method of a display control device that controls display of a captured image,
A position detecting step for detecting a subject position in the captured image;
An extraction step of extracting subject information of the captured image before and after the imaging process for image recording;
A conversion step of converting the subject information extracted in the extraction step according to a zoom state after extraction;
A control step for controlling to display a partial region of the captured image corresponding to the subject position detected in the position detection step,
Wherein the control step, a pre-imaging process when, for the extracted image recorded in the extraction step of displaying a partial area of the captured image when a plurality of object position is detected in the position detecting step A partial region of a captured image to be displayed is selected from a plurality of subject positions detected in the position detection step based on the subject information converted in the conversion step and the subject information extracted after the imaging process. Control method.

Images