JP2011250340A - Imaging apparatus and control method of same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus in which convenience with respect to the start of composition of images captured by a plurality of imaging means.SOLUTION: In response to the input of a speech from a photographer side to a sub microphone 211, an imaging apparatus 100 starts, at the time point, composing an image captured with a sub camera with an image captured with a main camera, displaying and recording the composed image.

Description

  The present invention relates to an imaging apparatus, and more particularly to an imaging apparatus including a plurality of imaging units and a control method thereof.

  Conventionally, an apparatus including a plurality of imaging units has been disclosed. For example, in Patent Document 1 (Japanese Patent Application Laid-Open No. 2005-80117), a photographer himself is imaged with one imaging means of a pair of imaging means, a subject is imaged with the other imaging means, and both A technique for synthesizing an image captured in (1) is disclosed. Note that the composition here refers to extracting the photographer by removing the background image from the image captured by one imaging means, and fitting the extracted image into the image captured by the other imaging means, Done. By doing so, it is possible to perform photography with high entertainment.

JP 2005-80117 A

  However, in order to synthesize the images as described above, it is necessary to make a setting such as “two-shot shooting mode” in the imaging apparatus. For example, the photographer described above during imaging with one imaging means When it is desired to synthesize such images, the shooting operation is stopped and an operation for setting is required, which is complicated.

  The present invention has been conceived in view of such circumstances, and an object of the present invention is to improve convenience in the start of synthesis of images captured by a plurality of imaging units in an imaging apparatus.

  An imaging apparatus according to an aspect of the present invention includes a first imaging unit that captures an image of a first subject, a second imaging unit that captures an image of a second subject, and a sound collection device that collects sound emitted from the first subject side. Recording on the recording medium in a manner in which the video imaged by the first imaging means and the video imaged by the second imaging means are synthesized due to the sound being collected by the means and the sound collecting means Means.

  In the imaging apparatus of the present invention, the sound collecting means includes a microphone that can be held in a direction facing the first subject, and the recording means is caused by the sound collected from the first subject by the microphone. It is preferable to record on the recording medium in such a manner that the video imaged by the first imaging unit is combined with the video imaged by the second imaging unit.

  Moreover, it is preferable that the imaging device of the present invention further includes a display unit that displays a video obtained by combining the video captured by the first imaging unit and the video captured by the second imaging unit.

  In the imaging apparatus of the present invention, the recording means captures an image captured by the first imaging means from the time when the sound collecting means collects the sound emitted from the first subject. It is preferable to start recording in the mode of synthesis.

  In the imaging apparatus of the present invention, the recording means uses the second imaging means to capture the video imaged by the first imaging means from a certain time before the time when the sound collecting means collects the sound emitted from the first subject. It is preferable to start recording in such a manner that it is combined with the captured image.

  In the image pickup apparatus of the present invention, the recording means outputs the second image captured by the first image pickup means when a predetermined time has elapsed since the sound collection means picked up the sound emitted from the first subject. It is preferable to stop the recording in a mode of combining with the video imaged by the imaging means.

  An imaging apparatus according to another aspect of the present invention includes a first imaging unit that images a first subject, a first recording unit that records a first video imaged by the first imaging unit, and a second recording unit. Second imaging means for imaging the subject, second recording means for recording the second video imaged by the second imaging means on a recording medium, sound collection means for collecting sound emitted from the first subject side, Due to the fact that the sound is collected by the sound collecting means, third recording means for recording information for reproduction in a mode of combining the first video and the second video, and the first video based on the information, And a reproducing means for reproducing the second video in a mode of synthesizing the second video.

  According to still another aspect of the present invention, there is provided a method for controlling an imaging apparatus, comprising: a first imaging unit that images a first subject; a second imaging unit that images a second subject; and a sound emitted from the first subject side. A method of controlling an image pickup apparatus including a sound collecting means for collecting sound, wherein the sound collecting means determines whether or not the sound collecting means has collected sound, and the sound collecting means determines that sound has been collected. And a step of recording on the recording medium in a manner to synthesize the video imaged by the first imaging unit and the video imaged by the second imaging unit.

  According to another aspect of the present invention, there is provided a method for controlling an imaging apparatus, comprising: a first imaging unit that images a first subject; a second imaging unit that images a second subject; and a sound emitted from the first subject side. An image pickup apparatus control method comprising: a sound collecting means for sounding, the method comprising: recording a first video imaged by the first imaging means on a recording medium; and a second video imaged by the second imaging means. Recording on the recording medium, recording information for reproduction in a manner of combining the first video and the second video due to the sound collected by the sound collecting means, and based on the information And a step of reproducing the first video and the second video in a combined mode.

  According to the present invention, in the imaging apparatus, a combination of videos captured by a plurality of imaging units is collected to collect sound emitted from the subject side of one imaging unit among the plurality of imaging units. It can be started due to the sound means collecting the sound.

  Thereby, the photographer can record the video imaged by the plurality of imaging means without stopping the photographing operation and performing a special operation, and the convenience of the imaging device can be improved. .

1 is an external view of an imaging apparatus that is a first embodiment of the present invention. 1 is an external view of an imaging apparatus that is a first embodiment of the present invention. It is a block diagram which shows the outline of a structure of the imaging device of FIG. It is a figure which shows the flow of the signal in the imaging device of FIG. It is a figure for demonstrating the synthesis | combination of the image | video of the main camera and the image | video of a subcamera in the imaging device of FIG. It is a flowchart of the synthetic | combination video recording process performed in the imaging device of FIG. It is a figure for demonstrating the production | generation aspect of the synthetic | combination image | video in the imaging device of FIG. It is a figure for demonstrating the production | generation aspect of the synthesized image in the imaging device which is the 2nd Embodiment of this invention. It is a block diagram which shows the outline of a structure of the imaging device which is the 3rd Embodiment of this invention. 10 is a flowchart of reproduction information registration processing executed in the imaging apparatus of FIG. It is a figure for demonstrating the content of the information registered regarding the image | video of a subcamera in the imaging device of FIG. 10 is a flowchart of a reproduction process that is executed in the imaging apparatus of FIG. 9. It is a figure which shows an example of the guidance screen displayed in the imaging device of FIG. It is an external view of the modification of the imaging device of the 1st Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the same or corresponding parts in the drawings are denoted by the same reference numerals and description thereof will not be repeated.

[First Embodiment]
(1. Overall configuration of imaging apparatus)
1 and 2 are external views of an image pickup apparatus 100 according to the first embodiment of the present invention. FIG. 1 is an external view of the imaging apparatus 100 as viewed from the lens side. FIG. 2 is an external view of the imaging apparatus 100 as viewed from the operation panel side.

  The imaging apparatus 100 may be a digital movie camera or a mobile communication terminal having a video camera function.

  The imaging apparatus 100 mainly includes a main body unit including an optical system 151 such as a lens and a monitor unit unit including a display unit 140. A power button 132 for switching power on or off is provided on the side surface of the main body that the monitor unit contacts when the monitor unit is folded into the main body. A zoom range button 131 for switching the zoom range to a wide angle or telephoto is provided on the operation panel side on the upper surface of the main body.

  The operation panel includes a zoom switch 133 for gradually zooming toward the wide-angle side or the telephoto side, a photo button 134 for capturing a still image, a movie button 135 for starting or stopping recording of a moving image, a shooting mode or playback. REC / PLAY button 136 for switching to a mode, menu button 137 for switching to various setting modes, and a set button for performing a selection operation by moving up, down, left and right in various settings and a determination operation by pressing 138 is provided.

  The optical system 151 and an imaging element 161 described later constitute a camera device for imaging a subject (second subject) and constitute a second imaging unit.

  A main microphone 181 is provided on the surface of the monitor unit that faces the subject when the monitor unit is in an open state. The main microphone 181 has good sensitivity on the subject side so as to mainly collect sound on the subject side.

  A display unit 140, an optical system 221, and a sub microphone 211 are provided on a surface of the monitor unit unit that faces the photographer when the monitor unit unit is open. The optical system 221 and an image sensor 231 to be described later mainly constitute a camera device for imaging a photographer (first subject), and constitute a first imaging means. In the opened state, the monitor unit unit holds the sub microphone 211 in a direction facing the photographer. The sub microphone 211 has good sensitivity on the photographer side so as to mainly collect the sound on the photographer side. In the present embodiment, the sub microphone 211 constitutes sound collecting means.

  In the present specification, hereinafter, the optical system 221 and the image sensor 231 constituting the first imaging unit may be collectively referred to as a “sub camera”. Further, the optical system 151 and the image sensor 161 constituting the second image pickup unit may be collectively referred to as “main camera”.

  In the monitor unit section, as is mainly understood from FIG. 2, the display section 140 is the main body of the imaging apparatus 100 in a state in which imaging is generally performed such that an operation button such as a movie button 135 is positioned above. The optical system 221 and the sub microphone 211 are provided on the left side of the display unit 140. Thus, when the photographer captures the image of the imaging apparatus 100 with the right hand, the optical system 221 can more reliably capture an image of the photographer's face and the like, and the sub-microphone 211 emits a voice from the photographer. Sound can be collected.

  In the imaging device 100, the display unit 140, the optical system 221 and the sub microphone 211 are provided in a monitor unit unit that can be folded with respect to the main body of the imaging device 100. However, these are not necessarily members that can be folded with respect to the main body. It does not have to be provided. That is, it is not an indispensable constituent requirement that the monitor unit can be folded in the imaging apparatus 100.

(2. Block configuration of imaging device)
FIG. 3 is a block diagram showing an outline of the configuration of the imaging apparatus 100 according to this embodiment. FIG. 4 is a diagram showing a signal flow of the imaging apparatus 100 in this embodiment. In FIG. 4, a solid line among arrows indicating flow indicates a signal related to an image, and a broken line indicates a signal related to control.

  3 and 4, the imaging apparatus 100 includes a control unit 110, a storage unit 120, an operation unit 130, a display unit 140, an optical system control unit 150, optical systems 151 and 221, and imaging. Element control units 160 and 230, image sensors 161 and 231, video processing units 170 and 240, AD (Analog Digital) conversion units 171 and 241, audio processing units 180 and 210, a main microphone 181, and a sub microphone 211. And a speaker 182, compression / decompression processing units 190 and 250, and a synthesis processing unit 260.

  The operation unit 130 transmits operation signals from the power button 132, the zoom range button 131, and various buttons on the operation panel described with reference to FIG.

  The storage unit 120 includes a non-volatile memory such as a ROM (Read Only Memory) (for example, a flash memory), a volatile memory such as a RAM (Random Access Memory) (for example, a SDRAM (synchronous Dynamic Random Access Memory)), and a memory. A memory card slot for reading / writing data of the card 121 (for example, an SD memory card and a USB memory) is included.

  Note that the storage unit 120 may include other internal storage devices such as a hard disk drive in addition to or instead of the semiconductor memory such as the nonvolatile memory and the volatile memory. In addition to or instead of the memory card slot, the storage unit 120 also stores data in other external storage devices, for example, recording media such as optical discs (eg, DVD (Digital Versatile Disc) and BD (Blu-ray Disc)). A device for reading and writing may be included.

  The storage unit 120 stores data of a program for controlling the imaging apparatus 100, data for controlling the imaging apparatus 100, setting data for setting various functions of the imaging apparatus 100, and images captured by the imaging apparatus 100. Record still image or video data.

  The storage unit 120 is used as a work memory when a program is executed and a buffer memory when an imaging process is performed.

  The control unit 110 includes a CPU (Central Processing Unit), and in accordance with an operation signal from the operation unit 130 according to a program for controlling the imaging device 100 recorded in the storage unit 120, the storage unit 120 and the display unit 140. The optical system control unit 150, the image sensor control units 160 and 230, the video processing units 170 and 240, the audio processing units 180 and 210, the compression / decompression processing units 190 and 250, and the synthesis processing unit 260 are controlled.

  The optical system 151 includes a plurality of lenses, a diaphragm, and an actuator for changing the position of the lens and the opening of the diaphragm. The optical system 151 guides light incident from an external subject or its background to the light receiving surface of the image sensor 161.

  The actuator of the optical system 151 adjusts the position of the lens and the aperture of the diaphragm according to a control signal from the optical system control unit 150, thereby forming an image formed on the light receiving surface of the image sensor 161 within a predetermined range. The zoom magnification is changed, the focus on the light receiving surface is adjusted, and the amount of light incident on the light receiving surface is limited.

  The optical system control unit 150 controls the actuator of the optical system 151 in accordance with a control signal from the control unit 110, and the zoom magnification and focus of the image formed on the image sensor 161 and the amount of light incident on the image sensor 161. To change.

  For example, when the zoom switch 133 is operated to the telephoto side or the wide angle side, the operation unit 130 transmits an operation signal to that effect to the control unit 110, respectively. When the zoom control unit 111 configured in the control unit 110 by executing a program by the CPU of the control unit 110 receives an operation signal for zooming to the telephoto side or the wide angle side, the zoom magnification of the optical zoom is set. A control signal for gradually increasing or decreasing is transmitted to the optical system control unit 150.

  When the control signal for gradually increasing the zoom magnification of the optical zoom is transmitted from the control unit 110, the optical system control unit 150 is a control signal for gradually shifting the angle of view of the optical system 151 to the telephoto side. Is transmitted to the actuator of the optical system 151, and a control signal for gradually decreasing the zoom magnification is transmitted, a control signal for gradually shifting the angle of view of the optical system 151 to the wide angle side is transmitted to the optical system 151. It transmits to the actuator of the system 151. The actuator of the optical system 151 moves the position of the lens so that the angle of view of the optical system 151 is gradually shifted to the telephoto side or the wide angle side in accordance with a control signal from the optical system control unit 150.

  The optical system 221 includes a plurality of lenses, and guides light incident from the outside to the light receiving surface of the image sensor 231 via the lenses.

  The image pickup devices 161 and 231 are area image sensors each having a light receiving surface in which a plurality of light receiving portions each including one photodiode that detects light and generates charges are arranged in two dimensions. (Charge Coupled Device) An image sensor. The imaging elements 161 and 231 may be CMOS (Complementary Metal Oxide Semiconductor) image sensors.

  The image pickup devices 161 and 231 are controlled by the image pickup device control units 160 and 230, and images formed by the light incident from the optical systems 151 and 221 are formed on the light receiving surfaces. It accumulates as electric charge by photoelectric conversion according to light and dark, sequentially converts the amount of electric charge for each light receiving unit into voltage according to the drive pulse from the image sensor control unit 160, and outputs the change in the voltage as an electric signal. .

  The AD conversion units 171 and 241 are controlled by the image sensor control units 160 and 230, respectively, and quantize analog electrical signals from the image sensors 161 and 231 to convert them into digital image signals.

  The imaging element control units 160 and 230 supply clock signals to the imaging elements 161 and 231 and the AD conversion units 171 and 241 and switch the driving modes of the imaging elements 161 and 231 in accordance with a control signal from the control unit 110. .

  For example, when the zoom range button 131 is operated, the operation unit 130 transmits an operation signal to that effect to the control unit 110. When the drive mode control unit 112 configured in the control unit 110 by executing a program by the CPU of the control unit 110 receives an operation signal indicating that the zoom range button 131 has been operated, the drive mode control unit 112 sets the drive mode of the image sensor 161. A control signal for switching is transmitted to the image sensor control unit 160. When the image sensor control unit 160 receives a control signal for switching the drive mode of the image sensor 161, the image sensor control unit 160 switches the drive mode of the image sensor 161. The image sensor 161 outputs an electrical signal according to the switched drive mode.

  The video processing units 170 and 240 convert the digitized Bayer pattern image signals from the AD conversion units 171 and 241 into RGB signals in accordance with control signals from the control unit 110, and tone correction processing such as gamma correction. Or converting RGB signals into YCbCr signals, or temporarily recording YCbCr signals in the video buffers 122 and 124 of the storage unit 120, respectively. In addition, the video processing units 170 and 240 may perform other processing for correcting the image.

  The main microphone 181 and the sub microphone 211 convert an external sound into an electric signal indicated by a change in voltage and transmit it to the sound processing units 180 and 210, respectively.

  The audio processing units 180 and 210 quantize the analog electrical signals from the main microphone 181 and the sub microphone 211 in accordance with the control signal from the control unit 110, convert them into digital audio signals, and record them in the buffer of the storage unit 120. Let Also, the audio processing units 180 and 210 convert the audio signal expanded by the compression / decompression processing units 190 and 250 into an analog electric signal according to the control signal from the control unit 110 and output the analog electric signal to the speaker 182.

  The speaker 182 converts the electrical signal output from the sound processing units 180 and 210 into sound and outputs the sound to the outside. Note that the speaker 182 includes a mixer that appropriately synthesizes the electrical signals output from the audio processing unit 180 and the audio processing unit 210.

  The compression / decompression processing units 190 and 250 read the YCbCr signals recorded in the video buffers 122 and 124 of the storage unit 120 in accordance with the control signal from the control unit 110, and appropriately combine them in the synthesis processing unit 260, thereby buffering the storage unit 120. Are read out, the read YCbCr signal and the audio signal are compressed, converted into video data in the MPEG-4 (Moving Picture Experts Group phase 4) format, and the video data is written into the write buffer 123 of the storage unit 120. To record.

  The control unit 110 causes the memory card 121 to record the video data recorded in the write buffer 123 of the storage unit 120. In the present embodiment, the storage unit 120 that records the composite video on the memory card 121 constitutes a recording unit.

  For example, when the movie button 135 is operated, the operation unit 130 transmits an operation signal to that effect to the control unit 110. When the imaging start / stop control unit 113 configured in the control unit 110 by executing a program by the CPU of the control unit 110 receives an operation signal indicating that the movie button 135 has been operated, if the video is being captured, Then, control for stopping the imaging is performed, and if the moving image is not being captured, the control for starting the imaging is performed.

  Specifically, as a control for starting imaging, a control signal is transmitted to the imaging element control units 160 and 230 so that the imaging element outputs an electrical signal corresponding to the imaging setting, and processing for imaging is performed. The video processing units 170 and 240, the compression / decompression processing units 190 and 250, the composition processing unit 260, and the buffers of the storage unit 120 are controlled to start.

  The display unit 140 includes a display device (for example, an LCD (Liquid Crystal Display)), and is recorded on the video processed by the video processing unit 170 or the memory card 121 of the storage unit 120 in accordance with a control signal from the control unit. Video, display indicating the state of the imaging apparatus 100 overlaid on the video (for example, remaining imaging time, remaining battery level, etc.), and various menu screens (for example, screens for setting various setting values) Display a screen for selecting an image).

(3. Video composition)
The imaging apparatus 100 according to the present embodiment records video captured by the sub camera with the main camera due to the fact that the sub microphone 211 collects sound while recording video captured by the main camera. Record and synthesize the captured video. This will be specifically described below with reference to FIG.

  FIG. 5A shows a state where the photographer PN1 is capturing an image of the visual field AR1 of the main camera including the subjects PN2 and PN3 with the imaging device 100. In this state, an image displayed on the display unit 140 and recorded in the storage unit 120 is shown in FIG. FIG. 5B shows a screen 900 on which the subjects PN2 and PN3 are shown.

  Here, when the photographer PN1 speaks, the voice is collected by the sub microphone 211. When sound collection by the sub microphone 211 is detected, a video image captured by the sub camera is combined with a video image captured by the main camera, displayed on the display unit 140, and recorded in the storage unit 120. In this state, an image displayed on the display unit 140 and recorded in the storage unit 120 is shown in FIG. FIG. 5C shows a screen 910 in which what is shown as the screen 900, the video, and the video of the field of view AR2 of the sub camera including the photographer PN1 are combined. On the screen 910, the video imaged by the sub camera (video image of the visual field AR2) is synthesized as an image in the frame 911 in a state that can be distinguished from other parts.

(4. Composite video recording process)
FIG. 6 is a flowchart of a process (synthesized video recording process) executed by the control unit 110 when the imaging apparatus 100 combines the video of the main camera and the video of the sub camera as described above. The composite video recording process is executed when moving image recording is being executed in the imaging apparatus 100.

  Referring to FIG. 6, in the composite video recording process, control unit 110 first determines in step S10 whether sub-microphone 211 has started sound collection, and if it determines that it has started, the process proceeds to step S20. Proceed.

  Note that, in step S10, the control unit 110, for example, when the volume collected by the sub microphone 211 is larger than the volume collected by the main microphone 181 or when the volume collected by the sub microphone 211 is larger than the main microphone. It is determined that the sub microphone 211 has collected sound when 181 is smaller than the collected sound volume, and the latter sound volume minus the former sound volume is equal to or less than a predetermined amount. Therefore, it is determined that the sub microphone 211 is not collecting sound.

  In step S20, as described with reference to FIG. 5C, the control unit 110 combines the video captured by the main camera and the video captured by the sub camera (hereinafter referred to as “composited video” as appropriate). ”) And the process proceeds to step S30. Note that the storage unit 120 stores in advance how to synthesize two images (for example, the fitting position of the image captured by the sub camera with respect to the image captured by the main camera and the size of the screen to be inserted). The setting content may be recorded, or may be based on information input via the operation unit 130 and registered in the storage unit 120.

  In step S30, the control unit 110 records the composite video generated in step S20 in the storage unit 120, and advances the process to step S40.

  In step S40, the control unit 110 determines whether or not the sub microphone 211 stops collecting sound. When it is determined that the sub microphone 211 stops collecting, the process proceeds to step S50, and when it is determined that sound collection is continued, the process proceeds to step S20. return.

  In step S50, the composite video generation is terminated, and the process returns to step S10. As a result, the video displayed on the display unit 140 and recorded in the storage unit 120 is only the video captured by the main camera as described with reference to FIG.

  In the composite video storage processing described above, in response to the sub microphone 211 collecting the sound emitted from the photographer, the video captured by the sub camera from that time is synthesized with the video captured by the main camera. Displayed and recorded. As a result, when the photographer speaks out the contents explaining the image captured by the main camera, the photographer who explains such an image is displayed and recorded together with the image of the main camera. The Rukoto.

  In this case, when the images of the main camera and the sub camera are combined, the sound collected by the main microphone 181 and the sub microphone 211 is also combined, output, and recorded.

  Here, in the present invention, the synthesis instruction trigger is not the button operation but the sound collection by the sound collection means. For this reason, the photographer is not required to perform complicated operations, and even when a compositing instruction is input while imaging with the main camera is continued, noise due to button operation is It is possible to avoid being placed in the sound of the main microphone 181.

  In the imaging apparatus 100, as shown in FIG. 7A, the imaging by the sub camera may be performed during a period during which a moving image is recorded, that is, during a period during which the main camera is imaging. However, as shown in FIG. 7B, it may be set only when a composite video is generated.

  In FIGS. 7A and 7B, video data captured by the main camera and the sub camera is schematically shown by a frame. The hatched portion corresponds to the composite video data. 7A and 7B, the time when the sound collection by the sub microphone 211 is started is indicated by “voice input ON”, and the time when the sound collection by the sub microphone 211 is ended is “voice input OFF”. It is shown in

  In the example shown in FIG. 7A, for example, before the sub microphone 211 starts collecting sound, such as when the sub camera starts imaging simultaneously with the start of imaging of the main camera, Imaging is being performed. When the sub microphone 211 starts collecting sound, the video of the main camera is combined with the video of the sub camera until the sound collection ends. When the recording operation is being performed, the main camera video (see FIG. 5B) is basically recorded in the storage unit 120 as the recorded video. Then, during the period in which the composite video is generated, the video from the main camera is replaced with the composite video and recorded in the storage unit 120 as a recorded video. Note that the sub-camera video for a period during which no composite video is generated is temporarily recorded in the video buffer 124 and then discarded as appropriate.

  In the example shown in FIG. 7B, imaging by the sub camera is started in response to detection of sound collection by the sub microphone 211, and when the sound collection ends, imaging by the sub camera is terminated. Then, the captured video of the sub camera is synthesized with the video of the main camera and recorded in the storage unit 120 as a recorded video.

  In the present embodiment, as shown in FIG. 7B, the imaging of the video that is not used for the generation of the synthesized video can be suppressed as much as possible by setting the imaging of the sub camera only during the synthesized video generation period. This has the advantage of reducing power consumption.

  On the other hand, according to the example shown in FIG. 7A, when sound collection by the sub microphone 211 is started, a composite video can be generated from a time point that is a predetermined time after that point.

  Further, in the present embodiment, the generation of the composite video is finished when the sound collection by the sub microphone 211 is finished, but when a specific time has passed since the end of the sound collection was detected (for example, for 3 seconds). ) May end the generation of the composite video.

  In addition, even if the generation of the composite video is ended on the condition that a predetermined time (for example, 3 seconds) has elapsed since the start of the sound collection, regardless of whether the sound collection by the sub microphone 211 is started or not. Good.

  Once the sound collection by the sub microphone 211 is detected, the generation of the composite video may be continued until an operation for stopping the recording is performed (regardless of whether or not the voice input is finished).

  That is, in the imaging apparatus 100 according to the present embodiment, if the generation start timing of the composite video corresponds to the start of the sound collection by the sub microphone 211, the start of the sound collection does not coincide with the generation start timing. In addition, the generation end timing may not coincide with the sound collection end timing.

[Second Embodiment]
The imaging apparatus according to the present embodiment is different from the imaging apparatus 100 according to the first embodiment only in the recording mode of video data when a composite video is generated.

  Specifically, as shown in FIGS. 8A and 8B, in this embodiment, when the synthesized video is generated, video data of the synthesized video is captured by the main camera. Separately from the video data and the video data picked up by the sub camera, it is recorded in the storage unit 120. That is, in this embodiment, the video imaged by the main camera is recorded on the memory card 121, and the video data of the synthesized video for the same recording time as the video image by the main camera is further recorded on the memory card 121. . The memory card 121 may further record video data captured by the sub camera. Note that FIG. 8A shows a case where imaging is performed by the sub camera during a period other than the generation period of the composite video, as in FIG. 7A, and FIG. 8B is a diagram in FIG. In the same manner as in FIG. 5, the case where imaging by the sub camera is performed only during the period in which the composite video is generated is shown.

  Start time information and end time information are attached to the video data of the composite video described above. The start time information is information representing a reproduction position when sound collection by the sub microphone 211 is started, and the end time information is information representing a reproduction position when sound collection by the sub microphone 211 is lost.

  In the composite video, the video from the time when the sound collection by the sub microphone 211 is started (audio input ON) to the time when the sound collection ends (audio input OFF) is an image obtained by combining the video of the main camera and the video of the sub camera. The other video is a video of the main camera (that is, a video obtained by copying the video of the main camera).

  By recording the composite video in such a manner, when the video data corresponding to the composite video is played back, the composite video is played back at the playback position corresponding to the period from the start time information to the end time information. Only the video imaged by the main camera is reproduced at the reproduction position corresponding to the other period. On the other hand, if the video data corresponding to the “main camera” is reproduced, only the video imaged by the main camera is reproduced even during the period from the start time information to the end time information.

  In the present embodiment, the storage unit 120 that records the composite video on the memory card 121 constitutes a recording unit. Note that the recording means of the present embodiment also records the video of the main camera.

  In the present embodiment, only the video data corresponding to the period from the start time information to the end time information may be recorded as the composite video. In this case, when the video data of the main camera is being played back, the video data of the composite image is played back only during the period in which the playback position is specified by the end time information from the start time information, so that FIG. The same video as that when the composite video recorded in the manner shown in FIG. 8B is played back is played back.

  In the present embodiment, the video from the sub camera may be recorded in the storage unit 120 or may not be recorded. Further, as the video of the sub camera, only the video of the period other than the period combined with the main camera may be recorded, or only the video of the period may be recorded.

[Third Embodiment]
(1. Overall configuration)
FIG. 9 shows a block diagram of an image pickup apparatus 100A that is the third embodiment of the image pickup apparatus of the present invention.

  The imaging device 100A is synthesized with the video of the main camera on the video of the sub camera instead of generating the synthesized video during the period when the imaging device 100 was generating the synthesized video in the first or second embodiment. Information for playback. For this reason, as illustrated in FIG. 9, in the imaging device 100 </ b> A, the synthesis processing unit 260 included in the imaging device 100 is omitted. Note that the configuration of the imaging apparatus 100A can be the same as that of the imaging apparatus 100 except that the composition processing unit 260 is not provided.

(2. Playback information registration process)
FIG. 10 is a flowchart of the process (reproduction information registration process) executed to add information to be synthesized and reproduced with the video of the main camera to the video of the sub camera as described above.

  Referring to FIG. 10, in the reproduction information registration process, control unit 110 first determines in step SA10 whether or not sub microphone 211 has collected sound and determines that sound has been collected. The process proceeds to step SA20.

  In step SA20, control unit 110 performs processing for registering information indicating the start position to be reproduced by being combined with the video of the main camera with respect to the video of the sub camera at that time, and goes to step SA30. Proceed with the process.

  In the present embodiment, for example, a combined information table shown in the following Table 1 can be given as information for combining and reproducing the video of the sub camera with the video of the main camera.

  The composite information table can specify the generation location of the composite video a plurality of times in the reproduction of one recorded data. In the composite information table, a serial number is assigned as an ID for each playback position of the composite video. Each ID has an associated start position and end position. In the present embodiment, the video of the sub camera is recorded in the storage unit 120 at least from the start position to the end position registered in association with each ID. When the recorded data is reproduced, the composite video as described with reference to FIG. 5C is reproduced from the start position to the end position of each ID, and for other periods, Only the main camera video is generated.

  In step SA20, a new ID is defined in the composite information table, and the recording position at that time is registered as the start position for the new ID.

  In step SA30, control unit 110 determines whether or not there is no sound collection by sub microphone 211. If it is determined that there is no sound collection, control unit 110 proceeds to step SA40.

  In step SA40, control unit 110 registers the recording time at that time as the end position for the ID whose start position is registered in step S20 in the composite information table, and returns the process to step SA10.

  According to the reproduction information registration process described above, as shown in FIG. 11, the start position is registered in the composite information table for the video of the sub camera corresponding to the position where the sound collection by the sub microphone 211 is started. The end position is registered corresponding to the time when the sound collection by 211 ends.

(3. Playback processing)
When a recorded video recorded in the storage unit 120 is played back in the imaging apparatus 100A according to the present embodiment, a composite video playback process executed in the background is described with reference to FIG. 12 which is a flowchart of the process. explain.

  Referring to FIG. 12, in the composite video playback process, first in step SB10, control unit 110 determines whether or not the playback position of the recorded video matches the start position registered in the composite information table. If it is determined that they match, the process proceeds to step SB20.

  In step SB20, the control unit 110 generates a synthesized video by synthesizing the video of the main camera and the video of the sub camera as a recorded video in a synthesis mode registered in advance in the storage unit 120, and displays the synthesized video. By outputting to the display device such as the unit 140, the synthesized video is reproduced, and the process proceeds to step SB30.

  In step SB30, the control unit 110 determines whether or not the playback position of the recorded video matches the position registered as the end position of the composite information table. If the control section 110 determines that they do not match yet, the process proceeds to step SB20. Go back and continue playing the composite video. On the other hand, if it is determined that the playback position matches the end position, the process returns to step SB10 without returning to step SB20. As a result, the playback of the composite video is terminated, that is, the video to be played back is not the composite video but the video of the main camera.

  As a result of the composite video playback process described above, the video of the hatched part of the sub camera, that is, the video from the position where the start position is registered to the position where the end position is registered in the composite information table is synthesized. The video is combined with the video from the main camera and played.

(4. Example of playback guidance display)
FIG. 13 shows an example of a guidance screen that is displayed when the recorded video recorded according to the reproduction information registration process described in the present embodiment is reproduced.

  Referring to FIG. 13, guidance screen 990 includes images 991 and 992 indicating positions where reproduction of the composite video is started for a certain recorded video recorded together with the composite information table in accordance with the reproduction information registration process. The image 991 can be, for example, a thumbnail image of a still image of the main camera at each position. In the guidance screen 990, each of the images 991 and 992 can be displayed together with information indicating the corresponding reproduction position (“XX: XX: XX” and “YY: YY: YY” in FIG. 13).

  When an operation for selecting a recorded video from the storage unit 120 is performed on the operation unit 130 and an operation for instructing to display a guidance screen is performed on the operation unit 130, the control unit 110 is registered in the synthesis table. The start position of each ID obtained is obtained, thumbnail images for the video of the main camera at those positions are generated, and the generated thumbnail image is fitted together with the start position in the frame of the screen that has been generated in advance. Information for displaying the guidance screen 990 is generated.

[Modifications, etc.]
In the first to third embodiments described above, the microphone (main microphone 181) provided on the subject side and the microphone (sub microphone 211) provided on the photographer side, that is, a plurality of microphones are provided. It was done. Then, in response to the timing when sound is collected by the latter microphone, a composite video is generated, or information for displaying the video of the sub camera as a composite video is registered.

  Instead of providing microphones at a plurality of locations, as shown in FIG. 14, a microphone 211A that collects sounds on both the subject side and the photographer side is provided, and among the sounds input to the microphone 211A, The composite video may be generated and information may be registered in response to the timing when it is determined that the sound emitted from the photographer has been collected. Such a microphone 211A is realized, for example, by combining a microphone having directivity on the subject side and a microphone having directivity on the photographer side.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims. In addition, the technical ideas described in the embodiments may be implemented singly or in combination as much as possible.

  100, 100A imaging device, 110 control unit, 120 storage unit, 121 memory card, 122, 124 video buffer, 123 write buffer, 130 operation unit, 132 power button, 140 display unit, 160, 230 image sensor control unit, 161 , 231 Image sensor, 170, 240 Video processing unit, 171, 241 AD conversion unit, 180, 210 Audio processing unit, 181 Main microphone, 182 Speaker, 190, 250 Compression / decompression processing unit, 211 Sub microphone.

Claims (9)

First imaging means for imaging a first subject;
Second imaging means for imaging a second subject;
Sound collecting means for collecting sound emitted from the first subject side;
Recording that is recorded on a recording medium in a manner in which the video imaged by the first imaging unit and the video imaged by the second imaging unit are synthesized due to the sound collected by the sound collecting unit And an imaging device. The sound collecting means includes a microphone that can be held in a direction facing the first subject,
The recording unit synthesizes the video imaged by the first imaging unit due to the sound collected from the first subject by the microphone with the video imaged by the second imaging unit. The imaging apparatus according to claim 1, wherein recording is performed on the recording medium.   The imaging apparatus according to claim 1, further comprising a display unit configured to display a video obtained by synthesizing the video captured by the first imaging unit and the video captured by the second imaging unit.   The recording unit synthesizes the video imaged by the first imaging unit with the video imaged by the second imaging unit from the time when the sound collecting unit collects the sound emitted from the first subject. The imaging device according to any one of claims 1 to 3, wherein recording is started.   The recording means captures an image captured by the first imaging means from a predetermined time before the time when the sound collecting means collects the sound emitted from the first subject. The imaging device according to any one of claims 1 to 3, wherein recording is started in a mode of combining with the imaging device.   The recording means picks up an image picked up by the first image pick-up means by the second image pick-up means when a predetermined time has elapsed from the time when the sound pick-up means picks up the sound emitted from the first subject. The imaging device according to claim 1, wherein recording in a mode of being combined with the recorded video is stopped. First imaging means for imaging a first subject;
First recording means for recording a first video imaged by the first imaging means on a recording medium;
Second imaging means for imaging a second subject;
Second recording means for recording a second video imaged by the second imaging means on a recording medium;
Sound collecting means for collecting sound emitted from the first subject side;
Third recording means for recording information for reproduction in a mode of combining the first video and the second video due to the sound being collected by the sound collecting means;
An imaging apparatus comprising: a reproducing unit that reproduces the first video and the second video based on the information. A control method of an imaging apparatus comprising: a first imaging unit that images a first subject; a second imaging unit that images a second subject; and a sound collecting unit that collects sound emitted from the first subject side. There,
Determining whether the sound collecting means has collected sound;
A step of recording on a recording medium in a manner to synthesize the video imaged by the first imaging unit and the video imaged by the second imaging unit when it is determined that the sound collecting unit has collected audio; An imaging apparatus control method comprising: A control method of an imaging apparatus comprising: a first imaging unit that images a first subject; a second imaging unit that images a second subject; and a sound collecting unit that collects sound emitted from the first subject side. There,
Recording a first video imaged by the first imaging means on a recording medium;
Recording a second video imaged by the second imaging means on a recording medium;
Recording information for reproduction in a mode of combining the first video and the second video due to the sound collected by the sound collecting means;
A control method for an imaging apparatus, comprising: reproducing the first video and the second video based on the information.

Images