JP5299034B2 - Imaging device - Google Patents

Description

  The present invention relates to a photographing apparatus.

  Conventionally, in a video camera or a still camera having a function of recording a moving image with sound, the sound is recorded in a state most suitable for the selected shooting mode corresponding to the arbitrary shooting mode selected at the time of shooting. (For example, Patent Document 1).

JP 2000-354190 A

  However, in the conventional photographing apparatus, the photographer has to set the photographing scene of the subject by selecting a photographing mode such as portrait or landscape, which is equipped on the camera before photographing. For this reason, there is a problem that a setting error occurs at the time of setting or the setting work is troublesome.

  The present invention has been made in view of such problems, and it is an object of the present invention to provide an imaging apparatus capable of eliminating a setting error and annoying setting work and recording audio suitable for a captured image.

The present invention solves the above problems by the following means. In addition, in order to make an understanding easy, although the code | symbol corresponding to embodiment of this invention is attached | subjected and demonstrated, it is not limited to this. That is, the invention of claim 1 includes an image acquisition unit (71) that acquires an image, a sound acquisition unit (74) that acquires sound corresponding to the image, and a characteristic for correcting image blur of the image. Acquired by an operation unit (31) that can be operated by the photographer, a recognition unit (58) that recognizes the state in which the image is acquired according to the operation state of the operation unit, and the sound acquisition unit A processing unit (76) for processing the processed voice, a control unit (64) for controlling the processing unit according to a recognition result of the recognition unit, and the image and the voice processed by the processing unit. And a storage unit (79) that stores the information in a storage medium in a corresponding manner.

According to a second aspect of the present invention, there is provided the photographing apparatus according to the first aspect, further comprising a face recognition unit (70) for recognizing whether a human face is reflected in the image. is there.
A third aspect of the present invention is the imaging apparatus according to the first or second aspect , wherein the recognition unit (58) acquires the image by performing a scene analysis using the image. It is an imaging device characterized by recognizing.

A fourth aspect of the present invention is the photographing apparatus according to the third aspect , wherein the photographing unit includes a scene operation unit (25) that can be operated by a photographer according to a photographing scene, and the recognition unit (58) includes the scene. The imaging device recognizes the state where the image is acquired by giving a weight to the scene analyzed by the scene analysis rather than the scene operated by the operation unit (25).

A fifth aspect of the present invention is the imaging apparatus according to any one of the first to fourth aspects, further comprising a detection unit (61, 61a) for detecting a shake of the apparatus, and the control unit (64) is an imaging device characterized in that the processing unit (76) is controlled so as to process the sound according to the shake of the device detected by the detection unit.

Invention of Claim 6 is an imaging device as described in any one of Claim 1-5 , Comprising: The said process part (76) is acquired by the said audio | voice acquisition part (74). The filter (76) changes the gain according to the frequency of the voice, and the control unit (64) controls the characteristic of the filter (76) according to the recognition result of the recognition unit (58). It is an imaging device.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which can record the audio | voice suitable for the image | photographed image can be provided.

It is a perspective view which shows typically the external appearance of the imaging device which concerns on 1st Embodiment. It is a figure which shows the structure of the imaging device which concerns on 1st Embodiment. It is a block diagram which shows the function of the imaging device which concerns on 1st Embodiment. It is a flowchart which shows the example of the scene analysis of a scene recognition part. It is a figure which shows the example of the frequency characteristic of the sound which an average microphone collects. It is a figure which shows the example of the characteristic of a filter. It is a block diagram which shows the function of the imaging device which concerns on 2nd Embodiment. In the imaging device concerning a 2nd embodiment, it is a figure showing the relation between the input state of each signal which is inputted into a control part and memorized by a storage part, and the time, respectively, and (a) is an input of an image signal. State, (b) is the input state of the right microphone audio signal, (c) is the input state of the left microphone audio signal, (d) is the angular velocity signal input state, and (e) is the active mode ON signal input state. Is shown.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a perspective view schematically showing the appearance of the photographing apparatus according to the first embodiment. FIG. 2 is a diagram illustrating the configuration of the photographing apparatus according to the first embodiment.

  The imaging apparatus according to the first embodiment is a digital single-lens reflex camera 1 (hereinafter abbreviated as camera 1), and can capture still images and moving images. The camera 1 includes a body part 4, a finder device part 7 provided on the upper part of the body part 4, and a lens part 10 provided on the front part of the body part 4. The lens unit 10 incorporates a photographing lens 11 composed of a plurality of lens groups. The photographing lens 11 constitutes a photographing optical system. The lens unit 10 is provided with a subject distance detection unit 13 and a focal length detection unit 16.

  Outside the body portion 4, a still image / video switching switch 19, a release button 22, a scene operation unit 25, a camera shake correction switch 28 for operating a camera shake correction mechanism, and a correction capable of changing the characteristics for correcting camera shake. A mode switch 31 is provided. In addition, a microphone 34 is provided on the front surface of the body portion 4, that is, below the surface on the subject side, for recording sound when shooting a moving image. The microphone 34 includes two microphones, a right microphone 34R provided on the right side and a left microphone 34L provided on the left side of the subject (not shown). On the back surface of the body portion 4, a liquid crystal display unit 37 is provided on which an image being shot during moving image shooting and an image captured by the shooting optical system during shooting standby are displayed. The liquid crystal display unit 37 displays the setting state of the camera 1 and various menus during operation. Speakers 40 are provided on both side surfaces of the body portion 4 to output sound when a captured moving image is reproduced. The speaker 40 includes two speakers, a right speaker 40R provided on the right side and a left speaker 40L provided on the left side facing the subject.

  An image sensor 43 for photographing a subject is provided inside the body portion 4. The image sensor 43 is a CCD or a CMOS. In the photographing optical path between the photographing lens 11 and the image sensor 43, a quick return mirror 46 for reflecting light from the subject that has passed through the photographing lens 11 to the finder device unit 7 is provided.

  When photographing a still image, light from the subject before the release passes through the photographing lens 11 and enters the body portion 4, is reflected by the quick return mirror 46, and is provided at a position conjugated with the image sensor 43. An image is formed on the screen 49. The subject image formed on the finder screen 49 is guided to the eyepiece lens 55 via the pentaprism 52 provided in the finder device section 7 and is observed by the photographer. After the release, the quick return mirror 46 jumps up, and the light from the subject that has passed through the photographing lens 11 forms an image on the image sensor 43.

  FIG. 3 is a block diagram illustrating functions of the camera 1 according to the first embodiment.

  Hereinafter, the function of the camera 1 will be described by taking as an example the case of shooting a moving image of a subject (not shown) with the camera 1.

  When the still image / moving image changeover switch 19 is operated to switch the camera 1 to the moving image capturing mode, the quick return mirror 46 jumps up and is held in a state of being retracted from the image capturing optical path, and enters a standby state for capturing moving images. The camera 1 allows the photographer to select a shooting mode suitable for the shooting state. The shooting mode is selected by operating the scene operation unit 25. The scene operation unit 25 is a shooting scene selection dial in which shooting modes such as a portrait shooting mode, a child shooting mode used in an athletic meet, a night view shooting mode, a macro shooting mode, and the like are set. When the photographer selects a photographing scene, the setting states of the photographing lens 11, aperture (not shown) and the like are automatically adjusted so that an image suitable for the selected photographing mode can be photographed. The setting state of the scene operation unit 25 is output to the scene recognition unit 58.

  The camera 1 is provided with a camera shake correction mechanism for reducing image blurring of a photographed image. When operating the camera shake correction mechanism, the camera shake correction switch 28 (hereinafter referred to as the VR switch 28) is turned on. Then, the blur detection unit 61 provided in the scene recognition unit 58 or the blur detection unit 61a provided in the lens unit 10 operates. The shake detection unit 61 detects a shake of the camera 1 by image processing using an image signal input to a scene recognition unit 58 described later. The shake detection unit 61a is a gyro sensor (hereinafter also referred to as a gyro sensor 61a). The shake detection units 61 and 61a detect the shake of the camera 1, that is, the angular velocity when the body part 4 is tilted. The photographing lens 11 incorporates a correction lens (not shown), and the optical axis is moved by moving the correction lens in a direction that cancels the inclination of the body portion 4 in accordance with the angular velocity detected by the shake detection units 61 and 61a. to correct. In the illustrated embodiment, both the shake detection unit 61 and the shake detection unit 61a may be used, or at least one of them may be used. Thereby, the movement of light reaching the light receiving surface of the image sensor 43 is suppressed, and camera shake is reduced.

  Further, the camera 1 according to the present embodiment can change the correction characteristics of the camera shake correction mechanism. The correction characteristic is changed by operating a correction mode changeover switch 31 (hereinafter referred to as a VR mode changeover switch 31). For example, the VR mode change-over switch 31 is switched between an active mode and a normal mode (hereinafter, the active mode is referred to as a VR active mode and the normal mode is referred to as a VR normal mode). The VR active mode is a mode used when a photographer takes a picture on a vehicle such as an automobile or a train. In the VR active mode, camera 1 shake due to vibrations peculiar to a vehicle such as an automobile or a train is detected and corrected. The VR normal mode is a mode used to correct camera shake when the photographer is not on the vehicle. By selecting the VR mode, it is possible to reduce image blur with correction characteristics suitable for the shooting state. The VR mode changeover switch 31 may be a changeover switch when using a tripod / when not using a tripod. The VR mode changeover switch 31 is set such that the VR active on signal is output when the VR mode changeover switch 31 is set to the VR active mode, and the VR active on signal is not output when the VR normal mode is set. The VR active on signal is output to the scene recognition unit 58 and the control unit 64, respectively.

  In a standby state for moving image shooting, light from a subject that has passed through the shooting optical system forms an image on the image sensor 43. An image of the subject captured by the photographing optical system is displayed on the liquid crystal display unit 37 on the back surface of the body unit 4. When the photographer presses the release button 22 halfway, the subject distance detection unit 13 and the focal length detection unit 16 operate to calculate the distance to the subject and the focal length of the subject, and automatically adjust the subject to be in focus. The photographic lens 11 is driven. Thus, the photographer can take an image while the subject is in focus. The distance to the subject and the focal length of the subject calculated by the subject distance detection unit 13 and the focal length detection unit 16 are output to the scene recognition unit 58.

  When the photographer fully presses the release button 22 and the full push switch is turned on, shooting of the moving image is started. When shooting of a moving image is started, an image being recorded is displayed on the liquid crystal display unit 37. Light from the subject imaged on the image sensor 43 is photoelectrically converted by the image sensor 43 and converted into an electrical signal. The electrical signal is subjected to analog processing by an analog processing circuit 67 and then converted to a digital signal by an A / D conversion circuit 68 to generate an image signal. The image signal is subjected to image processing such as white balance adjustment, saturation adjustment, contour adjustment, and gradation adjustment by an image processing circuit 69. An image acquisition unit 71 is configured by the imaging optical system, the image sensor 43, the analog processing circuit 67, the A / D conversion circuit 68, and the image processing circuit 69. The image signal subjected to the image processing is output to the scene recognition unit 58 and the control unit 64, respectively.

  The image processing circuit 69 is provided with a face recognition unit 70 for recognizing a person's face within the angle of view. The face recognition unit 70 calculates the area, number, and position of faces existing within the angle of view, and outputs the calculation result to the scene recognition unit 58 as face recognition information.

  When shooting a moving image, the signals and information described above, that is, the image signal from the image processing circuit 69, the face recognition information from the face recognition unit 70, the setting state of the scene operation unit 25, the setting state of the VR mode changeover switch 31, that is, VR. The active-on signal, the distance information to the subject, and the focal length information of the subject (hereinafter referred to as information such as an image signal) are continuously output, and the information such as the image signal is input to the scene recognition unit 58.

  The scene recognition unit 58 calculates the brightness of the subject, the distance to the subject, and the like based on the input information such as the image signal, and analyzes the scene. The scene recognizing unit 58 recognizes a state where the image is captured based on the scene analysis result. For example, based on information from the face recognition unit 58, it is recognized that a person is a main subject, or if there is a lot of blue information at the top of the angle of view, it is recognized that the image is taken outdoors on a sunny day. To do. When the information such as the input image signal includes the VR active on signal, the scene recognition unit 58 recognizes that the image is an image taken on a vehicle such as an automobile. The shooting state recognized by the scene recognition unit 58 does not necessarily match the shooting mode selected by the photographer using the scene operation unit 25. In this case, the scene recognizing unit 58 recognizes the state where the image is acquired by giving a weight to the scene analysis result. The scene analysis result of the scene recognition unit 58, that is, the scene recognition result is output to the control unit 64.

  FIG. 4 is an example of scene analysis performed by the scene recognition unit 58, and is a flowchart for recognizing a scene where a child is photographed as a result of executing the scene analysis. In this example, the signal and information input to the scene recognition unit 58 are information such as the image signal described above, that is, the image signal from the image processing circuit 69, the face recognition information from the face recognition unit 70, the scene operation unit 25 The setting state, the state of the VR mode setting switch 31 (in this example, the VR mode setting switch 31 is set to the VR normal mode), the distance information to the subject, and the focal length information of the subject.

  The scene recognition unit 58 performs scene analysis using face recognition information, focal length information, and subject distance information among information such as input image signals. The scene recognizing unit 58 acquires face recognition information, focal length information, and subject distance information (step 1 to step 3), and takes into account subject distance information and focal length information, and determines the absolute size of the subject's face from the face recognition information. Is calculated (step 4). An index indicating the absolute size of the face calculated by the scene recognition unit 58 is a face area S, and a parameter for determining the size of the face area S is an adult face area Sadult. The scene recognition unit 58 stores an area Sadult in advance. If Sadult is a threshold, if the absolute face size S calculated by the scene recognition unit 58 is smaller than Sadult (step 5), the scene recognition unit 58 recognizes that the subject is a child (step 6). In addition, since the VR active on signal of the VR mode setting switch 31 is not output, the scene recognizing unit 58 recognizes that the image is captured without being on the vehicle. Even if the setting state of the scene operation unit 25 is set to a mode other than the child shooting mode, the scene recognition unit 58 recognizes that the subject is a child from information such as an image signal of an actually shot image. . That is, the scene recognition unit 58 recognizes a scene with priority given to information such as an image signal of an actually captured image. The recognition result of the scene recognition unit 58 is output to the control unit 64 (step 7).

  In the case of the above-described scene analysis example, for example, when there are a plurality of faces in the angle of view and each size is smaller than Sadult, the scene recognition unit 58 recognizes that the subject is a child. Also, for example, when there are a plurality of faces and each of which has a size larger than Sadult and a size smaller than that, the scene recognition unit 58 determines that the subject includes a child. Further, for example, if a face having a size smaller than Sadult moves around indefinitely, the scene recognizing unit 58 determines that a scene where a child is playing is being photographed.

  Next, a description will be given of a processing function of audio recorded together with an image in moving image shooting.

  When shooting a movie, sound is recorded simultaneously with the image. Usually, the audible range of the human ear is said to be 20 Hz to 2 kHz (however, there are various theories). For this reason, many microphones that collect sound collect much over the audible range. FIG. 5 shows an example of frequency characteristics of sound collected by an average microphone. The microphone 34 in the first embodiment (hereinafter simply referred to as “microphone 34” means the right microphone 34R and the left microphone 34L) also has the frequency characteristics shown in FIG. It covers a lot. The audible range includes various types of sounds. For example, as shown in FIG. 5, the frequency characteristic of the sound of a car is relatively low, and the call of insects such as cicada and bellworm is high. Further, as shown in FIG. 5, the human voice has a wide range including the middle range. The microphone 34 collects these various types of sounds. For example, when a scene where a child plays is photographed, the microphone 34 collects not only the voice of the child but also the voice of an adult nearby and the sound of a car that has passed by.

  Such various types of sound collected by the right microphone 34R and the left microphone 34L together with the captured image are processed by the signal processing circuit 73, respectively. The signal processing circuit 73 includes an A / D conversion circuit (not shown) and a digital amplifier (not shown). The microphone 34 and the signal processing circuit 73 constitute an audio acquisition unit 74. The sound processed by the signal processing circuit 73 is output as a digital sound signal. As the audio signal, an audio signal of the sound collected by the right microphone 34R and an audio signal of the sound collected by the left microphone 34L are respectively output (hereinafter simply referred to as “audio signal”). An audio signal generated from the sound and an audio signal generated from the sound of the left microphone 34L). Note that the audio signal processed by the signal processing circuit 73 may be output as an analog audio signal. When outputting an analog audio signal, the signal processing circuit 73 has an A / D conversion circuit (not shown), a digital amplifier (not shown), and a D / A conversion circuit (not shown).

  In the camera 1 according to the first embodiment, the audio signal processed by the signal processing circuit 73 is processed by the processing unit 76. The processing unit 76 is a filter that changes the gain according to the frequency of the audio signal (hereinafter referred to as the filter 76). The sound signal is processed by such a filter 76 into sound suitable for the state in which the image is captured. For example, when a child is photographed as a main subject, the audio signal is processed by the filter 76 whose processing characteristics are changed so as to attenuate the voice of an adult at the side or the sound of a car passing by nearby. Then, it is possible to record an image together with a voice that emphasizes a child's voice. In general, the basic frequency of an adult male voice is around 100 Hz, and the basic frequency of a child's voice is 200 Hz or more. Therefore, when the audio signal is processed using a filter having a frequency characteristic as shown in FIG. 6, the voice of the adult male is attenuated and only the band of the child's voice can be extracted. As a result, the audio signal is focused on the child's voice. As described above, the camera 1 of the first embodiment adjusts the input audio signal using the filter 76 so that a desired audio band is emphasized, in other words, unnecessary audio is not recorded. . In the first embodiment, since the audio signal is digital, the filter 76 is also a digital filter. When outputting an audio signal in analog, an analog filter may be used.

  The control unit 64 controls how the characteristics of the filter 76 are set for the audio signal generated by the signal processing circuit 73. The control unit 64 receives an analysis result obtained by the scene recognition unit 58 performing scene analysis using information such as an image signal, that is, a scene recognition result regarding a state where an image is captured. Based on the input scene recognition result, the control unit 64 changes the processing characteristics of the filter 76 so that the sound most suitable for the state in which the image is captured is recorded. That is, the gain of the filter 76 is changed. For example, when the scene recognizing unit 58 recognizes that the image is a scene in which a child is playing, the control unit 64 performs processing with processing characteristics as shown in FIG. 6 based on the recognition result. Judge as the most suitable for the image. Then, the gain of the filter 76 is changed so as to have the processing characteristics shown in FIG. The audio signal is processed by a filter 76 having processing characteristics shown in FIG. As a result, the voice band of the adult male is cut from the voice signal, and a voice signal in which the voice band of the child is emphasized is generated. Further, for example, when the information such as the input image signal includes a VR active on signal, the scene recognition unit 58 recognizes that the image is an image taken on a vehicle such as an automobile. The control unit 64 changes the processing characteristics of the filter 76 based on the recognition result of the scene recognition unit 58. That is, the processing characteristics of the filter 76 are changed so as to reduce engine noise, tire running noise, wind noise, etc. that resonate inside the vehicle. If the audio signal is processed by the filter 76 having such characteristics, the engine sound of the automobile is reduced, and an image in which conversation in the vehicle is emphasized can be recorded.

  The image signal of the captured image is stored in the storage unit 79 in association with the audio signal processed by the filter 76. The storage unit 79 may be an external storage medium such as a memory card.

  In the first embodiment, the control unit 64 changes the processing characteristics of the filter 76 depending on whether or not the scene recognition result of the scene recognition unit 58 is a person as a main subject. For example, if a scene recognition result indicating that the image is a person as a main subject is input, the processing characteristics are changed to reduce the sound of a car or the sound of insects. Further, the control unit 64 changes the processing characteristics of the filter 76 depending on whether or not the scene recognition result includes a child in the main subject. If the main subject includes a child, the processing characteristics are changed to those shown in FIG.

  As described above, the camera 1 according to the first embodiment analyzes the captured image, recognizes the state of the captured image, and records the sound most suitable for the captured image. Adjust the gain. The control unit 64 that adjusts the gain of the filter 76 processes the audio signal by changing the filter 76 to have characteristics such as a high-pass filter, a low-pass filter, or a band-pass filter based on the scene recognition result of the scene recognition unit 58. . As a result, sound in a state most suitable for the photographed image is recorded.

  Here, for example, even if the photographer is shooting a scene where a child is playing in a state where the setting of the scene operation unit 25 is set to the portrait shooting mode, the scene recognizing unit 58 receives from the face recognizing unit 70. If it is determined from the face recognition information that the subject is a child, and the recognized face is moving around, the result of the scene analysis is that the image is captured as an image of a scene where the child is playing. To do. Then, based on the recognition result of the scene recognition unit 58, the control unit 64 changes the processing characteristics of the filter 76 so that processing that emphasizes the child's voice band can be performed. That is, the scene recognizing unit 58 recognizes the scene with emphasis on the actually captured image. As a result, even if the photographer makes a setting mistake or forgets to set the scene operation unit 25, the sound signal processing most suitable for the photographed image can be performed.

  When the scene recognition unit 58 performs scene analysis during image shooting and the analysis result is different from the setting state of the scene operation unit 25, the photographer is notified that the setting of the scene operation unit 25 is not suitable. You may make it do. The notification may be displayed on the liquid crystal display unit 37 so as to prompt the user to change the setting of the scene operation unit 25, or the alarm lamp may be turned on or blinked. In addition, control may be performed so as to switch to a shooting mode suitable for a state in which shooting is forcibly performed.

  In addition, when changing the gain of the filter 76 in accordance with the frequency of the audio signal, the control unit 64 may control to change the gain of the filter 76 uniformly for the sounds collected by the left and right microphones 34R and 34L. Alternatively, the volume of the sound collected by the right microphone 34R and the volume of the sound collected by the left microphone 34L may be controlled to be different. For example, when the child of the main subject moves from right to left within the angle of view and further moves from left to right, the gain of the filter 76 is changed so that the volume of the microphone on which the child is photographed is increased. . By controlling in this way, an image is recorded together with a sound in which the movement of a child moving around within the angle of view is emphasized. When the image and sound recorded in this way are reproduced, the sense of reality increases.

  Further, in the camera 1 according to the first embodiment, when the VR switch 28 is on, the shake of the body part 4 detected by the shake detection units 61 and 61a, that is, the angular velocity is detected regardless of the setting state of the VR mode changeover switch 31. Is input to the control unit 64. The control unit 64 controls the characteristics of the filter 76 so as to process the audio signal based on the input blur period of the body unit 4.

  For example, when an angular velocity of A is detected from the blur detection units 61 and 61a, the control unit 64 changes the characteristics of the filter 76 so that the gain of the filter 76 is multiplied by kA (k is an arbitrary constant). When the angular velocity B is detected from the blur detection units 61 and 61a, the control unit 64 changes the characteristics of the filter 76 so that the gain of the filter 76 is multiplied by kB (k is an arbitrary constant). In this way, by changing the characteristics of the filter 76 according to the angular velocity detected by the blur detection units 61 and 61a, when the photographer pans in accordance with the movement of the subject, the sound is changed and recorded in accordance with the panning speed. be able to. If this function is used, for example, in motor sports, when the camera 1 is mounted on a car and shot, the angular speed detected by the shake detection units 61 and 61a changes when the car turns a curve, so the car changes its direction. By changing the gain of the filter 76 to kA or kB times according to the change in angular velocity every time (turning a curve), the sound can be changed and recorded. When the image and sound recorded in this way are reproduced, the sound changes according to the movement of the automobile, and a sense of reality can be obtained. As another example, when the photographer pans from right to left, the volume of the sound collected by the right microphone 34R gradually decreases, and the volume of the sound collected by the left microphone 34L gradually increases. Therefore, the sound may be recorded by controlling the characteristics of the filter 76 so as to be recorded. When the image and sound recorded in this way are reproduced, the volume of the right speaker 40R gradually decreases and the volume of the left speaker 40L gradually increases as the screen pans from right to left. If the sound changes in accordance with the panning direction of the screen during playback, the sense of presence can be increased.

  As described above, according to the camera 1 of the first embodiment, it is possible to record the sound most suitable for the state in which the image is captured based on the scene analysis of the captured image, and further, the movement of the camera 1 and the left and right By linking the sound signals collected by the microphones 34R and 34L, it is possible to record realistic sounds.

(Second Embodiment)
Next, a photographing apparatus according to the second embodiment will be described. The photographing apparatus according to the second embodiment is also a digital single-lens reflex camera 100 (hereinafter abbreviated as “camera 100”), and can capture still images and moving images. In addition, about the structure similar to 1st Embodiment, it demonstrates using the same code | symbol (refer FIG. 1 and FIG. 2).

  FIG. 7 is a block diagram illustrating a functional configuration of the camera 100 according to the second embodiment.

  The configuration of the camera 100 according to the second embodiment is substantially the same as that of the camera 1 according to the first embodiment. However, the camera 100 includes a detachable interchangeable lens unit 82, and the interchangeable lens unit 82 has a camera shake for reducing image blur. A VR switch 28 and a VR mode changeover switch 31 for operating the correction mechanism and the camera shake correction mechanism are provided. The VR mode is a VR active mode and a VR normal mode, as in the first embodiment. The interchangeable lens unit 82 is provided with a subject distance detection unit 13 and a focal length detection unit 16. The interchangeable lens unit 82 includes a lens-side control unit 85, subject distance information from the subject distance detection unit 13, focal length information from the focal length detection unit 16, VR switch on / off information, and a VR mode switch. 31 setting state information is input.

  A still image / moving image changeover switch 19, a release button 22, and a scene operation unit 25 are provided outside the body unit 4. The body unit 4 is provided with a body side communication unit 88, and always communicates with the lens side communication unit 85. The subject distance information from the subject distance detection unit 13, the focal length information from the focal length detection unit 16, the ON / OFF information of the VR switch 28, and the setting state information of the VR mode changeover switch 31 are transmitted to the body unit 4 side through this communication. Has been sent to. Other configurations, that is, the image pickup device, the quick return mirror 46, the microphone 34, the liquid crystal display unit 37 on the back surface of the body unit 4, and the like are the same as those in the first embodiment.

  Hereinafter, the function of the camera 100 according to the second embodiment will be described taking an example of shooting a moving image of a subject.

  When the photographer fully presses the release button 22 from the half-pressed state of the release button 22, that is, in the standby state for moving image shooting, and operates the scene operation unit 25 to start moving image shooting, as in the first embodiment, Light from the subject imaged on the image sensor 43 is photoelectrically converted by the image sensor 43 and converted into an electrical signal. The electrical signal is subjected to analog processing by an analog processing circuit 67 and then converted to a digital signal by an A / D conversion circuit 68 to generate an image signal. The image signal is subjected to image processing such as white balance adjustment by the image processing circuit 69 and is output to the scene recognition unit 58 and the control unit 64, respectively.

  When the VR switch 28 is on, the blur detection units 61 and 61a are operated, and the blur of the body part 4 detected by the blur detection units 61 and 61a, that is, the angular velocity of the body unit 4 is input to the control unit 64. However, in the following description, the case where only the blur detection unit 61a is used will be described in detail.

  Information on the setting state of the VR mode changeover switch 31 is input to the scene recognition unit 58 and the control unit 64. The information on the setting state of the VR mode changeover switch 31 is a VR active on signal that is output when the VR mode changeover switch 31 is set to the VR active mode, and is not output when it is set to the VR normal mode.

  In addition, the face recognition unit 70 included in the image processing circuit 69 calculates the area, number, position, and the like of the face within the angle of view as in the first embodiment, and the calculation result is sent to the scene recognition unit 58 as face recognition information. Is output.

  As in the first embodiment, the scene recognition unit 58 includes information such as an image signal, that is, an image signal from the image processing circuit 69, face recognition information from the face recognition unit 70, setting state of the scene operation unit 25, VR mode. Information on the setting state of the changeover switch 31, that is, VR active-on signal, distance information to the subject, and focal length information of the subject are input. The scene recognizing unit 58 performs scene analysis based on the input information such as an image signal, and recognizes a state where the image is captured based on the scene analysis result. The scene recognition result of the scene recognition unit 58 is output to the control unit 64.

  In the camera 100 according to the second embodiment, sounds collected by the right microphone 34R and the left microphone 34L are converted into audio signals by the signal processing circuit 73 and input to the control unit 64, respectively. The audio signal input to the control unit 64 is an audio signal including all bands of various sounds collected by the respective microphones 34R and 34L (hereinafter, simply referred to as “the microphone 34”, the right microphone 34R and the left microphone 34R). This refers to the microphone 34L, and the term “audio signal” refers to an audio signal generated from the sound of the right microphone 34R and an audio signal generated from the sound of the left microphone 34L.)

  As described above, the control unit 64 includes the image signal from the image processing circuit 69, the scene recognition result of the scene recognition unit 58, the audio signal, the VR active on signal, and the angular velocity of the body unit 4 detected by the blur detection unit 61a. Is entered. The image signal is output from the image processing circuit 69 and input to the control unit 64 when the photographer fully presses the release button 22 and the full-press switch is turned on, and image capturing is started. When the release button 22 is fully pressed again and the full-press switch is turned off, no image signal is output from the image processing circuit 69. Therefore, the image signal is not input to the control unit 64, and image shooting is terminated.

  The control unit 64 measures the elapsed time from the start of shooting to the end of shooting, that is, from when the release button 22 is fully pressed in the moving image shooting mode to when the release button 22 is fully pressed again and the full press switch is turned off. ing. That is, the image signal and the elapsed time during which the image signal is input are linked. The control unit 64 further associates the input scene recognition result, the audio signal, the VR active on signal, and the angular velocity of the body unit 4 with the elapsed time from the start of shooting to the end of shooting. That is, the scene recognition result, the audio signal, the VR active on signal, and the angular velocity of the body unit 4 are each associated with the elapsed time of image signal input. As a result, the image signal, the scene recognition result, the audio signal, the VR active on signal, and the angular velocity of the body unit 4 are associated with each other according to the elapsed time of the image signal input. In this way, by associating the input signal and information with the elapsed time of image signal input, the control unit 64, for example, from which time point to which point in time from the start of image capture to the end of image capture enters the VR active mode. It can be ascertained whether it has been set or has not been set to the VR active mode.

  These signals and information associated with each other are stored in the storage unit 79. At this time, the audio signal associated with the elapsed time of image signal input is stored in the audio file 91 of the storage unit 79. The audio signal stored in the audio file 91 is an audio signal including all bands of various sounds collected by the microphone 34. The VR active on signal associated with the elapsed time of image signal input is stored in a correction mode storage file 94 (hereinafter referred to as a VR mode storage file 94) of the storage unit. The image signal associated with the elapsed time of image signal input, the scene recognition result, and the angular velocity of the body unit 4 are stored in another file 97 of the storage unit. One or more other files 97 may be provided. Further, the audio signal and the VR active on signal may be stored in the same file. The storage unit 79 may be an external storage medium such as a memory card.

  8A to 8E are graphs showing the relationship between the input state of each signal and time for each signal input to the control unit 64 and stored in the storage unit 79. FIG. However, the scene recognition result is not shown.

  8A, S1 in FIG. 8A is an image signal from the image processing circuit 69, S2 in FIG. 8B is an audio signal from the right microphone 34R, and S3 in FIG. 8C is an audio signal from the left microphone 34L. , (D) S4 indicates an angular velocity signal from the gyro sensor 61a, and (e) S5 indicates a VR active-on signal. The VR active on signal S5 is transmitted when the VR active mode is set.

  As shown in each drawing of FIG. 8, the image signal S1 is turned on at time t1. That is, the release button 22 is fully pressed at time t1, and image capturing is started. After the start of image capturing, the signal strength of the audio signal S2 from the right microphone 34R, the audio signal S3 from the left microphone 34L, and the angular velocity signal S4 change with time (time t2, t3, t4). At the beginning of shooting, the VR mode changeover switch 31 is set to the VR normal mode, and the gyro sensor 61a detects the angular velocity of the body part due to camera shake. The VR active on signal is not output. Here, for example, it is assumed that the photographer gets into the car in a state where shooting is continued, and the VR mode switch 31 is switched from the VR normal mode to the VR active mode at time t5. Then, the VR active-on signal S5 is input to the control unit 64. When shooting is performed in the vehicle of the traveling vehicle, the gyro sensor 61a detects a shake of the camera 100 due to vibrations peculiar to the traveling vehicle, and a shake angular velocity signal S4 is input to the control unit 64. In addition, engine sound, wind noise, and the like of the automobile are collected by the right microphone 34R and the left microphone 34L. After that, it is assumed that the photographer gets out of the automobile in a state where photographing is continued and the VR mode changeover switch 31 is changed from the VR active mode to the VR normal mode at time t6. Then, the VR active on signal S5 is turned off. The shake of the camera 100 due to the vibration unique to the automobile is not detected, and the engine sound or the like of the automobile is not collected. Then, the photographer fully presses the release button 22 again at time t7, and the photographing is finished.

  When shooting is performed in the above-described state, the images from time t1 to t7 are all the signals from S1 to S5 and the scene recognition result from the scene recognition unit 58 are all input image signals S1. In association with time, the image data is stored in the storage unit 79 as one piece of image data.

  Next, a case where the image data stored in the storage unit 79 is reproduced will be described.

  The reproduced image of the image data stored in the storage unit 79 can be viewed on the liquid crystal display unit 37 on the back of the body unit 4, and the reproduced audio can be heard through the speaker 40 on the side of the body unit 4. Alternatively, a personal computer can be connected to an external output terminal (not shown) provided in the body portion 4 and viewed on a monitor of the personal computer.

  The operator operates an operation button (not shown) or the like to select image data to be reproduced and gives an instruction for reproduction. At this time, the operator can select whether to process and reproduce the sound reproduction by operating an operation button or the like (not shown), or to reproduce without processing.

  First, a case will be described in which the operator selects the method of reproducing without processing the voice.

  When the operator selects the one to reproduce without processing the audio, the control unit 64 causes the audio signal 91, the VR mode storage file 94, and the other file 97 in the storage unit 79 to receive an audio signal, a VR active on signal, an image signal, A signal and information such as a scene recognition result and an angular velocity of the body part 4 are called. These signals and information called from the storage unit 79 are input to a reproduction signal processing unit 99 provided in the control unit 64. The reproduction signal processing unit 99 generates an image and audio reproduction signal from these input signals and information.

  For example, in the case of reproducing the image data from time t1 to t7 in FIG. 8 described above, the reproduction signal processing unit 99 generates an image reproduction signal from the image signal S1, and the audio signal, that is, the audio from the right microphone 34R. An audio reproduction signal is generated from the signal S2 and the audio signal S3 of the left microphone 34L. At this time, the control unit 64 does not consider the VR active on signal, the scene recognition result, and the angular velocity of the body unit 4. Then, the reproduction signal processing unit 99 generates an audio reproduction signal without processing the audio signal. That is, the audio reproduction signal is generated without applying the processing by the filter 76 as in the first embodiment to the audio signal. When the image and sound reproduction signals generated in this way are reproduced, the portion of the image taken while riding in the car attenuates the engine sound of the car, the running sound of the tire, the wind noise, etc. It is played without When the audio signal is reproduced without being processed in this way, it is emphasized that the image is taken while riding in a car, and the atmosphere at the time of shooting is not impaired. In addition to automobiles, when a train window is photographed on a train, it can be reproduced without impairing the atmosphere at the time of photographing.

  Next, a case will be described in which the operator selects a method of processing and playing back audio.

  When the operator selects one to process and reproduce the sound and gives an instruction to reproduce, the control unit 64 transmits the sound signal, VR active from the sound file 91, the VR mode storage file 94 and the other file 97 in the storage unit 79, respectively. Signals and information such as an ON signal, an image signal, a scene recognition result, and an angular velocity of the body unit 4 are called. These signals and information called from the storage unit 79 are input to a reproduction signal processing unit 99 provided in the control unit 64. The reproduction signal processing unit 99 generates an image and audio reproduction signal from these input signals and information. The reproduction signal processing unit 99 generates an image reproduction signal from the image signal S1. Further, the reproduction signal processing unit 99 generates an audio reproduction signal from the input audio signal and outputs it to the processing unit 76. The processing unit 76 is a filter that changes the gain according to the frequency of the audio signal, as in the first embodiment. Here, the control unit 64 refers to other signals included in the image data. Specifically, the scene recognition result from the scene recognition unit 58, the angular velocity signal S4 from the gyro sensor 61a, and the ON signal S5 in the VR active mode. The control unit 64 changes the characteristics of the filter 76 based on the scene recognition result as in the first embodiment. The audio reproduction signal input to the filter 76 is processed and output so as to be reproduced with audio most suitable for the captured image.

  For example, it is assumed that a scene recognition result for a child as a subject is included in the image data. Then, the control unit 64 changes the characteristics of the filter 76 so that the voice of the child is emphasized in the portion corresponding to the scene where the child is the subject in the elapsed time of the image signal S1. Then, an image in which the child's voice is emphasized is reproduced for a scene in which the child is a subject among the reproduced images. Further, if the VR active on signal S5 is stored in the image data, the control unit 64 determines the sound of the vehicle for the portion corresponding to the portion in which the VR active on signal S5 is stored in the elapsed time of the image signal S1. The characteristic of the filter 76 is changed so as to reduce. If it does so, about the part into which VR active on signal S5 is inputted among reproduction images, the engine sound and wind noise of a car will be reduced, and the picture which emphasized the conversation etc. in a car will be reproduced. The control unit 64 appropriately changes the characteristics of the filter 76 based on the scene recognition result for other portions of the elapsed time of the image signal S1.

  Further, the volume of the audio reproduction signal generated from the audio signals S2 and S3 of the left and right microphones 34R and 34L may be changed based on the angular velocity signal S4 from the gyro sensor 61a included in the image data. Then, the left and right sounds change according to the panning of the screen, and it can be reproduced with a sense of reality.

  As described above, the camera 100 according to the second embodiment stores the entire bandwidth of the sound collected when the image is captured. Then, when reproducing the acquired image, a pattern to be reproduced without processing, and a pattern to be reproduced with sound processed based on the scene recognition result of the scene recognition unit 58, the VR active on signal, and the angular velocity signal, etc. And can be played back. With such a configuration, it is possible to change the sound reproduction state according to the preference of the person viewing the image when reproducing the image.

  As described above, the first embodiment and the second embodiment have been described using the digital single-lens reflex camera as an example. However, these embodiments are not limited to the digital single-lens reflex camera, and can be applied to an imaging apparatus having a function of recording sound. For example, the present invention can be applied to a still camera, a video camera, a mobile phone with a built-in camera, and the like.

  The configuration of the present invention is not limited to the first and second embodiments, and can be changed as appropriate.

DESCRIPTION OF SYMBOLS 1,100 Camera 4 Body part 7 Finder apparatus part 10 Lens part 11 Shooting lens 13 Subject distance detection part 16 Focal length detection part 19 Still image / video switching switch 22 Release button 25 Scene operation part 28 Camera shake correction switch 31 Correction mode switching Switch 34 Microphone 37 Liquid crystal display unit 40 Speaker 43 Image sensor 46 Quick return mirror 49 Viewfinder screen 52 Penta prism 55 Eyepiece 58 Scene recognition unit 61, 61a Motion detection unit 64 Control unit 67 Analog processing circuit 70 Face recognition unit 71 Image acquisition unit 73 Signal Processing Circuit 74 Audio Acquisition Unit 76 Processing Unit (Filter)
79 Storage unit 82 Interchangeable lens unit 85 Lens side control unit 88 Body side communication unit 91 Audio file 94 Correction mode storage file (VR mode storage file)
97 Other files 99 Playback signal processor

Claims (6)

An image acquisition unit for acquiring images;
An audio acquisition unit that acquires audio corresponding to the image;
An operation unit that can be operated by a photographer to change characteristics for correcting image blur of the image;
A recognition unit for recognizing a state in which the image is acquired according to an operation state of the operation unit;
A processing unit for processing the voice acquired by the voice acquisition unit;
A control unit that controls the processing unit according to a recognition result of the recognition unit;
And a storage unit that stores the image and the sound processed by the processing unit in a storage medium in association with each other. An imaging apparatus according to claim 1,
An imaging apparatus, further comprising a face recognition unit for recognizing whether a human face is reflected in the image. The imaging device according to claim 1 or 2, wherein
The recognizing unit recognizes a state in which the image is acquired by performing scene analysis using the image. An imaging apparatus according to claim 3, wherein
Includes a scene operation unit that can be operated by the photographer according to the shooting scene,
The imaging apparatus recognizes a state where the image is acquired by giving a weight to a scene analyzed by the scene analysis rather than a scene operated by the scene operation unit. An imaging apparatus according to any one of claims 1 to 4 , wherein
It has a detection unit that detects the shake of the device,
The said control part controls the said process part so that the said audio | voice may be processed according to the blurring of the said apparatus detected by the said detection part, The imaging device characterized by the above-mentioned. An imaging apparatus according to any one of claims 1 to 5 , wherein
The processing unit is a filter that changes a gain according to the frequency of the sound acquired by the sound acquisition unit,
The control unit controls the characteristics of the filter according to a recognition result of the recognition unit.

Images