JP2011250439A - Imaging apparatus, recording method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize an imaging apparatus and a program for improving the accuracy of reproduction of sound recorded while sound is reproduced.
When a CPU 10 includes a subtraction processing unit 31 and performs moving image shooting with sound while reproducing sound, the sound data to be reproduced is emitted from the built-in speaker 26 and collected by the built-in microphone 21. The sound data being reproduced is subtracted from the sound data, and the sound data after the subtraction is recorded in the flash memory 13. As a result, even if the sound is played back during shooting and recording, the recorded sound will not erase the sound of the subject by the played back sound, and the playback accuracy of the sound that was recorded while the sound was played back Can be improved.
[Selection] Figure 2

Description

  The present invention relates to an imaging apparatus and a program thereof, and more particularly to an imaging apparatus having a recording function and a program thereof.

  In recent years, in an imaging apparatus such as an electronic camera, a technology having a function of recording and photographing a subject (photographing with sound) (Patent Document 1) and a function of reproducing sound data such as music has appeared (patents). Reference 2).

JP Patent Publication No. 2007-036427 Japanese Patent Laid-Open No. 2002-176578

  However, in the conventional technology, when shooting a subject with sound while reproducing music or the like, the recorded subject's voice (sound) is drowned out by the reproduced sound, and the subject's voice is often improved. It cannot be recorded, and when the recorded sound is reproduced, it becomes difficult to hear.

  Therefore, the present invention has been made in view of such a conventional problem, and an object of the present invention is to provide an imaging apparatus that improves the accuracy of reproduction of sound recorded while the sound is reproduced, and a program therefor.

In order to achieve the above object, an image pickup apparatus according to the first aspect of the present invention includes a speaker that emits sound,
Sound emission control means for emitting sound data from the speaker;
A microphone that collects the sound,
Recording control means for recording sound data collected by the microphone in a recording means;
While the sound data collected by the microphone is being recorded on the recording means by the recording control means, the sound data emitted by the sound emission control means is the sound data collected by the microphone. Subtracting means for subtracting from,
It is provided with.

For example, as described in claim 2, the subtracting unit includes:
Performing subtraction before sound data collected by the microphone is recorded in the recording means;
The recording control means includes
The sound data subtracted by the subtracting means may be recorded on the recording means.

For example, as described in claim 3, the subtracting unit includes:
Subtraction may be performed after recording the sound data collected by the microphone by the recording means.

Further, for example, as described in claim 4, a playback unit that plays back the sound data recorded by the recording control unit is provided,
The subtracting means is
When reproducing the sound data recorded in the recording means, subtraction is performed,
The reproducing means includes
The sound data subtracted by the subtracting means may be reproduced.

Further, for example, as described in claim 5, the apparatus includes a generation unit that generates sound data having a phase opposite to that of sound data emitted from the speaker.
The speaker is
A first speaker directed to the subject side and a second speaker directed to the photographer side;
The sound emission control means includes
The sound data to be emitted from the first speaker may be emitted, and the opposite phase sound data generated by the generating unit may be emitted from the second speaker.

In addition, for example, as described in claim 6, voice recognition means for performing voice recognition on the sound data collected by the microphone,
The subtracting means is
While the sound data collected by the microphone is being recorded on the recording means by the recording control means, ambient sounds other than the voice are collected by the microphone based on the recognition result by the voice recognition means. You may make it subtract from sound data.

For example, as described in claim 7, the subtraction unit includes:
Out of peripheral sounds other than sound, sound data other than sound data emitted by the sound emission control means is subtracted from sound data collected by the microphone, and sound emitted by the sound emission control means Data may also be subtracted from sound data collected by the microphone.

Further, for example, as described in claim 8, voice recognition means for performing voice recognition on the sound data collected by the microphone,
The subtracting means is
While the sound data collected by the microphone is recorded on the recording means by the recording control means, voice data having a high sound level is collected by the microphone based on the recognition result by the voice recognition means. Subtraction may be made from the recorded sound data.

In order to achieve the above object, a program according to a ninth aspect of the invention is a program for executing an imaging apparatus including an imaging unit that images a subject, a speaker that emits sound, and a microphone that collects sound. A program,
A sound emission control process for emitting sound data from the speaker;
A recording control process in which image data captured by the imaging unit and sound data collected by the microphone are associated and recorded on a recording medium;
While the sound data collected by the microphone is recorded on the recording medium by the recording control process, the sound data emitted by the sound emission control process is the sound data collected by the microphone. Subtraction processing to subtract from
It is characterized by including.

  According to the present invention, it is possible to improve the reproduction accuracy of the sound recorded while the sound is reproduced.

1 is a block diagram of a digital camera according to an embodiment of the present invention. It is a block diagram for demonstrating the function of the audio | voice processing part of this invention, and CPU10. It is a block diagram for demonstrating the function of the audio | voice processing part of this invention, and CPU10. It is a flowchart which shows operation | movement in the case of subtracting the sound data reproduced | regenerated at the time of recording. It is a flowchart which shows the operation | movement of sound operation | movement imaging | photography when subtracting at the time of reproduction | regeneration. It is a flowchart which shows operation | movement for the operation | movement reproduction | regeneration with sound in the case of subtracting at the time of reproduction | regeneration. It is a figure for demonstrating the function of the audio | voice processing part 15 and CPU10 in 2nd Embodiment. It is a flowchart which shows operation | movement of the digital camera 1 of 2nd Embodiment. It is a figure for demonstrating the function of the audio | voice process part 15 and CPU10 in 3rd Embodiment. It is a flowchart which shows operation | movement of the digital camera 1 of 3rd Embodiment.

Hereinafter, the present embodiment will be described in detail with reference to the drawings as an example in which the imaging apparatus of the present invention is applied to a digital camera.
[First embodiment]
A. Configuration of Digital Camera FIG. 1 is a block diagram showing a schematic electrical configuration of a digital camera 1 that implements the imaging apparatus of the present invention.
The digital camera 1 includes a photographing lens 2, a lens driving block 3, an aperture 4, a CCD 5, a driver 6, a TG (timing generator) 7, a unit circuit 8, a memory 9, a CPU 10, a DRAM 11, an image display unit 12, a flash memory 13, and a key. An input unit 14, an audio processing unit 15, and a bus 16 are provided.

  The photographing lens 2 includes a focus lens, a zoom lens, and the like that are constituted by a plurality of lens groups (not shown). A lens driving block 3 is connected to the photographing lens 2. The lens driving block 3 includes a focus motor and a zoom motor that drive the focus lens and the zoom lens along the optical axis direction, respectively, and a focus motor driver that drives the focus motor and the zoom motor according to a control signal sent from the CPU 10. And a zoom motor driver (not shown).

The diaphragm 4 includes a drive circuit (not shown), and the drive circuit operates the diaphragm 4 in accordance with a control signal sent from the CPU 10.
The diaphragm 4 is a mechanism that controls the amount of light that enters from the photographing lens 2.

  The CCD 5 is driven by the driver 6 and photoelectrically converts the intensity of light of each color of the RGB value of the subject image for every fixed period and outputs it to the unit circuit 8 as an imaging signal. The operation timing of the driver 6 and the unit circuit 8 is controlled by the CPU 10 via the TG 7. The CCD 5 has a Bayer color filter and also functions as an electronic shutter. The shutter speed of the electronic shutter is controlled by the CPU 10 via the driver 6 and TG7.

  A TG 7 is connected to the unit circuit 8, a CDS (Correlated Double Sampling) circuit that holds the imaged signal output from the CCD 5 by correlated double sampling, and an AGC that performs automatic gain adjustment of the imaged signal after the sampling. An (Automatic Gain Control) circuit and an A / D converter that converts an analog image pickup signal after the automatic gain adjustment into a digital signal, the image pickup signal output from the CCD 5 passes through the unit circuit 8 as a digital signal. It is sent to CPU10.

The CPU 10 stores the image data sent from the unit circuit 8 and the sound data collected by the built-in microphone of the sound processing unit 15 in the buffer memory (DRAM 11), and also performs gamma correction on the stored image data. It has functions for performing image processing such as interpolation processing, white balance processing, luminance color difference signal (YUV data) generation processing, and image data and sound data recording processing to the flash memory 13, and controls each part of the digital camera 1. It is a one-chip microcomputer.
In particular, the CPU 10 includes a subtracting unit that subtracts sound data reproduced at the time of recording from recorded sound data.

The memory 9 stores a control program and necessary data necessary for the CPU 10 to control each unit, and the CPU 10 operates according to the program.
The DRAM 11 is used as a buffer memory for temporarily storing image data picked up by the CCD 5 and then sent to the CPU 10 and sound data collected by the built-in microphone of the sound processing unit 15, and as a working memory of the CPU 10. used.
The flash memory 13 is a recording medium that stores image data and sound data.

The image display unit 12 includes a color LCD and its drive circuit, and displays the subject imaged by the CCD 5 as a through image when in a shooting standby state, and is read out from the flash memory 13 and decompressed when a recorded image is reproduced. The recorded image is displayed.
The key input unit 14 includes a plurality of operation keys such as a shutter button, a mode switching key, a sound reproduction key, a cross key, and a SET key, and outputs an operation signal according to the user's key operation to the CPU 10.

  The sound processing unit 15 includes a built-in microphone (microphone), an amplifier / low-pass filter, an A / D converter, a D / A converter, a low-pass filter / amplifier, a built-in speaker, and the like. The sound that has been sounded is converted into a digital signal and sent to the CPU 10. The CPU 10 sequentially stores the transmitted audio data in the buffer memory (DRAM 12), and records the stored audio data in the flash memory 13 in association with the captured moving image data. That is, moving image data with sound is recorded.

The sound processing unit 15 reproduces the sound data by emitting sound data sent from the CPU 10 from the built-in speaker.
Both the built-in microphone and built-in speaker are provided on the surface of the digital camera 1 on which the photographing lens 2 is provided. As a result, the reproduced sound can be heard easily by the subject, and the sound of the subject can be recorded well. The image display unit 12 is provided on a surface opposite to the surface on which the photographing lens 2 is provided (photographer side).

B. Next, functions of the configuration of the digital camera 1 of the present invention will be described.
2 and 3 are block diagrams for explaining the functions of the audio processing unit and the CPU 10 of the present invention.
FIG. 2 is a diagram for explaining the function of the CPU 10 in the case of shooting moving image with sound (recording process) while reproducing the sound data recorded in the flash memory 13 or the memory 9.
As can be seen from FIG. 2, the audio processing unit 15 includes a built-in microphone 21, an amplifier / low-pass filter 22, an A / D converter 23, a D / A converter 24, a low-pass filter / amplifier 25, and a built-in speaker 26. The CPU 10 includes a subtraction processing unit 31.

  First, the built-in microphone 21 converts the collected subject voice or the like into an electrical signal and outputs sound data to the amplifier / low-pass filter 22. The amplifier / low-pass filter 22 amplifies the sound data (AMP), cuts an unnecessary frequency band (LPF), and outputs it to the A / D converter 23. The A / D converter 23 converts the input sound data into a digital signal and outputs it to the CPU 10.

  At this time, the CPU 10 stores in the buffer memory (DRAM 11) the sound data (sound data such as music and play sound) recorded in advance in the flash memory 13 and the memory 9, and stores the stored sound. Data is output to the D / A converter 24. The D / A converter 24 converts the received sound data into an analog signal and outputs the analog signal to the low-pass filter / amplifier 25. The low-pass filter / amplifier 25 uses the unnecessary frequency band of the input sound data. Is amplified and output to the built-in speaker 26. The built-in speaker 26 converts the inputted sound data into sound and emits the sound.

  In this case, if sound is emitted from the built-in speaker 26 during recording, the sound (reproduced sound) reproduced from the built-in speaker 26 at a sound level higher than the sound of the subject is collected by the built-in microphone 21. This causes a problem that the sound of the subject is erased. This is because the built-in microphone 21 is closer to the built-in speaker 26 than the subject and emits sound from the built-in speaker 26 at a sound level that can be heard by the subject.

  Therefore, the subtraction processing unit 31 of the CPU 10 causes the built-in microphone 21 so that the sound level (volume) of the sound of the subject collected by the built-in microphone 21 and the reproduction sound collected by the built-in microphone 21 are the same. The sound data reproduced by the built-in speaker 26 is subtracted from the sound data collected by the above. This subtraction process is for subtracting so that the volume of the sound collected by the built-in microphone 21 is the same as that of the reproduced sound, and is not necessarily for subtracting to eliminate the reproduced sound. In addition, this subtraction subtracts the sound data of the reproduced sound in synchronization with the recorded reproduced sound.

  The degree of subtraction can be selected by the user, and the selected degree may be subtracted. For example, when the degree of subtraction is increased, recording can be performed so that the volume (sound level) of the reproduced sound is reduced, and when the degree of subtraction is decreased, recording can be performed so that the volume of the reproduced sound is increased. it can.

Then, the sound data subtracted by the subtraction processing unit 31 is stored in the buffer memory and recorded in the flash memory 13.
Thereby, the volume of the reproduction sound from the built-in speaker 26 collected by the built-in microphone 21 can be adjusted and recorded.

FIG. 3 is a diagram for explaining the function of the CPU 10 in the case of reproducing sound data recorded simply by moving-image shooting with sound (recording processing) without performing subtraction processing while reproducing sound data.
Sound recorded without sound subtraction from the sound data collected from the built-in microphone 21 is the sound reproduced from the sound of the subject. The volume is much larger and the subject's voice has been drowned out.

Therefore, when reproducing such a sound, the subtraction processing unit 31 of the CPU 10 records the sound so that the sound of the subject recorded during the recording process and the sound reproduced during the recording process have the same sound level. The sound data reproduced at the time of recording is subtracted from the recorded sound data. This subtraction process is for subtracting so that the volume of the sound collected by the built-in microphone 21 and the reproduced sound collected by the built-in microphone 21 are the same, and the sound reproduced during recording is not necessarily eliminated. Not for subtraction.
The degree of subtraction can be selected by the user, and the selected degree may be subtracted. For example, if the degree of subtraction is increased, the volume (sound level) of the sound played during recording can be reduced, and if the degree of subtraction is reduced, the volume of the sound played back during recording can be increased. Can be played.

  The sound data after being subtracted by the subtraction processing unit 31 is input to the D / A converter 24. The D / A converter 24 converts the input sound into an analog signal to convert it to a low-pass filter / amplifier. The low-pass filter / amplifier 25 cuts an unnecessary frequency band of the input sound data, amplifies it, and outputs it to the built-in speaker 26. The built-in speaker 26 outputs the input sound data to the sound. Convert to sound and emit. As a result, the volume of the sound reproduced during recording can be adjusted and reproduced.

C. Operation of Digital Camera 1 The operation of the digital camera 1 in the embodiment will be described separately for the case of subtraction during recording and the case of subtraction during playback.
C-1. First, the operation when subtracting during recording will be described.
The operation when subtracting sound data reproduced during recording will be described with reference to the flowchart of FIG.

  When the shooting mode with sound is set by operating the mode switching key of the user's key input unit 14, in step S1, the CPU 10 starts a process of causing the CCD 5 to image a subject at a predetermined frame rate (moving image capturing process). The frame image data (YUV data) of the luminance / color difference signal is generated from the frame image data sequentially obtained by the imaging, and the frame image data of the luminance / chrominance signal sequentially generated is buffered (DRAM 11). And through image display is started in which the frame image data of the stored subject is sequentially displayed on the image display unit 12.

Next, in step S2, the CPU 10 determines whether or not to reproduce sound.
At this time, when the user operates the sound reproduction key, the CPU 10 superimposes sound data (sound data such as music or play sound) recorded in the flash memory 13 or the memory 9 on the through image display. The user can select any sound data by operating the cross key and the SET key, and the CPU 10 determines that the sound is reproduced when the series of operations is performed.

If it is determined in step S2 that the sound is to be reproduced, the process proceeds to step S3, where the CPU 10 starts the reproduction process of the sound data selected by the user, and proceeds to step S4. That is, the selected sound data is read out and stored in the buffer memory, and the stored sound data is sequentially input to the sound processing unit 15. The sound processing unit 15 performs analog conversion or the like on the input sound data. Then, the process of emitting sound from the built-in speaker 26 is started. This sound data playback process automatically ends when the sound data is played back to the end, but also ends when the user performs a playback end instruction operation.
On the other hand, if it is determined in step S2 that the sound is not reproduced, the process proceeds to step S4.

In step S4, the CPU 10 determines whether to start recording. This determination is made based on whether or not an operation signal corresponding to pressing of the shutter button is sent from the key input unit 14.
If it is determined in step S4 that recording is not started, the process proceeds to step S5, and the CPU 10 determines whether or not sound reproduction processing is currently in progress. That is, it is determined whether or not sound is currently being emitted from the built-in speaker 26.
If it is determined in step S5 that sound reproduction processing is not being performed, the process returns to step S2, and if it is determined that sound reproduction processing is being performed, the process returns to step S4.

On the other hand, if it is determined in step S4 that recording is started, the process proceeds to step S6, and the CPU 10 starts moving image recording processing. That is, the process of recording the frame image data sequentially captured by the CCD 5 and stored in the buffer memory in the flash memory 13 is started.
Next, in step S7, the CPU 10 starts a recording process. That is, the sound collected by the built-in microphone 26 is converted into an electrical signal, the sound data converted into a digital signal is sequentially stored in the buffer memory, and the stored sound data is recorded in the flash memory 13. Start.

Next, in step S8, the CPU 10 determines whether or not sound reproduction processing is currently in progress.
If it is determined in step S8 that sound is being reproduced, the process proceeds to step S12. If it is determined in step S8 that sound is not being reproduced, the process proceeds to step S9 to determine whether or not to reproduce the sound. That is, the same determination as in step S2 is performed.

If it is determined in step S9 that no sound is reproduced, the process proceeds to step S10, and the CPU 10 determines whether or not to end the recording. This determination is made based on whether or not an operation signal corresponding to pressing of the shutter button is sent from the key input 14.
Here, recording is started when the shutter button is pressed, and it is determined that the recording is ended when the shutter button is pressed again. However, recording is performed while the shutter button is pressed, and the shutter button is released. Then, it may be determined that the recording is finished.
If it is determined in step S10 that the recording is not finished, the process returns to step S9.

On the other hand, if it is determined in step S9 that the sound is to be reproduced, the process proceeds to step S11, and the CPU 10 starts the reproduction process of the sound data selected by the user, and proceeds to step S12.
In step S12, the subtraction processing unit 31 of the CPU 10 starts a process of subtracting the sound data currently being emitted (reproduced) from the built-in speaker 26 from the sound data collected from the built-in microphone 21. Let The sound data subtracted by this subtraction process is stored in the buffer memory by the sound recording process and recorded in the flash memory 13.

Next, the process proceeds to step S13, and the CPU 10 determines whether or not to end the recording. This determination is the same as in step S10.
If it is determined in step S10 that the recording is not to be ended, the process proceeds to step S14, and the CPU 10 determines whether or not the sound reproduction processing has ended. The case where it is determined that the sound reproduction processing has been completed includes the case where the sound being reproduced has been reproduced to the end, the case where the user has performed an instruction to end reproduction, and the like. When the user performs a reproduction end instruction operation, the CPU 10 stops the sound reproduction process.

  If it is determined in step S14 that the sound reproduction process has not ended, the process returns to step S13. If it is determined in step S14 that the sound reproduction process has ended, the process proceeds to step S15, and the CPU 10 ends the subtraction process. Return to step S9. When the subtraction process is completed, the sound data acquired by the built-in microphone 21 is stored as it is in the buffer memory and recorded in the flash memory 13. That is, the subtraction processing unit 31 in FIG. 2 allows the sound data sent from the A / D converter 23 to pass through without being subtracted and to store and record it in the buffer memory or the like.

On the other hand, if it is determined in step S10 that the recording is to be ended, or if it is determined in step S13 that the recording is ended, the process proceeds to step S16, and the CPU 10 stores the frame image data recorded in the flash memory 13 by the moving image recording process, A sound-added moving image file is generated from the sound data recorded in the flash memory 13 by the recording process. Although the sound-added moving image file is generated, the sound data and the moving image data may be recorded in association with each other.
As a result, even if the sound is recorded while being reproduced, the sound of the subject is not erased by the reproduced sound, and the recording accuracy of the subject sound when the sound is reproduced during recording can be improved. it can.

C-2. Next, the operation when subtracting during reproduction will be described.
The operation in the case of subtraction at the time of reproduction will be further described separately for sounded moving image shooting and sounded moving image reproduction.
First, the operation of sounding operation shooting when subtracting during reproduction will be described with reference to the flowchart of FIG.

When the sound moving image shooting mode of the mode switching key of the user key input unit 14 is set, in step S31, the CPU 10 starts moving image capturing processing and starts displaying a through image of the subject.
Next, in step S32, the CPU 10 determines whether or not to reproduce the sound. This determination is the same as step S2 in FIG.
If it is determined in step S32 that the sound is to be reproduced, the process proceeds to step S33, and the CPU 10 starts the reproduction process of the sound data selected by the user, and proceeds to step S34.
On the other hand, if it is determined in step S32 that the sound is not reproduced, the process proceeds to step S34 as it is.

In step S34, the CPU 10 determines whether to start recording. This determination is the same as step S4 in FIG.
If it is determined in step S34 that the recording is not started, the process proceeds to step S35, and the CPU 10 determines whether or not a sound reproduction process is currently being performed. That is, it is determined whether or not sound is currently being emitted from the built-in speaker.
If it is determined in step S35 that sound reproduction processing is not being performed, the process returns to step S32. If it is determined that sound reproduction processing is being performed, the process returns to step S34.

On the other hand, if it is determined in step S34 that recording is started, the process proceeds to step S36, and the CPU 10 starts moving image recording processing. That is, the process of recording the frame image data sequentially captured by the CCD 5 and stored in the buffer memory in the flash memory 13 is started.
Next, the CPU 10 proceeds to step S37, and the CPU 10 starts a recording process. That is, the sound collected by the built-in microphone 26 is converted into an electrical signal, the sound data converted into a digital signal is sequentially stored in the buffer memory, and the stored sound data is recorded in the flash memory 13. Start.

Next, the process proceeds to step S38, and the CPU 10 determines whether or not a sound reproduction process is currently being performed. That is, it is determined whether or not sound is currently being emitted from the built-in speaker.
If it is determined in step S38 that sound reproduction processing has not been performed, the process proceeds to step S39, and the CPU 10 determines whether or not to reproduce sound. This determination is the same as step S2 in FIG.

If it is determined in step S39 that the sound is to be reproduced, the process proceeds to step S40, where the CPU 10 starts the reproduction process of the sound data selected by the user, and proceeds to step S41. This reproduction processing is automatically terminated when the reproduction is completed to the end, but is also terminated when a reproduction end instruction operation is performed by the user.
On the other hand, if it is determined in step S38 that no sound reproduction process is currently being performed, or if it is determined in step S39 that no sound is to be reproduced, the process proceeds directly to step S41.

In step S41, the CPU 10 determines whether to end the recording. This determination is performed in the same manner as step S10 in FIG.
If it is determined in step S41 that the recording is not to be ended, the process returns to step S38. If it is determined in step S41 that the recording is to be ended, the process proceeds to step S42, and the CPU 10 records the frame image data recorded in the flash memory 13 by the moving image recording process. A sound-added moving image file is generated from the sound data recorded in the flash memory 13 by the processing, and information (sound information) of the reproduced sound data is recorded in association with it. This sound information is the location information of the reproduced sound data, the timing at which the sound data was reproduced, the timing at which the reproduction of the sound data was completed, and from which part of the reproduced sound data was recorded. It is information such as Taka. For example, if sound data has already been played back at the start of the recording process, information on when the sound data was recorded from among the played sound data, and the sound data is played back after the recording process is started If it is, it is information on when the sound data was reproduced at the timing of recording or recording.

Next, the operation for sounding operation reproduction when subtracting during reproduction will be described with reference to the flowchart of FIG.
When the sound-added moving image shooting mode of the mode switching key of the user key input unit 14 is set, the process proceeds to step S51, and the CPU 10 selects a sound-added moving image file to be reproduced in accordance with the user operation.

Next, in step S52, the CPU 10 starts reproduction processing of the selected moving image file with sound. In other words, the sound-added moving image file is read out and stored in the buffer memory, the frame image data is sequentially displayed on the image display unit 12, and the process of sequentially emitting sound data from the built-in speaker 26 is started.
At this time, if sound information is recorded in association with the selected moving image file with sound, sound data reproduced at the time of recording is specified based on the sound information, read from the flash memory 13 or the memory 9, and buffered. The sound data is stored in the memory.

  Next, in step S53, the CPU 10 determines whether or not the timing at which recording of the sound reproduced at the time of shooting has started has arrived. This determination is made based on the sound information recorded in association with the sound-added moving image file selected in step S51. The timing at which recording of the sound played back at the time of shooting is started, for example, when sound data has already been played back at the start of video recording and recording, is the timing at which video recording and recording started, When moving image recording and recording are started and then reproduction of sound data is started, this is the timing at which the start of reproduction of the sound data is started.

If it is determined in step S53 that the recording timing of the reproduced sound has not started, the process proceeds to step S54, and the CPU 10 determines whether or not the reproduction processing of the sound-added moving image file has been completed.
If it is determined in step S54 that the reproduction process of the moving image file with sound has not been completed, the process returns to step S53.

  On the other hand, if it is determined in step S53 that the recording timing of the reproduced sound has started, the process proceeds to step S55, and the subtraction processing unit 31 of the CPU 10 determines from the sound data of the sound-added moving image file currently being reproduced. Then, the process of subtracting the sound data reproduced during recording stored in the buffer memory is started. At this time, the reproduced sound data is subtracted from the sound data of the sound-added moving image file in synchronization with the sound reproduced at the time of recording. The subtracted sound data is emitted from the built-in speaker 26.

Next, in step S56, the CPU 10 determines whether or not the reproduction of the moving image file with sound has been completed. This determination is made when the movie file with sound is played back to the end, or when the operation signal indicating that the playback is finished is sent by the user's operation of the key input unit 14, the playback of the movie file with sound is finished. Judge.
If it is determined in step S56 that the reproduction of the moving image file with sound has not ended, the process proceeds to step S57, and the CPU 10 determines whether or not the timing at which recording of the sound reproduced at the time of shooting (recording) has ended. Judging. This determination is also made based on the sound information recorded in association with the sound-added moving image file selected in step S51. As the timing when the recording of the sound played back at the time of shooting is finished, the sound played back at the time of shooting is played to the end and the playback of the sound automatically ends, or the playback of the sound played by the user at the time of shooting There is a timing when the ending operation is performed.

If it is determined in step S57 that the recording of the sound reproduced at the time of shooting has not ended, the process returns to step S56, and in step S57, it is determined that the timing of the recording of the sound reproduced at the time of shooting has come. Then, the process proceeds to step S58, and the CPU 10 ends the subtraction process of the sound data reproduced at the time of recording from the sound data of the currently-reproduced moving image file with sound, and returns to step S53.
On the other hand, if it is determined in steps S54 and S56 that the reproduction of the sound-added moving image file is finished, the process proceeds to step S59, and the CPU 10 ends the sound-added moving image file reproduction process.
As a result, even when the recorded sound is reproduced while the sound is reproduced, the sound of the subject is not drowned out by the sound reproduced at the time of recording, and the reproduction accuracy of the recorded sound can be improved. it can.

As described above, in the first embodiment, even when sound data is played back during shooting of sound-added moving images, the sound data to be played back is extracted from the sound data collected by the built-in microphone 21. Since the subtraction is performed, even if the sound is played back during shooting and recording, the sound of the subject is not drowned out by the played back sound, and the accuracy of playback of the sound recorded while the sound is played back can be improved. it can.
For example, even when a moving image with sound is taken while playing a song such as a dance, only the dance song is not played with an extremely loud sound when the movie is played.

In the first embodiment, the reproduced sound data is obtained from the sound data collected from the built-in microphone 21 so that the sound of the subject and the sound reproduced at the time of recording have the same sound level. Subtraction is performed, but the reproduced sound data is subtracted from the sound data collected from the built-in microphone 21 so that there is no sound reproduced during recording (the sound level is zero or substantially zero). Then, the subtracted sound data and the sound data reproduced at the time of recording may be synthesized. In this case, the sound level of the sound data reproduced at the time of recording may be synthesized so that the sound level of the sound of the subject is the same, or the recording may be performed so that the sound level selected by the user is obtained. Sometimes the reproduced sound data may be synthesized. Thereby, the reproduced sound can be recorded with a beautiful sound.
Alternatively, sound data collected by the built-in microphone 21 may be recorded as it is, and after recording, sound data obtained by subtracting sound data reproduced during recording from the recorded sound data may be generated and recorded. Thereby, it is possible to reduce the processing load at the time of shooting and reproduction.

[Second Embodiment]
Next, a second embodiment will be described.
In the second embodiment, even when sound data is reproduced during or during the recording of a moving image with sound, the sound of the subject is sufficiently not to be erased by the reproduced sound. The voice is made audible to the photographer.

D. Function of Digital Camera 1 The second embodiment also implements the imaging apparatus of the present invention by using the digital camera 1 having the same configuration as that shown in FIG.
However, in the second embodiment, the sound processing unit 15 includes two built-in speakers, a low-pass filter / amplifier, and an A / D converter, and one built-in speaker (described in the first embodiment). The built-in speaker 21) is provided on the surface (subject side) where the taking lens 2 is provided, and the other built-in speaker is provided on the opposite surface (photographer side) where the image display unit 12 is provided. . In addition, the CPU 10 includes an anti-phase generation unit that generates sound data having a phase opposite to that of the sound data.

FIG. 7 is a diagram for explaining the functions of the audio processing unit 15 and the CPU 10 in the second embodiment.
As can be seen from FIG. 7, the audio processing unit 15 includes a D / A converter 24, a low-pass filter / amplifier 25, a built-in speaker 26 (first speaker 26), a D / A converter 27, A low-pass filter / amplifier 28 and a built-in speaker 29 (second speaker 29) are provided. The built-in microphone 21, the amplifier / low pass filter 22, and the A / D converter 23 are not shown.

  First, the CPU 10 stores sound data (sound data such as music) of a digital signal recorded in the flash memory 13 in a buffer memory, and the anti-phase generation unit 32 of the CPU 10 calculates the sound data from the stored sound data. The sound data with the opposite phase to the sound data is sequentially generated. Since the generation of the audio data with the opposite phase is already a well-known technique, the description is omitted.

The CPU 10 sequentially outputs the stored sound data to the D / A converter 24 and sequentially outputs the generated antiphase sound data to the D / A converter 27. At this time, the sound data and the generated sound data of the opposite phase are synchronized and output to the D / A converters 24 and 27, respectively.
The A / D converters 24 and 27 convert the input digital signal sound data into analog signal sound data, and output the analog signal sound data to the low-pass filters / amplifiers 25 and 28, respectively. The unnecessary frequency band of the input sound data is cut and amplified, and then output to the built-in speakers 26 and 29. The built-in speakers 26 and 29 convert the input sound data into sound and emit sound. To do.

  In this way, sound data is emitted as it is to the built-in speaker 26 on the subject side, and sound data having a phase opposite to that of the sound data is emitted to the built-in speaker 29 on the photographer side. 26, the sound emitted from the built-in speaker 26 is transmitted to the photographer as it is, and the sound emitted from the built-in speaker 26 is attenuated by the reverse phase sound emitted from the built-in speaker 29. The sound to be heard is reduced, and the subject's voice is fully transmitted to the photographer without being erased.

E. Operation of Digital Camera 1 The operation of the digital camera 1 according to the second embodiment will be described below with reference to the flowchart of FIG.
First, in step S71, the CPU 10 determines whether or not to reproduce a sound. This determination is the same as in step S2 or step S9 in FIG. 4, and the sound recorded on the flash memory 13 or the like displayed on the through image display is displayed by the user operating the sound reproduction key. When data (sound data such as music or play sounds) is selected by the user's operation of the cross key or the SET key, it is determined that the sound is reproduced.

If it is determined in step S71 that the sound is to be reproduced, the process proceeds to step S72, and the anti-phase generation unit 32 of the CPU 10 starts a process of generating sound data having an anti-phase with the sound data selected by the user.
Next, in step S73, the sound data selected by the user is emitted from the first speaker, thereby starting the reproduction processing of the sound data.
Next, in step S74, the sound data reproduction process is started by emitting the generated sound data of opposite phase from the second speaker. At this time, the sound data in the opposite phase is emitted in synchronization with the sound data reproduced in step S73.

Next, in step S75, it is determined whether or not the sound reproduction processing has ended. This determination is the same as step S14 in FIG. 4. When the sound being played has been played to the end, or when the user has performed an operation to stop the playback, the playback process has ended. Judge.
If it is determined in step S75 that the sound reproduction process has not been completed, the process returns to step S75 until the process is completed. If it is determined that the sound reproduction process has been completed, the process proceeds to step S76, where the CPU 10 The reproduction process of steps S73 and 74 is finished, and the process returns to step S71.

  As described above, in the second embodiment, sound data having an opposite phase to the sound data being reproduced is generated, and the sound data being reproduced is emitted from the first speaker directed toward the subject side. Since the generated sound data having the opposite phase is emitted from the second speaker directed to the photographer, the photographer can hear the sound of the subject without being erased by the sound being reproduced. .

[Third Embodiment]
Next, a third embodiment will be described.
In the third embodiment, by suppressing peripheral sounds other than the sound of the subject among the recorded sounds, the sound of the subject is prevented from being erased by the peripheral sound.

F. Functions of Digital Camera 1 In the third embodiment, the imaging apparatus of the present invention is realized by using the digital camera 1 having the same configuration as that shown in FIG.
However, in the third embodiment, the CPU 10 includes a voice recognition processing unit, a sound extraction unit, and a subtraction processing unit.

FIG. 9 is a diagram for explaining the functions of the audio processing unit 15 and the CPU 10 in the third embodiment.
As can be seen from FIG. 9, the sound data of the digital signal (sound and ambient sound is converted by the built-in microphone 21 of the sound processing unit 15 and output from the A / D converter 23 via the amplifier / low-pass filter 22). Sound data) is sent to the speech recognition processing unit 33 of the CPU 10. The D / A converter 24, the low-pass filter / amplifier 25, and the built-in speaker 26 of the audio processing unit 15 are not shown.

Then, the voice recognition processing unit 33 recognizes the voice part of the sound data by performing voice recognition processing on the transmitted sound data (sound data including voice and surrounding sounds), and the sound extraction unit 34. Output to. This voice recognition makes it possible to distinguish between voice (human voice) and other ambient sounds (animal calls, car sounds, sea wave sounds, and sound of trees flying by the wind). . Since this voice recognition process is already a well-known technique, a description thereof is omitted.
The sound extraction unit 34 extracts sound data of peripheral sound from the transmitted sound data (sound data including sound and peripheral sound) based on the recognition result of the sound recognition processing unit. The sound extraction unit 34 outputs the received sound data (sound data including sound and peripheral sounds) to the subtraction processing unit 31 as it is, and also outputs the extracted sound data of the peripheral sounds to the subtraction processing unit 31.

The subtraction processing unit 31 subtracts the transmitted sound data of the peripheral sound from the transmitted sound data (sound data including sound and peripheral sound). This subtraction may be performed so that the peripheral sound is completely eliminated or may be subtracted so that the peripheral sound is slightly heard. Further, the user may arbitrarily select the degree of subtraction.
The subtracted sound data output from the subtraction processing unit 31 is stored in the buffer memory and recorded in the flash memory 13.
As a result, it is possible to record while suppressing the volume of the ambient sound, and the sound of the subject is not erased by the ambient sound.

G. Operation of Digital Camera 1 The operation of the digital camera 1 according to the third embodiment will be described below with reference to the flowchart of FIG.
When the sound switching moving image shooting mode of the mode switching key of the user key input unit 14 is set, the process proceeds to step S91 where the CPU 10 starts moving image capturing processing and starts displaying a through image of the subject.

Next, in step S92, the CPU 10 determines whether to start recording. This determination is the same as step S4 in FIG.
If it is determined in step S92 that the recording is not started, the process stays in step S92 until it is determined that the recording is started. If it is determined that the recording is started, the process proceeds to step S93, and the CPU 10 starts the moving image recording process. That is, the process of recording the frame image data sequentially captured by the CCD 5 and stored in the buffer memory in the flash memory 13 is started.

Next, in step S94, the CPU 10 starts a recording process. That is, the sound data converted into the electric signal by the built-in microphone 21 and converted into the digital signal is sequentially stored in the buffer memory, and the process of recording the stored sound data in the flash memory 13 is started.
Next, in step S95, the voice recognition processing unit 33 of the CPU 10 performs voice recognition processing on the sound data of the digital signal acquired by the built-in microphone 21 and sequentially converted by the A / D converter 23. The voice part is recognized, and the transmitted sound data is output to the sound extraction unit 34 as it is.

Next, in step S96, the CPU 10 determines whether or not the sound of the subject is included based on the recognition result obtained by the sound recognition process.
If it is determined in step S96 that the sound of the subject is included, the process proceeds to step S97, and the sound extraction unit 34 of the CPU 10 includes the sound data sent from the sound recognition processing unit 33 based on the sound recognition result. Then, the sound data of the peripheral sound is extracted, and the subtraction processing unit 31 performs a process of subtracting the extracted sound data of the peripheral sound from the transmitted sound data, and proceeds to step S98. The subtracted sound data is stored in the buffer memory and recorded in the flash memory 13.

On the other hand, if it is determined in step S96 that the subject's voice is not included, the process directly proceeds to step S98. At this time, the sound extraction unit 34 and the subtraction processing unit 31 output the transmitted sound data through, and the sound data transmitted to the speech recognition processing unit 33 is not subtracted as it is but in the buffer memory. And recorded in the flash memory 13.
In step S98, the CPU 10 determines whether to end recording. This determination is performed in the same manner as step S10 in FIG.
If it is determined in step S98 that the recording is not to be ended, the process returns to step S96. If it is determined in step S98 that the recording is to be ended, the process proceeds to step S99, where the CPU 10 captures and records the frame image data and the recorded sound data. Based on the above, a sound-added moving image file is generated.

  As described above, in the third embodiment, the sound data collected by the built-in microphone 21 is recognized by voice, and the sound data collected by the built-in microphone 21 based on the voice recognition result is used for other than voice. Since the sound data of the surrounding sound is subtracted, even if the surrounding sound is noisy, the sound of the subject is not drowned out by the surrounding sound, and can be recorded so that the sound of the subject becomes the loudest. , Recording and playback accuracy can be improved.

In the third embodiment, the sound data of the surrounding sound is subtracted from the sound data collected from the built-in microphone 21 and recorded (although it is recorded), but at the time of recording, Alternatively, the sound may be recorded without being subtracted, and the sound may be reproduced by subtracting the sound data of the surrounding sound from the sound data to be reproduced by performing a voice recognition process at the time of reproduction.
Alternatively, sound data collected from the built-in microphone 21 may be recorded as it is, and sound data obtained by subtracting the surrounding sound may be generated and recorded by performing a voice recognition process after the recording. Thereby, it is possible to reduce the processing load at the time of shooting and reproduction.

Further, for example, when the photographer and the subject have a conversation, the photographer's voice near the built-in microphone 21 becomes louder. Therefore, by recognizing the sound data collected by the built-in microphone 21, the sound level can be adjusted. Built-in so that loud sound (sound with high sound) and low sound level (sound with low sound) are distinguished, and sound with a high sound level and sound with a low sound level have the same sound level. Audio data with a high sound level may be subtracted from the sound data collected by the microphone 21, or a sound with a low sound level may be amplified to increase the sound level. As a result, the sound volume of the photographer and the subject can be recorded and reproduced with the same volume, and the recording accuracy and reproduction accuracy can be improved.
Also in this case, the user may be able to specify the degree of subtraction and the degree of amplification.

  In the first and third embodiments, voice recognition is performed on sound data collected by the built-in microphone 21, and no subtraction is performed on sound data collected by the built-in microphone 21, or While subtracting, the sound level of the sound may be increased and recorded and reproduced.

Moreover, the aspect which combined each said embodiment arbitrarily may be sufficient.
When the first embodiment and the third embodiment are combined, the sound collected by the built-in microphone 21 from the sound data of the surrounding sounds other than the sound data reproduced at the time of recording is collected. While subtracting from the data, sound data to be reproduced at the time of recording may be subtracted.
For example, when playing music with vocals during recording, the sound being played may be divided into a part that is not recognized as a peripheral sound and a part that is recognized as a peripheral sound. Since the music being played cannot be subtracted, the music being played and the sound data other than the music of the surrounding sounds are divided and subtracted from the sound data collected from the built-in microphone 21. Thereby, the reproduction accuracy can be improved.

  In each of the above-described embodiments, the case of moving image shooting with sound has been described. However, the present invention can also be applied to still image shooting with sound and simple recording without shooting.

The above embodiments of the present invention are merely examples as the best embodiments, and are described in order to better understand the principle and structure of the present invention. It is not intended to limit the scope of the appended claims.
Therefore, it should be understood that all the various variations and modifications that can be made to the above-described embodiments of the present invention are included in the scope of the present invention and protected by the appended claims.

  Finally, in each of the above embodiments, the case where the imaging apparatus of the present invention is applied to the digital camera 1 has been described. However, the present invention is not limited to the above embodiment, and in short, the subject is photographed, Any device that can record sound is applicable.

1 Digital Camera 2 Shooting Lens 3 Lens Drive Block 4 Aperture 5 CCD
6 Driver 7 TG
8 Unit circuit 9 Memory 10 CPU
11 DRAM
12 Image display unit 13 Flash memory 14 Key input unit 15 Audio processing unit 16 Bus

In order to achieve the above object, the imaging apparatus of the present invention provides:
Imaging means;
Obtaining means for obtaining sound data collected by a microphone;
Recognition means for performing speech recognition for recognizing a human voice for the acquired sound data;
Subtracting means for subtracting sound other than human voice from the acquired sound data based on the result of the speech recognition;
Recording control means for associating the image data picked up by the image pickup means with sound data obtained by subtracting sounds other than the human voice and recording them in a recording device;
It is provided with.

According to the present invention, it is possible to improve the reproduction accuracy of the recorded sound sounds.

Claims (9)

Imaging means for imaging a subject;
A speaker that emits sound;
Sound emission control means for emitting sound data from the speaker;
A microphone that collects the sound,
Recording control means for associating the image data picked up by the image pickup means with the sound data collected by the microphone to record in the recording means;
While the sound data collected by the microphone is being recorded on the recording means by the recording control means, the sound data emitted by the sound emission control means is the sound data collected by the microphone. Subtracting means for subtracting from,
An imaging apparatus comprising: The subtracting means is
Performing subtraction before sound data collected by the microphone is recorded in the recording means;
The recording control means includes
2. The imaging apparatus according to claim 1, wherein the sound data subtracted by the subtracting means is recorded in the recording means. The subtracting means is
The imaging apparatus according to claim 1, wherein subtraction is performed after recording of sound data collected by the microphone by the recording unit. Reproducing means for reproducing the sound data recorded by the recording control means,
The subtracting means is
When reproducing the sound data recorded in the recording means, subtraction is performed,
The reproducing means includes
4. The imaging apparatus according to claim 3, wherein the sound data subtracted by the subtracting means is reproduced. Generating means for generating sound data having an opposite phase to the sound data emitted from the speaker;
The speaker is
A first speaker directed to the subject side and a second speaker directed to the photographer side;
The sound emission control means includes
5. The sound data to be emitted from the first speaker is emitted, and the sound data of the opposite phase generated by the generating means is emitted from the second speaker. The imaging device according to any one of the above. Voice recognition means for performing voice recognition on sound data collected by the microphone;
The subtracting means is
While the sound data collected by the microphone is being recorded on the recording means by the recording control means, ambient sounds other than the voice are collected by the microphone based on the recognition result by the voice recognition means. 6. The imaging apparatus according to claim 1, wherein the imaging apparatus subtracts from the sound data. The subtracting means is
Out of peripheral sounds other than sound, sound data other than sound data emitted by the sound emission control means is subtracted from sound data collected by the microphone, and sound emitted by the sound emission control means The image pickup apparatus according to claim 6, wherein data is also subtracted from sound data collected by the microphone. Voice recognition means for performing voice recognition on the sound data collected by the microphone;
The subtracting means is
While the sound data collected by the microphone is being recorded in the recording means by the recording control means, a voice having a high sound level among a plurality of sounds is based on the recognition result by the voice recognition means. 6. The imaging apparatus according to claim 1, wherein subtraction is performed from sound data collected by the microphone. A program for executing an imaging apparatus including an imaging unit that images a subject, a speaker that emits sound, and a microphone that collects sound,
A sound emission control process for emitting sound data from the speaker;
A recording control process in which image data captured by the imaging unit and sound data collected by the microphone are associated and recorded on a recording medium;
While the sound data collected by the microphone is recorded on the recording medium by the recording control process, the sound data emitted by the sound emission control process is the sound data collected by the microphone. Subtraction processing to subtract from
The program characterized by including.

Images