JP2013047710A - Sound signal processing apparatus, imaging apparatus, sound signal processing method, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce buffer memory length required for estimating an interpolation signal from signals neighboring a sound section and to reduce delay of an output sound corresponding to an input sound in association with interpolation processing.SOLUTION: A sound signal processing apparatus includes: a first buffer memory that temporarily stores an input sound signal for every prescribed section; a second buffer memory that temporarily stores a sound signal in an (n-1)th section immediately preceding a sound signal in an n-th section which is stored in the first buffer memory; an interpolation signal generation unit which, on detecting that noise is included in the sound signal in the n-th section, generates an interpolation signal from at least the sound signal in the (n-1)th section which is stored in the second buffer memory; and a signal interpolation unit which uses the interpolation signal to interpolate the sound signal in the n-th section which includes the noise.

Description

  The present disclosure relates to an audio signal processing device, an imaging device, an audio signal processing method, a program, and a recording medium.

  Video cameras, digital cameras with a moving image capturing function, IC recorders, and the like are known as audio recording devices that collect and record external audio. When these devices are operated, there are cases where pulsed operating sound generated from the device main body is mixed into the recorded sound.

  For example, an imaging apparatus having a moving image capturing function picks up external sound around the apparatus with a microphone during moving image capturing, and records the sound together with the moving image. At the time of capturing a moving image, a pulsed operation sound such as a zoom drive sound, an aperture drive sound, an autofocus drive sound, and an operation button pressing sound is generated in the housing of the image pickup apparatus. In particular, at the start or end of operation of a drive device (zoom motor, aperture mechanism, focus motor, etc.) that drives the imaging optical system, for example, a pulsing mechanical drive sound is generated when the motor and gear mesh. To do.

  Such a pulsed operation sound is very annoying if it is mixed and recorded as noise in the external sound that the user desires to record. For this reason, in the audio recording apparatus, it is necessary to take countermeasures against silence and noise elimination to reduce the pulsed operating sound during recording.

  Several methods have been proposed so far for reducing the pulse-like mechanical drive sound. For example, Patent Document 1 proposes to interpolate an input voice signal in a noise section including noise generated in the lens driving unit with a voice signal in a section before and after the noise section.

JP-A-8-124299

  The noise reduction method described in the above-mentioned patent document estimates a sound signal of a background sound desired to be recorded using sound signals in a section before and after the noise section, and interpolates a signal in the noise section using the estimated signal. Thus, an audio signal with reduced noise is obtained. However, in this noise reduction method, it is necessary to use signals in a certain interval before and after the noise interval, and thus a long buffer memory is required to hold signals for all the intervals. Further, when noise reduction processing is performed during recording, the output sound is greatly delayed with respect to the input sound because the long buffer memory holds the noise. For this reason, the video and the control clock are synchronized with the delayed audio, and there is a concern about the delay of the entire camera system such as another video recording unit or the control unit.

  For example, FIG. 1 shows that when recording an audio signal including a pulsed mechanical drive sound as noise, the noise reduction method described in the above-mentioned Patent Document 1 uses the background sound data of sections before and after the noise section. It is explanatory drawing which shows the case where an area is interpolated. In this case, assuming that the noise interval length is N, a buffer memory having a length of approximately 3 * N is required to hold all of the audio signals in the noise interval and the preceding and succeeding intervals. In addition, when an interpolated signal is generated after all the audio signals in these sections are stored in the buffer memory and a noise section is interpolated using the interpolated signals, the audio signal is input and then output. There will be a delay of at least 2 * N.

  As described above, in the interpolation processing for noise reduction, it is necessary to estimate an interpolation signal using a signal in the vicinity of the noise section. At this time, in order to perform highly accurate estimation, it has conventionally been necessary to secure a buffer memory capable of holding a sound signal of a certain long section before and after noise. For this reason, there is a problem that not only the buffer memory necessary for estimating the interpolation signal is increased, but also the output sound is greatly delayed with respect to the input sound, which causes a delay of the entire camera system such as video recording and control. It was.

  Therefore, in view of the above circumstances, the buffer memory length necessary for estimating the interpolation signal from the signal in the vicinity of the noise interval can be reduced, and the delay of the output sound with respect to the input sound accompanying the interpolation process can be reduced. There has been a need for a noise reduction method.

  According to the present disclosure, a first buffer memory that temporarily stores an input audio signal for each predetermined interval, and an audio signal that is one before the nth interval audio signal that is stored in the first buffer memory. a second buffer memory for temporarily storing the audio signal of the (n-1) -th section, and at least the second buffer memory when it is detected that noise is included in the audio signal of the n-th section An interpolated signal generating unit that generates an interpolated signal from the n-1th section audio signal, and a signal interpolating unit that interpolates the nth section audio signal including the noise using the interpolated signal. An audio signal processing device is provided.

  Further, according to the present disclosure, a sound collection unit that converts external sound into an audio signal, a sound generation unit that is provided in the same casing as the sound collection unit, and generates noise, and input from the sound collection unit A first buffer memory that temporarily stores the audio signal for each predetermined interval, and an audio of the (n-1) th interval that is one previous to the audio signal of the nth interval that is stored in the first buffer memory A second buffer memory for temporarily storing a signal, and the n−1th buffer stored in at least the second buffer memory when it is detected that the audio signal in the nth section includes noise An imaging apparatus comprising: an interpolation signal generation unit that generates an interpolation signal from the audio signal in the interval; and a signal interpolation unit that interpolates the audio signal in the n-th interval including the noise using the interpolation signal. Provided.

  In addition, according to the present disclosure, the audio signal of the (n−1) th section stored in the first buffer memory is temporarily stored in the second buffer memory, and the input audio signal of the nth section is stored. Is temporarily stored in the first buffer memory, and at least the second is detected when it is detected that the nth section audio signal stored in the first buffer memory contains noise. Generating an interpolated signal from the audio signal of the (n-1) -th section stored in the buffer memory of the first, and interpolating the audio signal of the n-th section including the noise using the interpolated signal. An audio signal processing method is provided.

  In addition, according to the present disclosure, the audio signal of the (n−1) th section stored in the first buffer memory is temporarily stored in the second buffer memory, and the input audio signal of the nth section is stored. Is temporarily stored in the first buffer memory, and at least the second is detected when it is detected that the nth section audio signal stored in the first buffer memory contains noise. Generating an interpolated signal from the audio signal of the (n-1) -th section stored in the buffer memory of the first, and interpolating the audio signal of the n-th section including the noise using the interpolated signal. A program for causing a computer to execute is provided.

  In addition, according to the present disclosure, the audio signal of the (n−1) th section stored in the first buffer memory is temporarily stored in the second buffer memory, and the input audio signal of the nth section is stored. Is temporarily stored in the first buffer memory, and at least the second is detected when it is detected that the nth section audio signal stored in the first buffer memory contains noise. Generating an interpolated signal from the audio signal of the (n-1) -th section stored in the buffer memory of the first, and interpolating the audio signal of the n-th section including the noise using the interpolated signal. And a computer-readable recording medium on which a program for causing the computer to execute is recorded.

  With the above configuration, the input audio signal in the nth section is completely stored in the first buffer memory, and the nth section audio signal stored in the first buffer memory includes noise. As soon as is detected, an interpolated signal is generated from the audio signal of the (n-1) th section stored in the second buffer memory, and the audio signal of the nth section is interpolated using the interpolated signal. The audio signal of the nth section after interpolation is output. As a result, the input / output processing of the audio signal for each predetermined section and the interpolation processing of the noise included in the audio signal can be suitably realized with a small delay amount using the two buffer memories.

  As described above, according to the present disclosure, it is possible to reduce the buffer memory length necessary for estimating the interpolation signal from the signal in the vicinity of the noise interval, and to reduce the delay of the output sound with respect to the input sound accompanying the interpolation process. Can do.

It is explanatory drawing which shows the case where a noise area is interpolated using the background sound data of the area before and behind a noise area with the noise reduction method which concerns on the related technique of this indication. It is a block diagram which shows the hardware constitutions of the digital camera to which the audio | voice signal processing apparatus which concerns on 1st Embodiment of this indication was applied. It is a block diagram which shows the function structure of the audio | voice signal processing apparatus which concerns on the same embodiment. It is a conceptual diagram which shows the method to produce | generate an interpolation signal from the input audio | voice signal before the noise area which concerns on the same embodiment. It is a conceptual diagram which shows the method to produce | generate an interpolation signal from the input audio | voice signal before the noise area which concerns on the same embodiment. It is a schematic diagram which shows the operation | movement at the time of the normal of the audio | voice signal processing apparatus concerning the embodiment. It is a schematic diagram which shows the operation example at the time of noise generation of the audio | voice signal processing apparatus concerning the embodiment. It is a flowchart which shows the audio | voice signal processing method concerning the embodiment. It is a block diagram which shows the function structure of the audio | voice signal processing apparatus which concerns on 2nd Embodiment of this indication. It is a conceptual diagram which shows another method which produces | generates a temporary interpolation signal and an interpolation signal from the input audio | voice signal before and behind the noise area which concerns on the embodiment. It is a schematic diagram which shows the operation | movement at the time of the normal of the audio | voice signal processing apparatus concerning the embodiment. It is a schematic diagram which shows the operation example at the time of noise generation of the audio | voice signal processing apparatus concerning the embodiment. It is a schematic diagram which shows the operation example at the time of noise generation of the audio | voice signal processing apparatus concerning the embodiment. It is a flowchart which shows the audio | voice signal processing method concerning the embodiment. It is a block diagram which shows the function structure of the audio | voice signal processing apparatus which concerns on 3rd Embodiment of this indication. It is explanatory drawing which shows the positional relationship of the audio | voice signal containing a noise and frame which concern on the embodiment. It is a schematic diagram which shows the 1st operation example at the time of noise generation of the audio | voice signal processing apparatus concerning the embodiment. It is a schematic diagram which shows the 1st operation example at the time of noise generation of the audio | voice signal processing apparatus concerning the embodiment. It is a schematic diagram which shows the 2nd operation example at the time of noise generation of the audio | voice signal processing apparatus which concerns on the embodiment. It is a schematic diagram which shows the 2nd operation example at the time of noise generation of the audio | voice signal processing apparatus which concerns on the embodiment. It is a flowchart which shows the audio | voice signal processing method concerning the embodiment.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. 1. First embodiment 1.1. Outline of mechanical sound reduction method 1.2. Configuration of audio signal processing apparatus 1.2.1. Hardware configuration of audio signal processing apparatus 1.2.2. Functional configuration of audio signal processing apparatus 1.3. Operation of audio signal processing apparatus 1.3.1. Example of normal operation without noise 1.3.2. Example of operation when noise occurs 1.4. Audio signal processing method 1.5. Effect 2. Second Embodiment 2.1. Outline of mechanical noise reduction method 2.2. Functional configuration of audio signal processing apparatus 2.3. Operation of audio signal processing apparatus 2.3.1. Example of normal operation without noise 2.3.2. Example of operation when noise occurs 2.4. Audio signal processing method 2.5. Effect 3. Third embodiment 3.1. Outline of mechanical sound reduction method 3.2. Functional configuration of audio signal processing apparatus 3.3. Operation of audio signal processing apparatus 3.3.1. First operation example when noise occurs 3.3.2. First operation example when noise occurs 3.4. Audio signal processing method 3.5. effect

<1. First Embodiment>
[1.1. Outline of mechanical noise reduction method]
First, an outline of a mechanical sound reduction method using the audio signal processing device and method according to the first embodiment of the present disclosure will be described.

  The audio signal processing apparatus and method according to the present embodiment are, for example, a pulsed operation sound (noise) generated from a sounding unit (for example, a driving device) installed in a casing of the audio recording apparatus or audio reproduction apparatus. ). In particular, in the present embodiment, in an imaging apparatus having a moving image capturing function, when recording peripheral sounds while capturing a moving image, a pulse-like generated at the start or end of the operation of a drive device built in the imaging apparatus Reduces mechanical drive noise.

  Here, the driving device is a driving device built in the imaging device to perform an imaging operation using the imaging optical system. For example, a zoom motor that moves the zoom lens, a focus motor that moves the focus lens, an aperture A drive mechanism for controlling the mechanism or the shutter is included. These driving devices are provided in the same housing as the sound collection unit of the imaging device. The pulse-like mechanical driving sound (hereinafter referred to as “pulse mechanical sound”) is, for example, instantaneous noise (for example, driving of a zoom motor) generated when the various driving devices start or end the operation. Sound, focus motor drive sound, aperture mechanism drive sound, shutter sound, operation button press sound, and the like. For example, the pulse mechanical sound is a “click” or “click” sound generated when the motor and the gear mesh when the operation of the zoom motor or the like starts or ends.

  Hereinafter, an example will be described in which the audio signal processing device is a digital camera having a moving image capturing function, and the pulse mechanical sound that is the noise to be removed is a zoom start sound generated at the start of the optical zoom operation in the digital camera. However, the audio signal processing device and the pulse mechanical sound of the present disclosure are not limited to such examples. In addition, the noise targeted by the present disclosure is not limited to the pulsed operation sound, but is any type / characteristic noise mixed in the background sound desired to be recorded out of the sound input to the sound signal processing apparatus. Applicable.

  When a user performs a zoom operation during imaging and recording by a digital camera, a zoom motor is driven inside the camera and engaged with a gear for driving a zoom lens, and instantaneously a pulse mechanical sound (zoom) (Starting sound) occurs. Then, the microphone of the digital camera includes any sound collected by the microphone, such as external sound around the camera that the user desires to record (for example, environmental sound, human speech, etc.). "Sound"), as well as pulsed mechanical sound generated inside the camera. For this reason, since the pulse mechanical sound is recorded as noise in the desired sound, when the recorded sound is reproduced, the pulse mechanical sound mixed in the desired sound becomes annoying to the user. For example, since the pulse mechanical sound is generated near the microphone with vibration of the casing of 200 Hz or less, it is picked up with a louder volume than the desired sound. Thus, since there is a volume difference between the pulse mechanical sound and the desired sound, if the mechanical sound is mixed in the desired sound, the pulse mechanical sound becomes conspicuous when the recorded sound is reproduced. Accordingly, there has been a demand for a technique capable of recording only desired sound after appropriately removing pulse mechanical sound such as the zoom start sound at the time of recording or reproducing moving images and sounds.

  In the conventional noise reduction technology, as described in Patent Document 1, a mechanical drive sound generation interval (noise interval) is estimated based on the transmission timing of a drive signal for controlling the drive device, and before and after the noise interval. The interpolation signal is estimated using the signal in the section, and the noise is reduced by interpolating the signal in the noise section with the interpolation signal. However, in this noise reduction method, as described above, in order to hold the signals of the sections before and after the noise section and generate the interpolation signal, in order to simultaneously hold the signals of all these sections, the noise section length A buffer memory corresponding to about three times N was required (see FIG. 1). For this reason, not only the buffer memory required for the noise reduction processing increases, but also the output sound is significantly delayed with respect to the input sound by the time for holding the 3 * N signal in the buffer memory (at least 2). * N delay occurs).

  Therefore, the present embodiment is characterized in that the two buffer memories provided in the noise reduction processing circuit are effectively used to suitably control the processing of the audio signal in units of frames and generate the interpolation signal. As a result, the buffer memory length required for generating the interpolation signal can be reduced, and the delay of the output audio signal with respect to the input audio signal can be greatly reduced.

  Furthermore, in the present embodiment, the interpolation signal is generated using only the audio signal in the section before the noise section including the pulse mechanical sound, and the audio signal in the noise section is output after being interpolated with the interpolation signal. It is a feature. Thus, even if the interpolation signal is generated using only the audio signal in the section before the noise section, the pulse mechanical sound can be appropriately reduced. The reason is as follows.

  In the prior art described in Patent Document 1 and the like described above, as a signal in a section not including noise (a section before and after the noise section), for example, speech such as human speech is assumed. Such sound is composed of periodic signals when viewed in a narrow time. In order to interpolate noise in a periodic signal, it is necessary to generate an interpolation signal having the same period as the period of the signal before and after the noise and connect the front and back of the noise interval without disturbing the period. The reason for this is that when the signal period is disturbed by the interpolation process, the sound becomes uncomfortable in terms of hearing. Therefore, conventionally, it is common for those skilled in the art to generate an interpolation signal using signals before and after the noise interval, and generating an interpolation signal using only the signal before the noise interval is a problem in sound quality. It was thought that problems would occur.

  However, in an actual recording environment, periodic sounds such as human speech are not always generated, and in many cases, various sounds are mixed to generate non-periodic sounds. If there is a non-periodic speech before and after the noise section, it is not necessary to align the periods before and after the interpolation of the noise section, and it is difficult for a sound with an uncomfortable feeling to occur. As a result, even when interpolation is performed using only the speech in front of the noise, it is possible to remove noise substantially.

  It can also occur when the noise is periodic (such as human speech), but in most cases it is spoken near the camera. In this case, the sound is input to the microphone as a loud sound. Is done. Therefore, since the voice inputted from the outside is larger than the noise (pulse mechanical sound etc.) generated inside the camera, the noise itself is often inaudible due to the masking phenomenon. Therefore, in such a case, it is not necessary to perform an interpolation process in the noise section, and it can be said that there is no adverse effect due to the interpolation using the speech in front of the noise.

  Therefore, in the first embodiment described in detail below, when the nth section of the input audio signal is a noise section including noise, the (n−1) th one before the noise section. Interpolation signals for noise reduction are generated using only the audio signal in the interval (n: natural number). Even with such an interpolation process, it is possible to appropriately reduce noise for the above reasons. The audio signal processing apparatus and method according to the first embodiment will be described in detail below.

[1.2. Configuration of audio signal processing apparatus]
[1.2.1. Hardware configuration of audio signal processing apparatus]
First, a hardware configuration example of a digital camera to which the audio signal processing device according to this embodiment is applied will be described with reference to FIG. FIG. 2 is a block diagram illustrating a hardware configuration of the digital camera 1 to which the audio signal processing device according to the present embodiment is applied.

  The digital camera 1 according to the present embodiment is, for example, an imaging device that can record audio together with moving images during moving image imaging. The digital camera 1 captures an image of a subject, converts a captured image (either a still image or a moving image) obtained by the imaging into digital image data, and records the image together with sound on a recording medium.

  As shown in FIG. 2, the digital camera 1 according to the present embodiment schematically includes an imaging unit 10, an image signal processing unit 20, a display unit 30, a recording medium 40, a sound collection unit 50, An audio signal processing unit 60, a control unit 70, and an operation unit 80 are provided.

  The imaging unit 10 images a subject and outputs an analog image signal representing the captured image. The imaging unit 10 includes an imaging optical system 11, an imaging element 12, a timing generator 13, and a driving device 14.

  The imaging optical system 11 includes various lenses such as a focus lens, a zoom lens, and a correction lens, and optical components such as an optical filter that removes unnecessary wavelengths, a shutter, and a diaphragm. An optical image (subject image) incident from a subject is imaged on the exposure surface of the image sensor 12 via each optical component in the imaging optical system 11. The image pickup device 12 (image sensor) is configured by a solid-state image pickup device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), for example. The image pickup device 12 photoelectrically converts the optical image guided from the image pickup optical system 11 and outputs an electric signal (analog image signal) representing the picked-up image.

  A driving device 14 for driving the optical components of the imaging optical system 11 is mechanically connected to the imaging optical system 11. The driving device 14 includes, for example, a zoom motor 15, a focus motor 16, a diaphragm mechanism (not shown), and the like. The drive device 14 drives the optical components of the imaging optical system 11 according to an instruction from the control unit 70 described later, and moves the zoom lens and the focus lens or adjusts the diaphragm. For example, the zoom motor 15 performs a zoom operation for adjusting the angle of view by moving the zoom lens in the tele / wide direction. Further, the focus motor 16 performs a focus operation for focusing on the subject by moving the focus lens.

  Further, the timing generator 13 (hereinafter referred to as TG 13) generates an operation pulse necessary for the image sensor 12 in accordance with an instruction from the control unit 70. For example, the TG 13 generates various pulses such as a four-phase pulse for vertical transfer, a field shift pulse, a two-phase pulse for horizontal transfer, and a shutter pulse, and supplies them to the image sensor 12. By driving the image sensor 12 by the TG 13, a subject image is captured. Further, the exposure amount and the exposure period of the captured image are controlled by the TG 13 adjusting the shutter speed of the image sensor 12 (electronic shutter function). The image signal output from the image sensor 12 is input to the image signal processing unit 20.

  The image signal processing unit 20 is configured by an electronic circuit such as a microcontroller, performs predetermined image processing on the image signal output from the image sensor 12, and displays the image signal after the image processing on the display unit 30 or the control unit. Output to 70. The image signal processing unit 20 includes an analog signal processing unit 21, an analog / digital (A / D) conversion unit 22, and a digital signal processing unit 23.

  The analog signal processing unit 21 is a so-called analog front end that preprocesses an image signal. The analog signal processing unit 21 performs, for example, CDS (correlated double sampling) processing, gain processing using a programmable gain amplifier (PGA), and the like on the image signal output from the image sensor 12. The A / D conversion unit 22 converts the analog image signal input from the analog signal processing unit 21 into a digital image signal and outputs the digital image signal to the digital signal processing unit 23. The digital signal processing unit 23 performs, for example, digital signal processing such as noise removal, white balance adjustment, color correction, edge enhancement, and gamma correction on the input digital image signal, and the display unit 30 and the control unit 70. Etc.

  The display unit 30 includes, for example, a display device such as a liquid crystal display (LCD) or an organic EL display. The display unit 30 displays various input image data under the control of the control unit 70. For example, the display unit 30 displays a captured image (through image) input in real time from the image signal processing unit 20 during imaging. Accordingly, the user can operate the digital camera 1 while viewing the through image being captured by the digital camera 1. Further, when the captured image recorded on the recording medium 40 is reproduced, the display unit 30 displays the reproduced image. Thereby, the user can confirm the content of the captured image recorded on the recording medium 40.

  The recording medium 40 stores various types of data such as the captured image data, audio data, and metadata thereof. As the recording medium 40, for example, a semiconductor memory such as a memory card or a disk-shaped recording medium such as an optical disk or a hard disk can be used. The optical disc includes, for example, a Blu-ray Disc, a DVD (Digital Versatile Disc), a CD (Compact Disc), and the like. The recording medium 40 may be built in the digital camera 1 or a removable medium that can be attached to and detached from the digital camera 1.

  The sound collection unit 50 collects external sound around the digital camera 1. The sound collection unit 50 according to the present embodiment is a monaural microphone composed of one external sound recording microphone 51, but may be composed of a stereo microphone composed of two microphones. The microphone 51 outputs a sound signal obtained by collecting external sound. The sound collecting unit 50 collects external sound during moving image capturing and can record it together with the moving image. The microphone 51 is provided in the housing of the digital camera 1 to pick up external sound (desired sound), but the mechanical drive sound of the sound generation unit (the driving device 14) provided in the housing is also included. Sound is picked up as noise.

  The audio signal processing unit 60 is configured by an electronic circuit such as a microcontroller, performs predetermined audio processing on the audio signal, and outputs an audio signal for recording. This voice processing includes, for example, AD conversion processing and noise reduction processing. The present embodiment is characterized by noise reduction processing by the audio signal processing unit 60, and the detailed description thereof will be described later.

  The control unit 70 is configured by an electronic circuit such as a microcontroller, and controls the entire operation of the digital camera 1. The control unit 70 includes, for example, a CPU 71, an EEPROM (Electrically Erasable Programmable ROM) 72, a ROM (Read Only Memory) 73, and a RAM (Random Access Memory) 74. The control unit 70 controls each unit in the digital camera 1. For example, the control unit 70 controls the operation of the audio signal processing unit 60 to reduce the mechanical sound generated by the driving device 14 as noise from the audio signal collected by the microphone 51.

  The ROM 73 in the control unit 70 stores programs for causing the CPU 71 to execute various control processes. The CPU 71 operates based on the program and executes the necessary calculation / control processing for each control described above while using the RAM 74. The program can be stored in advance in a storage device (for example, EEPROM 72, ROM 73, etc.) built in the digital camera 1. Further, the program may be stored in a removable recording medium such as a disk-shaped recording medium or a memory card and provided to the digital camera 1 or downloaded to the digital camera 1 via a network such as a LAN or the Internet. Also good.

  Here, a specific example of control by the control unit 70 will be described. The control unit 70 controls the TG 13 and the driving device 14 of the imaging unit 10 to control the imaging process by the imaging unit 10. For example, the control unit 70 performs automatic exposure control (AE function) by adjusting the aperture of the imaging optical system 11, setting the electronic shutter speed of the imaging device 12, setting the AGC gain of the analog signal processing unit 21, and the like. Further, the control unit 70 moves the focus lens of the imaging optical system 11 and changes the focus position, thereby performing autofocus control for automatically focusing the imaging optical system 11 on a specific subject. (AF function). The control unit 70 adjusts the angle of view of the captured image by moving the zoom lens of the imaging optical system 11 and changing the zoom position. In addition, the control unit 70 records various data such as captured images and metadata on the recording medium 40, and reads and reproduces data recorded on the recording medium 40. Further, the control unit 70 generates various display images to be displayed on the display unit 30 and controls the display unit 30 to display the display image.

  The operation unit 80 and the display unit 30 function as a user interface for the user to operate the operation of the digital camera 1. The operation unit 80 includes various operation keys such as buttons and levers, or a touch panel, and includes, for example, a zoom button, a shutter button, and a power button. The operation unit 80 outputs instruction information for instructing various imaging operations to the control unit 70 in accordance with a user operation.

[1.2.2. Functional configuration of audio signal processing apparatus]
Next, a functional configuration example of the audio signal processing device applied to the digital camera 1 according to the present embodiment will be described with reference to FIG. FIG. 2 is a block diagram showing a functional configuration of the audio signal processing apparatus 100 according to the present embodiment.

  As shown in FIG. 3, the audio signal processing apparatus 100 includes a signal input unit 110, an input / output buffer memory 120 (first buffer memory), an interpolation buffer memory 130 (second buffer memory), noise, and the like. A detection unit 140, a noise reduction unit 150, and a signal output unit 160 are provided. The signal input unit 110 includes the microphone 51 of FIG. The noise reduction unit 150 includes an interpolation signal generation unit 152 and a signal interpolation unit 154. Further, the input / output buffer memory 120, the interpolation buffer memory 130, the noise detection unit 140, and the noise reduction unit 150 are the same as those shown in FIG. 2 audio signal processing units 60 are configured.

  Each unit of the audio signal processing apparatus 100 may be configured by dedicated hardware or software. When software is used, the processor of the audio signal processing device 100 may execute a program for realizing the functions of the functional units described below. The program may be provided to the audio signal processing apparatus 100 via a computer-readable recording medium (for example, an optical disc, a hard disk, a semiconductor memory, etc.), or may be provided via various communication means. . Below, each part of the audio | voice signal processing apparatus 100 is demonstrated.

  The signal input unit 110 includes a microphone 51 installed in the housing of the digital camera 1, an AD conversion unit (not shown), and the like. In the signal input unit 110, the microphone 51 collects a desired sound (sound to be recorded) around the digital camera 1, converts the external sound into a sound signal, and outputs the sound signal. In this audio signal, not only the desired sound but also noise such as pulse mechanical sound generated by the driving device 14 of the digital camera 1 and other mechanical driving sound are mixed. An AD converter (not shown) converts the analog audio signal output from the microphone 51 into a digital audio signal and outputs the digital audio signal.

  The input / output buffer memory 120 (first buffer memory) and the interpolation buffer memory 130 (second buffer memory) function as a signal holding unit that temporarily stores the audio signal input from the microphone and the generated interpolation signal. To do. As described above, the audio signal processing apparatus 100 according to the present embodiment includes two buffer memories, and processes audio signals for each predetermined section (that is, in units of frames) using the two buffer memories. Reduce noise. In the present embodiment, the input / output buffer memory 120 and the interpolation buffer memory 130 are connected in parallel between the signal input unit 110 and the signal output unit 160, whereby the audio signals of the two sections are paralleled. Can be processed.

  In order for the audio signal processing apparatus 100 to input / output and process audio signals in units of frames, the output buffer memory 120 temporarily stores one frame of the currently input audio signal. The interpolation buffer memory 130 holds an audio signal input in the past for one frame in order to interpolate the noise section. The memory lengths of these two buffer memories are the same. For example, each buffer memory can store a digital audio signal (number of sample data N) for one frame. Therefore, the length of the buffer memory included in the audio signal processing apparatus 100 is 2 * N. The input / output buffer memory 120 and the interpolation buffer memory 130 may be configured by two physically separated buffer memories, or may be configured by physically separating the storage areas of one buffer memory. May be.

  The input / output buffer memory 120 temporarily stores the audio signal input from the signal input unit 110 for each predetermined section (for example, one frame at a time). The input / output buffer memory 120 outputs the audio signal of one frame when the storage of all the frames of the input audio signal is completed. Thus, the audio signal input from the signal input unit 110 is sequentially stored in the input / output buffer memory 120 frame by frame and then output to the signal output unit 160.

  The audio signal of one frame output from the input / output buffer memory 120 is temporarily stored in the interpolation buffer memory 130. In other words, the interpolation buffer memory 130 stores the previous frame (n−1th frame) before the audio signal of the current frame (nth frame) stored in the input / output buffer memory 120. Temporarily save the audio signal. Therefore, while the audio signal of the nth frame input from the signal input unit 110 is being accumulated in the input / output buffer memory 120, the audio signal of the (n-1) th frame is stored in the interpolation buffer memory 130. Is saved. By these two buffer memories, the audio signal for two frames is always held in the audio signal processing apparatus 100.

  The noise detection unit 140 detects a section (noise section) in which noise such as pulse mechanical sound is included in the audio signal input from the signal input unit 110. The noise detection unit 140 detects whether or not the audio signal in the predetermined section stored in the input / output buffer memory 120 includes noise. If the noise is included, the noise detection section 140 determines that the section is a noise section. To do. When the noise detection unit 140 detects a noise interval, the noise detection unit 140 notifies the noise reduction unit 150 of information representing the interval.

  For example, when the noise is a pulse mechanical sound, the noise detection unit 140 detects a section in which the driving device 14 is operating as a noise section. The noise detection unit 140 can detect the operation period (noise interval) of the drive device 14 from the control information by acquiring the control information of the drive device 14.

  In addition, the noise detection unit 140 may analyze the actual speech signal input from the signal input unit 110 and extract a noise feature amount, thereby determining the presence or absence of noise and detecting the noise period. For example, since the pulse mechanical sound includes characteristic components such as a pulse component and a reverberation component, the presence or absence of the pulse mechanical sound can be accurately detected if these two types of components can be detected. Therefore, the noise detection unit 140, from the audio signal output from the microphone 51, features representing the pulse component of the pulse mechanical sound (for example, the maximum amplitude A of the pulse component, the pulse width W), the reverberation of the pulse mechanical sound. A feature amount representing a component (for example, a power value P of a narrowband signal representing a reverberation component of a pulse mechanical sound and the number of zero crossings M of the narrowband signal) is extracted. Then, the noise detection unit 140 determines whether or not a pulse mechanical sound is included in the audio signal based on the feature amount (the maximum amplitude value A, the pulse width W, the reverberation component power value P, etc.) representing the pulse mechanical sound. judge. For example, the noise detection unit 140 comprehensively determines the presence / absence of a pulse mechanical sound in the audio signal using the feature amount and a predetermined determination coefficient by a determination method using a statistical identification method or a table determination. Thus, it is possible to determine whether or not the pulse signal is included in the audio signal, and to specify the section in which the pulse signal is included in the audio signal.

  The noise reduction unit 150 performs noise reduction processing on the audio signal according to the detection result by the noise detection unit 140, and removes noise such as pulse mechanical sound from the audio signal. Specifically, when it is determined that noise such as pulse mechanical sound is included in the audio signal in the section stored in the input / output buffer memory 120, the noise reduction unit 150 includes the pulse mechanical sound. Noise reduction processing is performed on the audio signal in the section. On the other hand, when it is determined that no pulse mechanical sound is included, the noise reduction unit 150 does not perform noise reduction processing. Thus, only when pulse mechanical sound is included, by performing noise reduction processing on the audio signal in the section (noise section) including the pulse mechanical sound, the processing efficiency of the noise reduction processing is improved, Unnecessary processing load can be reduced.

  As a noise reduction method, the noise reduction unit 150 estimates a signal waveform of a background sound in the noise section from a signal in a section before or after the noise section, and interpolates a signal in the noise section using the estimated signal. use. In order to execute this interpolation method, the noise reduction unit 150 includes an interpolation signal generation unit 152 and a signal interpolation unit 154.

  The interpolation signal generation unit 152 generates an interpolation signal for interpolating the noise interval using the signal in the interval before the noise interval. This interpolation signal generation process is executed when it is detected that the audio signal of the current frame (n-th frame) stored in the input / output buffer memory 120 contains noise. At this time, the interpolation signal generation unit 152 is currently stored in the input / output buffer memory 120 using the audio signal of the previous frame (n−1 frame) stored in the interpolation buffer memory 130. An interpolated signal for interpolating the audio signal in the noise section is generated.

  Here, an example of the method of generating the interpolation signal will be described with reference to FIGS. 4 and 5 are conceptual diagrams showing a method for generating an interpolated signal from the input speech signal before the noise section according to the present embodiment.

(A) Simple Generation Method As shown in the upper part of FIG. 4, s (n) = {s ₀ , s ₁ ,..., S _{N -1} }. Here, s ₀ , s ₁ ,..., S _N−1 indicate the values of N sample data in the one frame. When generating the interpolated signal V (n) from the audio signal s (n), for example, as shown in the middle part of FIG. 4, the audio signal s (n) is inverted in the time axis direction and the interpolated signal v (n ) = {S _N−1 , s _N−2 ,..., S ₁ , s ₀ } may be generated. Further, as shown in the lower part of FIG. 4, the audio signal s (n) is inverted in the time axis direction and the amplitude direction, and the interpolation signal v (n) = {− s _N−1 , −s _N−2 ,. .., -S ₁ , -s ₀ } may be generated.

(B) Generation Method Using Window FIG. 5 shows another interpolation signal generation method. As shown in FIG. 5, a more natural interpolation signal v (n) is obtained by synthesizing signals p (n) and q (n) obtained by multiplying an audio signal s (n) by an appropriate window w (n). It can also be generated. Here, as the window w (n), a Hanning window or a bar and a let window can be used. More specifically, as shown in FIG. 5, first, the window w (n) = {w ₀ , w _{1 in the} audio signal s (n) = {s ₀ , s ₁ ,..., S _N−1 }. ,..., W _{N −1} } to generate a signal p (n) = {s ₀ w ₀ , s ₁ w ₁ ,..., S _N−1 w _N−1 }. Next, the signal p (n) is inverted in the time axis direction to generate signals q (n) = {s _N−1 w _N−1 ,..., S ₁ w ₁ , s ₀ w ₀ }. Then, the signal p (n) and the signal q (n) are added, and the interpolation signal v (n) = p (n) + q (n) = {s ₀ w ₀ + s _N−1 w _N−1 , s ₁ w ₁ + s _N−2 w _N−2 ,..., s _N−1 w _N−1 + s ₀ w ₀ } are generated. Alternatively, the signal q (n) is subtracted from the signal p (n), and the interpolation signal v (n) = p (n) −q (n) = {s ₀ w ₀ −s _N−1 w _N−1 , _{s 1 w 1 -s N-2} w N-2, ···, and generates an _{s N-1 w N-1} -s 0 w 0}. In this way, a more natural interpolation signal v (n) can be generated from the audio signal s (n).

  With reference to FIG. 3 again, description of each part of the audio signal processing apparatus 100 will be continued. As illustrated in FIG. 3, the signal interpolation unit 154 uses the interpolation signal generated by the interpolation signal generation unit 152 to generate an nth frame audio signal (noise interval) stored in the input / output buffer memory 120. Audio signal).

  For example, the signal interpolation unit 154 sets all amplitude values (that is, N sample data) of the audio signal in the noise section stored in the input / output buffer memory 120 to zero, and then overwrites the interpolation signal as it is. By doing so, the interpolation process may be executed. By this interpolation processing, the sound signal in the nth section including noise is replaced with the interpolation signal and output. Alternatively, the signal interpolation unit 154 may perform the interpolation process by synthesizing the speech signal in the noise interval stored in the input / output buffer memory 120 and the interpolation signal with an appropriate mixing ratio. By this interpolation processing, the audio signal in the noise section is output after noise is reduced.

  As a result of the interpolation processing by the signal interpolation unit 154, a voice signal interpolated with the interpolation signal is output instead of the voice signal of the input noise section, so that noise included in the noise section is reduced / removed. can do.

  The signal output unit 160 outputs the audio signal output from the input / output buffer memory 120 to the outside frame by frame. When noise reduction processing is performed by the noise reduction unit 150, the signal output unit 160 outputs an audio signal with reduced noise. For example, the signal output unit 160 may output the audio signal to a signal recording unit (configured by the control unit 70 and the recording medium 40 in FIG. 2), or an audio output such as a speaker or headphones. May be output to a unit (not shown). When an audio signal is output to the signal recording unit, the audio signal with reduced noise is recorded on a recording medium (not shown). The recording medium may be any recording medium such as a magnetic recording medium such as a hard disk or a magnetic tape, an optical recording medium such as a DVD or a Blu-ray disc, a semiconductor memory such as a flash memory or a USB memory.

[1.3. Operation of audio signal processing apparatus]
Next, the operation of the audio signal processing apparatus 100 according to the present embodiment will be described. In the following, an example of normal operation without noise and an example of operation when noise occurs will be described.

[1.3.1. Example of normal operation without noise]
First, with reference to FIG. 6, the operation of the audio signal processing apparatus 100 during normal time without noise will be described. FIG. 6 is a schematic diagram showing the normal operation of the audio signal processing apparatus 100 according to the present embodiment.

  As shown in FIG. 6, during normal times when noise is not generated, the audio signal input from the microphone 51 is temporarily stored in the input / output buffer memory 120 and the interpolation buffer memory 130 sequentially in units of frames. The frame stored in the interpolation buffer memory 130 is a frame one previous (past) before the frame stored in the input / output buffer memory 120. For example, as shown in FIG. 6A, when the audio signal s (n) of the nth frame is newly input and stored in the input / output buffer memory 120, it is input by one frame in the past. The audio signal s (n−1) of the (n−1) th frame is stored in the interpolation buffer memory 130.

  Then, as soon as all the audio signals s (n) of the nth frame have been accumulated in the input / output buffer memory 120, n stored in the input / output buffer memory 120 as shown in FIG. 6B. The audio signal s (n) of the second frame is output to the outside, and the data in the input / output buffer memory 120 is erased. At this time, since no noise is detected, the audio signal s (n) is output as it is without performing any special processing on the audio signal s (n) of the nth frame. The audio signal s (n) is copied to the interpolation buffer memory 130 together with the output of the audio signal s (n). This is because, when noise is detected in the audio signal s (n + 1) of the n + 1th frame that is input next, the n + 1th frame from the audio signal s (n) of the nth frame in the interpolation buffer memory 130. This is to generate the interpolation signal v (n + 1) for the frames.

[1.3.2. Example of operation when noise occurs]
Next, the operation of the audio signal processing apparatus 100 when noise occurs will be described with reference to FIG. FIG. 7 is a schematic diagram illustrating an operation example when noise is generated in the audio signal processing apparatus 100 according to the present embodiment.

  As shown in FIG. 7, even when noise (for example, pulse mechanical sound) is included in the input audio signal, the audio signal input from the microphone 51 is sequentially input / output buffer memory 120 in units of frames. , Temporarily stored in the interpolation buffer memory 130. As shown in FIG. 7A, when an audio signal s (n) of the nth frame including noise is newly input and stored in the input / output buffer memory 120, the previous n−1 is stored by one frame. The audio signal s (n−1) of the second frame is temporarily stored in the interpolation buffer memory 130.

  When it is detected that all of the audio signal s (n) of the nth frame has been accumulated in the input / output buffer memory 120 and noise is included in the audio signal s (n), FIG. The interpolation process shown in FIG. That is, as illustrated in FIG. 7B, the interpolation signal generation unit 152 generates a noise interval (nth frame) from the n−1th frame audio signal s (n−1) stored in the interpolation buffer memory 130. ) To generate an interpolated signal v (n) for interpolating the audio signal s (n). The method of generating the interpolation signal v (n) is as described above (see FIGS. 4 and 5). In the example of FIG. 6B, the interpolation signal v (n) is generated by inverting the audio signal s (n−1) of the (n−1) th frame in the time axis direction. Then, the signal interpolation unit 154 deletes the audio signal s (n) of the nth frame stored in the input / output buffer memory 120, and the interpolated signal v (n) is stored in the input / output buffer memory 120. save.

  Next, as illustrated in FIG. 7C, the signal interpolation unit 154 converts the interpolation signal v (n) stored in the input / output buffer memory 120 into the audio signal s of the nth frame actually input in FIG. 7A. Instead of (n), the data is output to the outside, and the data in the input / output buffer memory 120 is erased. Further, the signal interpolation unit 154 copies the interpolation signal v (n) to the interpolation buffer memory 130 together with the output of the interpolation signal v (n). This is because interpolation is performed for the (n + 1) th frame from the interpolation signal v (n) in the interpolation buffer memory 130 when noise is detected in the audio signal s (n + 1) of the (n + 1) th frame to be input next. This is because the signal v (n + 1) is generated.

  As described above, when noise is included in the audio signal s (n) of the nth frame, the audio signal s (n-1) of the (n-1) th frame is used to reduce the noise. Then, an interpolation signal v (n) is generated and interpolation processing is executed. By this interpolation processing, the interpolation signal v (n) not including noise is output to the outside instead of the input speech signal s (n) of the nth frame including the noise, so that noise can be suitably removed.

  When the audio signal is input and output in frame units as described above for interpolation processing, the memory lengths of the input / output buffer memory 120 and the interpolation buffer memory 130 may each be the number N of sample data for one frame. Therefore, the buffer memory length required for the entire apparatus is 2 * N. Further, since the interpolation signal v (n) can be generated and output to the outside immediately after the completion of the accumulation of the audio signal s (n) in the noise section in the input / output buffer memory 120, the output audio for the input audio can be output. The delay is zero.

[1.4. Audio signal processing method]
Next, an audio signal processing method (mechanical sound reduction method) using the audio signal processing apparatus 100 will be described with reference to FIG. FIG. 8 is a flowchart showing an audio signal processing method according to the present embodiment.

  During imaging and recording by the digital camera 1 including the audio signal processing apparatus 100 according to the present embodiment, surrounding external audio is collected by the microphone 51 and an audio signal is output. Then, the audio signal processing apparatus 100 converts the analog audio signal input from the microphone 51 into a digital audio signal, and processes the digital audio signal in units of frames. That is, the audio signal processing apparatus 100 stores the input audio signal frame by frame in the input / output buffer memory 120, and stores the audio signal of the frame immediately before the currently input frame in the interpolation buffer memory 130. To do. Then, the audio signal processing apparatus 100 detects the presence or absence of noise in units of frames, and when noise is detected, performs an interpolation process on the frame using the signal of the previous frame. FIG. 8 shows a detailed flow of this processing.

  As shown in FIG. 8, first, the audio signal processing apparatus 100 determines whether or not an audio signal for one frame input from the microphone 51 has been accumulated in the input / output buffer memory 120 (S100). Here, processing when the audio signal s (n) of the nth frame is currently being input will be described. As a result of the determination of S100, when the audio signal s (n) of the nth frame is completely stored in the input / output buffer memory 120, the noise detection unit 140 immediately determines whether the audio signal s (n) includes noise. Whether or not is detected (S102).

  If noise is detected as a result of the noise determination in S102, an interpolation process (see FIG. 7) is immediately executed. That is, the interpolation signal generation unit 152 uses the audio signal s (n−1) of the (n−1) th frame (1 frame past) stored in the interpolation buffer memory 130. The interpolation signal v (n) is generated (S104). Then, the signal interpolation unit 154 interpolates the audio signal s (n) of the nth frame including noise using the interpolation signal v (n) generated in S104, and inputs / outputs the interpolation signal v (n). Is stored in the buffer memory 120 (S106). In the interpolation processing of S106, the noise signal s (n) of the nth frame including noise may be replaced with the interpolation signal v (n), or the sound signal s (n) and the interpolation signal v (n) may be replaced. May be synthesized at an appropriate mixing ratio. Below, the substituted example is demonstrated.

  Next, the signal interpolation unit 154 copies the noise-reduced interpolation signal v (n) (corresponding to the nth frame) stored in the input / output buffer memory 120 to the interpolation buffer memory 130 ( In step S108, the interpolation signal v (n) is output to the signal output unit 160 (S110).

  On the other hand, if no noise is detected as a result of the noise determination in S102, the input nth frame audio signal s (n) is output as it is without performing the interpolation processing in S108 and S110. That is, the signal interpolation unit 154 copies the audio signal s (n) of the nth frame stored in the input / output buffer memory 120 to the interpolation buffer memory 130 (S108), and the audio signal s (n ) As it is from the input / output buffer memory 120 to the signal output unit 160 (S110).

  Thereafter, until the imaging and recording operations by the digital camera 1 are completed (S112), the processes of S100 to S100 are repeated for the audio signal s (n + 1) of the next frame of the input audio signal. As a result, noise detection processing is performed on the input audio signal for each frame, and if necessary, interpolation processing (noise reduction processing) is performed, and then a noise-free audio signal is output in frame units. The

[1.5. effect]
The configuration of the audio signal processing apparatus 100 according to the first embodiment of the present disclosure and the audio signal processing method using the same have been described above. According to the present embodiment, immediately after noise is detected in the audio signal s (n) of the frame input from the microphone 51 and being stored in the input / output buffer memory 120, it is stored in the interpolation buffer memory 130 in advance. The interpolated signal v (n) is generated using only the audio signal s (n−1) that is past one frame. Then, using the interpolated signal v (n), the speech signal s (n) in the noise section is interpolated and the interpolated speech signal is output.

  As a result, the buffer memory used for input / output of the audio signal can be effectively used for the interpolation processing, so that the buffer memory length necessary for estimating the interpolation signal can be shortened, and the buffer memory necessary for the entire apparatus can be reduced. That is, the memory lengths of the input / output buffer memory 120 and the interpolation buffer memory 130 may be the number N of sample data for each frame, so that the buffer memory length required for the entire apparatus is 2 * N. In the conventional interpolation method (see FIG. 1), since interpolation is performed using signals before and after the noise interval, a buffer memory length of at least 3 * N is required. On the other hand, in this embodiment, the buffer memory length may be 2 * N, and the buffer memory required for the interpolation processing can be greatly reduced.

  In addition, as described above, when there is aperiodic speech in which various sounds are mixed before and after the noise section, it is not necessary to align the periods before and after the interpolation of the noise section, and the generation of an uncomfortable sound Is unlikely to occur. Therefore, even when interpolation is performed using only the audio signal of the frame preceding the noise section, it is possible to remove noise substantially.

  Furthermore, according to the present embodiment, high-quality noise reduction processing with less delay can be realized by effectively controlling processing of an audio signal in units of frames by effectively using two buffer memories. That is, in the above conventional interpolation method (see FIG. 1), a delay of one frame occurs until the signal of the frame after the noise interval is completely stored in the buffer memory, and further, an interpolation signal is generated thereafter. Since a delay of one frame occurs, a delay of at least 2 * N (a delay of two frames) occurs.

  In contrast, in the interpolation processing according to the present embodiment, the (n + 1) th audio signal s (n−) before the noise interval is used without using the audio signal s (n + 1) of the (n + 1) th frame after the noise interval. The interpolation signal v (n) is generated using only 1). As a result, the interpolation process can be executed immediately after the accumulation of the audio signal s (n) of the nth frame, which is a noise interval, and the interpolated signal can be output, as in the conventional interpolation method. There is no need to wait for the interpolation process until the signal after the noise interval is accumulated. Accordingly, since the delay of the output sound with respect to the input sound can be made zero, the delay of the output sound accompanying the interpolation process can be greatly reduced as compared with the conventional case.

<2. Second Embodiment>
Next, an audio signal processing device and an audio signal processing method according to the second embodiment of the present disclosure will be described. The audio signal processing apparatus according to the second embodiment is characterized in that an interpolation signal is generated using signals before and after a noise interval, and interpolation processing is performed. The other functional configuration of the second embodiment is substantially the same as that of the first embodiment, and a detailed description thereof will be omitted.

[2.1. Outline of mechanical noise reduction method]
First, an outline of the mechanical sound reduction method according to the second embodiment will be described. In the first embodiment described above, the interpolation signal is generated using only the audio signal in the section (n−1th frame) before the noise section. On the other hand, in the second embodiment, not only the voice signal in the section (n−1th frame) before the noise section but also the voice signal in the section (n + 1th frame) after the noise section is used. To generate an interpolation signal and perform an interpolation process.

  Specifically, when noise is detected in the audio signal of the nth frame, a first temporary interpolation signal (front temporary interpolation signal) is generated from the audio signal of the (n−1) th frame, and the n + 1th frame. A second temporary interpolation signal (rear temporary interpolation signal) is generated from the audio signal of the frame. Then, the first temporary interpolation signal and the second temporary interpolation signal are combined to generate an interpolation signal, and the nth frame audio signal, which is a noise interval, is interpolated using the interpolation signal.

  Although the interpolation processing causes a delay of one frame in the output sound with respect to the input sound as compared with the first embodiment, the interpolation signal is generated by using the signals before and after the noise interval. Can be estimated with high accuracy. Therefore, higher quality noise reduction processing can be realized. Further, since the interpolation signal is efficiently generated by properly using the two buffer memories, it is possible to suppress the delay of the output sound with respect to the input sound to the maximum and to suppress it to one frame. The audio signal processing apparatus and method according to the second embodiment will be described in detail below.

[2.2. Functional configuration of audio signal processing apparatus]
Next, a functional configuration of the audio signal processing apparatus 100 according to the second embodiment will be described with reference to FIG. FIG. 9 is a block diagram showing a functional configuration of the audio signal processing apparatus 100 according to the second embodiment.

  As shown in FIG. 9, the audio signal processing apparatus 100 includes a signal input unit 110, an input buffer memory 122 (first buffer memory), an output buffer memory 132 (second buffer memory), and noise detection. Unit 140, noise reduction unit 150, and signal output unit 160. The input buffer memory 122, the output buffer memory 132, the noise detection unit 140, and the noise reduction unit 150 constitute the audio signal processing unit 60 of FIG. The signal input unit 110, the noise detection unit 140, and the signal output unit 160 according to the second embodiment have substantially the same functional configuration as that of the first embodiment, and thus detailed description thereof is omitted. .

  The audio signal processing apparatus 100 according to the second embodiment includes two buffer memories: an input buffer memory 122 and an output buffer memory 132. These buffer memories function as a signal holding unit that temporarily stores the audio signal input from the microphone and the generated interpolation signal. In the second embodiment, the input buffer memory 122 and the output buffer memory 132 are connected in series between the signal input unit 110 and the signal output unit 160.

  In order for the audio signal processing apparatus 100 to input / output and process audio signals in units of frames, the input buffer memory 122 temporarily stores one frame of the currently input audio signal, and the output buffer memory 132 stores the past. One frame of the audio signal input to is temporarily stored. The memory lengths of these two buffer memories are the same. For example, each buffer memory can store a digital audio signal (number of sample data N) for one frame. Therefore, the length of the buffer memory included in the audio signal processing apparatus 100 is 2 * N. The input buffer memory 122 and the output buffer memory 132 may be configured by two physically separated buffer memories, or may be configured by physically separating a storage area of one buffer memory. May be.

  The input buffer memory 122 temporarily stores the audio signal input from the signal input unit 110 at predetermined intervals (for example, one frame at a time). The input buffer memory 122 outputs the audio signal of one frame when the storage of all the frames of the input audio signal is completed.

  One frame of the audio signal output from the input buffer memory 122 is temporarily stored in the output buffer memory 132. That is, the output buffer memory 132 stores the audio signal (n−1) of the previous frame immediately before the audio signal of the current frame (the audio signal of the nth frame) stored in the input buffer memory 122. The audio signal of the second frame) is temporarily saved. Therefore, while the audio signal of the nth frame input from the signal input unit 110 is being accumulated in the input buffer memory 122, the audio signal of the (n-1) th frame is stored in the output buffer memory 132. It will be saved. The output buffer memory 132 outputs the audio signal of one frame to the signal output unit 160 when the storage of one frame of the audio signal input from the input buffer memory 122 is completed.

  Thus, the audio signal input from the signal input unit 110 is temporarily stored in the input buffer memory 122 and the output buffer memory 132 sequentially for each frame, and then output to the signal output unit 160. By these two buffer memories, the audio signal for two frames is always held in the audio signal processing apparatus 100.

  Next, the noise reduction unit 150 according to the second embodiment will be described. The noise reduction unit 150 includes an interpolation signal generation unit 152, a signal interpolation unit 154, a first temporary interpolation signal generation unit 156, and a second temporary interpolation signal generation unit 157.

  When noise is detected in the audio signal of the nth frame by the noise detection unit 140, the first temporary interpolation signal generation unit 156 stores the audio signal of the (n−1) th frame stored in the output buffer memory 132. To generate a first temporary interpolation signal. The first temporary interpolation signal is a temporary interpolation signal generated from the input speech signal in the section before the noise section. Thus, the first temporary interpolation signal generation unit 156 immediately follows the noise interval (n-th frame) immediately after the noise interval (n-th frame) is stored in the input buffer memory 122. The first temporary interpolation signal for interpolating the noise section (nth frame) is generated from the voice signal.

  Thereafter, when the audio signal of the (n + 1) th frame is stored in the input buffer memory 122, the second temporary interpolation signal generation unit 157 stores the audio signal of the (n + 1) th frame stored in the input buffer memory 122. To generate a second temporary interpolation signal. The second temporary interpolation signal is a temporary interpolation signal generated from the input speech signal in the section after the noise section. As described above, the second temporary interpolation signal generation unit 156 immediately after the noise interval (n-th frame) is stored in the output buffer memory 132, immediately after the noise interval (n + 1-th frame). A second temporary interpolation signal for interpolating a noise interval (nth frame) is generated from the signal.

  Then, the interpolation signal generation unit 152 generates an interpolation signal from the first and second temporary interpolation signals. The signal interpolation unit 154 uses the interpolation signal generated by the interpolation signal generation unit 152 to interpolate the nth frame audio signal (the audio signal in the noise interval) stored in the output buffer memory 132.

  For example, the signal interpolation unit 154 sets all amplitude values (that is, N pieces of sample data) of the audio signal in the noise section stored in the output buffer memory 132 to zero, and then overwrites the interpolation signal as it is. Thus, the interpolation process may be executed. By this interpolation processing, the sound signal in the nth section including noise is replaced with the interpolation signal and output. Alternatively, the signal interpolation unit 154 may perform the interpolation process by combining the speech signal in the noise section stored in the output buffer memory 132 and the interpolation signal with an appropriate mixing ratio. By the interpolation processing by the signal interpolation unit 154, a voice signal interpolated with the interpolation signal is output instead of the voice signal in the input noise section, so that noise included in the noise section is reduced / removed. can do.

  Here, an example of the temporary interpolation signal and the method of generating the interpolation signal will be described.

(A) Simple generation method When the n-th frame is a noise section, for example, in the same manner as the interpolation signal generation method shown in FIG. 4 or FIG. A first temporary interpolation signal p (n) is generated from (n−1), and a second temporary interpolation signal q (n) is generated from the input speech signal s (n + 1) of the (n + 1) th frame. Then, as shown in the following formula (1), the first temporary interpolation signal p (n) and the second temporary interpolation signal q (n) are used with a predetermined mixing count α (0 <α <1). By mixing, the interpolation signal v (n) is generated.

  v (n) = α · p (n) + (1−α) · q (n) (1)

  For example, by setting α = 0.5, the first temporary interpolation signal p (n) and the second temporary interpolation signal q (n) can be evenly mixed to generate the interpolation signal v (n). Further, when it is desired to bias the weighting of p (n) or q (n), the numerical value of α may be adjusted. With the generation method as described above, it is possible to generate an interpolation signal v (n) with high interpolation accuracy using the audio signals in the sections before and after the noise section.

(B) Generation Method Using Window FIG. 10 is a conceptual diagram showing another method for generating a temporary interpolation signal and an interpolation signal from input speech signals before and after the noise interval according to this embodiment.

In the same manner as the interpolation signal generation method shown in FIG. 4 or FIG. 5, the first temporary interpolation signal p (n) is generated from the input audio signal s (n−1) of the (n−1) th frame, A second temporary interpolation signal q (n) is generated from the input audio signal s (n + 1) of the (n + 1) th frame. Then, as shown in FIG. 10, the first temporary interpolation signal p (n) and the second temporary interpolation signal q (n) generated as described above are converted into an arbitrary window w ₁ such as a Hanning window or a Bartlett window. Mix using (n), w ₂ (n). Specifically, first, the temporary interpolation signals p (n) and q (n) are multiplied by the windows w ₁ (n) and w ₂ (n), respectively, to obtain the signals t (n) and u (n). Generate. Next, the signal t (n) and the signal u (n) are combined to generate an interpolation signal v (n). For example, the signal t (n) and the signal u (n) may be added to generate the interpolation signal v (n) = p (n) + q (n), or the signal p (n) The interpolated signal v (n) = p (n) −q (n) may be generated by subtracting q (n). By such a method, it is possible to generate a more natural interpolation signal v (n) from the temporary interpolation signals p (n) and q (n).

[2.3. Operation of audio signal processing apparatus]
Next, the operation of the audio signal processing apparatus 100 according to the second embodiment will be described. In the following, an example of normal operation without noise and an example of operation when noise occurs will be described.

[2.3.1. Example of normal operation without noise]
First, with reference to FIG. 11, the operation of the audio signal processing apparatus 100 in a normal time without noise will be described. FIG. 11 is a schematic diagram illustrating the normal operation of the audio signal processing apparatus 100 according to the second embodiment.

  As shown in FIG. 11, during normal times when noise is not generated, the audio signal input from the microphone 51 is temporarily stored in the input buffer memory 122 and the output buffer memory 132 sequentially in units of frames. The audio signal of the frame stored in the output buffer memory 132 is a frame one previous (past) before the audio signal of the frame currently stored in the input buffer memory 122. For example, as shown in FIG. 11A, when an audio signal of the nth frame is newly input and stored in the input buffer memory 122, the (n−1) th frame that has been input in the past by one frame is currently stored. The audio signal s (n−1) of the frame is stored in the output buffer memory 132.

  Then, as soon as all the audio signals s (n) of the nth frame have been accumulated in the input buffer memory 122, as shown in FIG. 11B, the nth frame stored in the output buffer memory 132. Audio signal s (n) is output to the outside. At this time, since no noise is detected, the audio signal s (n−1) of the (n−1) th frame is output as it is. In addition to the output of the audio signal s (n−1), the audio signal s (n) of the nth frame stored in the input buffer memory 122 is copied to the output buffer memory 132 to be input buffer. Data in the memory 122 is erased. This is because, when noise is detected in the next input audio signal s (n + 1) of the (n + 1) th frame, the n + 1th frame from the audio signal s (n) of the nth frame in the output buffer memory 132. This is to generate the interpolation signal v (n + 1) for the frames.

  Thereafter, when the audio signal s (n + 1) of the (n + 1) th frame is newly input and all the audio signals s (n + 1) of the (n + 1) th frame are completely accumulated in the input buffer memory 122, the nth frame is immediately The frame audio signal s (n) is output from the output buffer memory 132. For this reason, the output sound is delayed by one frame with respect to the input sound (the delay amount is N).

[2.3.2. Example of operation when noise occurs]
Next, the operation of the audio signal processing apparatus 100 when noise is generated will be described with reference to FIGS. 12 and 13 are schematic diagrams illustrating an operation example when noise is generated in the audio signal processing apparatus 100 according to the present embodiment.

  As shown in FIG. 12, even when noise (for example, pulse mechanical sound) is included in the input audio signal, the audio signal input from the microphone 51 is sequentially input to the input buffer memory 122, in units of frames. It is temporarily stored in the output buffer memory 132. As shown in FIG. 12A, when the nth frame audio signal s (n) including noise is newly input and stored in the input buffer memory 122, the previous (n−1) th frame by one frame is stored. The audio signal s (n−1) of the frame is temporarily stored in the output buffer memory 132.

  When it is detected that all of the audio signal s (n) of the nth frame has been accumulated in the input buffer memory 122 and noise is included in the audio signal s (n), FIG. The first temporary interpolation signal generation process shown is immediately executed. That is, as shown in FIG. 12B, the first temporary interpolation signal generation unit 156 generates a noise interval (from the audio signal s (n−1) of the (n−1) th frame stored in the output buffer memory 132. A first temporary interpolation signal p (n) for interpolating the audio signal s (n) of the (nth frame) is generated. In the example of FIG. 12B, the first temporary interpolation signal p (n) is generated by inverting the audio signal s (n−1) of the (n−1) th frame in the time axis direction. Then, the first temporary interpolation signal generation unit 156 deletes the audio signal s (n) of the nth frame stored in the output buffer memory 132, and the first temporary interpolation signal p (n). Is stored in the output buffer memory 132.

  Next, as illustrated in FIG. 12C, the signal interpolation unit 154 outputs the audio signal s (n−1) of the (n−1) th frame stored in the output buffer memory 132 to the outside. Further, the signal interpolation unit 154 moves the first temporary interpolation signal p (n) stored in the input buffer memory 122 to the output buffer memory 132 together with the output of the audio signal s (n−1). Let This is because the first temporary interpolation signal p (n) is stored in the output buffer memory 132, and the temporary interpolation signal p (n (n) when the audio signal s (n + 1) of the (n + 1) th frame is input next. ) To generate the interpolation signal v (n).

  Next, as shown in FIG. 13A, when the audio signal s (n + 1) of the next (n + 1) th frame is newly input and stored in the input buffer memory 122, the first temporary interpolation signal p (n ) Is temporarily stored in the output buffer memory 132.

  When all of the audio signal s (n + 1) of the (n + 1) th frame has been accumulated in the input buffer memory 122, the second temporary interpolation signal generation process and the interpolation signal generation process shown in FIG. 13B are immediately executed. Is done. That is, as shown in FIG. 13B, the second temporary interpolation signal generation unit 157 generates a noise interval (nth frame) from the audio signal s (n + 1) of the (n + 1) th frame stored in the input buffer memory 122. ) Of the second temporary interpolation signal q (n) for interpolating the audio signal s (n). Then, as shown in FIG. 13B, the interpolation signal generation unit 152 generates the generated second temporary interpolation signal q (n) and the first temporary interpolation signal p (n) stored in the output buffer memory 132. ) And the interpolation signal v (n) is generated.

  Next, as shown in FIG. 13C, immediately after the generation of the interpolation signal v (n), the signal interpolation unit 154 replaces the audio signal s (n) of the nth frame actually input in FIG. The interpolation signal v (n) is output to the outside. Further, the signal interpolation unit 154 moves the audio signal s (n + 1) of the (n + 1) th frame stored in the input buffer memory 122 to the output buffer memory 132 together with the output of the interpolation signal v (n). . This is because the audio signal s (n + 1) is output from the output buffer memory 132 when the next input audio signal s (n + 2) of the (n + 2) th frame has been accumulated in the input buffer memory 122. . In addition, when noise is detected in the audio signal s (n + 2) of the (n + 2) th frame that is input next, the n + 2th frame signal is detected from the audio signal s (n + 1) in the output buffer memory 132. One temporary interpolation signal p (n + 1) can also be generated.

  As described above, according to the present embodiment, when noise is included in the audio signal s (n) of the nth frame, in order to reduce the noise, the n−1th and n + 1th frames The interpolation signal v (n) is generated using the audio signals s (n−1) and s (n + 1), and the interpolation process is executed. By this interpolation processing, the interpolation signal v (n) not including noise is output to the outside instead of the input speech signal s (n) of the nth frame including the noise, so that noise can be suitably removed. Furthermore, since interpolation is performed using audio signals before and after the noise section, more natural and highly accurate interpolation processing can be executed, so that high-quality noise reduction can be realized.

  Further, in the case where the audio signal is input / output in units of frames as described above and the interpolation processing is performed, the memory lengths of the input buffer memory 122 and the output buffer memory 132 may each be the number N of sample data of one frame. Therefore, as in the first embodiment, the buffer memory length required for the entire apparatus is 2 * N. Also, immediately after the audio signal s (n + 1) of the next frame is completely stored in the input buffer memory 122, the audio signal s (n) of the previous frame is output to the outside. The delay of the output voice with respect to is only one frame.

[2.4. Audio signal processing method]
Next, an audio signal processing method (mechanical sound reduction method) using the audio signal processing apparatus 100 will be described with reference to FIG. FIG. 14 is a flowchart showing an audio signal processing method according to the second embodiment.

  As shown in FIG. 14, first, the audio signal processing apparatus 100 determines whether or not an audio signal for one frame input from the microphone 51 has been accumulated in the input buffer memory 122 (S200). Here, processing when the audio signal s (n) of the nth frame is currently being input will be described. As a result of the determination in S200, when all of the audio signal s (n) of the nth frame is completely stored in the input buffer memory 122, the noise detection unit 140 immediately includes noise in the audio signal s (n). Is detected (S202).

  If noise is detected as a result of the determination in S202, the first temporary interpolation signal generation process shown in FIG. 12 is immediately executed. That is, the first temporary interpolation signal generation unit 156 uses the audio signal s (n−1) of the (n−1) th frame (one frame past) stored in the output buffer memory 132 to The temporary interpolation signal p (n) is generated (S204). Then, the first temporary interpolation signal generation unit 156 outputs the audio signal s (n−1) of the (n−1) th frame from the output buffer memory 132 to the signal output unit 160 as it is, and also performs the first temporary interpolation. The signal p (n) is stored in the output buffer memory 132 (S206).

  Next, the newly input audio signal s (n + 1) of the (n + 1) th frame is accumulated in the input buffer memory 122, and whether or not all the audio signals s (n + 1) have been accumulated in the input buffer memory 122. Determine (S210). As a result, when all of the audio signal s (n + 1) of the (n + 1) th frame is completely stored in the input buffer memory 122, the second temporary interpolation signal generation process and the interpolation process shown in FIG. .

  That is, the second temporary interpolation signal generation unit 157 generates the second temporary interpolation signal q (n) using the audio signal s (n + 1) of the (n + 1) th frame stored in the input buffer memory 122. (S214). Then, the interpolation signal generation unit 152 generates an interpolation signal from the first temporary interpolation signal p (n) stored in the output buffer memory 132 and the second temporary interpolation signal q (n) generated in S214. v (n) is generated (S216). Further, the signal interpolation unit 154 uses the interpolation signal v (n) generated in S216 to interpolate the noise signal s (n) of the nth frame including noise, and outputs the interpolated signal as an output buffer memory. It is stored in 132 (S218). In the interpolation processing in S218, the audio signal s (n) of the nth frame including noise may be replaced with the interpolation signal v (n), or the audio signal s (n) and the interpolation signal v (n) may be replaced. May be synthesized at an appropriate mixing ratio. Below, the substituted example is demonstrated.

  Thereafter, the signal interpolation unit 154 outputs the interpolation signal v (n) stored in the output buffer memory 132 in S218 to the signal output unit 160 in place of the audio signal s (n) of the nth frame ( S220). Then, the audio signal s (n) of the (n + 1) th frame stored in the input buffer memory 122 is moved to the output buffer memory 132.

  On the other hand, if no noise is detected in the sound signal s (n) of the nth frame as a result of the noise determination in S202, normal input / output processing is performed without performing the above interpolation processing. That is, as shown in FIG. 11, the audio signal s (n−1) of the (n−1) th frame is output from the output buffer memory 132 as it is, and is output to the signal output unit 160 and stored in the input buffer memory 122. The audio signal s (n) of the th frame is moved to the output buffer memory 132 (S208). Then, when all the audio signals s (n + 1) of the next n + 1th frame have been accumulated in the input buffer memory 122 (S210), the audio signal s (n) of the nth frame is output from the output buffer memory 132. Is directly output to the signal output unit 160 (S220), and the audio signal s (n) of the (n + 1) th frame stored in the input buffer memory 122 is moved to the output buffer memory 132.

  Thereafter, the processes of S200 to S220 are repeated for the audio signal s (n + 2) of the next frame of the input audio signal until the imaging and recording operation by the digital camera 1 is completed (S222). As a result, noise detection processing is performed on the input audio signal for each frame, and if necessary, interpolation processing (noise reduction processing) is performed, and then a noise-free audio signal is output in frame units. The

[2.5. effect]
The configuration of the audio signal processing apparatus 100 according to the second embodiment of the present disclosure and the audio signal processing method using the same have been described above. According to the second embodiment, the interpolated signal is generated using the audio signals s (n−1) and s (n + 1) before and after the noise interval, so that the background sound in the noise interval (external sound excluding noise) is generated. ) Can be estimated with high accuracy. Accordingly, the accuracy of the interpolation processing can be increased to reduce the noise and the background sound can be reproduced with high accuracy, so that the accuracy of the noise reduction processing can be greatly improved.

  Similarly to the first embodiment, the buffer memory used for signal input / output is also effectively used for interpolation processing, so that the buffer memory length required for interpolation signal estimation can be reduced, which is necessary for the entire apparatus. Buffer memory can be reduced. Also in the second embodiment, the buffer memory length required for estimation of the interpolation signal is only 2 * N. Therefore, the conventional interpolation method (see FIG. 1) requires a buffer memory length of at least 3 * N. Compared to the above, the buffer memory required for the interpolation process can be greatly reduced.

  Furthermore, according to the present embodiment, high-quality noise reduction processing with less delay can be realized by effectively controlling processing of audio signals in units of frames by effectively using two buffer memories. That is, in the conventional interpolation method (see FIG. 1), since interpolation processing is performed using signals before and after the noise interval, a delay of at least 2 * N (delay of 2 frames) occurs as described above. Was. On the other hand, according to the present embodiment, the interpolated signal v (n) is generated using the audio signals s (n−1) and s (n + 1) before and after the noise section, but the audio to the input buffer memory 122 is generated. The interpolation signal v (n) can be generated and output immediately when the accumulation of the signal s (n + 1) is completed. Thereby, since the delay of the output sound with respect to the input sound can be suppressed to one frame (delay amount: N), the delay of the output sound accompanying the interpolation process can be reduced to half that of the conventional interpolation method.

<3. Third Embodiment>
Next, an audio signal processing device and an audio signal processing method according to the third embodiment of the present disclosure will be described. The audio signal processing apparatus according to the third embodiment detects a noise start point and an end point, generates an interpolation signal using signals before and after the noise, and generates a signal from the noise start point to the end point. It is characterized in that an interpolation process is performed on it. The remaining functional configuration of the third embodiment is substantially the same as that of the second embodiment, and a detailed description thereof will be omitted.

[3.1. Outline of mechanical noise reduction method]
First, an outline of the mechanical sound reduction method according to the third embodiment will be described.

  In the first and second embodiments described above, as shown in FIG. 7 and the like, interpolation processing is performed in units of frames on the assumption that noise such as pulse mechanical sound is within one frame of the audio signal. . However, in practice, one noise does not necessarily fall within one frame of the audio signal, and there may be a case where one noise exists over two frames as shown in FIG. That is, in such a case, it is difficult to suitably reduce noise with the interpolation methods according to the first and second embodiments described above.

Therefore, in the third embodiment, the noise reference point is detected by the noise reference point detection unit, so that even if the noise exists across two frames, the signals before and after the noise are used. Noise is effectively reduced. The noise reference points are reference points indicating the position of noise in the speech signal, and are three noise start points P _S , noise intermediate points P _M , and noise end points P _E as shown in FIG. By detecting this noise reference point, interpolation processing can be realized in an arbitrary section of the audio signal in addition to the frame unit.

  Here, the relationship between the frame position of the audio signal and the noise position will be described in more detail. When only audio signal processing is considered, the position of the frame, that is, how to determine the number N of sample data in one frame is arbitrary. In general, since FFT (Fast Fourier Transform) is often used to convert an audio signal to the frequency domain, the number N of sample data is “256”, “512”, “1024”. Etc. are widely used. However, this is not the case when frequency conversion is not performed.

  On the other hand, in digital cameras, video cameras, etc., it is necessary to synchronize the audio signal with the system control clock and video signal (moving image) inside the camera. It is difficult to decide. Here, taking a long frame (that is, increasing N) leads to an increase in the delay of the camera system. Therefore, in reality, the number N of sample data is often about 100 to 2000.

  For the above reason, it is actually difficult to arbitrarily determine the number N of sample data of the frame of the audio signal in accordance with the time length (total time width) of the pulse mechanical sound.

  By the way, in general, a pulse mechanical sound is characterized by a shorter time length than other noises. For this reason, there is no problem even if the time length of the pulse mechanical sound is regarded as being equal to or shorter than the number N of sample data of the frame of the audio signal. Therefore, if the entire pulse mechanical sound is within one frame (see FIG. 7 and the like), the interpolation processing as in the first and second embodiments can be performed without any problem.

  However, in reality, there are more cases where the pulse mechanical sound is shifted from the boundary of the frame and exists across two frames (see FIG. 16). Therefore, apart from the frame boundaries set in the audio signal, a pulse mechanical sound break (reference point) is detected, and the pulse mechanical sound is interpolated using signals before and after the noise break position. It is preferable.

Therefore, in the third embodiment, when a frame of an audio signal including noise (for example, pulse mechanical sound) is input, the noise reference point (noise start point P _S , noise intermediate point P _M, and noise end point P). _E )) is detected, and the noise interval is specified regardless of the frame. Then, from the previous signal than noise start point P _S, it generates a front interpolation signal (first interpolation signal), noise intermediate from the first half (noise start point P _S noise by using the front interpolation signal interpolating the section) to the point _{P M.} Further, when the next frame is input, it generates a rear interpolation signal (second interpolation signal) from the signal after the noise end point P _E, the latter part of the noise using the posterior interpolation signal (noise intermediate interpolating the interval) from the point _{P M} until the noise end point _{P E.}

  By such interpolation processing, even when noise exists over a plurality of frames of the audio signal, the interpolation processing can be performed using the audio signal in an arbitrary section before and after the noise regardless of the frame boundary. Therefore, the noise can be appropriately reduced. The audio signal processing apparatus and method according to the third embodiment will be described in detail below.

[3.2. Functional configuration of audio signal processing apparatus]
Next, a functional configuration of the audio signal processing apparatus 100 according to the third embodiment will be described with reference to FIG. FIG. 15 is a block diagram illustrating a functional configuration of the audio signal processing apparatus 100 according to the third embodiment.

  As shown in FIG. 15, the audio signal processing apparatus 100 includes a signal input unit 110, an input buffer memory 122 (first buffer memory), an output buffer memory 132 (second buffer memory), and noise detection. Unit 140, noise reference point detection unit 142, noise reduction unit 150, and signal output unit 160. The input buffer memory 122, output buffer memory 132, noise detection unit 140, noise reference point detection unit 142, and noise reduction unit 150 constitute the audio signal processing unit 60 of FIG. The signal input unit 110, the input buffer memory 122, the output buffer memory 132, the noise detection unit 140, and the signal output unit 160 according to the third embodiment are substantially the same as those in the second embodiment. Detailed description will be omitted.

The audio signal processing apparatus 100 according to the third embodiment is characterized by further including a noise reference point detection unit 142. The noise reference point detection unit 142 is configured to generate a reference point (noise start point P _S , noise intermediate point P _M, and noise end point) of noise (pulse mechanical sound) included in the audio signal based on the signal characteristics of noise included in the audio signal. The point P _E ) is detected. As shown in FIG. 16, the noise start point P _S is a position at which pulsed mechanical sound starts in the speech signal. Also, the noise midpoint P _M is an intermediate position of the pulsed mechanical sound in the speech signal (e.g., the amplitude of the pulse components is maximized position). Furthermore, the noise end point _PE is a position where the pulse mechanical sound ends in the audio signal. A method for detecting these reference points by the noise reference point detection unit 142 is, for example, as follows.

First, the noise reference point detection unit 142 detects a noise intermediate point _{P M.} Detection methods for noise midpoint P _M is, for example, the following (a) ~ (c) are exemplified.

(A) a position where the maximum value exists in the absolute value of the amplitude of the maximum amplitude available pulsed mechanical sound may be a noise midpoint P _M. As shown in FIG. 16, the pulse mechanical sound includes a pulse component and a reverberation component, and the pulse peak (maximum amplitude value) of the pulse component substantially coincides with the intermediate point of the pulse mechanical sound. Therefore, the absolute value of the amplitude of the pulsed mechanical sound becomes maximum position can be estimated to be the noise midpoint P _M.

(B) noise interval information utilizing also the may be noise midpoint P _M the position when a certain time has elapsed from when obtaining the noise section information from the noise detecting unit 140. The noise detection unit 140 can generate noise section information representing a section including noise and output the noise section information to the noise reference point detection unit 142. This noise section information may be generated by the noise detection process described above, or may be generated based on control information of the driving device 14 that generates a pulse mechanical sound.

Utilizing the inclination of the change value of (c) signal also the position of the change point immediately after the tilt of the pulsed mechanical sound signal from the noise detection unit 140 has changed abruptly may be noise midpoint P _M. Since the amplitude of the pulse component of the pulse mechanical sound changes abruptly, the position where the amplitude differential value becomes zero immediately after this abrupt change shows the peak of the pulse component. Therefore, the position of the change point of the differential value of the amplitude can be estimated to be the noise midpoint P _M.

Next, the noise reference point detection unit 142 detects a noise start point _{P S.} Detection methods for noise start point P _S, for example the following (a), is exemplified (b).

(A) Use of signal energy The point where the signal energy falls below the threshold in the audio signal before the noise intermediate point P _M may be set as the noise start point P _S. As shown in FIG. 16, since the pulse mechanical sound generally has a larger amplitude than the background sound, the signal energy of the portion where the pulse mechanical sound exists is larger than the signal energy of the portion where only the background sound exists. Therefore, it can be estimated that the point where the signal energy is equal to or lower than the predetermined threshold in the audio signal temporally before the detected noise intermediate point P _M is the noise start point P _S.

(B) The pre-use set number of sample data, the point before the preset sample data number only noise midpoint P _M, may be a noise start point P _S. The time width of the pulse mechanical sound is measured in advance, and the difference between the noise intermediate point P _M and the noise start point P _S is obtained in advance, and the number of sample data representing the difference may be set as a parameter. . Using this parameter can be estimated noise start point P _S from the noise midpoint P _M.

Furthermore, noise reference point detection unit 142 detects the noise end point _{P E.} Detection method of noise end point P _E is the same as the detection method of the noise start point P _S. However, rather than the previous signal than noise midpoint P _M, the noise end point P _E is detected in the signal later than the noise midpoint P _M.

As described above, the noise reference point detection unit 142 detects the actual noise reference point in the noise section of the input speech signal. From the noise start point P _S of the reference point of the noise to the noise end point P _E is representative of the range of actual noise. The noise start point P _S and the noise end point P _E are separation positions between noise and background sound in the audio signal.

  Next, the noise reduction unit 150 according to the third embodiment will be described. The noise reduction unit 150 includes an interpolation signal generation unit 152 and a signal interpolation unit 154. The interpolation signal generation unit 152 includes a front interpolation signal generation unit 158 (first interpolation signal generation unit) and a rear interpolation signal generation unit 159 (first interpolation signal generation unit).

If the noise in the audio signal of the n-th frame by the noise detection unit 140 is detected, the front interpolation signal generating unit 158, by using the audio signal before a predetermined interval than the noise start point P _S, noise A front interpolation signal (first interpolation signal) for interpolating the first half is generated. For example, the front interpolation signal generating unit 158, n-1-th, the n-th audio signal before the noise start point P _S of speech signal frame, the noise start point P _S and a noise intermediate point P _M from the interval of the audio signal located before the length amount corresponding noise start point P _S corresponding to between generates a front interpolation signal.

Then, when the n + 1-th frame of the speech signal is stored in the input buffer memory 122, a rear interpolation signal generating unit 159 uses the audio signal of a predetermined period later than the noise end point P _E, noise A rear interpolation signal (second interpolation signal) for interpolating the latter half of the signal is generated. For example, the rear interpolated signal generation unit 159 may determine the difference between the noise intermediate point P _M and the noise end point P _E in the audio signal after the noise end point P _E among the audio signals of the nth and n + 1th frames. from the interval of the speech signal located after the amount corresponding noise end point P _E corresponding to the length, to generate a rear interpolation signal.

  As described above, in the third embodiment, the interpolation signal is not generated on the basis of the frame as in the second embodiment, but the front interpolation signal and the rear portion are based on the section specified by the noise reference point. Generate an interpolation signal. Details of the method of generating these front interpolation signal and rear interpolation signal will be described later.

  Then, the signal interpolation unit 154 uses the front interpolation signal generated by the front interpolation signal generation unit 158 to calculate the first half of noise included in the audio signal of the (n−1) th and / or nth frame. Interpolate. Further, the signal interpolation unit 154 interpolates the latter half of the noise included in the audio signal of the nth and / or n + 1th frame using the rear interpolation signal generated by the rear interpolation signal generation unit 159.

  For example, the signal interpolation unit 154 may replace the first half of the noise included in the audio signal with the front interpolation signal and replace the second half of the noise with the rear interpolation signal. Alternatively, the signal interpolation unit 154 combines the first half of the noise and the front interpolation signal included in the audio signal with an appropriate mixing ratio, and combines the second half of the noise and the rear interpolation signal with an appropriate mixing ratio, Interpolation processing may be executed. By this interpolation processing, the noise portion in the input speech signal is interpolated and the speech signal with reduced noise is output, so that the noise can be reduced / removed.

[3.3. Operation of audio signal processing apparatus]
Next, the operation of the audio signal processing apparatus 100 according to the third embodiment will be described. Since the normal operation without noise is the same as in the second embodiment (see FIG. 11), detailed description is omitted. In the following, with respect to an operation example at the time of noise generation according to the third embodiment, when noise exists over the nth and n + 1th frames (first operation example), the noise is n−1th and nth. A case (second operation example) in which the frame exists over two frames will be described. In both cases, since the peak of the pulse component of noise (pulse mechanical sound) exists in the nth frame, it is assumed that noise is detected when the nth frame is input.

[3.3.1. First operation example when noise occurs]
First, a first operation example of the audio signal processing apparatus 100 in the case where noise exists across the nth and n + 1th frames will be described with reference to FIGS. 17 and 18. 17 and 18 are schematic diagrams illustrating a first operation example when noise occurs in the audio signal processing apparatus 100 according to the present embodiment.

  As shown in FIG. 17A, it is detected that all of the audio signal s (n) of the nth frame has been accumulated in the input buffer memory 122 and that the audio signal s (n) includes a noise peak. When this is done, the noise reference point detection process and the front interpolation signal generation process shown in FIG. 17A and the front interpolation process shown in FIG. 17B are immediately executed.

Specifically, first, as shown in FIG. 17A, the front interpolated signal generation unit 158, from the signal in the section S _A from the noise start point P _S to the point P _A before the noise front section length L _F , generating a front interpolation signal t (n) for interpolating the noise front section S _F. Here, the noise front section S _F is a section from the noise start point P _S to noise midpoint P _M, the noise front section length L _F is from the noise start point P _S to noise intermediate point P _M The length of the section.

Section S _A is present before the noise start point P _S of the pulsed mechanical sound, a section that does not contain noise. In the present embodiment, the section length of the section S _A is set equal to the noise front section length L _F. However, section length of the section S _A may be set as appropriate depending on the noise front section length L _F, shorter than L _F, or may be longer. Such interval S _A includes front section of at least n-th frame, by the noise front section length L _F also includes a rear section of the n-1 th frame. In the example of FIG. 17A, the section S _A is set across both the n-th and n-1 th frame.

Front interpolation signal generating unit 158, n-th and (n-1) th frame of the speech signal s (n), using the signal of the interval _{S A} of s (n-1), the front interpolation signal t ( n). The method for generating the front interpolation signal t (n) is the same as the method for generating the interpolation signal v (n) according to the first embodiment described above (see FIGS. 4 and 5). _The front interpolation signal t (n) is generated by inverting the _A signal in the time axis direction.

Then, signal interpolation unit 154, as shown in FIG. 17B, by using the front interpolation signal t (n), of the n th frame of the speech signal s (n), the noise front section S _F of the signal Is interpolated. In the example of the front interpolation processing FIG. 17B, the audio signal s of the noise front section S _F of the n-th frame stored in the input buffer memory 122 (n) is a front interpolation signal t (n) Has been replaced. Such front interpolation processing, noise of the noise front section S _F is reduced.

Next, as shown in FIG. 17C, immediately after the front interpolation process, the signal interpolation unit 154 receives the audio signal s (n−1) of the (n−1) th frame stored in the output buffer memory 132 as a signal. Output to the output unit 160. Further, the signal interpolation unit 154 outputs the front interpolated audio signal s (n) + t (n) stored in the input buffer memory 122 together with the output of the audio signal s (n−1) to the output buffer memory. Move to 132. Here, the front interpolated audio signal s (n) + t (n) is the nth frame audio signal s (n) in which the noise front section _SF is interpolated by the front interpolated signal t (n). It is. In this way, by moving the front interpolated audio signal s (n) + t (n) to the output buffer memory 132, when the audio signal s (n + 1) of the next n + 1th frame is input. The rear noise section of the front interpolated speech signal s (n) + t (n) can be interpolated.

  Next, as shown in FIG. 18A, when the audio signal s (n + 1) of the next n + 1-th frame is newly input and stored in the input buffer memory 122, the front interpolated audio signal s (n) + T (n) is temporarily stored in the output buffer memory 132.

  Then, when all of the audio signal s (n + 1) of the (n + 1) th frame has been accumulated in the input buffer memory 122, the rear interpolation signal generation process shown in FIG. 18A and the rear interpolation process shown in FIG. 18B are immediately executed. The

More specifically, first, the rear interpolation signal generating unit 159, as shown in FIG. 18A, the section S _B of the signal from the noise end point P _E to P _B point earlier by noise rear section length L _R, noise rear generating a rear interpolation signal u (n) for interpolating the section S _R. Here, the noise rear section S _R is a section from the noise intermediate point P _M to the noise end point P _E , and the noise rear section length L _R is a section of the section from the noise intermediate point P _M to the noise end point P _E. Length.

Section S _B is present after the noise end point P _E of pulsed mechanical sound, a section that does not contain noise. In the present embodiment, the section length of the section S _B is set equal to the noise rear section length L _R. However, section length of the section S _B may be set appropriately in accordance with the noise rear section length L _R, shorter than L _B, or may be longer. Such section S _B includes front section of at least n-th rear section of the frame and (n + 1) th frame, depending on the noise rear section length L _R, including a front section of the n + 2 th frame . In the example of FIG. 18A, the section S _B is set across both the n-th and (n + 1) th frame.

Rear interpolation signal generating unit 159, n-th and (n + 1) th frame of the speech signal s (n), using the signal of the section _{S B} of s (n + 1), and generates a rear interpolation signal u (n). The method of generating the rear interpolation signal u (n) is the same as the method of generating the interpolation signal v (n) according to the first embodiment described above (see FIGS. 4 and 5). For example, the section S _B Is inverted in the time axis direction to generate the rear interpolation signal u (n).

Next, as shown in FIG. 18B, the signal interpolation unit 154 uses the rear interpolation signal u (n) to generate noise among the audio signals s (n) and s (n + 1) of the nth and n + 1th frames. interpolating the signals of the rear section S _R. In the example of the rear interpolation processing of FIG. 18B, the front interpolation audio signal s (n) + t (n) of the nth frame stored in the output buffer memory 132 and the input buffer memory 122 are stored. among (n + 1) th frame of the speech signal s (n + 1), the signal of the noise rear section S _R is substituted at the rear interpolation signal u (n). Such rear interpolation processing, noise of the noise rear section S _R is reduced.

Next, as shown in FIG. 18C, immediately after the rear interpolation process, the signal interpolation unit 154 replaces the audio signal s (n) of the nth frame actually input in FIG. 17A with the output buffer memory 132. The front / rear interpolation signal s (n) + t (n) + u (n) stored in is output to the signal output unit 160. Here, the front and rear interpolated audio signal s (n) + t (n ) + u (n), the noise front section _{S F} by the front interpolation signal t (n) are interpolated, and the rear interpolation signal u (n) by the noise rear section S _R is interpolated n-th frame of the speech signal s (n).

  Further, as shown in FIG. 18C, the signal interpolating unit 154 includes the output of the front and rear interpolated audio signals s (n) + t (n) + u (n) and the (n + 1) th stored in the input buffer memory 122. The front interpolated audio signal u (n) + s (n + 1) of the current frame is moved to the output buffer memory 132. As a result, when the audio signal s (n + 1) of the (n + 2) th frame is input next, the front-interpolated audio signal u (n) + s (n + 1) of the (n + 1) th frame with reduced noise is output. Is possible.

As in the above first operation example, in the presence of noise across n-th and (n + 1) th frame, by using a signal just before the noise start point P _S of the n-th and n-1 th frame noise front section S _F is interpolated noise rear section S _R using the signal immediately after the n-th and n + 1 th noise end point P _E of the frames is interpolated.

[3.3.2. Second operation example when noise occurs]
Next, with reference to FIGS. 19 and 20, a second operation example of the audio signal processing apparatus 100 when noise is present across the (n−1) th and nth frames will be described. 19 and 20 are schematic diagrams illustrating a second operation example when noise occurs in the audio signal processing apparatus 100 according to the present embodiment.

  As shown in FIG. 19A, it is detected that all of the audio signal s (n) of the nth frame has been accumulated in the input buffer memory 122 and that the audio signal s (n) includes a noise peak. If so, the noise reference point detection process and the front interpolation signal generation process shown in FIG. 19A and the front interpolation process shown in FIG. 19B are immediately executed.

Specifically, first, as shown in FIG. 19A, the front interpolation signal generation unit 158, from the signal in the section S _A from the noise start point P _S to the point P _A before the noise front section length L _F , generating a front interpolation signal t (n) for interpolating the noise front section S _F. Defining such noise front section S _F and section S _A is the same as in the first operation example described above. However, in the second operation example, since the noise start point P _S is present in the n-1 th frame, the noise front section S _F is present across n-1 th and n th frame. Furthermore, the section S _A includes a portion of the section of at least (n-1) th frame, by the noise front section length L _F also includes a rear section of the (n-2) th frame. In the example of FIG. 19A, the section S _A is set in the n-1 th frame.

Front interpolation signal generating unit 158, by using the signal of the interval S _A among the n-1 th frame of the speech signal s (n-1), generates a front interpolation signal t (n). The method of generating the front interpolation signal t (n) is the same as that in the first operation example.

Next, as shown in FIG. 19B, the signal interpolation unit 154 uses the front interpolation signal t (n) to generate the audio signals s (n−1) and s (n (n−1) th and n−1th frames. of), interpolating the signal in the noise front section S _F. In the example of the front interpolation process of FIG. 19B, the noise front section S _F among the n th frame stored in the output buffer memory 132 and the n th frame stored in the input buffer memory 122. Audio signals s (n−1) and s (n) are replaced with the front interpolation signal t (n). Such front interpolation processing, noise of the noise front section S _F is reduced.

Next, as shown in FIG. 19C, immediately after the front interpolation process, the signal interpolation unit 154 receives the rear interpolation audio signal s (n−1) + t (n) stored in the output buffer memory 132. Output to the output unit 160. Here, the rear interpolated audio signal s (n−1) + t (n) is an audio signal s (n−1) th frame in which the noise rear section _SF is interpolated by the front interpolated signal t (n). n-1).

Further, as shown in FIG. 19C, the signal interpolation unit 154 outputs the front interpolated audio signal t stored in the input buffer memory 122 together with the output of the rear interpolated audio signal s (n−1) + t (n). (N) + s (n) is moved to the output buffer memory 132. Here, the front interpolated audio signal t (n) + s (n) is the nth frame audio signal s (n) in which the noise front section _SF is interpolated by the front interpolated signal t (n). It is. In this way, by moving the front interpolated audio signal t (n) + s (n) to the output buffer memory 132, when the audio signal s (n + 1) of the next (n + 1) th frame is input. The rear noise section of the front interpolated speech signal t (n) + s (n) can be interpolated.

  Next, as shown in FIG. 20A, when the audio signal s (n + 1) of the next n + 1-th frame is newly input and accumulated in the input buffer memory 122, the front interpolated audio signal t (n) + S (n) is temporarily stored in the output buffer memory 132.

  Then, when all of the audio signal s (n + 1) of the (n + 1) th frame has been accumulated in the input buffer memory 122, the rear interpolation signal generation process shown in FIG. 20A and the rear interpolation process shown in FIG. 20B are immediately executed. The

Specifically, first, as shown in FIG. 20A, the rear interpolation signal generation unit 159 generates a noise rear part from the signal in the section S _B from the noise end point P _E to the point P _B that is the noise rear section length L _R before. generating a rear interpolation signal u (n) for interpolating the section S _R. Defining such noise rear section S _R and segment S _B is the same as in the first operation example described above. However, in the second operation example, since the noise end point P _E is present in the n-th frame, the noise rear section S _F is present in the n + 1 th frame. Furthermore, the section S _B may comprise part of a section of at least n-th frame, by the noise rear section length L _R also includes a front rear section of the n-1 th frame. In the example of FIG. 20A, the section S _B is set across both the n-th and (n + 1) th frame.

Rear interpolation signal generating unit 159, n-th and (n + 1) th frame of the speech signal s (n), using the signal of the section _{S B} of s (n + 1), and generates a rear interpolation signal u (n). The method for generating the rear interpolation signal u (n) is the same as in the first operation example.

Then, signal interpolation unit 154, as shown in FIG. 20B, by using the rear interpolation signal u (n), of the n-th frame voice signal s (n), interpolates the signal noise rear section S _R To do. In the example of the rear interpolation processing FIG. 20B, among the front interpolated audio signal t of the n-th frame stored in the output buffer memory 132 (n) + s (n ), the signal of the noise rear section S _R is, It is replaced by the rear interpolation signal u (n). Such rear interpolation processing, noise of the noise rear section S _R is reduced.

Next, as shown in FIG. 20C, immediately after the rear interpolation process, the signal interpolation unit 154 replaces the audio signal s (n) of the nth frame actually input in FIG. 19A with the output buffer memory 132. The front / rear interpolation signal t (n) + u (n) + s (n) stored in is output to the signal output unit 160. Here, the front and rear interpolated audio signal t (n) + u (n ) + s (n), the noise front section _{S F} by the front interpolation signal t (n) are interpolated, and the rear interpolation signal u (n) by the noise rear section S _R is interpolated n-th frame of the speech signal s (n).

  Furthermore, as shown in FIG. 20C, the signal interpolation unit 154 stores the (n + 1) th stored in the input buffer memory 122 together with the output of the front and rear interpolated audio signals t (n) + u (n) + s (n). The audio signal s (n + 1) of the current frame is moved to the output buffer memory 132. As a result, when the audio signal s (n + 1) of the (n + 2) th frame is input next, the audio signal s (n + 1) of the (n + 1) th frame can be output.

As in the second operation example, in the presence of noise across n-1 th and n-th frame, by using a signal just before the noise start point P _S of the n-1 th frame noise front section S _F is interpolated noise rear section S _R using the signal immediately after the n-th and n + 1 th noise end point P _E of the frames is interpolated.

As described above, according to this embodiment, in the presence of noise over two frames, and detects the reference point of the noise, using a signal of a previous segment S _A from the noise start point P _S with interpolated noise front section S _F, it interpolates the noise rear section S _R using the signal of the section S _B after the noise end point P _E. This eliminates the need to perform interpolation processing on a frame-by-frame basis and allows interpolation processing to be performed using a signal in a free section closest to noise, so that even more natural and highly accurate interpolation processing can be executed. High quality noise reduction can be realized.

  Further, even when the noise reference point is detected and interpolation processing is performed as described above, the memory lengths of the input buffer memory 122 and the output buffer memory 132 may each be the number N of sample data of one frame. Therefore, as in the first and second embodiments, the buffer memory length required for the entire apparatus is 2 * N. Also, immediately after the audio signal s (n + 1) of the next frame is completely stored in the input buffer memory 122, the audio signal s (n) of the previous frame is output to the outside. The delay of the output voice with respect to is only one frame.

[3.4. Audio signal processing method]
Next, an audio signal processing method (mechanical sound reduction method) using the audio signal processing apparatus 100 will be described with reference to FIG. FIG. 21 is a flowchart showing an audio signal processing method according to the third embodiment.

  As shown in FIG. 21, first, the audio signal processing apparatus 100 determines whether or not an audio signal for one frame input from the microphone 51 has been accumulated in the input buffer memory 122 (S300). Here, processing when the audio signal s (n) of the nth frame is currently being input will be described. As a result of the determination in S300, as soon as all of the audio signal s (n) of the nth frame has been accumulated in the input buffer memory 122, the noise detection unit 140 includes noise in the audio signal s (n). Is detected (S302).

  If noise is detected as a result of the determination in S302, a noise reference point detection process (S304), a front interpolation signal generation process (S306), and a front interpolation process (S308) are immediately executed.

Specifically, first, the noise reference point detection unit 142, as described above, based on the characteristics of noise included in the audio signal s (s), the noise start point P _S , the noise intermediate point P _M , and the noise end point. calculating a P _E (S304). Then, the front interpolation signal generating unit 158, FIG. 17, as shown in FIG. 19, a predetermined section before the noise start point _{P S S A} speech signal s (n-1), with s (n) Te, generating the front interpolation signal t (n) for interpolating the noise front section _{S F} (S306).

Furthermore, signal interpolation unit 154 uses the front interpolation signal t (n) generated by S306, the speech signal s (n-1), s the signal of the noise front section _{S F} of the (n) Interpolate (S308). In front interpolation processing in step S308, to the signal in the noise front section S _F may be replaced by the front interpolation signal t (n), the signal of the noise front section S _F and the front interpolation signal t ( n) may be synthesized at an appropriate mixing ratio. Below, the substituted example is demonstrated.

  After that, the signal interpolation unit 154 moves the front interpolation audio signal s (n) + t (n) in the input buffer memory 122 to the output buffer memory 132 (S310).

  Next, the newly input audio signal s (n + 1) of the (n + 1) th frame is accumulated in the input buffer memory 122, and whether or not all the audio signals s (n + 1) have been accumulated in the input buffer memory 122. Determination is made (S312). As a result, when all of the audio signals s (n + 1) of the (n + 1) th frame are completely stored in the input buffer memory 122, the rear interpolation signal generation processing (S316) and the rear interpolation shown in FIGS. Processing (S318) is executed.

More specifically, first, the rear interpolation signal generating unit 159, FIG. 18, as shown in FIG. 20, a predetermined interval prior to back than the noise end point _{P E S B} of the audio signal s (n), s (n + 1) It is used to generate a rear interpolation signal u (n) for interpolating the noise rear section _{S R} (S316).

Then, signal interpolation unit 154, using the rear interpolation signal u (n) generated in S316, interpolating the signal in the noise rear section _{S R} of the speech signal s (n), s (n + 1) (S318 ). In the rear interpolation processing in S318 is to signal noise rear section S _R may be replaced with the rear interpolation signal u (n), an appropriate signal of the noise rear section S _R and the rear interpolation signal u (n) You may synthesize | combine by a mixture ratio. Below, the substituted example is demonstrated.

  Thereafter, the signal interpolation unit 154 interpolates with the front interpolation signal t (n) and the rear interpolation signal u (n) in S308 and S318, instead of the actually input n-th frame audio signal s (n). The front and rear interpolated audio signals s (n), t (n), and u (n) are output to the signal output unit 160 (S320). Then, the audio signal s (n) of the (n + 1) th frame stored in the input buffer memory 122 is moved to the output buffer memory 132.

  On the other hand, if no noise is detected in the audio signal s (n) of the nth frame as a result of the noise determination in S302, normal input / output processing is performed without performing the above interpolation processing. That is, as shown in FIG. 11, the audio signal s (n−1) of the (n−1) th frame is output from the output buffer memory 132 as it is, and is output to the signal output unit 160 and stored in the input buffer memory 122. The audio signal s (n) of the th frame is moved to the output buffer memory 132 (S310). Then, when all the audio signals s (n + 1) of the next n + 1th frame have been accumulated in the input buffer memory 122 (S312), the audio signal s (n) of the nth frame is output from the output buffer memory 132. Are directly output to the signal output unit 160 (S320), and the audio signal s (n) of the (n + 1) th frame stored in the input buffer memory 122 is moved to the output buffer memory 132.

  Thereafter, the processes of S300 to S320 are repeated for the audio signal s (n + 2) of the next frame of the input audio signal until the imaging and recording operation by the digital camera 1 is completed (S322). As a result, noise detection processing is performed on the input audio signal for each frame, and if necessary, interpolation processing (noise reduction processing) is performed, and then a noise-free audio signal is output in frame units. The

[3.5. effect]
The configuration of the audio signal processing apparatus 100 according to the third embodiment of the present disclosure and the audio signal processing method using the same have been described above. According to the third embodiment, in addition to the effects of the second embodiment described above, the following effects are further obtained.

According to the third embodiment, the noise reference point (noise start point P _S , noise intermediate point P _M , noise end point P _E ) is detected, so that the noise reference point can be obtained regardless of the frame unit of the audio signal. Based on the above, it is possible to freely select an arbitrary section of the audio signal before and after the noise and realize the interpolation processing. That generates a front interpolation signal t (n) from the signal of the section S _A immediately before the noise start point P _S, as well as interpolated noise front section S _F, immediately after the noise end point P _E section S from the signal of _B to generate a rear interpolation signal u (n), it interpolates the noise rear section S _R. Therefore, even when noise exists over a plurality of frames, it is possible to appropriately implement the interpolation process by suitably using signals in the sections immediately before and after the noise section.

  Furthermore, as in the second embodiment, since interpolation processing is performed using signals before and after the noise section, the background sound can be reproduced with high accuracy while increasing the accuracy of the interpolation processing and reducing noise. The accuracy of noise reduction processing can be greatly improved.

  Similarly to the first and second embodiments, in the third embodiment, the buffer memory length required for estimating the interpolation signal is 2 * N, so the conventional interpolation method (see FIG. 1) is used. Compared with the need for a buffer memory length of at least 3 * N, the buffer memory required for interpolation processing can be greatly reduced.

  Further, as in the second embodiment, since the delay of the output sound with respect to the input sound can be suppressed to one frame (delay amount: N), the delay of the output sound associated with the interpolation process is less than that of the conventional interpolation method. Can be reduced to half.

  The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present technology is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

  For example, in the above-described embodiment, the digital camera 1 is mainly exemplified as an audio signal processing device, and an example in which mechanical sound is reduced when recording with moving image capturing is described. However, the present technology is not limited to such an example. The audio signal processing apparatus according to the present technology can be applied to electronic apparatuses such as various audio signal recording apparatuses or audio signal reproduction apparatuses. For example, the audio signal processing apparatus includes a recording / reproducing apparatus (for example, a Blu-ray disc / DVD recorder), a television receiver, a system stereo apparatus, an imaging apparatus (for example, a digital camera, a digital video camera), and a portable terminal (for example, a portable type). Music / video player, portable game machine, IC recorder), personal computer, game machine, car navigation device, digital photo frame, home appliance, vending machine, ATM, kiosk terminal, etc.

  Further, in the above-described embodiment, the example in which the noise reduction process is performed when the audio signal is recorded by the digital camera 1 has been described. However, the present invention is not limited to this example, and if the audio signal processing device of the present technology is applied to an audio signal reproduction device, the noise included in the audio signal to be reproduced is appropriately reduced even when the recorded audio signal is reproduced. Can be reduced.

In addition, this technique can also take the following structures.
(1) a first buffer memory that temporarily stores an input audio signal for each predetermined section;
A second buffer memory for temporarily storing the audio signal of the (n-1) th section immediately before the audio signal of the nth section stored in the first buffer memory;
Interpolation that generates an interpolated signal from at least the (n−1) th section audio signal stored in the second buffer memory when it is detected that the nth section audio signal contains noise. A signal generator;
Using the interpolation signal, a signal interpolation unit that interpolates the sound signal of the nth section including the noise;
An audio signal processing apparatus comprising:

(2) further comprising a noise reference point detection unit for detecting a start point and an end point of the noise included in the audio signal;
The interpolation signal generator is
A first interpolation signal generating unit that generates a first interpolation signal from one or both of the n-1 and / or n-th audio signals;
A second interpolation signal generating unit that generates a second interpolation signal from one or both of the n + 1-th and n-th interval audio signals;
With
When the audio signal of the nth section is temporarily stored in the first buffer memory and it is detected that at least the audio signal of the nth section includes noise, the first interpolation signal The generating unit includes the n-1th section audio signal stored in the second buffer memory and the nth section audio signal stored in the first buffer memory. The first interpolation signal is generated from a signal in a predetermined interval before the start point, and the signal interpolation unit is configured to detect the noise front portion of the audio signals in the (n−1) th and nth intervals. A signal is interpolated using the first interpolation signal, and the audio signal of the n-th section after interpolation by the first interpolation signal is temporarily stored in the second buffer memory,
When the audio signal of the (n + 1) th section is temporarily stored in the first buffer memory, the second interpolation signal generation unit stores the first interpolation stored in the second buffer memory. Of the audio signal of the nth section after interpolation by the signal and the audio signal of the n + 1st section stored in the first buffer memory, from the signal of the predetermined section after the end point, A second interpolation signal is generated, and the signal interpolation unit outputs a signal behind the noise among the sound signal of the nth section and the sound signal of the n + 1th section after interpolation by the first interpolation signal. Interpolating using the first interpolation signal, and outputting the audio signal of the nth section after interpolation by the first interpolation signal and the second interpolation signal from the second buffer memory, Sound described in (1) Signal processor.

(3) The noise reference point detection unit detects an intermediate point of the noise, detects the start point and the end point based on the intermediate point,
When the audio signal of the nth section is temporarily stored in the first buffer memory and it is detected that at least the audio signal of the nth section includes noise, the first interpolation signal The generating unit includes the n-1th section audio signal stored in the second buffer memory and the nth section audio signal stored in the first buffer memory. The signal interpolation unit generates the first interpolation signal from a signal in a section located before the start point by an amount corresponding to the length between the start point and the intermediate point of noise. Of the speech signals of the (n-1) th and nth intervals, the signal in the interval from the start point to the intermediate point of the noise is replaced with the first interpolation signal, and the replacement by the first interpolation signal The audio signal of the nth section after Temporarily stored in the second buffer memory,
When the audio signal of the (n + 1) th section is temporarily stored in the first buffer memory, the second interpolation signal generation unit stores the first interpolation stored in the second buffer memory. Of the sound signal of the nth section after replacement by a signal and the sound signal of the n + 1th section stored in the first buffer memory, between the intermediate point and the end point of the noise The second interpolation signal is generated from the signal in the section located after the end point by an amount corresponding to the length of the first interpolation signal, and the signal interpolation unit replaces the nth section with the first interpolation signal. Of the noise signal and the sound signal of the (n + 1) th section are replaced with the second interpolation signal in the section from the intermediate point to the end point of the noise, and the first interpolation signal and the second interpolation signal are replaced. After replacement by interpolation signal The audio signal of the serial n th interval outputted from said second buffer memory, the audio signal processing apparatus according to (2).

(4) The interpolation signal generation unit
A first temporary interpolation signal generating unit that generates a first temporary interpolation signal from the audio signal of the (n-1) th section;
A second temporary interpolation signal generation unit that generates a second temporary interpolation signal from the audio signal of the (n + 1) th section;
With
When the audio signal of the nth section is temporarily stored in the first buffer memory and it is detected that the audio signal of the nth section includes noise, the first temporary interpolation signal The generation unit generates the first temporary interpolation signal from the audio signal of the (n−1) -th section stored in the second buffer memory, and the first temporary interpolation signal is generated in the second buffer. Temporarily store it in memory,
When the audio signal of the (n + 1) th section is temporarily stored in the first buffer memory, the second temporary interpolation signal generation unit is configured to store the n + 1th section stored in the first buffer memory. A second temporary interpolation signal is generated from the audio signal, and the signal interpolation unit performs the interpolation from the second temporary interpolation signal and the first temporary interpolation signal stored in the second buffer memory. The audio signal processing device according to (1), wherein a signal is generated and the interpolated signal is output from the second buffer memory in place of the audio signal in the n-th interval.

(5) generating the interpolated signal when the audio signal in the n-th section is temporarily stored in the first buffer memory and it is detected that the audio signal in the n-th section includes noise. The unit generates the interpolation signal from the audio signal of the (n-1) th section stored in the second buffer memory, and the signal interpolation unit is stored in the first buffer memory. The audio signal processing device according to (1), wherein the interpolation signal is output from the first buffer memory in place of the audio signal in the nth section.

(6) The noise is any one of (1) to (5), which is a pulsed operation sound generated from a sound generation unit provided in the same housing as the sound collection unit that outputs the audio signal. The audio signal processing device according to item.

(7) The sound generation unit is a driving device provided in the same housing as the sound collection unit,
The sound signal processing device according to (6), wherein the operation sound is a pulse-like mechanical drive sound generated when the operation of the drive device starts or ends.

(8) The sound according to any one of (1) to (7), wherein a time length of the predetermined section, which is a processing unit of the sound signal, is longer than a time length of the pulse-like mechanical drive sound. Signal processing device.

(9) a sound collection unit that converts external sound into a sound signal;
A sound generation unit that is provided in the same housing as the sound collection unit and generates noise;
A first buffer memory for temporarily storing the audio signal input from the sound collection unit for each predetermined section;
A second buffer memory for temporarily storing the audio signal of the (n-1) th section immediately before the audio signal of the nth section stored in the first buffer memory;
Interpolation that generates an interpolated signal from at least the (n−1) th section audio signal stored in the second buffer memory when it is detected that the nth section audio signal contains noise. A signal generator;
Using the interpolation signal, a signal interpolation unit that interpolates the sound signal of the nth section including the noise;
An imaging apparatus comprising:

(10) temporarily storing the audio signal of the (n-1) th section stored in the first buffer memory in the second buffer memory;
Temporarily storing the input audio signal of the nth section in the first buffer memory;
When it is detected that the nth section audio signal stored in the first buffer memory includes noise, at least the n−1th section stored in the second buffer memory. Generating an interpolated signal from the audio signal of the section;
Using the interpolated signal to interpolate the audio signal of the nth section containing the noise;
An audio signal processing method comprising:

(11) temporarily storing the audio signal of the (n−1) th section stored in the first buffer memory in the second buffer memory;
Temporarily storing the input audio signal of the nth section in the first buffer memory;
When it is detected that the nth section audio signal stored in the first buffer memory includes noise, at least the n−1th section stored in the second buffer memory. Generating an interpolated signal from the audio signal of the section;
Using the interpolated signal to interpolate the audio signal of the nth section containing the noise;
A program that causes a computer to execute.

(12) temporarily storing the audio signal of the (n−1) -th section stored in the first buffer memory in the second buffer memory;
Temporarily storing the input audio signal of the nth section in the first buffer memory;
When it is detected that the nth section audio signal stored in the first buffer memory includes noise, at least the n−1th section stored in the second buffer memory. Generating an interpolated signal from the audio signal of the section;
Using the interpolated signal to interpolate the audio signal of the nth section containing the noise;
A computer-readable recording medium on which a program for causing a computer to execute is recorded.

DESCRIPTION OF SYMBOLS 1 Digital camera 10 Image pick-up part 14 Drive apparatus 15 Zoom motor 16 Focus motor 51 Microphone 60 Audio | voice signal processing part 70 Control part 100 Audio | voice signal processing apparatus 110 Signal input part 120 Input / output buffer memory 122 Input buffer memory 130 Interpolation buffer memory 130 132 buffer memory for output 140 noise detection unit 142 noise reference point detection unit 150 noise reduction unit 152 interpolation signal generation unit 154 signal interpolation unit 156 first temporary interpolation signal generation unit 157 second temporary interpolation signal generation unit 158 front interpolation signal generating unit 159 rear interpolation signal generating unit 160 signal output unit s audio signal v interpolated signal p first temporary interpolated signal q second temporary interpolated signal t front interpolation signal u rear interpolation signal P _S noise start point P _M noise intermediate point P _E noise end point S _F noise front section S _R miscellaneous Rear section L _F noise front section length L _R noise rear section length

Claims (12)

A first buffer memory for temporarily storing the input audio signal for each predetermined section;
A second buffer memory for temporarily storing the audio signal of the (n-1) th section immediately before the audio signal of the nth section stored in the first buffer memory;
Interpolation that generates an interpolated signal from at least the (n−1) th section audio signal stored in the second buffer memory when it is detected that the nth section audio signal contains noise. A signal generator;
Using the interpolation signal, a signal interpolation unit that interpolates the sound signal of the nth section including the noise;
An audio signal processing apparatus comprising: A noise reference point detection unit for detecting a start point and an end point of the noise included in the audio signal;
The interpolation signal generator is
A first interpolation signal generating unit that generates a first interpolation signal from one or both of the n-1 and / or n-th audio signals;
A second interpolation signal generating unit that generates a second interpolation signal from one or both of the n + 1-th and n-th interval audio signals;
With
When the audio signal of the nth section is temporarily stored in the first buffer memory and it is detected that at least the audio signal of the nth section includes noise, the first interpolation signal The generating unit includes the n-1th section audio signal stored in the second buffer memory and the nth section audio signal stored in the first buffer memory. The first interpolation signal is generated from a signal in a predetermined interval before the start point, and the signal interpolation unit is configured to detect the noise front portion of the audio signals in the (n−1) th and nth intervals. A signal is interpolated using the first interpolation signal, and the audio signal of the n-th section after interpolation by the first interpolation signal is temporarily stored in the second buffer memory,
When the audio signal of the (n + 1) th section is temporarily stored in the first buffer memory, the second interpolation signal generation unit stores the first interpolation stored in the second buffer memory. Of the audio signal of the nth section after interpolation by the signal and the audio signal of the n + 1st section stored in the first buffer memory, from the signal of the predetermined section after the end point, A second interpolation signal is generated, and the signal interpolation unit outputs a signal behind the noise among the sound signal of the nth section and the sound signal of the n + 1th section after interpolation by the first interpolation signal. Interpolating using the first interpolation signal, and outputting the audio signal of the nth section after interpolation by the first interpolation signal and the second interpolation signal from the second buffer memory. The voice of item 1 No. processing apparatus. The noise reference point detection unit detects an intermediate point of the noise, detects the start point and the end point based on the intermediate point,
When the audio signal of the nth section is temporarily stored in the first buffer memory and it is detected that at least the audio signal of the nth section includes noise, the first interpolation signal The generating unit includes the n-1th section audio signal stored in the second buffer memory and the nth section audio signal stored in the first buffer memory. The signal interpolation unit generates the first interpolation signal from a signal in a section located before the start point by an amount corresponding to the length between the start point and the intermediate point of noise. Of the speech signals of the (n-1) th and nth intervals, the signal in the interval from the start point to the intermediate point of the noise is replaced with the first interpolation signal, and the replacement by the first interpolation signal The audio signal of the nth section after Temporarily stored in the second buffer memory,
When the audio signal of the (n + 1) th section is temporarily stored in the first buffer memory, the second interpolation signal generation unit stores the first interpolation stored in the second buffer memory. Of the sound signal of the nth section after replacement by a signal and the sound signal of the n + 1th section stored in the first buffer memory, between the intermediate point and the end point of the noise The second interpolation signal is generated from the signal in the section located after the end point by an amount corresponding to the length of the first interpolation signal, and the signal interpolation unit replaces the nth section with the first interpolation signal. Of the noise signal and the sound signal of the (n + 1) th section are replaced with the second interpolation signal in the section from the intermediate point to the end point of the noise, and the first interpolation signal and the second interpolation signal are replaced. After replacement by interpolation signal The audio signal of the serial n th interval outputted from said second buffer memory, the audio signal processing apparatus according to claim 2. The interpolation signal generator is
A first temporary interpolation signal generating unit that generates a first temporary interpolation signal from the audio signal of the (n-1) th section;
A second temporary interpolation signal generation unit that generates a second temporary interpolation signal from the audio signal of the (n + 1) th section;
With
When the audio signal of the nth section is temporarily stored in the first buffer memory and it is detected that the audio signal of the nth section includes noise, the first temporary interpolation signal The generation unit generates the first temporary interpolation signal from the audio signal of the (n−1) -th section stored in the second buffer memory, and the first temporary interpolation signal is generated in the second buffer. Temporarily store it in memory,
When the audio signal of the (n + 1) th section is temporarily stored in the first buffer memory, the second temporary interpolation signal generation unit is configured to store the n + 1th section stored in the first buffer memory. A second temporary interpolation signal is generated from the audio signal, and the signal interpolation unit performs the interpolation from the second temporary interpolation signal and the first temporary interpolation signal stored in the second buffer memory. 2. The audio signal processing device according to claim 1, wherein a signal is generated, and the interpolation signal is output from the second buffer memory in place of the n-th interval audio signal.   When the audio signal of the nth section is temporarily stored in the first buffer memory and it is detected that noise is included in the audio signal of the nth section, the interpolation signal generation unit is The interpolation signal is generated from the audio signal of the (n−1) -th section stored in the second buffer memory, and the signal interpolation unit is configured to generate the n-th section stored in the first buffer memory. The audio signal processing apparatus according to claim 1, wherein the interpolated signal is output from the first buffer memory in place of an audio signal in a section.   The audio signal processing apparatus according to claim 1, wherein the noise is a pulsed operation sound generated from a sound generation unit provided in the same housing as the sound collection unit that outputs the audio signal. The sound generation unit is a driving device provided in the same housing as the sound collection unit,
The audio signal processing device according to claim 6, wherein the operation sound is a pulse-like mechanical drive sound that is generated when the operation of the drive device starts or ends.   The audio signal processing device according to claim 1, wherein a time length of the predetermined section, which is a processing unit of the audio signal, is longer than a time length of the pulse-like mechanical drive sound. A sound collection unit that converts external sound into an audio signal;
A sound generation unit that is provided in the same housing as the sound collection unit and generates noise;
A first buffer memory for temporarily storing the audio signal input from the sound collection unit for each predetermined section;
A second buffer memory for temporarily storing the audio signal of the (n-1) th section immediately before the audio signal of the nth section stored in the first buffer memory;
Interpolation that generates an interpolated signal from at least the (n−1) th section audio signal stored in the second buffer memory when it is detected that the nth section audio signal contains noise. A signal generator;
Using the interpolation signal, a signal interpolation unit that interpolates the sound signal of the nth section including the noise;
An imaging apparatus comprising: Temporarily storing the audio signal of the (n−1) th section stored in the first buffer memory in the second buffer memory;
Temporarily storing the input audio signal of the nth section in the first buffer memory;
When it is detected that the nth section audio signal stored in the first buffer memory includes noise, at least the n−1th section stored in the second buffer memory. Generating an interpolated signal from the audio signal of the section;
Using the interpolated signal to interpolate the audio signal of the nth section containing the noise;
An audio signal processing method comprising: Temporarily storing the audio signal of the (n−1) th section stored in the first buffer memory in the second buffer memory;
Temporarily storing the input audio signal of the nth section in the first buffer memory;
When it is detected that the nth section audio signal stored in the first buffer memory includes noise, at least the n−1th section stored in the second buffer memory. Generating an interpolated signal from the audio signal of the section;
Using the interpolated signal to interpolate the audio signal of the nth section containing the noise;
A program that causes a computer to execute. Temporarily storing the audio signal of the (n−1) th section stored in the first buffer memory in the second buffer memory;
Temporarily storing the input audio signal of the nth section in the first buffer memory;
When it is detected that the nth section audio signal stored in the first buffer memory includes noise, at least the n−1th section stored in the second buffer memory. Generating an interpolated signal from the audio signal of the section;
Using the interpolated signal to interpolate the audio signal of the nth section containing the noise;
A computer-readable recording medium on which a program for causing a computer to execute is recorded.

Images