JP6540730B2 - Sound collection device, program and method, determination device, program and method - Google Patents

Description

  The present invention relates to a sound collection device, a program and method, and a determination device, program and method, and can be applied to, for example, a process of emphasizing the sound of a target area and suppressing the sound of other areas.

  There is a beam former (Beam Former; hereinafter referred to as BF) using a microphone array as a technology for separating and collecting only a sound in a specific direction in an environment where a plurality of sound sources exist. BF is a technology for forming directivity by using the time difference between signals arriving at each microphone (see Non-Patent Document 1). BF can be roughly divided into two types: addition and subtraction.

  In particular, the subtractive BF has an advantage that directivity can be formed with a smaller number of microphones than the additive BF.

  FIG. 7 is a block diagram showing a configuration according to a conventional subtraction type BF.

  In the conventional subtraction type BF shown in FIG. 7, the number of microphones is two.

  The conventional subtractive BF first calculates the time difference between the signals arriving at each target microphone (hereinafter also referred to as "target sound") by means of a delay device, and adds a delay to the target sound. Match the phase. In the conventional subtraction type BF delayer, the time difference is calculated by the following equation (1).

In the following equation (1), d is the distance between the microphones, c is the speed of sound, and τ _i is the delay amount. Further, in the following equation (1), θ _L is an angle from a perpendicular direction to a target direction with respect to a straight line connecting the microphones.
τ _L = (d sin θ _L ) / c (1)

Here, when the dead angle is in the direction of the first microphone with respect to the centers of the first microphone and the second microphone, the delay device in the conventional subtractive BF is the input signal x ₁ (the first microphone) Perform delay processing for t). Thereafter, the delayed input signal x ₁ (t) is subtracted according to equation (2).
a (t) = x ₂ (t) -x ₁ (t-τ _L ) (2)

The subtraction process in the conventional subtraction type BF can be similarly performed in the frequency domain, and in that case, the equation (2) is changed to the following equation (3).

Here, in the case of θ _L = ± π / 2, the directivity formed is a cardioid single directivity as shown in FIG. 8A, and in the case of θ _L = 0, π, FIG. It becomes a figure 8-shaped bi-directional like (B). In the following, a filter that forms unidirectionality from an input signal is referred to as a unidirectional filter, and a filter that forms bidirectionality is referred to as a bidirectional filter.

In addition, by using spectral subtraction (hereinafter also referred to as "SS"), strong directivity can be formed in a bidirectional blind spot. The formation of directivity by SS follows equation (4). Although the input signal X1 of the _first microphone is used in the equation (4), the same effect can be obtained with the input signal X2 of the _second microphone. Here, β is a coefficient for adjusting the intensity of SS. If the value becomes negative at the time of subtraction, floor processing is performed to replace 0 or the original value with a smaller value. In this method, bi-directional filters are used to extract sounds (hereinafter also referred to as "non-target sounds") present in other than the target direction, and the power spectrum of the non-target sounds extracted is subtracted from the power spectrum of the input signal. , Can emphasize the target sound.
| Y (ω) | = | X ₁ (ω) | -β | Α (ω) | (4)

  If you want to pick up only the sounds that exist in a specific area (hereinafter referred to as the "target area sound"), using the subtractive BF only, the sound sources that exist around that area ("non-target area") It may also pick up the sound). Therefore, Patent Document 1 proposes a method of collecting a target area sound by directing directivity to the target area from different directions and crossing the directivity in the target area using a plurality of microphone arrays.

  Next, an example of the sound pickup process of the target area sound described in Patent Document 1 will be described.

  FIG. 9 is an explanatory view showing a configuration example of each microphone array when the target area sound from the sound source of the target area is picked up using the two microphone arrays MA1 and MA2.

  FIG. 10 is an explanatory diagram (graph) showing the BF output of each of the microphone arrays MA1, MA2 shown in FIG. 9 in the frequency domain. FIGS. 10A and 10B are graphs (image views) showing the BF output of the microphone arrays MA1 and MA2 in the frequency domain, respectively.

In the method described in Patent Document 1, first, the ratio of the power of the target area sound included in the BF output of each of the microphone arrays MA1 and MA2 is estimated, and this is used as a correction coefficient. Specifically, when using two microphone arrays MA1 and MA2, the correction coefficient of the target area sound power may be calculated, for example, by (5), (6) or (7), (8) it can.

Here, Y _1k (n) and Y _2k (n) are the power spectra of the BF output of the microphone arrays MA 1 and MA 2, N is the total number of frequency bins, k is the frequency, and α (n) is the power correction coefficient for the BF output is there. Also, mode represents a mode value, and median represents a median value. Thereafter, each BF output is corrected by the correction coefficient, and the non-target area sound present in the direction of the target area is extracted by performing SS. Furthermore, the target area sound can be extracted by performing SS on the extracted non-target area sound from the output of each BF.

  FIG. 11 is an explanatory view (image) showing a change in power spectrum of each component when area sound collection processing is performed based on the BF output acquired using the microphone arrays MA1 and MA2 shown in FIG.

First, the input signal _{X 1} of the microphone array MA1, obtain BF output _{Y 1} that suppresses the non-target area sound _{N 2} (see FIG. 11 (a)).

In order to extract non-target area sounds N ₁ (n) present in the direction of the target area viewed from the microphone array MA 1, the BF output Y ₂ (n) of the microphone array MA ₁ is extracted from the microphone array MA ₂ as shown in equation (7). The product of the BF output Y ₂ (n) and the power correction coefficient α is SS (see FIG. 11B). Thereafter, according to the equation (8), the non-target area sound is SS from each BF output to extract the target area sound (see FIG. 11 (c)). γ (n) is a coefficient for changing the intensity at SS.
N ₁ (n) = Y ₁ (n) -α (n) Y ₂ (n) (7)
Z ₁ (n) = Y ₁ (n) -γ (n) N ₁ (n) (8)

  In order to extract the target area sound, the SS which is non-linear processing is performed by the equations (4) and (8), so that in a high noise environment, unpleasant noise called musical noise may be generated.

Therefore, in Patent Document 2, a section in which the target area sound is present and a section in which the target area sound does not exist are determined, and in the section where the target area sound is not present, abnormal noise such as musical noise is output. I'm holding back. Power spectrum ratio (area sound) between the input signal and the output from which the target area sound is extracted (hereinafter referred to as "area sound output") according to equation (9) to determine whether the target area sound exists Calculate the output / input signal). If there is a sound source in the destination area, since the destination area sound to the input signal X ₁ and Area sound output Z ₁ is included in the common power spectral ratios object area sound component is a value close to 1. Conversely, since the non-target area sound component is suppressed by the area sound output, the power spectrum ratio becomes a small value. In addition, since the SS is performed a plurality of times in the area sound collection process for other background noise components, the power spectrum ratio is reduced to a small value without using a dedicated noise suppression process in advance. Conversely, when the target area sound does not exist, the area sound output contains only weak residual noise as compared to the input signal, so the power spectral ratio has a small value in the entire region. Due to this feature, when the average of the power spectrum ratio obtained at each frequency according to equation (10) (hereinafter also referred to as "average power spectrum ratio") is taken, a large difference occurs between the presence and absence of the target area sound. Will be born. Here, m and n are respectively the upper limit and the lower limit of the processing band, and may be, for example, 100 Hz to 6 kHz in which audio information is sufficiently included. Then, in the device described in Patent Document 2, the average power spectrum ratio is determined by a preset threshold value, and when it is determined that the target area sound does not exist, the area sound output data is not output but silent or input Output sound with reduced sound gain.

JP, 2014-072708, A JP, 2016-127457, A

Asano Ta, "Sound Technology Series 16 Array signal processing of sound-Localization, tracking and separation of sound source", Japan Acoustical Society, edited by Corona, February 25, 2011

  If the method described in Patent Document 1 is used, the target area sound can be collected even if the non-target area sound exists around the target area. Moreover, if the method described in Patent Document 2 is used, the influence of musical noise generated in the area sound collection process can be suppressed. However, in a high noise environment such as an event site where there are many people or where music is flowing around, the SN ratio may deteriorate, and the power spectrum of the sound output by the area pickup may be reduced. is there. In such a situation, the average power spectrum ratio of the area pickup output and the input signal also becomes small. In particular, in components with originally small power spectrum such as unvoiced consonants, the difference with the average power spectrum ratio of the non-target area sound section becomes small, so that the judgment accuracy of the target area sound deteriorates and part of the target area sound is missing There is a risk of

  In view of the above problems, there is a demand for a sound collection device, program and method, determination device, program and method capable of improving the determination accuracy of the target area sound in an environment where background noise is strong. .

The sound pickup apparatus according to the first aspect of the present invention comprises (1) directivity forming means for forming directivity in the direction of a target area from an input signal by a beam former, and (2) directivity formed by the directivity forming means. Non-target area sound extraction means for extracting non-target area sound present in the target area direction; (3) non-target area present in the target area direction extracted by the non-target area sound extraction section from the output of the beam former (5) target area sound extraction means for outputting the extraction sound of the result of extracting the target area sound using the sound; (4) band division means for dividing the input signal and the extraction sound into a plurality of bands; ) Power spectrum ratio calculation means for calculating the power spectrum ratio of the input signal and the extracted sound for each divided band divided by the band division means, and (6) calculation of the power spectrum ratio Determining means for determining whether or not the target area sound is present in the input signal using the power spectrum ratio for each divided band calculated in the step; and (7) determining that the target area sound is present in the determination means And output means for outputting the extracted sound as a sound collection result when being received, and (8) an all-band average power spectrum ratio calculating means for calculating an average power spectrum ratio of all bands of the input signal and the extracted sound. (9) The determining means first determines whether the target area sound is present in the input signal based on the average power spectrum ratio of the entire band calculated by the all band average power spectrum ratio calculating means. And (10) the band dividing unit can not determine whether the target area sound is present in the input signal in the first determination process. And the extracted sound are each divided into a plurality of bands, and (11) the power spectrum ratio calculating means can not determine whether the target area sound is present in the input signal in the first determination process. In this case, the power spectrum ratio of the input signal and the extracted sound is calculated for each of the bands divided by the band division means, and (12) the determination means aims at the input signal in the first determination process. A second determination that determines whether or not a target area sound is present in the input signal from the power spectrum ratio calculated by the power spectrum ratio calculation means when it can not be determined whether the area sound is present or not. It is characterized by performing processing .

A sound pickup program according to a second aspect of the present invention comprises a computer, (1) directivity forming means for forming directivity in a target area direction from an input signal by a beam former, and (2) the directivity forming means Non-target area sound extraction means for extracting non-target area sound existing in the target area direction by directivity, and (3) present in the target area direction extracted by the non-target area sound extraction means from the output of the beam former Target area sound extraction means for outputting the extraction sound of the result of extracting the target area sound using non-purpose area sound; (4) Band division means for dividing the input signal and the extraction sound into a plurality of bands respectively (5) power spectrum ratio calculating means for calculating a power spectrum ratio of the input signal and the extracted sound for each divided band divided by the band dividing means; A determination unit that determines whether a target area sound is present in the input signal using the power spectrum ratio of each divided band calculated by the word spectrum ratio calculation unit; (7) the target area sound by the determination unit Means for outputting the extracted sound as a sound collection result when it is determined that there is a (8) total band average power spectrum ratio calculation for calculating an average power spectrum ratio of the entire band of the input signal and the extracted sound (9) The determination means first determines whether the target area sound is present in the input signal based on the average power spectrum ratio of the all bands calculated by the all band average power spectrum ratio calculation means. (10) The band dividing means determines whether the target area sound is present in the input signal in the first determination process. If it does not occur, the input signal and the extracted sound are divided into a plurality of bands respectively, and (11) the power spectrum ratio calculating means determines that the target area sound exists in the input signal in the first determination process. If it can not be determined, the power spectrum ratio of the input signal and the extracted sound is calculated for each of the bands divided by the band division means, and (12) the determination means determines the first When it is not possible to determine whether the target area sound exists in the input signal in the determination processing, whether the target area sound exists in the input signal from the power spectrum ratio calculated by the power spectrum ratio calculation means It is characterized in that a second determination process of determining

Sound collection method of the third invention, (1) directivity forming means, non-target area sound extraction unit, destination area sound extraction unit, the band dividing means, the power spectrum ratio calculating means, determining means, outputs means, and has a total band average power spectrum ratio calculating means, (2) the directional forming means, a directional formed from the input signal to the destination area direction by the beam former, (3) the non-target area sound extraction means, The non-target area sound present in the direction of the target area by directivity formed by the directivity forming means is extracted, and (4) the target area sound extraction means extracts the non-target area sound from the output of the beam former The extraction means outputs the extraction sound as a result of extracting the target area sound using the non-target area sound existing in the direction of the target area extracted, and (5) the band dividing means divides the input signal and the extraction sound (6) The power spectrum ratio calculation means calculates the power spectrum ratio of the input signal and the extracted sound for each divided band divided by the band division means (6) 7) The determination means determines whether or not the target area sound is present in the input signal using the power spectrum ratio of each divided band calculated by the power spectrum ratio calculation means, and (8) the output The means outputs the extracted sound as a sound collection result when it is determined that the target area sound exists by the determination means, and (9) the all-band average power spectral ratio calculating means calculates the input signal and the extracted sound Calculating the average power spectrum ratio of all the bands, and (10) the determining means first calculates the average power spectrum ratio of the all bands calculated by the all band average power spectrum ratio calculating means. A first determination process of determining whether or not the target area sound is present in the input signal, and (11) the band dividing means performs the target area sound in the input signal in the first determination process. When it can not be determined whether or not it exists, the input signal and the extracted sound are each divided into a plurality of bands, and (12) the power spectral ratio calculation means calculates the input signal in the first determination process. Power spectrum ratio of the input signal and the extracted sound is calculated for each of the bands divided by the band dividing means when it can not be determined whether the target area sound exists in the target area sound, and (13) the judgment When it is not possible to determine whether the target area sound is present in the input signal in the first determination process, the means determines the input signal from the power spectrum ratio calculated by the power spectrum ratio calculation means. A second determination process of determining whether a target area sound is present is performed .

  According to the present invention, it is possible to improve the determination accuracy of the target area sound under the environment where the background noise is strong.

It is the block diagram shown about the functional composition of the sound collection device (determination device) concerning a 1st embodiment. It is the figure (graph) shown about the example which the frequency band division part which concerns on 1st Embodiment divided | segmented the power spectrum of a process target signal for every division | segmentation band. It is the figure (graph) shown about the average power spectrum ratio for every division | segmentation band which the average power spectrum ratio calculation part classified by zone which concerns on 1st Embodiment calculated. It is the block diagram shown about the functional composition of the sound collection device (determination device) concerning a 2nd embodiment. It is the flowchart shown about operation | movement of the object area sound determination process of the sound collection apparatus (determination apparatus) which concerns on 2nd Embodiment. It is the block diagram shown about the functional composition of the sound collection device (determination device) concerning a 3rd embodiment. It is a block diagram which shows the structure which concerns on the conventional subtraction type BF in case the number of microphones is two. It is a figure which shows the directivity characteristic formed by the subtractive BF using two conventional microphones. It is explanatory drawing shown about the structural example of each microphone array in, when collecting the target area sound from the sound source of a target area using two conventional microphone arrays. It is explanatory drawing shown in the frequency domain about each BF output of two conventional microphone arrays. It is explanatory drawing shown about the change of the power spectrum of each component in the case of carrying out area sound collection processing based on BF output acquired using the two conventional microphone arrays.

(A) First Embodiment Hereinafter, a first embodiment of a sound collection device, program and method, determination device, program and method according to the present invention will be described in detail with reference to the drawings.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing a functional configuration of the sound collection device 100 of this embodiment.

  The sound collection device 100 performs the target area sound collection process of collecting the target area sound from the sound source of the target area using the two microphone arrays MA (MA1, MA2).

  The microphone arrays MA1 and MA2 are disposed at arbitrary places in the air where the target area exists. For example, as shown in FIG. 9, the positions of the microphone arrays MA1 and MA2 with respect to the target area may be anywhere as long as the directivity overlaps only in the target area. For example, the microphone arrays MA1 and MA2 may be disposed opposite to each other across the target area. Each microphone array MA is composed of two or more microphones M, and each microphone M picks up an acoustic signal. In this embodiment, two microphones M (M1 and M2) for picking up an acoustic signal will be described as being arranged in each microphone array MA. That is, each microphone array MA constitutes a 2ch microphone array. The number of microphone arrays MA is not limited to two. When there are a plurality of target areas, it is necessary to arrange the number of microphone arrays MA that can cover all the areas.

  The sound collection device 100 includes a data input unit 1, directivity formation unit 2, delay correction unit 3, space coordinate data 4, target area sound power correction coefficient calculation unit 5, target area sound extraction unit 6, frequency band division unit 7, A band-wise average power spectrum ratio calculating unit 8 and an area sound judging unit 9 are provided. Detailed processing of each functional block constituting the sound collection device 100 will be described later.

  In this embodiment, the sound collection device 100 that outputs the sound collection result of the target area sound will be described based on the determination processing result of whether the target area sound exists in the input signal. A determination device (determination program, determination method) that outputs the determination processing result of the target area sound by omitting the output unit (a part of the processing of the area sound determination unit 9) that outputs the sound collection result of the target area sound It may be configured.

  The sound collection device 100 may be configured entirely by hardware (for example, a dedicated chip or the like), or may be configured as software (program) for part or all. The sound collection device 100 may be configured, for example, by installing a program (including the determination program and the sound collection program of the embodiment) in a computer having a processor and a memory.

(A-2) Operation of First Embodiment Next, an operation (a determination method and a sound collection method according to the embodiment) of the sound collection device 100 of the first embodiment having the configuration as described above will be described. .

  The data input unit 1 converts an acoustic signal collected by each of the microphone arrays MA1 and MA2 from an analog signal to a digital signal. Then, the data input unit 1 performs conversion processing (for example, processing of converting from the time domain to the frequency domain using fast Fourier transform or the like) on the digital signal.

  The directivity forming unit 2 extracts non-target area sound present in other than the target direction (for example, by a bi-directional filter) for each microphone array MA, and extracts the amplitude spectrum of the extracted non-target area sound as an input signal. By subtracting from the amplitude spectrum, a sound (BF output) in which directivity is formed in the direction of the target area is acquired. Specifically, the directivity forming unit 2 acquires, as the BF output, the sound in which the directivity is formed in the direction of the target area by BF according to the equation (4) for each microphone array MA. When the input signal is not the microphone array MA but the signal input from the directional microphone, the processing of the directivity forming unit 2 is omitted, and the input signal is supplied to the subsequent stage as it is. It is also good.

  The delay correction unit 3 calculates and corrects a delay generated due to a difference in distance between the target area and each microphone array MA (MA1, MA2). The delay correction unit 3 acquires the position of the target area and the position of the microphone array from the space coordinate data 4, and calculates the difference in arrival time of the target area sound to each microphone array MA (MA1, MA2). Next, the delay correction unit 3 causes the target area sounds to simultaneously reach all the microphone arrays MA (MA1, MA2) on the basis of the microphone arrays MA (MA1, MA2) arranged farthest from the target area Add a delay to

  The spatial coordinate data 4 holds position information of all the target areas, the microphone arrays MA (MA1, MA2), and the microphones M (M1, M2) constituting the microphone arrays MA (MA1, MA2).

  The target area sound power correction coefficient calculation unit 5 calculates a correction coefficient for making the power of the target area sound component included in each BF output the same, according to equation (5) or (6).

  The target area sound extraction unit 6 SSs each BF output data corrected by the correction coefficient calculated by the target area sound power correction coefficient calculation unit 5 according to the equation (7) to extract noise present in the direction of the target area. Furthermore, the target area sound extraction unit 6 extracts the target area sound by performing SS on the extracted noise from the output of each BF according to the equation (8).

Frequency band division unit 7 obtains an input signal from the data input unit 1, and the area sound output Z ₁ from the target area sound extraction unit 6 divides each of a plurality of bands. Here, it is assumed that the bandwidths of the input signal and the area sound output are the same.

In the following, as an input signal to be processed in the frequency band division unit 7 and the band-by-band average power spectrum ratio calculating unit 8, it is assumed to be used on behalf of the input signal X ₁ of the microphone array MA1, other microphone (other It may be replaced by an input signal of the microphone array MA (which may be a microphone).

The frequency band dividing unit 7 divides, for example, signals to be processed (input signal X ₁ and area sound output Z ₁ ) at predetermined frequency bandwidths (constant intervals or indefinite intervals). Hereinafter, the frequency bands obtained by dividing the signal to be processed by the frequency band dividing unit 7 into a plurality of parts are referred to as “division bands”, and the signals of each division band (signals divided from the division target signals) are divided into “division band signals I shall call it ".

  The frequency band dividing unit 7 may set the bandwidths of the divided bands to be equal (equally spaced), or may be set to be biased depending on the frequency band. For example, the frequency band division unit 7 may set the division band wider as the frequency is higher (set the division band narrower as the lower frequency). For example, the frequency band division unit 7 sets division bands at 100 Hz intervals for low frequency bands (for example, less than 1 kHz), and sets division bands at 1 kHz intervals for bands that are not low frequencies (for example, 1 kHz or more). You may do so.

  Further, the frequency band dividing unit 7 sets divided bands within a predetermined range of band (for example, a range of 100 hz to 6 kHz) in which voice information (component of voice) is sufficiently included, and signals of other frequency bands are set. It may be made to round off (it cuts off as the object of a division | segmentation band).

  In the example of this embodiment, in order to simplify the description, the frequency band division unit 7 divides the signal to be processed into division bands at intervals of 1 kHz, and the following description will be made.

  FIG. 2 is a diagram (a graph showing the power spectrum for each band) showing an example of the processing target signal processed by the frequency band division unit 7.

FIG. 2 shows an example in which the frequency band dividing unit 7 divides the processing target signal in the band of 100 Hz to 6 kHz into six divided bands B _{1 to} B ₆ at intervals of approximately 1 khz.

The band-wise average power spectrum ratio calculation unit 8 calculates the power spectrum for each divided band (division band signal) divided by the frequency band division unit 7 for each processing target signal (input signal X ₁ and area sound output Z ₁ ). Extract (acquire) Then, the band-wise average power spectrum ratio calculating unit 8 calculates an average power spectrum ratio (average of power spectrum ratios in each divided band) based on equation (11) for each divided band.

In equation (11), “R _j ” is the average power spectral ratio in the j-th divided band (j is an integer from 1 to M; M is the total number of divided bands (number of divided bands)) . Further, in the equation (11), “X _1j ” is an average power spectrum (average value of power spectrum) within the j-th divided band in the input signal X ₁ of the microphone array MA ₁ , and “Z _1j ” is an area sound is the average power spectrum of the j-th divided band in the output Z ₁ (average value of the power spectrum).

For example, as shown in FIG. 2, it is assumed that the frequency band dividing unit 7 divides each processing target signal (input signal X ₁ and area sound output Z ₁ ) into six divided bands B _{1 to} B _6. . In this case, the band-by-band average power spectrum ratio calculating unit 8 and the band-by-band average power spectrum ratio calculating unit 8 respectively input the average power spectra X _{11 to} X _{16 of the} input signal from divided bands B _{1 to} B ₆ of the input signal X _1. To get Further, the band-wise average power spectrum ratio calculating unit 8 acquires the average power spectra Z _{11 to} Z ₁₆ of the area sound output from the divided bands B _{1 to} B ₆ of the area sound output Z ₁ respectively.

Then, the band average power spectrum ratio calculation unit 8 applies X _{11 to} X ₁₆ and Z _{11 to} Z ₁₆ to equation (11) to calculate average power spectrum ratios R _{1 to} R ₆ for each divided band. .

FIG. 3 is a diagram (graph) showing the average power spectrum ratios R _{1 to} R ₆ for each divided band calculated by the band average power spectrum ratio calculation unit 8.

FIG. 3 shows the average power spectrum ratio R _{1 to} R ₆ for each divided band and the average power spectrum (the value at the right end) in the entire band.

Then, the band average power spectrum ratio calculation unit 8 calculates the largest value (average power spectrum ratio) from the maximum power spectrum ratio U according to the equation (12) from the average power spectrum ratios R _{1 to} R ₆ for each divided band. Get as _max .

For example, when the values of the average power spectrum ratio R _{1 to} R ₆ for each division band are as shown in FIG. 3, the maximum average power spectrum ratio U _max is the value of the division band B ₆ and It can be seen that the value is larger than the average power spectrum.

The area sound determination unit 9 compares the maximum average power spectrum ratio U _max calculated by the band average power spectrum ratio calculation unit 8 with the threshold value T1 set in advance to determine whether the target area sound exists (the purpose of the input signal is It is determined whether an area sound is included. The area sound determination unit 9 determines that the target area sound exists, for example, when the maximum average power spectrum ratio U _max exceeds the threshold T1, and the target area sound is present when the maximum average power spectrum ratio U _max is less than or equal to the threshold T1. It may be determined that it does not exist.

When it is determined that the target area sound is present, the area sound determination unit 9 may output the area sound collection processing data (area sound output Z ₁ (extraction sound)) as it is. On the other hand, when it is determined that the target area sound does not exist, the area sound determination unit 9 outputs silent audio data without outputting the area sound collection processing data (area sound output Z ₁ (extraction sound)). You may do so. The area sound determination unit 9 may output a signal obtained by reducing the gain of an input signal (for example, the input signal X ₁ of the microphone array MA1) instead of silent voice data.

(A-3) Effects of the First Embodiment According to the first embodiment, the following effects can be achieved.

In the sound collection device 100 of the first embodiment, the input signal (in the above example, X ₁ ) and the area sound output Z ₁ are divided into a plurality of divided bands, and the average power spectral ratio for each divided band is determined. Based on the value, the maximum average power spectrum ratio U _max , it is determined whether the target area sound exists.

In other words, in the sound collection device 100 according to the first embodiment, it is determined that the target area sound is present if there is one divided band in which the average power spectrum ratio exceeds the threshold (T1 in the above example). When human voice is used as the target area sound, the power of unvoiced consonants is small, but there is a peak in the power spectrum, so if the band is divided, the power of the band including the peak becomes large. Because of the characteristics as described above, the maximum value of the average power spectrum ratio per division band (maximum average power spectrum ratio U _max ) is different from the average power spectrum ratio of all bands (eg, musical noise etc.) The difference between the non-target area sound section in which noise is occurring and the target area sound section becomes clear. Therefore, in the sound collection device 100 according to the first embodiment, the determination accuracy of the target area sound is strong in an environment where background noise is strong as compared with the conventional target area sound determination using the average power spectrum ratio of all bands. It can be improved.

Furthermore, in the first embodiment, since the target area sound is determined using the maximum value of the average power spectrum ratio (maximum average power spectrum ratio U _max ) for each divided band, the band used for the target area sound determination is a peak. Since the determination is not performed using only one sample while being localized to the periphery, it is possible to perform a stable determination process that is less susceptible to noise generated in a burst manner.

(B) Second Embodiment Hereinafter, a second embodiment of a sound collection device, program and method, determination device, program and method according to the present invention will be described in detail with reference to the drawings.

(B-1) Configuration of Second Embodiment FIG. 4 is a block diagram showing a functional configuration of the sound collection device 100A of this embodiment. In FIG. 4, the same reference numerals or corresponding reference numerals are given to the same or corresponding parts as those in FIG. 1 described above.

  In the following, differences from the first embodiment will be described in the sound collection device 100A of the second embodiment.

  The sound collection device 100A differs from the first embodiment in that the area sound determination unit 9 is replaced with the area sound determination unit 9A, and further, the all band average power spectrum ratio calculation unit 10 is added.

  The all-band average power spectrum ratio calculation unit 10 calculates an average power spectrum ratio in all bands.

  The area sound determination unit 9A controls the frequency band division unit 7, the band average power spectrum ratio calculation unit 8, and the all band average power spectrum ratio calculation unit 10 to determine the presence or absence of the target area sound.

(B-2) Operation of Second Embodiment Next, regarding the operation (the determination method and the sound collection method according to the embodiment) of the sound collection device 100A of the second embodiment having the configuration as described above. The difference from the embodiment of FIG.

  The sound collection device 100A is different from the first embodiment in that the determination process of the target area sound by the area sound determination unit 9A is different. Hereinafter, determination processing of the target area sound centering on the area sound determination unit 9A will be described.

  FIG. 5 is a flowchart showing the process of determining a target area sound by the sound collection device 100A (area sound determination unit 9A).

  In the flowchart of FIG. 5, T1, T2, and T3 used in the area sound determination process are threshold values. The threshold value T1 can be the same as that in the first embodiment. Further, the threshold T2 is a value larger than the threshold T3 (T2> T3). The magnitude relationship between “T1” and “T2, T3” is not limited, and a suitable value confirmed by an experiment or the like can be applied.

  First, the area sound determination unit 9A controls the all-band average power spectrum ratio calculation unit 10 to calculate the all-band average power spectrum ratio (S101).

  The all-band average power spectrum ratio calculation unit 10 calculates the all-band average power spectrum ratio according to equations (9) and (10).

  Next, the area sound determination unit 9A determines whether the all-band average power spectrum ratio calculated by the all-band average power spectrum ratio calculation unit 10 exceeds the threshold T2 (whether U> T2 or not) ( S102). The area sound determination unit 9A operates from step S104 described later when the all-band average power spectrum ratio exceeds the threshold value T2, and otherwise operates from step S103 described later.

  If the all-band average power spectrum ratio exceeds the threshold T2 (in the case of U> T2), the area sound determination unit 9A determines that the target area sound exists (S104), and ends the target area sound determination process. .

  On the other hand, if the all band average power spectrum ratio is less than or equal to the threshold T2 (in the case of U ≦ T2), the area sound determination unit 9A determines whether the all band average power spectrum ratio exceeds the threshold T3 (U> T3? Or not) is judged (S103). The area sound determination unit 9A operates from step S105 described later when the all-band average power spectrum ratio exceeds the threshold T3, and operates from step S108 described later otherwise.

  When the all band average power spectrum ratio exceeds the threshold T3 (in the case of U> T3), the area sound determination unit 9A controls the frequency band division unit 7 and the band average power spectrum ratio calculation unit 8 to The average power spectrum ratio is calculated for each divided band by the same processing as that of the first embodiment (S105).

Next, the area sound judging unit 9A controls the band average power spectrum ratio calculating unit 8 to calculate the maximum average power spectrum ratio U _max from the average power spectrum ratio for each divided band, as in the first embodiment. It is calculated and it is determined whether the maximum average power spectrum ratio U _max exceeds the threshold T1 (S106). In other words, the area sound determination unit 9A and the per-band average power spectrum ratio calculation unit 8 perform processing to determine whether or not there is an average power spectrum ratio for each divided band that exceeds the threshold T1.

If the maximum average power spectrum ratio U _max exceeds the threshold T 1 (if there is an average power spectrum ratio for each divided band exceeding the threshold T 1), the area sound determination unit 9 A operates from step S 107 described later, and not so In this case, the operation starts from step S108 described later.

When the maximum average power spectrum ratio U _max exceeds the threshold T1 (in the case of U _max > T1), the area sound determination unit 9A determines that the target area sound exists (S107), and ends the target area sound determination process. .

On the other hand, if the all band average power spectrum ratio is less than or equal to the threshold T3 in step S103 described above (U ≦ T3), or if the maximum average power spectrum ratio U _max is less than or equal to the threshold T1 in step S106 described above (U _max ≦ In the case of T1), the area sound determination unit 9A determines that the target area sound does not exist (S108), and ends the process of determining the target area sound.

  The area sound determination unit 9A first causes the all band average power spectrum ratio calculation unit 10 to calculate the all band average power spectrum ratio, and the target area sound determination process based on the all band average power spectrum ratio (hereinafter 1) is performed). Specifically, as described above, the area sound determination unit 9A determines that the target area sound is present when the all-band average power spectrum ratio is larger than the threshold T2, and the all-band average power spectrum ratio is less than or equal to the threshold T3. It is determined that the target area sound does not exist.

The area sound determination unit 9A determines that the target area sound can not be determined in the first determination process when the all-band average power spectrum ratio is less than or equal to the threshold T2 and exceeds the threshold T3 (T2 ≦ U> T3). Determine the maximum average power spectrum ratio U _max by the same processing as in the first embodiment by controlling the frequency band dividing unit 7 and the band average power spectrum ratio calculating unit 8, and calculating the maximum average power spectrum ratio A process of determining the presence or absence of the target area sound based on U _max (hereinafter, referred to as “second determination process”) is performed.

(B-3) Effects of Second Embodiment According to the second embodiment, the following effects can be achieved.

The sound collection device 100A (area sound determination unit 9A) according to the second embodiment first performs target area sound determination processing (first determination processing) based on the whole band average power spectrum ratio, and all band average power spectrum When the ratio is less than or equal to the threshold T2 and exceeds the threshold T3 (T2 ≦ U> T3), the target area sound determination process (second determination process) is performed based on the maximum average power spectrum ratio _Umax .

Thereby, the case where the target area sound determination processing can be performed with sufficient accuracy only with the first determination processing (the determination processing based on the all-band average power spectrum ratio) of the sound collection device 100A (area sound determination unit 9A) (For example, when the all band average power spectrum ratio is sufficiently large), the second determination processing (processing of band division and the like) is not performed. On the other hand, when the sound pickup device 100A (area sound judgment unit 9A) can not judge the target area sound with sufficient accuracy in the first judgment process (the judgment process based on the all band average power spectrum ratio) (for example, The second determination processing (processing for calculating the maximum average power spectrum ratio U _max by band division and determining the target area sound) is performed only when the average power spectrum ratio U is small as in the case of unvoiced consonants.

  That is, the sound collection device 100A (area sound determination unit 9A) performs the second process involving band division with a larger amount of processing only when the determination process of the target area sound can be performed with sufficient accuracy in the first determination process. Since the determination process is performed, the target area sound determination process can be performed efficiently.

(C) Third Embodiment Hereinafter, a third embodiment of a sound collection device, program and method, determination device, program and method according to the present invention will be described in detail with reference to the drawings.

(C-1) Configuration of Third Embodiment FIG. 6 is a block diagram showing a functional configuration of the sound collection device 100B of this embodiment. In FIG. 6, the same or corresponding parts as those in FIG. 1 described above are denoted by the same reference numerals.

  In the following, differences from the first embodiment will be described in the sound collection device 100B of the third embodiment.

  The sound collection device 100B differs from the first embodiment in that the area sound determination unit 9 is replaced with the area sound determination unit 9B and the inter-band power spectrum ratio calculation unit 11 is further added.

The inter-band power spectrum ratio calculation unit 11 calculates a minimum value (hereinafter referred to as "minimum average power spectrum ratio U _min ") from the average power spectrum ratio for each divided band obtained by the band average power spectrum ratio calculation unit 8 Do. Then, the inter-band power spectrum ratio calculation unit 11 calculates the maximum average power spectrum ratio U _max (the maximum value of the average power spectrum ratio R for each divided band) calculated by the band average power spectrum ratio calculation unit 8 and the minimum average power The ratio of the spectrum ratio U _min (hereinafter referred to as “inter-band power spectrum ratio V”) is determined.

  The area sound determination unit 9B is different from the first embodiment in that the target area sound is determined based on the inter-band power spectrum ratio V.

  In the second embodiment, the second determination process may be replaced with a determination process using the inter-band power spectrum ratio V.

(C-2) Operation of Third Embodiment Next, an operation (a determination method and a sound collection method according to the embodiment) of the sound collection device 100B of the third embodiment having the above configuration will be described. The difference from the embodiment of FIG.

  The sound collection device 100B is different from the first embodiment in that the determination process of the target area sound by the area sound determination unit 9B is different. Hereinafter, determination processing of the target area sound centering on the area sound determination unit 9B will be described.

The inter-band power spectrum ratio calculating unit 11 obtains the minimum average power spectrum ratio U _min from the average power spectrum ratio for each divided band obtained by the band average power spectrum ratio calculating unit 8 according to the equation (13).

Then, the inter-band power spectrum ratio calculation unit 11 calculates the inter-band power spectrum ratio V based on the maximum average power spectrum ratio U _max and the minimum average power spectrum ratio U _min according to equation (14).

For example, when the value of the average power spectrum ratio (R _{1 to} R ₆ ) for each division band is as shown in FIG. 3, the maximum average power spectrum ratio U _max is the value of division band B ₆ and the minimum average the value of the power spectral ratios _{U min} is a value divided band _{B 3.}

  The area sound determination unit 9B compares the inter-band power spectrum ratio V with the threshold T4, and determines that the target area sound exists if the inter-band power spectrum ratio V is larger than the threshold T4 (V> T4), When the inter-band power spectrum ratio V is equal to or less than the threshold value T4 (when V ≦ T4), it is determined that the target area sound does not exist.

(C-3) Effects of Third Embodiment According to the third embodiment, the following effects can be achieved as compared to the first embodiment.

  In the sound collection device 100B of the third embodiment, the target area sound is detected based on the inter-band power spectrum ratio V, so that the target area sound component of a smaller power spectrum can also be detected.

(D) Other Embodiments The present invention is not limited to the above-described embodiments, and may include modified embodiments as exemplified below.

(D-1) In the above embodiments, when the maximum average power spectrum ratio U _max is larger than the threshold T1 by a certain amount or more, the area sound determination unit 9 (9A, 9B) determines the maximum average power for several seconds thereafter. It may be made to correspond to the function (hangover function) of judging that the target area sound exists regardless of the spectral ratio U _max .

(D-2) In the sound collection device (determination device) of each of the above embodiments, the average power spectrum ratio for each divided band is calculated, and the maximum average power spectrum ratio U _max as its maximum value is used as the target area sound determination. Although it is used, one representative value of the power spectrum ratio in each division band is acquired instead of the average value (average power spectrum ratio) of the power spectrum ratio in each division band, and the representative value (hereinafter referred to as “representation power” The maximum value (hereinafter referred to as “maximum representative power spectrum ratio”) of the “spectral ratio” may be used by replacing it with the maximum average power spectrum ratio U _max .

That is, in each of the above embodiments, the band average power spectrum ratio calculating unit 8 obtains the representative power spectrum ratio from each divided band, and sets the maximum value of the representative power spectrum ratio of each divided band as the maximum representative power spectrum ratio. It may be acquired and replaced with the maximum average power spectrum ratio U _max to be used for target area sound determination. In each of the above-described embodiments, the position at which the representative power spectrum ratio (representative value) is obtained from each divided band is not limited. However, for example, a median or the like may be obtained.

As described above, in each of the above embodiments, in the target area sound determination, the maximum value (for example, the maximum average power spectrum ratio U) of the power spectrum ratio (for example, the average power spectrum ratio or the representative power spectrum ratio) for each divided band _{Use max} , maximum representative power spectrum ratio, etc.).

  100 ... sound collecting device (determination device), 1 ... signal input unit, 2 ... directivity forming unit, 3 ... delay correction unit, 4 ... space coordinate data, 5 ... target area sound power correction coefficient calculation unit, 6 ... target area Sound extraction unit 7 Frequency band division unit 8 Average power spectrum ratio calculation unit according to band 9 Area sound determination unit MA, MA1, MA2 Microphone array M, M1, M2 Microphone.

Claims (5)

Directivity forming means for forming directivity from an input signal to a target area direction by a beam former;
Non-target area sound extraction means for extracting non-target area sound existing in the direction of a target area according to directivity formed by the directivity forming means;
Target area sound extraction means for extracting an extraction sound of a result of extracting a target area sound using the non-target area sound present in the direction of the target area extracted by the non-target area sound extraction means from the output of the beamformer ,
Band dividing means for dividing the input signal and the extracted sound into a plurality of bands respectively;
Power spectrum ratio calculating means for calculating a power spectrum ratio of the input signal and the extracted sound for each divided band divided by the band dividing means;
A determination unit that determines whether a target area sound is present in the input signal using the power spectrum ratio for each divided band calculated by the power spectrum ratio calculation unit;
An output unit that outputs the extracted sound as a sound collection result when it is determined by the determination unit that the target area sound is present ;
And an all-band average power spectrum ratio calculating unit configured to calculate an average power spectrum ratio of all the bands of the input signal and the extracted sound;
The determination means first determines whether the target area sound is present in the input signal based on the average power spectrum ratio of the entire band calculated by the all band average power spectrum ratio calculation means. Do the processing,
The band dividing means divides the input signal and the extracted sound into a plurality of bands when it is not possible to determine whether the target area sound is present in the input signal in the first determination process,
The power spectrum ratio calculation means may not perform the input for each of the bands divided by the band division means when it is not possible to determine whether the target area sound is present in the input signal in the first determination process. Calculating the power spectrum ratio of the signal and the extracted sound,
The determination means may not determine whether the target area sound is present in the input signal in the first determination process, the input signal may be calculated from the power spectrum ratio calculated by the power spectrum ratio calculation means. And a second determination process of determining whether a target area sound is present or not . When the average power spectrum ratio of the entire band calculated by the all-band average power spectrum ratio calculation unit exceeds the second threshold value in the first determination process, the determination unit determines the target area of the input signal. If it is determined that sound is present, and the average power spectrum ratio of the entire band calculated by the all-band average power spectrum ratio calculating means is a value equal to or less than a third threshold value smaller than the second threshold value When it is determined that the target area sound does not exist, and the average power spectrum ratio of the entire band calculated by the all-band average power spectrum ratio calculating unit exceeds the third threshold and is equal to or less than the second threshold. the pickup device according to claim 1, characterized in that to obtain a result that can not be determined whether the sound object area is present in the input signal. The determination means is a target area for the input signal according to the comparison result of the maximum value of the power spectrum ratio for each divided band calculated by the power spectrum ratio calculation means and the first threshold value in the second determination process. The sound pickup device according to claim 1 or 2 , wherein it is determined whether a sound is present. Computer,
Directivity forming means for forming directivity from an input signal to a target area direction by a beam former;
Non-target area sound extraction means for extracting non-target area sound existing in the direction of a target area according to directivity formed by the directivity forming means;
Target area sound extraction means for extracting an extraction sound of a result of extracting a target area sound using the non-target area sound present in the direction of the target area extracted by the non-target area sound extraction means from the output of the beamformer ,
Band dividing means for dividing the input signal and the extracted sound into a plurality of bands respectively;
Power spectrum ratio calculating means for calculating a power spectrum ratio of the input signal and the extracted sound for each divided band divided by the band dividing means;
A determination unit that determines whether a target area sound is present in the input signal using the power spectrum ratio for each divided band calculated by the power spectrum ratio calculation unit;
An output unit that outputs the extracted sound as a sound collection result when it is determined by the determination unit that the target area sound is present;
Function as total band average power spectrum ratio calculating means for calculating an average power spectrum ratio of all bands of the input signal and the extracted sound;
The determination means first determines whether the target area sound is present in the input signal based on the average power spectrum ratio of the entire band calculated by the all band average power spectrum ratio calculation means. Do the processing,
The band dividing means divides the input signal and the extracted sound into a plurality of bands when it is not possible to determine whether the target area sound is present in the input signal in the first determination process,
The power spectrum ratio calculation means may not perform the input for each of the bands divided by the band division means when it is not possible to determine whether the target area sound is present in the input signal in the first determination process. Calculating the power spectrum ratio of the signal and the extracted sound,
The determination means may not determine whether the target area sound is present in the input signal in the first determination process, the input signal may be calculated from the power spectrum ratio calculated by the power spectrum ratio calculation means. A sound collecting program characterized by performing a second determination process of determining whether or not there is a target area sound . A directivity forming means, non-target area sound extraction unit, destination area sound extraction unit, the band dividing means, the power spectrum ratio calculating means, determining means, outputs means, and the entire band average power spectrum ratio calculating means,
The directivity forming unit forms directivity from the input signal to a target area direction by a beam former,
The non-target area sound extraction unit extracts non-target area sound existing in the direction of the target area according to the directivity formed by the directivity formation unit.
The target area sound extraction means extracts an extraction sound as a result of extracting a target area sound from the output of the beamformer using the non-target area sound existing in the direction of the target area extracted by the non-target area sound extraction means. Output
The band dividing means divides the input signal and the extracted sound into a plurality of bands, respectively.
The power spectrum ratio calculating means calculates a power spectrum ratio of the input signal and the extracted sound for each divided band divided by the band dividing means,
The determination means determines whether or not a target area sound is present in the input signal, using the power spectrum ratio for each divided band calculated by the power spectrum ratio calculation means.
The output unit outputs the extracted sound as a sound collection result when it is determined by the determination unit that the target area sound exists.
The all band average power spectrum ratio calculating means calculates an average power spectrum ratio of all bands of the input signal and the extracted sound;
The determination means first determines whether the target area sound is present in the input signal based on the average power spectrum ratio of the entire band calculated by the all band average power spectrum ratio calculation means. Do the processing,
The band dividing means divides the input signal and the extracted sound into a plurality of bands when it is not possible to determine whether the target area sound is present in the input signal in the first determination process,
The power spectrum ratio calculation means may not perform the input for each of the bands divided by the band division means when it is not possible to determine whether the target area sound is present in the input signal in the first determination process. Calculating the power spectrum ratio of the signal and the extracted sound,
The determination means may not determine whether the target area sound is present in the input signal in the first determination process, the input signal may be calculated from the power spectrum ratio calculated by the power spectrum ratio calculation means. A second determination process of determining whether a target area sound is present in the second area .

Images