CN108597531B - A method to improve dual-channel blind signal separation by multi-source activity detection - Google Patents

Abstract

The invention discloses a method for improving dual-channel blind signal separation through multi-sound source activity detection. This method performs blind signal separation based on the dual-channel TRINICON algorithm, and compares the power before and after preliminary processing. If the signal power of one output channel is significantly lower than that of the other output channel, the target sound source to be suppressed in this signal can be determined. In the active state, according to this, it can be judged whether each target sound source in each segment of data is in the active state. The TRINICON algorithm is modified by using the results of multi-sound source activity judgment, and the filter coefficients are updated with the data of the target sound source activity, so as to achieve the purpose of removing interference and improving the performance of speech separation. The method of the invention can effectively improve the separation performance of the TRINICON method in the scene of discontinuous interlaced mixing and sparse mixing.

Description

A method to improve dual-channel blind signal separation by multi-source activity detection

technical field

The invention relates to the technical field of blind signal separation, in particular to a method for improving blind signal separation performance through multi-sound source activity detection based on a dual-channel system. An algorithm for multi-source activity detection is added to the signal separation process.

Background technique

The problem of separating convolutional mixtures of unknown time series has important applications in several fields. An important example is the so-called cocktail party problem, where a single speech signal is extracted from the mixture of multiple speakers in a reverberant acoustic environment. Due to the presence of reverberation, the source signal of this separation problem is filtered through a linear multiple-input multiple-output (MIMO) system before being collected by the microphone array. Blind Signal Separation (BSS) is an algorithm that uses a microphone array to separate the signals of multiple sound sources without prior information and based on the basic assumption that the signals of different sound sources are statistically independent of each other (S. Makino, H. Sawada). , and T.W. Lee, Blind Speech Separation. Springer Netherlands, 2007, pp. 169-192.).

Blind signal separation based on independent component analysis (ICA) is an effective method to extract desired speech signals from interfering speech signals. However, the frequency domain ICA method faces the problem of ambiguous ordering of each frequency point, which needs to be corrected by an appropriate supplementary repair mechanism based on the correlation information between frequencies. Currently, this problem can be solved by extending ICA to the multivariate case, performing the Independent Vector Analysis (IVA) algorithm, or by utilizing the TRINICON scheme based on the broadband standard. The TRINICON method can be efficiently implemented in the frequency domain using second order statistics (SOS).

Offline TRINICON methods in the frequency domain generally have the best performance when the sound sources are continuously mixed. In practical applications, the speech signal sometimes detects errors intermittently or is sparsely mixed, and the offline algorithm does not consider the activity state of the speech and counts each segment of speech equally into the calculation, which will lead to performance degradation, especially when Acausal filters need to be generated when each sound source is on one side of the midline of the microphone array.

SUMMARY OF THE INVENTION

The purpose of the present invention is to improve the performance of the frequency-domain TRINICON algorithm in the scenarios of discontinuous interlaced mixing and sparse mixing, and to provide a method for improving dual-channel blind signal separation through multi-sound source activity detection.

The technical scheme that the present invention takes for solving the above-mentioned technical problems is:

A method for improving dual-channel blind signal separation through multi-sound source activity detection, comprising the following steps:

(1) Initialize the filter matrix parameters in the frequency domain TRINICON algorithm;

(2) The sound source signal received by the microphone is input into the two-channel system, the input signal of each channel is divided into blocks, and the short-time Fourier transform is used to transform the input signal of each block into the frequency domain;

(3) Calculate the output signal of each block by Y ^(k) = X ^(k) W ^(k) , where Y ^(k) , X ^(k) and W ^(k) are the output signal in the frequency domain, the The filter coefficient matrix of the input signal and the frequency domain, the superscript (k) is the sequence number of the intermediate frequency point of the short-time discrete Fourier transform; then calculate the power spectral density matrix Φ _yy of each output signal;

(4) update the filter coefficients according to the natural gradient descent method, use the inverse short-time Fourier transform to transform the filter coefficient matrix back to the time domain and set the filter coefficients in the time domain to zero if the filter coefficients are greater than the filter length minus 1;

(5) using the filter coefficients obtained in step (4), repeat steps (3) and (4) until the maximum number of iterations is reached;

(6) obtain preliminary filter coefficient convergence result, utilize overlapping retention method to convolve the input signal to obtain the preliminary output signal of the first channel and the preliminary output signal of the second channel;

(7) Calculate the output and input power of each block in the two channels according to the following two formulas:

where E _x (m) is the input power, E _yp (m) is the power of the preliminary output signal, m is the sequence number of the signal block, ku and _{k l} represent the upper and lower limits of the frequency points included in the calculation, respectively, and the subscript p is the serial number of the output channel, the subscripts 1 and 2 represent the first channel and the second channel respectively;

Then judge the active state of each signal, and get the revised weight function;

(8) Re-initialize the weight function according to step (1), and the corrected gradient expression is:

in

N _sig is the block number of the whole signal, ε _p (m) is the modification weight, when the state of the p-th sound source in the m-th block is active, ε _p (m) is equal to 1, otherwise, ε _p (m) is equal to 0;

According to the above modified gradient expression, repeat steps (3) and (4), that is, perform an iteration again. In this iteration, only the data of the target sound source activity to be suppressed by the first channel is used to update the data of the first channel. filter coefficient, update the filter coefficient of the second channel with only the data of the target sound source activity to be suppressed by the second channel until the maximum number of iterations is reached;

(9) The final filter coefficient convergence result is obtained, and the input signal is convolved by the overlap preservation method to obtain the final output signal of the two channels.

The detection method of the present invention can approximately identify the activity of each source, extract the respective effective activity frames of the sound source, and adjust the filter update process according to the result of the sound source activity detection, which can effectively improve the performance of intermittent interlaced mixing and sparse mixing. The separation performance of the TRINICON method in the scene.

Description of drawings

FIG. 1 is a signal model diagram in an embodiment of the present invention: (a) only causal filters are required, (b) causal and non-causal filters are required.

FIG. 2 is a flowchart of multi-sound source activity detection in an embodiment of the present invention.

FIG. 3 is a flow chart of the frequency domain blind signal separation algorithm with multi-sound source activity detection added in the present invention.

Fig. 4 is the spatial response diagram of the final optimized filter in the embodiment of the present invention: (a) common frequency domain TRINICON output channel 1, (b) common frequency domain TRINICON output channel 2, (c) adding multi-sound source activity detection Frequency domain TRINICON output channel 1, (d) join frequency domain TRINICON output channel 2 for multi-sound source activity detection.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

1. Dual-channel frequency domain TRINICON algorithm

The present invention is suitable for a two-channel microphone array system, assuming that the number of sound sources is one or two, and the signal model of the algorithm is shown in Figure 1 (when there is only one signal, only one of the two sound sources in the model emits sound).

The cost function of the TRINICON algorithm is:

是第q通道的D维的多元概率密函数的估计，

是所有输出通道(这里Q＝2)的联合概率密度的估计，m是块序号，j＝0,…,N-1长度为N的块中时移的序号，β(i,m)是一个权重函数，根据

归一化。in,

is the estimate of the D-dimensional multivariate probability density function of the qth channel,

is the estimate of the joint probability density of all output channels (here Q=2), m is the block sequence number, j=0,...,N-1 The sequence number of the time shift in blocks of length N, β(i,m) is a weight function, according to

Normalized.

The cost function of TRINICON using SOS is expressed as (H. Buchner, R. Aichner, and W. Kellermann, "A generalization of blind source separation algorithms for convolutive mixtures based on second-order statistics," IEEE Transactions on Speech&Audio Processing, vol. 13, no .1, pp. 120-134, 2004):

The bdiag operation is to zero out the off-diagonal submatrices of a block matrix.

The filter coefficients W are updated according to the natural gradient descent method:

in

j _iter is the number of iterations, μ is the step size.

In the frequency domain method, the filter matrix and the signal are expressed as:

and

where L is the length of the filter and F _4L×4L is the Fourier transform matrix. Variables underlined are variables in the frequency domain.

The power spectral density matrix Φ _yy of the output signal and the power spectral density matrix Φ _xx of the input signal are expressed as

and

Among them, the role of the constraint matrix G of different dimensions is to prevent the decoupling between each frequency point to avoid the ordering ambiguity problem and the circular convolution effect in the frequency domain blind signal separation. When applied to the frequency domain calculation, if G is approximated as a unit matrix, it can be iterated independently at each frequency point (the block number m is omitted in the formula):

The superscript (k) is the serial number of the frequency point in the STFT. After each iteration, the demarcation matrix must be converted back to the time domain and the value of w _qq,l for l>L-1 is zeroed to avoid the circular convolution effect.

2. Add dual-channel frequency domain TRINICON method for multi-sound source activity detection

In the offline TRINICON method, when the signals are not continuously mixed, especially when there is no speech segment or only one sound source is active for a long time, the estimation of the power spectral density matrix Φ is biased, thus misleading the separation system Convergence direction, resulting in poor speech separation effect. This situation is more pronounced in the situation shown in Figure 1(b) where acausal filters are required.

The present invention executes off-line TRINICONO twice, uses the first convergence preliminary result to perform a rough multi-sound source activity estimation, and adjusts the second iterative process according to the estimated result, thereby obtaining an improved speech separation effect . For the case where a non-causal filter is required, the general algorithm uses a unit impulse response with a displacement of L/2 for initialization, but in the frequency domain algorithm, this initialization method has poor convergence characteristics. The method used in this embodiment is: The filter is initialized with a unit impulse response with a displacement point slightly larger than the maximum sound path difference between the two microphones.

First calculate the power input and output in each block:

and

Among them, k _u and k _l are the upper and lower limits of the frequency points included in the calculation (the subscript u is the abbreviation of upper bound, indicating the upper limit, and the subscript l is the abbreviation of lower bound, indicating the lower limit), which can be adjusted according to the spectral characteristics of the signal. To better estimate signal power and save computation. It should be noted that the effect of blind signal separation in the low frequency band is generally poor, so the low frequency band is generally not included in the calculation.

After calculating the input and output power of each channel, according to the power comparison before and after the first processing, a rough judgment can be made on the activity of the two sound sources in each block. The specific judgment steps are shown in Figure 2. Assume that the sound source s1 is suppressed in the output channel y1. First, if the power of the input signal is less than the set minimum power threshold E _min (the threshold can be determined by the stationary noise power estimated by methods such as MCRA), then it is judged that the two sound sources do not emit sound, and ε ₁ (m )=0, ε ₂ (m)=0. If the power is greater than the threshold, it is judged that there is at least one sound source emitting sound, and the next step is to judge: if the power ratio before and after processing of channel y1 is greater than the power ratio before and after processing of channel y2 multiplied by the coefficient λ, then it is considered that the sound source s1 is inactive and the sound Source s2 is active, and set ε ₁ (m)=0, ε ₂ (m)=1; if the power ratio before and after processing of channel y2 is greater than the power ratio before and after processing of channel y1 multiplied by the coefficient λ, it is considered that the sound source s2 is inactive, sound source s1 is active, and set ε ₁ (m)=1, ε ₂ (m)=0; if none of the above conditions are met, consider both sound sources active, and set ε ₁ (m)=1, ε ₂ (m)=1. The coefficient λ can be adjusted according to the specific sound source and usage scenario, and is generally a real number greater than 1.

After judging the active state of the sound source, the gradient of equation (9) is modified as

in

where N _sig is the block number of the entire signal, ε _p (m) is the correction weight, when the state of the p-th sound source in the m-th block is active, ε _p (m) is equal to 1, otherwise, ε _p (m ) is equal to 0.

The coefficients are updated using the modified gradient expression (12), that is, the filter of output channel y1 is updated using only the data of sound source s1 activity, and the filter of output channel y2 is updated using only the data of sound source s2 activity. In this way, an effective frame can be selected to update the two filters respectively, so as to improve the convergence performance and effectively improve the effect of speech separation. It should be noted that the above-mentioned multi-sound source activity detection is relatively accurate when the initial signal-to-interference ratio is below 10dB, but it is not completely accurate. In this embodiment, it is only necessary to roughly determine the time period of each sound source activity. That's it.

The flow of the whole algorithm after adjustment is shown in Figure 3. Using the input signals x1 and x2 received by the microphone, first use the general frequency domain TRINICON method to perform a speech separation, and then use the separation result of the first frequency domain TRINICON, according to the multi-sound source activity detection method described above, determine the two. The respective active time periods of the two sound sources; then update the two filter coefficients with the respective active time periods of the two sound sources, and perform the adjusted second frequency-domain TRINICON speech separation to obtain the final output signals y1 and y2 .

Figure 4 shows the spatial responses of the filters converged using the normal frequency-domain TRINICON method and the frequency-domain TRINICON method with multi-source activity detection added to a set of simulated data. The sound source positions are -70 degrees and -15 degrees, the room reverberation time is 0.2 seconds, and the microphone array unit spacing is 10 cm.

It can be seen that after adding multi-sound source activity detection, the valley points obtained by filter optimization are more accurate and deeper, and the suppression of sound sources is more obvious.

Table 1 presents the simulation results of multiple sets of data, and compares the improvement of the separation effect of the present invention in different scenarios. The simulated input signal is obtained by convolving the room impulse response generated by the sound of the Image Model tool with the pure speech signal. The indicators tested are signal to interference ratio (SIR) and signal to distortion ratio (SDR), and the test tool is BSSEVAL. The scores given in the table are the average of the scores of the two channels under 4 experiments. The higher the SIR, the better the interference sound is suppressed and the purer the target speech; the higher the SDR, the smaller the distortion of the output speech compared to the pure speech.

It can be seen that the improvement of the separation effect of the present invention is very obvious in most cases, the improvement of SIR can reach more than 5dB to 10dB, and at the same time, the sound quality can be kept from serious distortion.

Table 1 SIR and SDR scores of the common method and the processing results of the present invention under different scenarios

Example

1. Test conditions:

The size of the room is set to 6m×6m×4m, the reverberation time is 250ms, the spacing of the microphone array units is 10cm, and it is placed close to the center of the room. The sound source is 1.5m away from the center of the array, at -70 degrees and -15 degrees, respectively. The pure signals of the two sound sources come from the TIMIT database, and the sounding time of each of them accounts for 60% of the total duration of the recorded signal, of which about one-third of the total duration is the two sound sources sounding simultaneously. The signal-to-interference ratio (SIR) of the signals of the two sound sources is set to 0 dB, that is, the sound power is basically the same. In addition, white noise 30dB lower than the sound source signal is added to the whole signal. The sampling frequency of the signal is 16000Hz. Use the Image Model to generate the room impulse response according to the above conditions and use it to convolve with the pure speech to obtain the input signal.

2. Algorithm process:

1) Algorithm initialization: set w ₁₁ and w ₂₂ in formula (5) as the displacement unit impulse response with 10 displacement points (that is, w _11,10 =1, and the remaining values are 0), w ₁₂ , w ₂₁ is set to the zero vector; the step size μ is set to 0.02, the maximum number of iterations is set to 200, and the filter length L is set to 1024. Frame shift is set to half of L. The weight function is set to 1/N _sig .

2) Input the signals x ₁ and x ₂ received by the microphone, divide the input signal into blocks with a length of 4L and a frame shift of L/2 according to formula (6), and transform them into the frequency domain with short-time Fourier transform.

3) Calculate the output signal of each block by Y ^(k) = X ^(k) W ^(k) , and calculate the power spectral density matrix Φ _yy of the output signal according to formula (7).

4) According to equations (3) and (9), update the filter coefficient W according to the natural gradient descent method, use the inverse short-time Fourier transform to transform back to the time domain and set the part of w greater than L-1 to zero.

5) Using the updated filter coefficients W , repeat steps 3) and 4) until the maximum number of iterations is reached.

6) Obtain a preliminary filter coefficient convergence result W , and convolve the input signal with the overlap-preserving method to obtain preliminary output signals y ₁₀ and y ₂₀ of two channels.

7) Calculate the output and input power of each block of the two channels according to formula (10) and formula (11) respectively, and according to the process of Figure 2, judge the active state of each block signal, and obtain the revised weight function B ( m).

8) Re-initialize the weight function according to step 1), change formula (9) in step 4) to formula (12), and repeat steps 3) and 4), that is, perform an iteration again, this time In the iteration, only the active data of the sound source s1 is used to update the filter coefficient of the channel y1, and only the active data of the sound source s2 is used to update the filter coefficient of the channel y2 until the maximum number of iterations is reached.

9) The final filter coefficient convergence result W is obtained, and the input signal is convolved by the overlap preservation method to obtain the final output signals y ₁ and y ₂ of the two channels.

Claims (4)

1. A method for improving dual channel blind signal separation by multiple source activity detection, comprising the steps of:

(1) initializing filter matrix parameters in a frequency domain TRINICON algorithm;

(2) inputting sound source signals received by a microphone into a dual-channel system, partitioning input signals of each channel, and transforming the input signals of each block into a frequency domain by using short-time Fourier transform;

(3) byY ^(k)＝X ^(k) W ^(k)Calculating an output signal of each block, whereinY ^(k)、X ^(k)AndW ^(k)the filter coefficient matrixes of the frequency domain, the frequency domain output signal and the frequency domain input signal are respectively, and the superscript (k) is the serial number of the frequency point in the short-time discrete Fourier transform; then calculating the power spectral density matrix phi of each output signal_yy；

(4) Updating the filter coefficient according to a natural gradient descent method, transforming the filter coefficient matrix back to a time domain by using inverse short-time Fourier transform, and setting the part of the filter coefficient of the time domain, which is larger than the filter length minus 1, to zero;

(5) repeating the steps (3) and (4) by using the filter coefficient obtained in the step (4) until the maximum iteration number is reached;

(6) obtaining a preliminary filter coefficient convergence result, and performing convolution on the input signal by using an overlap preservation method to obtain a preliminary output signal of the first channel and a preliminary output signal of the second channel;

(7) firstly, the output and input power of each block in the two channels are respectively calculated according to the following two formulas:

wherein E_x(m) is input power, E_yp(m) is the power of the preliminary output signal, m is the number of signal blocks, k_uAnd k_lRespectively representing the upper limit and the lower limit of the frequency points which are counted and calculated, wherein subscript p is the serial number of an output channel, and subscripts 1 and 2 respectively represent a first channel and a second channel;

then judging the activity state of each block of signal and obtaining a modified weight function;

(8) re-initializing the weight function according to the step (1), wherein the modified gradient expression is as follows:

wherein

N_sigIs the number of blocks, epsilon, of the entire signal_p(m) is a correction weight, ε when the status of the p-th sound source in the m-th block is active_p(m) equals 1, otherwise,. epsilon_p(m) is equal to 0;

repeating the steps (3) and (4) according to the modified gradient expression, namely, repeating an iteration, wherein in the iteration, the filter coefficient of the first channel is updated only by the data of the target sound source activity to be suppressed by the first channel, and the filter coefficient of the second channel is updated only by the data of the target sound source activity to be suppressed by the second channel until the maximum iteration number is reached;

(9) and obtaining a final filter coefficient convergence result, and performing convolution on the input signal by using an overlap preservation method to obtain final output signals of the two channels.

2. The method of claim 1, wherein in step (1), the filter matrix parameters are initialized with a unity impulse response with a number of displacement points greater than the maximum path difference between adjacent microphones.

3. A method for improving the two-channel blind signal separation by multi-source activity detection as claimed in claim 1, wherein in step (2), the input signal is divided into blocks of length four times the filter length and frame-shifted by half the filter length.

4. The method for improving the dual-channel blind signal separation through multi-sound-source activity detection according to claim 1, wherein in the step (7), the specific process of judging the activity state of each signal is as follows:

after calculating the input power for each channel and the power of the preliminary output signal, the input power is compared to the power of the preliminary output signal for both channels, assuming that there are two sources and that the first source is suppressed in the first channel:

firstly, if the power of the input signal is less than the set minimum power threshold value E_minIf it is determined that all the sound sources are not sounding, ε is set₁(m)＝0，ε₂(m) ═ 0; if the power of the input signal is greater than the threshold value E_minIf so, judging that at least one sound source generates sound, and entering the next judgment:

if the power ratio of the first channel input power and the preliminary output signal is greater than the power ratio of the second channel input power and the preliminary output signal multiplied by the factor lambda, then the first sound source is considered inactive, the second sound source is considered active, and epsilon is set₁(m)＝0，ε₂(m) 1; if the power ratio of the second channel input power to the preliminary output signal is greater than the power ratio of the first channel input power to the preliminary output signal multiplied by the factor lambda, the second sound source is considered inactive, the first sound source is active, and epsilon is set₁(m)＝1，ε₂(m) ═ 0; if none of the above conditions are met, both sound sources are considered to be active and ε is set₁(m)＝1，ε₂(m)＝1。

Images