CN112530451A - Speech enhancement method based on denoising autoencoder - Google Patents

Abstract

A speech enhancement method based on denoising auto-encoder of the present invention is characterized in that the content includes: constructing a de-noising auto-encoder training model, obtaining time-domain difference values from a multi-microphone array, and reconstructing an original sound prediction model for de-noising Noise processing can effectively reduce the interference of noise on speech signals and significantly improve the signal-to-noise ratio of speech signals.

Description

Speech enhancement method based on denoising autoencoder

technical field

The invention belongs to the technical field of speech signal processing, and relates to a speech enhancement method based on a denoising autoencoder.

Background technique

Speech noise reduction is an important front-end for speech processing systems. Background noise and human-voice interference can reduce the quality and intelligibility of speech signals and cause performance degradation in real-world applications, including voice communications, hearing aids, and speech recognition. A key goal of speech noise reduction is to improve the quality and intelligibility in the presence of interfering noise.

In speech noise reduction algorithms, the most commonly used method is spectral subtraction. Spectral subtraction has the characteristics of simple algorithm and small computational complexity. The downside of this algorithm is that it produces "musical noise" that sounds like music after processing. The speech noise reduction algorithm based on the adaptive filter method can use the filter parameters and filtering results of the previous frame to automatically adjust the filter parameters of the current frame, which requires less prior knowledge of clean speech signals and noise. Therefore, it can adapt to the unknown random changes and statistics of clean speech signals and noise, so the speech after noise reduction has obvious improvement in signal-to-noise ratio and hearing sense. However, such algorithms often have slow convergence speed and are not suitable for non-stationary noise problems. The speech noise reduction algorithm based on Minimum Mean Square Error Estimation (MMSE) can effectively suppress the residual "music noise". However, in the case of low signal-to-noise ratio, the recognition of speech frames and non-speech frames is extremely error-prone, resulting in severe distortion of speech after noise reduction. The speech noise reduction algorithm based on subspace divides the whole space into pure noise subspace and pure speech subspace through space decomposition. By designing an estimator that not only guarantees the residual signal spectrum, but also considers minimizing the speech distortion, the noise subspace is removed and the eigenvalues of the speech signal are estimated to achieve speech noise reduction. One of the most commonly used subspace speech noise reduction based on optimal constraint estimator, but this speech noise reduction algorithm is very complex and difficult to implement on embedded platforms. The wavelet transform method is a new transform analysis method, which can perform local analysis of frequency in time or space. The signal is gradually refined by scaling and translation operations, which has the characteristics of multi-resolution analysis and can adapt to the requirements of signal analysis. It has been widely used in the fields of audio and image processing. According to the wavelet transform can effectively remove the correlation characteristics of the data, the clean speech signal energy is concentrated in the larger wavelet coefficients in the wavelet domain, and the noise energy is concentrated in the smaller wavelet coefficients. It is essentially a wavelet domain filtering algorithm, and choosing an appropriate threshold is the key to the performance of the system. However, the threshold value is difficult to obtain and the algorithm complexity is getting higher and higher, and it is difficult to be used for real-time communication. Deep neural networks (DNNs) are becoming more and more popular for speech noise reduction work. The speech noise reduction algorithm based on deep neural network is to form a deep neural network by stacking auto-encoders. Although the network has better noise reduction effect than the traditional single-channel speech algorithm, it has the problems of difficulty in network training and poor performance under the condition of low signal-to-noise ratio.

SUMMARY OF THE INVENTION

The purpose of the present invention is to propose a speech enhancement method based on denoising autoencoder to realize the enhancement of speech signal in order to reduce the interference of noise on speech signal and improve the signal-to-noise ratio of speech signal.

The object of the present invention is achieved by the following technical solutions: a speech enhancement method based on denoising autoencoder, characterized in that the content it includes includes: constructing a denoising autoencoder training model, when multiple microphone arrays acquire Domain difference, reconstruct the original sound prediction model for denoising,

1) Build a denoising autoencoder training model

The denoising autoencoder training model is designed as a three-layer network model. The first layer is the input layer, the middle layer is the hidden layer, the number of design nodes is 1024, and the third layer is the output layer. The output layer is compared with the original lossless data. Alignment, minimize the loss value:

是样本x经过损坏过程

后得到的损坏样本，通常分布p_decoder是因子的分布，平局参数由前馈网络给出，这里对负对数释然

进行基于梯度下降法的近似最小化，

即是样本

的概率分布，这样构成了确定的自编码器，也就是一个前馈的网络，并且能够使用与其他前馈网络完全相同的方式进行训练，因此整个自动编码器就可类比为下一个期望的梯度下降：In the formula,

is the sample x that has undergone the damage process

The damaged samples obtained later, usually the distribution p _decoder is the distribution of factors, and the draw parameters are given by the feedforward network, where the negative logarithm is relieved

perform an approximate minimization based on gradient descent,

the sample

The probability distribution of , which constitutes a deterministic auto-encoder, which is a feed-forward network, and can be trained in exactly the same way as other feed-forward networks, so the entire auto-encoder can be analogized to the next expected gradient decline:

是训练数据的分布，

表示对

分布的期望值，

表示对

样本

在全量x上的下一个期望值；in,

is the distribution of training data,

express right

the expected value of the distribution,

express right

sample

the next expected value on the full x;

2) Multi-microphone array to obtain time domain difference

The advantage of the voice enhancement method of the microphone array is that it considers the position information of the sound source and can realize spatial filtering, so it has an excellent suppression effect on directional noise. Therefore, the technology of the microphone array is applied to suppress the interfering voice. The realization is to reserve the speech signal in the desired direction;

First of all, due to the different positions of different microphones, there must be time deviations in the received voice signals. Therefore, tapped delay-lines (TDLs) are used to realize beamforming of broadband voice signals, and fixed beamforming of TDLs structure is used. The algorithm generates components of different frequencies through multi-tap delay, and then constrains the input signal of each microphone through the filter coefficient description, so that the signal in the desired direction is retained, and a null is formed in the undesired direction, so as to realize the fixed For beamforming in the direction of the sound source, the fixed beamforming algorithm of the TDLs structure can suppress the signal in the direction of the fixed noise source, and can effectively suppress both coherent and incoherent noise. Its expression is Equation (3):

F=WD (3)

In the formula, the matrix D is the direction matrix, which is used to align the speech signals of different angles in the frequency domain, W is the speech signals of different incident angles, ω ₀ , ..., ω _J-1 , respectively represent different frequency components, the matrix F is the target response matrix. Similarly, each component corresponds to the target response of signals with different incident angles. By setting the target response matrix F, it can be determined which directions of speech signals are reserved by the fixed beamforming structure, and which directions are reserved. To suppress the speech signal, the matrix W is the weight coefficient matrix, which is also the part that needs to be designed in the TDLs structure. By solving the formula (3), the obtained matrix coefficient solution ω _i,j is the final designed filter coefficient;

Then, the output of the signal is used to adaptively adjust the weight coefficient ω _i,j in the similar TDLs structure to achieve a certain robustness to changes in the acoustic environment. In the adaptive beamforming algorithm, the LCMV structure is used to adjust , the LCMV structure is adjusted on the basis of formula (3), and adjusted to formula (4):

Among them, R _yy is the expectation of the autocorrelation matrix of the input signal Y, which is estimated by R _yy ≈ YY ^H , and argmin _W W ^H R _yy W represents the adaptive adjustment of the weight coefficient W by minimizing the output power, so as to make the interference target The signal in the direction is suppressed, and equations (3) and (4) are solved to obtain the value of the coefficient matrix W:

According to the value of the above-mentioned solution coefficient matrix W, the difference value in the time domain is calculated;

3) Reconstruct the original sound prediction model for denoising processing

After calculating the difference in the time domain, the obtained speech signal is a distorted speech signal, because the structure of the multi-microphone array algorithm is used alone, there will be a situation where the same frequency speech is reduced and canceled. At the same time, for speech signals in different domains, there are The wind noise is not completely eliminated, which leads to the problem of "music noise". The model processed here does not have good robustness, so it is necessary to re-predict the distorted speech signal, and the distorted speech is introduced into the first layer as the input layer. Before the autoencoder model of the first step, a further step of filtering and denoising is required:

是估计的先验信噪比(a prior SNR)，所以整个求解的过程都是围绕如何求解这个先验信噪比进行的，而在这之前，先要估计后验信噪比和语音存在概率，后验信噪比的定义如下：

is the estimated prior signal-to-noise ratio (a prior SNR), so the entire solution process revolves around how to solve this prior signal-to-noise ratio, and before that, the posterior signal-to-noise ratio and the probability of speech existence must be estimated , the posterior signal-to-noise ratio is defined as follows:

是噪声的功率谱，是通过Cohen提出的OMLSA方法求得的，对比γ(t,d)和预先设定的阈值Tr，如果大于这个阈值，则语音的存在的索引I(d)设为1，否则为0，其实这有点类似理想二值掩蔽的概念，即如果是语音主导的就设定为1，否则就是设定为0，那么语音存在概率就能够通过以下方式进行估计：

is the power spectrum of the noise, which is obtained by the OMLSA method proposed by Cohen. Compare γ(t, d) with the preset threshold Tr. If it is greater than this threshold, the index I(d) of the existence of speech is set to 1 , otherwise it is 0. In fact, this is somewhat similar to the concept of ideal binary masking, that is, if it is voice-dominant, it is set to 1, otherwise it is set to 0, then the probability of voice existence can be estimated in the following ways:

p(t,d)=0.95p(t-1,d)+0.05I(d) (8)

It can be seen that the speech existence probability is the iterative average result of the speech existence probability of the previous moment and the speech existence index of the current frequency band, and the final prior signal-to-noise ratio can be estimated in the following way:

The prior signal-to-noise ratio consists of three parts. The first part is the prior signal-to-noise ratio at the previous moment. The second part is the prior signal-to-noise ratio calculated from the speech estimated by DNN and the noise spectrum estimated by the OMLSA method. The last part is to use the maximum likelihood estimation of the prior signal-to-noise ratio by the posterior signal-to-noise ratio, and then re-input the auto-encoder model of the first step after obtaining the result, and the result is the final noise-reduced speech.

A speech enhancement method based on denoising auto-encoder of the present invention includes the following steps: constructing a de-noising auto-encoder training model, acquiring time-domain difference values from a multi-microphone array, reconstructing an original sound prediction model for de-noising processing, etc. The steps can effectively reduce the interference of noise on the speech signal, improve the signal-to-noise ratio of the speech signal, and have the advantages of scientific and reasonable, simple structure, good denoising effect, wide application range and the like.

Description of drawings

FIG. 1 is a flow chart of a speech enhancement method based on denoising autoencoder.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

Referring to Fig. 1, the speech enhancement method based on the denoising autoencoder of the present invention includes the following: constructing a denoising autoencoder training model, obtaining a time domain difference with a multi-microphone array, reconstructing the original sound prediction model for denoising deal with.

1) Build a denoising autoencoder training model

The denoising autoencoder training model is designed as a three-layer network model. The first layer is the input layer, the middle layer is the hidden layer, the number of design nodes is 1024, and the third layer is the output layer. The output layer is compared with the original lossless data. Alignment, minimize the loss value:

是样本x经过损坏过程

后得到的损坏样本，通常分布p_decoder是因子的分布，平局参数由前馈网络给出，这里对负对数释然

进行基于梯度下降法的近似最小化，

即是样本

的概率分布，这样构成了确定的自编码器，也就是一个前馈的网络，并且能够使用与其他前馈网络完全相同的方式进行训练，因此整个自动编码器就可类比为下一个期望的梯度下降：In the formula,

is the sample x that has undergone the damage process

The damaged samples obtained later, usually the distribution p _decoder is the distribution of factors, and the draw parameters are given by the feedforward network, where the negative logarithm is relieved

perform an approximate minimization based on gradient descent,

the sample

The probability distribution of , which constitutes a deterministic auto-encoder, which is a feed-forward network, and can be trained in exactly the same way as other feed-forward networks, so the entire auto-encoder can be analogized to the next expected gradient decline:

是训练数据的分布，

表示对

分布的期望值，

表示对

样本

在全量x上的下一个期望值。in,

is the distribution of training data,

express right

the expected value of the distribution,

express right

sample

The next expected value on full x.

2) Multi-microphone array to obtain time domain difference

The advantage of the voice enhancement method of the microphone array is that it considers the position information of the sound source and can realize spatial filtering, so it has an excellent suppression effect on directional noise. Therefore, the technology of the microphone array is applied to suppress the interfering voice. The realization is to reserve the speech signal in the desired direction;

First of all, due to the different positions of different microphones, there must be time deviations in the received voice signals. Therefore, tapped delay-lines (TDLs) are used to realize beamforming of broadband voice signals, and fixed beamforming of TDLs structure is used. The algorithm generates components of different frequencies through multi-tap delay, and then constrains the input signal of each microphone through the filter coefficient description, so that the signal in the desired direction is retained, and a null is formed in the undesired direction, so as to realize the fixed For beamforming in the direction of the sound source, the fixed beamforming algorithm of the TDLs structure can suppress the signal in the direction of the fixed noise source, and can effectively suppress both coherent and incoherent noise. Its expression is Equation (3):

F=WD (3)

In the formula, the matrix D is the direction matrix, which is used to align the speech signals of different angles in the frequency domain, W is the speech signals of different incident angles, ω ₀ , ..., ω _J-1 , respectively represent different frequency components, the matrix F is the target response matrix. Similarly, each component corresponds to the target response of signals with different incident angles. By setting the target response matrix F, it can be determined which directions of speech signals are reserved by the fixed beamforming structure, and which directions are reserved. To suppress the speech signal, the matrix W is the weight coefficient matrix, which is also the part that needs to be designed in the TDLs structure. By solving the formula (3), the obtained matrix coefficient solution ω _i,j is the final designed filter coefficient;

Then, the output of the signal is used to adaptively adjust the weight coefficient ω _i,j in the similar TDLs structure to achieve a certain robustness to changes in the acoustic environment. In the adaptive beamforming algorithm, the LCMV structure is used to adjust , the LCMV structure is adjusted on the basis of formula (3), and adjusted to formula (4):

Among them, R _yy is the expectation of the autocorrelation matrix of the input signal Y, which is estimated by R _yy ≈ YY ^H , and argmin _W W ^H R _yy W represents the adaptive adjustment of the weight coefficient W by minimizing the output power, so as to make the interference target The signal in the direction is suppressed, and equations (3) and (4) are solved to obtain the value of the coefficient matrix W:

According to the value of the above solution coefficient matrix W, the difference value in the time domain is calculated.

3) Reconstruct the original sound prediction model for denoising processing

After calculating the difference in the time domain, the obtained speech signal is a distorted speech signal, because the structure of the multi-microphone array algorithm is used alone, there will be a situation where the same frequency speech is reduced and canceled. At the same time, for speech signals in different domains, there are The wind noise is not completely eliminated, which leads to the problem of "music noise". The model processed here does not have good robustness. Therefore, it is necessary to re-predict the distorted speech signal, and the distorted speech is used as the input layer to transmit milk first. Before the autoencoder model of the first step, a further step of filtering and denoising is required:

是估计的先验信噪比(a prior SNR)，所以整个求解的过程都是围绕如何求解这个先验信噪比进行的，而在这之前，先要估计后验信噪比和(aposteriorSNR)和语音存在概率，后验信噪比的定义如下：here

is the estimated prior signal-to-noise ratio (a prior SNR), so the entire solution process revolves around how to solve this prior signal-to-noise ratio, and before this, the posterior signal-to-noise ratio sum (aposteriorSNR) must be estimated and the probability of speech existence, the posterior signal-to-noise ratio is defined as follows:

是噪声的功率谱，是通过Cohen提出的OMLSA方法求得的(Cohen,2003)，对比γ(t,d)和预先设定的阈值Tr，如果大于这个阈值，则语音的存在的索引I(d)设为1，否则为0，其实这有点类似理想二值掩蔽的概念，即如果是语音主导的就设定为1，否则就是设定为0，那么语音存在概率就能够通过以下方式进行估计：here

is the power spectrum of the noise, which is obtained by the OMLSA method proposed by Cohen (Cohen, 2003). Compare γ(t, d) with the preset threshold Tr. If it is greater than this threshold, the index I ( d) is set to 1, otherwise it is 0. In fact, this is a bit similar to the concept of ideal binary masking, that is, if it is voice-dominant, it is set to 1, otherwise it is set to 0, then the probability of voice existence can be carried out in the following ways. estimate:

p(t,d)=0.95p(t-1,d)+0.05I(d) (8)

It can be seen that the speech existence probability is the iterative average result of the speech existence probability of the previous moment and the speech existence index of the current frequency band, and the final prior signal-to-noise ratio can be estimated in the following way:

The prior signal-to-noise ratio here consists of three parts, the first is the prior signal-to-noise ratio at the previous moment, and the second is the prior signal-to-noise calculated from the speech estimated by DNN and the noise spectrum estimated by the OMLSA method The last part is the maximum likelihood estimation of the prior signal-to-noise ratio using the posterior signal-to-noise ratio, and then re-input the auto-encoder model of the first step after obtaining the result, and the result is the final noise reduction speech.

The software program of the present invention is compiled according to automation, network and computer processing technology, and is a technology familiar to those skilled in the art.

The embodiments of the present invention are only used to further illustrate the present invention, are not exhaustive, and do not constitute a limitation on the protection scope of the claims. Those skilled in the art can think of other Substantially equivalent substitutions are all within the protection scope of the present invention.

Claims (1)

1. a speech enhancement method based on denoising self-encoder, it is characterized in that, the content it includes is: build denoising self-encoder training model, multi-microphone array obtains time domain difference, reconstructs the original sound prediction model to remove noise processing, 1) Build a denoising autoencoder training model The denoising autoencoder training model is designed as a three-layer network model. The first layer is the input layer, the middle layer is the hidden layer, the number of design nodes is 1024, and the third layer is the output layer. The output layer is compared with the original lossless data. Alignment, minimize the loss value:

In the formula,

is the sample x that has undergone the damage process

The damaged samples obtained later, usually the distribution p _decoder is the distribution of factors, and the draw parameters are given by the feedforward network, where the negative logarithm is relieved

perform an approximate minimization based on gradient descent,

the sample

The probability distribution of , which constitutes a deterministic auto-encoder, which is a feed-forward network, and can be trained in exactly the same way as other feed-forward networks, so the entire auto-encoder can be analogized to the next expected gradient decline:

in,

is the distribution of training data,

express right

the expected value of the distribution,

express right

sample

the next expected value on the full x; 2) Multi-microphone array to obtain time domain difference The advantage of the voice enhancement method of the microphone array is that it considers the position information of the sound source and can realize spatial filtering, so it has an excellent suppression effect on directional noise. Therefore, the technology of the microphone array is applied to suppress the interfering voice. The realization is to reserve the speech signal in the desired direction; First of all, due to the different positions of different microphones, there must be time deviations in the received voice signals. Therefore, tapped delay-lines (TDLs) are used to realize beamforming of broadband voice signals, and fixed beamforming of TDLs structure is used. The algorithm generates components of different frequencies through multi-tap delay, and then constrains the input signal of each microphone through the filter coefficient description, so that the signal in the desired direction is retained, and a null is formed in the undesired direction, so as to realize the fixed For beamforming in the direction of the sound source, the fixed beamforming algorithm of the TDLs structure can suppress the signal in the direction of the fixed noise source, and can effectively suppress both coherent and incoherent noise. Its expression is Equation (3): F=WD (3) In the formula, the matrix D is the direction matrix, which is used to align the speech signals of different angles in the frequency domain, W is the speech signals of different incident angles, ω ₀ , ..., ω _J-1 , respectively represent different frequency components, the matrix F is the target response matrix. Similarly, each component corresponds to the target response of signals with different incident angles. By setting the target response matrix F, it can be determined which directions of speech signals are reserved by the fixed beamforming structure, and which directions are reserved. The speech signal is suppressed, and the matrix W is the weight coefficient matrix, which is also the part of the TDLs structure that needs to be designed. By solving the formula (3), the obtained matrix coefficient solution ω _{i, j} is the final designed filter coefficient; Then, the output of the signal is used to adaptively adjust the weight coefficient ω _i,j in the structure similar to TDLs, so as to achieve a certain robustness to the changes of the acoustic environment. In the adaptive beamforming algorithm, the LCMV structure is used to adjust , the LCMV structure is adjusted on the basis of formula (3), and adjusted to formula (4):

Among them, R _yy is the expectation of the autocorrelation matrix of the input signal Y, which is estimated by R _yy ≈ YY ^H , and argmin _w W ^H R _yy W represents the adaptive adjustment of the weight coefficient W by minimizing the output power, so as to make the interference target The signal in the direction is suppressed, and equations (3) and (4) are solved to obtain the value of the coefficient matrix W:

According to the value of the above-mentioned solution coefficient matrix W, the difference value in the time domain is calculated; 3) Reconstruct the original sound prediction model for denoising processing After calculating the difference in the time domain, the obtained speech signal is a distorted speech signal, because the structure of the multi-microphone array algorithm is used alone, there will be a situation where the same frequency speech is reduced and canceled. At the same time, for speech signals in different domains, there are The wind noise is not completely eliminated, which leads to the problem of "music noise". The model processed here does not have good robustness, so it is necessary to re-predict the distorted speech signal, and the distorted speech is introduced into the first layer as the input layer. Before the autoencoder model of the first step, a further step of filtering and denoising is required:

is the estimated prior signal-to-noise ratio (a prior SNR), so the entire solution process revolves around how to solve this prior signal-to-noise ratio, and before that, the posterior signal-to-noise ratio and the probability of speech existence must be estimated , the posterior signal-to-noise ratio is defined as follows:

is the power spectrum of the noise, which is obtained by the OMLSA method proposed by Cohen. Compare γ(t, d) with the preset threshold Tr. If it is greater than this threshold, the index I(d) of the existence of speech is set to 1 , otherwise it is 0. In fact, this is somewhat similar to the concept of ideal binary masking, that is, if it is voice-dominant, it is set to 1, otherwise it is set to 0, then the probability of voice existence can be estimated in the following ways: p(t,d)=0.95p(t-1,d)+0.05I(d) (8) It can be seen that the speech existence probability is the iterative average result of the speech existence probability of the previous moment and the speech existence index of the current frequency band, and the final prior signal-to-noise ratio can be estimated in the following way:

The prior signal-to-noise ratio consists of three parts. The first part is the prior signal-to-noise ratio at the previous moment. The second part is the prior signal-to-noise ratio calculated from the speech estimated by DNN and the noise spectrum estimated by the OMLSA method. The last part is to use the maximum likelihood estimation of the prior signal-to-noise ratio by the posterior signal-to-noise ratio, and then re-input the auto-encoder model of the first step after obtaining the result, and the result is the final noise-reduced speech.

Images