CN118609587A - Signal noise reduction method, device, equipment and readable storage medium - Google Patents

Abstract

The present application discloses a signal denoising method, device, equipment and readable storage medium. The present application relates to the field of signal processing technology, including: obtaining an original signal to be denoised and obtaining a noise signal; inputting the original signal and the noise signal into a pre-trained signal denoising model, and outputting a denoising processing result, wherein the trained signal denoising model is obtained by training with first noise speech data and second noise speech data as model input data, and with an ideal processing result as a model training label, and the signal-to-noise ratio of the first noise speech data is higher than the signal-to-noise ratio of the second noise speech data. The present application realizes effective denoising of non-steady-state noise.

Description

Signal noise reduction method, device, equipment and readable storage medium

Technical Field

The present application relates to the field of signal processing technology, and in particular to a signal noise reduction method, device, equipment and readable storage medium.

Background Art

Speech is always inevitably interfered by external environmental noise, including noise introduced by the transmission medium, electrical noise inside the communication equipment, and even interference from other speakers. These interferences make the speech signal received by the microphone not a pure original speech signal, but a noisy speech signal contaminated by noise, causing the performance of many speech processing systems to deteriorate sharply. Therefore, in order to obtain the purest original speech signal from the noisy speech signal, speech noise reduction is required.

Compared with a single microphone, a microphone array adds a spatial domain (i.e., air domain) on the basis of the time-frequency domain, making full use of the air, time, and frequency domain information of the speech signal, and at the same time has the characteristics of high spatial resolution, high signal gain, and strong anti-interference ability. It can make up for the shortcomings of a single microphone in noise processing, speech extraction and separation, etc.

In the related art, when there are multiple sound pickup microphones (i.e., microphone arrays), the usual practice is to first fuse the signals received by the microphone arrays (i.e., multi-channel signals), and then input the fused signals into a signal noise reduction model, such as a deep learning model, to perform noise reduction on the fused signals. However, this noise reduction processing method has the problem of poor processing effect on non-steady-state noise (such as knocking sounds, horn sounds, etc.) and obvious noise residual.

Summary of the invention

The main purpose of the present application is to provide a signal noise reduction method, device, equipment and readable storage medium, aiming to solve the technical problems that the current noise reduction processing method has poor noise reduction effect on non-steady-state noise and obvious noise residue.

To achieve the above object, the present application provides a signal noise reduction method, the signal noise reduction method comprising:

Acquire an original signal to be denoised and acquire a noise signal, wherein the original signal is a signal obtained by fusing a multi-channel signal received by a microphone array, and the noise signal is a signal obtained by estimating the noise of the multi-channel signal based on a preset noise estimation algorithm;

The original signal and the noise signal are input into a pre-trained signal denoising model, and a denoising processing result is output, wherein the trained signal denoising model is trained using first noisy speech data and second noisy speech data as model input data and an ideal processing result as a model training label, and the signal-to-noise ratio of the first noisy speech data is higher than the signal-to-noise ratio of the second noisy speech data.

In one embodiment, after the step of inputting the original signal and the noise signal into a pre-trained signal denoising model and outputting a denoising result, the method further comprises:

If the noise reduction processing result is a gain factor, noise reduction processing is performed on the original signal based on the gain factor to obtain a noise-reduced signal.

In one embodiment, before the steps of obtaining the original signal to be denoised and obtaining the noise signal, the method further includes:

Acquire a training data set, wherein the training data set includes clean speech data and noise data;

Mixing the clean speech data with the noise data to obtain first noise speech data and second noise speech data, wherein the signal-to-noise ratio of the first noise speech data is higher than the signal-to-noise ratio of the second noise speech data;

An ideal processing result is determined based on the clean speech data, and the first noisy speech data and the second noisy speech data are used as inputs. The ideal processing result is used as a training label to train a preset signal denoising model to be trained, so as to obtain the trained signal denoising model.

In one embodiment, before the step of determining an ideal processing result based on the clean speech data, the method further includes:

Performing Fourier transform on the first noisy speech data, the second noisy speech data and the clean speech data respectively to obtain the first noisy speech data in the frequency domain, the second noisy speech data in the frequency domain and the clean speech data in the frequency domain;

The step of determining an ideal processing result based on the clean speech data is performed based on the first noisy speech data in the frequency domain, the second noisy speech data in the frequency domain, and the clean speech data in the frequency domain.

In one embodiment, the step of determining an ideal processing result based on the clean speech data includes:

Taking the pure speech data in the frequency domain as the ideal processing result; or,

An ideal gain factor is calculated based on the first noisy speech data in the frequency domain and the clean speech data in the frequency domain, and the ideal gain factor is used as an ideal processing result.

In one embodiment, the step of calculating the ideal gain factor based on the first noisy speech data in the frequency domain and the clean speech data in the frequency domain comprises:

Acquire the signal amplitude of the first noisy speech data in the frequency domain, and acquire the signal amplitude of the clean speech data in the frequency domain;

Using the signal amplitude of the first noisy speech data in the frequency domain as the first signal amplitude, and using the signal amplitude of the clean speech data in the frequency domain as the second signal amplitude;

The ratio between the second signal amplitude and the first signal amplitude is taken as the ideal gain factor.

In one embodiment, the preset signal denoising model is a neural network model, comprising a convolutional neural network layer, a first gated recurrent unit, and a second gated recurrent unit connected in sequence.

In addition, to achieve the above-mentioned purpose, the present application also provides a signal noise reduction device, the signal noise reduction device comprising:

An acquisition module, used to acquire an original signal to be denoised, and to acquire a noise signal, wherein the original signal is a signal obtained by fusing a multi-channel signal received by a microphone array, and the noise signal is a signal obtained by performing noise estimation on the multi-channel signal based on a preset noise estimation algorithm;

A denoising module is used to input the original signal and the noise signal into a pre-trained signal denoising model, and output a denoising processing result, wherein the trained signal denoising model is trained using the first noisy speech data and the second noisy speech data as model input data and an ideal processing result as a model training label, and the signal-to-noise ratio of the first noisy speech data is higher than the signal-to-noise ratio of the second noisy speech data.

In addition, to achieve the above-mentioned purpose, the present application also provides a signal noise reduction device, which includes a microphone array and a processor, and the microphone array is electrically connected to the processor; the processor is used to execute the steps of the signal noise reduction method as described above.

In addition, to achieve the above-mentioned purpose, the present application also provides a readable storage medium, which is a computer-readable storage medium, and a program for implementing the signal noise reduction method is stored on the computer-readable storage medium. The program for implementing the signal noise reduction method is executed by a processor to implement the steps of the signal noise reduction method as described above.

The present application also provides a computer program product, including a computer program, which implements the steps of the signal noise reduction method as described above when executed by a processor.

In the present application, an original signal to be denoised is obtained, and a noise signal is obtained, wherein the original signal is a signal obtained by fusing a multi-channel signal received by a microphone array, and the noise signal is a signal obtained by performing noise estimation on the multi-channel signal based on a preset noise estimation algorithm; the original signal and the noise signal are input into a pre-trained signal denoising model, and a denoising processing result is output, wherein the trained signal denoising model is trained using first noisy speech data and second noisy speech data as model input data, and using an ideal processing result as a model training label, and the signal-to-noise ratio of the first noisy speech data is higher than the signal-to-noise ratio of the second noisy speech data. In this way, compared with the signal denoising method of directly inputting the fused signal into the signal denoising model for noise reduction, the embodiment of the present application obtains the original signal after multi-channel signal fusion processing and the noise signal obtained by noise estimation, and inputs the original signal and the noise signal into the pre-trained signal denoising model. Since the signal denoising model is trained with the first noise speech data and the second noise speech data as model input data, the original signal with a higher signal-to-noise ratio is represented by the first noise speech data, and the noise signal with a lower signal-to-noise ratio is represented by the second noise speech data, the trained signal denoising model can effectively predict the output based on the actual input original signal and noise signal, and the original signal is denoised based on the reference information provided by the input noise signal, such as the position of the noise signal, the amplitude of the noise signal, etc., so that the signal denoising model also has a better processing effect on non-steady-state noise.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

FIG1 is a schematic diagram of a flow chart of a first embodiment of a signal noise reduction method of the present application;

FIG2 is a schematic diagram of a noise reduction result of a traditional noise reduction method according to an embodiment of the present application;

FIG3 is a schematic diagram of a noise reduction result obtained by using the signal noise reduction method of the present application;

FIG4 is a schematic diagram of a simplified signal noise reduction process involved in an embodiment of a signal noise reduction method of the present application;

FIG5 is a schematic diagram of a brief process of training a noise reduction model according to an embodiment of the signal noise reduction method of the present application;

FIG6 is a schematic diagram of the device structure of the signal noise reduction device of the present application;

FIG. 7 is a schematic diagram of the device structure of the hardware operating environment involved in the signal noise reduction device in the embodiment of the present application.

The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

DETAILED DESCRIPTION

In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

The main solution of the present application is: obtaining an original signal to be denoised and obtaining a noise signal, wherein the original signal is a signal obtained by fusion processing of a multi-channel signal received by a microphone array, and the noise signal is a signal obtained by noise estimation of the multi-channel signal based on a preset noise estimation algorithm; inputting the original signal and the noise signal into a pre-trained signal denoising model, and outputting a denoising processing result, wherein the trained signal denoising model is trained using first noisy speech data and second noisy speech data as model input data, and using an ideal processing result as a model training label, and the signal-to-noise ratio of the first noisy speech data is higher than the signal-to-noise ratio of the second noisy speech data.

The present application obtains the original signal after multi-channel signal fusion processing and the noise signal obtained by noise estimation, and inputs the original signal and the noise signal into a pre-trained signal denoising model. Since the signal denoising model is trained with the first noise speech data and the second noise speech data as model input data, the original signal with a higher signal-to-noise ratio is represented by the first noise speech data, and the noise signal with a lower signal-to-noise ratio is represented by the second noise speech data, the trained signal denoising model can effectively predict the output based on the actual input original signal and noise signal, and the original signal is denoised based on the reference information provided by the noise signal, such as the position of the noise signal, the amplitude of the noise signal, etc., so that the signal denoising model also has a better processing effect on non-steady-state noise.

It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a signal noise reduction device that can achieve the above functions, such as headphones, wearable devices, etc. For example, headphones are used as the execution subject to describe the embodiments of this application.

Based on this, the present application proposes a signal noise reduction method of the first embodiment. Please refer to FIG. 1 . The signal noise reduction method includes steps S10 to S20:

Step S10, obtaining an original signal to be denoised, and obtaining a noise signal, wherein the original signal is a signal obtained by fusing a multi-channel signal received by a microphone array, and the noise signal is a signal obtained by performing noise estimation on the multi-channel signal based on a preset noise estimation algorithm;

The original signal can be obtained by the headset itself fusing the multi-channel signal received by the microphone array, or by other devices fusing the multi-channel signal received by the microphone array and sending the fused multi-channel signal to the headset. Similarly, the noise signal can also be obtained by the headset itself performing noise estimation on the multi-channel signal based on a preset noise estimation algorithm, or by other devices performing noise estimation on the multi-channel signal based on a preset noise estimation algorithm and sending the obtained noise signal to the headset. This embodiment does not impose specific restrictions on this.

It can be understood that, considering that the positions of the microphones in the microphone array are different from the sound source, the multi-channel signals received by the microphone array can be time-aligned, and corresponding processing can be performed based on the time-aligned multi-channel signals to obtain the original signal and the noise signal.

After the microphone array picks up the signal, the signal picked up by the microphone array is used as a multi-channel signal, and the multi-channel signal is fused. Specifically, a beamforming algorithm can be used to fuse the multi-channel signal. The beamforming algorithm can be an existing beamforming algorithm, such as a generalized sidelobe canceller algorithm (Generalized Sidelobe Canceller, GSC), a minimum variance distortionless response (Minimum Variance Distortionless Response, MVDR) algorithm, a linearly constrained minimum variance (Linearly Constrained Minimum Variance, LCMV) algorithm, etc.

The preset noise estimation algorithm may be an algorithm set in advance for estimating noise in a multi-channel signal, such as a mean square error algorithm, a spectrum estimation algorithm, etc. Considering that the present embodiment adopts a beamforming algorithm to fuse the multi-channel signal to obtain the original signal, based on this, in a preferred embodiment, the same beamforming algorithm as the fusion process may be used to estimate the noise of the multi-channel signal, such as when the GSC algorithm is used to fuse the multi-channel signal, that is, the signal output after the multi-channel signal passes through the blocking matrix of the GSC algorithm is used as the noise signal, so that the original signal and the noise signal can be obtained through one algorithm processing, reducing the number of times the multi-channel signal is processed, and the original signal and the noise signal are obtained based on the same algorithm processing, and the consistency of the data between the original signal and the noise signal can also be maintained.

Exemplarily, the embodiments of the present application are described by taking a signal obtained by fusing multi-channel signals using the GSC algorithm as the original signal and a signal output after passing through a blocking matrix of the GSC algorithm as a noise signal as an example.

The GSC algorithm mainly consists of three parts: Fixed Beamforming (FBF), Blocking Matrix (BM) and Adaptive Noise Canceller (ANC), among which the fixed beamforming is the upper branch, and the blocking matrix and adaptive noise canceller are the lower branch. It is assumed that the main beam is aligned with the direction of the sound source, and the others are reference beams. The upper branch allows the main beam to pass, while the reference beam cannot pass; the lower branch blocks the main beam and allows the reference beam to pass. Specifically, the blocking matrix in the lower branch is used to block the main beam, and the ANC is used to equalize the signal Z output by the blocking matrix to make it consistent with the residual noise in the output result of the upper branch. The multi-channel signal x is output to the GSC algorithm, and the upper branch signal _yc is obtained after the upper branch processing, and the blocking signal _yb is obtained after the lower branch processing. The upper branch signal _yc and the blocking signal _yb are then filtered. For example, the two signals are subjected to Wiener filtering, and the noise of the upper and lower branches is offset to obtain the processing result _yp . When the GSC algorithm is used to perform fusion processing on the multi-channel beam, the processing result _yp can be the original signal to be denoised, and the signal Z output by the blocking matrix can be the noise signal.

It should be noted that since the multi-channel signals collected by the microphone array are time domain signals, if the multi-channel signals in the time domain are fused or estimated in the process of noise estimation, they are not subjected to Fourier transform processing, but are directly fused or estimated based on the time domain signals. At this time, there may be a signal in the noise signal and the original signal that is a time domain signal. Then, the time domain signal can be Fourier transformed to obtain a frequency domain signal, and the noise signal and the original signal in the frequency domain are subsequently processed. Exemplarily, if the time domain signal is framed in the time domain, the length of each frame and the overlapping length can be set as needed, such as when the frame length is between 7.5ms and 30ms, and each frame of data is subjected to STFT (short-time Fourier transform) to obtain a frequency domain signal. It can be understood that each time domain signal is converted into a frequency domain signal using the same framing method to ensure consistency between signals.

Step S20, input the original signal and the noise signal into a pre-trained signal denoising model, and output a denoising processing result, wherein the trained signal denoising model is trained using the first noisy speech data and the second noisy speech data as model input data, and using the ideal processing result as a model training label, and the signal-to-noise ratio of the first noisy speech data is higher than the signal-to-noise ratio of the second noisy speech data.

It should be noted that the pre-trained signal denoising model can be specifically a signal denoising model trained based on the first noisy speech data and the second noisy speech data as inputs, and the ideal processing result as a training label, wherein the ideal processing result is used to indicate the expected processing result of the first noisy speech data, such as pure speech informationization, ideal gain factor, etc. The first noisy speech data is used to represent the original signal with a high signal-to-noise ratio, the second noisy speech data is used to represent the noise signal with a low signal-to-noise ratio, and the ideal processing result is used to represent the ideal output, so that the trained signal denoising model can effectively predict the output based on the actual input original signal and noise signal.

In a possible implementation manner, after the step of inputting the original signal and the noise signal into a pre-trained signal denoising model and outputting a denoising result, the method further includes:

Step S201: If the noise reduction processing result is a gain factor, noise reduction processing is performed on the original signal based on the gain factor to obtain a noise-reduced signal.

It should be noted that when the model is trained, if an ideal gain factor is used as a training label, the noise reduction processing result output by the signal noise reduction model is also the gain factor. At this time, the original signal can be subjected to noise reduction processing based on the gain factor output by the model. Specifically, the signal amplitude of the original signal can be adjusted using the gain factor. It can be understood that the processing method of signal noise reduction processing using the gain factor can adopt the existing noise reduction processing method of the signal based on the gain factor, which will not be described in detail in this embodiment.

Furthermore, when the model is trained, if a pure speech signal is used as a training label, the denoising result output by the signal denoising model is also a speech signal (that is, a denoised signal). Alternatively, after the original signal is denoised based on the gain factor output by the model to obtain a denoised signal, an inverse Fourier transform can be performed on the denoised signal to convert the frequency domain signal to the time domain for easy output.

In this embodiment, an original signal to be denoised is obtained, and a noise signal is obtained, wherein the original signal is a signal obtained by fusing a multi-channel signal received by a microphone array, and the noise signal is a signal obtained by performing noise estimation on the multi-channel signal based on a preset noise estimation algorithm; the original signal and the noise signal are input into a pre-trained signal denoising model, and a denoising processing result is output, wherein the trained signal denoising model is trained using first noisy speech data and second noisy speech data as model input data, and using an ideal processing result as a model training label, and the signal-to-noise ratio of the first noisy speech data is higher than the signal-to-noise ratio of the second noisy speech data. In this way, compared with the signal denoising method of directly inputting the fused signal into the signal denoising model for denoising, the present embodiment obtains the original signal after multi-channel signal fusion processing and the noise signal obtained by noise estimation, and inputs the original signal and the noise signal into the pre-trained signal denoising model. Since the signal denoising model is trained with the first noise speech data and the second noise speech data as model input data, the original signal with a higher signal-to-noise ratio is represented by the first noise speech data, and the noise signal with a lower signal-to-noise ratio is represented by the second noise speech data, the trained signal denoising model can effectively predict the output based on the actual input original signal and noise signal, and performs denoising on the original signal based on the reference information provided by the input noise signal, such as the position of the noise signal, the amplitude of the noise signal, etc., so that the signal denoising model also has a better processing effect on non-steady-state noise.

Exemplarily, referring to Figures 2 to 3, the interior of the boxes in Figures 2 to 3 is divided into non-steady-state noise; Figure 2 shows the noise reduction processing effect obtained by taking the original signal as input and adopting the traditional signal noise reduction method to process it; Figure 3 shows the noise reduction processing effect obtained by taking the original signal and the noise signal as input and adopting the signal noise reduction method of the embodiment of the present application to process it; by comparing Figures 2 and 3, it can be seen that the noise reduction processing method of the embodiment of the present application has a better processing effect on non-steady-state noise and less noise residue.

For example, to help understand the technical concept or technical principle of this embodiment, please refer to FIG. 4 , which provides a brief flow chart of signal noise reduction, as follows:

1. Get the multi-channel signals collected by microphone arrays mic1 and mic2.

2. The collected multi-channel signals are input into the GSC algorithm, and the noise signal is obtained through the blocking matrix output of the GSC algorithm. The output of the blocking matrix and the upper branch output of the GSC algorithm are adaptively filtered to obtain the original noisy signal.

3. The noise signal obtained in step 2 and the original noisy signal are input into a pre-trained signal denoising model, which is specifically a neural network model, and the gain factor IRM is output.

4. Perform noise reduction processing on the original signal based on the gain factor IRM to obtain a noise-reduced signal.

Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description and will not be described in detail later. On this basis, before the steps of obtaining the original signal to be denoised and obtaining the noise signal, the method further includes:

Step A10, obtaining a training data set, wherein the training data set includes clean speech data and noise data;

It should be noted that the clean speech data and the noise data can specifically be single-channel time domain signal data. The clean speech data is data that only contains valid speech signals and no noise signals, and the noise data is data that only contains noise signals and no valid speech signals.

Step A20, mixing the clean speech data and the noise data to obtain first noise speech data and second noise speech data, wherein the signal-to-noise ratio of the first noise speech data is higher than the signal-to-noise ratio of the second noise speech data;

It should be noted that, as one of the implementation modes, the clean voice data and the noisy voice data can be directly mixed with the preset first signal-to-noise ratio and the second signal-to-noise ratio to obtain the first noisy voice data and the second noisy voice data. As another implementation mode, the clean voice data can be played at a sound source position and the noisy data can be played at other positions to mix the clean voice data and collect signals through a microphone array to simulate a multi-channel signal, and the signals collected by the microphone array are input into the GSC algorithm, and the output of the GSC algorithm is used as the first noisy voice data, and the output of the blocking matrix in the GSC algorithm is used as the second noisy voice data. In this way, the first noisy voice data and the second noisy voice data are the outputs of the actual GSC algorithm, which can further improve the training effect of the model, and further improve the noise reduction processing effect of the model.

Step A30, determining an ideal processing result based on the clean speech data, taking the first noisy speech data and the second noisy speech data as input and the ideal processing result as output to train a preset signal denoising model, and obtaining a trained signal denoising model.

The ideal processing result is used to indicate the ideal output expected by the user after processing the first noisy speech data, such as pure speech data, ideal gain factor, etc. The preset signal denoising model is trained with the first noisy speech data and the second noisy speech data as input and the ideal processing result as output, so as to simulate a signal with a high signal-to-noise ratio through the first noisy speech data and simulate a signal with a low signal-to-noise ratio through the second noisy speech data, so that the signal denoising model obtained after training can effectively predict the output based on the two input signals with high signal-to-noise ratio and low signal-to-noise ratio.

Based on the first embodiment and/or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first and second embodiments can be referred to the above description, and will not be described in detail later. On this basis, before the step of determining the ideal processing result based on the clean voice data, the method further includes:

Step B10, performing Fourier transform on the first noisy speech data, the second noisy speech data and the clean speech data respectively to obtain the first noisy speech data in the frequency domain, the second noisy speech data in the frequency domain and the clean speech data in the frequency domain;

Step B20, performing the step of determining an ideal processing result based on the clean speech data based on the first noisy speech data in the frequency domain, the second noisy speech data in the frequency domain, and the clean speech data in the frequency domain.

The time domain signal is processed by Fourier transform to obtain the frequency domain signal, and the signal noise reduction model is trained based on the frequency domain signal. Exemplarily, the time domain signal can be framed in the time domain, and the length of each frame and the overlap length can be set as needed, such as the frame length is between 7.5ms and 30ms, and each frame of data is subjected to STFT (short-time Fourier transform) to obtain the frequency domain signal. It can be understood that each time domain signal is converted into a frequency domain signal using the same framing method to ensure consistency between signals.

In a possible implementation, the step of determining an ideal processing result based on the clean speech data includes:

Step C10, taking the pure speech data in the frequency domain as an ideal processing result; or,

Step C20, calculating an ideal gain factor based on the first noisy speech data in the frequency domain and the clean speech data in the frequency domain, and taking the ideal gain factor as an ideal processing result.

It can be understood that, considering that the first noisy speech data is used to simulate a signal to be de-noised with a high signal-to-noise ratio, ideally, after the signal to be de-noised is input into the model, the model is expected to output a signal with noise completely eliminated. Based on this, the ideal gain factor calculated based on the first noisy speech data and the clean speech data is the clean speech data with noise completely eliminated after the first noisy speech data is de-noised based on the ideal gain factor.

In a possible implementation, the step of calculating the ideal gain factor based on the first noisy speech data in the frequency domain and the clean speech data in the frequency domain includes:

Step D10, obtaining the signal amplitude of the first noisy speech data in the frequency domain, and obtaining the signal amplitude of the clean speech data in the frequency domain;

Specifically, the signal amplitude may be the cumulative signal amplitude or average signal amplitude of all signal frequency points in the signal of the preset signal frame number. It can be understood that after Fourier transform of N sampling points, the Fourier transform results of N signal points can be obtained, each sampling point corresponds to a signal frequency point after Fourier transform, and each signal frequency point corresponds to the value after Fourier transform (recorded as FT value). Based on this, the signal amplitude of a certain signal frequency point is specifically the modulus of the FT value corresponding to the signal frequency point.

Step D20, using the signal amplitude of the first noisy speech data in the frequency domain as the first signal amplitude, and using the signal amplitude of the clean speech data in the frequency domain as the second signal amplitude;

Step D30: taking the ratio between the second signal amplitude and the first signal amplitude as an ideal gain factor.

Specifically, the ratio between the second signal amplitude and the first signal amplitude is the second signal amplitude divided by the first signal amplitude, so that the signal amplitude obtained by multiplying the first signal amplitude by the gain factor is the signal amplitude of pure voice data, that is, the ideal signal amplitude.

In a possible implementation, the preset signal denoising model is a neural network model, comprising a convolutional neural network layer, a first gated recurrent unit, and a second gated recurrent unit connected in sequence.

Considering that the neural network has powerful nonlinear computing capabilities, this embodiment adopts a neural network model as a signal denoising model. Specifically, as shown in Figure 5, the signal denoising model includes a convolutional network layer (Convolutional Neural Network, convolutional neural network), a first gated recurrent unit (Gate Recurrent Unit, GRU) and a second gated recurrent unit connected in sequence.

For example, in order to help understand the technical concept or technical principle of signal noise reduction after combining this embodiment with the above-mentioned embodiment 2, a specific embodiment is listed. In this specific embodiment, the signal noise reduction process is:

1. Prepare data. Prepare a large amount of single-channel pure speech data and noise data, and mix them according to high and low signal-to-noise ratios. The mixed data is regarded as one group, and a total of three groups of data are mixed. They are 5dB and -10dB, 10dB and -5dB, and 15dB and 0dB. Among them, the low signal-to-noise ratio signal simulates the noise estimated by the blocking matrix, and the high signal-to-noise ratio signal simulates the signal obtained after fusion processing.

2. Model training. As shown in Figure 5, a neural network with two-channel input and one-channel output is designed as a noise reduction module based on the DNN neural network. For each pair of signals obtained in the above process: ① Calculate the input of the neural network. The specific steps are to perform Fourier transform (frame length is 256 points) on the two groups of high and low signal-to-noise ratio signals, and form a group of three consecutive frames. ② Calculate the output of the neural network. Perform Fourier transform (frame length is 256 points) on the high signal-to-noise ratio signal and the corresponding pure speech signal, divide the speech signal after FFT transformation by the corresponding high signal-to-noise ratio signal, and obtain the ideal gain value of each frame (denoted as IRM). ③ Perform training, first splice (stack) the results of the Fourier transform of the two groups of high and low signal-to-noise ratio signals, and obtain a set of 2×3×129 (select 129 asymmetric points out of 256 points) data as the input of the neural network. Take the IRM corresponding to the high signal-to-noise ratio signal of the middle frame as the output. Perform neural network training.

It should be noted that the above specific embodiments are only used to understand the present application and do not constitute a limitation on the signal noise reduction process of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

In addition, the embodiment of the present application further provides a signal noise reduction device. Referring to FIG. 6 , the signal noise reduction device includes:

An acquisition module 10 is used to acquire an original signal to be denoised and to acquire a noise signal, wherein the original signal is a signal obtained by fusing a multi-channel signal received by a microphone array, and the noise signal is a signal obtained by estimating the noise of the multi-channel signal based on a preset noise estimation algorithm;

The denoising module 20 is used to input the original signal and the noise signal into a pre-trained signal denoising model, and output a denoising processing result, wherein the trained signal denoising model is trained using the first noisy speech data and the second noisy speech data as model input data and an ideal processing result as a model training label, and the signal-to-noise ratio of the first noisy speech data is higher than the signal-to-noise ratio of the second noisy speech data.

In one embodiment, the noise reduction module 20 is further used for:

If the noise reduction processing result is a gain factor, noise reduction processing is performed on the original signal based on the gain factor to obtain a noise-reduced signal.

In one embodiment, the signal denoising device further includes a model training module, wherein the model training module is used to:

Acquire a training data set, wherein the training data set includes clean speech data and noise data;

Mixing the clean speech data with the noise data to obtain first noise speech data and second noise speech data, wherein the signal-to-noise ratio of the first noise speech data is higher than the signal-to-noise ratio of the second noise speech data;

An ideal processing result is determined based on the clean speech data, and the first noisy speech data and the second noisy speech data are used as inputs. The ideal processing result is used as a training label to train a preset signal denoising model to be trained, so as to obtain the trained signal denoising model.

The model training module is also used for:

Performing Fourier transform on the first noisy speech data, the second noisy speech data and the clean speech data respectively to obtain the first noisy speech data in the frequency domain, the second noisy speech data in the frequency domain and the clean speech data in the frequency domain;

The step of determining an ideal processing result based on the clean speech data is performed based on the first noisy speech data in the frequency domain, the second noisy speech data in the frequency domain, and the clean speech data in the frequency domain.

The model training module is also used for:

Taking the pure speech data in the frequency domain as the ideal processing result; or,

An ideal gain factor is calculated based on the first noisy speech data in the frequency domain and the clean speech data in the frequency domain, and the ideal gain factor is used as an ideal processing result.

The model training module is also used for:

Acquire the signal amplitude of the first noisy speech data in the frequency domain, and acquire the signal amplitude of the clean speech data in the frequency domain;

Using the signal amplitude of the first noisy speech data in the frequency domain as the first signal amplitude, and using the signal amplitude of the clean speech data in the frequency domain as the second signal amplitude;

The ratio between the second signal amplitude and the first signal amplitude is taken as the ideal gain factor.

In one embodiment, the preset signal denoising model is a neural network model, comprising a convolutional neural network layer, a first gated recurrent unit, and a second gated recurrent unit connected in sequence.

In addition, an embodiment of the present application also proposes a signal noise reduction device, which includes a memory, a processor, and a signal noise reduction program stored in the memory and executable on the processor. When the signal noise reduction program is executed by the processor, the steps of the signal noise reduction method as described above are implemented.

Referring to FIG. 7 below, a schematic diagram of the structure of a signal noise reduction device suitable for implementing an embodiment of the present application is shown. The signal noise reduction device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), etc., and fixed terminals such as digital TVs, desktop computers, etc. The signal noise reduction device shown in FIG. 7 is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

As shown in FIG7 , the signal noise reduction device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM) 1004. Various programs and data required for the operation of the signal noise reduction device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other via a bus 1005. An input/output (I/O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to the I/O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the signal noise reduction device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a signal noise reduction device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

The signal noise reduction device provided in the embodiment of the present application adopts the signal noise reduction method in the above embodiment to solve the technical problem of signal noise reduction. Compared with the prior art, the beneficial effects of the signal noise reduction device provided in the present application are the same as the beneficial effects of the signal noise reduction method provided in the above embodiment, and other technical features in the signal noise reduction device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

In addition, to achieve the above-mentioned purpose, an embodiment of the present application further provides a readable storage medium having computer-readable program instructions (ie, computer program) stored thereon, and the computer-readable program instructions are used to execute the signal noise reduction method in the above-mentioned embodiment.

The computer-readable storage medium provided in the embodiment of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

The computer-readable storage medium may be included in the signal noise reduction device; or may exist independently without being assembled into the signal noise reduction device.

The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the signal noise reduction device, the signal noise reduction device: obtains the original signal to be denoised and obtains the noise signal, wherein the original signal is a signal obtained by fusion processing of the multi-channel signal received by the microphone array, and the noise signal is a signal obtained by noise estimation of the multi-channel signal based on a preset noise estimation algorithm; inputs the original signal and the noise signal into a pre-trained signal noise reduction model, and outputs a noise reduction processing result, wherein the trained signal noise reduction model is trained using the first noisy speech data and the second noisy speech data as model input data, and using the ideal processing result as the model training label, and the signal-to-noise ratio of the first noisy speech data is higher than the signal-to-noise ratio of the second noisy speech data.

Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and/or flow chart, and the combination of the boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The modules involved in the embodiments of the present application may be implemented by software or hardware, wherein the name of the module does not, in some cases, constitute a limitation on the unit itself.

The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned signal noise reduction method, and can solve the technical problem of signal noise reduction. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the signal noise reduction method provided in the above-mentioned embodiment, and will not be repeated here.

In addition, an embodiment of the present application further proposes a computer program product, including a signal noise reduction program, which implements the steps of the signal noise reduction method described above when executed by a processor.

The specific implementation of the computer program product of the present application is basically the same as the above-mentioned signal noise reduction method embodiments, and will not be repeated here.

It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software sensor, which is stored in a storage medium (such as ROM/RAM, disk, CD) as described above, including a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims (10)

1. A signal noise reduction method, characterized in that the signal noise reduction method comprises the following steps: Acquire an original signal to be denoised, and acquire a noise signal, wherein the original signal is a signal obtained by fusing a multi-channel signal received by a microphone array, and the noise signal is a signal obtained by estimating the noise of the multi-channel signal based on a preset noise estimation algorithm; The original signal and the noise signal are input into a pre-trained signal denoising model, and a denoising processing result is output, wherein the trained signal denoising model is trained using first noisy speech data and second noisy speech data as model input data and an ideal processing result as a model training label, and the signal-to-noise ratio of the first noisy speech data is higher than the signal-to-noise ratio of the second noisy speech data. 2. The method according to claim 1, characterized in that after the step of inputting the original signal and the noise signal into a pre-trained signal denoising model and outputting a denoising result, the method further comprises: If the noise reduction processing result is a gain factor, noise reduction processing is performed on the original signal based on the gain factor to obtain a noise-reduced signal. 3. The method according to claim 1, characterized in that before the steps of obtaining the original signal to be denoised and obtaining the noise signal, the method further comprises: Acquire a training data set, wherein the training data set includes clean speech data and noise data; Mixing the clean speech data with the noise data to obtain first noise speech data and second noise speech data, wherein the signal-to-noise ratio of the first noise speech data is higher than the signal-to-noise ratio of the second noise speech data; An ideal processing result is determined based on the clean speech data, and the first noisy speech data and the second noisy speech data are used as inputs. The ideal processing result is used as a training label to train a preset signal denoising model to be trained, so as to obtain the trained signal denoising model. 4. The method according to claim 3, characterized in that before the step of determining an ideal processing result based on the clean speech data, the method further comprises: Performing Fourier transform on the first noisy speech data, the second noisy speech data and the clean speech data respectively to obtain the first noisy speech data in the frequency domain, the second noisy speech data in the frequency domain and the clean speech data in the frequency domain; The step of determining an ideal processing result based on the clean speech data is performed based on the first noisy speech data in the frequency domain, the second noisy speech data in the frequency domain, and the clean speech data in the frequency domain. 5. The method according to claim 4, wherein the step of determining an ideal processing result based on the clean speech data comprises: Taking the pure speech data in the frequency domain as the ideal processing result; or, An ideal gain factor is calculated based on the first noisy speech data in the frequency domain and the clean speech data in the frequency domain, and the ideal gain factor is used as an ideal processing result. 6. The method according to claim 5, wherein the step of calculating the ideal gain factor based on the first noisy speech data in the frequency domain and the clean speech data in the frequency domain comprises: Acquire the signal amplitude of the first noisy speech data in the frequency domain, and acquire the signal amplitude of the clean speech data in the frequency domain; Using the signal amplitude of the first noisy speech data in the frequency domain as the first signal amplitude, and using the signal amplitude of the clean speech data in the frequency domain as the second signal amplitude; The ratio between the second signal amplitude and the first signal amplitude is taken as the ideal gain factor. 7. The method according to any one of claims 3 to 6 is characterized in that the preset signal denoising model is a neural network model, comprising a convolutional neural network layer, a first gated recurrent unit and a second gated recurrent unit connected in sequence. 8. A signal noise reduction device, characterized in that the signal noise reduction device comprises: An acquisition module, used to acquire an original signal to be denoised, and to acquire a noise signal, wherein the original signal is a signal obtained by fusing a multi-channel signal received by a microphone array, and the noise signal is a signal obtained by performing noise estimation on the multi-channel signal based on a preset noise estimation algorithm; A denoising module is used to input the original signal and the noise signal into a pre-trained signal denoising model, and output a denoising processing result, wherein the trained signal denoising model is trained using the first noisy speech data and the second noisy speech data as model input data and an ideal processing result as a model training label, and the signal-to-noise ratio of the first noisy speech data is higher than the signal-to-noise ratio of the second noisy speech data. 9. A signal noise reduction device, characterized in that the signal noise reduction device comprises: a memory, a processor, and a signal noise reduction program stored in the memory and executable on the processor, wherein when the signal noise reduction program is executed by the processor, the steps of the signal noise reduction method according to any one of claims 1 to 7 are implemented. 10. A readable storage medium, characterized in that the readable storage medium is a computer-readable storage medium, and a program for implementing a signal noise reduction method is stored on the computer-readable storage medium, and the program for implementing the signal noise reduction method is executed by a processor to implement the steps of the signal noise reduction method as described in any one of claims 1 to 7.