CN114495962A - An audio noise reduction method, apparatus, system and computer-readable storage medium - Google Patents

Abstract

The application discloses an audio noise reduction method, an audio noise reduction device, an audio noise reduction system and a computer readable storage medium, wherein the audio noise reduction method comprises the following steps: calculating a power spectrum of data to be denoised; initializing the noise estimation parameters based on the power spectrum to obtain a noise spectrum of the noise data; carrying out minimum tracking on the initialized noise estimation parameters in a first time period to obtain a first array; calculating a posterior signal-to-noise ratio and a prior signal-to-noise ratio of the data to be denoised based on the first array; carrying out minimum tracking on the initialized noise estimation parameters in a second time period to obtain a second array; calculating a gain estimation value of the noise-free data based on the second array, the silence probability estimation value, the sound probability estimation value and the noise power spectrum estimation value; and performing noise reduction processing on the data to be subjected to noise reduction based on the gain estimation value, the noise spectrum and the noise power spectrum estimation value to obtain noise-free data. By the aid of the mode, the noise reduction effect on the audio data can be improved.

Description

An audio noise reduction method, apparatus, system and computer-readable storage medium

technical field

The present application relates to the technical field of audio processing, and in particular, to an audio noise reduction method, apparatus, system, and computer-readable storage medium.

Background technique

In the process of voice communication, the audio data will be subject to various interferences to varying degrees, which affects the quality and naturalness of the audio. Therefore, it is necessary to extract the original audio data as pure as possible from the audio data with noise. The noise frequency data is subjected to audio noise reduction processing, thereby achieving an anti-noise effect, while the existing audio data processing process cannot achieve a better audio noise reduction effect.

SUMMARY OF THE INVENTION

The present application provides an audio noise reduction method, device, system and computer-readable storage medium, which can improve the noise reduction effect on audio data.

In order to solve the above technical problems, the technical solution adopted in the present application is to provide an audio noise reduction method, the audio noise reduction method includes: acquiring data to be denoised, and calculating a power spectrum of the data to be denoised, the data to be denoised comprising: Noise data and noise-free data; initialize the noise estimation parameters based on the power spectrum to obtain the noise spectrum of the noise data; perform minimum value tracking on the initialized noise estimation parameters in the first time period to obtain the first array; based on the first array , calculate the posterior signal-to-noise ratio and the prior signal-to-noise ratio of the data to be denoised; based on the confidence of the posterior signal-to-noise ratio, calculate the estimated value of the probability of no sound, the estimated value of the probability of sound and the estimated value of the noise power spectrum; Perform minimum value tracking on the initialized noise estimation parameters in the second time period to obtain a second array; based on the second array, the soundless probability estimation value, the sounding probability estimation value and the noise power spectrum estimation value, calculate the noiseless data Based on the estimated gain value, the noise spectrum and the estimated value of the noise power spectrum, noise reduction processing is performed on the data to be denoised to obtain noise-free data.

In order to solve the above-mentioned technical problem, another technical solution adopted in the present application is to provide an audio noise reduction device, the audio noise reduction device includes a memory and a processor connected to each other, wherein the memory is used for storing a computer program, and the computer program is stored in the computer program. When executed by the processor, it is used to implement the audio noise reduction method in the above technical solution.

In order to solve the above-mentioned technical problems, another technical solution adopted in the present application is to provide an audio noise reduction device, which is used to perform a receiving task and a noise reduction task at the same time, the audio noise reduction device includes a scheduling circuit and a noise reduction circuit, and the scheduling The circuit is used to receive the data to be noise-reduced corresponding to the receiving task, and perform branch processing on the data to be noise-reduced to obtain multi-channel sub-audio data; the noise-reduction circuit is connected to the scheduling circuit for all sub-audio corresponding to the noise reduction task. The data is subjected to parallel noise reduction processing to obtain noise-reduced audio data; wherein, the noise reduction circuit is used to implement the audio noise reduction method in the above technical solution.

In order to solve the above technical problem, another technical solution adopted in the present application is to provide an audio noise reduction system, the audio noise reduction system includes an audio collection device and an audio noise reduction device, and the audio collection device is used to collect the sound in the target scene , to obtain the data to be denoised; the audio denoising device is connected to the audio collection device, which is used to perform denoising processing on the denoising data to obtain the denoised audio data; wherein, the audio denoising device is the audio in the above technical solution Noise reduction device.

In order to solve the above-mentioned technical problem, another technical solution adopted in this application is to provide a computer-readable storage medium, which is used for storing a computer program, and when the computer program is executed by a processor, is used to realize the audio frequency in the above-mentioned technical solution Noise reduction method.

Through the above scheme, the beneficial effects of the present application are: the audio noise reduction method provided by the present application can improve the operation speed of the audio noise reduction by streamlining the steps of the audio noise reduction method, thereby improving the efficiency of the audio noise reduction; By performing multiple minimum value tracking operations on the initialized noise estimation parameters, the accuracy of the subsequent calculation of the gain estimation value can be improved, thereby improving the noise reduction effect of the data to be denoised.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort. in:

1 is a schematic flowchart of an audio noise reduction method provided by the application;

Fig. 2 is the schematic diagram of speech noise reduction provided by the application;

3 is a schematic diagram of an implementation process of FFT and IFFT provided by the present application;

Fig. 4 is the schematic diagram of butterfly operation in FFT and IFFT provided by this application;

Fig. 5 is the schematic diagram of processing the operation result of IFFT provided by the present application;

6 is a schematic diagram of the noise spectrum estimation principle provided by the present application;

Fig. 7 is the schematic diagram of the gain calculation principle provided by the application;

8 is a schematic flowchart of an embodiment of an audio noise reduction device provided by the present application;

9 is a schematic structural diagram of an embodiment of an audio noise reduction device provided by the present application;

10 is a schematic structural diagram of another embodiment of an audio noise reduction device provided by the present application;

11 is a schematic diagram of the connection between the branching module and the scheduling module in the embodiment shown in FIG. 10;

12 is a schematic structural diagram of an embodiment of an audio noise reduction system provided by the present application;

FIG. 13 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided by the present application.

Detailed ways

The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is particularly pointed out that the following examples are only used to illustrate the present application, but do not limit the scope of the present application. Similarly, the following embodiments are only some of the embodiments of the present application, but not all of the embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

Reference in this application to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

It should be noted that the terms "first", "second" and "third" in this application are only used for description purposes, and should not be interpreted as indicating or implying relative importance or indicating the indicated technical features. quantity. Thus, a feature defined as "first", "second", "third" may expressly or implicitly include at least one of that feature. In the description of the present application, "a plurality of" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined. Furthermore, the terms "comprising" and "having", and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of an embodiment of an audio noise reduction method provided by the present application. The audio noise reduction method includes:

Step 11: Acquire the data to be denoised, and calculate the power spectrum of the data to be denoised.

The data to be denoised includes noise data and non-noise data. Specifically, the power spectrum of the first half of the data (such as the first 128 data points) in the data to be denoised can be calculated to obtain the first power spectrum, and then calculated based on symmetry. The power spectrum of the second half of the data to be denoised (such as the last 128 data points) is obtained to obtain the second power spectrum; then the first power spectrum and the second power spectrum are combined to obtain the power spectrum; Before the power spectrum of the denoised data, a Fast Fourier Transform (FFT) can be performed on the denoised data, and then the power spectrum of the first half of the data obtained by the FFT transform is calculated first, and then calculated based on the symmetry. The power spectrum of the second half of the data can be obtained to obtain the power spectrum of the entire data, so that it is not necessary to perform the power spectrum calculation for all the data, which can improve the operation efficiency.

Step 12: Initialize the noise estimation parameters based on the power spectrum to obtain the noise spectrum of the noise data.

After calculating the power spectrum of the data to be denoised, the noise estimation parameters can be initialized based on the power spectrum to obtain the noise spectrum of the noise data, and assign the power spectrum to the noise spectrum; The power spectrum is smoothed to obtain the smoothed power spectrum, and then the noise estimation parameters are initialized based on the smoothed power spectrum to obtain the noise spectrum; it is understandable that the three-point mean filtering method is a conventional operation method in the technical field. This will not be described in detail.

Step 13: Perform minimum value tracking on the initialized noise estimation parameters in the first time period to obtain a first array.

In a specific embodiment, the minimum value of the initialized noise estimation parameter can be calculated first to obtain the fourth array; then the minimum value tracking is performed on the fourth array to obtain the first array; the minimum value of the fourth array is obtained by Tracking, can check the accuracy of the fourth array, determine whether the fourth array is the minimum value of the initialized noise estimation parameters, if the fourth array is not the minimum value of the initialized noise estimation parameters, the minimum value can be performed Track, update the actual minimum value into the fourth array.

Step 14: Based on the first array, calculate a posteriori SNR and a priori SNR of the data to be denoised.

The posterior SNR may be calculated based on the first array, and the minimum value of the posterior SNR may be obtained to obtain a third array, and then the prior SNR may be calculated based on the third array.

Step 15: Based on the confidence of the posterior signal-to-noise ratio, calculate the estimated value of the probability of no sound, the estimated value of the probability of having a sound, and the estimated value of the noise power spectrum.

Step 16: Perform minimum value tracking on the initialized noise estimation parameters in the second time period to obtain a second array.

The second time period is after the first time period, that is, after the first time period ends, the second time period is started, and the minimum value tracking of the initialized noise estimation parameters is performed in the first time period. After the end, the minimum value tracking of the initialized noise estimation parameter is performed again in the second time period; it can be understood that the operation of the minimum value tracking performed in step 16 is the same as the operation of the minimum value tracking in the above-mentioned step 13, It is not repeated here.

In a specific embodiment, the step of performing minimum value tracking on the initialized noise estimation parameter can be performed twice or more, and the data to be denoised may include multiple frames of audio data, and the interval of a preset number of frames of audio data may be sufficient. By performing the operation of minimum value tracking once, the accuracy of the subsequent calculation of the gain estimation value can be improved by performing the minimum value tracking multiple times, thereby improving the noise reduction effect of the data to be denoised.

Step 17: Calculate the gain estimation value of the noise-free data based on the second array, the soundless probability estimation value, the sounding probability estimation value, and the noise power spectrum estimation value.

The soundless probability estimation value is the probability estimation value that the prior speech does not exist, and the sound probability estimation value is the probability estimation value that the conditional speech exists.

Step 18: Based on the estimated gain value, the noise spectrum, and the estimated value of the noise power spectrum, perform noise reduction processing on the data to be denoised to obtain noise-free data.

In a specific embodiment, after the second array is obtained, the estimated value of silent probability can be updated based on the second array to obtain the updated estimated value of silent probability; The spectrum is updated to obtain the updated noise spectrum; thus, the gain estimate is calculated based on the prior signal-to-noise ratio, the posterior signal-to-noise ratio, the updated silent probability estimate and the sound probability estimate; finally, based on the gain estimate and After the updated noise spectrum, noise reduction processing is performed on the data to be denoised to obtain noise-free data.

Specifically, the step of updating the estimated value of silent probability based on the second array, and obtaining the updated estimated value of silent probability may include: calculating the first estimated value of silent probability based on the second array; using three-point mean filtering method to smooth the first estimated value of silence probability to obtain the second estimated value of silence probability; perform window processing on the second estimated value of silence probability to obtain the third estimated value of silence probability; The estimated value is subjected to numerical transformation processing to obtain the updated silent probability estimated value.

In this embodiment, by streamlining the steps of the audio noise reduction method, the operation speed of audio noise reduction can be improved, thereby improving the efficiency of audio noise reduction; Improve the accuracy of the subsequent calculation of the gain estimation value, thereby improving the noise reduction effect of the denoised data.

In a specific implementation manner, the audio noise reduction method in the above-mentioned embodiment can also be used to realize the parallel processing of the data to be denoised, so as to improve the noise reduction rate; Multi-channel sub-audio data; then use the noise reduction processing method to perform parallel noise reduction processing on all sub-audio data to obtain noise-reduced sub-audio data, and then combine the noise-reduced sub-audio data to obtain noise-free data; wherein , the noise reduction processing method is the audio noise reduction method in the above embodiment.

Further, the audio noise reduction method in the above-mentioned embodiment can be applied to a programmable logic device (FieldProgrammable Gate Array, FPGA) to realize the parallel processing of the data to be noise reduction, and the steps of the audio noise reduction method based on the FPGA platform are carried out below. Specific introduction:

First, as shown in Figure 2, the implementation process of speech noise reduction is mainly divided into the following parts: time domain windowing, FFT transformation, noise reduction operation (Log-Spectral Amplitude estimator, LSA), inverse Fourier transform (Inverse Fourier transform) FastFourier Transform, IFFT), weight superposition and storage, etc.; specifically, before performing the IFFT operation, the audio data (that is, the data to be denoised in the above embodiment) may be preprocessed, such as: removing low frequencies, removing high frequencies or Multiply with the gain coefficient, etc.; after the IFFT operation, use weighted overlap-add, windowing and de-scaling to restore the audio time-domain data; among them, because the input audio data is before the FFT operation, do Frame-by-frame processing (1 frame is divided into 4 frames), so it is necessary to do weight overlap addition and other processing after IFFT operation to convert these 4 frames of data into one frame, so as to ensure that the number of input audio data and output audio data are consistent. The storage of the processed audio data is similar to the storage of the audio data input before entering the FFT operation, and the data before and after are correlated.

Specifically, the audio data to be denoised may be a noisy signal, which includes a noise signal and a pure signal, and the audio denoising is realized by extracting the pure signal, and the steps of speech denoising may include: 1) to the input audio data ( That is, the noisy signal) is divided into frames and windowed; 2) FFT operation is performed on each frame of the noisy signal; 3) The posterior SNR is estimated first, and then the a priori SNR is estimated by the decision-guided method, wherein, in Estimate the energy spectrum of noise in non-verbal segments (such as several frames before speech or speech gaps); 4) Use the optimal MMSE-LSA estimator to estimate the strength of the enhanced signal (equivalent to the following gain-seeking steps); 5. ) to reconstruct the enhanced signal spectrum, and then perform an IFFT operation on the enhanced signal spectrum to obtain the corresponding time domain signal of the enhanced speech (ie, the audio data after noise reduction).

(1) Since the human ear's perception of sound intensity is proportional to the logarithm of the spectral amplitude, assuming that the noise signal and the speech signal are not correlated with each other, the noisy signal can be expressed as y=x+d, where y is the band Noise speech, x is pure speech, d is additive stationary noise.

First, Y _k , X _k and D _k can be used to represent the k-th spectral component of the above y, x and d after FFT operation, respectively, and Y _k and B _k can be calculated using the following formulas (1) and (2) get:

Among them, R _k and B _k in the above formula (1) and formula (2) are the amplitudes of the noisy speech and the pure speech at the frequency point k, respectively, and θ _k and α _k are the frequency points of the noisy speech and the pure speech, respectively. phase of k.

Then use the following formulas (3) to (6) to estimate B _{k from Y k :}

为B_k的估计，可由上述公式(3)推导出上述公式(4)，上述公式(6)中G(ξ_k，γ_k)为增益函数，ξ_k和γ_k分别为先验信噪比(Signal-Noise Ratio，SNR)和后验信噪比，v_k＝(ξ_k/1+ξ_k)*γ_k，ξ_k＝λ_s(k)/λ_n(k)，γ_k＝R_k ²/λ_n(k)，λ_s为纯净语音方差，λ_n为噪声方差；由上述公式(5)可知，R_k ²与公式(6)所示的增益函数相乘即可得到纯净语音估计

Among them, in the above formula (3),

is the estimation of B _k , the above formula (4) can be derived from the above formula (3), G(ξ _k , γ _k ) in the above formula (6) is the gain function, and ξ _k and γ _k are the prior signal-to-noise ratios, respectively (Signal-Noise Ratio, SNR) and a posteriori SNR, v _k =(ξ _k /1+ξ _k )*γ _k , ξ _k =λ _s (k)/λ _n (k), γ _k =R _k ² /λ _n (k), λ _s is the variance of pure speech, and λ _n is the variance of noise; it can be seen from the above formula (5) that the pure speech can be obtained by multiplying R _k ² with the gain function shown in formula (6). estimate

(2) For the operations of FFT and IFFT, the radix-2 frequency domain/time domain decimation method can be used to realize the operation by transplanting the C source code. As shown in Figure 3, the operation of FFT and IFFT under the FPGA platform The implementation process is as follows:

The original audio data before the first butterfly operation can be stored through the Norml_ram (random access memory) shown in Figure 3. Norml_ram can include a real part memory (not shown in the figure) and an imaginary part memory (not shown in the figure). ), the size of each real part/imaginary part memory can be 256*16bit, DATA RAM0 and DATA RAM1 shown in Figure 3 can respectively store real part data and imaginary part data in FFT or IFFT operation, DATA RAM0 and DATA RAM1 The size can be 256*32bit.

When the butterfly operation is performed for the first time, the audio data can be directly input to the butterfly operation module without using the memory buffer, which can save the operation time. The size of the real part data and imaginary part data of the output audio data processed by FFT or IFFT is 16 bits. When performing FFT or IFFT operation, you can obtain FFT by checking the FFT co/sine table or IFFT co/sine table. Or the number of sine and cosine in IFFT operation, so as to adjust the data order of butterfly operation, for example: FFT co/sine table is FFTg_FFTCos or g_FFTReverse, the size of FFTg_FFTCos and g_FFTReverse can be 512*16bit and 256*16bit respectively.

Further, as shown in Figure 4, in the butterfly operation process of FFT, the audio data of the first 128 points and the audio data of the last 128 points are calculated separately, and the audio data of the first 128 points can be calculated first. Then, according to the symmetry, the FFT value of the audio data of the last 128 points is calculated; and in the butterfly operation process of IFFT, it is divided into The two butterfly operations directly calculate the audio data of 256 points, thereby obtaining the IFFT value of the audio data of 256 points. , IP) core and adder IP core to achieve, because the operation of the signed number is involved, the delay of the multiplier and the adder can be set to 2 clock cycles.

Understandably, as shown in Figure 5, the data after IFFT operation is 256 points, the current operation result needs to be superimposed on the previous operation result, and then output the lower 64-bit data as the result obtained by the current operation of the entire noise reduction algorithm. , after outputting the operation result, move the 64-255-bit data to 0-191-bit, and fill the upper 64-bit data with 0 for the next operation.

表示噪声谱估计值，G表示增益，Y_a ²表示功率谱，具体地，下面对计算数噪声谱估计值

和增益G的步骤进行介绍(即上述实施例中的音频降噪方法)：(3) The realization process of noise reduction operation is actually to calculate the noise spectrum estimation and gain. Specifically, noise reduction can be realized based on the noise spectrum estimation principle and gain calculation principle as shown in Figure 6 and Figure 7, where |Y| ² represents the energy of the speech signal (that is, the data after FFT operation), with λ _d representing the noise spectrum, with

represents the estimated value of the noise spectrum, G represents the gain, and Y _a ² represents the power spectrum. Specifically, the following is the estimated value of the calculated noise spectrum.

The steps of and gain G are introduced (that is, the audio noise reduction method in the above-mentioned embodiment):

1) Calculate the power spectrum of the data of the first 128 points, that is, Y _a ² =Real ² +Image ² , and restore the data according to the scaling value during the FFT operation, so as to obtain the value of the power spectrum, and store it in In the corresponding memory, the size of the memory can be 129*32bit; at the same time, three-point mean filtering is performed on the power spectrum, and the weights of the data of the previous point, the middle point and the latter point are set to 1/4, 1/ 2 and 1/4 to achieve S _f [i]=(Ya ² [i-1]>>2)+(Ya ² [i]>>1)+(Ya ² [i+1]>>2) , and then set the initial value of some intermediate array variables to the value after three-point mean filtering.

2) Initialize the noise spectrum based on the power spectrum, that is, λ _d =Y _a ² , and then initialize some intermediate array variables (ie, noise estimation parameters) in the memory (Blockram) according to the initialized noise spectrum, for example: nShiftYa2 (m_nShiftYa2) and eta(m_eta).

3) Limited by the number of adders and multipliers, the minimum value of some array variables can be calculated first, and then the first minimum value search in the array variables can be performed. The operation/assignment methods can be different for the first 14 frames and after 14 frames.

4) Calculate the minimum value of the posterior SNR, and calculate the prior SNR.

5) According to the confidence of the posterior signal-to-noise ratio (ie, the range), calculate the probability estimation value of the absence of a priori speech, the probability estimation value of the conditional speech existence, and the noise power spectrum estimation value by recursive averaging.

6) Execute the second minimum value search in the array variable. Except for the data of the 10th frame, the minimum value is calculated every 10 frames of data, and 5 values are reserved for each frequency point, and then stored in In five 129*32bit memories (Blockram), the minimum value of each frequency point is obtained.

7) Calculate the probability estimate of the absence of speech a priori, update the noise spectrum according to the estimated value of the noise power spectrum, and respectively add a local window (three-point mean filtering) and a global window (windowing, and restore data according to the calibration) to the probability estimate; Finally, through column transformation and other operations, a probability estimate of the absence of a priori speech is obtained.

8) Update some of the intermediate array variables for calculating the noise spectrum, for example, m_gamma&m_eta&m_v.

9) Update the value of the intermediate variable for calculating the estimated gain value, calculate the estimated value of the minimum gain value, and thereby calculate the gain G; wherein, in the calculation process, 5 temporary array variables are involved: ivUInt32m_min_temp[129], m_lambda_d_global[129], m_GH0[129], m_GH1[129] and ivUInt16m_PH1[129], the gain G and the temporary array variable m_PH1 can be multiplexed with a memory (Blockram), the depth of the memory is 129*16bit, and the temporary array variable m_GH1 is multiplexed with the power spectrum Ya ² A memory (Blockram).

和增益G中间变量eta_2term，完成降噪处理。11) Finally update the calculated noise spectrum estimate

and the gain G intermediate variable eta_2term to complete the noise reduction process.

The audio noise reduction method based on the FPGA platform in this embodiment can implement source code block according to the coupling degree of function and context by transplanting the source C, so that the coupling degree between each noise reduction and/or operation module can be reduced, thereby The parallel operation of the modules is realized, which greatly improves the effect and efficiency of noise reduction processing; in addition, all operations such as multiplication, addition or division are performed in a pipeline manner, which can improve the operation efficiency; in addition, some memories and operation modules can be reused, The resource occupation of the platform can be saved, and a clock frequency of more than 100M can be used in the platform to further improve the operation speed.

Please refer to FIG. 8. FIG. 8 is a schematic structural diagram of an embodiment of an audio noise reduction device provided by the present application. The audio noise reduction device 80 includes a memory 81 and a processor 82 that are connected to each other, wherein the memory 81 is used for storing computer programs. When executed by the processor 82, the computer program is used to implement the audio noise reduction method in the above-mentioned embodiment.

Please refer to FIG. 9 . FIG. 9 is a schematic structural diagram of an embodiment of an audio noise reduction apparatus provided by the present application. The audio noise reduction apparatus 10 includes a scheduling circuit 11 and a noise reduction circuit 12 .

The scheduling circuit 11 is used to receive the data to be noise-reduced corresponding to the receiving task, and perform demultiplexing processing on the data to be noise-reduced to obtain multi-channel sub-audio data; All corresponding sub-audio data are subjected to parallel noise reduction processing to obtain noise-reduced data to be noise-reduced.

In a specific embodiment, multiple sub-noise reduction circuits (not shown in the figure) can be set, and then the multiple sub-audio data are processed respectively by the multiple sub-noise reduction circuits, so as to realize parallel noise reduction processing of the sub-audio data, The number of sub-noise reduction circuits can be set according to actual needs, which is not limited here. It can be understood that the "noise reduction" in this embodiment is not limited to the noise reduction processing of the data to be denoised, but may also include operations such as separation or de-reverberation of the data to be denoised.

Specifically, the sub-audio data may be the data to be noise-reduced that needs noise reduction processing in the data to be noise-reduced. The scheduling circuit 11 can split the received data to be noise-reduced, identify the sub-audio data, and then The noise reduction data is divided into multi-channel sub-audio data; among them, "multi-channel" can be understood as "multi-channel", that is, each channel of sub-audio data is transmitted in parallel to the noise reduction circuit 12 by using multiple different transmission channels, so as to improve the data transmission efficiency. efficiency. It can be understood that the data to be denoised can be divided into four channels of sub-audio data or eight channels of sub-audio data, etc. The number of channels of the sub-audio data can be increased or decreased according to actual needs, which is not limited here.

Further, the receiving task and the noise reduction task can be performed at the same time, that is, the audio noise reduction device 10 can perform noise reduction processing on the received data to be denoised in real time, wherein the data to be denoised may be generated by an audio collection device (such as a pickup, etc.). ) data containing speech collected in real time, in this case, the scheduling circuit 11 can directly transmit the collected sub-audio data to the noise reduction circuit 12 at the first time, so that all the sub-audio data can be processed by the noise reduction circuit 12. Parallel noise reduction processing, to obtain the noise reduction data after noise reduction, does not need to transfer the noise reduction data through the computer, saves data transmission time, and can greatly improve the real-time performance of the noise reduction data processing. After practical application verification, compared with Compared with the existing solution, the solution in this embodiment can save 4-8 ms of time; or, in other implementations, the data to be noise-reduced can also be the data to be noise-reduced that is pre-stored in the audio collection device and collected before, here Not limited.

In a specific implementation manner, for each transmission channel, the scheduling circuit 11 may serially transmit the corresponding sub-audio data, and transmit part of the sub-audio data of a preset number (eg, 64 data points) each time. to the noise reduction circuit 12 to perform noise reduction processing by the noise reduction circuit 12; then transmit the next preset number of partial sub-audio data to the noise reduction circuit 12, and so on, until the noise reduction of all the sub-audio data is completed Processing; through this method of processing while transmitting, the processing pressure of the noise reduction circuit 12 can be reduced, and the efficiency of audio noise reduction can be improved. After practical application verification, the noise reduction time of the noise reduction circuit 12 for each sub-audio data is about 250us , which is far lower than the conventional noise reduction processing time of 2ms to 6ms in the related art.

In this embodiment, the scheduling circuit is used to perform the receiving task, that is, the data to be noise-reduced is received, and the data to be noise-reduced is demultiplexed to obtain multi-channel sub-audio data, so as to transmit the multi-channel sub-audio data to the noise-reduction circuit in parallel; The circuit performs the noise reduction task, that is, performs parallel noise reduction processing on all sub-audio data to obtain the noise reduction data to be noise reduction, which can greatly save the time of data transmission and noise reduction processing, thereby improving the efficiency of noise reduction processing; moreover, The receiving task and the noise reduction task are executed at the same time, which can receive the collected data to be denoised at the first time, and then transmit the data to be denoised to the denoising circuit without transferring the denoised data, which can greatly improve the processing efficiency of the denoised data. real-time.

Please refer to FIG. 10 . FIG. 10 is a schematic structural diagram of another embodiment of an audio noise reduction apparatus provided by the present application. The audio noise reduction apparatus 20 includes a scheduling circuit 21 and a noise reduction circuit 22 .

The scheduling circuit 21 includes a branching module 211 and a scheduling module 212. The branching module 211 is used to perform branch processing on the data to be denoised to obtain multiple channels of data to be denoised; the scheduling module 212 and the branching module 211 and The noise reduction circuit 22 is connected, and is used for scheduling and processing multiple channels of data to be noise reduction to obtain multiple channels of sub-audio data to the noise reduction circuit 22; specifically, the branching module 211 may be a data selector (multiplexer, MUX).

In a specific embodiment, the data to be denoised may include identification information and original audio information, the identification information includes a channel number identification and a noise reduction information identification, the channel number identification is used to identify the channel number of the data to be denoised, and each bit The data to be denoised may correspond to a channel number, and the denoising information identifier is used to identify denoising information of the data to be denoised, wherein the denoising information is used to indicate whether the data to be denoised needs denoising processing.

Specifically, the branching module 211 can identify the channel number in the channel number identifier, and use the transmission channel corresponding to the channel number to transmit the data to be noise-reduced; the scheduling module 212 can also identify the noise-reduction information in the noise-reduction information identifier, And based on the noise reduction information, it is judged whether the data to be noise-reduced is sub-audio data, that is, to determine whether the data to be noise-reduced needs noise reduction processing, and if the data to be noise-reduced is sub-audio data, the sub-audio data is input to the noise reduction circuit 22 ; If the data to be noise-reduced is not sub-audio data, the data to be noise-reduced can be input into other circuits to implement other processing operations or output directly.

As shown in FIG. 11 , the audio noise reduction device 20 further includes a storage module 23, and the storage module 23 is connected to the branching module 211, which is used to store the multi-channel data to be noise-reduced; specifically, the storage module 23 It can include a plurality of sub-storage modules 231, each sub-storage module 231 corresponds to the channel number in the channel number identification, and the branching module 211 can also output the data to be denoised to the corresponding sub-storage module 231 based on the channel number. .

The noise reduction circuit 22 may further include a plurality of sub noise reduction circuits 221 (three sub noise reduction circuits 221 are taken as an example in FIG. 10 ), the noise reduction data to be noise reduction after noise reduction may include a plurality of sub noise reduction audio data, each sub noise reduction circuit 221 It is connected to the scheduling circuit 21, and each sub-noise reduction circuit 221 performs noise reduction processing on the corresponding to-be-processed noise-reduced data to obtain corresponding sub-noise reduction audio data; in other embodiments, the audio noise reduction device 20 can also A demultiplexer (DEMUX) is included to output a plurality of sub noise reduction audio data through the DEMUX.

In a specific embodiment, the audio noise reduction device 20 can be implemented based on an FPGA. After an actual test, the size of the FPGA-based audio noise reduction device 20 can be controlled within 50mm×40mm, and the audio noise reduction device 20 can be integrated in a In one IP core, in different application scenarios, flexible transplantation can be realized, so that the audio noise reduction device 20 can be widely used in any FPGA platform; and because the FPGA has low cost, large scale, high integration, and low power The audio noise reduction device 20 based on the FPGA platform can save noise reduction costs and improve the real-time performance and efficiency of data noise reduction.

Moreover, in the audio noise reduction scheme based on other platforms (such as IMAX6Q or STM32, etc.), after the noise reduction data is collected, it is necessary to start the noise reduction after a frame of the noise reduction data is received and the data is updated. For example, taking a frame of data to be noise-reduced contains 256 data points as an example, only 64 data points can be updated at a time, then it needs to be updated four times before the noise-reduction processing can be started. The collection of the data to be denoised in the noise reduction device 20 is implemented in the FPGA. It is not necessary to wait for a frame of the denoised data to be updated before starting the dimensionality reduction process, and the audio can be started after receiving 1/4 of the frame of the denoised data to be denoised. Noise reduction processing, thereby greatly improving the efficiency of data transmission to be noise reduction and noise reduction processing.

Specifically, the noise reduction circuit 22 can be used to implement the audio noise reduction method in the above-mentioned embodiment, compared with other speech noise reduction algorithms (such as spectral subtraction, adaptive filtering, Wiener filtering or minimum mean square error estimation (Minimum mean square error estimation). Mean Square Error, MMSE), etc.), using the audio noise reduction algorithm in the above embodiment to perform noise reduction processing, can optimize the effect of speech noise reduction processing, achieve higher suppression of background noise, and make the distortion of the data to be noise reduced more. Low.

In this embodiment, the branching module and the scheduling module are used to realize the branching and scheduling of the data to be denoised, and in different transmission channels, multiple channels of data to be denoised that need to be denoised are transmitted in parallel to a plurality of subsequent sub-denoisers In the circuit, by setting a plurality of sub-noise reduction circuits, parallel noise reduction processing for multiple sub-audio data is realized, thereby greatly improving the efficiency of audio noise reduction; moreover, the audio noise reduction device in this embodiment can be implemented on an FPGA platform. At the same time, the parallel processing of multi-channel audio noise reduction methods is supported by the parallelism and rapidity of data processing on the FPGA platform, which can improve the noise reduction effect while still ensuring the processing speed, and can solve the problem of time-consuming audio noise reduction and slow response. problem, and further improve the efficiency of audio noise reduction.

Please refer to FIG. 12. FIG. 12 is a schematic structural diagram of an embodiment of an audio noise reduction system provided by the present application. The audio noise reduction system 120 includes an audio collection device 121 and an audio noise reduction device 122. The audio collection device 121 is used in the collection target scene. The audio noise reduction device 122 is connected to the audio collection device 121, which is used to perform noise reduction processing on the noise reduction data to obtain the audio data after noise reduction; wherein, the audio noise reduction device 122 is the above-mentioned The audio noise reduction device in the embodiment.

Please refer to FIG. 13. FIG. 13 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 130 is used to store a computer program 131. When the computer program 131 is executed by a processor, it is used to realize The audio noise reduction method in the above embodiment.

The computer-readable storage medium 130 may be a server, a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, etc. medium of program code.

In the several embodiments provided in this application, it should be understood that the disclosed method and device may be implemented in other manners. For example, the device implementations described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other divisions. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.

Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this implementation manner.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The above are only the embodiments of the present application, and are not intended to limit the scope of the patent of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present application, or directly or indirectly applied to other related technologies Fields are similarly included within the scope of patent protection of this application.

Claims (19)

1. A method for audio noise reduction, the method comprising:

acquiring data to be denoised, and calculating a power spectrum of the data to be denoised, wherein the data to be denoised comprises noise data and noise-free data;

initializing a noise estimation parameter based on the power spectrum to obtain a noise spectrum of the noise data;

carrying out minimum tracking on the initialized noise estimation parameters in a first time period to obtain a first array;

calculating a posterior signal-to-noise ratio and a prior signal-to-noise ratio of the data to be denoised based on the first array;

calculating a silence probability estimation value, a sound probability estimation value and a noise power spectrum estimation value based on the confidence coefficient of the posterior signal-to-noise ratio;

carrying out minimum tracking on the initialized noise estimation parameters in a second time period to obtain a second array;

calculating a gain estimation value of the noise-free data based on the second array, the silence probability estimation value, the sound probability estimation value and the noise power spectrum estimation value;

and performing noise reduction processing on the data to be subjected to noise reduction based on the gain estimation value, the noise spectrum and the noise power spectrum estimation value to obtain the noise-free data.

2. The audio noise reduction method of claim 1, further comprising:

updating the silence probability estimation value based on the second array to obtain an updated silence probability estimation value;

updating the noise spectrum based on the noise power spectrum estimated value to obtain an updated noise spectrum;

calculating the gain estimation value based on the prior signal-to-noise ratio, the posterior signal-to-noise ratio, the updated silence probability estimation and the voiced probability estimation;

and performing noise reduction processing on the data to be subjected to noise reduction based on the gain estimation value and the updated noise spectrum to obtain the noise-free data.

3. The audio denoising method of claim 1, wherein the step of initializing a noise estimation parameter based on the power spectrum to obtain a noise spectrum of the noise data comprises:

smoothing the power spectrum by adopting a three-point average filtering method to obtain a smoothed power spectrum;

initializing the noise estimation parameters based on the smoothed power spectrum to obtain the noise spectrum.

4. The method of claim 1, wherein the step of calculating a posteriori snr and a priori snr of the data to be denoised based on the first array comprises:

calculating a posterior signal-to-noise ratio based on the first array, and acquiring the minimum value of the posterior signal-to-noise ratio to obtain a third array;

and calculating the prior signal-to-noise ratio based on the third array.

5. The audio denoising method according to claim 1, wherein the step of calculating the power spectrum of the data to be denoised comprises:

calculating the power spectrum of the first half part of data in the data to be denoised to obtain a first power spectrum;

calculating a power spectrum of the latter half data in the data to be denoised based on symmetry to obtain a second power spectrum;

and combining the first power spectrum and the second power spectrum to obtain the power spectrum.

6. The method of claim 1, wherein the step of performing minimum tracking on the initialized noise estimation parameters in the first time period to obtain the first array comprises:

calculating the minimum value of the initialized noise estimation parameters to obtain a fourth array;

and carrying out minimum value tracking on the fourth array to obtain the first array.

7. The audio denoising method of claim 1, wherein the data to be denoised comprises a plurality of frames of audio data, the method further comprising:

and performing the minimum tracking operation at intervals of a preset number of frames of audio data.

8. The audio noise reduction method of claim 2, wherein the method comprises:

the step of updating the silence probability estimation value based on the second array to obtain an updated silence probability estimation value includes:

calculating a first silence probability estimation value based on the second array;

smoothing the first silence probability estimation value by adopting a three-point mean filtering method to obtain a second silence probability estimation value;

windowing the second silence probability estimation value to obtain a third silence probability estimation value;

and carrying out numerical transformation processing on the third silence probability estimation value to obtain an updated silence probability estimation value.

9. The audio noise reduction method of claim 1, further comprising:

carrying out shunt processing on the data to be denoised to obtain multi-path sub-audio data;

performing parallel noise reduction processing on all the sub-audio data by adopting a noise reduction processing method to obtain noise-reduced sub-audio data, wherein the noise reduction processing method is the audio noise reduction method of any one of claims 1 to 8;

and merging the sub audio data subjected to noise reduction to obtain the noiseless data.

10. An audio noise reduction arrangement comprising a memory and a processor connected to each other, wherein the memory is adapted to store a computer program, which when executed by the processor is adapted to implement the audio noise reduction method of any of claims 1-9.

11. An audio noise reduction apparatus for simultaneously performing a reception task and a noise reduction task, the audio noise reduction apparatus comprising:

the scheduling circuit is used for receiving the data to be denoised corresponding to the receiving task and carrying out shunt processing on the data to be denoised to obtain multi-path sub-audio data;

the noise reduction circuit is connected with the scheduling circuit and is used for carrying out parallel noise reduction processing on all the sub-audio data corresponding to the noise reduction task to obtain noise-reduced audio data; wherein the noise reduction circuit is configured to implement the audio noise reduction method of any of claims 1-9.

12. The audio noise reduction device of claim 11,

the denoised audio data comprises a plurality of sub-denoised audio data; the noise reduction circuit comprises a plurality of sub noise reduction circuits, and each sub noise reduction circuit is connected with the scheduling circuit and is used for carrying out noise reduction processing on corresponding sub audio data to obtain the sub noise reduction audio data.

13. The audio noise reduction device of claim 11, wherein the scheduling circuit comprises:

the shunting module is used for carrying out shunting processing on the data to be subjected to noise reduction to obtain a plurality of paths of data to be subjected to noise reduction;

and the scheduling module is connected with the shunt module and the noise reduction circuit and used for scheduling the multi-path data to be subjected to noise reduction to obtain the multi-path sub-audio data and transmitting the multi-path sub-audio data to the noise reduction circuit.

14. The audio noise reduction device of claim 13,

the data to be denoised comprises identification information and original audio information, wherein the identification information comprises a channel number identification and a denoising information identification; and the shunting module is also used for identifying the channel number in the channel number identification and transmitting the data to be denoised by adopting the transmission channel corresponding to the channel number.

15. The audio noise reduction device of claim 14,

the scheduling module is further configured to identify noise reduction information in the noise reduction information identifier, and determine whether the data to be noise reduced is the sub-audio data based on the noise reduction information; and if so, inputting the sub-audio data to the noise reduction circuit.

16. The audio noise reduction device of claim 14,

the audio noise reduction device further comprises a storage module, wherein the storage module is connected with the shunt module and used for storing the multi-path data to be subjected to noise reduction.

17. The audio noise reduction device of claim 16,

the storage module comprises a plurality of sub-storage modules, each sub-storage module corresponds to a channel number in the channel number identification, and the branching module is further used for outputting the multi-channel data to be denoised to the corresponding sub-storage module based on the channel number.

18. An audio noise reduction system, comprising:

the audio acquisition device is used for acquiring sound in a target scene to obtain data to be denoised;

the audio noise reduction device is connected with the audio acquisition device and is used for carrying out noise reduction processing on the data to be subjected to noise reduction to obtain noise-reduced audio data; wherein the audio noise reduction device is as claimed in any of the preceding claims 11-17.

19. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, is adapted to implement the audio noise reduction method of any of claims 1-9.

Images