CN101261833B - A Method for Audio Error Concealment Using Sine Model - Google Patents

Abstract

The invention provides a method for using a sine model to conceal an audio mistake, which belongs to the network and communication field. The operation procedures include: 1. the rebuilding of the whole system is classified into four parallel instances according to the mistakes of a previous frame and a current frame: A. the previous frame and the current frame are both correct and return after being correctly decoded; B. the previous frame is correct and the current frame is wrong, then a time domain model is analyzed and synthesized, and a rebuilding frame is made by overlapping the odd symmetrical section of a synthesized frame and the residual even symmetrical section; C. the previous frame is wrong and the current frame is correct, then a rebuilding frame is made by overlapping the even symmetrical section of the synthesized frame obtained from the model analysis and the residual odd symmetrical section; D. both the previous frame and the current frame are wrong, then the current frame is treated in a simple way; 2. output the obtained decoding result or the wrongly concealed time domain rebuilding. The method is suitable for treatment under various frame loss rates, and the continuous frame loss probability is relatively small for the medium or low frame loss rates, while the medium or high frame loss rates are treated by using a simple frame loss strategy, which can not cause greater audio distortion.

Description

A kind of sinusoidal model that uses carries out the method that audio error is hidden processing

Technical field

The invention belongs to network and communications field, specially refer to a kind of sinusoidal model that uses and carry out the method that audio error is hidden processing.

Background technology

Along with developing rapidly of wired and wireless network, the user by the Internet to the demand of audio service more and more widely comprises video conference, Online Music program request, IP phone etc.Though yet have channel coding/decoding to realize error correction and error detection, but can not guarantee that still the Internet voice communication provides the service of quality (QoS), in network service, especially in the wireless communication, owing to reasons such as multipath fading, noise and network congestions, often cause the mistake of digital audio compression data-bag lost or packed data.Because audio compressed data is to burst error, particularly the frame head information errors of packed data or lose is very responsive, can cause the decoding failure of packed data, thereby causes the receiving end audio quality to worsen when audio error is hidden module not introducing.Hide about audio error, the definition that audio error is hidden has two kinds of broad sense and narrow senses, in fact the error concealing of narrow sense carries out at demoder, at the mistake that occurs in the decoding tonic train is rebuild regeneration, and the audio error of broad sense is hidden and also to be included in coding side and just to add certain redundant information, is beneficial to the reconstruction of decoding end.General said error concealing is exactly the error concealing of narrow sense, different with error correction algorithms such as traditional forward error correction and automatic repeat requests as can be seen, the error concealing of narrow sense does not take extra bandwidth, the residual risk that the signal that only utilizes receiving end to receive provides has higher channel bandwidth utilization ratio.

The channel forward error correction coding is a kind of processing at the mistake that may occur in the Channel Transmission, need not be concerned about the content of information source, just when information source is transmitted, adds the redundant information of upper signal channel so that decoding end is carried out error detection or error correction.Different with the channel forward error correction coding, rebuild the audio decoder that generally is based on time domain at the sound signal of message source process and export the problem that occurs, disturb as time domains such as click, high frequency noises, use a kind of wrong cancellation scheme to remove them.The error concealing of audio frequency then comprises wider, can be the processing of time domain, can be in the decoder decode process, utilizes the information of frequency domain, transform domain to carry out the mistake reconstruction, can also utilize the additional data of making a start as instructing signal to be rebuild.And error concealing generally is the packet drop that may occur in the network application, whole audio frame promptly occurs and loses, and substituting of reconstructing lost frame makes the subjective auditory effect of people's ear better, reduces the tonequality damaged condition.Audio error is hidden pass with Error Correction of Coding and signal reconstruction and is tied up in the audio communication system as shown in Figure 1.For recording-media, as CD etc., because of reasons such as damages, often wave file is produced certain local interference, as the existence of clickly noise click, like this when playing, without any handling, just can hear as distortions such as buzzing, spines, influence people's the sound quality of listening.Thereby press for some and effectively handle and method at the mistake of voice data.Coding/decoding system based on conversion, in order to solve the blocking effect that produces through coding/decoding system between transform block, generally all adopt adjacent coded frame to have overlap-add, this thinking allows the present coding/decoding system based on conversion adopt a kind of time-frequency conversion method of overlap-add, the correction cosine transform of adopting in the audio coding and decoding system of the more representational MPEG of being (is Modified Discrete Cosine Transform, MDCT abridges) and the G.722.1 middle superimposed modulation conversion of adopting of ITU (being Modulated Lapped Transform, abbreviation MLT).

The overlap-add strategy that uses based on the coding/decoding system of conversion makes that the reconstruction of the generation of coded sequence of a frame or decoding sequence all is that coded sequence or interface sequence by front and back two frames obtains.MDCT time-frequency conversion with 50% overlap-add commonly used in the MPEG sensing audio code decode algorithm is an example, according to the characteristic of MDCT, if with 2 isometric sequence { x ₁, { x ₂Superposeing in proper order obtains [{ x ₁, { x ₂] after carry out the MDCT conversion, and then transformation results is carried out the IMDCT conversion, Shu Chu result is so

$\mathrm{IMDCT}\left(\mathrm{MDCT}\left(\right[\left\{x_1\right\},\left\{x_2\right\}\left]\right)\right)=\left[\right\{x_1-x_1^r\},\{x_2+x_2^r\left\}\right]$

X wherein ₁ ^rThe backward of expression sequence x is arranged, if carry out for twice the data of MDCT conversion have 50% overlapping, so final overlapping x ₂Be resumed out.The characteristic of this overlap-add allows the encoding code stream of a frame sequence in former and later two coded frame, the complete decoding of one frame sequence also needs the decoded result of former and later two coded frame to do overlap-add, if take place under the loss situation of a frame sequence simultaneously, also can cause losing of two continuous frames sequence, its error model as shown in Figure 2.

Sinusoidal analysis and comprehensive (Sinusoidal analysis and synthesis, the SAS of abbreviation) at first to the signal windowing, then the signal of fixing segment being analyzed, is signal indication the stack of one group of sinusoidal signal, and the parameter of these sinusoidal signals is exactly amplitude, frequency and phase place.If all amplitudes, frequency and the phase place of the sinusoidal model that analysis is obtained all are used for unified model, then the tonequality of Chong Jianing can reach near transparent tonequality.

Along with the Internet network and the third generation, constantly improving and development of super 3 g mobile communication, communication bandwidth is constantly expanded, the audio-visual multimedia service is more and more abundanter, interactive online service requires high to the data transmission real-time, traditional error correction mode has occupied massive band width and has caused bigger time delay, be difficult to satisfy the requirement of height real-time, the high-quality high real-time of service requires occupying bandwidth as few as possible, increase the high-quality video/audio communication of realization under the time delay condition as few as possible, the Error concealment techniques that can satisfy these requirements simultaneously becomes the focus that people pay close attention to.Audio error is hidden and is with a wide range of applications the error correction of Web conference, IP phone, mobile communication, online request, digital broadcasting and disc record.

Network flow-medium such as video conference and remote teaching service request is guaranteed the real-time Transmission of sound, video media stream, the RTP that is based on UDP (User Datagram Protoco (UDP)) (RTP) that present transmission over networks sound, video data generally adopt.Udp protocol can utilize the bandwidth for transmission data more fully, obtain better result of use, but this agreement itself does not have retransmission mechanism, can be because the obstruction of network or the delay of transmission cause the receiving end data-bag lost, influence sound, video quality, therefore need to adopt Error concealment techniques to cover up and lose the mistake that causes, improve the medium output quality.

The nineties in 20th century, error concealing method is suggested in ETSI as the optional suggestion of gsm system, be used to improve the voice quality of cell mobile communication systems, and in the wideband speech coding AMR-WB+ of 3G (Third Generation) Moblie, adopted corresponding error to hide scheme again and hidden error code or lose the influence that voice quality is caused.

Summary of the invention

The technical issues that need to address of the present invention are, because fast development along with the network and communications, traditional error correction mode occupies massive band width because of it, cause bigger time-delay can not satisfy the requirement of height real-time, in order to overcome these deficiencies of the prior art, realize high-quality shadow, sound communication, a kind of new disposal route of the proposition that will innovate, the objective of the invention is a kind of sinusoidal model that uses and carry out the method that audio error is hidden processing, this method has been utilized the characteristic based on the coding/decoding system of overlap-add, at first set model is analyzed, the parameter of extraction and analysis model is done suitable processing then, comprehensively obtain an audio frequency reconstruction frames at last, extract the suitable part and the residual fraction addition of this reconstruction frames, just the sequence that can obtain rebuilding.

The concrete technical scheme of finishing purpose employing of the present invention is as follows: a kind of sinusoidal model that uses carries out the method that audio error is hidden processing, it is characterized in that, operates as follows:

(1) at first, the reconstruction of total system is divided into 4 kinds of situations arranged side by side according to the mistake of former frame and present frame, frame and present frame all do not have mistake before the A., are correctly decoded and return; B. preceding frame is errorless, and the present frame mistake is then carried out analysis integratedly to Model in Time Domain, obtain comprehensive frame, the odd symmetry of comprehensive frame part and residual even symmetry part addition as reconstruction frames; C. preceding frame mistake, present frame is errorless, and then the even symmetry of the comprehensive frame that model analysis is obtained part and the addition of residual odd symmetry part are as reconstruction frames; D. preceding frame and present frame be mistake all, and then simple strategy is handled present frame;

(2). then, the time domain that is correctly decoded result or error concealing that output obtains is above rebuild.

Mistake according to former frame and present frame of the present invention is divided into 4 kinds of situations arranged side by side and at length is described below:

(A), preceding frame and present frame all do not have mistake

In this case, two frames all are correctly decoded, and do not have mistake this moment, and directly the output decoder result gets final product.

(B) errorless, the present frame mistake of former frame

The process of reconstruction of this frame is divided into two parts, promptly at first pass through the Model in Time Domain analysis, by the steps such as peak value calculating, the adjacent compensation of frequency spectrum peak dot, phase compensation and energy compensating of frequency spectrum being obtained the reconstruction of frequency domain, through IFFT, the i.e. time-domain signal that can obtain rebuilding, then for the time domain continuity between consecutive frame, get the addition of overlapping of the odd symmetry part of signal of reconstruction and residual even symmetry.The process of reconstruction frames comprises spectrum peak exactly and detects, the amplitude compensation (according to the characteristic of sinusoidal signal) of adjacent peak point, then the original phase of analyzing template is done compensation, then to doing energy compensating because spectrum peak detects the energy minimizing that causes reconstruction frames, promptly obtain the reconstruction frames of frequency domain form, realize the conversion of frequency domain to time domain then by IFFT.

(C), former frame mistake, present frame are errorless

The process of reconstruction of this frame and top situation are similar, also be at first analytical model to be carried out frequency-domain analysis, extract spectrum peak, carry out the reconstruction that steps such as the adjacent compensation of frequency spectrum peak dot, phase compensation and energy compensating obtain frequency domain then, through IFFT, promptly the time-domain signal that can obtain rebuilding is overlapped addition to the even symmetry of the reconstruction signal of time domain part and residual odd symmetry as final reconstruction.If the complexity that taking into account system is realized, because present frame is errorless, so need decode to it, if add top SAS process, the additional algorithm complexity that may cause is higher, so do not have under the wrong situation about taking place at present frame, for reducing the complexity of error concealing, can consider to lose the reconstruction frames of reconstruction frames, then the signal of rebuilding be done overlap-add again through certain decay, as shown in Figure 5 as present frame with former frame.

(D) preceding frame and present frame all make a mistake

At this moment, successive frame takes place lose, mistake is comparatively serious, can directly use simple error concealment strategy, is with the frame replication strategy of decaying or quiet strategy or the like as the reconstructed results of preceding frame is directly used.

The invention has the beneficial effects as follows, this method is suitable for the processing under the various frame loss rates, for in low frame loss rate, the probability that successive frame is lost is less, if single LOF takes place, strategy (B) above using and (C) carry out error concealing will improve greatly and rebuild quality afterwards, and people's ear can not listened distortion substantially.And in high frame loss rate, use simple frame losing strategy, can not cause bigger audible distortion.The strategy of branch state processing allows this method be fit to the situation that various channel conditions cause frame losing.This method realizes not increasing the time-delay of coding/decoding system, in to rebuild tonequality under the low frame loss rate higher, can not introduce bigger audible distortion under the high frame loss rate yet, tonequality people's ear can be stood, the moderate complexity that while method realizes is fit to be applied in the field that complexity is had relatively high expectations such as mobile, wireless.

The present invention be advantageous in that for every kind of SAS disposal system, the analysis template of sinusoidal model can be chosen arbitrarily before lost frames.Make a mistake as present frame, and former frame does not have mistake, this moment, the time-domain signal that is used to analyze can directly adopt the time-domain signal of former frame, also can select the template of the part of the part of former frame and residual signal as analysis frame.

The present invention be advantageous in that the improvement that peak value is detected strategy.Because error concealing is a kind of method of fuzzy reconstruction,,, judge that promptly current sample value is greater than the front and back sampling point so use simple peak value to detect strategy so do not need point-device estimation.For the adjacent compensation of peak dot, promptly to the interpolation of adjacent peak point, because the operational precision of DFT (Discrete Fourier Transform, i.e. discrete Fourier transformation) is limited, its frequency resolution only is F in addition _s/ N (F wherein _sBe sampling rate, N is that conversion is counted), all be the sinusoidal component of complete meaning so can not guarantee each peak point, and may be one of sinusoidal component approximate.If at this moment directly represent sinusoidal component with the peak point of DFT, because its limited accuracy, may cause estimating inaccurate, only can cause the variation of adjacent peak point discontinuous with peak point, be equivalent to peak point is weighted with a rectangular window, can cause the leakage of time domain like this, so consider with a more level and smooth window function, so consider to carry out interpolation at adjacent peak point, promptly except keeping peak point, also keep near several sampling points of peak point to keep the level and smooth of frequency spectrum, do certain amplitude attenuation near the sampling point the peak point simultaneously.

The present invention be advantageous in that after carrying out the spectrum peak detection frequency domain phase place of analyzing template is compensated.The compensation of phase place can be carried out according to the residual phase of residual frame, also can directly compensate analyzing the template phase place.A kind of effective phase compensation strategy is a linear phase characteristic of considering discrete Fourier transform (DFT) (DFT), can do the linear phase adjustment according to the skew of analyzing between template sequence and the reconstruction sequence.

The present invention be advantageous in that additional energy compensating on the basis of amplitude, phase compensation of offset of sinusoidal model.Energy after this compensation detects at peak value should guarantee that the energy of reconstruction frames can not be too little, does not allow the energy of reconstruction frames excessive again, produces the noise of pleasant audible.

The method that reduces the error concealing added complexity that provides is provided.If promptly present frame does not have mistake, and the former frame mistake,, certainly will cause decoding complex degree to increase if also carry out analysis integratedly with sinusoidal model according to Fig. 5.So the reconstruction frames that this method allows comprehensively to have come out with the front has so just reduced the complexity increase that causes because introduce error concealment module through the comprehensive frame of certain decay as present frame.

The present invention be advantageous in that provides the error concealment schemes make that is fit to various time delay conditions, the error concealing method that makes arbitrarily of analyzing the template selection can not increase the time-delay of original coding/decoding system, also can increase the time-delay of system in order to pursue higher reconstruction tonequality.This algorithm is rebuild tonequality, the algorithm time-delay is applicable to the disparate networks condition with customizable this algorithm that makes of algorithm implementation complexity.

Description of drawings

Fig. 1 is the square frame connection diagram of audio communication system;

Fig. 2 is the frame losing error pattern synoptic diagram based on the coding/decoding system of overlap-add conversion;

Fig. 3 is for hiding the scheme synoptic diagram based on using sinusoidal model to carry out audio error in the transition coding system;

Fig. 4 has only the SAS of present frame mistake to handle block diagram;

Fig. 5 has only the SAS of former frame mistake to handle block diagram;

Fig. 6 is the analysis window position view of sinusoidal model;

Fig. 7 is the energy compensating process flow diagram.

Embodiment

With reference to Fig. 1, the square frame annexation synoptic diagram of expression audio communication system is expressed audio error and is hidden with the difference of Error Correction of Coding and signal reconstruction and get in touch among the figure.

With reference to 2 expressions, based on the frame losing error pattern of the coding/decoding system of overlap-add time-frequency conversion, represent among the figure, if take place also can cause losing of continuous two sequences under the loss situation of a frame sequence, with reference to Fig. 2 as can be seen, if the current [{ x that comprises ₂, { x ₃] frame data of coded message lose, the present frame even symmetry part { x of remaining real time sequence only then ₂+ x ₂ ^r, and the only remaining odd symmetry part of next frame { x ₃-x ₃ ^r.Promptly work as losing of last frame data and can cause two continuous frame mistakes.Processing mode under this error pattern, be the reconstruction of total system according to former frame and present frame whether mistake is divided into four kinds of situations: (A) preceding frame and present frame all do not have mistake, (B) errorless, the present frame mistake of former frame, (C) former frame mistake, present frame are errorless, and (D) preceding frame and present frame all make a mistake.

With reference to Fig. 3 and Fig. 4, expression is based on using sinusoidal model to carry out the error concealment schemes make synoptic diagram in the transition coding system and having only the SAS of present frame mistake to handle block diagram.

Represent to have only preceding frame mistake SAS to handle block diagram with reference to Fig. 5, carry out analysis integrated with sinusoidal model, expression can be used for the implementation of shortcut calculation, the i.e. decay that the reconstruction frames process of directly comprehensively having come out with the front is certain has reduced the added complexity that error concealment algorithm realizes as the comprehensive frame of present frame.

With reference to Fig. 6 is the analysis window position view of sinusoidal model, if present frame is the i frame, at first residual present frame information is connected with the time-domain information that the correct interface of former frame obtains, analysis window can be chosen in the optional position (as shown in Figure 6) of this section time domain data then, as the center of analysis window is placed on the initial of i frame residual risk.Also similar for the situation of having only former frame to make a mistake, the analysis template of the reconstruction sinusoidal model of this part can be chosen in the optional position of 4 listed among Fig. 6 frame time domain sequences.For window function, can select in addition, as using sinusoidal windows among the AAC analyzing the template windowing according to concrete coding/decoding system.

With reference to Fig. 7 is the energy compensating process flow diagram, because peak value detects, some non-peak points directly make zero, cause signal energy to descend, so utilize the signal energy of peak value before and after detecting to try to achieve a compensating factor, then this factor through certain decay, as decay to original 0.8, do an amplitude limit then, excessive with the energy that prevents reconstruction frames.Amplitude limit value and decay factor can be adjusted.

Claims (4)

1. A method of using a sinusoidal model to carry out audio error concealment processing, characterized in that it operates as follows: (1) first, the reconstruction of the whole system is divided into 4 types of juxtaposition according to the errors of the previous frame and the current frame In the case, A. The previous frame and the current frame have no errors, and the correct decoding returns; B. The previous frame is correct, but the current frame is wrong, then use the sinusoidal model to analyze and synthesize the time domain signal to obtain the integrated frame, and the odd symmetric part of the integrated frame Add the remaining even symmetric part to the reconstructed frame; C. The previous frame is wrong and the current frame is correct, then use the sinusoidal model to analyze and synthesize the time domain signal, and then add the even symmetric part of the comprehensive frame and the residual odd symmetric part as Reconstruct the frame; D. Both the previous frame and the current frame are wrong, then a simple strategy is used to process the current frame; (2) Then, output the correct decoding result obtained above or the time domain reconstruction of the error concealment; The analysis and synthesis is to use the input time-domain signal as an analysis template, and synthesize the reconstructed time-domain signal by adjusting the three components of the amplitude, frequency and phase of the sinusoidal model, specifically including: Performing peak detection, the peak detection is to enhance the extraction accuracy of frequency components through interpolation of adjacent peak points; According to the time-frequency transformation characteristics, the phase of the analysis template is adjusted linearly; adjusting the amplitude of the sinusoidal component according to the energy of the signal before and after the peak detection; Perform energy compensation after peak detection. 2. the method for using sinusoidal model to carry out audio error concealment processing according to claim 1, is characterized in that, described peak detection, is to carry out interpolation at adjacent peak point, keep some sample points near the peak point to keep frequency spectrum smoothness. 3. the method for using the sinusoidal model to carry out audio error concealment processing according to claim 1, wherein the linear phase adjustment is to do linear phase compensation according to the time offset between the analysis template and the lost frames for error concealment . 4. The method for using a sinusoidal model to conceal audio errors according to claim 1, wherein the energy compensation is to calculate a compensation factor according to the energy change after peak detection, and through attenuation and limiting to ensure audio Continuity of energy between frames.

Images