RU2414009C2 - Signal encoding and decoding device and method - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: decoding method involves extraction of a set of encoded signals from an input bit stream, determination of which method from a set of decoding methods is to be used to decode each of the encoded signals, decoding the encoded signals using defined decoding methods and synthesising the decoded signals. Accordingly, signals having different characteristics can be encoded with optimum bit rate by attributing the said signals to one or more classes in accordance with characteristics of these signals and encoding each of the signals using the encoding unit which best suits the class to which the corresponding signal belongs.

EFFECT: efficient encoding of different signals, including audio and speech signals.

16 cl, 16 dwg

Description

FIELD OF THE INVENTION

The present invention relates to encoding and decoding devices and to encoding and decoding methods, and more particularly, to encoding and decoding devices and to encoding and decoding methods that can encode or decode signals with an optimal bit rate (bit rate) according to signal characteristics.

State of the art

Conventional audio encoders can provide high-quality audio signals with a high bit rate of 48 Kbps or more, but are inefficient for processing speech signals. Conventional speech encoders, on the other hand, can efficiently encode speech signals with a low bit rate of 12 Kbit / s or less, but are not suitable for encoding various audio signals.

Disclosure of invention

Technical problem

The present invention provides encoding and decoding devices and encoding and decoding methods that can encode or decode signals (eg, speech and audio signals) having various characteristics with an optimal bit rate.

Technical solution

According to an aspect of the present invention, there is provided a decoding method including extracting a plurality of encoded signals from an input bitstream, determining which of a plurality of decoding methods should be used to decode each of the encoded signals, decoding the encoded signals using specific decoding methods, and synthesizing the decoded signals.

According to another aspect of the present invention, there is provided a decoding device including a bit decompression module that extracts a plurality of encoded signals from an input bit stream, a decoder determination module that determines which of the plurality of decoding blocks should be used to decode each of the encoded signals, a decoding module which includes decoding units and decodes encoded signals using specific decoding units, and a synthesizer bar that synthesizes decoded signals.

According to another aspect of the present invention, there is provided a coding method including dividing an input signal into a plurality of divided signals, determining, based on the characteristics of each of the divided signals, which of the plurality of encoding methods should be used to encode each of the divided signals, encoding the divided signals using certain encoding methods and the formation of a bit stream based on encoded separated signals.

According to another aspect of the present invention, there is provided an encoding apparatus including a signal splitter module that splits an input signal into a plurality of split signals, an encoder determination module that determines, based on the characteristics of each of the split signals, which of the plurality of coding units should be used to encode each of the divided signals, a coding module that includes coding blocks and encodes the divided signals using certain blocks coding shafts, and a bit packing module that generates a bitstream based on encoded separated signals.

Benefits

Accordingly, it is possible to encode signals having different characteristics with an optimal bit rate by assigning said signals to one or more classes according to the characteristics of these signals and encoding each of the signals using the encoding unit that is best suited for the class to which the corresponding signal belongs. In addition, it is possible to efficiently encode various signals, including audio and speech signals.

BRIEF DESCRIPTION OF THE DRAWINGS

1 is a block diagram of an encoding device according to an embodiment of the present invention.

FIG. 2 is a block diagram of an embodiment of a classification module shown in FIG.

FIG. 3 is a block diagram of an embodiment of a pre-processing unit shown in FIG. 2.

4 is a block diagram of a device for calculating the perceptual entropy of an input signal according to an embodiment of the present invention.

FIG. 5 is a block diagram of another embodiment of a classification module shown in FIG.

6 is a block diagram of an embodiment of a signal splitter shown in FIG. 5.

FIG. 7 and FIG. 8 are diagrams for explaining methods for combining a plurality of separated signals according to embodiments of the present invention.

FIG. 9 is a block diagram of another embodiment of a signal separation unit shown in FIG.

10 is a diagram for explaining a method of dividing an input signal into a plurality of divided signals according to an embodiment of the present invention.

11 is a block diagram of an embodiment of the determination unit shown in FIG.

Fig. 12 is a block diagram of an embodiment of the coding unit of Fig. 1.

FIG. 13 is a block diagram of another embodiment of a coding unit shown in FIG.

14 is a block diagram of an encoding apparatus according to another embodiment of the present invention.

15 is a block diagram of a decoding apparatus according to an embodiment of the present invention.

FIG. 16 is a block diagram of an embodiment of a synthesis unit shown in FIG.

Preferred Embodiments

Hereinafter, the present invention will be more fully described according to the accompanying drawings, in which illustrative embodiments of the invention are presented.

1 is a block diagram of an encoding device according to an embodiment of the present invention. 1, an encoding device includes a classification module 100, an encoding module 200, and a bit packing module 300.

Encoding module 200 includes a plurality of first through mth encoding units 210 and 220 that perform various encoding methods.

The classification module 100 divides the input signal into a plurality of divided signals and maps to each of the divided signals one of the coding units 210 and 220, from the first to the mth. Some of the coding blocks 210 and 220, from the first to the mth, may be matched to two or more separated signals or not (not mapped) to any divided signal at all.

The classification module 100 may allocate the number of bits to encode each of the divided signals, or determine the order in which the divided signals should be encoded.

An encoding unit 200 encodes each of the divided signals using one of the first through mth encoding units 210 and 220 that is mapped to the corresponding divided signal. The classification module 100 analyzes the characteristics of each of the separated signals and selects one of the first to mth coding units 210 and 220, which, according to the analysis results, can most effectively encode each of the separated signals.

The encoding unit that can most effectively encode the split signal can be considered capable of achieving the highest compression efficiency.

For example, a split signal that can be easily modeled as a coefficient and a residual can be efficiently encoded by a speech encoder, and a split signal that cannot be easily modeled as a coefficient and a remainder can be effectively encoded by a sound encoder.

If the ratio of the residual power obtained by modeling the divided signal to the power of the divided signal is less than a predetermined threshold, then the divided signal can be considered a signal that can be easily modeled.

Since a split signal that detects high redundancy along the time axis can be well modeled using a linear prediction method in which the current signal is predicted based on a previous signal, it can be most efficiently encoded by a speech encoder that uses a linear coding method predictions.

The bit packing unit 300 generates a transmitted bitstream based on the encoded separated signals provided by the encoding unit 200 and additional encoding information related to the encoded divided signals. The bit packing module 300 may generate a bit stream having a variable bit rate using a simple bit method or a bit-partitioned arithmetic coding method.

Separated signals or frequency ranges that are not encoded due to bit rate limitations can be reconstructed from the decoded signals or frequency ranges provided by the decoder using an interpolation, extrapolation or duplication method. In addition, compensation information related to separated signals that are not encoded may be included in the transmitted bitstream.

1, classification module 110 may include a plurality of classification blocks 110 and 120, first to nth. Each of the classification blocks 110 and 120, from the first to the n-th, can divide the input signal into a plurality of divided signals, transform the domain of definition of the input signal, extract the characteristics of the input signal, classify the input signal according to the characteristics of the input signal, or map the input signal to one of the blocks 210 and 220 coding, from first to mth.

One of the classification blocks 110 and 120, first through nth, may be a preprocessing unit that performs an input signal preprocessing operation so that the input signal can be converted into a signal that can be effectively encoded. The pre-processing unit can divide the input signal into many components, for example, a coefficient component and a signal component, and can perform an input signal pre-processing operation before other classification units perform their operations.

The input signal can be pre-processed optionally according to the characteristics of the input signal, environmental factors and the target bit rate, and only some of the plurality of separated signals obtained from the input signal can be pre-processed.

The classification module 100 may classify the input signal according to perceptual information of the input signal provided by the psychoacoustic modeling module 400. Examples of perceptual characteristic information include a masking threshold, a signal-to-mask ratio (SMR), and perceptual entropy.

In other words, the classification module 100 may split the input signal into a plurality of split signals, or may map to each of the split signals one or more of the first to mth coding blocks 210 to 220, according to information of a perceptual characteristic of the input signal, for example, a masking threshold and SNR of the input signal.

In addition, the classification module 100 may receive information such as tonality, zero-crossing frequency (ZCR), and linear prediction coefficient of the input signal, and classification information of previous frames, and may classify the input signal according to the received information.

1, the encoded result information output by the encoding module 200 may be returned to the classification module 100.

After the input signal classification module 100 divides into a plurality of separated signals and determines which of the first to mth coding units 210 and 220, by how many bits and in what order the divided signals are to be encoded, the separated signals are encoded according to the determination results. The number of bits actually used to encode each of the divided signals need not be identical to the number of bits that are allocated by the classification module 100.

Information determining the difference between the actually used number of bits and the allocated number of bits can be returned to the classification module 100 so that the classification module 100 can increase the number of bits allocated to other separated signals. If the actual number of bits used is greater than the allocated number of bits, then the classification module 100 may reduce the number of bits allocated to other separated signals.

An encoding unit that actually encodes the divided signal does not need to be an encoding unit identical to that associated with the divided signal by the classification module 100. In this case, the information may be returned to the classification module 100 indicating that the encoding unit that actually encodes the divided signal is different from the encoding unit associated with the divided signal by the classification module 100. Further, the classification module 100 may map the encoding unit to the divided signal different from the encoding unit previously associated with the divided signal.

The classification module 100 may re-split the input signal into a plurality of split signals according to the encoded result information returned to it. In this case, the classification module 100 may receive a plurality of divided signals having a structure different from the structure of the previously obtained divided signals.

If the encoding operation selected by the classification module 100 is different from the actual encoding operation, then information related to the difference between them can be returned to the classification module 100 so that the classification module 100 can re-determine the information associated with the encoding operation.

FIG. 2 is a block diagram of an embodiment of a classification module 100 shown in FIG. 2, the first classification unit may be a preprocessing unit that performs an input signal preprocessing operation so that the input signal can be effectively encoded.

2, classification block 110 may include a plurality of first through n-th preprocessors 111 and 112 that perform various preprocessing methods. The first classification unit 110 may use one of the first through n-th preprocessors 111 and 112 to pre-process the input signal according to the characteristics of this input signal, environmental factors, and target bit rate. In addition, the first classification unit 110 may perform two or more input signal preprocessing operations using preprocessors 111 and 112, from the first to the n-th.

FIG. 3 is a block diagram of an embodiment of preprocessors 111 and 112, first through n-th, shown in FIG. 2. 3, the preprocessor includes a coefficient extractor 113 and a residue extractor 114.

The coefficient extractor 113 analyzes the input signal and extracts from the input signal a coefficient representing the characteristics of the input signal. The residue extractor 114 extracts a residue from the input signal with excess components removed from it using the extracted coefficient.

The preprocessor may perform a coding operation based on linear prediction of the input signal. In this case, the coefficient extractor 113 extracts the linear prediction coefficient from the input signal by performing a linear prediction analysis of the input signal, and the residue extractor 114 extracts the remainder from the input signal using the linear prediction coefficient provided by the coefficient extractor 113. The residue with redundancy removed from it may have a format identical to (format) white noise.

Hereinafter, a linear prediction analysis method according to an embodiment of the present invention will be described in detail.

The predicted signal obtained by linear prediction analysis may consist of a linear combination of previous input signals, as shown in equation (1):

MatFig. 1

where p denotes the linear prediction order, 1 to p denote the linear prediction coefficients that are obtained by minimizing the mean square error (MSE) between the input signal and the estimated signal.

The transfer function P (z) for analysis based on linear prediction can be represented by equation (2):

MatFig. 2

3, the preprocessor can extract a linear prediction coefficient and a remainder from an input signal using a warped linear prediction (WLPC) coding method, which is another type of linear prediction analysis. The WLPC method can be implemented by substituting a phase filter having a transfer function A (z), instead of delaying the block Z ^-1 . The transfer function A (z) can be represented by equation (3):

MatFig. 3

where denotes the phase coefficient. By changing the phase coefficient, you can change the resolution of the analyzed signal. For example, if the analyzed signal is a signal with a high concentration in a certain frequency range, for example, if the analyzed signal is an audio signal that is a signal with a high concentration in the low frequency range, then the analyzed signal can be effectively encoded by setting such a phase coefficient so that the resolution can be increased low-frequency signals.

In the WLPC method, low-frequency signals are analyzed with higher resolution than high-frequency signals. Accordingly, the WLPC method can achieve high prediction characteristics for low frequency signals and can better simulate low frequency signals.

The phase coefficient can vary along the time axis according to the characteristics of the input signal, environmental factors and the target bit rate. If the phase coefficient changes over time, then the audio signal obtained by decoding can be significantly distorted. Accordingly, when the phase coefficient changes, a smoothing method can be applied to the phase coefficient so that the phase coefficient can change gradually and that signal distortion can be minimized. A range of values that can be defined as the current value of the phase coefficient can be determined by previous phase coefficient values.

At the input, a masking threshold can be used instead of the original signal to estimate the linear prediction coefficient. More specifically, the masking threshold can be converted to a time-domain signal, and WLPC can be performed using a time-domain signal at the input. It is also possible to perform linear prediction coefficient prediction using an input remainder. In other words, linear prediction analysis can be performed several times, thereby obtaining a remainder that is increasingly approaching white noise.

2, the first classification unit 110 may include a first preprocessor 111 that performs linear prediction analysis described above according to equations (1) and (2), and a second preprocessor (not shown) that performs WLPC. The first classification unit 100 may select one of the first processor 111 and the second preprocessor, or may decide not to perform an analysis based on linear prediction of the input signal according to the characteristics of this input signal, environmental factors, and target bit rate.

If the phase coefficient value is 0, then the second preprocessor may be identical to the first preprocessor 111. In this case, the first classification unit 110 may include only the second preprocessor and select one of the linear prediction analysis method and the WLPC method according to the phase coefficient value. In addition, the first classification unit 110 may perform linear prediction analysis or that of a linear prediction analysis method and a WLPC method that is selected in the blocks of frames.

Information indicating whether to perform linear prediction analysis and information indicating which of the linear prediction analysis method and WLPC methods are selected can be included in the transmitted bitstream.

The bit packing unit 300 receives, from the first classification block 110, a linear prediction coefficient, information indicating whether to perform coding based on the linear prediction, and information identifying the linear prediction encoder that is actually used. Further, the bit packing module 300 inserts all the received information into the transmitted bitstream.

The number of bits required to encode an input signal into a signal having a sound quality almost indistinguishable from the quality of the original input signal can be determined by calculating the perceptual entropy of the input signal.

4 is a block diagram of an apparatus for calculating perceptual entropy according to an embodiment of the present invention. According to FIG. 4, this device includes a filter bank 115, a linear prediction block 116, a psychoacoustic modeling block 117, a first bit calculator 118 and a second bit calculator 119.

The perceptual entropy PE of the input signal can be calculated using equation (4):

MatFig. 4

where X (e ^jw ) denotes the power level of the original input signal and T (e ^jw ) denotes a masking threshold.

In the WLPC method, which involves the use of a phase filter, the perceptual entropy of the input signal can be calculated using the ratio of the remainder power of the input signal and the remainder masking threshold. More specifically, an encoding device that uses the WLPC method can calculate the perceptual entropy PE of the input signal using equation (5):

MatFig. 5

where R (e ^jw ) denotes the remainder power of the input signal, T (e ^jw ) denotes the residual masking threshold.

The masking threshold T (e ^jw ) can be represented by equation (6):

MatFig. 6

where T (e ^jw ) denotes the masking threshold of the original signal and H (e ^jw ) denotes the transfer function for the WLPC. The psychoacoustic modeling unit 320 can calculate the masking threshold T (e ^jw ) using the masking threshold T (e ^jw ) in the range of the scale factor and using the transfer function H (e ^jw ).

4, the first bit calculation unit 118 receives the remainder obtained by the WLPC, which is executed by the linear prediction unit 116, and the masking threshold output by the psychoacoustic modeling unit 117. A set of filters 116 may perform frequency conversion of the original signal, and the frequency conversion result may be input to the psychoacoustic modeling unit 117 and the second bit calculating unit 119. A set of filters 115 may perform Fourier transform of the original signal.

The first bit calculating unit 118 can calculate perceptual entropy using the ratio of the masking threshold of the original signal divided by the spectrum of the transfer function of the WLPC synthesis filter and the remainder power.

The deformed perceptual entropy of the WPE signal, which is divided into 60 or more unevenly divided ranges with different bandwidths, can be calculated using WLPC, as shown in equation (7):

MatFig. 7

where b is the index of the partition range obtained using the psychoacoustic model, e _res (b) is the sum of the powers of the residues in the partition range b, w _low (b) and w _high (b) respectively denote the lowest and highest frequencies in the partition range b , nb _linear (w) denotes the mask threshold of the linearly displayed partitioning range, h (w) ² denotes a linear prediction (LPC) frame coding spectrum, and nb _res (w) denotes the linear mask threshold corresponding to the remainder.

On the other hand, the deformed perceptual entropy of the WPE _sub signal, which is divided into 60 or more uniformly divided ranges with the same bandwidth, can be calculated using WLPC, as shown in equation (8):

MatFig. 8

where s denotes the index of the linearly divided subband, s _low (w) and s _high (w) respectively denote the lowest and highest frequencies in the linearly divided subband s, nb _sub (s) denotes the masking threshold of the linearly divided subband s and e _sub (s ) denotes the power of the linearly divided subband s, that is, the sum of the frequencies in the linearly divided subband s. The masking threshold nb _sub (s) is the minimum of the set of masking thresholds in the linearly divided subband s.

Perceptual entropy cannot be calculated for ranges with identical range widths and with thresholds higher than the sum of the input spectra. Accordingly, the deformed perceptual entropy of the WPE _{sub of} equation (8) may be lower than the deformed perceptual entropy of the WPE sub of equation (7), which provides high resolution for low-frequency ranges.

The WPE _sf deformed perceptual entropy can be calculated for scale factor bands with different bandwidths using WLPC, as shown in equation (9):

MatFig. 9

where f is the index of the scale factor range, nb _sf (f) is the minimum mask threshold of the scale factor range f, WPE _sf is the ratio of the input signal of the scale factor range f and the mask threshold of the scale factor range f, and e _sf (s) is the sum of all frequencies in range f of the scale factor, that is, the power of range f of the scale factor.

FIG. 5 is a block diagram of another embodiment of a classification module 100 shown in FIG. 5, the classification module includes a signal separation unit 121 and a determination unit 122.

More specifically, the signal splitting unit 121 splits an input signal into a plurality of divided signals. For example, signal splitter 121 may split an input signal into multiple frequency ranges using a subband filter. Frequency bands may have the same bandwidth or different bandwidths. As described above, the divided signal can be encoded separately from other separated signals by a coding unit that is best suited for the characteristics of the divided signal.

The signal splitter 121 may split the input signal into a plurality of split signals, for example a plurality of range signals, so that mutual interference between the range signals can be minimized. The signal splitting unit 121 may have a dual filter set structure. In this case, the signal separation unit 121 may also separate each of the divided signals.

Separation information related to split signals obtained by the signal split unit 121, for example, the total number of split signals, and range information of each of the split signals may be included in the transmitted bitstream. The decoding device can decode the divided signals separately and synthesize the decoded signals according to the separation information, thereby restoring the original input signal.

The partitioning information may be stored in a table. The bitstream may include identification information of a table used to split the original input signal.

The importance of each of the separated signals (for example, a plurality of frequency range signals) for sound quality can be determined, and for each of the divided signals, the bit rate can be set according to the results of the determination. More specifically, the importance of a split signal can be defined as a constant value or as a variable value that varies according to the characteristics of the input signal for each frame.

If the speech and audio signals are mixed in the input signal, the signal splitting unit 121 may separate the input signal into a speech signal and an audio signal according to the characteristics of the speech signals and the characteristics of the audio signals.

The determining unit 122 may determine which of the first to mth encoding units 210 and 220 in the encoding unit 200 can most effectively encode each of the divided signals.

The determination unit 122 attributes the divided signals to several groups. For example, the determination unit 122 may classify the divided signals as N classes and determine which of the first to mth coding units 210 and 220 should be used to encode each of the divided signals, with each of the N classes being assigned one of the blocks 210 and 220 coding, from first to mth.

More specifically, given that the encoding module 200 includes first-to-mth coding units 210 and 220, the determining unit 122 can classify the divided signals into first-to-mth classes that can be most efficiently encoded blocks 210 and 220 coding, from the first to the mth, respectively.

For this, it is possible to pre-determine the characteristics of the signals that can be most effectively encoded by each of the first to mth coding units 210 and 220, and the characteristics of the classes from first to mth can be determined according to the determination results. After that, the determining unit 122 can extract the characteristics of each of the divided signals and assign each of the divided signals to one of the classes from the first to the mth, which shares characteristics identical to the corresponding divided signal according to the extraction results.

Examples of classes 1 through m include a voiced speech class, an unvoiced speech class, a background noise class, a pause class, a tonal sound class, a non-tonal sound class, and a vocalized speech / sound mixture class.

The determination unit 122 may determine which of the first to mth encoding units 210 and 220 should be used to encode each of the separated signals according to perceptual characteristic information related to the separated signals provided by the psychoacoustic modeling module 400, for example, masking thresholds , SMR ratios, or perceptual entropy levels of separated signals.

The determining unit 122 may determine the number of bits for encoding each of the divided signals, or determine the order in which the divided signals are to be encoded according to perceptual characteristic information related to the divided signals.

Information obtained through determination made by determination block 122, for example, information indicating which of the first to mth coding blocks 210 and 220, and by how many bits each of the separated signals should be encoded, and information indicating the order, in which the separated signals are to be encoded, may be included in the transmitted bitstream.

6 is a block diagram of an embodiment of a signal separation unit 121 of FIG. 5. 6, the signal separation unit includes both a separation device 123 and a combining device 124.

Separator 123 may divide the input signal into a plurality of separated signals. The combiner 124 may combine separated signals having similar characteristics into a single signal. To this end, combiner 124 may include a set of synthesis filters.

For example, the separation device 123 may divide the input signal into 256 bands. Of these 256 ranges, those that have similar characteristics can be combined by combining device 124 into a single range.

7, a combiner 124 may combine a plurality of separated signals that are adjacent into a single combined signal. In this case, the combiner 124 may combine a plurality of adjacent separated signals into a single combined signal according to a predetermined rule without taking into account the characteristics of the adjacent separated signals.

Alternatively, as shown in FIG. 8, the combiner 124 may combine multiple separated signals having similar characteristics into a single combined signal, regardless of whether the separated signals are adjacent. In this case, the combiner 124 may combine a plurality of separated signals, which can be efficiently encoded by an identical coding unit, into a single combined signal.

FIG. 9 is a block diagram of another embodiment of a signal separation unit 121 of FIG. 5. 9, the signal separation unit includes a first separation device 125, a second separation device 126, and a third separation device 127.

More specifically, the signal splitting unit 121 may hierarchically split the input signal. For example, the input signal may be divided into two separated signals by the first separation device 125, one of these two divided signals may be divided into three divided signals by the second separation device 126, and one of these three divided signals may be divided into three divided signals by the third device separation 127. Consequently, the input signal can be divided into a total of six divided signals. The signal splitting unit 121 may hierarchically divide an input signal into a plurality of ranges with different range widths.

In the embodiment shown in FIG. 9, the input signal is divided according to a three-level hierarchy, but the present invention is not limited to this. In other words, the input signal can be divided into many divided signals according to a two-level or four-level hierarchy or hierarchy with a large number of levels.

One of the separation devices 125 to 127, first to third, in the signal separation unit 121 may divide the input signal into a plurality of time-domain signals.

10, an explanation is given of an embodiment for splitting an input signal into a plurality of split signals by a signal splitting unit 121.

Speech or sound signals are generally stationary for a short period of frame length. However, sometimes speech or sound signals may have non-stationary characteristics, for example, during a transition period.

In order to efficiently analyze non-stationary signals and increase the coding efficiency of such non-stationary signals, the encoding device according to the present embodiment may use the empirical decomposition method (EMD) or wavelet. In other words, the encoding device according to the present embodiment can analyze the characteristics of the input signal using a variable conversion function. For example, the signal splitting unit 121 may split the input signal into multiple variable bandwidth ranges using a variable band subband filtering method.

Hereinafter, a method for dividing an input signal into a plurality of separated signals by EMD will be described in detail.

In the EMD method, the input signal can be decomposed into one or more native view functions (IMF). The IMF function must satisfy the following conditions: the number of extrema should be equal to the number of transitions through zero or differ from it by at most one; and the average value of the envelope defined by local maxima and the envelope defined by local minima is zero.

The IMF presents a simple oscillating view, similar to a component in a simple harmonic function, thereby enabling the efficient decomposition of the input signal using the EMD method.

More specifically, to extract IMF from the input signal s (t), the upper envelope can be generated by connecting all the local extrema defined by the local maxima of the input signal s (t) using the spatial splines interpolation method, and the lower envelope can be generated by connecting all the local extrema defined by the local minima of the input signal s (t), using the interpolation method based on spatial splines. All values that the input signal s (t) can have can be between the upper envelope and the lower envelope.

After that, the average value m (t) of the upper envelope and the lower envelope can be calculated. After that, the first component h ₁ (t) can be calculated by subtracting the average value of m (t) from the input signal s (t), as shown in equation (10):

MatFig. 10

If the first component h ₁ (t) does not satisfy the above IMF conditions, then the first component h ₁ (t) can be determined to be identical to the input signal s (t), and the above operation can be repeated until the first IMF C ₁ is received (t) satisfying the above IMF conditions.

After receiving the first IMF C ₁ (t) obtained residue r ₁ (t) by subtracting the first IMF C ₁ (t), as shown in equation (11):

MatFig. 11

Thereafter, the aforementioned IMF extraction operation can be repeated using the remainder r ₁ (t) as a new input signal, thereby obtaining second IMF C ₂ (t) and the remainder r ₂ (t).

If the remainder r _n (t) obtained during the aforementioned IMF extraction operation has a constant value or is either a monotonically increasing function or a function with a single period with only one extremum or no extremum at all, then the aforementioned IMF extraction operation can be completed.

As a result of the above IMF extraction operation, the input signal s (t) can be represented by the sum of the set of IMF functions from C ₀ (t) to C _M (t) and the final remainder r _m (t), as shown in equation (12):

MatFig. 12

where M denotes the total number of extracted IMF functions. The final remainder r _m (t) may reflect the general characteristics of the input signal s (t).

10 depicts eleven IMF functions and the final remainder obtained by decomposing the original input signal using the EMD method. 10, the IMF obtained from the original input at an early stage of extracting IMF is higher than the frequency IMF obtained from the original input at a later stage of extracting IMF.

Extraction of IMF can be simplified by using the standard deviation SD between the previous remainder h _{1 (k-1)} and the current remainder h _1k , as shown in equation (13):

MatFig. 13

If the standard deviation SD is less than the reference value, for example 0.3, then the current remainder h _1k can be considered as IMF.

At the same time, the signal x (t) can be converted into an analytical signal by means of the Hilbert transform, as shown in equation (14):

MatFig. 14

where (t) is the instantaneous value, (t) is the instantaneous phase and H {} is the Hilbert transform.

As a result of the Hilbert transform, the input signal can be converted into an analytical signal consisting of the real part and the imaginary part.

Using the Hilbert transform to a signal with an average of 0, it is possible to obtain frequency components that can provide high resolution for both the time and frequency domains.

Hereinafter, it will be described in detail how the determination unit 122 shown in FIG. 4 determines which of the plurality of coding units should be used to encode each of the plurality of divided signals obtained by decomposing the input signal.

The determining unit 122 may determine which of the speech encoder and sound encoder can more efficiently encode each of the separated signals. In other words, the determination unit 122 may decide to encode the divided signals, which can be effectively encoded by the speech encoder, using that one of the first through mth encoding units 210 and 220, which is the speech encoder, and decide on the encoding of the separated signals that can be efficiently encoded by a sound encoder, using one of the first to mth coding blocks 210 and 220, which is a sound encoder.

Hereinafter, it will be described in detail how the determining unit 122 determines which of the speech encoder and the audio encoder can more efficiently encode the divided signal.

The determining unit 122 may measure variation in the divided signal and determine that the divided signal can be encoded by the speech encoder more efficiently than the audio encoder if the measurement result is larger than a predetermined reference value.

Alternatively, the determining unit 122 may measure a tonal component included in a specific part of the divided signal, and determine that this divided signal can be encoded by the audio encoder more efficiently than the speech encoder if the measurement result is larger than a predetermined reference value.

11 is a block diagram of an embodiment of a determination unit 122 of FIG. 5. 11, the determination unit includes a speech encoding / decoding unit 500, a first filter set 510, a second filter set 520, a determination unit 530, and a psychoacoustic modeling unit 540.

The determination unit shown in FIG. 11 can determine which of the speech encoder and audio encoder can more effectively encode each split signal.

11, the input signal is encoded by the speech encoding / decoding unit 500, and the encoded signal is decoded by the speech encoding / decoding unit 500, thereby restoring the original input signal. The speech encoding / decoding unit 500 may include an adaptive multi-speed wideband (AMR-WB) speech encoder / decoder, and the AMR-WB speech encoder / decoder may have a code-excited linear prediction (CELP) structure.

The input signal may be downsampled before being input to speech coding / decoding unit 500. The signal outputted by the speech encoding / decoding unit 500 can be upsampled, thereby restoring the input signal.

The input signal may be frequency converted by a first set of filters 510.

The signal output by the speech encoding / decoding unit 500 is converted into a frequency domain signal by a second set of filters 520. The first filter set 510 or the second filter set 520 can perform a cosine transform, for example a modified discrete transform (MDCT), of the signal introduced into them.

The frequency component of the original input signal output by the first filter set 510 and the frequency component of the restored input signal output by the second filter set 520 are both input to the determination unit 530. The determining unit 530 may determine which of the speech encoder and sound encoder can more efficiently encode the input signal based on the frequency components inputted thereto.

More specifically, the determining unit 530 can determine which of the speech encoder and sound encoder can more efficiently encode the input signal based on the frequency components introduced into it, by calculating the perceptual entropy PE of each of the frequency components using equation (15):

MatFig. 15

Where

where x (j) denotes the coefficient of the frequency component, j denotes the index of the frequency component, denotes the size of the quantization step, nint () is a function that returns the integer closest to its argument, j _{low (i)} and j _{high (i)} are respectively the index of the initial frequency and the index of the final frequency of the range of the scale factor.

The determination unit 530 using equation (15) can calculate the perceptual entropy of the frequency component of the original input signal and the perceptual entropy of the frequency component of the reconstructed input signal, and based on the calculation results, determine which of the audio encoder and speech encoder is more effective for use in encoding the input signal.

For example, if the perceptual entropy of the frequency component of the original input signal is less than the perceptual entropy of the frequency component of the reconstructed input signal, the determining unit 530 can determine that the input signal can be encoded by the audio encoder more efficiently than the speech encoder. On the other hand, if the perceptual entropy of the frequency component of the reconstructed input signal is less than the perceptual entropy of the frequency component of the original input signal, then the determining unit 530 can determine that the input signal can be encoded by the speech encoder more efficiently than the audio encoder.

FIG. 12 is a block diagram of an embodiment of one of the first through mth coding units 210 and 220 shown in FIG. The coding unit shown in FIG. 12 may be a speech encoder.

In general, speech encoders can perform an LPC of an input signal in blocks of frames and extract an LPC coefficient, such as a 16th order LPC coefficient, from each frame of the input signal using the Levinson-Durbin algorithm. The excitation signal can be quantized through a search in the adaptive codebook or a search in a fixed codebook. The excitation signal can be quantized using a linear prediction method with algebraic code excitation. Vector quantization can be performed on the gain of the excitation signal using a quantization table having a conjugate structure.

The speech encoder of FIG. 12 includes a linear prediction analysis unit 600, a pitch estimation unit 610, a codebook search unit 620, a linear spectral pair (LSP) unit 630, and a quantization unit 640.

A linear prediction analysis unit 600 performs linear prediction analysis of the input signal using an autocorrelation coefficient that is obtained using an asymmetric window. If the predictive estimation time, that is, the asymmetric window, is 30 ms, then the linear prediction analysis unit 600 may perform linear prediction analysis using the predictive estimation time of 5 ms.

The autocorrelation coefficient is converted to a linear prediction coefficient using the Levinson-Durbin algorithm. For quantization and linear interpolation, the LSP unit 630 converts the linear prediction coefficient into an LSP. Quantization block 640 quantizes the LSP.

The pitch estimation unit 610 estimates the pitch (in the circuit) with an open loop to reduce the complexity of the search in the adaptive codebook. More specifically, the pitch estimator 610 estimates the pitch period (in the circuit) with an open loop using the weighted speech area of each frame. After that, a noise filter harmonics filter is formed using the estimated pitch (in the circuit) with an open loop. After that, the impulse response is calculated using a noise filter harmonics filter, a linear prediction synthesis filter and a formant perceptual weighting filter. The impulse response can be used to generate the target signal to quantize the excitation signal.

The codebook search unit 620 searches the adaptive codebook and searches the fixed codebook. Search in the adaptive codebook can be performed in blocks of subframes by computing the adaptive codebook vector by searching for the pitch (in the circuit) with a closed loop and through interpolation of previous excitation signals. Adaptive codebook parameters may include the pitch period and the gain of the pitch filter. The excitation signal can be generated by a linear prediction synthesizing filter to simplify the search for a closed loop.

The fixed codebook structure is established based on the intermittent single pulse permutation model (ISSP). A codebook vector containing 64 positions, in which 64 pulses are respectively located, is divided into four tracks, each track containing 16 positions. In each of the four tracks, according to the transmission speed, a predetermined number of pulses can be found. Since the codebook index indicates the position of the track and the sign of the pulse, there is no need to save the codebook, and the excitation signal can be generated simply using the codebook index.

The speech encoder depicted in FIG. 12 may perform the aforementioned time-domain coding methods. In addition, if the input signal is encoded using the linear prediction coding method by the classification module 100 shown in FIG. 1, then the linear prediction analysis unit 600 may be optional.

The present invention is not limited to the speech encoder shown in FIG. In other words, in addition to the speech encoder shown in FIG. 12, various speech encoders that can efficiently encode speech signals can be used within the scope of the present invention.

FIG. 13 is a block diagram of another embodiment of one of the first through mth coding units 210 and 220 shown in FIG. The coding unit shown in FIG. 13 may be a sound encoder.

13, the sound encoder includes a set of filters 700, a psychoacoustic modeling unit 710, and a quantization unit 720.

A set of 700 filters converts the input signal into a frequency domain signal. A set of filters 700 may perform a cosine transform, for example a modified discrete transform (MDCT) of the input signal.

A psychoacoustic simulation unit 710 calculates a masking threshold for the input signal or SMR of the input signal. The quantization unit 720 quantizes the MDCT coefficients output by the filter bank 700 using the masking threshold calculated by the psychoacoustic modeling unit 710. Alternatively, to minimize audible distortion within a given range of bit rate, quantization unit 720 may use an input signal SMR.

The sound encoder depicted in FIG. 13 may perform the aforementioned coding methods in the frequency domain.

The present invention is not limited to the sound encoder shown in FIG. 13. In other words, in addition to the audio encoder shown in FIG. 13, various sound encoders (eg, perspective audio encoders) that can efficiently encode audio signals can be used within the scope of the present invention.

Promising audio encoders perform temporal noise limitation (TNS), intensity / communication, prediction and coding of stereo sound with modulation of the carrier and subcarrier, respectively, the sum and difference of the signals of the left and right channels (mid / side, M / S). TNS is the operation of the corresponding distribution of time-domain quantization noise in a filter set window so that the quantization noise becomes inaudible. Intensity / communication is an operation that can reduce the amount of spatial information transmitted by encoding an audio signal and transmitting the power of the audio signal only on the basis that the perception of the direction of sound in the high frequency range depends mainly on the time scale of the power.

Prediction is the operation of removing redundancy from a signal whose statistical characteristics do not change when using correlation between the components of the frame spectrum. M / S stereo audio coding is the operation of transmitting the normalized sum (i.e. middle) and the difference (i.e. side) of the stereo signal instead of the left and right channel signals.

A signal that has undergone TNS operations, intensity / communication, M / S stereo sound prediction and encoding is quantized by a quantizer that performs synthesis through analysis (AbS) using the SMR obtained from the psychoacoustic model.

As described above, since the audio encoder encodes the input signal using a modeling method, for example, a linear prediction encoding method, the determination unit 122 shown in FIG. 5 can determine whether the input signal can be easily modeled according to a predetermined list of rules. After that, if it is determined that the input signal can be easily modeled, then the determining unit 122 may decide to encode the input signal using a speech encoder. On the other hand, if it is determined that the input signal cannot be easily modeled, then the determining unit 122 may decide to encode the input signal using a sound encoder.

14 is a block diagram of an encoding device according to another embodiment of the present invention. 1 to 14, like reference numbers represent like elements, and accordingly, detailed descriptions thereof will be omitted.

According to FIG. 14, the classification module 100 divides the input signal into a plurality of divided signals, first through n-th, and determines which of the plurality of coding units 230, 240, 250, 260, and 270 should be used to encode each of the divided signals, c first to nth.

According to FIG. 14, coding units 230, 240, 250, 260, and 270 can sequentially encode the separated signals, from first to nth, respectively. In addition, if the input signal is divided into a plurality of frequency range signals, then the frequency range signals can be encoded in order from the signal of the lowest frequency range to the signal of the highest frequency range.

If the separated signals are encoded sequentially, then to encode the current signal, you can use the encoding error of the previous signal. As a result, split signals can be encoded using various coding methods and, accordingly, signal distortion can be prevented and the frequency range scalable.

14, an encoding unit 230 encodes a first divided signal, decodes an encoded first divided signal, and outputs to the encoding unit 240 an error between the decoded signal and the first divided signal. Block 240 coding encodes the second split signal using the error output by block 230 coding. Therefore, the second to mth separated signals are encoded taking into account the coding errors of the corresponding previous separated signals. Therefore, it is possible to perform error-free coding and increase the sound quality.

The encoding device depicted in FIG. 14 may recover a signal from an input bitstream by performing the operations of the encoding device shown in FIG. 1 to FIG. 14 in the reverse order.

15 is a block diagram of a decoding apparatus according to an embodiment of the present invention. 15, a decoding apparatus includes a bit decompression module 800, a decoder determination module 810, a decoding module 820, and a synthesis module 830.

The bit decompression unit 800 extracts, from the input bitstream, one or more encoded signals and additional information necessary for decoding the encoded signals.

Decoding module 820 includes a plurality of first through mth decoding units 821 and 822 that perform various decoding methods.

The decoder determination module 810 determines which of the first to mth decoding units 821 and 822 can decode each of the encoded signals most efficiently. The decoder determination module 810 may use a method similar to that of the classification module 100 shown in FIG. 1 to determine which of the first to mth decoding units 821 and 822 can decode each of the encoded signals most efficiently. In other words, the decoder determination module 810 can determine which of the first through mth decoding units 821 and 822 can decode each of the encoded signals most efficiently based on the characteristics of each of the encoded signals. Preferably, the decoder determination module 810 can determine which of the first through mth decoding units 821 and 822 can decode each of the encoded signals most efficiently based on additional information extracted from the input bitstream.

Additional information may include class information identifying the class to which the encoded signal is assigned by the encoding device, information on the encoding unit identifying the encoding unit used to generate the encoded signal, and information on the decoding unit identifying the decoding unit to be used to decode the encoded signal.

For example, the decoder determination module 810, based on the additional information, can determine which class the encoded signal belongs to and select, for the encoded signal, one of the first through mth decoding units 821 and 822 that corresponds to the encoded signal class. In this case, the selected decoding unit may have such a structure that it can most efficiently decode signals belonging to a class that is identical to the class of the encoded signal.

Alternatively, the decoder determination module 810 may, based on additional information, identify the encoding unit used to generate the encoded signal, and select, for the encoded signal, that one of the first through mth decoding units 821 and 822 that corresponds to the identified encoding unit . For example, if the encoded signal was generated by a speech encoder, then the decoder determination module 810 may select, for the encoded signal, that one of the first through mth decoding units 821 and 822, which is the speech decoder.

Alternatively, the decoder determination module 810 may, based on additional information, identify a decoding unit that can decode the encoded signal, and select, for the encoded signal, that one of the first through mth decoding units 821 and 822 that corresponds to the identified decoding unit .

Alternatively, the decoder determination module 810 may obtain the characteristics of the encoded signal from additional information and select one of the first through mth decoding units 821 and 822 that can most efficiently decode signals whose characteristics are identical to the characteristics of the encoded signal.

Therefore, each of the encoded signals extracted from the input bitstream is encoded by that of the first through mth decoding units 821 and 822, with respect to which it is determined that it can most effectively decode the corresponding encoded signal. The decoded signals are synthesized by the synthesis module 830, thereby restoring the original signal.

The bit decompression unit 800 extracts separation information related to the encoded signals, for example, the number of encoded signals, and range information of each of the encoded signals, and the synthesis module 830 can synthesize the decoded signals provided by the decoding module 820 according to the separation information.

Synthesizer 830 may include a plurality of first to nth synthesizer 831 and 832. Each of the first to nth synthesizing units 831 and 832 can synthesize the decoded signals provided by the decoding unit 820, or perform area conversion or additional decoding of some or all of the decoded signals.

One of the synthesis blocks 831 and 832, from the first to the nth, may perform a post-processing operation of the synthesized signal, which is an operation inverse to the pre-processing operation performed by the encoding device. Information indicating whether to perform the post-processing operation and decoding information used to perform the post-processing operation can be extracted from the input bitstream.

According to FIG. 16, one of the first to nth synthesis blocks 831 and 832, in particular the second synthesis block 833, may include a plurality of first to nth postprocessors 834 and 835. The first synthesizing unit 831 synthesizes a plurality of decoded signals into a single signal, and one of the postprocessors 834 and 835, from the first to the n-th, performs the post-processing operation of a single signal obtained by the synthesis.

Information indicating which of the postprocessors 834 and 835, from the first to the n-th, should perform the post-processing operation of a single signal obtained by synthesis, can be included in the input bit stream.

One of the synthesizers 831 and 832, from the first to the n-th, can perform decoding based on linear prediction of a single signal obtained by synthesis using a linear prediction coefficient extracted from the input bit stream, thereby restoring the original signal.

The present invention can be implemented in the form of a machine-readable code recorded on a computer-readable storage medium. A computer-readable storage medium may be any type of recording device on which data is stored in a computer-readable manner. Examples of computer-readable storage media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical storage media, and a carrier (e.g., data transmission over the Internet). A computer-readable storage medium may be distributed across a plurality of computer systems connected to the network so that the computer-readable code is written to and executed from it in a decentralized manner. One of ordinary skill in the art can easily review the functional programs, code, and code segments needed to implement the present invention.

Although the present invention has, in particular, been shown and described in relation to its illustrative embodiments, those skilled in the art will understand that various changes in form and detail can be made therein without departing from the spirit and scope of the present inventions as defined by the following claims.

Industrial applicability

As described above, according to the present invention, it is possible to encode signals having different characteristics with an optimal bit rate by adding said signals to one or more classes according to the characteristics of these signals and encoding each of the signals using the encoding unit that is best suited for the class, to which the corresponding signal belongs. Therefore, it is possible to efficiently encode various signals, including audio and speech signals.

Claims (16)

1. A decoding method comprising:
receiving a signal including at least one of an audio signal and a speech signal, the signal being divided into multiple frames,
extracting decoding method information related to how to decode a signal in each frame,
determining a decoding method in each frame based on information about the decoding method and
decoding a signal using a specific decoding method in each frame. 2. The decoding method according to claim 1, wherein the information on the decoding method comprises at least one of information about a coding unit identifying an encoding unit that generated the encoded signal, information about a decoding unit identifying a decoding unit that decodes the encoded signal , and information indicating the characteristic of the encoded signal. 3. The decoding method according to claim 1, in which the determination comprises the choice of the decoding method that most effectively decodes each of the encoded signals. 4. The decoding method according to claim 1, further comprising extracting signal separation information from the input bitstream,
in which the signal frame is determined based on the separation information in the time domain. 5. The decoding method according to claim 1, further comprising extracting information about the number of bits of the signal from the input bit stream,
in which the decoding comprises decoding the signal according to information about the number of bits. 6. The decoding method according to claim 1, further comprising extracting information about the decoding order of the signal from the input bit stream,
wherein the decoding comprises decoding a signal according to decoding order information. 7. A decoding device comprising:
a bit decompression unit, which receives a signal including at least one of an audio signal and a speech signal, the signal being divided into a plurality of frames, and extracts decoding method information related to how to decode the signal in each frame,
a decoder determination module that determines a decoding method in each frame based on information on a decoding method, and
a decoding module that comprises a plurality of decoding blocks, each of the decoding blocks decoding a signal using a specific decoding method in each frame. 8. The decoding device according to claim 7, in which the decoder determination module selects a decoding unit that most effectively decodes the encoded signals in each frame. 9. The decoding apparatus of claim 7, wherein the bit decompression unit further extracts signal separation information from the input bit stream,
in which the signal frame is determined based on the separation information in the time domain. 10. An encoding method comprising:
receiving an input signal including at least one of an audio signal and a speech signal;
determination of the encoding method of the input signal in accordance with the characteristics of the input signal in each frame,
encoding an input signal using a speech encoding method or an audio encoding method based on a specific encoding method corresponding to an input signal in each frame; and
generating a bitstream including an encoded input signal,
moreover, the method of speech coding uses analysis based on linear prediction coding (LPC), and the method of sound coding uses psychoacoustic modeling and a set of filters. 11. The encoding method according to claim 10, in which the determination comprises the selection of the encoding method that most effectively encodes the divided signals in each frame. 12. The encoding method of claim 10, further comprising allocating the number of bits for encoding the signal. 13. The encoding method of claim 10, further comprising determining the order in which multiple signals in each frame should be encoded. 14. An encoding device comprising:
a bit decompression unit that receives an input signal including at least one of an audio signal and a speech signal;
an encoder determination module that determines an encoding method of the input signal in accordance with the characteristics of the input signal in each frame;
an encoding module encoding an input signal based on a specific encoding method corresponding to an input signal in each frame, comprising:
a speech coding unit encoding an input signal using linear prediction coding (LPC) analysis, and
an audio coding module encoding an input signal using psychoacoustic modeling and a set of filters, and
a bit packing module that generates a bitstream using an encoded input signal. 15. The encoding device according to 14, in which the encoder determination module selects the encoding unit that encodes the signal most efficiently. 16. A computer-readable storage medium containing a program for executing a decoding method according to any one of claims 1 to 6 of the claims.

Images