CN106205626B - A kind of compensation coding and decoding device and method for the subspace component being rejected - Google Patents

Abstract

The present invention provides a kind of compensation coding and decoding devices and method for the subspace component being rejected, wherein, compensating code device includes basic compensation vector generation module, basis compensation set generation module, compensation coding parameter coding module and code stream multiplex module, and provides corresponding compensation coding and execute step；Compensating decoding apparatus includes basic compensation vector generation module, basis compensation set generation module, code stream demultiplexing module, compensation coding parameter decoder module and subspace component compensation module, and provides corresponding compensation decoding and execute step.The compensation coding and decoding device and method can carry out effective compensation to the subspace component being rejected, and effectively improve signal reconstruction quality, and the audio power after reconstruction is also close with primary energy.

Description

Compensation coding and decoding device and method for discarded subspace component

Technical Field

The present invention relates to a compensation encoding and decoding device and method, and more particularly, to a compensation encoding and decoding device and method for discarded subspace components.

Background

With the development of technology, a variety of coding techniques for sound signals have appeared, and the sound signals are generally digital sounds including signals perceivable to human ears, such as speech, music, natural sounds, and artificially synthesized sounds. At present, many sound encoding technologies have become industrial standards to be applied in a large amount and are incorporated into people's daily lives, and the common sound encoding technologies include AC-3 of dolby laboratories, DTS of digital cinema systems, MP3 and AAC of Moving Picture Experts Group (MPEG) organization, WMA of microsoft corporation, and ATRAC of sony corporation.

In order to reproduce sound effects of Stereo, a multichannel sound signal is played to a user by using a plurality of channels, and a coding method of the multichannel sound signal is also evolving from waveform coding techniques such as sum and difference Stereo (M/S Stereo) and Intensity Stereo (Intensity Stereo) represented by AC-3 and MP3 to Parametric Stereo (Parametric Stereo) and Parametric Surround (Parametric Surround) represented by MP3Pro, ITU EAAC +, mpeg Surround, Dolby DD +.

However, both the conventional PS technique and the MPEG Surround technique rely heavily on binaural psychoacoustic characteristics, and ignore statistical characteristics of the multi-channel sound signal itself. For example, neither conventional PS techniques nor MPEG Surround techniques take advantage of statistical redundancy between channel pairs. Moreover, when the MPEG Surround adopts residual information coding, statistical redundancy still exists between the sum channel signal and the residual channel signal, so that both coding efficiency and quality of the coded signal cannot be considered. Patent applications 201410395806.5 and 201410404895.5 propose methods for multi-channel audio encoding and decoding based on subspace mapping (principal component analysis (PCA), Independent Component Analysis (ICA), and other methods), which effectively remove redundancy through subspace mapping, take account of statistical characteristics and psychoacoustic characteristics of multi-channel audio signals, and improve encoding efficiency. The method performs perceptual coding by selecting a plurality of important items in the mapped subspace components (the number of K), for example, a plurality of principal component components (the number of p) are coded, and the rest subspace components (the number of K-p) are discarded, so that the effect of reducing the dimension is achieved. Such a dimension reduction process sometimes causes a difference in reconstruction quality between multi-channel audios, and if a certain channel audio is highly correlated with a discarded subspace component (a subspace component that is not perceptually encoded), the reconstruction quality of the channel is significantly deteriorated, and the energy of the reconstructed audio is also significantly lower than the original energy. How to ensure that the reconstruction quality of each channel cannot be obviously reduced when dimension reduction processing is carried out through subspace mapping becomes a key problem.

Disclosure of Invention

The invention aims to solve the technical problem that the signal reconstruction quality is poor after the dimension reduction processing by adopting the abandoning method, and the reconstructed audio energy is obviously lower than the original energy.

In order to solve the above technical problem, the present invention provides a compensation encoding apparatus for discarded subspace components, comprising:

the basic compensation vector generation module is used for generating a fixed compensation vector as a basic compensation vector according to a perceptually coded signal model, or generating a self-adaptive compensation vector as a basic compensation vector according to a coded principal component copy in a current frame;

the basic compensation set generation module is used for placing each basic compensation vector in the basic compensation vector set and generating a one-to-one corresponding sequence number or characteristic parameter for each basic compensation vector;

and the compensation coding parameter coding module is used for selecting basic compensation vectors for compensation according to the compensation requirement, and performing quantization coding by taking the serial numbers or the characteristic parameters of the basic compensation vectors as compensation coding parameters to obtain a compensation coding code stream.

A basic compensation vector generating module is adopted to generate a basic compensation vector for compensation, the basic compensation vector can be used for performing substitution compensation on the abandoned subspace component, the signal reconstruction quality is effectively improved, and the reconstructed audio energy is similar to the original energy; the basic compensation set generation module is adopted to place the basic compensation vectors in the basic compensation vector set, and corresponding serial numbers or characteristic parameters are generated for the basic compensation vectors, so that the basic compensation vectors in the basic compensation vector set can be called conveniently; and the compensation coding parameter coding module is adopted to carry out quantization coding on the compensation coding parameters, so that the compensation coding parameters are conveniently stored or forwarded.

As a further improvement of the encoding apparatus of the present invention, the encoding apparatus further includes the following modules:

and the code stream multiplexing module is used for storing the compensation coding code stream as a code stream file or directly sending the compensation coding code stream to a communication channel.

The code stream multiplexing module can store the compensation coding code stream as a code stream file or directly send the compensation coding code stream to a communication channel, so that a decoding device can conveniently read or receive the compensation coding code stream, and compensation coding parameters can be multiplexed in real time.

As a further limiting solution of the encoding apparatus of the present invention, the basic compensation vector set includes a fixed basic compensation vector set and/or an adaptive basic compensation vector set; the fixed compensation vector is placed in the fixed basic compensation vector set; the adaptive compensation vector is placed in the set of adaptive base compensation vectors. The fixed compensation vector and the self-adaptive compensation vector are respectively placed in the fixed basic compensation vector set and the self-adaptive basic compensation vector set, so that the fixed compensation vector and the self-adaptive compensation vector are conveniently and quickly called, and the compensation efficiency is improved.

As a further limiting scheme of the encoding device of the present invention, the fixed compensation vector is generated according to one or more of a sinusoidal signal model, a random noise signal model and a HILN signal model, and does not change following the change of each frame of encoded data; the adaptive compensation vector changes following the change in the encoded data for each frame. A known signal model is adopted to generate a fixed compensation vector which is fixed and unchangeable, and the fixed compensation vector does not change along with the change of frame coding data, so that the reliability is higher; the adaptive compensation vector can change along with the change of each frame of coded data in real time, and has high real-time performance.

The invention also provides a compensation coding method aiming at the discarded subspace component, which comprises the following steps:

step 1, generating a fixed compensation vector as a basic compensation vector according to a perceptually encoded signal model, or generating a self-adaptive compensation vector as a basic compensation vector according to a copy of a coded principal component in a current frame;

step 2, placing each basic compensation vector in a basic compensation vector set, and generating a one-to-one corresponding serial number or characteristic parameter for each basic compensation vector;

and 3, selecting basic compensation vectors for compensation according to compensation requirements, and carrying out quantization coding by taking the serial numbers or the characteristic parameters of the compensation vectors as compensation coding parameters to obtain compensation coding code streams.

As a further definition of the compensation coding method of the present invention, the method further comprises the following steps:

and 4, storing the compensation coding code stream as a code stream file or directly sending the compensation coding code stream to a communication channel.

As a further limiting scheme of the compensation coding method of the present invention, the basic compensation vector set comprises a fixed basic compensation vector set and/or an adaptive basic compensation vector set; the fixed compensation vector is placed in the fixed basic compensation vector set; the adaptive compensation vector is placed in the set of adaptive base compensation vectors.

As a further limiting scheme of the compensation coding method of the present invention, the fixed compensation vector is generated according to one or more of a sinusoidal signal model, a random noise signal model and a HILN signal model, and does not change following the change of each frame of coded data; the adaptive compensation vector changes following the change in the encoded data for each frame.

The invention also provides a compensation decoding device for the discarded subspace component, which comprises the following modules:

the basic compensation vector generation module is used for generating a fixed compensation vector as a basic compensation vector according to the perceptually decoded signal model or generating a self-adaptive compensation vector as a basic compensation vector according to the decoded principal component copy in the current frame;

the basic compensation set generation module is used for placing each basic compensation vector in the basic compensation vector set and generating a one-to-one corresponding sequence number or characteristic parameter for each basic compensation vector;

the code stream demultiplexing module is used for reading a code stream file or directly reading a compensation coding code stream from a communication channel;

the compensation coding parameter decoding module is used for decoding the compensation coding code stream to obtain the serial number or the characteristic parameter of each basic compensation vector for compensation;

and the subspace component compensation module is used for selecting corresponding basic compensation vectors from the basic compensation vector set according to the sequence numbers or the characteristic parameters, and then replacing the discarded subspace components with the selected basic compensation vectors.

A basic compensation vector generating module is adopted to generate a basic compensation vector for compensation, the basic compensation vector is used for performing substitution compensation on the abandoned subspace component, the signal reconstruction quality is effectively improved, and the reconstructed audio energy is similar to the original energy; the basic compensation set generation module is adopted to place the basic compensation vectors in the basic compensation vector set, and generate corresponding serial numbers or characteristic parameters for each basic compensation vector, so that the basic compensation vectors in the basic compensation vector set can be conveniently called for decoding compensation; decoding the compensation coding parameters by adopting a compensation coding parameter decoding module so as to obtain the compensation coding parameters; and selecting a corresponding basic compensation vector from the basic compensation vector set by adopting a subspace component compensation module according to the compensation coding parameters, thereby further carrying out substitution compensation on the abandoned subspace component.

As a further limiting solution of the compensation decoding apparatus of the present invention, the set of basic compensation vectors includes a set of fixed basic compensation vectors and/or a set of adaptive basic compensation vectors; the fixed compensation vector is placed in the fixed basic compensation vector set; the adaptive compensation vector is placed in the set of adaptive base compensation vectors. The fixed compensation vector and the self-adaptive compensation vector are respectively placed in the fixed basic compensation vector set and the self-adaptive basic compensation vector set, so that the fixed compensation vector and the self-adaptive compensation vector are conveniently and quickly called, and the compensation efficiency is improved.

As a further limiting scheme of the compensation decoding device of the present invention, the fixed compensation vector is generated according to one or more of a sinusoidal signal model, a random noise signal model, and a HILN signal model, and does not change following the change of each frame of decoded data; the adaptive compensation vector changes following the change of the decoded data per frame. A known signal model is adopted to generate a fixed compensation vector which is fixed and unchangeable, and the fixed compensation vector does not change along with the change of frame decoding data, so that the reliability is higher; the adaptive compensation vector can change along with the change of each frame of decoded data in real time, and has high real-time performance.

The invention also provides a compensation decoding method aiming at the discarded subspace component, which comprises the following steps:

step 1, generating a fixed compensation vector as a basic compensation vector according to a signal model which is sensed and decoded, or generating a self-adaptive compensation vector as a basic compensation vector according to a main component copy which is decoded in a current frame;

step 2, placing each basic compensation vector in a basic compensation vector set, and generating a one-to-one corresponding serial number or characteristic parameter for each basic compensation vector;

step 3, reading the code stream file or directly reading the compensation code stream from the communication channel;

step 4, decoding the compensation coding code stream to obtain the serial numbers or characteristic parameters of all basic compensation vectors for compensation;

and 5, selecting corresponding basic compensation vectors from the basic compensation vector set according to the sequence numbers or the characteristic parameters, and replacing the discarded subspace components with the selected basic compensation vectors.

As a further limiting scheme of the compensation decoding method of the present invention, the basic compensation vector set comprises a fixed basic compensation vector set and/or an adaptive basic compensation vector set; the fixed compensation vector is placed in the fixed basic compensation vector set; the adaptive compensation vector is placed in the set of adaptive base compensation vectors.

As a further limiting scheme of the compensation decoding method of the present invention, the fixed compensation vector is generated according to one or more of a sinusoidal signal model, a random noise signal model, and a HILN signal model, and does not change following the change of each frame of decoded data; the adaptive compensation vector changes following the change of the decoded data per frame.

The invention has the beneficial effects that: (1) a basic compensation vector generating module is adopted to generate a basic compensation vector for compensation, the basic compensation vector is used for performing substitution compensation on the abandoned subspace component, the signal reconstruction quality is effectively improved, and the reconstructed audio energy is similar to the original energy; (2) the basic compensation set generation module is adopted to place the basic compensation vectors in the basic compensation vector set, and generate corresponding serial numbers or characteristic parameters for each basic compensation vector, so that the basic compensation vectors in the basic compensation vector set can be conveniently called for decoding compensation; (3) a compensation coding parameter coding module is adopted in the coding device to carry out quantization coding on the compensation coding parameters, so that the compensation coding parameters are conveniently stored or forwarded; (4) decoding the compensation coding parameters by adopting a compensation coding parameter decoding module in a decoding device so as to obtain the compensation coding parameters; (5) in the decoding device, a subspace component compensation module is adopted to select a corresponding basic compensation vector from a basic compensation vector set according to the compensation coding parameters, so that the discarded subspace component is further subjected to substitution compensation.

Drawings

FIG. 1 is a flow chart of a conventional multi-channel audio signal encoding method;

FIG. 2 is a schematic structural diagram of a compensation encoding apparatus according to the present invention;

FIG. 3 is a schematic flow chart illustrating a compensation application of the compensation encoding apparatus of the present invention to a multi-channel sound signal encoding method;

FIG. 4 is a flowchart illustrating the step 207 of performing the present invention;

FIG. 5 is a schematic diagram of a compensation decoding apparatus according to the present invention;

FIG. 6 is a schematic flow chart illustrating a compensation application of the multi-channel sound signal decoding method using the compensation decoding apparatus of the present invention;

FIG. 7 is a flowchart illustrating the step 303 according to the present invention;

FIG. 8 is a schematic flow chart of generating a fixed basis compensation vector set A1 according to the present invention;

FIG. 9 is a schematic flow chart of generating an adaptive basic compensation vector set A2 according to the present invention;

FIG. 10 is a flow chart illustrating a method of encoding a PCA principal component with a compensation process according to the present invention;

FIG. 11 is a flow chart illustrating a method of decoding PCA principal component compensation processing according to the present invention;

FIG. 12 is a schematic flowchart of a method for encoding PCA principal component by compensation processing using model parameters as compensation encoding parameters according to the present invention;

FIG. 13 is a schematic flowchart of a method for decoding PCA principal component by compensation processing using model parameters as compensation encoding parameters according to the present invention;

FIG. 14 is a flowchart illustrating a method for performing compensation processing coding on PCA principal component without fixed basis compensation vector set storage according to the present invention;

FIG. 15 is a flowchart illustrating a method for performing compensation processing decoding on PCA principal component without fixed basis compensation vector set storage according to the present invention;

FIG. 16 is a flow chart illustrating a method of encoding PCA principal component compensation processing of a vector combination according to the present invention;

FIG. 17 is a flowchart illustrating a method for decoding PCA principal component compensation processing of a vector combination according to the present invention;

FIG. 18 is a flowchart illustrating a method for adjusting mapping matrix coefficients to perform compensation processing coding on PCA principal component according to the present invention;

FIG. 19 is a block diagram of an apparatus for efficient multi-channel audio coding based on subspace mapping according to the present invention;

FIG. 20 is a block diagram of an apparatus for efficient multi-channel audio decoding based on subspace mapping according to the present invention.

Detailed Description

As shown in fig. 1, a multi-channel sound signal encoding method in the related art includes the steps of:

step 101, mapping a first multi-channel sound signal into a first frequency domain signal by adopting Modified Discrete Cosine Transform (MDCT) or Modified Discrete Sine Transform (MDST);

step 102, dividing the first frequency domain signal into different time-frequency sub-bands;

step 103, calculating a first statistical characteristic of the first multi-channel sound signal in each time-frequency sub-band of different time-frequency sub-bands;

step 104, estimating a Principal Component Analysis (PCA) mapping model according to the first statistical characteristic;

step 105, mapping the first multi-channel sound signal into a second multi-channel sound signal by adopting a PCA mapping model;

and 106, performing perceptual coding on at least one group of second multi-channel sound signals and the PCA mapping model according to the difference of time, frequency and channels, and multiplexing into a coding multi-channel code stream.

The method performs perceptual coding by selecting a plurality of important items in the mapped subspace components (the number of K), for example, a plurality of principal component components (the number of p) are coded, and the rest subspace components (the number of K-p) are discarded, so that the effect of reducing the dimension is achieved. Such a dimension reduction process sometimes causes a difference in reconstruction quality between multi-channel audios, and if a certain channel audio is highly correlated with a discarded subspace component (a subspace component that is not perceptually encoded), the reconstruction quality of the channel is significantly deteriorated, and the energy of the reconstructed audio is also significantly lower than the original energy.

The technical solution of the present invention is described in detail below with reference to fig. 2-20, but the scope of the present invention is not limited to the embodiments.

In view of the problems of the multi-channel coding method in the prior art, the present invention provides a scheme for performing compensation coding/decoding on discarded subspace components, and embodiment 1 is an illustration of a compensation coding apparatus for performing compensation coding on discarded subspace components according to the present invention; embodiment 2 describes a multi-channel encoding method to which this compensation encoding method is applied; corresponding to the encoding, embodiment 3 is an explanation of a compensation decoding apparatus for performing compensation decoding on discarded subspace components according to the present invention; embodiment 4 describes a multi-channel decoding method to which this compensation decoding method is applied; embodiment 13 describes a multi-channel encoding apparatus to which the compensation encoding apparatus described in embodiment 1 is applied; embodiment 14 describes a multi-channel decoding apparatus to which the compensation decoding apparatus described in embodiment 3 is applied; embodiment 5 is a description of a method for generating a fixed base compensation vector set a1 in compensation coding; embodiment 6 is a description of a method for generating an adaptive basic compensation vector set a2 in compensation coding; embodiments 7 to 12 describe several more specific compensation encoding/decoding methods, wherein embodiments 7 and 8 describe the cases when the sequence number indext of the basic compensation vector in the set is the compensation encoding parameter; example 9 describes the case when the model parameters are compensation encoding parameters; embodiment 10 describes the case when the set of basic compensation vectors is not stored; example 11 describes a case where two vectors are selected from a set to be combined; embodiment 10 describes a variation of the compensation process by adjusting the coefficients of the mapping matrix.

Example 1:

as shown in fig. 2, the compensation encoding apparatus of the present invention includes a basic compensation vector generation module, a basic compensation set generation module, a compensation encoding parameter encoding module, and a code stream multiplexing module; wherein,

a base compensation vector generation module for generating a fixed compensation vector as a base compensation according to the perceptually encoded signal modelVectorOr generating an adaptive compensation vector as a base compensation vector based on the copy of the encoded principal component in the current frameThe fixed compensation vector can be generated according to one or more of a perceptually coded sinusoidal signal model, a random noise signal model and a harmonic and single-chord plus noise HILN (harmonic and industrial Line plus noise) model, and does not change along with the change of each frame of coded data, and the adaptive compensation vector changes along with the change of each frame of coded data;

a basic compensation set generation module for generating each basic compensation vectorIs placed in the basic compensation vector set A and is used as each basic compensation vectorGenerating one-to-one corresponding serial numbers or characteristic parameters, wherein the basic compensation vector set A comprises a fixed basic compensation vector set A1 and/or an adaptive basic compensation vector set A2, fixed compensation vectors are placed in the fixed basic compensation vector set A1, and adaptive compensation vectors are placed in the adaptive basic compensation vector set A2;

a compensation coding parameter coding module for selecting a basic compensation vector for compensation according to the compensation requirementAnd compensating the basic compensation vectorsThe serial number or the characteristic parameter is used as a compensation coding parameter to carry out quantization coding to obtain a compensation coding code stream;

and the code stream multiplexing module is used for storing the compensation coding code stream as a code stream file or directly sending the compensation coding code stream to a communication channel, so that the compensation decoding device can conveniently read or directly receive the compensation coding code stream.

The compensation coding device of the invention utilizes the basic compensation vector generation module to generate the basic compensation vector for compensation, the basic compensation vector can be used for replacing and compensating the abandoned subspace component, the signal reconstruction quality is effectively improved, and the reconstructed audio energy is similar to the original energy; the basic compensation set generation module is utilized to place the basic compensation vectors in the basic compensation vector set, and corresponding serial numbers or characteristic parameters are generated for the basic compensation vectors, so that the basic compensation vectors in the basic compensation vector set can be called conveniently; the compensation coding parameter coding module is used for carrying out quantization coding on the compensation coding parameters, so that the compensation coding parameters are conveniently stored or forwarded; the code stream multiplexing module can be used for storing the compensation coding code stream into a code stream file or directly sending the compensation coding code stream to a communication channel, so that a decoding device can conveniently read or receive the compensation coding code stream, and the compensation coding parameters are multiplexed in real time; the fixed compensation vector and the self-adaptive compensation vector are respectively placed in the fixed basic compensation vector set and the self-adaptive basic compensation vector set, so that the fixed compensation vector and the self-adaptive compensation vector are conveniently and quickly called, and the compensation efficiency is improved; the fixed compensation vector which is fixed and unchangeable is generated by utilizing the known signal model, the fixed compensation vector does not change along with the change of the frame coding data, the reliability is high, the adaptive compensation vector can change along with the change of each frame coding data in real time, and the real-time performance is high.

Example 2:

as shown in fig. 3, the compensation encoding apparatus of the present invention is applied to compensate the multi-channel sound signal encoding method, and specifically includes the following steps:

step 201, mapping the first multi-channel sound signal into a first frequency domain signal by using Modified Discrete Cosine Transform (MDCT) or Modified Discrete Sine Transform (MDST);

step 202, dividing the first frequency domain signal into different time-frequency sub-bands;

step 203, calculating a first statistical characteristic of the first multi-channel sound signal in each time-frequency sub-band of different time-frequency sub-bands;

step 204, estimating a PCA mapping model according to the first statistical characteristic;

step 205, mapping the first multi-channel sound signal into a second multi-channel sound signal by adopting a PCA mapping model;

step 206, performing perceptual coding on at least one group of the second multi-channel sound signals according to the difference of time, frequency and sound channels to obtain perceptual coding code streams;

step 207, according to the difference of time, frequency and sound channel, using the coding compensation device of the invention to perform compensation coding processing on at least one group of the second multi-sound-channel sound signal which is not subjected to perceptual coding, so as to obtain a compensation coding code stream;

and 208, coding the PCA mapping model to obtain a PCA mapping model code stream, and multiplexing the PCA mapping model code stream with the perception coding code stream, the compensation coding code stream and the like to obtain a coding multi-channel code stream.

Wherein, the initial expression form of the first multi-channel sound signal in step 201 is time domain signal u (m, t), and after the mapping process in step 201, a multi-channel frequency domain signal x (m, k) can be obtained, where m is a channel number, t is a frame (or sub-frame) number, and k is a frequency number;

in step 202, if the first frequency-domain signal obtained in step 201 is x (m, k), x (m, k) may be divided into different time-frequency sub-bands x_i(t, k), m is the sound track serial number, i is the serial number of the time frequency sub-band, t is the frame (or sub-frame) serial number, k is the frequency serial number;

in the step of execution205, a multi-channel sound signal x is calculated in different frequency sub-bands_i(t, k) and estimating an optimized subspace mapping model W_i(t, k) using the estimated mapping model to map the multi-channel signal into a new subspace, obtaining a new set of multi-channel signals z_i(t,k)；

When step 207 is executed, the specific steps are as shown in fig. 4, and include:

step 1-1, generating a fixed compensation vector as a base compensation vector based on a perceptually encoded signal modelOr generating an adaptive compensation vector as a base compensation vector based on a copy of the perceptually encoded principal component (second multi-channel sound signal) in the current frameThe fixed compensation vector is generated according to one or more of a perceptually encoded sinusoidal signal model, a random noise signal model and a HILN signal model and does not change along with the change of each frame of encoded data, and the adaptive compensation vector changes along with the change of each frame of encoded data;

step 1-2, placing each basic compensation vector in a basic compensation vector set A, and generating a one-to-one corresponding sequence number or characteristic parameter for each basic compensation vector, wherein the basic compensation vector set comprises a fixed basic compensation vector set A1 and/or an adaptive basic compensation vector set A2, the fixed compensation vector is placed in a fixed basic compensation vector set A1, and the adaptive compensation vector is placed in an adaptive basic compensation vector set A2;

step 1-3 for processing a second multi-channel signal according to a compensationNeed to select a base compensation vector for compensationAnd compensating the basic compensation vectorsThe serial number or the characteristic parameter of the code is taken as a compensation coding parameter to carry out quantization coding, and a compensation coding code stream is obtained.

In step 208, the mapping model may be encoded by encoding the corresponding mapping matrix (i.e., eigenvector), other transformations of the model, or by directly encoding the covariance matrix from which the mapping matrix is calculated. In the mapping model coding, well-known methods such as scalar quantization, vector quantization, predictive coding, and the like may be used, or entropy coding (such as huffman coding or arithmetic coding) may be used to further improve the coding efficiency.

Example 3:

as shown in fig. 5, the compensation decoding apparatus of the present invention includes a basic compensation vector generation module, a basic compensation set generation module, a code stream de-multiplexing module, a compensation coding parameter decoding module, and a subspace component compensation module. Wherein,

a base compensation vector generation module for generating a fixed compensation vector as a base compensation vector from the perceptually decoded signal modelOr generating an adaptive compensation vector from a copy of the decoded principal component (second multi-channel sound signal) in the current frame as a base compensation vectorThe fixed compensation vector is generated according to one or more of a perceptually decoded sinusoidal signal model, a random noise signal model and a HILN signal model and does not change along with the change of each frame of decoded data; fromThe adaptive compensation vector changes along with the change of each frame of decoded data;

a basic compensation set generation module for generating each basic compensation vectorIs placed in the basic compensation vector set A and is used as each basic compensation vectorGenerating one-to-one corresponding serial numbers or characteristic parameters, wherein the basic compensation vector set A comprises a fixed basic compensation vector set A1 and/or an adaptive basic compensation vector set A2, the fixed compensation vectors are placed in the fixed basic compensation vector set A1, and the adaptive basic compensation vectors are placed in the adaptive basic compensation vector set A2;

the code stream demultiplexing module is used for reading a code stream file or directly reading a compensation coding code stream from a communication channel;

a compensation coding parameter decoding module for decoding the compensation coding code stream to obtain each basic compensation vector for compensationThe serial number or characteristic parameter of (a);

a subspace component compensation module for selecting corresponding basic compensation vectors from the basic compensation vector set A according to the serial numbers or the characteristic parametersThen, the selected basic compensation vector is usedReplacing a second multi-channel signalThe discarded subspace component.

Generating a basic compensation vector for compensation by using a basic compensation vector generation module, wherein the basic compensation vector is used for performing substitution compensation on the abandoned subspace component, so that the signal reconstruction quality is effectively improved, and the reconstructed audio energy is similar to the original energy; the basic compensation set generation module is used for placing the basic compensation vectors into the basic compensation vector set, and corresponding serial numbers or characteristic parameters are generated for the basic compensation vectors, so that the basic compensation vectors in the basic compensation vector set can be conveniently called for decoding compensation; decoding the compensation coding parameters by using a compensation coding parameter decoding module so as to obtain compensation coding parameters; selecting a corresponding basic compensation vector from the basic compensation vector set by using a subspace component compensation module according to the compensation coding parameters, thereby further performing substitution compensation on the discarded subspace component; the fixed compensation vector and the self-adaptive compensation vector are respectively placed in the fixed basic compensation vector set and the self-adaptive basic compensation vector set, so that the fixed compensation vector and the self-adaptive compensation vector are conveniently and quickly called, and the compensation efficiency is improved; the fixed compensation vector which is fixed and unchangeable is generated by utilizing a known signal model, and the fixed compensation vector does not change along with the change of frame decoding data, so that the reliability is higher; the self-adaptive compensation vector can change along with the change of each frame of decoded data in real time, and has high real-time performance.

Example 4:

as shown in fig. 6, the compensation application of the decoding apparatus of the present invention to the method for decoding a multi-channel sound signal includes the following steps:

step 301, demultiplexing the encoded multi-channel code stream to obtain a perception encoded code stream, a compensation encoded code stream and a PCA mapping model code stream;

step 302, decoding the sensing code stream and the PCA mapping model code stream in the multi-channel code stream to obtain at least one group of second multi-channel sound signals and a PCA mapping model;

step 303, decoding the compensation coding stream in the coding multi-channel code stream by using the compensation decoding device of the present invention to obtain a compensation coding parameter, and performing compensation processing on the second multi-channel sound signal by using the compensation coding parameter to obtain a second multi-channel sound signal after the compensation processing;

step 304, mapping the compensated second multi-channel sound signal back to the first multi-channel sound signal by adopting a PCA mapping model;

step 305, mapping the first multi-channel sound signal from the frequency domain to the time domain by using the Inverse Modified Discrete Cosine Transform (IMDCT) or the Inverse Modified Discrete Sine Transform (IMDST).

When step 303 is executed, the specific steps are as shown in fig. 7, and include:

step 2-1, generating a fixed compensation vector as a basic compensation vector according to a perceptually decoded signal model, or generating an adaptive compensation vector as a basic compensation vector according to a decoded principal component copy in a current frame, wherein a basic compensation vector set a comprises a fixed basic compensation vector set a1 and/or an adaptive basic compensation vector set a2, the fixed compensation vector is placed in a fixed basic compensation vector set a1, and the adaptive compensation vector is placed in an adaptive basic compensation vector set a 2;

step 2-2, placing each basic compensation vector in a basic compensation vector set, and generating a one-to-one corresponding serial number or characteristic parameter for each basic compensation vector;

step 2-3, decoding the compensation coding code stream to obtain each basic compensation vector for compensationThe serial number or characteristic parameter of (a);

step 2-4, used for selecting corresponding basic compensation vector from the basic compensation vector set A according to the serial number or the characteristic parameterThen, the selected basic compensation vector is usedReplacing a second multi-channel signalThe discarded subspace component.

When the first multi-channel sound signal is a plurality of grouped sound signals in the frequency domain in executing step 305, before step 304, the method may further include: decoding the sound channel grouping information in the code stream to obtain decoded sound channel grouping information; the plurality of packet audio signals are packet-restored according to the decoded channel packet information to obtain a third multi-channel audio signal, and step 304 is performed with the third multi-channel audio signal as the first multi-channel audio signal.

Example 5:

the fixed basis compensation vector set a1 may be generated according to a perceptually encoded signal model or a combination of several signal models, such as a sinusoidal model, a random noise model, a hiln (harmonic and industrial Line plus noise) model, etc. Each basic compensation vector in the fixed basic compensation vector set A1Are determined by the corresponding compensation coding parameters, and the compensation coding parameters are in one-to-one correspondence, for example, if there are Nt vectors in A1, there are Nt compensation coding parameters Paraset [ i [ ]](i-1 … Nt) corresponds to it, e.g., when a1 is generated from the HILN signal model, Paraset [ i ═ i [ ] is generated]Is a set of fixed basis compensation vectors for a set of HILN models.

When the fixed base compensation vector set A1 is generated, the base compensation vector is first determinedAnd calculating the tonality parameter Tnrset to be expressed according to the tonality parameter Tnrset to obtain normalized energy of the corresponding signal model, and finally generating a fixed compensation vector of the corresponding signal model according to the normalized energy.

The specific steps are illustrated in fig. 8, taking as an example the generation of a combined signal of a sinusoidal model signal and a random noise model signal from the tonality parameters, where Paraset [ i ] is the tonality parameter set Tnrset [ i ]. The method comprises the following specific steps:

step 3-1, determining the model parameter set expressed by the set a1, namely the number Nt of elements in the set a1, and a tonality parameter Tnrset [ i ] (i ═ 1 … Nt, 0 ≦ Tnrset [ i ] ≦ 1) corresponding to each element, wherein the tonality describes the tonality of the signal, and when Tnrset equals to 1, the signal is a pure chord signal, and when Tnrset equals to 0, the signal is white noise;

step 3-2, enabling indext to be 1;

step 3-3, according to the tone tuning parameter Tnrset [ indext [ ] ] [, a tone tuning parameter]Constructing a base compensation vectorThe method specifically comprises the following steps:

firstly, noise energy engo _ n and tonal energy engo _ t in a normalized compensation vector are obtained according to a tonal parameter Tnrset [ indext ], and the requirement that engo _ n + engo _ t is 1 is met, and the calculation method can adopt the method in patent application 201210085257.2;

then, a noise signal with energy of engo _ n is generatedAnd chordal signal with energy of engo _ tThe generation method can adopt the method in patent application 201210085257.2;

finally, the noise signal is analyzedChord signalCombining to obtain a base compensation vectorComprises the following steps:

step 3-4, the basic compensation vector is processedAdded to set A1, in set A1The corresponding sequence number is indext.

And 3-5, enabling indext to be indext +1, and then turning to the step 3-3, and repeating until indext is greater than Nt.

Example 6:

the adaptive basic compensation vector set a2 may be generated from the decoded signals of the current frame or previous and next frames, such as the perceptually decoded p principal components of the current frame, the perceptually decoded p principal components of the previous frame, the decoded low frequency data of the current frame, etc. The adaptive basic compensation vector set a2 needs to be updated every frame of data processing.

When the adaptive basic compensation vector set A2 is generated, firstly, the coded principal component components in the current frame are read, and normalization processing is carried out on each coded principal component; then, the normalized encoded principal component is used as a variation vector, and encoding parameters corresponding to each variation vector are set.

As shown in fig. 9, the generation process of a2 is described by taking the generation of an adaptive basic compensation vector set a2 according to p principal components of perceptual coding/decoding of a current frame as an example:

step 4-1, enabling indext to be 1;

step 4-2, the indext main component of the perception coding/decoding of the current frameReplacement of base compensation vectorsAnd compensating the vector to the basisThe normalization processing is carried out as follows:

step 4-3, the basic compensation vector is processedAdded to the adaptive basic compensation vector set A2 in set A2The corresponding compensation coding parameter is indext;

and 4-4, enabling indext to be indext +1, then turning to the step 4-2, and repeating until indext > p.

Embodiments 7 and 8 describe the compensation encoding method and the compensation decoding method when the sequence number indext of the base compensation vector in the set is the compensation encoding parameter.

Example 7:

as shown in fig. 10, the method for coding the PCA principal component by compensation processing includes the following steps:

step 5-1, calculating a certain perceptually uncoded principal component to be compensatedThe energy engo is subjected to quantization coding, and a known method can be adopted for the quantization coding, and scalar quantization or vector quantization can be adopted;

step 5-2, selecting the basic compensation vector for compensation processing from the basic compensation vector set A The index in the set a is used as a compensation coding parameter, the compensation coding parameter index is quantized and coded by a known method, either scalar quantization or vector quantization, and a base compensation vector for compensation processing is selected from the base compensation vector set aThere may be several ways of selecting the tonality of the signal, e.g. based on the criterion that the difference in tonality is minimalThe closest base compensation vector;

step 5-3, adjustingCorresponding mapping matrix coefficient wok (K1 … K), as described above, the o-th principal componentInverse mapping to data of k channelThe mapping matrix coefficient corresponding to above is wok, for each K (K equals 1 … K), ifData with the k channelThe absolute value of the difference of the pitch tonality is greater than a given Threshold _1 (the pitch tonality is in the range of 0,1]If the difference between the tonality of the two signals is large, for example, greater than 0.5, which indicates that the characteristics of the two signals are very different, and it is likely that one is chordal and the other is noisy), then the mapping matrix coefficient wok is adjusted to 0, or the adjustment of the mapping matrix coefficient may not be performed, and step 5-3 may be omitted.

Example 8:

as shown in fig. 11, the method for performing compensation processing decoding on the principal component of PCA specifically includes the following steps:

step 6-1, decoding a discarded principal component to be compensatedEnergy engo of (a).

Step 6-2, decoding the compensated encodingThe parameter indext, and accordingly, the corresponding basic compensation vector is selected from the basic compensation vector set A

6-3, performing compensation treatment, namely performing basic compensation vectorCopy toAnd amplitude adjustment is carried out:

embodiment 9 describes a compensation encoding method and a compensation decoding method when a model parameter is used as a compensation encoding parameter.

Example 9:

for the basic compensation vector set A, each vector thereinThe uniquely determined indext corresponds to the vector, so that the indext is taken as a compensation coding parameter and coded in the embodiment, and the decoding end can determine the corresponding vector according to the decoded indext and the set AFor the fixed basis compensation vector set A1, the model parameter Paraset [ indext ] is similar to indext]There is also such a one-to-one correspondence between (indext ═ 1 … Nt) and the vectors of the fixed base compensation vector set a1, and the model parametersParaset[indext]There is also such a one-to-one correspondence between (index ═ 1 … Nt) and the sequence number index. Therefore, the model parameter Paraset [ index ] may be selected]This can be seen as a special encoding of the sequence number indext, as the compensation encoding parameter is encoded. Using the model parameter as tone parameter Tnrset]The description will be given by taking coding as an example.

As shown in FIG. 12, the method of encoding PCA principal component by compensation processing using model parameters as compensation encoding parameters is to discard the principal componentThe steps of performing compensation encoding are as follows:

step 7-1, calculating a certain discarded principal component to be compensatedThe energy engo is coded, and a known method can be adopted as a quantization coding method, and scalar quantization or vector quantization can be adopted;

7-2, selecting the basic compensation vector for compensation processing from the basic compensation vector set A(The sequence number in the set A is indext, and the corresponding model parameter is the model parameter Tnrset [ indext ]]) For Tnrset [ indext ]]Quantization coding is carried out, and a known method can be adopted as the method for quantization coding, and scalar quantization or vector quantization can be adopted;

step 7-3, adjustingThe corresponding mapping matrix coefficient wok (K is 1 … K), and the adjustment procedure is the same as the aforementioned step 5-3, and may not be performedSuch an adjustment, step 7-3 may be omitted.

As shown in fig. 13, the method of decoding the PCA principal component by the compensation process using the model parameters as the compensation encoding parameters is to discard the principal componentThe steps of line compensation decoding are as follows:

step 8-1 discarded principal component components to be compensatedDecoding the energy engo;

step 8-2, decoding the tonal parameter Tnrset [ indext [ ]]And selecting corresponding basic compensation vector from the basic compensation vector set A according to the basic compensation vector

Step 8-3, compensation processing is carried out, namely compensation components are obtainedCopy toAnd amplitude adjustment is carried out:

as can be seen from the fixed basis compensation vector set generation of the foregoing embodiment 5 and the adaptive basis compensation vector set generation of the foregoing embodiment 6, each vector in the basis compensation vector set a or a1 or a2 is constructed by a set of determined model parameters paramet or a determined sequence number indext, and thus, the set a or a1 or a2 may not be stored, but the compensation vector may be generated directly in the decoder according to the model parameters (the method in embodiment 5) or by normalizing the perceptually decoded principal component according to the sequence number indext (corresponding to the indext principal component in the perceptually decoded p principal components) (the method in embodiment 6); accordingly, the encoder does not need to search the sets a or a1 or a2 to determine the index or the model parameter Paraset, but directly obtains the sequence number index or the model parameter Paraset through signal analysis and encodes the same. Directly generating a compensation vector according to the model parameters (the method in embodiment 5), which is equivalent to the method of performing model parameter coding on the principal component to be compensated, and is equivalent to the method in embodiment 9; generating a compensation vector by normalizing the perceptually decoded principal component according to the sequence number indext (the index-th principal component among the perceptually decoded p principal component components) corresponds to performing compensation processing by copying data from the perceptually decoded principal component, and is equivalent to the method of embodiment 7/embodiment 8. A compensation encoding/compensation decoding method when a fixed base compensation vector set is not stored will be described as an example.

Example 10:

for the fixed base compensation vector set A1, each vector thereinAre all defined by a unique set of model parameters Paraset [ index [ ] ]](indext ═ 1 … Nt), and therefore, set a1 may not be stored, but generated directly at the decoder from the model; accordingly, the encoder does not need to search the set A1 to determine indext and Paraset [ indext ]]Instead, the model Paraset [ index ] is directly obtained by signal analysis]And encodes it. This processing method is equivalent to the method of example 9, and the description will be given by taking the case where the model parameter is the tonality parameter Tnrset as an example.

As shown in fig. 14, the method for coding the PCA principal component by compensation processing stored in the fixed basis compensation vector set a is not required. In the encoding, the discarded principal component is subjected toThe steps of performing compensation encoding are as follows:

step 9-1, calculateThe energy engo is subjected to quantization coding, and a known method can be adopted for the quantization coding, and the quantization coding can be scalar quantization or vector quantization;

step 9-2, calculateThe tonality parameter Tnrset of (a) and performing quantization coding, and the method for calculating the tonality can adopt the method in patent application 201210085257.2;

step 9-3, adjustingThe corresponding mapping matrix coefficient is wok (K equals 1 … K), the adjustment procedure is the same as that in step 5-3, and if no such adjustment is required, step 9-3 may be omitted.

As shown in FIG. 15, the method for performing compensation processing decoding on PCA principal component without fixed basis compensation vector set storage, in which the discarded principal component is decodedThe steps of line compensation decoding are as follows:

step 10-1, decodingEnergy engo of (a);

step 10-2, decoding the tonal parameter Tnrset and generating a compensation component therefromThe method comprises the following specific steps:

firstly, the noise energy engo _ n and the tonal energy engo _ t of the normalized compensation vector are obtained according to the tonality parameter Tnrset, and the requirement that engo _ n + engo _ t is 1 is met, and the calculation method can adopt the method in the patent application 201210085257.2;

then, a noise signal with energy of engo _ n is generatedAnd chordal signal with energy of engo _ tThe method can be the method of patent application 201210085257.2;

finally, the noise signal is analyzedChord signalCombined to obtain a compensation component

Step 10-3, compensation processing is carried out, namely compensation component is carried outCopy toAnd amplitude adjustment is carried out:

example 11:

in addition to selecting one basic compensation vector from the set a (a — a1 ∪ a2) for compensation processing, a combination of two or more basic compensation vectors from the set a may be selected for compensation processing, and a description will be given by taking as an example a combination of one vector from each of the fixed basic compensation vector set a1 and the adaptive basic compensation vector set a 2.

As shown in fig. 16, the method of vector combination for encoding the PCA principal component by the compensation process. In the encoder, the steps of performing compensation processing encoding are as follows:

step 11-1, calculating a discarded principal component to be compensatedEnergy engo of (a);

step 11-2, selecting a basic compensation vector for compensation processing from the adaptive basic compensation vector set A2(Index 1 in set a2 as a compensation encoding parameter), a base compensation vector for compensation processing is selected from the base compensation vector setCan be prepared byBy selecting tonality or tonality according to a plurality of criteria, e.g. the criterion that the difference in tonality of signals is minimalThe closest base vector;

step 11-3, determining the compensation processing of the adaptive basic compensation vector set A2 and the fixed basic compensation vector set A1The specific gravity β 1 and β 2 in the energy of (1), and β 1 and β 2 satisfy β 1+ β 2 as 1, for example β 1 as 1, and β 2 as 0, which is equivalent to processing by using only the vector in the adaptive basic compensation vector set a2, for example β 1 as 0.6 and β 2 as 0.4, which is equivalent to processing by using the combination of the fixed basic compensation vector set a1 and the adaptive basic compensation vector set a 2;

such as by calculationAndto determine β 1, β 2 values, the similarity measure may have various methods, such as correlation coefficient, pitch similarity, etc., taking correlation coefficient as an example,andis cor1, then:

β1＝cor1*cor1

β2＝1-cor1*cor1

taking the similarity of pitch as a measure,andare TNR1 and TNR2, respectively, then β 1 and β 2 may be determined using the following formulas:

β1＝1-2*abs(TNR1-TNR2)/(TNR1+TNR2)

β2＝1-β1

wherein abs () is an absolute value function;

step 11-4, determining the value of the target tonality of the selected vector from the set of fixed basis compensation vectors a1 to be TNR3, which may be, for example, TNR3 by the following formula:

TNR3＝TNR1-TNR2；

step 11-5, selecting a basic compensation vector for compensation processing from the basic compensation vector set A1(Index 2 in the set a1 as a compensation encoding parameter), a base compensation vector for compensation processing is selected from the adaptive base compensation vector set a2There may be several ways to select the base compensation vector having a tonality closest to TNR3 as the basis for the criterion that the difference in tonality is minimal

And 11-6, performing quantitative coding on indext1, indext2, engo1 and engo2, wherein engo2 is engo × β 2, and engo1 is engo × β 1.

As shown in fig. 17, the method of vector combining for decoding the PCA principal component by the compensation process. In the decoder, forThe steps of decoding the compensation process are as follows:

step 12-1, decoding indext1, indext2, engo1 and engo 2;

step 12-2, obtaining the adaptive basic compensation vector set A2 according to indext1

Step 12-3, obtaining the fixed basic compensation vector set A1 according to indext2

Step 12-4, compensation processing is carried out, namely compensation components are carried outCopy toAnd amplitude adjustment is carried out:

example 12:

when the compensation component is generated by selecting the basic compensation vector from the adaptive basic compensation vector set a2, the effect is equivalent to adjusting the mapping matrix coefficients, which forms a variation of the compensation processing method: compensation for adjusting mapping matrix coefficientsThe treatment method comprises the following steps: by adjusting the inverse mapping matrix W^TTo achieve the use of perceptually encoded p principal componentsFor the purpose of compensating for the discarded principal component or components.

As shown in fig. 18, the method for coding the PCA principal component by adjusting the mapping matrix coefficients. Perceptually uncoded principal component to be compensated forThe steps of performing the compensation process encoding in the encoder are as follows:

step 13-1, selecting perceptually encoded p principal componentsIn which one main component is selectedCome to rightThe compensation process is carried out and the compensation process is carried out,the sequence numbers in the p principal components are denoted by l,the selection method of (2) can be various, for example, p principal components can be selectedLast one of (1) Can selectTone property of middle tone andthe closest one asCan selectFlatness of the middle spectrum andthe closest one as

Step 13-2, calculatingAndenergy engo and engl of (a);

step 13-3, adjusting mapping matrix coefficients wlk:

step 13-4, adjusting mapping matrix coefficients wok:

wok＝0，k＝1,…,K。

in the decoder, only normal decoding processing is required for the mapping matrix, and special processing is not required.

Example 13:

as shown in fig. 19, an efficient multi-channel audio coding apparatus based on subspace mapping is provided, which combines the advantages of perceptual coding and compensation processing of mapped subspace components by using the aforementioned compensation processing method of the present invention based on the methods of patent applications 201410395806.5 and 201410404895.5, and can effectively improve the coding efficiency and reconstruction quality of multi-channel audio. The coding part of the device comprises a time-frequency transformation module, a frequency band division module, a subspace mapping module, a coding/compensation selection module, a perception coding module, a compensation processing coding module, a mapping matrix coding module and a multichannel audio code stream multiplexing module. Wherein,

the time-frequency transformation module is used for mapping the multi-channel sound signals into frequency domain signals by adopting time-frequency transformation or subband filtering;

the frequency band dividing module is used for dividing the frequency domain signals into different time frequency sub-bands;

the subspace mapping module is used for calculating the statistical characteristics of the multi-channel sound signals X in different frequency subbands, estimating a PCA mapping model W, and mapping the multi-channel signals to a new subspace by adopting the estimated mapping model to obtain a new group of multi-channel signals Z;

a coding/compensation selection module for selecting p principal component components that need to be perceptually coded and q principal component components that need to be compensated (q ═ K-p);

the perception coding module is used for respectively carrying out perception coding on p principal component components selected in the coding/compensation selection to obtain a perception coding code stream;

the compensation processing coding module is used for updating the self-adaptive basic compensation vector in the basic compensation vector set A according to p main component components of the perceptual coding of the frame, and respectively carrying out compensation processing coding on q main component components selected in coding/compensation selection to obtain a compensation coding code stream;

the mapping model coding module is used for coding a corresponding mapping matrix (namely, a characteristic vector) to obtain a mapping model coding code stream, coding other transformation forms of the model, and directly coding to calculate a covariance matrix of the mapping matrix. When the mapping model is coded, the well-known methods such as scalar quantization, vector quantization, predictive coding and the like can be adopted, and entropy coding (such as huffman coding or arithmetic coding) can also be adopted to further improve the coding efficiency;

and the multi-channel audio code stream multiplexing module is used for multiplexing the perception coding code stream, the compensation coding code stream, the mapping model coding code stream and the like to obtain a coding multi-channel code stream.

Wherein the coding/compensation selection portion may be selected using a variety of methods. E.g. in order of energy size, selecting the p principal components with the largest energy for perceptual coding, and performing compensation coding on q of the remaining K-p principal components, q<Or K-p, can also be according toTo pairThe method comprises the following specific steps:

first, the importance matrix Q ═ Q (qij) is calculated_K*KQij characterizeTo pairThe importance of can be calculated by various methods, such as:

the importance may also be weighted by the magnitude value:

then, the matrix Q is (qij) according to the degree of importance_K*KComputingImportance of (D) P (j)

P(j)＝max{qij,i＝1,…,K}；

Finally, according to P (j) pairsSorting, selecting p principal components with the maximum P (j) value for perceptual coding, and performing compensation coding on q of the remaining K-p principal components, q<＝K-p。

As shown in fig. 20, the present invention correspondingly provides a subspace mapping-based efficient multi-channel audio decoding apparatus, which includes a multi-channel audio code stream de-multiplexing module, a perceptual decoding module, a mapping model decoding module, a compensation processing decoding module, a subspace inverse mapping module, and a frequency-time transformation module. Wherein,

the multichannel audio code stream demultiplexing module is used for demultiplexing the coding multichannel code stream to obtain a perception coding code stream, a compensation coding code stream, a mapping model coding code stream and the like;

the perception decoding module is used for decoding the perception coding code stream in the coding multichannel code stream to obtain p principal component components;

the mapping model decoding module is used for decoding the mapping model coding code stream to obtain a PCA mapping model, namely a mapping matrix;

the compensation processing decoding module is used for updating the self-adaptive basic compensation vector in the basic compensation vector set A according to the p main component components of the frame which are subjected to the perception decoding, and performing compensation processing decoding on the q main component components which need to be subjected to the compensation processing;

the subspace inverse mapping module is used for mapping the principal component obtained by decoding back to the original space of the multi-channel data by adopting the PCA mapping model obtained by decoding;

and the frequency-time transformation module is used for mapping the multi-channel sound signals from a frequency domain to a time domain by adopting inverse time-frequency transformation, or mapping the multi-channel sound signals from a sub-band domain to the time domain by adopting inverse sub-band filtering.

Claims (14)

1. A complementary encoding apparatus for discarded subspace components, comprising:

the basic compensation vector generation module is used for generating a fixed compensation vector as a basic compensation vector according to a perceptually coded signal model, or generating a self-adaptive compensation vector as a basic compensation vector according to a coded principal component copy in a current frame;

the basic compensation set generation module is used for placing each basic compensation vector in the basic compensation vector set and generating a one-to-one corresponding sequence number or characteristic parameter for each basic compensation vector;

and the compensation coding parameter coding module is used for selecting basic compensation vectors for compensation according to the compensation requirement, and performing quantization coding by taking the serial numbers or the characteristic parameters of the basic compensation vectors as compensation coding parameters to obtain a compensation coding code stream.

2. A complementary encoding device for discarded subspace components as recited in claim 1, further comprising:

and the code stream multiplexing module is used for storing the compensation coding code stream as a code stream file or directly sending the compensation coding code stream to a communication channel.

3. The apparatus according to claim 1 or 2, wherein said set of base compensation vectors comprises a set of fixed base compensation vectors and/or a set of adaptive base compensation vectors; the fixed compensation vector is placed in a fixed basic compensation vector set; the adaptive compensation vector is placed in an adaptive base compensation vector set.

4. The apparatus of claim 1 or 2, wherein the fixed compensation vector is generated according to one or more of a sinusoidal signal model, a random noise signal model, and a HILN signal model, and does not vary with the variation of each frame of encoded data; the adaptive compensation vector changes following the change in the encoded data for each frame.

5. A method of compensating encoding for a discarded subspace component, comprising the steps of:

step 1, generating a fixed compensation vector as a basic compensation vector according to a perceptually encoded signal model, or generating a self-adaptive compensation vector as a basic compensation vector according to a copy of a coded principal component in a current frame;

step 2, placing each basic compensation vector in a basic compensation vector set, and generating a one-to-one corresponding serial number or characteristic parameter for each basic compensation vector;

and 3, selecting basic compensation vectors for compensation according to compensation requirements, and performing quantization coding by taking the serial numbers or the characteristic parameters of the basic compensation vectors as compensation coding parameters to obtain a compensation coding code stream.

6. A method of complementary encoding for a discarded subspace component according to claim 5, further comprising the steps of:

and 4, storing the compensation coding code stream as a code stream file or directly sending the compensation coding code stream to a communication channel.

7. The method according to claim 5 or 6, wherein said set of base compensation vectors comprises a set of fixed base compensation vectors and/or a set of adaptive base compensation vectors; the fixed compensation vector is placed in a fixed basic compensation vector set; the adaptive compensation vector is placed in an adaptive base compensation vector set.

8. A method of compensating coding for rejected subspace components according to claim 5 or 6, wherein said fixed compensation vector is generated according to one or more of a sinusoidal signal model, a random noise signal model and a HILN signal model, and does not follow the variations of the coded data per frame; the adaptive compensation vector changes following the change in the encoded data for each frame.

9. A device for compensated decoding of rejected subspace components, comprising:

the basic compensation vector generation module is used for generating a fixed compensation vector as a basic compensation vector according to the perceptually decoded signal model or generating a self-adaptive compensation vector as a basic compensation vector according to the decoded principal component copy in the current frame;

the basic compensation set generation module is used for placing each basic compensation vector in the basic compensation vector set and generating a one-to-one corresponding sequence number or characteristic parameter for each basic compensation vector;

the code stream demultiplexing module is used for reading a code stream file or directly reading a compensation coding code stream from a communication channel;

the compensation coding parameter decoding module is used for decoding the compensation coding code stream to obtain the serial number or the characteristic parameter of each basic compensation vector for compensation;

and the subspace component compensation module is used for selecting corresponding basic compensation vectors from the basic compensation vector set according to the sequence numbers or the characteristic parameters, and then replacing the discarded subspace components with the selected basic compensation vectors.

10. The apparatus for compensatory decoding for a discarded subspace component according to claim 9, wherein the set of base compensation vectors includes a set of fixed base compensation vectors and/or a set of adaptive base compensation vectors; the fixed compensation vector is placed in a fixed basic compensation vector set; the adaptive compensation vector is placed in an adaptive base compensation vector set.

11. The apparatus for compensatory decoding according to claim 9 or 10, wherein the fixed compensation vector is generated according to one or more of a sinusoidal signal model, a random noise signal model, and a HILN signal model, and does not vary with variation of decoded data per frame; the adaptive compensation vector varies following the variation of the decoded data per frame.

12. A method of compensated decoding for rejected subspace components, comprising the steps of:

step 1, generating a fixed compensation vector as a basic compensation vector according to a signal model which is sensed and decoded, or generating a self-adaptive compensation vector as a basic compensation vector according to a main component copy which is decoded in a current frame;

step 2, placing each basic compensation vector in a basic compensation vector set, and generating a one-to-one corresponding serial number or characteristic parameter for each basic compensation vector;

step 3, reading the code stream file or directly reading the compensation code stream from the communication channel;

step 4, decoding the compensation coding code stream to obtain the serial numbers or characteristic parameters of all basic compensation vectors for compensation;

and 5, selecting corresponding basic compensation vectors from the basic compensation vector set according to the sequence numbers or the characteristic parameters, and replacing the discarded subspace components with the selected basic compensation vectors.

13. The method of compensated decoding for discarded subspace components according to claim 12, wherein said set of base compensation vectors comprises a set of fixed base compensation vectors and/or a set of adaptive base compensation vectors; the fixed compensation vector is placed in a fixed basic compensation vector set; the adaptive compensation vector is placed in an adaptive base compensation vector set.

14. The compensation decoding method for a discarded subspace component according to claim 12 or 13, wherein said fixed compensation vector is generated according to one or more of a sinusoidal signal model, a random noise signal model and a HILN signal model, and does not vary with a variation of decoded data per frame; the adaptive compensation vector varies following the variation of the decoded data per frame.