TW202447609A - Scene audio signal encoding method and electronic device - Google Patents

Abstract

The present application discloses a scene audio signal encoding method. The decoding method includes: receiving and decoding a first bitstream to obtain a first reconstructed signal, attribute information of a target virtual speaker, and a higher order energy gain coding result, where the first reconstructed signal is a reconstructed signal of a first audio signal in the scene audio signal; generating, based on the attribute information of the target virtual speaker and the first audio signal, a virtual speaker signal corresponding to the target virtual speaker; performing reconstruction based on the attribute information of the target virtual speaker, so as to obtain a first reconstructed scene audio signal; determining a fading factor according to a frequency band sequence number of a reconstructed signal in the first reconstructed scene audio signal and/or an order of the first reconstructed scene audio signal; and adjusting the first reconstructed scene audio signal according to the higher order energy gain coding result and the attenuation factor, so as to obtain a reconstructed scene audio signal.

Description

Scene audio decoding method and electronic device

The present invention relates to the field of audio decoding, and more particularly to a scene audio decoding method and electronic device.

Three-dimensional audio technology is an audio technology that acquires, processes, transmits, renders and replays sound events and three-dimensional sound field information in the real world through computers and signal processing. Three-dimensional audio gives sound a strong sense of space, enclosure and immersion, giving people an extraordinary auditory experience of "sound immersion". Among them, the Higher Order Ambisonics (HOA) technology has the properties of being independent of the speaker layout in the recording, encoding and replay stages, as well as the rotatable replay characteristics of HOA format data. It has higher flexibility in three-dimensional audio replay, and has therefore received more extensive attention and research.

For an N-order HOA signal, the corresponding number of channels is (N+1) ² . As the HOA order increases, the information used to record more detailed sound scenes in the HOA signal will also increase; however, the amount of data in the HOA signal will also increase, and a large amount of data will cause difficulties in transmission and storage, so the HOA signal needs to be encoded and decoded. However, the existing technology has the problem of low accuracy in reconstructing HOA signals.

The present invention provides a scene audio encoding and decoding method and an electronic device.

In a first aspect, an embodiment of the present invention provides a scene audio coding method, the method comprising: obtaining a scene audio signal to be encoded, the scene audio signal comprising audio signals of C1 channels, C1 being a positive integer; obtaining attribute information of a target virtual speaker corresponding to the scene audio signal; obtaining a high-order energy gain of the scene audio signal; encoding the high-order energy gain to obtain a high-order energy gain coding result; encoding a first audio signal in the scene audio signal, the attribute information of the target virtual speaker and the high-order energy gain coding result to obtain a first bit stream; wherein the first audio signal is an audio signal of K channels in the scene audio signal, K being a positive integer less than or equal to C1.

In one possible manner, the scene audio signal is an N1-order high-order stereo reverberation HOA signal, the N1-order HOA signal includes a second audio signal, the second audio signal is an audio signal in the N1-order HOA signal except the first audio signal, and C1 is equal to the square of (N1+1); obtaining the high-order energy gain of the scene audio signal includes: obtaining the high-order energy gain according to feature information of the second audio signal and feature information of the first audio signal.

Exemplarily, the N1-order HOA signal includes the second audio signal, which can be understood as the N1-order HOA signal only includes the second audio signal.

Exemplarily, the N1-order HOA signal includes the second audio signal, which can be understood as the N1-order HOA signal includes the second audio signal and other audio signals.

Exemplarily, the first audio signal may be referred to as the low-level portion of the scene audio signal, and the second audio signal may be referred to as the high-level portion of the scene audio signal. In other words, a portion of the low-level portion of the scene audio signal and a portion of the high-level portion of the scene audio signal may be encoded.

It should be understood that, compared with the N1-order HOA signal including the second audio signal, when the N1-order HOA signal only includes the first audio signal, the number of channels of the encoded N1-order HOA signal is smaller and the corresponding bit rate is lower.

In one possible manner, obtaining the high-order energy gain based on the feature information of the second audio signal and the feature information of the first audio signal includes: obtaining the energy gain of the first audio signal and the energy gain of the second audio signal; obtaining the high-order energy gain based on the energy gain of the first audio signal and the energy gain of the second audio signal.

In one possible manner, obtaining the high-order energy gain according to the energy gain of the first audio signal and the energy gain of the second audio signal includes: obtaining the high-order energy gain Gain'(i, b) by the following method:

Gain'(i，b) = 10*log10( );

Wherein, log10 represents the logarithmic function log, * represents the multiplication operation, E(1, b) is the channel energy of the b-th frequency band of the first audio signal, E(i, b) is the i-th channel energy of the b-th frequency band of the second audio signal, i is the number of the i-th channel of the second audio signal, and b is the frequency band sequence number of the second audio signal.

In one possible manner, encoding the high-order energy gain to obtain a high-order energy gain encoding result includes: quantizing the high-order energy gain to obtain a quantized high-order energy gain; and entropy encoding the quantized high-order energy gain to obtain the high-order energy gain encoding result.

It should be noted that the position of the target virtual speaker matches the position of the sound source in the scene audio signal; based on the attribute information of the target virtual speaker and the first audio signal in the scene audio signal, a virtual speaker signal corresponding to the target virtual speaker can be generated; based on the virtual speaker signal and the high-order energy gain coding result, the scene audio signal can be reconstructed. Therefore, the encoder encodes the first audio signal in the scene audio signal, the property information of the target virtual speaker, and the high-order energy gain coding result, and sends them to the decoder. The decoder can obtain the first reconstructed signal (i.e., the reconstructed signal of the first audio signal in the scene audio signal), the property information of the target virtual speaker, and the high-order energy gain coding result based on the decoding, and reconstruct the scene audio signal.

Compared with other methods of reconstructing scene audio signals in the prior art, the scene audio signals reconstructed based on the virtual speaker signals have higher audio quality; therefore, when K is equal to C1, at the same bit rate, the scene audio signals reconstructed by the present invention have higher audio quality.

When K is less than C1, compared with the prior art, the number of channels of the audio signal encoded by the present invention is smaller, and the amount of data of the property information of the target virtual speaker is much smaller than the amount of data of the audio signal of one channel; therefore, under the premise of achieving the same quality, the encoding bit rate of the present invention is lower.

In addition, the prior art converts the scene audio signal into a virtual speaker signal and a residual signal and then encodes the signal. However, the encoding end of the present invention directly encodes the first audio signal in the scene audio signal without calculating the virtual speaker signal and the residual signal, and the encoding complexity of the encoding end is lower.

Exemplarily, the scene audio signal involved in the embodiments of the present invention may refer to a signal used to describe a sound field; wherein the scene audio signal may include: an HOA signal (wherein the HOA signal may include a three-dimensional HOA signal and a two-dimensional HOA signal (also referred to as a planar HOA signal)) and a three-dimensional audio signal; the three-dimensional audio signal may refer to other audio signals in the scene audio signal except the HOA signal.

In one possible manner, when N1 is equal to 1, K may be equal to C1; when N1 is greater than 1, K may be less than C1. It should be understood that when N1 is equal to 1, K may also be less than C1.

Exemplarily, the process of encoding the property information of the first audio signal and the target virtual speaker in the scene audio signal may include operations such as downmixing, transformation, quantization, and entropy coding, which are not limited in the present invention.

Exemplarily, the first code stream may include coded data of a first audio signal in the scene audio signal and coded data of property information of a target virtual speaker.

In one possible manner, a target virtual speaker may be selected from a plurality of candidate virtual speakers based on a scene audio signal, and then the property information of the target virtual speaker may be determined. Exemplarily, the virtual speakers (including the candidate virtual speakers and the target virtual speakers) are virtual speakers, not real speakers.

Exemplarily, a plurality of candidate virtual speakers may be evenly distributed on a sphere, and the number of target virtual speakers may be one or more.

In one possible approach, a preset target virtual speaker may be obtained, and then the property information of the target virtual speaker may be determined.

It should be understood that the present invention is not limited to the manner of determining the target virtual speaker.

In a second aspect, an embodiment of the present invention provides a scene audio decoding method, the scene audio decoding method comprising: receiving a first code stream; decoding the first code stream to obtain a first reconstructed signal, property information of a target virtual speaker, and a high-order energy gain coding result, wherein the first reconstructed signal is a reconstructed signal of a first audio signal in a scene audio signal, the scene audio signal comprises an audio signal of C1 channels, the first audio signal is an audio signal of K channels in the scene audio signal, C1 is a positive integer, and K is a positive integer less than or equal to C1; based on the property information of the target virtual speaker and the first audio signal, generating a virtual speaker signal corresponding to the target virtual speaker; reconstructing based on the attribute information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer; determining an attenuation factor according to a frequency band sequence number of a reconstructed signal in the first reconstructed scene audio signal and/or an order of the first reconstructed scene audio signal; adjusting the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain a reconstructed scene audio signal.

In one possible manner, the scene audio signal is an N1-order high-order stereo reverberation HOA signal, the N1-order HOA signal includes a second audio signal, the second audio signal is an audio signal in the N1-order HOA signal except the first audio signal, and C1 is equal to the square of (N1+1); and/or, the first reconstructed scene audio signal is an N2-order HOA signal, the N2-order HOA signal includes a third audio signal, the third audio signal is a reconstructed signal in the N2-order HOA signal corresponding to each channel of the second audio signal, and C2 is equal to the square of (N2+1).

In one possible manner, the adjusting the first reconstructed scene audio signal according to the high-order energy gain encoding result and the attenuation factor to obtain the reconstructed scene audio signal includes: entropy decoding the high-order energy gain encoding result to obtain the high-order energy gain after entropy decoding; dequantizing the high-order energy gain after entropy decoding to obtain the high-order energy gain; adjusting the high-order energy gain according to the characteristic information of the second audio signal and the characteristic information of the first audio signal to obtain the adjusted decoded high-order energy gain; adjusting the third audio signal in the N2-order HOA signal according to the adjusted decoded high-order energy gain and the attenuation factor to obtain the adjusted third audio signal, and the adjusted third audio signal belongs to the reconstructed scene audio signal. In the above scheme, after the adjusted decoded high-order energy gain is obtained, the gain of the third audio signal of the current frame can be weighted, and the gain is attenuated along with the frequency band number of the third audio signal and/or the order of the N2-order HOA signal. The attenuation factor can be first obtained according to the frequency band number of the third audio signal and/or the order of the N2-order HOA signal. For example, the attenuation factor can be attenuated along with the two factors of frequency band and Ambisonic order, and then the adjusted decoded high-order energy gain and the obtained attenuation factor are applied to the high-order channel of the third audio signal reconstructed in the current frame, so that the energy of the high-order channel is more uniform and smooth, thereby improving the quality of the reconstructed audio signal.

In one possible manner, the adjusting the high-order energy gain according to the characteristic information of the second audio signal and the characteristic information of the first audio signal includes: obtaining the high-order energy of the second audio signal according to the channel energy of the first audio signal and the high-order energy gain; obtaining a decoding energy proportional factor according to the channel energy of the third audio signal and the high-order energy of the second audio signal; obtaining a decoding high-order energy gain of the third audio signal according to the channel energy of the third audio signal and the channel energy of the first audio signal; and adjusting the decoding high-order energy gain of the third audio signal according to the decoding energy proportional factor to obtain the adjusted decoding high-order energy gain. In the above scheme, in order to make the energy of the high-order channel more uniform and smooth, the decoding high-order energy gain of the third audio signal is adjusted using the decoding energy proportional factor, and the adjusted decoding high-order energy gain is determined. After the adjustment using the decoding energy proportional factor, the energy of the high-order channel is more uniform and smooth, and the quality of the reconstructed audio signal is better.

In one possible manner, determining the attenuation factor according to the frequency band number of the reconstructed signal in the first reconstructed scene audio signal and/or the order of the first reconstructed scene audio signal includes: obtaining the attenuation factor according to the frequency band number of the third audio signal and/or the order of the N2-order HOA signal. In the above scheme, the decoding end can obtain the attenuation factor according to the frequency band number where the third audio signal is located, or the decoding end can obtain the attenuation factor according to the order of the N2-order HOA signal, and the order of the N2-order HOA signal can specifically be the Ambisonic order, or the decoding end can obtain the attenuation factor according to the above-mentioned frequency band number and the order of the N2-order HOA signal, and the attenuation factor can be called a double attenuation factor.

In one possible manner, after adjusting the third audio signal in the N2-order HOA signal according to the high-order energy gain coding result and the attenuation factor, the method further includes: obtaining the channel energy of the fourth audio signal corresponding to the adjusted third audio signal, the third audio signal includes the audio signal of the current frame, and the fourth audio signal includes the audio signal of the previous frame of the current frame; and adjusting the adjusted third audio signal again according to the channel energy of the fourth audio signal. In the above scheme, the decoding end can also use the previous frame of the third audio signal to adjust the adjusted third audio signal of the current frame again, so as to improve the quality of the reconstructed audio signal.

In one possible manner, the adjusting the adjusted third audio signal again according to the channel energy of the fourth audio signal includes: obtaining a channel energy average value of the fourth audio signal and a channel energy of the adjusted third audio signal; obtaining an energy average threshold according to the channel energy average value of the fourth audio signal and the channel energy of the adjusted third audio signal; performing weighted averaging calculation on the channel energy average value of the fourth audio signal and the channel energy of the adjusted third audio signal according to the energy average threshold to obtain a target energy; obtaining an energy smoothing factor according to the target energy and the channel energy of the adjusted third audio signal; and adjusting the third audio signal according to the energy smoothing factor. In the above scheme, the adjusted third audio signal is further adjusted by using the energy smoothing factor q(i, b) to further improve the decoding quality of the third audio signal.

In one possible manner, obtaining the attenuation factor according to the frequency band number of the third audio signal and/or the order of the N2-order HOA signal includes:

The attenuation factor g'(i, b) is obtained as follows:

Wherein, i is the number of the i-th channel of the third audio signal, b is the frequency band number of the third audio signal, represents the number of the target virtual speakers, represents the number of mapping channels of the N2-order HOA signal, , M is the order of the N2-order HOA signal, γ represents the order corresponding to the channel number i of the third audio signal, Represents a multiplication operation.

In the above scheme, through the above attenuation factor g'(i, b) calculation method, b is the frequency band number of the third audio signal, Indicates the number of target virtual speakers. Indicates the number of mapping channels of N2-order HOA signals. , M is the order of the N2-order HOA signal, γ represents the order corresponding to the channel number i of the third audio signal. The attenuation factor can be accurately calculated through the above parameters. The attenuation factor changes with the number of speakers, the number of HOA signal mapping channels, and the HOA order through parameter adjustment, so that when the attenuation factor and the adjusted decoded high-order energy gain are used to adjust the third audio signal, the quality of the reconstructed audio signal is improved.

In one possible manner, the method further comprises: when b≤d, Updated to , ; d is the default first threshold; when b＞d, the Updated to , .

In one possible manner, the method further comprises: Updated to , ; wherein bands represents the number of frequency bands of the third audio signal.

In the above scheme, when the frequency band b is less than the first threshold value, that is to say, the bth frequency band can represent the low frequency band, then the attenuation coefficient is set to a smaller value, for example, the attenuation coefficient is equal to 0.375, and when the frequency band b is greater than the threshold value, that is to say, the bth frequency band can represent the high frequency band, then the attenuation coefficient is set to a larger value, for example, the attenuation coefficient is equal to 0.5. The above attenuation coefficient of 0.375 or 0.5 is only a possible example implementation method, for example, the above 0.375 can also be replaced by 0.38 or 0.37, and the above 0.5 can also be replaced by 0.55 or 0.6. The specific value of the attenuation coefficient needs to be determined in combination with the application scenario, and is not limited here. In the above scheme, the value of band b can be adjusted flexibly. The value of is set so that the attenuation effect becomes more significant as the frequency band becomes higher, so that the reconstructed audio signal is more in line with the hearing characteristics of the human ear.

In one possible manner, the method further comprises: when b≤d, Updated to , ; d is the default first threshold, w is the default adjustment ratio threshold.

In the above scheme, when b ≤ the first threshold, the above scheme can be flexibly adjusted according to the value of the frequency band b. The value of is set so that the attenuation effect becomes more significant as the frequency band becomes higher, so that the reconstructed audio signal is more in line with the hearing characteristics of the human ear.

In one possible manner, the method further comprises: updating w to w2, w2=w+ ×0.05. In the above scheme, Update the value of w so that the weight w in the attenuation factor is consistent with the parameter Build relationships, As w increases, the attenuation effect becomes more significant as the frequency band increases, making the reconstructed audio signal more in line with the auditory characteristics of the human ear.

In one possible manner, when the value of i is 0, 1, 2, or 3, the value of γ is 1; when the value of i is 4, 5, 6, 7, or 8, the value of γ is 2; when the value of i is 9, 10, 11, 12, 13, 14, or 15, the value of γ is 3. It can be seen from the above value of γ that the attenuation factor is determined by the value of γ, and the value of γ is a piecewise function related to the HOA order of the channel. As the HOA order of i increases, the value of γ increases, but will not exceed the maximum HOA order. The attenuation factor is used to adjust the third audio signal to improve the quality of the reconstructed audio signal.

In one possible manner, the adjusting the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor includes: entropy decoding the high-order energy gain coding result to obtain the high-order energy gain after entropy decoding; dequantizing the high-order energy gain after entropy decoding to obtain the high-order energy gain; obtaining the high-order energy of the second audio signal according to the channel energy of the first audio signal and the high-order energy gain; and adjusting the first reconstructed scene audio signal according to the high-order energy gain. The channel energy of the third audio signal and the high-order energy of the second audio signal are used to obtain a decoding energy proportional factor; the dispersion factor of the first reconstructed scene audio signal is determined according to the first bitstream; the attenuation factor is linearly weighted according to the dispersion factor to obtain a weighted attenuation factor; the third audio signal in the N2-order HOA signal is adjusted according to the weighted attenuation factor and the decoding energy proportional factor to obtain an adjusted third audio signal. In the above scheme, the dispersion factor can also be used for linear weighting of the attenuation factor, and the weighted attenuation factor is used to adjust the third audio signal. A weighted method is used to balance the proportion between the dispersion component and the directional component in the attenuation factor. The dispersion factor can be used to measure the energy ratio of the non-directional component in the HOA signal to be encoded, and is used to adjust the energy of each channel of the reconstructed HOA signal, so that the reconstructed HOA signal after energy adjustment is closer to the energy of the HOA signal to be encoded.

In one possible manner, linearly weighting the attenuation factor according to the divergence factor to obtain a weighted attenuation factor includes: obtaining the weighted attenuation factor gd(i,b) by at least one of the following methods:

gd(i,b)= w diffusion(b)+(1-w) ; Where w is the preset adjustment ratio threshold, represents a multiplication operation, diffusion(b) represents the diffusion factor of the b-th frequency band of the third audio signal, represents an attenuation factor of the b-th frequency band of the third audio signal;

or,

When the attenuation factor is a full-band signal, gd(i,b)=w mean(diffusion)+ (1-w) , wherein mean(diffusion) is the average value of the dispersion factors of multiple frequency bands of the third audio signal, represents the attenuation factor of the bth frequency band of the third audio signal, and w is a preset adjustment proportional threshold;

or,

gd(i,b)= w diffusion(b)+(1-w) +offset(i,b), wherein offset(i,b) represents the offset constant of the bth frequency band on the i-th channel of the third audio signal, represents the attenuation factor of the b-th frequency band of the third audio signal, w is a preset adjustment proportional threshold, and diffusion(b) represents the diffusion factor of the b-th frequency band of the third audio signal;

or,

gd(i,b)= w diffusion(b)+(1-w) +direction(i,b), wherein direction(i,b) represents the direction parameter of the bth frequency band on the i-th channel of the third audio signal, represents the attenuation factor of the b-th frequency band of the third audio signal, w is a preset adjustment proportional threshold, and diffusion(b) represents the diffusion factor of the b-th frequency band of the third audio signal.

In one possible manner, the third audio signal in the N2-order HOA signal is adjusted according to the weighted attenuation factor and the decoding energy proportional factor to obtain an adjusted third audio signal:

The adjusted third audio signal X'(i,b) is obtained by the following method:

X'(i,b) = X(i) g(i,b) gd(i,b);

Wherein, gd(i,b) represents the weighted attenuation factor, g(i,b) represents the decoding energy proportional factor, and X(i) represents the third audio signal.

In the above scheme, since the dispersion factor can be used to measure the energy ratio of the non-directional component in the HOA signal to be encoded, and the decoding energy ratio factor can be used to measure the energy ratio of the reconstructed HOA signal, the dispersion factor and the decoding energy ratio factor are used to adjust the energy of each channel of the reconstructed HOA signal, so that the reconstructed HOA signal after energy adjustment is closer to the energy of the HOA signal to be encoded.

In one possible manner, obtaining the energy average threshold according to the channel energy average of the fourth audio signal and the adjusted channel energy of the third audio signal includes:

The energy average threshold k is obtained as follows:

Wherein, E_mean(i, b) represents the channel energy average value of the fourth audio signal, and E'_dec(i, b) represents the energy of the adjusted third audio signal.

In one possible manner, the energy smoothing factor is obtained according to the target energy and the adjusted channel energy of the third audio signal, including:

The energy smoothing factor q(i, b) is obtained as follows:

q(i, b) = sqrt(E_target(i, b))/sqrt(E’_dec(i, b));

Wherein, E_target(i, b) represents the target energy, and E’_dec(i, b) represents the energy of the adjusted third audio signal.

In one possible manner, the step of adjusting the decoding high-order energy gain of the third audio signal according to the decoding energy proportional factor to obtain the adjusted decoding high-order energy gain includes:

The adjusted decoding high-order energy gain Gain_dec’(i, b) is obtained by the following method:

Gain_dec’(i,b) = w*min(g(i,b),Gain_dec(i,b)) + (1 - w) * g(i,b);

Among them, g(i, b) represents the decoding energy proportional factor, Gain_dec(i, b) represents the decoding high-order energy gain of the third audio signal, w is the default adjustment proportional threshold, min represents the minimum value operation, and * represents the multiplication operation.

Compared with other methods of reconstructing scene audio signals in the prior art, the scene audio signals reconstructed based on the virtual speaker signals have higher audio quality; therefore, when K is equal to C1, at the same bit rate, the scene audio signals reconstructed by the present invention have higher audio quality.

When K is less than C1, in the process of encoding the scene audio signal, the number of channels of the audio signal encoded by the present invention is less than the number of channels of the audio signal encoded by the prior art, and the amount of data of the attribute information of the target virtual speaker is much smaller than the amount of data of the audio signal of one channel; therefore, under the premise of the same bit rate, the audio quality of the reconstructed scene audio signal obtained by decoding of the present invention is higher.

Secondly, since the virtual speaker signal and residual information encoded and transmitted by the existing technology are converted from the original audio signal (i.e., the scene audio signal to be encoded), and are not the original audio signal, errors will be introduced; while the present invention encodes part of the original audio signal (i.e., the audio signals of K channels in the scene audio signal to be encoded), thus avoiding the introduction of errors, thereby being able to improve the audio quality of the reconstructed scene audio signal obtained by decoding; and it is also possible to avoid fluctuations in the reconstruction quality of the reconstructed scene audio signal obtained by decoding, and has high stability.

In addition, since the existing technology encodes and transmits virtual speaker signals, and the data volume of virtual speaker signals is relatively large, the number of target virtual speakers selected by the existing technology is subject to greater bandwidth restrictions. The present invention encodes and transmits attribute information of virtual speakers, and the data volume of attribute information is much smaller than the data volume of virtual speaker signals; therefore, the number of target virtual speakers selected by the present invention is subject to less bandwidth restrictions. The more target virtual speakers selected, the higher the quality of the scene audio signal reconstructed based on the virtual speaker signals of the target virtual speakers. Therefore, compared with the prior art, under the same bit rate, the present invention can select a larger number of target virtual speakers, so that the quality of the reconstructed scene audio signal decoded by the present invention is higher.

In addition, compared with the encoding end and decoding end of the prior art, the encoding end and decoding end of the present invention do not need to perform residual and superposition operations, so the comprehensive complexity of the encoding end and decoding end of the present invention is lower than the comprehensive complexity of the encoding end and decoding end of the prior art.

It should be understood that when the encoding end performs distortion compression on the first audio signal in the scene audio signal, there is a difference between the first reconstructed signal decoded by the decoding end and the first audio signal encoded by the encoding end. When the encoding end performs lossless compression on the first audio signal, the first reconstructed signal decoded by the decoding end is the same as the first audio signal encoded by the encoding end.

It should be understood that when the encoding end performs distortion compression on the attribute information of the target virtual speaker, there is a difference between the attribute information decoded by the decoding end and the attribute information encoded by the encoding end. When the encoding end performs lossless compression on the attribute information of the virtual speaker, the attribute information decoded by the decoding end is the same as the attribute information encoded by the encoding end. In particular, the present invention does not distinguish between the attribute information encoded by the encoding end and the attribute information decoded by the decoding end in terms of name.

The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

Regarding third parties, an embodiment of the present invention provides a method for generating a bit stream, which can generate a bit stream according to the first aspect and any one of the implementation methods of the first aspect.

The third party aspect and any implementation of the third party aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third party aspect and any implementation of the third party aspect can be found in the first aspect and any implementation of the first aspect, which will not be elaborated here.

In a fourth aspect, an embodiment of the present invention provides a scene audio coding device, the device comprising:

An acquisition module, used for acquiring a scene audio signal to be encoded, wherein the scene audio signal includes audio signals of C1 channels, where C1 is a positive integer;

The acquisition module is also used to acquire the property information of the target virtual speaker corresponding to the scene audio signal;

The acquisition module is also used to obtain the high-order energy gain of the scene audio signal;

A coding module, used for coding the high-order energy gain to obtain a high-order energy gain coding result;

The encoding module is also used to encode the first audio signal in the scene audio signal, the attribute information of the target virtual speaker and the high-order energy gain encoding result to obtain a first code stream; wherein the first audio signal is the audio signal of K channels in the scene audio signal, and K is a positive integer less than or equal to C1.

The scene audio coding device of the fourth aspect can execute the steps of the first aspect and any one of the implementations of the first aspect, which will not be described in detail herein.

The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

In a fifth aspect, an embodiment of the present invention provides a scene audio decoding device, the device comprising:

A code stream receiving module, used for receiving a first code stream;

a decoding module, configured to decode the first bit stream to obtain a first reconstructed signal, property information of a target virtual speaker, and a high-order energy gain coding result, wherein the first reconstructed signal is a reconstructed signal of a first audio signal in a scene audio signal, the scene audio signal includes audio signals of C1 channels, the first audio signal is audio signals of K channels in the scene audio signal, C1 is a positive integer, and K is a positive integer less than or equal to C1;

a virtual speaker signal generating module, configured to generate a virtual speaker signal corresponding to the target virtual speaker based on the attribute information of the target virtual speaker and the first audio signal;

A scene audio signal reconstruction modeling group, used for reconstructing based on the attribute information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer;

an attenuation factor determination module, configured to determine an attenuation factor according to a frequency band sequence number of a reconstructed signal in the first reconstructed scene audio signal and/or an order of the first reconstructed scene audio signal;

The scene audio signal adjustment module is used to adjust the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain a reconstructed scene audio signal.

The scene audio decoding device of the fifth aspect can execute the steps of the second aspect and any one of the implementations of the second aspect, which will not be elaborated here.

The fifth aspect and any implementation of the fifth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, which will not be repeated here.

In a sixth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, and when the program instructions are executed by the processor, the electronic device executes the scene audio coding method in the first aspect or any possible implementation of the first aspect.

The sixth aspect and any implementation of the sixth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

In the seventh aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, and when the program instructions are executed by the processor, the electronic device executes the scene audio decoding method in the second aspect or any possible implementation of the second aspect.

The seventh aspect and any implementation of the seventh aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the seventh aspect and any implementation of the seventh aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, which will not be repeated here.

In an eighth aspect, an embodiment of the present invention provides a chip comprising one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals comprising computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the scene audio coding method of the first aspect or any possible implementation of the first aspect.

The eighth aspect and any implementation of the eighth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the eighth aspect and any implementation of the eighth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

In the ninth aspect, an embodiment of the present invention provides a chip comprising one or more interface circuits and one or more processors; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the scene audio decoding method in the second aspect or any possible implementation of the second aspect.

The ninth aspect and any implementation of the ninth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the ninth aspect and any implementation of the ninth aspect can refer to the technical effects corresponding to the second aspect and any implementation of the second aspect, which will not be repeated here.

In a tenth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer or a processor, the computer or the processor executes the scene audio encoding method in the first aspect or any possible implementation of the first aspect.

The tenth aspect and any implementation of the tenth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the tenth aspect and any implementation of the tenth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

In the eleventh aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer or a processor, the computer or the processor executes the scene audio decoding method in the second aspect or any possible implementation of the second aspect.

The eleventh aspect and any implementation of the eleventh aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the eleventh aspect and any implementation of the eleventh aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, which will not be repeated here.

In a twelfth aspect, an embodiment of the present invention provides a computer program product, which includes a software program. When the software program is executed by a computer or a processor, the computer or the processor executes the scene audio coding method in the first aspect or any possible implementation of the first aspect.

The twelfth aspect and any implementation of the twelfth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the twelfth aspect and any implementation of the twelfth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

In a thirteenth aspect, an embodiment of the present invention provides a computer program product, which includes a software program. When the software program is executed by a computer or a processor, the computer or the processor executes the scene audio decoding method in the second aspect or any possible implementation of the second aspect.

The thirteenth aspect and any implementation of the thirteenth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the thirteenth aspect and any implementation of the thirteenth aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, which will not be repeated here.

In a fourteenth aspect, an embodiment of the present invention provides a device for storing a code stream, the device comprising: a receiver and at least one storage medium, the receiver is used to receive the code stream; at least one storage medium is used to store the code stream; the code stream is generated according to the first aspect and any one of the implementation methods of the first aspect.

The fourteenth aspect and any implementation of the fourteenth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourteenth aspect and any implementation of the fourteenth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

In a fifteenth aspect, an embodiment of the present invention provides a device for transmitting a code stream, the device comprising: a transmitter and at least one storage medium, the at least one storage medium is used to store the code stream, the code stream is generated according to the first aspect and any one of the implementation methods of the first aspect; the transmitter is used to obtain the code stream from the storage medium and send the code stream to the end device through the transmission medium.

The fifteenth aspect and any implementation of the fifteenth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifteenth aspect and any implementation of the fifteenth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

In a sixteenth aspect, an embodiment of the present invention provides a system for distributing a bit stream, the system comprising: at least one storage medium for storing at least one bit stream, the at least one bit stream being generated according to the first aspect and any one of the implementation methods of the first aspect, a streaming media device for obtaining a target bit stream from the at least one storage medium and sending the target bit stream to an end device, wherein the streaming media device comprises a content server or a content distribution server.

The sixteenth aspect and any implementation of the sixteenth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixteenth aspect and any implementation of the sixteenth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative labor are within the scope of protection of the present invention.

The term "and/or" in this article is only a description of the association relationship between related objects, indicating that three types of relationships may exist. For example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

The terms "first" and "second" in the description and patent application of the embodiments of the present invention are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

In the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

In the description of the embodiments of the present invention, unless otherwise specified, the meaning of "plurality" refers to two or more. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

In order to make the description of the following embodiments clear and concise, a brief introduction to the relevant technology is first given.

Sound is a continuous wave generated by the vibration of an object. The object that vibrates and emits sound waves is called the sound source. When sound waves propagate through a medium (such as air, solid or liquid), the hearing organs of humans or animals can perceive the sound.

The characteristics of sound waves include pitch, intensity and timbre. Pitch refers to the high or low pitch of a sound. Intensity refers to the size of a sound. Intensity can also be called loudness or volume. The unit of intensity is decibel (dB). Tone is also called timbre.

The frequency of sound waves determines the pitch of the sound. The higher the frequency, the higher the pitch. The number of times an object vibrates in one second is called frequency, and the unit of frequency is Hertz (Hz). The frequency of sound that the human ear can recognize is between 20 Hz and 20,000 Hz.

The amplitude of the sound wave determines the intensity of the sound. The greater the amplitude, the greater the intensity of the sound. The closer to the sound source, the greater the intensity of the sound.

The waveform of the sound wave determines the timbre. The waveform of the sound wave includes square wave, sawtooth wave, sine wave and pulse wave.

According to the characteristics of sound waves, sound can be divided into regular sound and irregular sound. Irregular sound refers to the sound produced by the irregular vibration of the sound source. Irregular sound is, for example, noise that affects people's work, study and rest. Regular sound refers to the sound produced by the regular vibration of the sound source. Regular sound includes speech and music. When sound is represented electrically, regular sound is an analog signal that changes continuously in the time-frequency domain. This analog signal can be called an audio signal. An audio signal is an information carrier that carries speech, music and sound effects.

Since human hearing has the ability to distinguish the positional distribution of sound sources in space, when listeners hear sounds in space, in addition to being able to feel the pitch, intensity and timbre of the sounds, they can also feel the direction of the sounds.

As people pay more and more attention to the experience of hearing systems and demand more quality, three-dimensional audio technology has emerged to enhance the depth, presence and spatial sense of sound. The listener not only feels the sound coming from the front, back, left and right sound sources, but also feels that the space they are in is surrounded by the spatial sound field (abbreviated as "sound field") generated by these sound sources, and the sound diffuses to the surroundings, creating an "immersive" sound effect that makes the listener feel like they are in a theater or concert hall.

The scene audio signal involved in the embodiments of the present invention may refer to a signal used to describe a sound field; wherein the scene audio signal may include: an HOA signal (wherein the HOA signal may include a three-dimensional HOA signal and a two-dimensional HOA signal (also referred to as a planar HOA signal)) and a three-dimensional audio signal; the three-dimensional audio signal may refer to other audio signals in the scene audio signal except the HOA signal. The following is an explanation using the HOA signal as an example.

As we all know, sound waves propagate in an ideal medium with a wave number of , the angular frequency is ,in, is the sound wave frequency, is the speed of sound. Satisfying formula (1, b), is the Laplace operator. (1)

Assume that the space system outside the human ear is a sphere, and the listener is at the center of the sphere. The sound coming from outside the sphere has a projection on the sphere, filtering out the sound outside the sphere. Assume that the sound source is distributed on this sphere, and use the sound field generated by the sound source on the sphere to fit the sound field generated by the original sound source. That is, three-dimensional audio technology is a method of fitting the sound field. Specifically, solve the equation (1, b) in the spherical coordinate system. In the passive spherical area, the equation (1, b) is solved as follows: equation (2). (2)

in, represents the radius of the sphere, Represents horizontal angle information (or azimuth information), Indicates pitch angle information (or elevation information). represents the wave number, represents the amplitude of an ideal plane wave, Indicates the order number of the HOA signal (or called the order number of the HOA signal). represents the spherical Bessel function, which is also called the radial basis function. The first j represents an imaginary unit. Does not change with angle. express , The spherical harmonic function of the direction, The spherical harmonic function represents the direction of the sound source. The HOA signal satisfies formula (3). (3)

Substituting formula (3) into formula (2), formula (2) can be transformed into formula (4). (4)

Among them, m is truncated to the Nth item, that is, m=N, so As an approximate description of the sound field; at this point, It can be called the HOA coefficient (can be used to represent the N-order HOA signal). The sound field refers to the area in the medium where sound waves exist. N is an integer greater than or equal to 1.

The scene audio signal is an information carrier that carries the spatial position information of the sound source in the sound field, and describes the sound field of the listener in space. Formula (4) shows that the sound field can be expanded on the sphere according to the spherical harmonic function, that is, the sound field can be decomposed into the superposition of multiple plane waves. Therefore, the sound field described by the HOA signal can be expressed by the superposition of multiple plane waves, and the sound field can be reconstructed by the HOA coefficient.

The HOA signal to be encoded involved in the embodiment of the present invention may refer to an N1-order HOA signal, which may be represented by an HOA coefficient or an Ambisonic (stereo reverberation) coefficient, where N1 is an integer greater than or equal to 1 (wherein, when N1 is equal to, the 1-order HOA signal may be referred to as a FOA (First Order Ambisonic, first-order stereo reverberation) signal). The N1-order HOA signal includes channels of audio signals.

Fig. 1a is a schematic diagram of an exemplary application scenario. Fig. 1a shows the encoding and decoding scenario of a scene audio signal.

1a, exemplarily, the first electronic device may include a first audio acquisition module, a first scene audio encoding module, a first channel encoding module, a first channel decoding module, a first scene audio decoding module, and a first audio playback module. It should be understood that the first electronic device may include more or fewer modules than those shown in FIG. 1a, and the present invention is not limited thereto.

1a, exemplarily, the second electronic device may include a second audio acquisition module, a second scene audio encoding module, a second channel encoding module, a second channel decoding module, a second scene audio decoding module, and a second audio playback module. It should be understood that the second electronic device may include more or fewer modules than those shown in FIG. 1a, and the present invention is not limited thereto.

Exemplarily, the process in which a first electronic device encodes and transmits a scene audio signal to a second electronic device, and the second electronic device decodes and replays the audio may be as follows: a first audio acquisition module may perform audio acquisition and output the scene audio signal to a first scene audio encoding module. Next, the first scene audio encoding module may encode the scene audio signal and output a code stream to a first channel encoding module. Thereafter, the first channel encoding module may channel-code the code stream and transmit the channel-coded code stream to a second electronic device via a wireless or wired network communication device. Then, a second channel decoding module of the second electronic device may channel-decode the received data to obtain a code stream and output the code stream to a second scene audio decoding module. Next, the second scene audio decoding module can decode the code stream to obtain a reconstructed scene audio signal; and then output the reconstructed scene audio signal to the second audio replay module, which performs audio replay.

It should be noted that the second audio playback module can perform post-processing (such as audio rendering) on the reconstructed scene audio signal (for example, The invention relates to a method for converting a reconstructed scene audio signal of a plurality of channel audio signals into an audio signal having the same number of channels as the number of speakers in the second electronic device), resonant normalization, user interaction, audio format conversion or noise removal, etc., so as to convert the reconstructed scene audio signal into an audio signal suitable for playing by the speakers in the second electronic device.

It should be understood that the process of the second electronic device encoding and transmitting the scene audio signal to the first electronic device, and the first electronic device decoding and audio replaying is similar to the above-mentioned process of the first electronic device transmitting the scene audio signal to the second electronic device, and the second electronic device replaying the audio, and will not be repeated here.

Exemplarily, the first electronic device and the second electronic device may include but are not limited to: personal computers, computer workstations, smart phones, tablet computers, servers, smart cameras, smart cars or other types of cellular phones, media consumption devices, wearable devices, set-top boxes, game consoles, etc.

Exemplarily, the present invention can be specifically applied to VR (Virtual Reality)/AR (Augmented Reality) scenes. In one possible manner, the first electronic device is a server, and the second electronic device is a VR/AR device. In one possible manner, the second electronic device is a server, and the first electronic device is a VR/AR device.

Exemplarily, the first scene audio encoding module and the second scene audio encoding module may be scene audio encoders. The first scene audio decoding module and the second scene audio decoding module may be scene audio decoders.

For example, when the first electronic device encodes the scene audio signal and the second electronic device reconstructs the scene audio signal, the first electronic device can be called the encoding end and the second electronic device can be called the decoding end. When the second electronic device encodes the scene audio signal and the first electronic device reconstructs the scene audio signal, the second electronic device can be called the encoding end and the first electronic device can be called the decoding end.

Fig. 1b is a schematic diagram of an exemplary application scenario. Fig. 1b shows a transcoding scenario of a scene audio signal.

Referring to FIG. 1 b ( 1 ), illustratively, the wireless or core network device may include: a channel decoding module, other audio decoding modules, a scene audio encoding module and a channel encoding module. The wireless or core network device may be used for audio transcoding.

Exemplarily, the specific application scenario of Figure 1b (1) may be: when the first electronic device is not provided with a scene audio encoding module but only with other audio encoding modules; and the second electronic device is only provided with a scene audio decoding module but not with other audio decoding modules, in order to enable the second electronic device to decode and replay the scene audio signal encoded by the first electronic device using other audio encoding modules, wireless or core network equipment may be used for transcoding.

Specifically, the first electronic device uses other audio encoding modules to encode the scene audio signal to obtain a first code stream; and the first code stream is channel-encoded and sent to the wireless or core network device. Then, the channel decoding module of the wireless or core network device can perform channel decoding and output the first code stream decoded by the channel to other audio decoding modules. Afterwards, other audio decoding modules decode the first code stream to obtain the scene audio signal and output the scene audio signal to the scene audio encoding module. Then, the scene audio encoding module can encode the scene audio signal to obtain a second code stream and output the second code stream to the channel encoding module, and the channel encoding module channel-encodes the second code stream and sends it to the second electronic device. In this way, the second electronic device can call the scene audio decoding module to decode the second code stream obtained by channel decoding to obtain a reconstructed scene audio signal; subsequently, the reconstructed scene audio signal can be replayed.

Referring to FIG. 1 b ( 2 ), illustratively, the wireless or core network device may include: a channel decoding module, a scene audio decoding module, other audio encoding modules and a channel encoding module. The wireless or core network device may be used for audio transcoding.

Exemplarily, the specific application scenario of Figure 1b (2) may be: when the first electronic device is only provided with a scene audio encoding module and no other audio encoding modules are provided; and the second electronic device is not provided with a scene audio decoding module and only provided with other audio decoding modules, in order to enable the second electronic device to decode and replay the scene audio signal encoded by the scene audio encoding module of the first electronic device, a wireless or core network device may be used for transcoding.

Specifically, the first electronic device uses a scene audio encoding module to encode the scene audio signal to obtain a first code stream; and the first code stream is channel-encoded and sent to the wireless or core network device. Then, the channel decoding module of the wireless or core network device can perform channel decoding and output the first code stream decoded by the channel to the scene audio decoding module. Afterwards, the scene audio decoding module decodes the first code stream to obtain a scene audio signal and outputs the scene audio signal to other audio encoding modules. Then, other audio encoding modules can encode the scene audio signal to obtain a second code stream and output the second code stream to the channel encoding module, and the channel encoding module channel-encodes the second code stream and sends it to the second electronic device. In this way, the second electronic device can call other audio decoding modules to decode the second code stream obtained by channel decoding to obtain a reconstructed scene audio signal; subsequently, the reconstructed scene audio signal can be replayed.

The following describes the encoding and decoding process of scene audio signals.

FIG. 2a is a schematic diagram showing an exemplary encoding process.

S201, obtaining a scene audio signal to be encoded, wherein the scene audio signal includes audio signals of C1 channels, where C1 is a positive integer.

For example, when the scene audio signal is an HOA signal, the HOA signal may be an N1-order HOA signal, that is, the N1-term in the above formula (3) .

For example, the N1-order HOA signal may include C1 channels of audio signals, where C1= For example, when N1=3, the N1-order HOA signal includes 16 channels of audio signals; when N1=4, the N1-order HOA signal includes 25 channels of audio signals.

S202, obtaining property information of a target virtual speaker corresponding to the scene audio signal.

Based on the scene audio signal, a target virtual speaker is selected from multiple candidate virtual speakers, and property information of the target virtual speaker is obtained.

S203, obtaining a high-order energy gain of the scene audio signal.

Exemplarily, characteristic information of the HOA signal is obtained from the HOA signal to be encoded, and a high-order energy gain is obtained through the characteristic information of the HOA signal. The high-order energy gain can be used to indicate the energy gain of the high-order channel signal of the scene audio signal.

Exemplarily, the scene audio signal includes audio signals of C1 channels, the first audio signal is audio signals of K channels in the scene audio signal, K is a positive integer less than or equal to C1, and the value of K is not limited.

The scene audio signal is an N1-order HOA signal, the N1-order HOA signal includes a first audio signal and a second audio signal, the second audio signal is an audio signal in the N1-order HOA signal except the first audio signal, and C1 is equal to the square of (N1+1).

In a possible manner, assuming that N1=3 and C1=10, the N1-order HOA signal includes 1st to 16th channel audio signals, the first audio signal is the 1st to 10th channel audio signals in the N1-order HOA signal, and the second audio signal is the 11th to 16th channel audio signals in the N1-order HOA signal.

Exemplarily, N1=3, C1=9. The N1-order HOA signal includes 1st to 16th channel audio signals, the first audio signal is the 1st to 9th channel audio signals in the N1-order HOA signal, and the second audio signal is the 10th to 16th channel audio signals in the N1-order HOA signal.

Exemplarily, N1=3, C1=8. The N1-order HOA signal includes the 1st to 16th channel audio signals, the first audio signal is the 1st to 6th and 8th, 9th channel audio signals in the N1-order HOA signal, and the second audio signal is the 7th and 10th to 16th channel audio signals in the N1-order HOA signal.

In a possible implementation, obtaining a high-order energy gain of a scene audio signal includes:

A high-order energy gain is obtained according to the feature information of the second audio signal and the feature information of the first audio signal.

The scene audio signal includes a first audio signal and a second audio signal, and feature information of the second audio signal and feature information of the first audio signal are obtained respectively. Feature information corresponding to the scene audio signal includes but is not limited to: gain information and diffusion information. The high-order energy gain of the scene audio signal can be obtained according to the feature information of the second audio signal and the feature information of the first audio signal.

Exemplarily, the gain information Gain(i, b) of the second audio signal in the scene audio signal may be calculated according to the following formula:

Among them, i is the number of the i-th channel contained in the second audio signal in the scene audio signal, and the number can also be called the channel number, b is the frequency band number of the second audio signal, E(i, b) is the i-th channel energy of the b-th frequency band of the second audio signal, and E(1, b) is the channel energy of the b-th frequency band of the first audio signal. For example, the channel of the first audio signal can specifically be the first channel of the N1-order HOA signal.

The following steps can be performed in a frame signal or on a subframe. The following steps can be performed in the full band or on a subband.

For example, after Gain(i, b) is calculated, Gain'(i, b) is calculated as follows:

Gain’(i, b) = 10*log10(Gain(i, b)).

S204, encoding the high-order energy gain to obtain a high-order energy gain encoding result.

After the encoder obtains the high-order energy gain of the scene audio signal, it can encode the high-order energy gain to generate a high-order energy gain encoding result. The role of the high-order energy gain is to adjust the high-order channel energy at the decoder to make the HOA signal encoding and decoding quality higher.

S205, encode the attribute information and high-order energy gain encoding result of the first audio signal and the target virtual speaker in the scene audio signal to obtain a first bit stream; wherein the first audio signal is the audio signal of K channels in the scene audio signal, and K is a positive integer less than or equal to C1.

Illustratively, a virtual speaker is a virtual speaker, not a real speaker.

For example, based on the above, the scene audio signal can be expressed by superposition of multiple plane waves, and then the target virtual speaker used to simulate the sound source in the scene audio signal can be determined; in this way, in the subsequent decoding process, the virtual speaker signal corresponding to the target virtual speaker is used to reconstruct the scene audio signal.

In one possible approach, a plurality of candidate virtual speakers at different positions may be arranged on a sphere; then, a target virtual speaker whose position matches the position of a sound source in a scene audio signal may be selected from the plurality of candidate virtual speakers.

Fig. 2b is a schematic diagram of candidate virtual speaker distribution. In Fig. 2b, multiple candidate virtual speakers can be evenly distributed on a sphere, and a point on the sphere represents a candidate virtual speaker.

It should be noted that the present invention does not limit the number and distribution of candidate virtual speakers, which can be set according to needs, as will be described in detail later.

Exemplarily, based on the scene audio signal, a target virtual speaker whose position corresponds to the sound source position in the scene audio signal can be selected from the multiple candidate virtual speakers; wherein the number of target virtual speakers can be one or more, and the present invention is not limited thereto.

In one possible approach, a target virtual speaker may be pre-set.

It should be understood that the present invention is not limited to the manner of determining the target virtual speaker.

Exemplarily, in one possible manner, during the decoding process, the scene audio signal can be reconstructed based on the virtual speaker signal; however, directly transmitting the virtual speaker signal of the target virtual speaker will increase the bit rate; and the virtual speaker signal of the target virtual speaker can be generated based on the attribute information of the target virtual speaker and the scene audio signal of some or all channels; therefore, the attribute information of the target virtual speaker and the audio signals of K channels in the scene audio signal can be obtained as the first audio signal; then the first audio signal, the attribute information of the target virtual speaker and the high-order energy gain encoding result are encoded to obtain a first bit stream.

Exemplarily, the first audio signal and the property information of the target virtual speaker may be downmixed, transformed, quantized, and entropy encoded to obtain a first bitstream, and the high-order energy gain encoding result may be written into the first bitstream. In other words, the first bitstream may include the encoding data of the first audio signal in the scene audio signal, the encoding data of the property information of the target virtual speaker, and the high-order energy gain encoding result.

Compared with other methods of reconstructing scene audio signals in the prior art, the scene audio signals reconstructed based on the virtual speaker signals have higher audio quality; therefore, when K is equal to C1, at the same bit rate, the scene audio signals reconstructed by the present invention have higher audio quality.

When K is less than C1, in the process of encoding the scene audio signal, the number of channels of the audio signal encoded by the present invention is less than the number of channels of the audio signal encoded by the prior art, and the amount of data of the attribute information of the target virtual speaker is also much smaller than the amount of data of the audio signal of one channel; therefore, under the premise of achieving the same quality, the encoding bit rate of the present invention is lower.

In addition, the prior art converts the scene audio signal into a virtual speaker signal and a residual signal and then encodes the signal. However, the encoding end of the present invention directly encodes the audio signals of some channels in the scene audio signal without calculating the virtual speaker signal and the residual signal, and the encoding complexity of the encoding end is lower.

Fig. 3 is a schematic diagram of an exemplary decoding process. Fig. 3 is a decoding process corresponding to the encoding process of Fig. 2.

S301, receiving a first code stream.

S302: Decode the first bit stream to obtain a first reconstructed signal and property information of a target virtual speaker.

Exemplarily, the coded data of the first audio signal in the scene audio signal included in the first bitstream can be decoded to obtain the first reconstructed signal; that is, the first reconstructed signal is a reconstructed signal of the first audio signal. And the coded data of the property information of the target virtual speaker included in the first bitstream can be decoded to obtain the property information of the target virtual speaker.

It should be understood that when the encoding end performs distortion compression on the first audio signal in the scene audio signal, there is a difference between the first reconstructed signal decoded by the decoding end and the first audio signal encoded by the encoding end. When the encoding end performs lossless compression on the first audio signal, the first reconstructed signal decoded by the decoding end is the same as the first audio signal encoded by the encoding end.

It should be understood that when the encoding end performs distortion compression on the attribute information of the target virtual speaker, there is a difference between the attribute information decoded by the decoding end and the attribute information encoded by the encoding end. When the encoding end performs lossless compression on the attribute information of the virtual speaker, the attribute information decoded by the decoding end is the same as the attribute information encoded by the encoding end. In particular, the present invention does not distinguish between the attribute information encoded by the encoding end and the attribute information decoded by the decoding end in terms of name.

S303: Generate a virtual speaker signal corresponding to the target virtual speaker based on the property information of the target virtual speaker and the first reconstructed signal.

S304: Reconstruct based on the property information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal. The first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer.

For example, the scene audio signal can be reconstructed based on the virtual speaker signal; then, the virtual speaker signal corresponding to the target virtual speaker can be generated based on the property information of the target virtual speaker and the first reconstructed signal. One target virtual speaker corresponds to one virtual speaker signal, and the virtual speaker signal is a plane wave. Then, the scene audio signal is reconstructed based on the property information of the target virtual speaker and the virtual speaker signal to generate a first reconstructed scene audio signal.

Exemplarily, when the scene audio signal is an HOA signal, the reconstructed first reconstructed scene audio signal may also be an HOA signal, and the HOA signal may be an N2-order HOA signal, where N2 is a positive integer. Exemplarily, the N2-order HOA signal may include audio signals of C2 channels, where C2= .

Exemplarily, the order N2 of the first reconstructed scene audio signal may be greater than or equal to the order N1 of the scene audio signal in the embodiment of FIG. 2a ; correspondingly, the number of channels C2 of the audio signal included in the first reconstructed scene audio signal may be greater than or equal to the number of channels C1 of the audio signal included in the scene audio signal in the embodiment of FIG. 2a .

Exemplarily, the scene audio signal is an N1-order HOA signal, the N1-order HOA signal includes a second audio signal, the second audio signal is an audio signal in the N1-order HOA signal except the first audio signal, and C1 is equal to the square of (N1+1); and/or,

The first reconstructed scene audio signal is an N2-order HOA signal, the N2-order HOA signal includes a third audio signal, the third audio signal is a reconstructed signal corresponding to each channel of the second audio signal in the N2-order HOA signal, and C2 is equal to the square of (N2+1).

In one possible approach, the first reconstructed scene audio signal can be directly used as the final decoding result.

S305: Determine an attenuation factor according to a frequency band number of a reconstructed signal in the first reconstructed scene audio signal and/or an order of the first reconstructed scene audio signal.

For example, the decoder can obtain the attenuation factor according to the frequency band number of the reconstructed signal in the first reconstructed scene audio signal, or the decoder can obtain the attenuation factor according to the order of the first reconstructed scene audio signal, where the order of the first reconstructed scene audio signal can specifically be the order of an N2-order HOA signal, such as an Ambisonic order, or the decoder can obtain the attenuation factor according to the above-mentioned frequency band number and the order of the first reconstructed scene audio signal, and the attenuation factor can be called a double attenuation factor. The attenuation factor can be used to attenuate according to at least one of the frequency band number of the reconstructed signal and/or the order of the first reconstructed scene audio signal. The attenuation factor can be used to adjust the first reconstructed scene audio signal to make the quality of the reconstructed scene audio signal higher.

S306: Adjust the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain a reconstructed scene audio signal.

The decoding end obtains the high-order energy gain coding result from the first bit stream, and uses the high-order energy gain coding result and the attenuation factor to adjust the energy of the first reconstructed scene audio signal. The decoding end uses the high-order energy gain coding result to adjust the high-order channel energy of the first reconstructed scene audio signal, so that the decoding quality of the scene audio signal is higher.

Compared with other methods of reconstructing scene audio signals in the prior art, the scene audio signals reconstructed based on the virtual speaker signals have higher audio quality; therefore, when K is equal to C1, at the same bit rate, the scene audio signals reconstructed by the present invention have higher audio quality.

When K is less than C1, in the process of encoding the scene audio signal, the number of channels of the audio signal encoded by the present invention is less than the number of channels of the audio signal encoded by the prior art, and the amount of data of the attribute information of the target virtual speaker is much smaller than the amount of data of the audio signal of one channel; therefore, under the premise of the same bit rate, the audio quality of the reconstructed scene audio signal obtained by decoding of the present invention is higher.

Secondly, since the virtual speaker signal and residual information encoded and transmitted by the existing technology are converted from the original audio signal (i.e., the scene audio signal to be encoded), and are not the original audio signal, errors will be introduced; while the present invention encodes part of the original audio signal (i.e., the audio signals of K channels in the scene audio signal to be encoded), avoiding the introduction of errors, thereby improving the audio quality of the reconstructed scene audio signal obtained by decoding; and it can also avoid the fluctuation of the reconstruction quality of the reconstructed scene audio signal obtained by decoding, and has high stability.

In addition, since the existing technology encodes and transmits virtual speaker signals, and the data volume of virtual speaker signals is relatively large, the number of target virtual speakers selected by the existing technology is subject to greater bandwidth restrictions. The present invention encodes and transmits attribute information of virtual speakers, and the data volume of attribute information is much smaller than the data volume of virtual speaker signals; therefore, the number of target virtual speakers selected by the present invention is subject to less bandwidth restrictions. The more target virtual speakers selected, the higher the quality of the scene audio signal reconstructed based on the virtual speaker signals of the target virtual speakers. Therefore, compared with the prior art, under the same bit rate, the present invention can select a larger number of target virtual speakers, so that the quality of the reconstructed scene audio signal decoded by the present invention is higher.

In addition, the integrated coding end and decoding end do not need to perform residual and superposition operations compared to the coding end and decoding end of the prior art, so the comprehensive complexity of the coding end and decoding end of the present invention is lower than that of the coding end and decoding end of the prior art. Since the first bit stream sent by the coding end includes the high-order energy gain coding result, the high-order energy gain can be used to adjust the high-order channel energy at the decoding end, so that the coding and decoding quality of the scene audio signal is higher.

The following describes the encoding process of the high-order energy gain in the encoding process and the adjustment process of the high-order energy gain on the audio signal in the decoding process.

FIG. 4 is a schematic diagram showing an exemplary encoding process.

S401, obtaining a scene audio signal to be encoded, wherein the scene audio signal includes audio signals of C1 channels, where C1 is a positive integer.

Exemplarily, S401 may refer to the description of S201 above, which will not be repeated here.

S402, obtaining property information of a target virtual speaker corresponding to the scene audio signal.

In one possible manner, based on the position information of the target virtual speaker, the attribute information of the target virtual speaker is generated. In one possible manner, the position information of the target virtual speaker (including the pitch angle information and the horizontal angle information) can be used as the attribute information of the target virtual speaker. In one possible manner, the position index corresponding to the position information of the target virtual speaker (including the pitch angle index (which can be used to uniquely identify the pitch angle information) and the horizontal angle index (which can be used to uniquely identify the horizontal angle information)) is used as the attribute information of the target virtual speaker.

In a possible manner, a virtual speaker index (eg, virtual speaker identification) of the target virtual speaker may be used as the attribute information of the target virtual speaker, wherein the virtual speaker index corresponds to the position information one by one.

In one possible manner, the virtual speaker coefficient of the target virtual speaker can be used as the attribute information of the target virtual speaker. Exemplarily, C2 virtual speaker coefficients of the target virtual speaker can be determined, and the C2 virtual speaker coefficients of the target virtual speaker can be used as the attribute information of the target virtual speaker; wherein the C2 virtual speaker coefficients of the target virtual speaker correspond one-to-one to the audio signal of C2 channels included in the first reconstructed scene audio signal.

It should be noted that the amount of data of virtual speaker coefficients is much larger than the amount of data of location information, location information index, and virtual speaker index; based on the bandwidth, it can be decided which of the location information, location information index, virtual speaker index, and virtual speaker coefficients to use as the attribute information of the target virtual speaker. For example, when the bandwidth is large, the virtual speaker coefficients can be used as the attribute information of the target virtual speaker; in this way, the decoder does not need to calculate the virtual speaker coefficients of the target virtual speaker, which can save the computing power of the decoder. When the bandwidth is small, any one of the position information, the index of the position information, and the virtual speaker index can be used as the attribute information of the target virtual speaker; in this way, the bit rate can be saved. It should be understood that it is also possible to pre-set which of the position information, the index of the position information, the virtual speaker index, and the virtual speaker coefficient is used as the attribute information of the target virtual speaker; the present invention is not limited to this.

S403: Obtain energy gain of the first audio signal and energy gain of the second audio signal.

The feature information corresponding to the scene audio signal includes gain information. The scene audio signal includes a first audio signal and a second audio signal. The energy gain E(1, b) of the first audio signal and the energy gain E(i, b) of the second audio signal are calculated respectively.

S404: Obtain a high-order energy gain according to the energy gain of the first audio signal and the energy gain of the second audio signal.

After the encoder obtains the high-order energy gain of the scene audio signal, it can encode the high-order energy gain to generate a high-order energy gain encoding result. The role of the high-order energy gain is to adjust the high-order channel energy at the decoder to make the HOA signal encoding and decoding quality higher.

Exemplarily, obtaining a high-order energy gain according to an energy gain of a first audio signal and an energy gain of a second audio signal comprises:

The high-order energy gain Gain’(i, b) is obtained as follows:

Gain'(i，b) = 10*log10( );

Wherein, log10 represents the logarithmic function log, * represents the multiplication operation, E(1, b) is the channel energy of the first audio signal, E(i, b) is the energy of each channel of the second audio signal, i is the number of the i-th channel of the second audio signal, and b is the frequency band number of the second audio signal.

Exemplarily, the characteristic information of the second audio signal may be the high-order energy gain of the N1-order HOA signal, specifically the energy ratio of each channel of the second audio signal to the W channel (the first channel of the N1-order HOA signal), and the W channel may specifically be the channel of the first audio signal.

Exemplarily, the characteristic information of the second audio signal may be obtained by referring to the following steps:

The N1-order HOA signal is subjected to time-frequency transformation, and the time-domain N1-order HOA signal is transformed into a frequency-domain N1-order HOA signal.

Calculate W channel energies E(1, b) and each channel energy E(i, b) of the second audio signal, where i is the channel number of the second audio signal.

The following formula can be used to calculate the high-order energy gain Gain’(i, b):

;

Gain’(i,b) = 10*log10(Gain(i,b)).

S405: quantize the high-order energy gain to obtain a quantized high-order energy gain.

S406: Perform entropy coding on the quantized high-order energy gain to obtain the high-order energy gain coding result.

The characteristic information of the second audio signal in the scene audio signal is obtained, the high-order energy gain of the scene audio signal is obtained through the characteristic information of the second audio signal, and the high-order energy gain is sequentially quantized and entropy encoded.

Exemplarily, scalar quantization may be used to quantize high-order energy gains.

The quantized high-order energy gain is entropy-coded. The entropy coding method is not limited.

Exemplarily, the high-order energy gain is differentially encoded, and then the number of bits of entropy coding is estimated. If the estimated number of bits is less than the fixed-length coding, the high-order energy gain is variable-length encoded, such as Huffman coding; otherwise, the high-order energy gain is fixed-length encoded.

After obtaining the high-order energy gain coding result, the coding result is written into the bitstream.

S407, encoding the first audio signal and the property information of the target virtual speaker and the high-order energy gain encoding result in the scene audio signal to obtain a first bit stream.

It should be understood that the number of channels of the audio signal included in the first audio signal can be determined according to demand and bandwidth, and the present invention does not impose any limitation on this.

Exemplarily, S401 may refer to the description of S201 above, which will not be repeated here.

In the embodiment of the present invention, the encoder can calculate the energy ratio of the high-order channel to the W channel to obtain the high-order energy gain coding result, and then select Huffman coding or direct coding according to the bit number estimation of the difference result between subframes. As a result, the first bit stream sent by the encoder includes the high-order energy gain coding result, so the high-order energy gain can be used to adjust the high-order channel energy at the decoder, so that the encoding and decoding quality of the scene audio signal is higher.

Fig. 5a is a schematic diagram of an exemplary decoding process. Fig. 5a is a decoding process corresponding to the encoding process in Fig. 4.

S501, receiving a first code stream.

S502, decoding the first bit stream to obtain the first reconstructed signal and the property information of the target virtual speaker and the high-order energy gain coding result.

S503, generating a virtual speaker signal corresponding to the target virtual speaker based on the attribute information of the target virtual speaker and the first audio signal;

S504, reconstructing based on the property information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer.

For example, S501 to S504 can refer to the description of S301 to S304, which will not be repeated here.

Exemplarily, the above S306 can refer to the description of S505~S508.

S505, entropy decoding is performed on the high-order energy gain encoding result to obtain the high-order energy gain after entropy decoding.

S506, dequantizing the high-order energy gain after entropy decoding to obtain the high-order energy gain.

Exemplarily, a high-order energy gain coding result is read from the first bitstream. Entropy decoding is performed on the high-order energy gain coding result. The entropy decoding method is the inverse process of entropy coding at the encoding end.

For example, if the encoding end adopts fixed-length coding, the decoding end uses the corresponding fixed-length decoding. If the encoding end adopts coding coding, the decoding end uses the corresponding edge-length decoding, such as Huffman decoding.

The entropy decoding result is dequantized, and the dequantization method is the inverse process of the quantization method at the encoding end.

S507, adjusting the high-order energy gain according to the characteristic information of the second audio signal and the characteristic information of the first audio signal to obtain an adjusted decoding high-order energy gain.

Among them, the decoding end performs signal reconstruction, and after obtaining the first reconstructed scene audio signal, the first audio signal and the third audio signal are determined from the first reconstructed scene audio signal, the third audio signal is a reconstructed signal corresponding to each channel of the second audio signal in the N2-order HOA signal, and the characteristic information of the second audio signal is determined according to the characteristic information of the third audio signal. Finally, the high-order energy gain is adjusted according to the characteristic information of the second audio signal and the characteristic information of the first audio signal to obtain the adjusted decoded high-order energy gain. The high-order energy gain is adjusted so that the high-order channel energy is more uniform and smooth, and the quality of the reconstructed audio signal is better.

Exemplarily, S507 adjusts the high-order energy gain according to the characteristic information of the second audio signal and the characteristic information of the first audio signal, including:

S5071, obtaining high-order energy of the second audio signal according to the channel energy and high-order energy gain of the first audio signal;

The decoding end obtains the high-order energy gain coding result from the first bit stream, performs entropy decoding on the high-order energy gain coding result, and dequantizes it to obtain the high-order energy gain. Then, the energy of the second audio signal is estimated according to the channel energy and the high-order energy gain of the first audio signal to determine the high-order energy of the second audio signal.

Exemplarily, the first reconstructed scene audio signal is an N2-order HOA signal, and the N2-order HOA signal is subjected to a time-frequency transformation, and the time-domain N2-order HOA signal is transformed into a frequency-domain N2-order HOA signal.

The high-order energy E_Ref(i, b) of the second audio signal can be calculated using the following formula:

E_Ref(i,b) = E_dec(1,b) * 10^(Gain’(i,b)/10)

Wherein, E_dec(1, b) is the channel energy of the bth frequency band of the first audio signal in the N2-order HOA signal, i is the channel number corresponding to the second audio signal, Gain’(i, b) is the high-order energy gain, and b is the frequency band number of the first audio signal.

S5072: Obtain a decoding energy proportional factor according to the channel energy of the third audio signal and the high-order energy of the second audio signal.

Specifically, the third audio signal is a reconstructed signal corresponding to each channel of the second audio signal in the N2-order HOA signal, and the decoding energy ratio factor is obtained by calculating the energy ratio of the third audio signal and the second audio signal.

Exemplarily, the decoding energy proportional factor g(i, b) may be calculated using the following formula:

g(i,b) = sqrt(E_Ref(i,b)) / sqrt(E_dec(i,b))

Wherein, sqrt() is a square root operation, E_dec(i, b) is the channel energy of the b-th frequency band of the third audio signal, i is the channel number corresponding to the third audio signal, and E_Ref(i, b) is the high-order energy of the b-th frequency band of the second audio signal.

S5073: Obtain a decoded high-order energy gain of the third audio signal according to the channel energy of the third audio signal and the channel energy of the first audio signal.

The channel energy of the first audio signal is used as a reference to calculate the gain of the channel energy of the third audio signal to obtain the decoded high-order energy gain of the third audio signal.

Exemplarily, the following formula may be used to calculate the decoding high-order energy gain Gain_dec(i, b):

Gain_dec(i,b) = E_dec(i,b)/E_dec(1,b)

Wherein, E_dec(1, b) is the channel energy of the b-th frequency band of the first audio signal in the N2-order HOA signal, and E_dec(i, b) is the channel energy of the b-th channel of the third audio signal.

S5074, adjusting the decoding high-order energy gain of the third audio signal according to the decoding energy proportional factor to obtain an adjusted decoding high-order energy gain.

Specifically, in order to make the energy of the high-order channel more uniform and smooth, the decoding high-order energy gain of the third audio signal is adjusted using the decoding energy proportional factor, and the adjusted decoding high-order energy gain is determined. After the decoding energy proportional factor is adjusted, the energy of the high-order channel is more uniform and smooth, and the quality of the reconstructed audio signal is better.

Exemplarily, adjusting the decoding high-order energy gain of the third audio signal according to the decoding energy proportional factor to obtain the adjusted decoding high-order energy gain includes:

The adjusted decoding high-order energy gain Gain_dec’(i, b) is obtained as follows:

Gain_dec’(i,b) = w*min(g(i,b),Gain_dec(i,b)) + (1 - w) * g(i,b);

Wherein, g(i, b) represents the decoding energy proportional factor, Gain_dec(i, b) represents the decoding high-order energy gain of the b-th frequency band of the third audio signal, w is the default adjustment proportional threshold, min represents the minimum value operation, and * represents the multiplication operation.

For example, min(a,b) is the minimum value of a and b, and w is the adjustment ratio threshold. There are many ways to take the value of w, for example, the value of w is 0.25.

S508: Adjust the third audio signal in the N2-order HOA signal according to the adjusted decoded high-order energy gain to obtain an adjusted third audio signal.

The decoding end obtains the high-order energy gain coding result from the first bit stream, and uses the high-order energy gain coding result to adjust the energy of the third audio signal in the N2-order HOA signal. The decoding end uses the high-order energy gain coding result to adjust the high-order channel energy of the third audio signal, so that the decoding quality of the third audio signal is higher.

The third audio signal is a channel audio signal corresponding to each channel of the second audio signal in the N2-order HOA signal.

Exemplarily, the third audio signal may be adjusted based on the feature information corresponding to the second audio signal in the N1-order HOA signal to improve the quality of the N2-order HOA signal.

Exemplarily, S508 adjusts the third audio signal in the N2-order HOA signal according to the adjusted decoded high-order energy gain, including:

S5081: Obtain an attenuation factor according to the frequency band number of the third audio signal and/or the order of the N2-order HOA signal.

For example, the decoding end can obtain the attenuation factor according to the frequency band number where the third audio signal is located, or the decoding end can obtain the attenuation factor according to the order of the N2-order HOA signal, and the order of the N2-order HOA signal can specifically be the Ambisonic order, or the decoding end can obtain the attenuation factor according to the above-mentioned frequency band number and the order of the N2-order HOA signal, and the attenuation factor can be called a double attenuation factor.

S5082, adjusting the third audio signal according to the adjusted decoded high-order energy gain and attenuation factor to obtain an adjusted third audio signal, wherein the adjusted third audio signal belongs to the reconstructed scene audio signal.

Among them, after obtaining the adjusted decoded high-order energy gain, the gain of the third audio signal of the current frame can be weighted, and the gain can be attenuated along with the frequency band sequence number of the third audio signal and/or the order of the N2-order HOA signal. The attenuation factor can be first obtained according to the frequency band sequence number of the third audio signal and/or the order of the N2-order HOA signal. For example, the attenuation factor can be attenuated along with the two factors of frequency band and Ambisonic order, and then the adjusted decoded high-order energy gain and the obtained attenuation factor are applied to the high-order channel of the third audio signal reconstructed in the current frame, so that the energy of the high-order channel is more uniform and smooth, and the quality of the reconstructed audio signal is improved.

Exemplarily, the third audio signal is adjusted using the adjusted decoded high-order energy gain Gain_dec’(i, b) and attenuation factor g’(i, b).

For example, the following formula may be used for adjustment:

X’(i,b) = X(i,b) * Gain_dec’(i,b) * g’(i,b);

Wherein, X(i, b) is the third audio signal before adjustment, and X'(i, b) is the third audio signal after adjustment.

In one possible manner, S5081 obtains the attenuation factor according to the frequency band number of the third audio signal and the order of the N2-order HOA signal, including:

The attenuation factor g'(i, b) is obtained as follows:

Wherein, i is the number of the i-th channel of the third audio signal, b is the frequency band number of the third audio signal, represents the number of the target virtual speakers, represents the number of mapping channels of the N2-order HOA signal, , M is the order of the N2-order HOA signal, γ represents the order corresponding to the channel number i of the third audio signal, Represents a multiplication operation.

Exemplarily, b is the frequency band number of the third audio signal, and the frequency band number can also be called the sub-band number, b=0, 1, 2, …, 11.

By calculating the attenuation factor g'(i, b) as above, b is the frequency band number of the third audio signal. Indicates the number of target virtual speakers. Indicates the number of mapping channels of N2-order HOA signals. , M is the order of the N2-order HOA signal, γ represents the order corresponding to the channel number i of the third audio signal. The attenuation factor can be accurately calculated through the above parameters. The attenuation factor changes with the number of speakers, the number of HOA signal mapping channels, and the HOA order through parameter adjustment, so that when the attenuation factor and the adjusted decoded high-order energy gain are used to adjust the third audio signal, the quality of the reconstructed audio signal is improved.

In one possible manner, the scene audio signal decoding method provided by the embodiment of the present invention further includes:

When b≤d, Updated to , ; d is the default first threshold;

When b＞d, Updated to , .

For example, d is a preset first threshold value, and there is no limitation on the value of the first threshold value. When b≤d, , When b＞d, we can The value of is reduced to , . When the frequency band b is less than the first threshold, that is to say, the bth frequency band can represent the low frequency band, then the attenuation coefficient is set to a smaller value, for example, the attenuation coefficient is equal to 0.375. When the frequency band b is greater than the threshold, that is to say, the bth frequency band can represent the high frequency band, then the attenuation coefficient is set to a larger value, for example, the attenuation coefficient is equal to 0.5. The above attenuation coefficient of 0.375 or 0.5 is only a possible example implementation method. For example, the above 0.375 can also be replaced by 0.38 or 0.37, and the above 0.5 can also be replaced by 0.55 or 0.6. The specific value of the attenuation coefficient needs to be determined in combination with the application scenario, and is not limited here. In the above scheme, the value of band b can be adjusted flexibly. The value of is set so that the attenuation effect becomes more significant as the frequency band becomes higher, so that the reconstructed audio signal is more in line with the hearing characteristics of the human ear.

For example, Indicates the number of target virtual speakers, with a value of 0, 1, 2, 3…N. For example 2, 3, 4. Further adjustment: When subband b ≤ the first threshold, , otherwise .

In one possible manner, the scene audio signal decoding method provided by the embodiment of the present invention further includes:

will be described Updated to , ;

Bands represents the number of frequency bands of the third audio signal, that is, bands represents the number of sub-bands, and for example, the value of bands can be configured as 12. This is only an example for illustration and is not intended to limit the embodiments of the present invention.

In the embodiment of the present invention, The value of is updated to , , that is, by The value of is expanded so that the value of the attenuation factor is related to the i-th frequency band, achieving different attenuation efficiencies for the low-band and the high-band, so that when the attenuation factor is used to adjust the third audio signal, the quality of the reconstructed audio signal is improved.

In one possible manner, the scene audio signal decoding method provided by the embodiment of the present invention further includes:

When b≤d, Updated to , ; d is the default first threshold, w is the default adjustment ratio threshold.

For example, Indicates the number of mapping channels of HOA signals, and its value is , M is the HOA signal order, The value is 0, 1, 2, 3...N. When sub-band b ≤ the first threshold, When b ≤ the first threshold, the above scheme can be flexibly adjusted according to the value of the frequency band b. The value of is set so that the attenuation effect becomes more significant as the frequency band becomes higher, so that the reconstructed audio signal is more in line with the hearing characteristics of the human ear.

In one possible manner, the scene audio signal decoding method provided by the embodiment of the present invention further includes: updating w to w2, where w2=w+ ×0.05.

For example, the method of obtaining the value of w is as follows. The initial value of w is 0, and the values are traversed in sequence. Take the value and calculate the final adjustment value of w. For example, update w to w2, w2 = w + × 0.05. Update the value of w so that the weight w in the attenuation factor is consistent with the parameter Build relationships, As w increases, the attenuation effect becomes more significant as the frequency band increases, making the reconstructed audio signal more in line with the auditory characteristics of the human ear.

In one possible approach, when the value of i is 0, 1, 2, or 3, the value of γ is 1;

When the value of i is 4, 5, 6, 7, or 8, the value of γ is 2;

When the value of i is 9, 10, 11, 12, 13, 14, or 15, the value of γ is 3;

Wherein, i is the number of the i-th channel of the third audio signal.

Exemplarily, γ represents the order of the HOA signal in the i-th channel, and γ and i satisfy the following Table 1: Table 1 is a corresponding relationship table of γ and i: Table 1 i 0 1 1 1 2 1 3 1 4 2 5 2 6 2 7 2 8 2 9 3 10 3 11 3 12 3 13 3 14 3 15 3

It can be seen from the above-mentioned value of γ that the attenuation factor is determined by the value of γ. The value of γ is a piecewise function related to the HOA order of the channel. As the HOA order of i increases, the value of γ increases, but will not exceed the maximum HOA order. This attenuation factor is used to improve the quality of the reconstructed audio signal when adjusting the third audio signal.

Exemplarily, after S5082 adjusts the third audio signal in the N2-order HOA signal according to the adjusted decoded high-order energy gain and attenuation factor, the method further includes:

S5083, obtaining channel energy of a fourth audio signal corresponding to the adjusted third audio signal, wherein the third audio signal includes an audio signal of a current frame, and the fourth audio signal includes an audio signal of a previous frame of the current frame;

S5084: adjust the adjusted third audio signal again according to the channel energy of the fourth audio signal.

Among them, the decoding end can also use the previous frame of the third audio signal to adjust the third audio signal after the adjustment of the current frame again, so as to improve the quality of the reconstructed audio signal. The third audio signal includes the audio signal of the current frame, and the fourth audio signal includes the audio signal of the previous frame of the current frame. For example, the previous frame can be an audio signal of a previous frame adjacent to the current frame, or the previous frame can also be an audio signal of a previous frame not adjacent to the current frame. The channel energy of the fourth audio signal can be used to adjust the third audio signal. For example, the decoding end linearly weights the high-order channel of the current frame of the third audio signal and the high-order channel corresponding subbands of the previous two frames to obtain the high-order channel of the current frame after energy smoothing.

Exemplarily, S5084 adjusts the third audio signal according to the channel energy of the fourth audio signal, including:

S50841, obtaining a channel energy average value of the fourth audio signal and an adjusted channel energy of the third audio signal;

The average value of the channel energy of the fourth audio signal may be an average value of all channel energies of the fourth audio signal.

S50842: Obtain an energy average threshold according to the channel energy average of the fourth audio signal and the channel energy of the third audio signal.

The energy average threshold is a threshold obtained by calculating the channel energies of the third audio signal and the fourth audio signal.

Exemplarily, obtaining the energy average threshold according to the channel energy average of the fourth audio signal and the channel energy of the third audio signal includes:

The energy average threshold k is obtained as follows:

The following formula can be used:

Wherein, E_mean(i) represents the channel energy average value of the fourth audio signal, and E’_dec(i) represents the energy of the adjusted third audio signal.

S50843, performing weighted average calculation on the channel energy average value of the fourth audio signal and the adjusted channel energy of the third audio signal according to the energy average threshold to obtain a target energy;

To calculate the target energy E_target(i, b), the following formula can be used:

E_target(i,b) = k * E_mean(i,b) + (1-k)*E’_dec(i,b);

Among them, E_mean(i, b) is the average value of the energy in the previous frame, and E’_dec(i, b) is the energy of the adjusted third audio signal.

S50844, obtaining an energy smoothing factor according to the target energy and the adjusted channel energy of the third audio signal;

The energy smoothing factor can be used to adjust the third audio signal so that the decoding quality of the third audio signal is higher.

Exemplarily, obtaining an energy smoothing factor according to the target energy and the adjusted channel energy of the third audio signal includes:

The energy smoothing factor q(i, b) is obtained as follows:

q(i, b) = sqrt(E_target(i, b))/sqrt(E’_dec(i, b));

Wherein, E_target(i, b) represents the target energy, and E’_dec(i, b) represents the energy of the third audio signal.

S50845. Adjust the third audio signal according to the energy smoothing factor.

The adjusted third audio signal is further adjusted by using the energy smoothing factor q(i,b) to further improve the decoding quality of the third audio signal.

Exemplarily, the third audio signal may be adjusted according to the following formula:

X’’(i, b) = X’(i, b) * q(i, b);

Exemplarily, after obtaining the adjusted third audio signal, the energy of the adjusted third audio signal may be used to update the average value of the energy of the previous frame.

Fig. 5b is a schematic diagram of an exemplary decoding process. Fig. 5b is a decoding process corresponding to the encoding process in Fig. 4.

S501, receiving a first code stream.

S502, decoding the first bit stream to obtain the first reconstructed signal and the property information of the target virtual speaker and the high-order energy gain coding result.

S503, generating a virtual speaker signal corresponding to the target virtual speaker based on the attribute information of the target virtual speaker and the first audio signal;

S504, reconstructing based on the property information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer.

For example, S501 to S504 can refer to the description of S301 to S304, which will not be repeated here.

Exemplarily, the above S306 can refer to the description of S505~S508.

S505, entropy decoding is performed on the high-order energy gain encoding result to obtain the high-order energy gain after entropy decoding.

S506, dequantizing the high-order energy gain after entropy decoding to obtain the high-order energy gain.

Exemplarily, a high-order energy gain coding result is read from the first bitstream. Entropy decoding is performed on the high-order energy gain coding result. The entropy decoding method is the reverse process of the entropy coding at the encoding end.

For example, if the encoding end adopts fixed-length coding, the decoding end uses the corresponding fixed-length decoding. If the encoding end adopts coding coding, the decoding end uses the corresponding edge-length decoding, such as Huffman decoding.

The entropy decoding result is dequantized, and the dequantization method is the inverse process of the quantization method at the encoding end.

S509, determining a diffusion factor of the first reconstructed scene audio signal according to the first bitstream.

Since the reconstructed HOA signal is calculated from the virtual speaker signal, it only contains the sound source component with a clear direction in the HOA signal to be encoded, also called the directional component, and lacks the environmental component in the HOA signal to be encoded, also called the non-directional component. Therefore, by adjusting the energy of each channel of the reconstructed HOA signal through the audio gain, it can only make the energy of the directional component closer to the energy of the HOA signal to be encoded, while the adjustment of the non-directional component is unstable.

The dispersion can be used as a parameter to describe the degree of diffusion of the HOA signal, and is applied to the HOA signal sound field reconstruction and sound field replay. In the embodiment of the present invention, the dispersion of the HOA signal is used to measure the energy ratio of the non-directional components in the HOA signal to be encoded, and is used to adjust the energy of each channel of the reconstructed HOA signal, so that the reconstructed HOA signal after energy adjustment is closer to the energy of the HOA signal to be encoded. Using the dispersion as a parameter for adjusting the energy of the reconstructed HOA signal, the dispersion can also use its own ability to describe the directionality of the sound source to self-adjust the non-directional components when adjusting the energy of the reconstructed HOA signal, and not compensate for the directional components. Therefore, the dispersion can make up for the shortcomings of the gain adjustment method based on the signal energy ratio without introducing additional errors.

In the embodiment of the present invention, the divergence is used to adjust the energy gain so as to reconstruct the HOA signal energy more accurately.

In a possible implementation, S509 determines the dispersion factor of the first reconstructed scene audio signal according to the first bitstream, including:

The dispersion factor is obtained by decoding the first code stream, wherein the first code stream includes the dispersion factor.

Specifically, the dispersion is calculated according to the HOA signal to be encoded at the encoding end, the dispersion is quantized and encoded into the first bit stream, the decoding end decodes the first bit stream, and then the dispersion is obtained by inverse quantization, which is then used to adjust the gain of the uncoded channel in the reconstructed HOA signal.

In a possible implementation, S509 determines the dispersion factor of the first reconstructed scene audio signal according to the first bitstream, including:

The dispersion factor is obtained according to the first reconstructed signal decoded from the first code stream.

There are many methods for calculating the dispersion. At the decoding end, the dispersion is calculated based on the low-order HOA signal in the decoded transmission channel, and then used to adjust the gain of the uncoded channel in the reconstructed HOA signal.

In a possible implementation, S509 obtains the dispersion factor according to the first reconstructed signal obtained by decoding the first bit stream, including:

S5091, obtaining the sound field intensity of each frequency band in the first reconstructed signal;

S5092, obtaining energy of each frequency band;

S5093: Determine the dispersion factor according to the sound field intensity of each frequency band and the energy of each frequency band.

Exemplarily, S5091 obtains the sound field intensity of each frequency band in the first reconstructed signal, including:

The sound field intensity of the bth frequency band is obtained as follows: :

Among them, Re represents the real part operation, Represents four channel signals of the bth frequency band in the first reconstructed signal.

Exemplarily, the energy of each frequency band in S5092 can be obtained by calculating the sum of the square of the real part and the square of the imaginary part of each frequency band.

S5093 determines the dispersion factor according to the sound field intensity of each frequency band and the energy of each frequency band, including:

The dispersion factor is obtained by: :

Among them, E represents the expected operation. Express Express I finds the second norm, represents the energy of the bth frequency band.

It is understandable that the calculation of the divergence in the above S5091 to S5093 is only an example implementation method, and is not intended to limit the embodiments of the present invention.

S510: Determine an attenuation factor according to a frequency band sequence number of a reconstructed signal in a first reconstructed scene audio signal and/or an order of the first reconstructed scene audio signal, and linearly weight the attenuation factor according to a dispersion factor to obtain a weighted attenuation factor.

Specifically, the dispersion factor can also be used to linearly weight the attenuation factor, and the weighted attenuation factor is used to adjust the third audio signal. The weighting method is used to balance the proportion between the dispersion component and the directional component in the attenuation factor. Since the dispersion factor can be used to measure the energy ratio of the non-directional component in the HOA signal to be encoded, and is used to adjust the energy of each channel of the reconstructed HOA signal, the reconstructed HOA signal after energy adjustment is closer to the energy of the HOA signal to be encoded.

There is no limitation on the specific implementation method of linearly weighting the attenuation factor by the dispersion factor, and an example is given as follows:

In one possible implementation, the attenuation factor is linearly weighted according to the dispersion factor to obtain a weighted attenuation factor, including:

The weighted attenuation factor gd(i,b) is obtained by at least one of the following methods:

gd(i,b)= w diffusion(b)+(1-w) ; Where w is the preset adjustment ratio threshold, represents the multiplication operation, diffusion(b) represents the diffusion factor of the bth frequency band of the third audio signal, represents the attenuation factor of the b-th frequency band of the third audio signal;

or,

When the attenuation factor is full-band signal, gd(i,b)=w mean(diffusion)+ (1-w) , where mean(diffusion) is the average value of the dispersion factors of multiple frequency bands of the third audio signal, represents the attenuation factor of the bth frequency band of the third audio signal, and w is the preset adjustment proportional threshold;

or,

gd(i,b)= w diffusion(b)+(1-w) +offset(i,b), where offset(i,b) represents the offset constant of the bth frequency band on the i-th channel of the third audio signal. represents the attenuation factor of the b-th frequency band of the third audio signal, w is the preset adjustment proportional threshold, and diffusion(b) represents the diffusion factor of the b-th frequency band of the third audio signal;

or,

gd(i,b)= w diffusion(b)+(1-w) +direction(i,b), where direction(i,b) represents the direction parameter of the bth frequency band on the i-th channel of the third audio signal. represents the attenuation factor of the b-th frequency band of the third audio signal, w is the default adjustment proportional threshold, and diffusion(b) represents the diffusion factor of the b-th frequency band of the third audio signal.

It is understandable that the weighted calculation of the attenuation factor in the above example is only an example implementation method, and is not intended to limit the embodiments of the present invention.

S511, obtaining high-order energy of the second audio signal according to the channel energy and high-order energy gain of the first audio signal; obtaining a decoding energy proportional factor according to the channel energy of the third audio signal and the high-order energy of the second audio signal.

Specifically, the third audio signal is a reconstructed signal corresponding to each channel of the second audio signal in the N2-order HOA signal, and the decoding energy ratio factor is obtained by calculating the energy ratio of the third audio signal and the second audio signal.

Exemplarily, the decoding energy proportional factor g(i, b) may be calculated using the following formula:

g(i,b) = sqrt(E_Ref(i,b)) / sqrt(E_dec(i,b));

Wherein, sqrt() is a square root operation, E_dec(i, b) is the channel energy of the b-th frequency band of the third audio signal, i is the channel number corresponding to the third audio signal, and E_Ref(i, b) is the high-order energy of the b-th frequency band of the second audio signal.

S512: Adjust the third audio signal in the N2-order HOA signal according to the weighted attenuation factor and the decoding energy proportional factor to obtain an adjusted third audio signal.

The decoder adjusts the third audio signal in the N2-order HOA signal according to the weighted attenuation factor and the decoding energy proportional factor. The decoder uses the weighted attenuation factor and the decoding energy proportional factor to adjust the high-order channel energy of the third audio signal, so that the decoding quality of the third audio signal is higher.

In one possible implementation, the third audio signal in the N2-order HOA signal is adjusted according to the weighted attenuation factor and the decoding energy proportional factor to obtain an adjusted third audio signal:

The adjusted third audio signal X’(i,b) is obtained by the following method:

X'(i,b) = X(i) g(i,b) gd(i,b);

Wherein, gd(i,b) represents the weighted attenuation factor, g(i,b) represents the decoding energy proportional factor, and X(i) represents the third audio signal.

In the above scheme, since the dispersion factor can be used to measure the energy ratio of the non-directional component in the HOA signal to be encoded, and the decoding energy ratio factor can be used to measure the energy ratio of the reconstructed HOA signal, the dispersion factor and the decoding energy ratio factor are used to adjust the energy of each channel of the reconstructed HOA signal, so that the reconstructed HOA signal after energy adjustment is closer to the energy of the HOA signal to be encoded.

It is understandable that the adjustment calculation of the third audio signal in the above example is only an example implementation method, and is not intended to limit the embodiments of the present invention.

Take an example as follows, the input signal in the encoding end is a 3rd order HOA signal, and the 3rd order HOA signal includes a total of 16 channels of audio signals, the first audio signal is the audio signal of the 1st to 5th channels, the 7th channel, and the 9th to 10th channels, and the second audio signal is the audio signal of the 6th channel, the 8th channel, and the 11th to 16th channels. There are three ways to implement the bit stream encoded by the encoding end: 1. The bit stream obtained by encoding does not include the high-order energy gain coding result. 2. The bit stream obtained by encoding includes the high-order energy gain coding result. There are three ways to implement the scene audio decoding method executed by the decoding end: 1. When the received bit stream does not include the high-order energy gain coding result, the decoding end reconstructs the scene audio signal in the bit stream. 2. When the received bitstream includes the high-order energy gain coding result, the decoding end reconstructs the scene audio signal in the bitstream and adjusts the reconstructed scene audio signal according to the high-order energy gain coding result to obtain the reconstructed scene audio signal. 3. In the embodiment of the present invention, when the received bitstream includes the high-order energy gain coding result, the decoding end reconstructs the scene audio signal in the bitstream and adjusts the reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain the reconstructed scene audio signal.

By analyzing the signal quality of the reconstructed scene audio signal, it can be seen that the decoded HOA signal without the high-order energy gain coding result has very poor quality. The decoded HOA signal with the high-order energy gain coding result, but without adjusting the reconstructed scene audio signal through the attenuation factor, has medium quality. The decoded HOA signal with the high-order energy gain coding result and with the reconstructed scene audio signal adjusted through the attenuation factor has the best quality.

Through the above analysis, it can be seen that in the embodiment of the present invention, the decoder can adjust the reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor, so that the high-order channel energy of the reconstructed scene audio signal is more uniform and smooth, and the quality of the reconstructed scene audio signal is better. For example, the attenuation factor can be attenuated according to the two factors of the frequency band and the Ambisonic order of the reconstructed scene audio signal, which effectively improves the encoding and decoding quality of the HOA signal.

FIG. 6a is a schematic diagram showing the structure of an encoding end.

Parameter Figure 6a, exemplary, the encoding end may include a configuration unit, a virtual speaker generation unit, a target speaker generation unit, and a core encoder. It should be understood that Figure 6a is only an example of the present invention, and the encoding end of the present invention may include more or less modules than those shown in Figure 6a, which will not be repeated here.

Exemplarily, the configuration unit may be used to determine configuration information of a candidate virtual speaker.

Exemplarily, the virtual speaker generation unit may be used to generate a plurality of candidate virtual speakers according to the configuration information of the candidate virtual speakers and determine the virtual speaker coefficient corresponding to each candidate virtual speaker.

Exemplarily, the target speaker generation unit may be configured to select a target virtual speaker from a plurality of candidate virtual speakers and determine property information of the target virtual speaker based on a scene audio signal and a plurality of sets of virtual speaker coefficients.

Exemplarily, the core encoder can be used to obtain the high-order energy gain of the scene audio signal and obtain the high-order energy gain encoding result; encode the first audio signal in the scene audio signal, the attribute information of the target virtual speaker and the high-order energy gain encoding result.

Exemplarily, the scene audio coding module in FIG. 1a and FIG. 1b may include the configuration unit, the virtual speaker generation unit, the target speaker generation unit, and the core encoder of FIG. 6a; or, may only include the core encoder.

FIG6b is a schematic diagram showing the structure of a decoding end.

6b, exemplarily, the decoding end may include a core decoder, a virtual speaker coefficient generating unit, a virtual speaker signal generating unit, a reconstruction unit and a signal adjustment unit. It should be understood that FIG6b is only an example of the present invention, and the decoding end of the present invention may include more or less modules than those shown in FIG6b, which will not be repeated here.

Exemplarily, the core decoder can be used to decode the first code stream to obtain a first reconstructed signal, property information of a target virtual speaker, and a high-order energy gain coding result.

Exemplarily, the virtual speaker coefficient generation unit may be configured to determine the virtual speaker coefficient based on the property information of the target virtual speaker.

Exemplarily, the virtual speaker signal generating unit may be configured to generate a virtual speaker signal based on the first reconstructed signal and the virtual speaker coefficient.

Exemplarily, the reconstruction unit may be configured to perform reconstruction based on the virtual speaker signal and the attribute information to obtain a first reconstructed scene audio signal.

Exemplarily, the signal adjustment unit can be used to determine the attenuation factor according to the frequency band number of the reconstructed signal in the first reconstructed scene audio signal and/or the order of the first reconstructed scene audio signal; adjust the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain the reconstructed scene audio signal.

Exemplarily, the scene audio decoding module in FIG. 1a and FIG. 1b may include the core decoder, the virtual speaker coefficient generation unit, the virtual speaker signal generation unit, the reconstruction unit and the signal adjustment unit of FIG. 6b; or, may only include the core decoder.

FIG7 is a schematic diagram of the structure of a scene audio coding device. The scene audio coding device in FIG7 can be used to execute the coding method of the aforementioned embodiment. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding method provided above, which will not be repeated here. The scene audio coding device may include:

An acquisition module 701 is used to acquire a scene audio signal to be encoded, wherein the scene audio signal includes audio signals of C1 channels, where C1 is a positive integer;

The acquisition module is also used to acquire the property information of the target virtual speaker corresponding to the scene audio signal;

The acquisition module is also used to obtain the high-order energy gain of the scene audio signal;

The encoding module 702 is used to encode the high-order energy gain to obtain a high-order energy gain encoding result;

The encoding module is also used to encode the first audio signal in the scene audio signal, the attribute information of the target virtual speaker and the high-order energy gain encoding result to obtain a first code stream; wherein the first audio signal is the audio signal of K channels in the scene audio signal, and K is a positive integer less than or equal to C1.

FIG8 is a schematic diagram of the structure of an exemplary scene audio decoding device. The scene audio decoding device in FIG8 can be used to execute the decoding method of the aforementioned embodiment. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding method provided above, which will not be repeated here. The scene audio decoding device may include:

The code stream receiving module 801 is used to receive a first code stream;

A decoding module 802 is used to decode the first bit stream to obtain a first reconstructed signal, property information of a target virtual speaker, and a high-order energy gain coding result, wherein the first reconstructed signal is a reconstructed signal of a first audio signal in a scene audio signal, the scene audio signal includes audio signals of C1 channels, the first audio signal is audio signals of K channels in the scene audio signal, C1 is a positive integer, and K is a positive integer less than or equal to C1;

A virtual speaker signal generating module 803 is used to generate a virtual speaker signal corresponding to the target virtual speaker based on the attribute information and the first audio signal;

The scene audio signal reconstruction group 804 is used to reconstruct based on the property information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer;

an attenuation factor determination module 805, configured to determine an attenuation factor according to a frequency band sequence number of a reconstructed signal in the first reconstructed scene audio signal and/or an order of the first reconstructed scene audio signal;

The scene audio signal adjustment module 806 is used to adjust the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain a reconstructed scene audio signal.

In an example, FIG. 9 shows a schematic block diagram of a device 900 according to an embodiment of the present invention. The device 900 may include: a processor 901 and a transceiver/transceiver pin 902, and optionally, a memory 903.

The various components of the device 900 are coupled together via a bus 904, wherein the bus 904 includes a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, all the buses are referred to as the bus 904 in the figure.

Optionally, the memory 903 can be used to store the instructions in the aforementioned method embodiment. The processor 901 can be used to execute the instructions in the memory 903, and control the receiving pin to receive the signal, and control the sending pin to send the signal.

The apparatus 900 may be an electronic device or a chip of an electronic device in the above method embodiments.

Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.

This embodiment also provides a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the method in the above embodiment. The interface circuit may refer to the transceiver 902 in FIG. 9 .

This embodiment also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are run on an electronic device, the electronic device executes the above-mentioned related method steps to implement the scene audio encoding and decoding method in the above-mentioned embodiment.

This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer executes the above-mentioned related steps to implement the scene audio encoding and decoding method in the above-mentioned embodiment.

This embodiment also provides a device for storing a code stream, which includes: a receiver and at least one storage medium, the receiver is used to receive the code stream; at least one storage medium is used to store the code stream; the code stream is generated according to the scene audio encoding method in the above embodiment.

An embodiment of the present invention provides a device for transmitting a code stream, which includes: a transmitter and at least one storage medium, wherein the at least one storage medium is used to store the code stream, and the code stream is generated according to the scene audio encoding method in the above embodiment; the transmitter is used to obtain the code stream from the storage medium and send the code stream to the end device through the transmission medium.

An embodiment of the present invention provides a system for distributing bit streams, the system comprising: at least one storage medium for storing at least one bit stream, the at least one bit stream being generated according to the scene audio encoding method in the above embodiment, a streaming media device for obtaining a target bit stream from the at least one storage medium and sending the target bit stream to an end device, wherein the streaming media device comprises a content server or a content distribution server.

In addition, an embodiment of the present invention further provides a device, which may be a chip, a component or a module, and the device may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor may execute the computer execution instructions stored in the memory, so that the chip executes the scene audio encoding and decoding method in the above-mentioned method embodiments.

Among them, the electronic device, computer readable storage medium, computer program product or chip provided in this embodiment are used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

Through the description of the above implementation method, technical personnel in the relevant field can understand that for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual application, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

In addition, each functional unit in each embodiment of the present invention can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional unit.

Any content of each embodiment of the present invention, as well as any content of the same embodiment, can be freely combined. Any combination of the above contents is within the scope of the present invention.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage media include: USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, and other media that can store program code.

The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are only illustrative and not restrictive. Under the inspiration of the present invention, ordinary technical personnel in this field can make many forms without departing from the scope protected by the purpose of the present invention and the scope of the patent application, all of which are protected by the present invention.

The steps of the method or algorithm described in the disclosure of the embodiments of the present invention may be implemented in hardware or by a processor executing software instructions. The software instructions may be composed of corresponding software modules, which may be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable ROM (EPROM), electronically erasable programmable ROM (EEPROM), register, hard disk, removable hard disk, CD-ROM or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

It should be appreciated by those skilled in the art that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media may be any available media that can be accessed by a general or special-purpose computer.

The embodiments of the present invention are described above in conjunction with the attached drawings, but the present invention is not limited to the above-mentioned specific embodiments. The above-mentioned specific embodiments are only illustrative and not restrictive. Under the inspiration of the present invention, ordinary technicians in this field can make many forms without departing from the scope of protection of the purpose and claims of the present invention, which are all within the protection of the present invention. The above is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should be within the scope of the present invention.

701: Acquisition module 702: Encoding module 801: Code stream receiving module 802: Decoding module 803: Virtual speaker signal generation module 804: Scene audio signal reconstruction module 805: Attenuation factor determination module 806: Scene audio signal adjustment module 900: Device 901: Processor 902: Transceiver/transceiver pins 903: Memory 904: Bus S201,S202,S203,S204,S205,S301,S302,S303,S304,S305,S306,S401,S402,S403,S404,S405,S406,S407,S501,S502,S503,S504,S505,S506,S507,S508,S509,S510,S511,S512: Steps

Figure 1a is a schematic diagram of an application scenario exemplarily shown in the present invention; Figure 1b is a schematic diagram of an application scenario exemplarily shown in the present invention; Figure 2a is a schematic diagram of a coding process exemplarily shown in the present invention; Figure 2b is a schematic diagram of candidate virtual speaker distribution exemplarily shown in the present invention; Figure 3 is a schematic diagram of a decoding process exemplarily shown in the present invention; Figure 4 is a schematic diagram of a coding process exemplarily shown in the present invention; Figure 5a is a schematic diagram of a decoding process exemplarily shown in the present invention; Figure 5b is a schematic diagram of another decoding process exemplarily shown in the present invention; Figure 6a is a schematic diagram of the structure of an encoding end exemplarily shown in the present invention; Figure 6b is a schematic diagram of the structure of a decoding end exemplarily shown in the present invention; Figure 7 is a schematic diagram of the structure of a scene audio coding device exemplarily shown in the present invention; FIG8 is a schematic diagram of the structure of a scene audio decoding device exemplarily shown in the present invention; FIG9 is a schematic diagram of the structure of a device exemplarily shown in the present invention.

S301, S302, S303, S304, S305, S306: Steps

Claims (21)

A scene audio decoding method, characterized in that the method comprises: Receiving a first code stream; Decoding the first code stream to obtain a first reconstructed signal, property information of a target virtual speaker and a high-order energy gain coding result, wherein the first reconstructed signal is a reconstructed signal of a first audio signal in a scene audio signal, the scene audio signal comprises audio signals of C1 channels, the first audio signal is audio signals of K channels in the scene audio signal, C1 is a positive integer, and K is a positive integer less than or equal to C1; Based on the property information of the target virtual speaker and the first audio signal, generating a virtual speaker signal corresponding to the target virtual speaker; Reconstructing based on the property information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer; Determining an attenuation factor according to a frequency band sequence number of a reconstructed signal in the first reconstructed scene audio signal and/or an order of the first reconstructed scene audio signal; Adjusting the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain a reconstructed scene audio signal. The method of claim 1 is characterized in that, the scene audio signal is an N1-order high-order stereo reverberation HOA signal, the N1-order HOA signal includes a second audio signal, the second audio signal is an audio signal in the N1-order HOA signal other than the first audio signal, C1 is equal to the square of (N1+1); and/or, the first reconstructed scene audio signal is an N2-order HOA signal, the N2-order HOA signal includes a third audio signal, the third audio signal is a reconstructed signal in the N2-order HOA signal corresponding to each channel of the second audio signal, and C2 is equal to the square of (N2+1). The method as claimed in claim 2 is characterized in that the adjusting the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain the reconstructed scene audio signal comprises: performing entropy decoding on the high-order energy gain coding result to obtain the high-order energy gain after entropy decoding; performing inverse quantization on the high-order energy gain after entropy decoding to obtain the high-order energy gain; adjusting the high-order energy gain according to the characteristic information of the second audio signal and the characteristic information of the first audio signal to obtain the adjusted decoded high-order energy gain; The third audio signal in the N2-order HOA signal is adjusted according to the adjusted decoded high-order energy gain and the attenuation factor to obtain an adjusted third audio signal, and the adjusted third audio signal belongs to the reconstructed scene audio signal. The method as claimed in claim 3 is characterized in that the high-order energy gain is adjusted according to the characteristic information of the second audio signal and the characteristic information of the first audio signal, including: Obtaining the high-order energy of the second audio signal according to the channel energy of the first audio signal and the high-order energy gain; Obtaining a decoding energy proportional factor according to the channel energy of the third audio signal and the high-order energy of the second audio signal; Obtaining the decoding high-order energy gain of the third audio signal according to the channel energy of the third audio signal and the channel energy of the first audio signal; Adjusting the decoding high-order energy gain of the third audio signal according to the decoding energy proportional factor to obtain the adjusted decoding high-order energy gain. The method as described in any one of claims 3 to 4 is characterized in that the attenuation factor is determined according to the frequency band number of the reconstructed signal in the first reconstructed scene audio signal and/or the order of the first reconstructed scene audio signal, including: Obtaining the attenuation factor according to the frequency band number of the third audio signal and/or the order of the N2-order HOA signal. The method as described in any one of claims 3 to 5 is characterized in that after adjusting the third audio signal in the N2-order HOA signal according to the high-order energy gain coding result and the attenuation factor, the method further includes: Obtaining the channel energy of the fourth audio signal corresponding to the adjusted third audio signal, the third audio signal includes the audio signal of the current frame, and the fourth audio signal includes the audio signal of the previous frame of the current frame; Adjusting the adjusted third audio signal again according to the channel energy of the fourth audio signal. The method as described in claim 6 is characterized in that the adjusting the adjusted third audio signal again according to the channel energy of the fourth audio signal comprises: Obtaining the channel energy average value of the fourth audio signal and the channel energy of the adjusted third audio signal; Obtaining an energy average threshold according to the channel energy average value of the fourth audio signal and the channel energy of the adjusted third audio signal; Performing weighted averaging calculation on the channel energy average value of the fourth audio signal and the channel energy of the adjusted third audio signal according to the energy average threshold to obtain a target energy; Obtaining an energy smoothing factor according to the target energy and the channel energy of the adjusted third audio signal; Adjusting the third audio signal according to the energy smoothing factor. The method of claim 7, characterized in that the step of obtaining the attenuation factor according to the frequency band number of the third audio signal and/or the order of the N2-order HOA signal comprises: obtaining the attenuation factor g'(i, b) by: Wherein, i is the number of the i-th channel of the third audio signal, b is the frequency band number of the third audio signal, represents the number of the target virtual speakers, represents the number of mapping channels of the N2-order HOA signal, , M is the order of the N2-order HOA signal, γ represents the order corresponding to the channel number i of the third audio signal, Represents a multiplication operation. The method as claimed in claim 8, characterized in that the method further comprises: when b≤d, Updated to , ; d is the default first threshold; When b＞d, the Updated to , . The method as claimed in claim 8 is characterized in that the method further comprises: Updated to , ; Wherein, bands represents the number of frequency bands of the third audio signal. The method as claimed in any one of claims 8 to 10, characterized in that the method further comprises: when b≤d, Updated to , ; d is the default first threshold, w is the default adjustment ratio threshold. The method as claimed in any one of claims 8 to 11, characterized in that the method further comprises: updating w to w2, w2=w+ ×0.05. A method as described in any one of claims 8 to 12, characterized in that, when the value of i is 0, 1, 2, or 3, the value of γ is 1; when the value of i is 4, 5, 6, 7, or 8, the value of γ is 2; when the value of i is 9, 10, 11, 12, 13, 14, or 15, the value of γ is 3. The method as claimed in claim 2 is characterized in that the adjusting the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor comprises: performing entropy decoding on the high-order energy gain coding result to obtain the high-order energy gain after entropy decoding; performing inverse quantization on the high-order energy gain after entropy decoding to obtain the high-order energy gain; obtaining the high-order energy of the second audio signal according to the channel energy of the first audio signal and the high-order energy gain; obtaining the decoded energy ratio factor according to the channel energy of the third audio signal and the high-order energy of the second audio signal; determining the dispersion factor of the first reconstructed scene audio signal according to the first bit stream; The attenuation factor is linearly weighted according to the dispersion factor to obtain a weighted attenuation factor: The third audio signal in the N2-order HOA signal is adjusted according to the weighted attenuation factor and the decoding energy proportional factor to obtain an adjusted third audio signal. The method as claimed in claim 14 is characterized in that the attenuation factor is linearly weighted according to the divergence factor to obtain the weighted attenuation factor, including: obtaining the weighted attenuation factor gd(i,b) by at least one of the following methods: gd(i,b)= w diffusion(b)+(1-w) ; Where w is the preset adjustment ratio threshold, represents a multiplication operation, diffusion(b) represents the diffusion factor of the b-th frequency band of the third audio signal, represents the attenuation factor of the bth frequency band of the third audio signal; or, when the attenuation factor is a full-band signal, gd(i,b)=w mean(diffusion)+ (1-w) , wherein mean(diffusion) is the average value of the dispersion factors of multiple frequency bands of the third audio signal, represents the attenuation factor of the bth frequency band of the third audio signal, w is the preset adjustment proportional threshold; or, gd(i,b)= w diffusion(b)+(1-w) +offset(i,b), wherein offset(i,b) represents the offset constant of the bth frequency band on the i-th channel of the third audio signal, represents the attenuation factor of the b-th frequency band of the third audio signal, w is the preset adjustment proportional threshold, and diffusion(b) represents the diffusion factor of the b-th frequency band of the third audio signal; or, gd(i,b)= w diffusion(b)+(1-w) +direction(i,b), wherein direction(i,b) represents the direction parameter of the bth frequency band on the i-th channel of the third audio signal, represents the attenuation factor of the b-th frequency band of the third audio signal, w is a preset adjustment proportional threshold, and diffusion(b) represents the diffusion factor of the b-th frequency band of the third audio signal. The method of any one of claims 14 to 15, characterized in that the third audio signal in the N2-order HOA signal is adjusted according to the weighted attenuation factor and the decoding energy proportional factor to obtain an adjusted third audio signal: The adjusted third audio signal X'(i,b) is obtained in the following manner: X'(i,b) = X(i) g(i,b) gd(i,b); wherein gd(i,b) represents the weighted attenuation factor, g(i,b) represents the decoding energy proportional factor, and X(i) represents the third audio signal. A scene audio decoding device, characterized in that the device comprises: A code stream receiving module, used for receiving a first code stream; A decoding module, used for decoding the first code stream to obtain a first reconstructed signal, property information of a target virtual speaker and a high-order energy gain coding result, wherein the first reconstructed signal is a reconstructed signal of a first audio signal in a scene audio signal, wherein the scene audio signal comprises audio signals of C1 channels, wherein the first audio signal is audio signals of K channels in the scene audio signal, wherein C1 is a positive integer, and K is a positive integer less than or equal to C1; A virtual speaker signal generation module, used to generate a virtual speaker signal corresponding to the target virtual speaker based on the attribute information of the target virtual speaker and the first audio signal; A scene audio signal reconstruction module, used to reconstruct based on the attribute information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes audio signals of C2 channels, C2 is a positive integer; An attenuation factor determination module, used to determine the attenuation factor according to the frequency band sequence number of the reconstructed signal in the first reconstructed scene audio signal and/or the order of the first reconstructed scene audio signal; The scene audio signal adjustment module is used to adjust the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain the reconstructed scene audio signal. An electronic device, characterized in that it includes: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the scene audio decoding method described in any one of request items 1 to 16. A chip, characterized in that it includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from the memory of an electronic device and send the signals to the processor, wherein the signals include computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the scene audio decoding method described in any one of request items 1 to 16. A computer-readable storage medium is characterized in that the computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor executes a scene audio decoding method as described in any one of request items 1 to 16. A computer program product, characterized in that the computer program product includes a software program, and when the software program is executed by a computer or a processor, the steps of the method described in any one of claim items 1 to 16 are executed.