CN117730367A - Grouping methods, encoders, decoders, and storage media - Google Patents

Abstract

The present disclosure relates to a grouping method, an apparatus, and a storage medium, a grouping method, the method being performed by an encoder, comprising: and grouping the plurality of channels to obtain at least one channel group, wherein N first channel groups exist in the at least one channel group, the first channel groups comprise three channels, M second channel groups exist in the at least one channel group, the second channel groups comprise two channels, N is 1, and M is a non-negative integer. In the above embodiment, the problem that a single channel cannot be grouped when the channels are grouped is solved, and a method for grouping channels including three channels is provided, so that each channel can be correspondingly grouped when the channels are grouped, the accuracy of grouping the channels is ensured, the redundancy among the channels is reduced, the accuracy of encoding the channels is ensured, and the accuracy of audio encoding is further ensured.

Description

Grouping methods, encoders, decoders, and storage media

Technical field

The present disclosure relates to the field of multimedia technology, and in particular, to a packetization method, an encoder, a decoder, and a storage medium.

Background technique

With the rapid development of multimedia technology, audio can be used in various fields. In order to improve the sense of space and direction of audio, audio can be three-dimensionally encoded and three-dimensionally decoded, so that the audio heard by the user is consistent with what is heard in the actual environment. There is no difference in the audio.

Contents of the invention

The present disclosure solves the problem that a single channel cannot be grouped when grouping channels, and provides a method of grouping channels including three channels to ensure that when grouping channels, each channel can have The corresponding grouping ensures the accuracy of grouping the channels, thereby ensuring the reduction of redundancy between channels, ensuring the accuracy of encoding the channels, and thereby ensuring the accuracy of audio encoding.

The embodiments of the present disclosure provide a grouping method, device and storage medium.

According to a first aspect of an embodiment of the present disclosure, a grouping method is proposed, the method is performed by an encoder, the method includes:

Multiple channels are grouped to obtain at least one channel group, wherein there are N first channel groups in the at least one channel group, the first channel group includes three channels, and the at least There are M second channel groups in one channel group, and the second channel group includes two channels, where N is 1 and M is a non-negative integer.

According to a second aspect of the embodiment of the present disclosure, a grouping method is proposed, the method is performed by a decoder, the method includes:

The first information is decoded to obtain channel information of at least one channel group, there are N first channel groups in the at least one channel group, the first channel group includes three channels, and the There are M second channel groups in at least one channel group, and the second channel group includes two channels, where N is 1 and M is a non-negative integer.

According to a third aspect of the embodiment of the present disclosure, a grouping method is proposed, which method includes:

The encoder groups multiple channels to obtain at least one channel group, wherein there are N first channel groups in the at least one channel group, and the first channel group includes three channels, so There are M second channel groups in the at least one channel group, and the second channel group includes two channels, where N is 1 and M is a non-negative integer;

The decoder decodes the first information to obtain channel information of at least one channel group.

According to a fourth aspect of the embodiments of the present disclosure, a coding and decoding device is proposed, including:

A processing module configured to group multiple sound channels to obtain at least one sound channel grouping, wherein there are N first sound channel groupings in the at least one sound channel grouping, and the first sound channel grouping includes three sound channel groups. channel, there are M second channel groups in the at least one channel group, and the second channel group includes two channels, where N is 1 and M is a non-negative integer.

According to the fifth aspect of the embodiment of the present disclosure, a coding and decoding device is proposed, including:

A processing module configured to decode the first information to obtain channel information of at least one channel group, where there are N first channel groups in the at least one channel group, and the first channel group includes three channel, there are M second channel groups in the at least one channel group, and the second channel group includes two channels, where N is 1 and M is a non-negative integer.

According to the sixth aspect of the embodiment of the present disclosure, a coding and decoding device is proposed, including:

one or more processors;

Wherein, the encoding and decoding device is used to perform any of the methods described in the first aspect.

According to the seventh aspect of the embodiment of the present disclosure, a coding and decoding device is proposed, including:

one or more processors;

Wherein, the encoding and decoding device is used to perform any of the methods described in the second aspect.

According to an eighth aspect of the embodiments of the present disclosure, a coding and decoding system is proposed, including:

An encoder and a decoder, wherein the encoder is configured to implement the grouping method described in the first aspect, and the decoder is configured to implement the grouping method described in the second aspect.

According to a ninth aspect of the embodiment of the present disclosure, a storage medium is proposed, the storage medium stores instructions, and when the instructions are run on a communication device, the communication device causes the communication device to perform the first aspect or the second aspect. any one of the methods.

Description of the drawings

The drawings described here are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the present disclosure. The schematic embodiments of the embodiments of the present disclosure and their descriptions are used to explain the embodiments of the present disclosure and do not constitute an explanation of the embodiments of the present disclosure. undue limitation. In the attached picture:

Figure 1 is an architectural schematic diagram of a coding and decoding system according to an embodiment of the present disclosure;

Figure 2A is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure;

Figure 2B is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure;

Figure 2C is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure;

Figure 2D is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure;

Figure 2E is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure;

Figure 3A is a schematic flowchart of a grouping method according to an embodiment of the present disclosure;

Figure 3B is a schematic flowchart of a grouping method according to an embodiment of the present disclosure;

Figure 4A is a schematic flowchart of a grouping method according to an embodiment of the present disclosure;

Figure 4B is a schematic flowchart of a grouping method according to an embodiment of the present disclosure;

Figure 5 is a schematic flowchart of a grouping method according to an embodiment of the present disclosure;

Figure 6 is a schematic flowchart of a grouping method according to an embodiment of the present disclosure;

Figure 7A is a schematic structural diagram of a coding and decoding device proposed by an embodiment of the present disclosure;

Figure 7B is a schematic structural diagram of a coding and decoding device proposed by an embodiment of the present disclosure;

Figure 8A is a schematic structural diagram of a communication device proposed by an embodiment of the present disclosure;

FIG. 8B is a schematic structural diagram of a chip proposed by an embodiment of the present disclosure.

Detailed ways

The present disclosure provides a grouping method, device and storage medium.

According to a first aspect of an embodiment of the present disclosure, a grouping method is proposed, the method is performed by an encoder, the method includes:

Multiple channels are grouped to obtain at least one channel group, wherein there are N first channel groups in the at least one channel group, the first channel group includes three channels, and the at least There are M second channel groups in one channel group, and the second channel group includes two channels, where N is 1 and M is a non-negative integer.

In the above embodiment, the problem that a single channel cannot be grouped when channel grouping is solved, and a method of dividing channel groups including three channels is provided to ensure that each channel is grouped into channels. All can have corresponding groupings to ensure the accuracy of channel grouping, thereby ensuring the reduction of redundancy between channels, ensuring the accuracy of encoding channels, and thus ensuring the accuracy of audio encoding.

Combined with some embodiments of the first aspect, in some embodiments,

The grouping of multiple sound channels to obtain at least one sound channel grouping includes:

Obtain the similarity between any two of the plurality of channels;

The plurality of sound channels are grouped based on the similarity between any two sound channels to obtain the at least one sound channel grouping.

In the above embodiment, the channels are grouped based on the similarity between each two channels, ensuring that the similarity of the channels included in each channel group meets the requirements, ensuring the accuracy of channel grouping, and further ensuring The redundancy between channels is reduced, ensuring the accuracy of encoding the channels, and thus the accuracy of audio encoding.

Combined with some embodiments of the first aspect, the grouping of the plurality of sound channels based on the similarity between any two sound channels to obtain the at least one sound channel grouping includes:

For any two channels among the plurality of channels, determine the two channels with the greatest similarity as a candidate channel group;

Exclude channels other than the two channels with the greatest similarity, and determine the two channels with the greatest similarity as a candidate channel group until one independent channel remains or no channels remain;

The at least one channel grouping is determined based on the resulting candidate channel groupings and/or the independent channels.

In the above embodiment, the channels are divided into the same channel group in order of similarity from large to small, ensuring that the similarity of the channels included in each channel group is a channel with high correlation, ensuring that The accuracy of grouping channels, thereby ensuring the reduction of redundancy between channels, ensuring the accuracy of encoding channels, thereby ensuring the accuracy of audio encoding.

With reference to some embodiments of the first aspect, determining the at least one channel grouping based on the obtained candidate channel groupings and/or the independent channels includes:

Obtain the similarity between the independent channel and each channel in each candidate channel group;

When the similarity between the independent channel and each channel in the first candidate channel group is greater than the similarity threshold, determine the independent channel and each channel in the first candidate channel group as one said first channel grouping;

Each remaining candidate channel group is determined as one of the second channel groups.

In the above embodiment, the remaining single channels are also divided into a channel group to ensure that there is no situation where a single channel is not divided into a channel group, thereby ensuring that the redundancy between channels is reduced and ensuring that the The accuracy of audio channel encoding, thereby ensuring the accuracy of audio encoding.

With reference to some embodiments of the first aspect, in some embodiments, grouping multiple channels to obtain at least one channel grouping includes:

The plurality of sound channels are grouped to directly obtain the N first sound channel groups and the M second sound channel groups.

Combined with some embodiments of the first aspect, in some embodiments, the grouping of the plurality of channels directly obtains the N first channel groups and the M second channel groups, including :

By performing a global search on the plurality of sound channels, the plurality of sound channels are grouped according to the similarity between any two sound channels to obtain N first sound channel groups and the M second sound channels. Road grouping.

With reference to some embodiments of the first aspect, in some embodiments, grouping multiple channels to obtain at least one channel grouping includes:

Group the plurality of channels to obtain a plurality of candidate channel groups including two channels and at least one independent channel;

Divide one of the at least one independent channel into one of the plurality of candidate channel groups to obtain the N first channel groups and the M second channel groups. Road grouping.

Combined with some embodiments of the first aspect, in some embodiments, the method further includes:

Downmix the audio signal in each of the channel groups to obtain a downmixed audio signal;

The downmixed audio signal is encoded to obtain an audio stream.

In the above embodiment, the audio signal of the channel grouping is downmixed and encoded to obtain an audio stream, which ensures that the encoded audio stream can be transmitted normally and the accuracy of the audio stream is also ensured.

Combined with some embodiments of the first aspect, in some embodiments, the method further includes:

First information is sent, where the first information is used to indicate channel information of the plurality of channel groups.

In the above embodiment, the channel information of each channel group is informed through the first information to ensure the reliability of transmitting the channel group.

With reference to some embodiments of the first aspect, in some embodiments, the vocal channel information includes at least one of the following:

The number of channel groups including two channels;

Channel identifiers of channels included in a channel grouping that includes two channels;

Energy parameters for channel groupings including two channels;

The number of channel groups including three channels;

Channel identifiers of channels included in a channel grouping that includes three channels;

Energy parameters for channel groupings including three channels;

Wherein, the energy parameter is used to adjust the energy of the channels in the channel grouping.

In the above embodiment, the channel information includes channel grouping information of two channels or three channels, ensuring the comprehensiveness of the channel information.

In a second aspect, embodiments of the present disclosure provide a grouping method, which is performed by a decoder. The method includes:

The first information is decoded to obtain channel information of at least one channel group, there are N first channel groups in the at least one channel group, the first channel group includes three channels, and the There are M second channel groups in at least one channel group, and the second channel group includes two channels, where N is 1 and M is a non-negative integer.

Combined with some embodiments of the second aspect, in some embodiments, the vocal channel information includes at least one of the following:

The number of channel groups including two channels;

Channel identifiers of channels included in a channel grouping that includes two channels;

Energy parameters for channel groupings including two channels;

The number of channel groups including three channels;

Channel identifiers of channels included in a channel grouping that includes three channels;

Energy parameters for channel groupings including three channels;

Wherein, the energy parameter is used to adjust the energy of the channels in the channel grouping.

Combined with some embodiments of the second aspect, in some embodiments, the method further includes:

When it is determined that the channel grouping does not include a channel grouping of three channels, a two-channel upmix is performed on the audio stream.

Combined with some embodiments of the second aspect, in some embodiments, the method further includes:

When it is determined that the channel grouping includes three channels, three-channel upmixing is performed on the audio stream.

In a third aspect, embodiments of the present disclosure provide a grouping method, which method includes:

The encoder groups multiple channels to obtain at least one channel group, wherein there are N first channel groups in the at least one channel group, and the first channel group includes three channels, so There are M second channel groups in the at least one channel group, and the second channel group includes two channels, where N is 1 and M is a non-negative integer;

The decoder decodes the first information to obtain the channel information of the at least one channel group. At least one channel group among the plurality of channel groups includes three channels.

In a fourth aspect, embodiments of the present disclosure provide a codec device. The codec device includes at least one of a transceiver module and a processing module; wherein the encoder is used to perform optional implementations of the first aspect and the third aspect. Way.

In a fifth aspect, embodiments of the present disclosure provide a codec device. The codec device includes at least one of a transceiver module and a processing module; wherein the access network device is configured to perform the steps of the second and third aspects. Optional implementation.

In a sixth aspect, an embodiment of the present disclosure provides a coding and decoding device, including:

one or more processors;

Wherein, the encoding and decoding device is used to perform the method described in any one of the first aspect and the third aspect.

In a seventh aspect, an embodiment of the present disclosure provides a coding and decoding device, including:

one or more processors;

Wherein, the encoding and decoding device is used to perform the method described in any one of the second aspect and the third aspect.

In an eighth aspect, an embodiment of the present disclosure provides a storage medium that stores first information. When the first information is run on a communication device, the communication device performs the steps of the first aspect and the third aspect. The method described in any one of the second aspect and the third aspect.

In a ninth aspect, embodiments of the present disclosure provide a program product. When the program product is executed by a communication device, the communication device causes the communication device to execute the method described in any one of the first aspect, the second aspect, and the third aspect.

In a tenth aspect, embodiments of the present disclosure provide a computer program that, when run on a communication device, causes the communication device to perform the method described in any one of the first, second and third aspects.

In an eleventh aspect, embodiments of the present disclosure provide a chip or chip system. The chip or chip system includes a processing circuit configured to perform the method described in any one of the first, second and third aspects.

It can be understood that the above-mentioned encoders, storage media, program products, computer programs, chips or chip systems are all used to execute the methods proposed in the embodiments of the present disclosure. Therefore, the beneficial effects it can achieve can be referred to the beneficial effects in the corresponding methods, and will not be described again here.

The embodiments of the present disclosure provide a grouping method, device and storage medium. In some embodiments, terms such as grouping method and information grouping method, grouping method may be interchanged with each other, terms such as codec device and information processing device, indicating device may be interchanged with each other, and terms such as information processing system and codec system may be interchanged with each other.

The embodiments of this disclosure are not exhaustive, but are only illustrative of some embodiments, and are not intended to specifically limit the scope of protection of this disclosure. If there is no contradiction, each step in a certain embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution in which some steps are removed from a certain embodiment can also be used. It is implemented as an independent embodiment, and the order of each step in a certain embodiment can be arbitrarily exchanged. In addition, the optional implementation methods in a certain embodiment can be arbitrarily combined; in addition, various embodiments can be arbitrarily combined, for example , some or all steps of different embodiments can be arbitrarily combined, and a certain embodiment can be arbitrarily combined with optional implementations of other embodiments.

In each embodiment of the present disclosure, if there is no special explanation or logical conflict, the terms and/or descriptions between the embodiments are consistent and can be referenced to each other. The technical features in different embodiments can be used according to their inherent logical relationships. Combined to form new embodiments.

The terms used in the embodiments of the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure.

In the embodiments of the present disclosure, unless otherwise specified, elements expressed in the singular form, such as "a", "an", "the", "above", "said", "aforesaid", "this" etc., can mean "one and only one", "one or more", "at least one", etc. For example, when articles (articles) such as "a", "an", and "the" in English are used in translation, the noun after the article can be understood as a singular expression or a plural expression. .

In the embodiment of the present disclosure, "plurality" refers to two or more.

In some embodiments, "at least one of", "one or more", "a plurality of", "a plurality of" Terms such as multiple can be used interchangeably.

In some embodiments, "at least one of A, B", "A and/or B", "A in one situation, B in another situation", "in response to one situation A, in response to another situation A" "A situation B" and other recording methods may include the following technical solutions depending on the situation: in some embodiments A (execute A independently of B); in some embodiments B (execute B independently of A); in some embodiments Select execution from A and B (A and B are selectively executed); in some embodiments, A and B (both A and B are executed). It is similar to the above when there are more branches such as A, B, C, etc.

In some embodiments, descriptions such as "A or B" may include the following technical solutions depending on the situation: in some embodiments A (execute A independently of B); in some embodiments B (execute independently of A) B); in some embodiments select execution from A and B (A and B are selectively executed). It is similar to the above when there are more branches such as A, B, C, etc.

The prefixes such as "first" and "second" in the embodiments of this disclosure are only used to distinguish different description objects and do not limit the position, order, priority, quantity or content of the description objects. See the description of the context in the claims or embodiments, and the use of prefixes should not constitute unnecessary limitations. For example, if the description object is "field", then the ordinal word before "field" in "first field" and "second field" does not limit the position or order between "fields". "First" and "Second field" " does not limit whether the "fields" it modifies are in the same message, nor does it limit the order of the "first field" and "second field". For another example, if the description object is "level", then the ordinal words before "level" in "first level" and "second level" do not limit the priority between "levels". For another example, the number of description objects is not limited by ordinal words, and can be one or more. Taking "first device" as an example, the number of "devices" can be one or more. In addition, the objects modified by different prefixes may be the same or different. For example, if the description object is "device", then the "first device" and the "second device" may be the same device or different devices, and their types may be the same or different. ; For another example, if the description object is "information", then the "first information" and the "second information" can be the same information or different information, and their contents can be the same or different.

In some embodiments, "includes A", "includes A", "is used to indicate A", and "carries A" can be interpreted as directly carrying A, or as indirectly indicating A.

In some embodiments, terms such as "time/frequency" and "time-frequency domain" refer to the time domain and/or frequency domain.

In some embodiments, "in response to...", "in response to determining...", "in the case of...", "when...", "when...", "if...", Terms such as "if..." are interchangeable.

In some embodiments, "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to" , "not less than", "above" and other terms can be interchanged, "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "not more than", " Terms such as "less than", "less than or equal to", "no higher than" and "below" may be used interchangeably.

In some embodiments, devices and equipment can be interpreted as physical or virtual, and their names are not limited to the names recorded in the embodiments. In some cases, they can also be understood as "equipment". , "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "chip", "chip System", "entity", "subject", etc.

In some embodiments, "network" may be interpreted as devices included in the network, such as access network equipment, core network equipment, etc.

In some embodiments, "access network device (AN device)" may also be called "radio access network device (RAN device)" or "base station (BS)" , "radio base station", "fixed station", in some embodiments can also be understood as "node", "access point", "transmitting point" (transmissionpoint, TP)", "reception point (RP)", "transmission and/or reception point (transmission/reception point, TRP)", "panel", "antenna panel" , "antenna array", "cell", "macro cell", "small cell", "femto cell", "pico cell" cell", "sector", "cellgroup", "serving cell", "carrier", "component carrier", "bandwidth part (BWP)" "wait.

In some embodiments, an "encoder (terminal)" or "encoder device (terminal device)" may be referred to as "user equipment (encoder)", "user terminal", " Mobile station (MS)", "mobile terminal (MT)", subscriber station, mobile unit, subscriber unit, wireless unit, remote unit (remote unit), mobile device (mobile device), wireless device (wireless device), wireless communication device (wireless communication device), remote device (remote device), mobile subscriber station (mobile subscriber station), access terminal ), mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client )wait.

In some embodiments, obtaining data, information, etc. may comply with the laws and regulations of the country where the location is located.

In some embodiments, data, information, etc. may be obtained with user consent.

In addition, each element, each row, or each column in the table in the embodiment of the present disclosure can be implemented as an independent embodiment, and the combination of any element, any row, and any column can also be implemented as an independent embodiment.

Figure 1 is a schematic architectural diagram of a codec system according to an embodiment of the present disclosure. As shown in Figure 1, the method provided by the embodiment of the present disclosure can be applied to a codec system 100. The codec system can include an encoder 101, a decoder 102. It should be noted that the coding and decoding system 100 may also include other devices, and this disclosure does not limit the devices included in the coding and decoding system 100 .

In some embodiments, both the encoder 101 and the decoder 102 are provided in the terminal. In some embodiments, the terminal may be a variety of devices. For example, they include mobile phones, wearable devices, Internet of Things devices, cars with communication functions, smart cars, tablets (Pads), computers with wireless transceiver functions, virtual reality (VR) encoder equipment, enhanced Augmented reality (AR) encoder equipment, wireless encoder equipment in industrial control (industrial control), wireless encoder equipment in self-driving (self-driving), wireless coding in remote medical surgery encoder equipment, wireless encoder equipment in smart grid, wireless encoder equipment in transportation safety, wireless encoder equipment in smart city, and smart home At least one of, but not limited to, wireless encoder devices.

It can be understood that the encoding and decoding system described in the embodiments of the present disclosure is for the purpose of more clearly illustrating the technical solutions of the embodiments of the present disclosure, and does not constitute a limitation on the technical solutions proposed by the embodiments of the present disclosure. Those of ordinary skill in the art will know that With the evolution of system architecture and the emergence of new business scenarios, the technical solutions proposed in the embodiments of the present disclosure are also applicable to similar technical problems.

The following embodiments of the present disclosure can be applied to the encoding and decoding system 100 shown in Figure 1, or part of the main body, but are not limited thereto. The subjects shown in Figure 1 are examples. The coding and decoding system may include all or part of the subjects in Figure 1, or may include other subjects outside of Figure 1. The number and form of each subject is arbitrary, and each subject may be physical or physical. It is virtual. The connection relationship between the subjects is just an example. The subjects may not be connected or connected. The connection may be in any way. It may be a direct connection or an indirect connection. It may be a wired connection or a wireless connection. connect.

Each embodiment of the present disclosure can be applied to Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, and fourth-generation mobile codec systems (4th generation mobile communication system, 4G),), fifth generation mobile communication system (5G), 5G new radio (new radio, NR), future radio access (Future Radio Access, FRA), New-Radio Access Technology (RAT), New Radio (NR), New radio access (NX), Futuregeneration radio access (FX), Global System for Mobile communications (GSM (registered trademark)), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (Bl encoder tooth (registered trademark)), Public Land Mobile Network (PLMN) network, Device-to-Device (D2D) Systems, Machine to Machine (M2M) systems, Internet of Things (IoT) systems, Vehicle-to-Everything (V2X), systems utilizing other grouping methods, and extensions based on them Generation system etc. In addition, a combination of multiple systems (for example, a combination of LTE or LTE-A and 5G, etc.) can also be applied.

In some embodiments, the present disclosure is used for three-dimensional acoustic audio coding. Among them, the three-dimensional audio encoding is one of the key technologies of immersive audio technology. Compared with traditional sound, three-dimensional sound increases the sense of space and direction, allowing listeners to reproduce the sounds heard in the real world, thereby meeting people's needs for a highly restored and highly immersive experience of sound, and at the same time, it can have personalized selection and interactive experience .

In some embodiments, in order to reproduce the sense of space and direction of sound, technology can rely on a channel-based approach, an object-based approach, a sound field-based approach, and a combination of the above three forms, where: sound-based Channel audio is a set of interrelated channels. Common ones include 5.1 channel, 7.1 channel, 5.1.4 channel, 7.1.4 channel, etc. Each format corresponds to a speaker layout. In the corresponding speaker The best playback effect can be obtained under the layout.

In some embodiments, object-based audio is a collection of monophonic audio elements and corresponding metadata. Metadata represents information such as the object's location, intensity, size, etc. During playback, according to the metadata information, the object is mapped to one or more speakers or binaurally rendered to headphones for playback to achieve the desired spatial audio effect.

In some embodiments, sound field-based audio is a 3D sound field modeling format defined on the surface of a sphere. The principle is that sound is transmitted as a pressure wave, and for a sound scene at a given time, each point needs to be represented by several pressure functions. If the pressure value of each point in the space is known, the sound in the space can be reconstructed. There is a certain relationship between the pressure of each point in space and its adjacent points. In order to give full play to the advantages of the scene-based audio production method, it is necessary to accurately obtain the coefficients and improve the encoding quality of the sound field spatial coefficients. The collected sound field signal is called Higher Order Ambisonics (HOA). HOA system performance increases as the HOA order increases, but the number of HOA signals also increases.

Figure 2 is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure. As shown in Figure 2, the embodiment of the present disclosure relates to a grouping method. The above method includes:

Step S2101: The encoder groups multiple channels to obtain at least one channel group.

In some embodiments, vocal channels refer to mutually independent audio signals that are collected or played back at different spatial locations when sound is recorded or played. Channels refer to elements of audio, and different types of audio have different numbers of channels. For example, the HOA3 signal has 16 channels, and the MC22.2 signal has 24 channels.

In some embodiments, the audio signal includes audio data and side information.

In some embodiments, a channel grouping refers to a grouping that includes at least two channels. The encoder will encode the channels in units of channel groups to obtain the audio stream.

In some embodiments, the name of the channel grouping is not limited. Examples thereof include channel groups, channel groups, and the like.

In some embodiments, there are N first channel groups in at least one channel group, the first channel group includes three channels, and M is a non-negative integer.

In some embodiments, there are M second channel groups in at least one channel group, and the second channel group includes two channels, where N is 1.

In some embodiments, N may also be 0, that is, when multiple channels are grouped, no channel grouping including three channels is obtained, that is, 0 first channel groupings are obtained.

Optionally, for the value of M, M should not be greater than half the number of channels. Optionally, the number of channels to group the channels is P, then the value of M is greater than or equal to 0 and less than or equal to P/2, where P is a positive integer.

In some embodiments, before the channels are grouped, the signals included in the channels need to be pre-processed, and the channels are grouped after the pre-processing is completed.

Optionally, the method of preprocessing the signal included in the vocal channel includes at least one of the following: transient detection, window type judgment, time-frequency transformation, frequency domain noise shaping, time domain noise shaping, and frequency band extension coding.

In some embodiments, grouping multiple channels to obtain at least one channel grouping includes: obtaining a similarity between any two channels among the multiple channels, based on the similarity between any two channels. Group multiple channels to obtain at least one channel group.

In this embodiment of the present disclosure, the encoder can obtain the similarity between each two channels, and then group the multiple channels according to the similarity between each two channels to obtain at least one channel. Group.

In some embodiments, grouping the plurality of channels based on the similarity between any two channels to obtain at least one channel grouping includes: for any two channels among the plurality of channels, the similarity is The two largest channels are determined as a candidate channel group, and other channels except the two channels with the greatest similarity are excluded, and the two channels with the greatest similarity are determined as a candidate channel group until one is left. Independent channels or no remaining channels, at least one channel grouping is determined based on the obtained candidate channel groupings and/or independent channels.

For example, if the multiple channels are channel 1, channel 2, channel 3, channel 4 and channel 5, then the similarity between each two channels is obtained respectively, such as channel 1 and channel 2. If the similarity between them is the highest, then group channel 1 and channel 2 as one channel, and then exclude channel 1 and channel 2, and select the channel with the greatest similarity among channel 3, channel 4, and channel 5. As a channel group, there is one channel 5 left at this time, and this channel 5 is an independent channel.

In some embodiments, determining at least one channel group based on the obtained candidate channel groupings and/or independent channels includes: obtaining a similarity between the independent channel and each channel in each candidate channel group, in When the similarity between the independent channel and each channel in the first candidate channel group is greater than the similarity threshold, each channel in the independent channel and the first candidate channel group is determined to be a first channel group, Each remaining candidate channel group is determined as a second channel group.

In the embodiment of the present disclosure, when the similarity between the independent channel and each channel in the first candidate channel group is greater than the similarity threshold, it means that the independent channel and the two channels in the first candidate channel group If the channels are all similar, then the independent channel can be divided into the first candidate channel group. At this time, the first candidate channel group includes three channels, that is to say, it is the first channel group, and the remaining The channel grouping still includes two channels, which is the second channel grouping.

In some embodiments, when multiple channels are grouped based on the similarity between any two channels, and an odd number of channels are determined to meet the grouping conditions, and then grouped to obtain at least one channel group, independent The similarity between the channel and each channel in each candidate channel group. When the similarity between the independent channel and each channel in the first candidate channel group is greater than the similarity threshold, the independent channel will be compared with the third candidate channel group. Each channel in a candidate channel group is determined as a first channel group, and each remaining candidate channel group is determined as a second channel group.

It should be noted that the embodiment of the present disclosure takes as an example that independent sound channels can be obtained. In another embodiment, for any two channels among the plurality of channels, the two channels with the greatest similarity are determined as a candidate channel group, and other channels other than the two channels with the greatest similarity are excluded. Among the channels, the two channels with the greatest similarity are determined as a candidate channel group until no channels remain. Based on the obtained candidate channel groups, at least one channel group is determined.

It should be noted that, for any two channels among the plurality of channels, if the similarity between any two channels is not greater than the similarity threshold, the channels will not be grouped.

It should be noted that in the embodiment of the present disclosure, there may be an independent channel whose similarity to two channels in multiple candidate channel groups is greater than the similarity threshold. In this case, additional judgment is required to divide the independent channels into into which candidate channel group.

In some embodiments, the sum of the similarities between the independent channels and the two channels in each candidate channel group is obtained, the largest sum is selected from the sum corresponding to each candidate channel group, and the independent channel is Divide it into the channel group corresponding to the largest sum.

For example, if the independent channel is channel 5, the first candidate channel group includes channel 1 and channel 2, and the second candidate channel group includes channel 3 and channel 4. If channel 5 and channel The sum of the similarities between channel 1 and channel 2 is 1.8, and the sum of the similarities between channel 5 and channel 3 and channel 4 is 1.9. Then channel 5 is divided into the second candidate channel group.

In some embodiments, the candidate channel group corresponding to the largest similarity among the similarities between the independent channel and the two channels in each candidate channel group is determined, and the independent channel is divided into the candidate channel group with the largest similarity. in the corresponding channel grouping.

For example, if the independent channel is channel 5, the first candidate channel group includes channel 1 and channel 2, and the second candidate channel group includes channel 3 and channel 4. If channel 5 and channel The similarity of channel 1 is 0.7, the similarity with channel 2 is 0.9, the similarity between channel 5 and channel 3 is 0.8, and the similarity with channel 4 is 0.6, then channel 5 is divided into the first place. candidate channel groupings.

It should be noted that in the above embodiment, if there is an independent channel that is the same as the sum or maximum value of the channels in at least two candidate channel groups, then one of the candidate channel groups is randomly selected to divide the independent channels into to the selected channel group.

It should be noted that the grouping in the embodiment of the present disclosure includes two grouping schemes. Below, each grouping scheme is explained.

In some embodiments, multiple channels are grouped to directly obtain N first channel groups and M second channel groups. Optionally, in the embodiment of the present disclosure, when multiple channels are grouped, the N first channel groupings and M second channel groupings obtained by grouping are directly output, and other grouping results will not be output during this period.

Optionally, group multiple channels to directly obtain N first channel groups and M second channel groups, including: performing a global search on multiple channels, based on the relationship between any two channels Multiple channels are grouped by the similarity to obtain N first channel groups and M second channel groups.

In one possible implementation, the similarity between any two channels of every three channels in the multiple channels is obtained, if the similarity between any two channels of every three channels is greater than If the similarity threshold is determined, the three channels are divided into a candidate channel group. If a candidate channel group is finally obtained, the candidate channel group is determined as the first channel group. If multiple candidate channel groups are finally obtained, the first channel group is determined from the multiple candidate channel groups based on the similarity relationship of each candidate channel group.

Optionally, if multiple candidate channel groups are finally obtained, then based on the similarity relationship of each candidate channel group, determine the first channel group from the multiple candidate channel groups, including: obtaining each candidate channel group The sum of the similarities between every two channels in the channel group, select the largest sum from the sum corresponding to each candidate channel group, and determine the channel group corresponding to the largest sum as the first channel group .

For example, the first candidate channel group includes channel 1, channel 2, and channel 3, and the second candidate channel group includes channel 4, channel 5, and channel 6. If channel 1, channel The sum of the similarities between each two channels in channel 2 and channel 3 is 2.7, and the sum of the similarities between channels 4, channel 5 and channel 6 is 2.6, then the first candidate channel grouping is determined as First channel grouping.

Optionally, if multiple candidate channel groups are finally obtained, the first channel group is determined from the multiple candidate channel groups based on the similarity relationship of each candidate channel group, including: determining the independent channel and Among the similarities of every two channels in each candidate channel group, the candidate channel group corresponding to the largest similarity is determined as the first channel group.

In some embodiments, the process of grouping multiple channels is similar to the process of grouping based on similarity in the above embodiments, and will not be described again here.

In some embodiments, multiple channels are grouped to obtain multiple candidate channel groups including two channels and at least one independent channel, and one independent channel in the at least one independent channel is divided into multiple candidates. A candidate channel grouping in the channel grouping results in N first channel groupings and M second channel groupings.

It should be noted that the embodiment of the present disclosure takes grouping of multiple channels as an example for explanation. In another embodiment, after multiple channels are grouped, channel information for each channel group is also generated.

In some embodiments, the vocal tract information includes at least one of the following:

(1) The number of channel groups including two channels;

In some embodiments, the number of channel groups including two channels may also be referred to as the number of second channel groups, or the number of second channel groups, or the number of channel group pairs, or the number of groups. etc., the embodiments of this disclosure are not limiting.

In some embodiments, the number of channel groupings including two channels is used to represent the number of second channel grouping pairs of the current frame.

(2) Channel identifiers of channels included in a channel grouping that includes two channels;

In some embodiments, the channel identifiers of the channels included in the channel group including two channels are used to represent the index of the channel pair, and the index values of the two channels in the current channel pair can be parsed to obtain.

(3) Energy parameters of channel groupings including two channels;

(4) The number of channel groups including three channels;

In some embodiments, the number of channel groups including three channels may also be referred to as the number of first channel groups, or the number of first channel groups, or the number of channel group pairs, or the number of groups. etc., the embodiments of this disclosure are not limiting.

In some embodiments, the number of channel groupings including three channels is used to represent the number of first channel grouping pairs of the current frame.

(5) Channel identifiers of the channels included in the channel grouping including three channels;

In some embodiments, the channel identifiers of the channels included in the channel grouping including three channels are used to represent the index of the channel pair, and the index values of the three channels in the current channel pair can be parsed to obtain.

(6) Energy parameters of channel groupings including three channels;

Among them, the energy parameter is used to adjust the energy of the channels in the channel grouping.

In some embodiments, the energy parameter is used as an inter-channel amplitude difference ILD parameter quantization index for the first and second channels of the current channel pair for inter-channel energy/amplitude adjustment.

In some embodiments, multiple channels are grouped to directly obtain N first channel groupings and M second channel groupings, and channel information of the first channel grouping and the second channel grouping is generated.

In some embodiments, multiple channels are grouped to obtain multiple candidate channel groupings including two channels and at least one independent channel, and a plurality of candidate channel groupings and at least one independent channel are generated. Information, divide one independent channel among at least one independent channel into one candidate channel group among multiple candidate channel groups, obtain N first channel groups and M second channel groups, and classify multiple The candidate channel grouping and the channel information of at least one independent channel are rewritten to obtain the channel information of the first channel grouping and the second channel grouping.

Optionally, group multiple channels to obtain M+1 candidate channel groups including two channels and at least one independent channel, and generate multiple candidate channel groups and at least one independent channel. Information, divide one independent channel among at least one independent channel into one candidate channel group among multiple candidate channel groups, and obtain N first channel groups and M second channel groups.

It should be noted that the method of grouping multiple channels to obtain the first channel grouping can be called 3-channel sum-difference coding. The method of grouping multiple channels to obtain a second channel group can be called 2-channel sum-difference coding.

It should be noted that the execution method in the embodiment of the present disclosure is executed by the decision module or by other modules, which is not limited by the embodiment of the present disclosure.

Step S2102: The encoder downmixes the audio signal in each channel group to obtain a downmixed audio signal.

In some embodiments, downmixing refers to using an orthogonal normalization matrix to mix the grouped channels to obtain a mixed channel for each channel.

Optionally, the orthogonal normalization matrix is set in advance, and the embodiment of the present disclosure does not limit the orthogonal normalization matrix.

For example, for a channel grouping that includes two channels, the orthogonal normalization matrix used is Among them, the first row is the sum vector, and the second row is the difference vector.

For channel groupings including three channels, the orthogonal normalization matrix used is Among them, the first row is the sum vector, and the second and third rows are the difference vector.

In some embodiments M _{3ch ms} I _{3ch ms} =O _{3ch ms} , M _{2ch ms} I _{2ch ms} =O _{2ch ms} . Among them, I _{3ch ms} is a 1*3 column vector, and this column vector refers to audio data in a channel group including three channels. I _{2ch ms} is a 1*2 column vector, and this column vector refers to audio data in a channel group including two channels.

It should be noted that the steps performed in the embodiment of the present disclosure are performed by the downmix module or in other ways, which are not limited by the embodiment of the present disclosure.

Step S2103: The encoder encodes the downmixed audio signal to obtain an audio stream.

In some embodiments, encoding includes bit allocation quantization entropy encoding and code stream multiplexing.

Below, the implementation of the above steps is illustrated with an example.

For example, referring to FIG. 2B, a decision module (step S2101) is executed to determine which sum-difference encoding method or a combination of the two is used. Among them, the channels include L channel, R channel, C channel, LS channel and RS channel. Among them, the similarity between any two channels is shown in Table 1.

Table 1

- L R C LS RS L - 0.85 0.74 0.43 0.36 R - - 0.78 0.37 0.32 C - - - 0.39 0.31 LS - - - - 0.66 RS - - - - -

The downmix threshold is 0.5. The decision module obtains the used sum-and-difference coding method through a certain algorithm.

Among them, 3CH M/S refers to a module with three channels divided into a first channel group, and 2CH M/S refers to a module with two channels divided into a second channel group.

1. Use the maximum similarity iteration to filter out the first channel pair L channel and R channel (maximum similarity cox_L_R = 0.85), and the second channel pair LS channel and RS channel (the largest of the remaining undownmixed channels). Similarity cox_LS_RS=0.66).

2. Calculate the similarity between all channels that are not downmixed (C channel) and the two channels of the first channel pair and the second channel pair.

The similarity between the C channel and the L channel, cox_C_L=0.74, is greater than the downmixing threshold, and the similarity between the C channel and the R channel, cox_C_R=0.78, is greater than the downmixing threshold.

3. Output the judgment results of 3CH M/S and 2CH M/S. In this embodiment, the 3CH M/S decision result outputs L channel, R channel, and C channel. 2CH M/S decision result outputs LS channel and RS channel.

Subsequently, execute the downmix module.

M _{2ch ms} I _{2ch ms} ＝O _{2ch ms}

M _{3ch ms} I _{3ch ms} =O _{3ch ms}

Among them, the two-channel downmix matrix M _{2ch ms} is a 2x2 orthogonal normalized matrix, in which the first row is the sum vector and the second row is the difference vector. The three-channel downmix matrix M _{3ch ms} is a 3x3 orthogonal normalized matrix, in which the first row is the sum vector, and the second and third rows are the difference vectors.

I _{2ch ms} is a 1x2 column vector, and I _{3ch ms} is a 1x3 column vector. The data contained in the vector is preprocessed audio data, and the unit is a sampling point or frequency point.

Generate channel information for each channel grouping.

For another example, see Figure 2C, a two-channel group pairing decision is executed, and two-channel group pairing edge information is obtained, including the number of group pairs and the channel pair index.

Among them, the channels include L channel, R channel, C channel, LS channel and RS channel. The similarity between any two channels is shown in Table 1, and the downmixing threshold is 0.5.

The two-channel group pair judgment uses the maximum similarity iteration to filter out the first channel pair L channel and R channel (maximum similarity cox_L_R = 0.85), and the second channel pair LS channel and RS channel (the remaining ones are not listed below). The maximum similarity in the mixed channel is cox_LS_RS=0.66).

The number of second channel groups finally obtained is 2, including the first channel grouping L channel and R channel, and the second channel grouping LS channel and RS channel.

Secondly, the three-channel pairing decision is performed. If a three-channel pairing occurs, the above-mentioned channel grouping result will be changed.

First, calculate the similarity between all channels that are not downmixed (C channel) and the two channels of the first channel pair and the second channel pair.

The similarity between the C channel and the L channel, cox_C_L=0.74, is greater than the downmixing threshold, and the similarity between the C channel and the R channel, cox_C_R=0.78, is greater than the downmixing threshold.

Output the judgment results of 3CH M/S and 2CH M/S. In this embodiment, the 3CH M/S decision result outputs L channel, R channel, and C channel. After the 2CH M/S judgment result is changed, 1 channel grouping is output, that is, LS channel and RS channel are output.

Generates bi- and tri-channel channel information.

Implement two-channel and three-channel downmix modules.

M _{2ch ms} I _{2ch ms} ＝O _{2ch ms}

M _{3ch ms} I _{3ch ms} =O _{3ch ms}

Among them, the two-channel downmix matrix M _{2ch ms} is a 2x2 orthogonal normalized matrix, in which the first row is the sum vector and the second row is the difference vector. The three-channel downmix matrix M _{3ch ms} is a 3x3 orthogonal normalized matrix, in which the first row is the sum vector, and the second and third rows are the difference vectors.

Rewrite the above vocal channel information to obtain new vocal channel information.

In some embodiments, the encoder obtains the audio stream by performing the above steps. Referring to Figure 2D, after the channel signal is obtained, the channel signal is preprocessed, and then the preprocessed channel signal is subjected to inter-channel pair downmixing, and then bit allocation quantization entropy coding is performed, and finally coding is performed. Stream multiplexing to obtain an audio stream.

Among them, preprocessing includes transient detection window type judgment, time-frequency transformation, frequency domain noise shaping, time domain noise shaping, frequency band extension coding and other processes. Inter-channel pairing downmixing includes three-channel pairing downmixing of the L channel, R channel, and C channel to obtain the M1 channel, S11 channel, and S12 channel; and the LS channel and RS channel. The two channels are combined and downmixed to obtain the M2 channel and the S2 channel, and the LFE channel is not processed.

Step S2104: The encoder sends the first information and audio stream.

In some embodiments, the decoder receives the first information and audio stream.

In some embodiments, the encoder may send the first information and the audio stream separately. For example, the encoder sends the first information first and then the audio stream. Alternatively, the encoder sends the audio stream first and then the first information. In some embodiments, the encoder may send the first information and the audio stream simultaneously.

In some embodiments, the vocal tract information includes at least one of the following:

The number of channel groups including two channels;

Channel identifiers of channels included in a channel grouping that includes two channels;

Energy parameters for channel groupings including two channels;

The number of channel groups including three channels;

Channel identifiers of channels included in a channel grouping that includes three channels;

Energy parameters for channel groupings including three channels;

Among them, the energy parameter is used to adjust the energy of the channels in the channel grouping.

Step S2105: The decoder decodes the first information to obtain channel information of at least one channel group.

Step S2106: When the decoder determines that the channel grouping does not include a channel grouping of three channels, the decoder performs two-channel upmixing on the audio stream. When it determines that the channel grouping includes a channel grouping of three channels, the decoder performs a two-channel upmix on the audio stream. The audio stream is three-channel upmixed.

In some embodiments, the decoder determines whether the encoder has performed two-channel downmixing and three-channel downmixing. If the encoder performs a two-channel downmix, the decoder performs a two-channel upmix. If the encoder performs a three-channel downmix, the decoder performs a three-channel upmix.

Optionally, audio post-processing includes but is not limited to general decoding processes, such as time-frequency inverse transform, time-domain noise shaping inverse transform, frequency domain noise shaping inverse transform, frequency band extension inverse transform and other modules, as well as modules targeting certain types of signal characteristics. Decoding processing performed, such as multi-channel decoding processing, HOA channel decoding processing, object metadata decoding processing, etc.

In some embodiments, the decoder decodes and obtains each channel signal by performing the above steps. Referring to Figure 2E, after the audio stream is obtained, the audio stream is demultiplexed, and then bit allocation inverse quantization entropy coding, inter-channel upmixing, and post-processing are performed to obtain a decoded channel signal.

Among them, post-processing includes frequency band extension decoding, inverse time domain noise shaping, inverse frequency domain noise shaping, inverse time-frequency transformation and other processes. The inter-channel pairing and upmixing includes three-channel pairing of M1 channel, S11 channel, and S12 sound, and upmixing of L channel, R channel, and C channel to obtain channels; and the M2 channel, S2 channel The LS channel and the RS channel are upmixed by two-channel grouping, and the LFE channel is not processed.

In some embodiments, the name of the information is not limited to the names recorded in the embodiments, such as "information", "message", "signal", "signaling", "report", "configuration", "indication", "instruction", "command", "channel", "parameter", "domain", "Field", "symbol", "symbol", "codebook", "codeword", "codepoint", "bit", Terms such as "data", "program" and "chip" are interchangeable.

In some embodiments, terms such as "uplink", "uplink" and "physical uplink" may be replaced by each other, and terms such as "downlink", "downlink" and "physical downlink" may be replaced by each other. "side", "sidelink", "side communication", "side link communication", "direct connection", "direct link", "direct communication", The terms "direct link communications" are used interchangeably.

In some embodiments, "obtain", "obtain", "get", "receive", "transmit", "bi-directional transmission", "send and/or receive" may be interchanged, which may be interpreted as receiving from other entities , obtained from the protocol, obtained from the high-level, processed by oneself, implemented independently, etc.

In some embodiments, terms such as "send", "transmit", "report", "deliver", "transmit", "bidirectional transmission", "send and/or receive" may be replaced with each other.

In some embodiments, terms such as "moment", "time point", "time", "time position", etc. may be replaced with each other, and "duration", "period", "time window", "window", "time", etc. The terms are interchangeable.

In some embodiments, "certain", "preseted", "preset", "setting", "indicated", "certain", "any", "first" Terms such as "specific A", "predetermined A", "preset A", "set A", "instruction A", "a certain A", "any A" and "first A" can be used interchangeably. It can be interpreted as A that is predetermined in an agreement, etc., or it can be interpreted as A that is obtained through setting, configuration, or instruction, etc. It can also be interpreted as a specific A, a certain A, any A, or the first A, etc., but not Limited to this.

The grouping method involved in the embodiment of the present disclosure may include at least one of steps S2101 to S2106. For example, step S2101 can be implemented as an independent embodiment, step S2102 can be implemented as an independent embodiment, step S2103 can be implemented as an independent embodiment, step S2104 can be implemented as an independent embodiment, and step S2105 can be implemented as an independent embodiment. Implementation, step S2106 can be implemented as an independent embodiment, step S2101 and step S2102 can be implemented as an independent embodiment, step S2103 and step S2104 can be implemented as an independent embodiment, and step S2105 and step S2106 can be implemented as an independent embodiment. , Step S2101, Step S2102, Step S2103, and Step S2104 can be implemented as independent embodiments. Step S2101, Step S2102, Step S2105, and Step S2106 can be implemented as independent embodiments. Step S2103, Step S2104, Step S2105, and Step S2106 It can be implemented as an independent embodiment, but is not limited thereto.

In some embodiments, step S2101 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2102 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2103 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2104 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2105 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2106 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, steps S2101 and S2102 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, steps S2103 and S2104 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, steps S215 and S2106 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, please refer to other optional implementations described before or after the description corresponding to Figure 2 .

FIG. 3A is a schematic flowchart of a grouping method according to an embodiment of the present disclosure, applied to an encoder. As shown in Figure 3A, the embodiment of the present disclosure relates to a grouping method. The above method includes:

Step S3101: The encoder groups multiple channels to obtain at least one channel group.

For the optional implementation of step S3101, please refer to the optional implementation of step S2101 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be described again here.

Step S3102: The encoder downmixes the audio signal in each channel group to obtain a downmixed audio signal.

For the optional implementation of step S3102, please refer to the optional implementation of step S2102 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be described again here.

Step S3103: The encoder encodes the downmixed audio signal to obtain an audio stream.

For the optional implementation of step S3103, please refer to the optional implementation of step S2103 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be described again here.

Step S3104: The encoder sends the first information and audio stream.

For the optional implementation of step S3103, please refer to the optional implementation of step S2104 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be described again here.

The grouping method involved in the embodiment of the present disclosure may include at least one of steps S3101 to S3104. For example, step S3101 can be implemented as an independent embodiment, step S3102 can be implemented as an independent embodiment, step S3103 can be implemented as an independent embodiment, step S3104 can be implemented as an independent embodiment, or at least two steps can be combined. , but not limited to this.

In some embodiments, step S3101 is optional, step S3102 is optional, step S3103 is optional, and step S3103 is optional. One or more of these steps may be performed in different embodiments. omit or substitute. But not limited to this.

FIG. 3B is a schematic flowchart of a grouping method according to an embodiment of the present disclosure, applied to an encoder. As shown in Figure 3B, the embodiment of the present disclosure relates to a grouping method. The above method includes:

Step S3201: The encoder groups multiple channels to obtain at least one channel group.

For optional implementation methods of step S3201, please refer to step S2101 in Figure 2, step S3101 in Figure 3A, and other related parts in the embodiments involved in Figure 2 and Figure 3A, which will not be described again here.

Figure 4A is a schematic flowchart of a grouping method according to an embodiment of the present disclosure, applied to a decoder. As shown in Figure 4A, an embodiment of the present disclosure relates to a grouping method, and the above method includes:

Step S4101: The decoder decodes the first information to obtain channel information of at least one channel group.

For optional implementation methods of step S4101, please refer to step S2105 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be described again here.

Step S4102: When the decoder determines that the channel grouping does not include a channel grouping of three channels, the decoder performs two-channel upmixing on the audio stream. When it determines that the channel grouping includes a channel grouping of three channels, the decoder performs a two-channel upmixing on the audio stream. The audio stream is three-channel upmixed.

For optional implementation methods of step S4102, please refer to step S2106 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be described again here.

The grouping method involved in the embodiment of the present disclosure may include at least one of steps S4101 to S4102. For example, step S4101 can be implemented as an independent embodiment, step S4102 can be implemented as an independent embodiment, or at least two steps can be combined, but are not limited thereto.

In some embodiments, step S4101 is optional and step S4102 is optional, and one or more of these steps may be omitted or replaced in different embodiments. But not limited to this.

Figure 4B is a schematic flowchart of a grouping method according to an embodiment of the present disclosure, applied to a decoder. As shown in Figure 4B, the embodiment of the present disclosure relates to a grouping method, and the above method includes:

Step S4201: The decoder decodes the first information to obtain channel information of at least one channel group.

For optional implementation methods of step S4201, please refer to step S2105 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be described again here.

In some embodiments, there are N first channel groupings in the at least one channel grouping, the first channel grouping includes three channels, and M second channel groups are present in the at least one channel grouping. Channel grouping, the second channel grouping includes two channels, where N is 1 and M is a non-negative integer.

In some embodiments, the vocal channel information includes at least one of the following:

The number of channel groups including two channels;

Channel identifiers of channels included in a channel grouping that includes two channels;

Energy parameters for channel groupings including two channels;

The number of channel groups including three channels;

Channel identifiers of channels included in a channel grouping that includes three channels;

Energy parameters for channel groupings including three channels;

Wherein, the energy parameter is used to adjust the energy of the channels in the channel grouping.

In some embodiments, the method further includes:

When it is determined that the channel grouping does not include a channel grouping of three channels, a two-channel upmix is performed on the audio stream.

In some embodiments, the method further includes:

When it is determined that the channel grouping includes three channels, three-channel upmixing is performed on the audio stream.

Figure 5 is a schematic flowchart of a grouping method according to an embodiment of the present disclosure. As shown in Figure 5, the embodiment of the present disclosure relates to a grouping method. The above method includes:

Step S5101: The encoder groups multiple channels to obtain at least one channel group.

In some embodiments, there are N first channel groupings in the at least one channel grouping, the first channel grouping includes three channels, and M second channel groups are present in the at least one channel grouping. Channel grouping, the second channel grouping includes two channels, where N is 1 and M is a non-negative integer.

Step S5102: The decoder decodes the first information to obtain the channel information of at least one channel group.

For optional implementation methods of step S5101, please refer to step S2101 in Fig. 2, step S3101 in Fig. 3A, and other related parts in the embodiments involved in Fig. 2 and Fig. 4A, which will not be described again here.

For optional implementation methods of step S5102, please refer to step S2105 in Figure 2, step S4101 in Figure 4A, and other related parts in the embodiments involved in Figures 2 and 3A, which will not be described again here.

In some embodiments, the above method may include the method of the above embodiments on the encoding and decoding system side, encoder side, decoder side, etc., which will not be described again here.

Figure 6 is a schematic flowchart of a grouping method according to an embodiment of the present disclosure. As shown in Figure 6, the embodiment of the present disclosure relates to a grouping method. The above method includes:

Step S6101: The encoding end uses a combination of two-channel sum-difference coding and three-channel sum-difference coding to code multiple channels.

The encoding end completes the audio preprocessing and enters the inter-channel pair downmix module. Audio preprocessing includes but is not limited to general coding processes, such as transient analysis, time-frequency transformation, time domain noise shaping, frequency domain noise shaping, frequency band expansion and other modules. It also includes processing for certain types of signal characteristics, such as multi-voice Channel encoding processing, HOA channel encoding processing, object metadata encoding processing, etc.

The inter-channel pair downmix module introduces the three-channel sum-difference coding method and is used in combination with the two-channel sum-difference coding framework. This module includes a decision module and a downmix module.

The decision module is used to determine which sum and difference coding method or combination to use. The decision criterion is inter-channel correlation and is compared to group pair thresholds. The judgment result is that the L, R, and C channels are subjected to 3-channel sum and difference coding, LS and LS are subjected to 2-channel sum and difference coding, and the LFE channel is not processed.

The downmix module performs 3-channel sum and difference coding on the L, R, and C channels, and LS and LS perform 2-channel sum and difference coding.

After passing through the inter-channel pairing downmixing module, the three-channel sum-difference-coded downmixed channels (M1 channel, S11 channel, S12 channel), the two-channel sum-difference-coded downmixed channel ( M2 channel, S2 channel) and the undownmixed channel (LFE channel) all pass through the bit allocation quantization entropy coding module, and are multiplexed to form the encoded bit stream E. .

In the embodiments of the present disclosure, some or all of the steps and their optional implementations can be arbitrarily combined with some or all of the steps in other embodiments, and can also be arbitrarily combined with the optional implementations of other embodiments.

Embodiments of the present disclosure also provide a device for implementing any of the above methods. For example, a device is provided. The device includes units or modules for implementing each step performed by the encoder in any of the above methods. As another example, another device is also proposed, including units or modules for implementing each step performed by the decoder in any of the above methods.

It should be understood that the division of each unit or module in the above device is only a division of logical functions. In actual implementation, it can be fully or partially integrated into a physical entity, or it can also be physically separated. In addition, the units or modules in the device can be implemented in the form of the processor calling software: for example, the device includes a processor, the processor is connected to a memory, instructions are stored in the memory, and the processor calls the instructions stored in the memory to implement any of the above. Method or implement the functions of each unit or module of the above device, where the processor is, for example, a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuits, and some or all of the functions of the units or modules can be implemented through the design of the hardware circuits. The above-mentioned hardware circuits can be understood as one or more processors; for example, in a In one implementation, the above hardware circuit is an application-specific integrated circuit (ASIC), and through the design of the logical relationship of the components in the circuit, the functions of some or all of the above units or modules are realized; for another example, in another implementation Among them, the above hardware circuit can be implemented by a programmable logic device (PLD). Taking Field Programmable Gate Array (FPGA) as an example, it can include a large number of logic gate circuits and can be configured through configuration files. Configure the connection relationship between logic gate circuits to realize the functions of some or all of the above units or modules. All units or modules of the above device may be fully implemented by the processor calling software, or may be fully implemented by hardware circuits, or part of the units or modules may be implemented by the processor calling software, and the remaining part may be implemented by hardware circuits.

In the embodiment of the present disclosure, the processor is a circuit with signal processing capabilities. In one implementation, the processor may be a circuit with instruction reading and execution capabilities, such as a central processing unit (Central Processing Unit, CPU), microprocessor, etc. Processor, graphics processing unit (GPU) (can be understood as a microprocessor), or digital signal processor (DSP), etc.; in another implementation, the processor can be configured through a hardware circuit Logical relationships realize certain functions. The logical relationships of the above-mentioned hardware circuits are fixed or can be reconstructed. For example, the processor is implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). Hardware circuits, such as FPGA. In a reconfigurable hardware circuit, the process of the processor loading the configuration file and realizing the hardware circuit configuration can be understood as the process of the processor loading instructions to realize the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (Neural Network Processing Unit, NPU), tensor processing unit (Tensor Processing Unit, TPU), deep learning processing unit ( Deep learning Processing Unit (DPU), etc.

FIG. 7A is a schematic structural diagram of a coding and decoding device proposed by an embodiment of the present disclosure. As shown in Figure 7A, the encoding and decoding device 7100 may include: at least one of a transceiver module 7101, a processing module 7102, and the like. In some embodiments, the processing module 7102 is used to group multiple channels to obtain at least one channel group, wherein there are N first channel groups in the at least one channel group, and the first channel group is The channel group includes three channels, and there are M second channel groups in the at least one channel group. The second channel group includes two channels, where N is 1 and M is a non-negative integer. Optionally, the above-mentioned transceiving module 7101 is used to perform at least one of the communication steps such as sending and/or receiving performed by the encoder in any of the above methods (such as step S2101 but not limited to this), which will not be described again here. Optionally, the above processing module is used to perform at least one of the other steps performed by the encoder in any of the above methods, which will not be described again here.

Optionally, the processing module 7102 is configured to perform at least one of the communication steps such as processing performed by the encoder in any of the above methods, which will not be described again here.

FIG. 7B is a schematic structural diagram of a coding and decoding device proposed by an embodiment of the present disclosure. As shown in Figure 7B, the encoding and decoding device 7200 may include: at least one of a transceiver module 7201, a processing module 7202, and the like. In some embodiments, the processing module 7202 is used to decode the first information to obtain channel information of at least one channel group, where there are N first channel groups in the at least one channel group, and the first channel group is The channel group includes three channels, and there are M second channel groups in the at least one channel group. The second channel group includes two channels, where N is 1 and M is a non-negative integer. . Optionally, the above-mentioned transceiver module is used to perform at least one of the communication steps such as sending and/or receiving performed by the decoder in any of the above methods (such as step S2102 but not limited to this), which will not be described again here.

Optionally, the processing module 7202 is configured to perform at least one of the communication steps such as processing performed by the decoder in any of the above methods, which will not be described again here.

In some embodiments, the transceiver module may include a sending module and/or a receiving module, and the sending module and the receiving module may be separated or integrated together. Optionally, the transceiver module can be interchangeable with the transceiver.

In some embodiments, the processing module may be one module or may include multiple sub-modules. Optionally, the above multiple sub-modules respectively perform all or part of the steps required by the processing module. Optionally, the processing module may be interchangeable with the processor.

FIG. 8A is a schematic structural diagram of a communication device 8100 proposed by an embodiment of the present disclosure. The communication device 8100 may be a decoder (such as access network equipment, core network equipment, etc.), an encoder (such as user equipment, etc.), or a chip, chip system, or chip system that supports the decoder to implement any of the above methods. The processor, etc., may also be a chip, chip system, or processor that supports the encoder to implement any of the above methods. The communication device 8100 may be used to implement the method described in the above method embodiment. For details, please refer to the description in the above method embodiment.

As shown in Figure 8A, communications device 8100 includes one or more processors 8101. The processor 8101 may be a general-purpose processor or a special-purpose processor, for example, it may be a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, and the central processor can be used to control encoding and decoding devices (such as base stations, baseband chips, encoder equipment, encoder equipment chips, DU or CU, etc.), Execute the program and process the program's data. The communication device 8100 is used to perform any of the above methods.

In some embodiments, communications device 8100 also includes one or more memories 8102 for storing instructions. Optionally, all or part of memory 8102 may also be external to communication device 8100.

In some embodiments, communications device 8100 also includes one or more transceivers 8103. When the communication device 8100 includes one or more transceivers 8103, the transceiver 8103 performs communication steps such as sending and/or receiving in the above method (such as step S2101, step S2102, step S2103, step S2104, but is not limited to this). At least one of.

In some embodiments, the transceiver may include a receiver and/or a transmitter, and the receiver and transmitter may be separate or integrated together. Optionally, terms such as transceiver, transceiver unit, transceiver, and transceiver circuit may be interchanged with each other, terms such as transmitter, transmitting unit, transmitter, transmitting circuit may be interchanged with each other, and terms such as receiver, receiving unit, receiver, receiving circuit, etc. The terms are interchangeable.

In some embodiments, communication device 8100 may include one or more interface circuits 8104. Optionally, the interface circuit 8104 is connected to the memory 8102. The interface circuit 8104 can be used to receive signals from the memory 8102 or other devices, and can be used to send signals to the memory 8102 or other devices. For example, interface circuit 8104 may read instructions stored in memory 8102 and send the instructions to processor 8101.

The communication device 8100 in the above embodiment description may be a decoder or an encoder, but the scope of the communication device 8100 described in the present disclosure is not limited thereto, and the structure of the communication device 8100 may not be limited by FIG. 8A. The communication device may be a stand-alone device or may be part of a larger device. For example, the communication device may be: 1) an independent integrated circuit IC, a chip, or a chip system or a subsystem; (2) a collection of one or more ICs. Optionally, the above-mentioned IC collection may also include Storage components for storing data and programs; (3) ASIC, such as modem; (4) Modules that can be embedded in other devices; (5) Receivers, encoder devices, smart encoder devices, cellular phones, Wireless devices, handheld devices, mobile units, vehicle-mounted equipment, decoders, cloud equipment, artificial intelligence equipment, etc.; (6) Others, etc.

FIG. 8B is a schematic structural diagram of the chip 8200 proposed by the embodiment of the present disclosure. For the case where the communication device 8100 may be a chip or a chip system, reference may be made to the schematic structural diagram of the chip 8200 shown in FIG. 8B , but is not limited thereto.

The chip 8200 includes one or more processors 8201, and the chip 8200 is used to perform any of the above methods.

In some embodiments, chip 8200 also includes one or more interface circuits 8202. Optionally, the interface circuit 8202 is connected to the memory 8203. The interface circuit 8202 can be used to receive signals from the memory 8203 or other devices, and the interface circuit 8202 can be used to send signals to the memory 8203 or other devices. For example, the interface circuit 8202 can read instructions stored in the memory 8203 and send the instructions to the processor 8201.

In some embodiments, the interface circuit 8202 performs at least one of the communication steps such as sending and/or receiving in the above method, and the processor 8201 performs at least one of the other steps.

In some embodiments, terms such as interface circuit, interface, transceiver pin, and transceiver may be used interchangeably.

In some embodiments, chip 8200 also includes one or more memories 8203 for storing instructions. Alternatively, all or part of memory 8203 may be external to chip 8200.

The present disclosure also proposes a storage medium, and instructions are stored on the storage medium. When the instructions are run on the communication device 8100, the communication device 8100 is caused to perform any of the above methods. Optionally, the above storage medium is an electronic storage medium. Optionally, the above-mentioned storage medium is a computer-readable storage medium, but is not limited thereto. It may also be a storage medium readable by other devices. Optionally, the above storage medium may be a non-transitory storage medium, but is not limited thereto, it may also be a transitory storage medium.

The present disclosure also proposes a program product. When the program product is executed by the communication device 8100, it causes the communication device 8100 to execute any of the above methods. Optionally, the above program product is a computer program product.

The present disclosure also proposes a computer program that, when run on a computer, causes the computer to perform any of the above methods.

Claims (21)

1. A method of grouping, the method performed by an encoder, the method comprising:

and grouping the plurality of channels to obtain at least one channel group, wherein N first channel groups exist in the at least one channel group, the first channel groups comprise three channels, M second channel groups exist in the at least one channel group, the second channel groups comprise two channels, N is 1, and M is a non-negative integer.

2. The method of claim 1, wherein grouping the plurality of channels to obtain at least one channel group comprises:

acquiring the similarity between any two channels of the plurality of channels;

and grouping the plurality of channels based on the similarity between any two channels to obtain the at least one channel group.

3. The method of claim 2, wherein grouping the plurality of channels based on the similarity between the any two channels results in the at least one channel group, comprising:

for any two channels of the plurality of channels, determining the two channels with the greatest similarity as a candidate channel group;

excluding the two channels with the maximum similarity from other channels, and determining the two channels with the maximum similarity as a candidate channel group until an independent channel or no channel remains;

the at least one channel group is determined based on the resulting candidate channel group and/or the independent channels.

4. A method according to claim 3, wherein said determining said at least one channel group based on the resulting candidate channel group and/or said independent channel comprises:

obtaining the similarity between the independent channel and each channel in each candidate channel group;

determining each channel in the independent channel and first candidate channel group as one first channel group when the similarity of each channel in the independent channel and first candidate channel group is larger than a similarity threshold;

Each remaining candidate channel group is determined as one of the second channel groups, respectively.

5. The method of claim 1, wherein grouping the plurality of channels to obtain at least one channel group comprises:

and grouping the plurality of channels to directly obtain the N first channel groups and the M second channel groups.

6. The method of claim 5, wherein the grouping the plurality of channels directly results in the N first channel groups and the M second channel groups, comprising:

and carrying out global search on the plurality of channels, and grouping the plurality of channels according to the similarity between any two channels to obtain N first channel groups and M second channel groups.

7. The method of claim 1, wherein grouping the plurality of channels to obtain at least one channel group comprises:

grouping the plurality of channels to obtain a plurality of candidate channel groups comprising two channels and at least one independent channel;

dividing one independent channel in the at least one independent channel into one candidate channel group in the plurality of candidate channel groups to obtain the N first channel groups and the M second channel groups.

8. The method according to any one of claims 1 to 7, further comprising:

down-mixing the audio signals in each channel group to obtain the down-mixed audio signals;

and encoding the audio signal after the down mixing to obtain an audio stream.

9. The method according to any one of claims 1 to 8, further comprising:

and transmitting first information, wherein the first information is used for indicating channel information of the plurality of channel groups.

10. The method of claim 9, wherein the channel information includes at least one of:

the number of channel groups comprising two channels;

channel identifications of channels included in a channel group including two channels;

energy parameters of a channel group comprising two channels;

the number of channel groups including three channels;

channel identifications of channels included in a channel group including three channels;

energy parameters of a channel group comprising three channels;

wherein the energy parameter is used for adjusting the energy of the channels in the channel group.

11. A method of grouping, the method performed by a decoder, the method comprising:

Decoding the first information to obtain channel information of at least one channel group, wherein N first channel groups exist in the at least one channel group, the first channel group comprises three channels, M second channel groups exist in the at least one channel group, the second channel group comprises two channels, N is 1, and M is a non-negative integer.

12. The method of claim 11, wherein the channel information comprises at least one of:

the number of channel groups comprising two channels;

channel identifications of channels included in a channel group including two channels;

energy parameters of a channel group comprising two channels;

the number of channel groups including three channels;

channel identifications of channels included in a channel group including three channels;

energy parameters of a channel group comprising three channels;

wherein the energy parameter is used for adjusting the energy of the channels in the channel group.

13. The method according to claim 11 or 12, characterized in that the method further comprises:

upon determining that the channel group does not include three channels of the channel group, two-channel upmixing is performed on the audio stream.

14. The method according to claim 11 or 12, characterized in that the method further comprises:

upon determining that the channel group includes a channel group of three channels, the audio stream is upmixed in three channels.

15. A method of grouping, the method comprising:

the method comprises the steps that an encoder groups a plurality of channels to obtain at least one channel group, wherein N first channel groups exist in the at least one channel group, the first channel groups comprise three channels, M second channel groups exist in the at least one channel group, the second channel groups comprise two channels, N is 1, and M is a non-negative integer;

the decoder decodes the first information to obtain channel information of the at least one channel group, wherein at least one channel group of the plurality of channel groups comprises three channels.

16. A codec apparatus, the codec apparatus comprising:

the processing module is used for grouping the plurality of channels to obtain at least one channel group, wherein N first channel groups exist in the at least one channel group, the first channel groups comprise three channels, M second channel groups exist in the at least one channel group, the second channel groups comprise two channels, N is 1, and M is a non-negative integer.

17. A codec apparatus, the codec apparatus comprising:

the processing module is used for decoding the first information to obtain channel information of at least one channel group, N first channel groups exist in the at least one channel group, the first channel group comprises three channels, M second channel groups exist in the at least one channel group, the second channel group comprises two channels, N is 1, and M is a non-negative integer.

18. A codec apparatus, the codec apparatus comprising:

one or more processors;

wherein the processor is configured to perform the grouping method of any one of claims 1 to 10.

19. A codec apparatus, the codec apparatus comprising:

one or more processors;

wherein the processor is configured to perform the grouping method of any one of claims 11 to 14.

20. A codec system comprising an encoder configured to implement the grouping method of any one of claims 1 to 10 and a decoder configured to implement the grouping method of any one of claims 11 to 14.

21. A storage medium storing instructions that, when executed on a communication device, cause the communication device to perform the grouping method of any one of claims 1 to 10 or perform the grouping method of any one of claims 11 to 14.